What is the llms.txt file, and how do you create one for your website?

The llms.txt file is a document that enables artificial intelligence (AI) to understand your website content more easily. It contains information about the structure of your site, including the page URLs and their content, written in Markdown syntax.

The information in llms.txt is crucial – without it, AI might have difficulty navigating complex site structures and determining content hierarchy. This reduces the chance of a given website being featured in AI-powered search engine results.

This digital document is similar to robots.txt in that it helps make your website’s content more easily parsed by web crawlers. However, while these files work hand-in-hand, they serve different purposes.

The llms.txt file is located inside your website’s root directory and can easily be created using a web-based tool. Let’s examine it more closely in the following section.

What is the content of the llms.txt file?

llms.txt contains an index of your website’s pages, including their content, URL, and title, written in the human- and machine-readable Markdown language. It can also include optional descriptions of the website structure, which provide more context to AI. Watch the following video for more info, or read on underneath.

Content in llms.txt starts with the website address, followed by the URLs of pages on your site, and some optional descriptive information. Here’s how it looks:

# Title
> Optional description goes here
Optional details go here

## Section name
- [Link title](https://link_url): Optional link details

## Optional
- [Link title](https://link_url)

Here’s an example of an llms.txt file for a basic website with two pages:

# your-website.com

## Posts

- [Hello world!](https://your-website.com/hello-world/): Welcome to WordPress. This is your first post. Edit or delete it, then start writing!

## Pages

- [Privacy Policy](https://your-website.com/privacy-policy/): Who we are Suggested text: Our website address is: your-website.com. Comments Suggested text: When visitors leave comments on the site we collect the data shown in the comments form, and also the visitor’s IP address and browser user agent string to help spam detection. An anonymized string created from your email address (also called a […]

The llms.txt document is located in your website’s root folder, and is viewable in a browser by adding /llms.txt to the end of your site’s URL as follows, with your-website.com being the actual domain name:

https://www.your-website.com/llms.txt

Why is the llms.txt file important?

The llms.txt file is an integral component of generative engine optimization (GEO), the practice of optimizing a website’s content and structure to allow AI to understand it more easily. This can improve your website’s visibility in AI-driven search engines.

GEO is similar to traditional search engine optimization (SEO), but focuses on making website content easier for AI to interpret, rather than improving its indexability on search engine results pages (SERPs).

Implementing GEO is crucial because people are increasingly using AI-driven search to find information, whether on AI overview snippets or platforms like ChatGPT. Statista’s 2024 survey shows that 36 million adults in the US will use generative AI for searches in 2028, more than double that of 2024.

How LLMs use the llms.txt file

Large language models (LLMs) are underlying systems of AI tools that use a massive amount of data to learn patterns in text and interpret user inputs.

The llms.txt file explicitly tells AI web crawlers which parts of your website are the most important, eliminating the need for the AI to navigate your site.

This also enables AI tools to gather information about your website’s content and structure more accurately, minimizing ambiguity and improving parsing ability.

This then allows AI to assess whether content on your website is relevant and valuable for specific user queries. If so, it is more likely to feature content from your site as part of an answer.

By excluding certain content from your llms.txt, you can also limit areas of your website that you don’t want AI tools to fetch and use for training. This pertains mainly to proprietary or copyrighted material.

How to create an llms.txt file for your website

You can create the llms.txt file for your website manually, but using an online llms.txt generator is easier. Note that depending on the tool, the steps and generated markdown may differ.

For example, here’s how to do it using Wordlift’s LLMs.txt Generator:

1. Enter your website’s address and choose the pages and posts you want to include in your llms.txt.
2. Click Process URL.
3. Once done, copy the generated content.

4. Create a new llms.txt file on your computer and paste in the copied markdown. Save the changes.
5. Open your website’s root directory using your web host’s file manager or an SFTP tool like FileZilla. By default, it is /var/www/html/.
6. Upload the llms.txt file to the website’s root directory.

How to create an llms.txt file for a WordPress site

You can generate an llms.txt file and upload it to your WordPress website easily using a plugin. For example, Hostinger’s managed WordPress hosting users can do so using the built-in Hostinger plugin by following these steps:

1. Open hPanel and go to the sidebar →Websites → Website list.
2. Find your WordPress website and click WP admin.

3. On the Hostinger page, click the Tools tab.
4. Scroll down to the AI Preferences section and click the Turn on LLMs.txt toggle.

That’s it! Now, you can check if the llms.txt is valid by visiting your website’s URL followed by /llms.txt, or using our llms.txt Validator.

How to create an llms.txt file for a Hostinger Horizons site

If you built your website using Hostinger Horizons, the llms.txt file will be automatically generated once you publish it live. You can skip these steps if your site is already online. Otherwise, follow this procedure:

1. On hPanel, navigate to the sidebar and select Horizons.
2. Click Manage Project on your website.
3. Hit Publish in the top right corner of your Hostinger Horizons dashboard.

Once published, check whether the llms.txt is configured properly using our llms.txt Validator tool. If you encounter an error, ask Hostinger Horizons to resolve the issue.

What’s the difference between llms.txt and robots.txt?

The llms.txt file serves a similar function to that of standard robots.txt: improving your website’s content crawlability and indexability.

Unlike llms.txt, robots.txt provides search engines with information about your website’s structure, allowing them to navigate through different pages and crawl your content. It also serves to block indexing of password-protected areas of your website, like the admin area, preventing them from being listed on SERPs.

In other words, robots.txt improves the visibility of your site’s public pages in search results. Meanwhile, llms.txt enables AI to interpret and outline your site’s most valuable content to help answer users’ questions.

Although both are TXT files, they use slightly different formatting. Unlike the Markdown syntax used in llms.txt, robots.txt’s content is in standard text. Here’s what it looks like:

User-agent: Googlebot

Disallow: /nogooglebot/

User-agent: *

Allow: /

Sitemap: https://www.example.com/sitemap.xml

Is llms.txt going to be the future of GEO?

Although still not fully supported by major LLM providers, llms.txt is currently the proposed standard for generative engine optimization (GEO). As internet users are increasingly moving towards AI-powered search engines, implementing it as soon as possible can be beneficial to your website.

Implementing llms.txt now gives you a head start over your competition. It increases the likelihood that AI-powered search engines and platforms will feature content from your website, improving its visibility.

Generating an llms.txt document for your site is easy, especially using an online tool like Hostinger plugin. It’s a low-effort, high-value improvement you can make in minutes.

All of the tutorial content on this website is subject to Hostinger's rigorous editorial standards and values.