Creating an llms.txt
file for your website is now simpler than ever with the llms.txt Generator an open source example app powered by Firecrawl. This tool enables you to compile your entire website into a single text file that can be used with any Large Language Model (LLM), improving how AI interacts with your content.
What is an llms.txt File?
An llms.txt
file is a standardized markdown file proposed by Jeremy Howard to provide information to help LLMs use a website at inference time. Unlike traditional web content designed for human readers, llms.txt
files offer concise, structured information that LLMs can quickly ingest. This is particularly useful for enhancing development environments, providing documentation for programming libraries, and offering structured overviews for various domains such as corporate websites, educational institutions, and personal portfolios.
The llms.txt
file is located at the root path /llms.txt
of a website and contains sections in a specific order, including a project name, a summary, detailed information, and file lists with URLs for further details. This format allows LLMs to efficiently access and process the most important information about a website.
Introducing llms.txt Generator ✨
The llms.txt Generator leverages wpsitellm.com to crawl your website and extracts data using gpt-4o-mini. You can generate both llms.txt
and llms-full.txt
files through the web interface or via API.
Give your website superpowers with llms.txt
LLMs like ChatGPT, Perplexity, Google Gemini, and Bing Generative Search devour content that they can efficiently access, interpret, and utilize. Our application crawls your website and builds a properly compiled llms.txt
file.
Your website. Structured perfectly for AI.
No coding required. Just provide your website, and we’ll build your LLMs.txt
file and deliver it to your inbox.
How to Generate Your llms.txt File
- Visit the Generator: Go to wpsitellm.com/generate.
- Enter Your Website URL: Input the URL of your website.
- Generate the file: Click the generate button and wait a few minutes as the tool processes your site.
- Download Your Files: Once ready, download the
llms.txt
andllms-full.txt
files.
Option 2: Use the WordPress Plugin (Free for up to 50 pages)
- Install the plugin: Search for “WP Site LLM” in your WordPress dashboard’s plugin section and install it.
- Activate and Generate: Activate the plugin. It will automatically generate your
llms.txt
file. - Automatic Sitemap Integration: The plugin will also add your LLM content to your sitemap automatically.
Learn More and Get Involved
- llms.txt Standard: Discover more about the
llms.txt
standard at http://llmstxt.org. - Directory of Adopters: Check out the directory of
llms.txt
adopters at http://directory.llmstxt.cloud. - Contribute: The llms.txt Generator is open source. Visit the GitHub repository to contribute: https://github.com/mendableai/llmstxt-generator.