AI workflows
AGENTS.md, CLAUDE.md and llms.txt: what an AI-ready template needs
What goes in AGENTS.md and CLAUDE.md, what llms.txt is for, what research and server logs say about whether they help, and how to check a template in 10 minutes.
Every new session, an AI helper starts from a blank slate: it doesn’t remember earlier conversations. What it should know about your project from the first minute, you write into instruction files that it reads at the start of every session.[1] That’s why “AI-ready” templates now ship with rule files: AGENTS.md, CLAUDE.md and, separately, llms.txt.
People often mix these three up, but they do different jobs. AGENTS.md and CLAUDE.md are a memo for a helper that edits your project’s files on your computer. llms.txt is a short map of an already published site, meant for tools that read it from the internet. We’ll go through each one: what to put in it, who reads it, and what is known about whether it helps.
A word on honesty up front: we sell Astro website templates, so we have an interest here. We haven’t run our own “with the file and without it” measurements. So we rely on tool documentation and research, and where there is no evidence, we say so.
Current as of October 26, 2026. Claude Code reads AGENTS.md directly starting with version 2.1.277.[1] These tools change quickly, so check the documentation before you start.
In short
- For a template you’ll work on with AI, the main file is AGENTS.md: short, with commands, where things live, and what not to do.
- Claude Code 2.1.277 and later reads AGENTS.md on its own. You need a CLAUDE.md only for Claude-only rules or setups that don’t read AGENTS.md, and then one line,
@AGENTS.md, pulls the shared rules in.[1] - llms.txt is an optional site map. Google Search doesn’t use it,[2] and we found no evidence that it helps AI assistants answer questions.
- The files are advice, not rules, and on average they don’t improve results.[3] What matters more: text in files with a schema, one check command, and styling in one place.
The three files in one table
| File | Who reads it | What’s inside | Do you need it |
|---|---|---|---|
| AGENTS.md | AI helpers that edit your project: Codex, Cursor, Copilot and others | Commands, where things live, what not to do | Yes, a short one |
| CLAUDE.md | Claude Code | Claude-only rules, plus one line pulling in AGENTS.md | Rarely: Claude Code 2.1.277+ reads AGENTS.md itself |
| llms.txt | A proposal for AI assistants that read a site from the internet | A site map in Markdown | Optional |
Details and sources are in the sections below.
AGENTS.md: a memo for the helper
AGENTS.md is a “README for AI agents”: a simple, open format for storing instructions for coding helpers.[4] It’s plain Markdown text. There are no required fields, and the headings can be anything.[4] The file goes in the project root. Large projects sometimes have files in subfolders too: the helper reads the nearest one, and a direct request in chat overrides any file.[4]
The agents.md site itself lists the tools that read it: Codex, Cursor, GitHub Copilot, Gemini CLI and others. That is the format authors’ own claim, and we haven’t verified it.[4] Claude Code reads AGENTS.md if the project has no CLAUDE.md.[1]
What to put in it
Claude Code’s documentation advises writing down what the helper can’t get from the code: the commands that check its work, places where the code is misleading, decisions that aren’t recorded anywhere in the project, and things you’ve had to correct twice.[5] For a website template that comes down to three blocks: commands, “where things live” and “what not to do.” Here is an example for an imaginary site (the commands in your template may differ):
# Rules for the AI helper
## Commands
- npm install — install dependencies
- npm run dev — run the site on your computer
- npm run build — build the site; it must pass without errors after any change
## Where things live
- Page text is in src/content/pages. Don't write text directly into components.
- Colors and fonts live only in src/styles/tokens.css.
- Contact details and social links are in src/content/site.json.
## Don't
- Don't add CSS frameworks or new packages without approval.
- Don't remove alt text from images or lang from the page.
- Don't change src/layouts without a separate request.
What not to put in it
The same page suggests a test for each line: can the helper find this in the code, and will it go looking? If both answers are yes, don’t write it.[5] A description of the architecture and a map of every file are on its list of outdated advice.[5] Long explanations, details that change often and general appeals like “write clean code” don’t pass the test either (our assessment).
What the research says
Researchers tested files like this on real tasks from software repositories: issues that had to be resolved in code. The result: the files did not reliably raise the share of tasks solved, and they made the work more expensive, by over 20% on average for files written by AI and by up to 19% for files written by developers. Overviews of how a project is organized, although model providers recommend them, did not help. Non-standard rules turned out to be useful, and the authors advise testing any attempt to improve results with such a file first.[3]
A caveat: this study is about tasks in program code, not about editing text on a brochure site. We found no data for our exact case. Our assessment: the takeaway “write briefly, and only what the helper doesn’t know” applies here too, but that is an assessment, not a measurement.
CLAUDE.md: when you need it
CLAUDE.md is a similar memo, but for Claude Code. Claude Code reads it at the start of every session.[1] It lives in the project root (or in the .claude folder).[1]
If the project has AGENTS.md and no CLAUDE.md, there is nothing to do: starting with version 2.1.277, Claude Code reads AGENTS.md on its own.[1] But by default, if the project has both files, Claude Code reads only CLAUDE.md (a setting lets it read both).[1] And some sessions can’t read AGENTS.md directly.[1] For those cases, put one line in CLAUDE.md that pulls in AGENTS.md:
@AGENTS.md
Claude Code inserts the contents of AGENTS.md, and you can add Claude-only rules below it.[1] The result is one source of truth with two entry points. If CLAUDE.md contains nothing but that line and your Claude Code reads AGENTS.md itself, the documentation says you can delete it.[1]
A symbolic link (ln -s AGENTS.md CLAUDE.md) also works, but on Windows it needs administrator rights or developer mode, and Git may check it out as an ordinary one-line file. Importing with @AGENTS.md is more reliable.[1]
Advice, not a rule
Claude Code treats CLAUDE.md as context, not as required configuration: the shorter and more specific the rules, the more consistently it follows them.[1] For hard bans you need not words but automatic checks (in Claude Code they’re called hooks): they fire every time, while text rules are only advisory.[5] Keep the length in check too: the target is under 200 lines, otherwise the file eats context and rules get lost.[1]
llms.txt: a site map for AI
What it is
llms.txt is a proposal by Jeremy Howard from September 3, 2024: put a short Markdown file in the root of your site. Only a heading with the site’s name is required; a brief description and lists of links to important pages are optional. The page now shows the second version of the proposal, changed on August 10, 2026.[6] The idea is to give an AI agent a small entry point, so it doesn’t have to parse whole pages full of menus, ads and scripts.[6] It is not an official standard: the project’s own page calls it a proposal.[6] The figures on how many sites publish it are also the project’s own words, and we haven’t verified them.[6]
What Google says
Google’s guide for Search says that to appear in Google Search you don’t need new machine-readable files, “AI text files,” special markup or Markdown, and that Google Search itself doesn’t use them.[2] The page was updated on July 10, 2026.
But Google has another side. Lighthouse, Chrome’s site-checking tool, has an experimental category called Agentic Browsing (a site “for agents”). It includes an llms.txt check, which looks at whether a machine-readable summary sits at the domain root. The category requires Chrome 150 or later, has no overall 0–100 score, and the documentation says nothing about any effect on search or citations.[7]
What shows up in server logs
We found no academic studies on this: there are only measurements by companies on their own data. Prosopo counted, on one site over 12 days in October 2026, three llms.txt requests from verified AI crawlers against 511 robots.txt requests.[8] EZY.ai, over 12 weeks across 83 sites: OpenAI requested llms.txt 7 times (robots.txt: 3,990), Anthropic 9 (3,120), PerplexityBot never (775).[9] Caveats: the samples are small; a request for the file doesn’t mean it was later used in answers (our assessment); and EZY sells software for “AI visibility” and has a stake in the topic.[9] Prosopo notes that one of the requests came from Claude Code acting on a person’s request.[8]
Where it makes sense
A clear example is documentation. At the top of Claude Code’s documentation pages there is a hint: to find the page you need, load the documentation index from the llms.txt file.[1] For a large reference, such a map makes sense: an agent needs to find the right page quickly. For a small brochure site or landing page, we found no evidence that it helps.
Our assessment: llms.txt is cheap insurance, not a way to get traffic. Our own site publishes one, but we haven’t measured whether anyone reads it, and we don’t promise any result. If you make your own, generate it automatically from the same data as your pages so it doesn’t go stale. That is our advice, not a requirement of the format.
What matters more than the files
A rules file asks; a check catches mistakes. So in an AI-ready template, three things matter to us, and they work even without any “AI files.”
1. Text in files with a schema. In Astro you can keep a site’s content in “collections.” Each one has a schema: a list of fields and their types. If a file doesn’t match the schema, Astro shows a clear error.[10] Collections are defined in the file src/content.config.ts.[10] That way a helper can’t quietly break a page’s structure: the error is visible right away. Exactly when it appears (when you start the dev server or at build time), the documentation doesn’t say, so test this on your own template (see the next section).
2. One check command. astro build builds the site, and astro check checks the code, including types in .astro files, and exits with code 1 if it finds errors.[11] What matters most is having a check the helper can run by itself. Claude Code’s documentation says that without such a check, “looks done” is the only signal the helper has, and you end up doing the checking.[5] For a website template the minimum is a build command that passes without errors. It’s worth writing it into AGENTS.md.
3. Styling in one place. It’s convenient to keep colors and fonts in CSS variables: a value is set once and used everywhere.[12] Then a request like “make the accent color darker” means editing one line, not searching through a hundred files. That is our reasoning, not a measured result.
A build has its limit: it catches technical errors. Whether the text is good, the color readable and the links correct stays up to your eye. We wrote about such checks in our checklist for buying a template and in how to avoid “AI slop” on your site.
How to check a template in 10 minutes
You don’t need to read code. Open the project folder and check five things. If you’re only buying, ask the seller to show you, or ask directly.
- Is there an AGENTS.md? Open it. It should have commands, the places where text and styling live, and a list of don’ts. It should run a couple of screens, not hundreds of lines.
- Does the text live in files? Open the site in a browser, copy the heading of any page and search for it across the project. It should turn up in a text file, not inside a component.
- Is there a schema? Find
src/content.config.ts. The test: delete a required field in one text file and run the build. A clear error should appear. If the build passes with no errors, there is no schema, or it doesn’t check that field. Put the field back afterward. - Does the check pass on a clean copy? Download the project fresh, install the dependencies and run the build. It should finish without errors.
- Are colors and fonts in one place? Find the file with the CSS variables, change the main color and see whether it changes on every page.
Red flags:
- An AGENTS.md hundreds of lines long that describes every file. Overviews of how a project is organized didn’t help in the study,[3] and long files reduce how reliably the rules are followed.[1]
- Text written directly into components, and no schema.
- There is no build command, or it doesn’t pass.
- llms.txt exists, but there is no AGENTS.md, no schema and no check. The storefront is ready, but the template isn’t prepared for work. That is our assessment.
This check is item 10 of the same checklist.
Where these files won’t help
- They’re advice, not rules. Claude Code itself calls such files context, not required configuration.[1] The helper can break a rule. Your safety net is the build and your own review of the result.
- No proven overall gain. In the study, success didn’t reliably improve, and the cost rose by over 20% on average for files written by AI.[3] Results may differ for your site, but we have no measurements.
- A long file hurts. The documentation advises staying within 200 lines; otherwise rules get lost.[1]
- The file goes stale. You change a command or rename a folder, and the old version stays in the file. Our assessment: a wrong instruction is worse than none. The documentation advises reviewing the files regularly and removing outdated or conflicting rules.[1]
- llms.txt won’t replace a good site. It has no effect on Google Search,[2] and it won’t fix problems on the site itself (slow pages, empty text).
- Files won’t replace taste. Rules don’t make text good or design thoughtful. We cover this in how to avoid “AI slop” on your site.
Summary
- When you choose a template, check five things in 10 minutes: AGENTS.md, text in files, a schema, a check command, and styling in one place.
- Keep AGENTS.md short and about the non-standard. Claude Code 2.1.277+ reads it directly; add a CLAUDE.md with
@AGENTS.mdonly if you need Claude-only rules. - llms.txt is optional, and you shouldn’t expect miracles from it.
- The files are advice; the build and your own review are the real check.
If you want to see what this looks like in practice, take a look at the ShipThemes template catalog. Before you buy, ask us which of the five checks a template passes: hello@shipthemes.com. We described how to work with a ready-made template together with AI in our guide to working with a website template using AI.
Frequently asked questions
Do I need to write an AGENTS.md if I’m just editing text on my site with AI?
Most likely not: the file should come with the template. If you change the structure or add pages, a couple of lines with commands and places where things live won’t hurt. Write only what the helper doesn’t know on its own: the study found no overall benefit from such files.[3]
Can I have one file for all helpers?
Yes. Make AGENTS.md the main one: Claude Code 2.1.277 and later reads it without a CLAUDE.md. If you also keep a CLAUDE.md, start it with the line @AGENTS.md.[1] The agents.md site lists which tools read AGENTS.md; we haven’t verified that list.[4] So if you work in a different tool, check its documentation.
Can I generate the file automatically?
Claude Code has a /init command: it creates a starter CLAUDE.md based on the project structure.[5] But in the study, the lack of improvement showed up both for AI-written files and for developer-written ones.[3] Our advice: generate a draft, then keep only what the helper can’t find out on its own.[5]
Do I need llms.txt for my site to appear in AI answers or in Google?
There is no evidence for it. Google Search doesn’t use it,[2] and in site logs it is requested far less often than robots.txt.[8, 9] And even when the file is requested, that doesn’t mean it influences answers (our assessment).
What should I do if the helper breaks the rules in the file?
The documentation names two reasons: the file is too long and the rule gets lost in it, or the rule can’t have exceptions and needs an automatic check (a hook) instead of text.[5] Adding the word IMPORTANT probably won’t fix it, the same page says: shorten the file or move the rule into a hook.[5]
Where should llms.txt live, and how do I check that it’s there?
In the site root, that is, at your-site/llms.txt. A subfolder is allowed too: then the file describes the pages under that path.[6] You can check by simply opening the address in a browser; there is also an automatic presence check in Lighthouse (it needs Chrome 150 or later).[7]
Sources
Pages checked on October 26, 2026.
- Anthropic, “How Claude remembers your project” (Claude Code documentation)
- Google Search Central, “Google’s Guide to Optimizing for Generative AI Features on Google Search.” Page updated July 10, 2026.
- Gloaguen, T., Mündler-Sasahara, N., Müller, M. N., Raychev, V., Vechev, M., “Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?” arXiv:2602.11988 (v1: February 12, 2026; v3: September 29, 2026)
- AGENTS.md, “A simple, open format for guiding coding agents.”
- Anthropic, “Best practices for Claude Code.”
- llmstxt.org, “The /llms.txt file” (proposal by Jeremy Howard, September 3, 2024; version 2 changed August 10, 2026)
- Chrome for Developers, “Lighthouse agentic browsing scoring.” Page updated May 5, 2026.
- Chris Taylor, Prosopo, “llms.txt: we publish one, and no AI crawler has ever read it.” Published October 5, 2026, updated October 18, 2026. A measurement on a single site.
- EZY.ai, “We put llms.txt on 83 websites. In 12 weeks, OpenAI’s crawler read it 7 times.” Published July 27, 2026. The company sells software for “AI visibility.”
- Astro Docs, “Content collections.”
- Astro Docs, “CLI commands.”
- MDN Web Docs, “Using CSS custom properties (variables).”