What it does
ai.txt is a file format from Spawning, the company behind the Do Not Train registry. It sits at the root of your site and says which kinds of files AI training datasets may use.
It works by file type rather than by crawler. That makes it useful when you're fine with AI reading your articles but don't want your photos or recordings in a training set.
How to use it
- 1Under “Which files may AI training datasets use?”, set “Allow” or “Block” for Text and documents, Images, Audio, Video and Code. Images, audio and video start blocked.
- 2Check the file extensions listed under each type, so you know what each switch covers.
- 3Add “Your site (optional, for the comment line)” if you want your domain noted at the top.
- 4Press “Copy” or “Download” and upload the file as /ai.txt at the root of your domain.
ai.txt and robots.txt do different jobs
robots.txt asks crawlers not to fetch pages. ai.txt doesn't stop anything from fetching. It states what you allow for training, for tools that check it.
If you want to stop AI training crawlers like GPTBot, ClaudeBot or CCBot from collecting your pages, use robots.txt. ai.txt adds a statement about file types on top of that. Many sites use both.
What the file says
The output starts with a comment naming the format, then a “User-Agent: *” line and one Allow or Disallow line per file extension, such as “Disallow: *.jpg”. If you block every type, it writes a single “Disallow: /” instead.
Keep the file at the root, served as plain text. Change it whenever your policy changes; there's nothing to register.
Questions
Who reads ai.txt?
Tools that respect Spawning's format, such as its Do Not Train registry. It isn't a web standard, so don't rely on it alone. Pair it with robots.txt.
Is ai.txt legally binding?
It's a statement of your wishes, not a technical block or a contract. Whether it carries legal weight depends on where you are, so ask a lawyer if that matters to you.
Does ai.txt affect search engines?
No. Search crawlers don't use it. Your search visibility is set by robots.txt and your page tags.
Can I block only some images?
This generator works by file type across the whole site. For finer control, edit the output by hand or use robots.txt paths.
How do I see whether AI tools still visit?
Look for AI crawler user agents in your server logs. atrby tracks AI crawler visits and shows which AI assistants send visitors who buy.