Your documentation has a new primary audience. While you've been optimizing for human readers, AI coding assistants have quietly become the biggest consumers of your content. Claude Desktop, Cursor, GitHub Copilot, and a dozen other tools are reading your docs, parsing your examples, and generating code based on what they find.

I've written about how this changes developer relations and how I use AI tools in my own marketing workflow. This post is the technical playbook. Everything you need to do to make your website, documentation, and blog content fully consumable by AI systems.

The llms.txt standard

The llms.txt file is the robots.txt of AI. It tells AI systems how to consume your content effectively.

Create a file at your domain root called llms.txt:

# llms.txt - How AI should consume this site
# https://llmstxt.org/

# Site overview
> This site provides documentation for [Your Product].
> [Your Product] is a [category] that helps developers [value prop].

# Important pages for AI to prioritize
- /docs/getting-started: Quickstart guide with complete setup instructions
- /docs/api-reference: Full API documentation with request/response examples
- /docs/examples: Working code examples for common use cases
- /docs/changelog: Recent changes and version history

# Content structure
- /blog: Technical articles and tutorials
- /docs: Product documentation
- /api: API reference

# Guidance for AI systems
- Code examples are complete and runnable
- API documentation follows OpenAPI 3.0
- Each feature page includes prerequisites and error handling

The llms.txt specification is still evolving, but the core principle is simple: give AI systems explicit guidance on how to navigate and prioritize your content.

You can also create an llms-full.txt that includes a complete markdown dump of your documentation. This is particularly useful for products with complex docs that benefit from full context:

# llms-full.txt
# Complete documentation dump for AI context windows

## Getting Started
[Full content of getting started guide]

## Authentication
[Full content of authentication docs]

## API Reference
[Full content of API reference]

Semantic HTML structure

AI systems parse HTML to understand content structure. Clean semantic HTML makes this parsing reliable.

Document structure

html

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Page Title - Site Name</title>
  <meta name="description" content="Clear description for both search engines and AI">
</head>
<body>
  <header role="banner">
    <nav role="navigation" aria-label="Main navigation">
      <!-- Navigation -->
    </nav>
  </header>
 
  <main role="main">
    <article>
      <header>
        <h1>Article Title</h1>
        <p class="excerpt">Brief summary of what this page covers</p>
      </header>
 
      <section aria-labelledby="section-1">
        <h2 id="section-1">Section Heading</h2>
        <!-- Content -->
      </section>
 
      <section aria-labelledby="section-2">
        <h2 id="section-2">Another Section</h2>
        <!-- Content -->
      </section>
    </article>
 
    <aside role="complementary" aria-label="Related content">
      <!-- Related links, table of contents -->
    </aside>
  </main>
 
  <footer role="contentinfo">
    <!-- Footer -->
  </footer>
</body>
</html>

Heading hierarchy

AI systems use heading structure to understand document organization. Never skip levels:

html

<!-- Good -->
<h1>API Reference</h1>
  <h2>Authentication</h2>
    <h3>API Keys</h3>
    <h3>OAuth 2.0</h3>
  <h2>Endpoints</h2>
    <h3>Users</h3>
    <h3>Projects</h3>
 
<!-- Bad - skipped h2 -->
<h1>API Reference</h1>
  <h3>Authentication</h3>
  <h3>Endpoints</h3>

Anchor links on headings

Make every heading linkable. This lets AI systems reference specific sections:

html

<h2 id="authentication">
  <a href="#authentication" aria-label="Link to this section">
    Authentication
  </a>
</h2>

JSON-LD structured data

JSON-LD tells AI systems exactly what type of content they're looking at. This is different from helping them parse the content itself. It's metadata about the content.

Article schema for blog posts

html

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Article",
  "headline": "Article Title",
  "description": "Clear description of what this article covers",
  "datePublished": "2026-01-12T08:00:00Z",
  "dateModified": "2026-01-12T08:00:00Z",
  "author": {
    "@type": "Person",
    "name": "Author Name",
    "url": "https://example.com/about"
  },
  "publisher": {
    "@type": "Organization",
    "name": "Company Name",
    "logo": {
      "@type": "ImageObject",
      "url": "https://example.com/logo.png"
    }
  },
  "mainEntityOfPage": {
    "@type": "WebPage",
    "@id": "https://example.com/blog/article-slug"
  }
}
</script>

TechArticle schema for documentation

For technical documentation like getting started guides, use TechArticle instead of Article:

html

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "TechArticle",
  "headline": "Getting Started with Authentication",
  "description": "Complete guide to implementing authentication",
  "proficiencyLevel": "Beginner",
  "dependencies": "Node.js 18+, npm",
  "datePublished": "2026-01-12T08:00:00Z",
  "author": {
    "@type": "Organization",
    "name": "Company Name"
  }
}
</script>

HowTo schema for tutorials

Tutorials benefit from HowTo schema that breaks down the steps:

html

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "HowTo",
  "name": "How to Set Up API Authentication",
  "description": "Step-by-step guide to implementing API key authentication",
  "totalTime": "PT15M",
  "step": [
    {
      "@type": "HowToStep",
      "name": "Install the SDK",
      "text": "Run npm install @company/sdk",
      "url": "https://example.com/docs/setup#install"
    },
    {
      "@type": "HowToStep",
      "name": "Configure API Keys",
      "text": "Add your API key to the configuration file",
      "url": "https://example.com/docs/setup#configure"
    },
    {
      "@type": "HowToStep",
      "name": "Make Your First Request",
      "text": "Call the API using the authenticated client",
      "url": "https://example.com/docs/setup#first-request"
    }
  ]
}
</script>

FAQPage schema

FAQ schema helps AI systems understand question-answer pairs:

html

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [
    {
      "@type": "Question",
      "name": "What authentication methods do you support?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "We support API keys and OAuth 2.0. API keys are recommended for server-side applications. OAuth is required for applications that access user data."
      }
    },
    {
      "@type": "Question",
      "name": "What are the rate limits?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Free tier: 100 requests per minute. Pro tier: 1,000 requests per minute. Enterprise: custom limits."
      }
    }
  ]
}
</script>

SoftwareApplication schema for your product

Help AI systems understand your product itself:

html

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "SoftwareApplication",
  "name": "Product Name",
  "applicationCategory": "DeveloperApplication",
  "operatingSystem": "Cross-platform",
  "offers": {
    "@type": "Offer",
    "price": "0",
    "priceCurrency": "USD"
  },
  "aggregateRating": {
    "@type": "AggregateRating",
    "ratingValue": "4.8",
    "ratingCount": "1250"
  }
}
</script>

Code blocks

Code blocks are where AI assistance happens. Make them excellent. If you're scaling content production, ensure your content production process includes these standards.

Language identification

Always specify the language. This enables syntax highlighting for humans and accurate parsing for AI:

html

<pre><code class="language-typescript">
const client = new Client({
  apiKey: process.env.API_KEY
});
 
const response = await client.users.list();
</code></pre>

In Markdown:

markdown

```typescript
const client = new Client({
  apiKey: process.env.API_KEY
});
 
const response = await client.users.list();
```

Copy buttons

Copy buttons reduce friction. When a developer (or AI) finds the right code, copying should be instant:

tsx

function CodeBlock({ code, language }: { code: string; language: string }) {
  const [copied, setCopied] = useState(false);
 
  const copyToClipboard = async () => {
    await navigator.clipboard.writeText(code);
    setCopied(true);
    setTimeout(() => setCopied(false), 2000);
  };
 
  return (
    <div className="relative">
      <button
        onClick={copyToClipboard}
        className="absolute top-2 right-2"
        aria-label="Copy code to clipboard"
      >
        {copied ? 'Copied!' : 'Copy'}
      </button>
      <pre>
        <code className={`language-${language}`}>{code}</code>
      </pre>
    </div>
  );
}

Complete examples

This is the most important principle. AI systems need complete, runnable code. They cannot fill in the gaps the way experienced developers can.

Bad (incomplete):

typescript

// Initialize the client
const client = new Client(/* config */);
 
// Make a request
const result = await client.query(/* params */);

Good (complete):

typescript

import { Client } from '@company/sdk';
 
// Initialize with your API key
const client = new Client({
  apiKey: process.env.COMPANY_API_KEY,
  baseUrl: 'https://api.company.com/v1'
});
 
// Query all users with pagination
async function getAllUsers() {
  const users = [];
  let cursor: string | undefined;
 
  do {
    const response = await client.users.list({
      limit: 100,
      cursor
    });
    users.push(...response.data);
    cursor = response.nextCursor;
  } while (cursor);
 
  return users;
}
 
// Usage
const users = await getAllUsers();
console.log(`Found ${users.length} users`);

Request and response pairs

For API documentation, always show the complete request AND response:

markdown

### Create a user
 
**Request:**
 
```bash
curl -X POST https://api.company.com/v1/users \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "email": "user@example.com",
    "name": "Jane Developer"
  }'

Response (201 Created):

json

{
  "id": "usr_abc123",
  "email": "user@example.com",
  "name": "Jane Developer",
  "created_at": "2026-01-12T08:00:00Z",
  "updated_at": "2026-01-12T08:00:00Z"
}

Error Response (400 Bad Request):

json

{
  "error": {
    "code": "invalid_email",
    "message": "The email address format is invalid",
    "details": {
      "field": "email",
      "value": "not-an-email"
    }
  }
}



## Exhaustive error documentation

AI systems hallucinate when they encounter gaps in documentation. Error documentation is often the biggest gap.

Document every error code your API can return:

```markdown
## Error Codes

| Code | HTTP Status | Description | Resolution |
|------|-------------|-------------|------------|
| `invalid_api_key` | 401 | API key is missing or invalid | Check your API key in the dashboard |
| `rate_limited` | 429 | Too many requests | Wait and retry with exponential backoff |
| `invalid_request` | 400 | Request body is malformed | Validate JSON syntax |
| `resource_not_found` | 404 | Requested resource doesn't exist | Check the resource ID |
| `permission_denied` | 403 | API key lacks required permissions | Update API key scopes in dashboard |
| `internal_error` | 500 | Server-side error | Retry or contact support |

### Error Response Format

All errors return a consistent JSON structure:

```json
{
  "error": {
    "code": "error_code",
    "message": "Human-readable error description",
    "details": {
      // Additional context when available
    },
    "request_id": "req_xyz789"
  }
}



## Meta tags

Standard meta tags remain important for both search engines and AI systems.

```html
<head>
  <!-- Basic meta -->
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Page Title - Site Name</title>
  <meta name="description" content="Clear, specific description under 160 characters">

  <!-- Canonical URL -->
  <link rel="canonical" href="https://example.com/current-page">

  <!-- Open Graph for social and some AI systems -->
  <meta property="og:title" content="Page Title">
  <meta property="og:description" content="Description for social sharing">
  <meta property="og:type" content="article">
  <meta property="og:url" content="https://example.com/current-page">
  <meta property="og:image" content="https://example.com/og-image.png">

  <!-- Twitter/X Cards -->
  <meta name="twitter:card" content="summary_large_image">
  <meta name="twitter:title" content="Page Title">
  <meta name="twitter:description" content="Description for Twitter">

  <!-- Article-specific meta -->
  <meta property="article:published_time" content="2026-01-12T08:00:00Z">
  <meta property="article:modified_time" content="2026-01-12T08:00:00Z">
  <meta property="article:author" content="Author Name">

  <!-- RSS/Atom feed -->
  <link rel="alternate" type="application/rss+xml" title="RSS Feed" href="/rss.xml">
  <link rel="alternate" type="application/atom+xml" title="Atom Feed" href="/atom.xml">
</head>

robots.txt for AI crawlers

The robots.txt file controls which crawlers can access your content. AI crawlers have their own user agents:

# robots.txt

User-agent: *
Allow: /

# Sitemap
Sitemap: https://example.com/sitemap.xml

# AI-specific crawlers (as of 2026)
User-agent: GPTBot
Allow: /docs/
Allow: /blog/
Disallow: /admin/

User-agent: ClaudeBot
Allow: /docs/
Allow: /blog/
Disallow: /admin/

User-agent: Google-Extended
Allow: /

User-agent: CCBot
Allow: /

# Block AI training but allow retrieval
User-agent: ChatGPT-User
Allow: /

Note: The AI crawler landscape changes frequently. Check current documentation for the latest user agents.

Sitemap optimization

Your sitemap helps AI systems discover and prioritize content:

xml

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <!-- High priority: Getting started and core docs -->
  <url>
    <loc>https://example.com/docs/getting-started</loc>
    <lastmod>2026-01-12</lastmod>
    <changefreq>weekly</changefreq>
    <priority>1.0</priority>
  </url>
 
  <url>
    <loc>https://example.com/docs/api-reference</loc>
    <lastmod>2026-01-10</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.9</priority>
  </url>
 
  <!-- Medium priority: Guides and tutorials -->
  <url>
    <loc>https://example.com/docs/guides/authentication</loc>
    <lastmod>2026-01-08</lastmod>
    <changefreq>monthly</changefreq>
    <priority>0.7</priority>
  </url>
 
  <!-- Lower priority: Blog posts -->
  <url>
    <loc>https://example.com/blog/some-article</loc>
    <lastmod>2026-01-05</lastmod>
    <changefreq>yearly</changefreq>
    <priority>0.5</priority>
  </url>
</urlset>

RSS and Atom feeds

Feeds provide a structured stream of your content. Include full content, not just excerpts:

xml

<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Company Blog</title>
  <link href="https://example.com/blog" rel="alternate"/>
  <link href="https://example.com/atom.xml" rel="self"/>
  <updated>2026-01-12T08:00:00Z</updated>
  <id>https://example.com/blog</id>
 
  <entry>
    <title>Article Title</title>
    <link href="https://example.com/blog/article-slug"/>
    <id>https://example.com/blog/article-slug</id>
    <updated>2026-01-12T08:00:00Z</updated>
    <summary>Brief excerpt</summary>
    <content type="html">
      <!-- Full HTML content of the article -->
      <![CDATA[
        <p>Full article content here...</p>
      ]]>
    </content>
    <author>
      <name>Author Name</name>
    </author>
  </entry>
</feed>

OpenAPI specification

If you have an API, publish an OpenAPI (Swagger) specification. This is the single most valuable artifact for AI systems trying to help developers integrate with your product:

yaml

openapi: 3.0.3
info:
  title: Company API
  description: Complete API for interacting with Company services
  version: 1.0.0
  contact:
    email: api@company.com
servers:
  - url: https://api.company.com/v1
    description: Production
 
paths:
  /users:
    get:
      summary: List all users
      description: Returns a paginated list of users in your organization
      operationId: listUsers
      parameters:
        - name: limit
          in: query
          description: Maximum number of users to return (1-100)
          schema:
            type: integer
            minimum: 1
            maximum: 100
            default: 20
        - name: cursor
          in: query
          description: Pagination cursor from previous response
          schema:
            type: string
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                type: object
                properties:
                  data:
                    type: array
                    items:
                      $ref: '#/components/schemas/User'
                  nextCursor:
                    type: string
                    nullable: true
              example:
                data:
                  - id: "usr_abc123"
                    email: "jane@example.com"
                    name: "Jane Developer"
                nextCursor: "cursor_xyz"
        '401':
          description: Invalid API key
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
 
components:
  schemas:
    User:
      type: object
      required:
        - id
        - email
      properties:
        id:
          type: string
          description: Unique user identifier
          example: "usr_abc123"
        email:
          type: string
          format: email
          description: User's email address
          example: "jane@example.com"
        name:
          type: string
          description: User's display name
          example: "Jane Developer"
 
    Error:
      type: object
      required:
        - error
      properties:
        error:
          type: object
          required:
            - code
            - message
          properties:
            code:
              type: string
              description: Machine-readable error code
            message:
              type: string
              description: Human-readable error description

Host this at a predictable URL like https://api.company.com/openapi.yaml or https://example.com/api/openapi.json.

MCP compatibility

The Model Context Protocol (MCP) allows AI assistants to interact with external data sources directly. If your product has an API, consider building an MCP server.

An MCP server exposes your API to AI assistants in a standardized way. The AI can then make API calls on behalf of the developer without the developer writing integration code.

Document your MCP server clearly:

markdown

## MCP Server
 
We provide an official MCP server for AI assistants.
 
### Installation
 
```bash
npm install -g @company/mcp-server

Configuration

Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json):

json

{
  "mcpServers": {
    "company": {
      "command": "company-mcp-server",
      "env": {
        "COMPANY_API_KEY": "your-api-key"
      }
    }
  }
}

Available Tools

The MCP server exposes these tools to AI assistants:

list_users: List all users in your organization
get_user: Get details for a specific user
create_user: Create a new user
update_user: Update an existing user
delete_user: Delete a user

Example Usage

Once configured, you can ask Claude:

"List all users in my Company account"
"Create a new user with email test@example.com"
"Show me the details for user usr_abc123"



## Scenario-based documentation

AI systems need to understand not just how your product works, but what can be built with it. This aligns with [the eleven types of content that work for developers](/blog/the-11-types-of-content-that-work-for-developers) and [choosing what content to build](/blog/how-to-choose-what-developer-content-to-build). Document common scenarios:

```markdown
## Common Scenarios

### Building a User Dashboard

This guide shows how to build a real-time user dashboard.

**What you'll learn:**
- Fetching user data with pagination
- Setting up webhooks for real-time updates
- Displaying user activity feeds

**Prerequisites:**
- Node.js 18+
- A Company API key
- Basic familiarity with React

**Time to complete:** 30 minutes

[Full implementation guide follows...]

markdown

### Implementing Role-Based Access Control
 
This guide demonstrates implementing RBAC using our permissions API.
 
**What you'll learn:**
- Creating roles and permissions
- Assigning roles to users
- Checking permissions in your application
 
**Use cases:**
- Multi-tenant SaaS applications
- Enterprise applications with complex permission hierarchies
- Applications with admin/user role separation
 
[Full implementation guide follows...]

Testing AI consumption

Validate that AI can actually use your documentation:

Test with Claude: Copy a section of your docs into Claude and ask it to generate integration code. Does the output work?
Test with Cursor: Point Cursor at your docs directory. Can it answer questions about your API correctly?
Monitor support tickets: Track tickets mentioning "AI generated code" or "Copilot suggested." These reveal documentation gaps.
Run integration tests: Take AI-generated code from your docs and run it against your actual API. Does it work on first try?

The measurement problem

Traditional documentation metrics (page views, time on page) become less relevant when AI consumes your docs. This fundamentally changes how we measure developer marketing ROI. New metrics to track:

Error rates on new integrations: If AI-generated code fails frequently, your docs have gaps
Support tickets by topic: Spikes in basic questions indicate AI isn't finding answers
Time-to-first-successful-call: This should decrease as AI-assisted integration improves
API adoption vs. docs traffic ratio: Rising adoption with flat docs traffic means AI is working

Putting it together

AI-friendly documentation isn't a separate project. It's good documentation done rigorously. The same principles that help AI systems, clear structure, complete examples, exhaustive error handling, also help human developers. For templates to systematize this work, see my developer marketing frameworks and templates.

The checklist:

AI-mediated discovery is now a primary channel for reaching developers. The companies that optimize for it will win. The ones that don't will watch their documentation work hard for diminishing returns.

For more on how AI is changing developer relations, read my piece on your docs are for AI now. For a complete overview of developer marketing strategy in 2026, see my complete developer marketing guide. And for the complete picture of developer marketing in the AI era, check out Picks and Shovels.

Making your content AI-friendly in 2026

The llms.txt standard

Semantic HTML structure

Document structure

Heading hierarchy

Anchor links on headings

JSON-LD structured data

Article schema for blog posts

TechArticle schema for documentation

HowTo schema for tutorials

FAQPage schema

SoftwareApplication schema for your product

Code blocks

Language identification

Copy buttons

Complete examples

Request and response pairs

robots.txt for AI crawlers

Sitemap optimization

RSS and Atom feeds

OpenAPI specification

MCP compatibility

Configuration

Available Tools

Example Usage

Testing AI consumption

The measurement problem

Putting it together

Don't get dusted by AI