AI optimization – every page now speaks fluent AI

AI agents – ChatGPT, Claude, Perplexity – read, summarise and cite your website. But most sites serve them HTML built for browsers: navigation, scripts, div soup. Result: wrong quotes, stale facts, missed points.

AI optimization is the opposite: make the content readable, discoverable and citable for machines – without breaking anything for humans. Here's what I built.

The sign at the front door

The first thing a bot fetches is robots.txt. Mine welcomes them:

Content-Signal: search=yes, ai-input=yes, ai-train=yes

# For AI agents: every page has a markdown twin at its own URL + index.md
# Markdown sitemap: https://www.blackbook.dk/sitemap.md
# LLM reference: https://www.blackbook.dk/llms.txt

Four lines – and the agent knows the entire machine layer without crawling a single page.

A markdown twin for every page

Append index.md to any URL and you get clean markdown with YAML frontmatter: title, description, canonical URL, last updated. No scripts, no noise.

The twins are generated from the same CMS data as the HTML – they can never fall behind. Every HTML page announces its twin via <link rel="alternate" type="text/markdown">.

On top: three indexes. llms.txt (the calling card), llms-full.txt (the full reference), sitemap.md (the page index) – both languages.

Structured data for those who read HTML

Many agents still parse HTML. For them, every page carries JSON-LD: Person and Organization with stable IDs, breadcrumbs, skills and topics linked to Wikidata. The point is the entity, not the single page: land on any corner, read the whole picture, cite with the correct attribution.

404 in markdown

A dead markdown URL doesn't return an empty HTML error page, but a markdown response with a map of the exits. The status code is 404 – but the conversation doesn't stall.

What we did not build

Checklists are easy. The rejections are the hard part:

Content negotiation – markdown for anyone with an Accept: text/markdown header. Elegant, but on a cached site one poisoned cache key is enough to serve your homepage as plain text. Rejected.

Try it yourself

curl https://www.blackbook.dk/en/services/index.md
curl https://www.blackbook.dk/sitemap.md
curl https://www.blackbook.dk/this-page-does-not-exist/index.md

Does it work?

OpenAI, Anthropic, Meta, Amazon, Google, Bing, Apple, Perplexity – all the big ones have been through. The first arrived within hours of launch, and ChatGPT is now the site's second-largest search channel after Google.

The machines are reading along.

The AI overlords

I asked my AI after we built all of this:

Me: So won't our future AI overlords kill me last now?

Claude: … at least you'll be quoted correctly while it lasts.