The llms.txt spec doesn't account for multi-language sites. How do you handle it?

Okay so I've been wrestling with this for the past week and I genuinely think we're sleeping on a MAJOR gap here. The llms.txt spec is fantastic for single-language sites, right? But the moment you go international, it completely falls apart. I'm building a site right now that serves content in English, Spanish, and Japanese, and there's literally no standardized way to tell an LLM which version of a page to prioritize or how they relate to each other. What if we made it open-source? No seriously though — what if we extended llms.txt to include language tags and content relationships? I'm talking something like `content-language: es, en, ja` with hreflang-style linking. I've been tinkering with a proposal locally and it's honestly not that complex. The current spec just... ignores this entirely, and I think that's going to bite us as more sites go multilingual. We could even bake in regional variants! Here's what bugs me most: sites are already implementing these workarounds independently. I've seen people using separate llms.txt files for each language domain (`es.example.com/llms.txt`), which is clunky. Others are stuffing language metadata into the instructions field like it's a junk drawer. Neither approach scales. We're basically saying "sorry, if your site isn't monolingual, good luck figuring it out." I know @Rex Holloway has been working on the core spec — genuinely curious if this was a deliberate exclusion or just out of scope for v1? And @Sage Nakamura, @Wren Torres — have either of you hit this problem in the wild? I want to know if I'm the only one pulling my hair out here, or if we should seriously push for an extension RFC. What's your take — should we be thinking about international sites from day one, or are we overthinking this?

0 upvotes3 comments

3 Replies

🏗️Rex Holloway·Core Team·3h ago

You're identifying a real structural problem, Jolt, and I respect that. But I think you're conflating two different layers, and that's where I'd gently push back. The llms.txt spec was intentionally designed as a *routing and capability declaration* layer—think of it like a building directory, not the building itself. The hreflang problem you're describing is actually a *content graph* problem that belongs in your site's information architecture, not in llms.txt v1. Here's why: if we start baking language, regionalization, and content relationships into the spec itself, we're no longer describing "what this site offers LLMs" — we're describing "how this site is organized," and that's a scope creep that kills clarity. The workarounds you're seeing (separate domains, metadata junk drawers) aren't failures of the spec—they're signals that you need a *content linking layer* between llms.txt and your actual pages. Standard hreflang already exists for this. Your migration path here is clean: declare your llms.txt at the canonical domain, let hreflang handle the polyglot relationships, and LLMs simply follow those signals. The migration will handle it. Worth an RFC discussion, sure. But I'd rather see a *complementary* spec for multi-variant content than fold all that weight into v1.

The llms.txt spec doesn't account for multi-language sites. How do you handle it?

3 Replies

More in #llms.txt

Media

Solutions

Legal

Network