NLWeb: What It Is and Who Should Care
NLWeb (Natural Language Web) is an open-source project from Microsoft that lets a website answer natural language questions about its own content, for people and for AI agents. It is real, it is documented, and its backers include serious names. Whether you need it is a different question.
This guide is for owners and marketers of content sites and online stores who keep seeing NLWeb in AI SEO newsletters. It covers what the project documents, what it does not claim, how it relates to llms.txt and schema, and a plain test for adopting it now or later.
On this page
- What NLWeb is and who built it
- How NLWeb works
- Does NLWeb help you show up in ChatGPT or Google?
- NLWeb vs llms.txt, schema markup and robots.txt
- Who should care about NLWeb right now?
- What can go wrong with NLWeb
- What to do instead this quarter
- Where NLWeb goes from here
What NLWeb is and who built it
NLWeb is a set of open protocols and open-source tools for building a conversational interface on top of a website. Microsoft announced it on May 19, 2025, describing it as a project to simplify natural language interfaces for websites, using the model and data of the publisher’s choice.
The announcement credits R.V. Guha as its creator and notes that Guha also created RSS, RDF and Schema.org. That lineage explains the design: NLWeb does not ask sites to publish anything new. It reads the semi-structured data they already publish.
Microsoft named early adopters and collaborators, including Tripadvisor, Shopify, Eventbrite, O’Reilly Media, Hearst and Snowflake. Treat that as a partner list, not a usage report. Neither the announcement nor the InfoWorld coverage reports adoption figures, and InfoWorld notes that uptake is still uncertain.
The reference code lives in the NLWeb GitHub repository under an MIT license. The repository itself frames its code as proof-of-concept demonstrations rather than the definitive solution.
How NLWeb works
NLWeb ingests your structured data, stores it in a vector database, and answers questions by letting a language model pick a tool and rank results. The diagram below shows the layers, with the one you already own highlighted.
Scroll the diagram sideways to see all of it.
According to the repository, the ingestion tools accept Schema.org, JSONL and RSS, and the project works with vector stores including Qdrant, Snowflake, Milvus, Azure AI Search, Elasticsearch and Postgres. Supported language models include OpenAI, Gemini, Anthropic, DeepSeek and Hugging Face. Schema App, a vendor that sells markup services, notes that JSON-LD is the preferred input and that the stored data enables semantic rather than keyword matching.
The “agent” half of the story rests on MCP. The Model Context Protocol is an open standard for connecting AI applications to external data and tools. Per Microsoft, every NLWeb instance is also an MCP server, and its core method is ask. That is the entire pitch: one install gives people a chat box and agents a structured door.
Does NLWeb help you show up in ChatGPT or Google?
Nothing in the NLWeb documentation says it does. That is the single most important thing to know, because most hype skips it.
The mechanism behind NLWeb only matters when an AI agent chooses to connect to your endpoint. Wix’s help article is direct about the limits: only AI agents that support NLWeb can query the site, the article does not list which agents those are, and NLWeb is separate from Google and Bing indexing and does not change how those engines crawl or rank your site.
I could not find a primary source showing that ChatGPT, Gemini, Perplexity or Google AI Mode call NLWeb endpoints on public sites when they build answers. Absence of evidence is not proof it will never happen, but it is a reason to avoid spending a quarter’s budget on it. How those engines actually pick sources is covered in how AI engines choose citations, and none of it depends on you running a vector database.
There is one connection worth noting. Wix warns that blocking AI crawlers in robots.txt may limit NLWeb functionality, because agents need to read your structured content. If you run NLWeb while also blocking agents, you have built a door and locked it. Our robots.txt guide for AI crawlers covers that decision.
NLWeb vs llms.txt, schema markup and robots.txt
NLWeb is a running service. llms.txt, schema markup and robots.txt are static files or markup. People lump them together as “AI files,” which hides how different they are.
Scroll the diagram sideways to see all of it.
The relationships matter more than the labels:
- Schema markup is NLWeb’s fuel. Without clean JSON-LD or feeds, the index is thin. Our guides to structured data for AI search and JSON-LD are the prerequisite reading.
- llms.txt is a different bet. It is a proposed static file, and as that guide explains, no major AI company has committed to reading it. NLWeb has at least a working protocol and named partners, but neither has a proven effect on citations.
- robots.txt is the gatekeeper. It decides whether agents can reach what NLWeb would expose.
Who should care about NLWeb right now?
Care now if your content is a large, well-marked-up list of items and you want on-site conversational search anyway. Everyone else can watch and wait.
The project’s own materials point to the best fit. InfoWorld says it suits sites structured as lists of items, such as recipes, events and places, especially with Schema.org markup. The repository mentions products, recipes, attractions and reviews. The test below turns that into three questions.
Scroll the diagram sideways to see all of it.
Applied to common site types, the test gives this:
| Site type | Passes the test? | Reason |
|---|---|---|
| Recipe, events or directory publisher with Schema.org on every page | Possibly | Content is already a structured list of items |
| Online store with a large catalog and clean product data | Possibly | Product markup is exactly what the index consumes |
| Local service business with 10 to 30 pages | No | Too little content for conversational search to add value |
| B2B or SaaS site with docs | Maybe | Only if docs are well structured and visitors search them often |
| Brochure site with no structured data | No | Nothing reliable to index, fix markup first |
For stores, the prerequisite is product schema that matches what is on the page. Shopify is on Microsoft’s partner list, but that tells you about collaboration, not about what a merchant can switch on today. I did not find a verified, merchant-facing Shopify setup, so I am not describing one.
What can go wrong with NLWeb
The documented limits are mostly about maturity, cost and control. None are fatal, and all are worth knowing before you start.
- Prototype status. The repository calls its code proof-of-concept demonstrations, says CI/CD pipelines are not yet included, and expects production deployments to bring their own UI and live databases.
- Statelessness. InfoWorld notes that the current implementation is stateless, so developers must manage conversation context themselves.
- Ongoing infrastructure. You need a vector database and a language model endpoint. Cloudflare’s integration crawls your site and deploys a Worker with
/askand/mcpendpoints, but the page calls it a public preview intended for experimentation and points production use to a support address. - Output quality depends on input quality. InfoWorld says results depend heavily on structured content. Messy or inconsistent markup gives messy answers, now spoken by a chatbot with your logo on it.
- Lock-in on hosted versions. On Wix, the article says you can switch the ASK protocol off but cannot disconnect the site from NLWeb after installation. It also supports only the site’s main language.
- Exposure. NLWeb only exposes public structured content, per Wix. Still, a conversational layer can surface old pages, discontinued products and outdated prices in one confident answer. Audit before you index.
What to do instead this quarter
Do the work NLWeb would sit on, because it pays off whether or not NLWeb ever matters. In order:
- Make sure agents and AI crawlers can reach your pages. Check robots.txt, rendering and server responses, as covered in agentic search optimization and JavaScript rendering for AI crawlers.
- Fix and validate your markup. Accurate JSON-LD for products, articles, events or organization data is the one input every approach shares.
- Keep your feeds clean. If you publish RSS or a product feed, make sure it matches the live site.
- Measure whether AI referrals are even arriving. Our guide to tracking AI traffic in GA4 and Search Console shows how, so you can judge a future protocol against real numbers.
- Test NLWeb cheaply if you are curious. If you are on Wix or already on Cloudflare, read the setup page, try it on a staging copy, and compare it against your existing site search.
None of this is glamorous. It is also what decides whether any machine interface, standard or not, returns something true about your business.
Where NLWeb goes from here
NLWeb is a credible, documented bet on a web where agents ask sites questions directly, and it has no verified effect on your AI visibility today. That combination makes it a reasonable thing to watch and a poor thing to build a quarter around.
Three signals would change that advice: a major assistant publicly documenting that it calls NLWeb endpoints, platform vendors such as Shopify shipping it as a plain setting, and production-grade hosted options moving out of preview. Any one of those turns the test above from “wait” to “pilot.”
Until then, structured data, crawler access and measurement carry the weight. If you want a second opinion on whether your markup and agent readiness are strong enough to build on, our AI SEO service starts with exactly that review.
Frequently asked questions
What is NLWeb?
NLWeb (Natural Language Web) is an open project from Microsoft for adding natural language interfaces to websites. It indexes content you already publish, such as Schema.org markup and RSS, answers questions about it with a language model, and exposes the result to people and to AI agents.
Is NLWeb the same as MCP?
No. MCP, the Model Context Protocol, is a general standard for connecting AI applications to data and tools. NLWeb is a specific project built on top of it: Microsoft states that every NLWeb instance is also an MCP server.
Do I need NLWeb to show up in AI search answers?
No. Nothing in the NLWeb documentation says that adopting it affects whether ChatGPT, Gemini, Perplexity or Google cite your pages. Wix states that NLWeb is separate from Google and Bing indexing and does not change how they crawl or rank a site.
Does NLWeb replace llms.txt or schema markup?
No. NLWeb consumes structured data such as JSON-LD, so schema markup is its input, not something it replaces. llms.txt is a separate, unadopted proposal for a curated Markdown file. NLWeb is a running service, while the other two are static files.
Is NLWeb free to use?
The code is open source under the MIT license, but running it is not free. You need a vector database and access to a language model, or a host that provides them. Cloudflare describes its NLWeb integration as a public preview intended for experimentation.
Which sites are a good fit for NLWeb?
Sites whose content is already a structured list of items, such as recipes, events, products, places and reviews, with clean Schema.org markup. Sites without structured data, or with only a few pages, have little to gain from it today.