How do AI shopping agents find products?

Most start from a crawl or a search index rather than your homepage: robots.txt to establish access, sitemap.xml to enumerate product URLs, then the product pages themselves to extract structured data. Some also consult retail feeds, but feeds do not replace an unreadable product page.

Do I need llms.txt for AI shopping agents?

It is a useful signal, not a requirement. llms.txt tells an agent where the meaningful parts of your store are and which policies apply; it does not compensate for missing Product schema. Roughly half of the stores in our sample serve one, mostly because their platform generates it automatically.

Will blocking AI crawlers protect my content?

It removes you from consideration. Retrieval crawlers such as OAI-SearchBot, ChatGPT-User, PerplexityBot and Claude-User are the ones that surface a store to a buyer in the moment — they are distinct from training crawlers, and blocking both together is the most common accidental self-exclusion we find.

What is the single highest-impact fix?

Server-rendered Product JSON-LD with price, currency and availability. Two-thirds of the stores we scanned failed here, and every downstream signal — comparison, policy weighting, preference — depends on it existing.

How do I know if my store is ready?

Run the free scan. It fetches your robots.txt, sitemap, a sample of product pages and your policy pages, then reports which of the five steps an agent would fail at and what to change.

ReadinessUpdated 2026-08-17

Make your store ready for AI shopping agents

AI shopping agents — ChatGPT shopping, Perplexity, Gemini, and the browser agents built on Claude and Gemini — now sit between your ecommerce store and a growing share of buyers. This is what they read, what silently disqualifies you, and how to close the gaps in the order that actually changes the outcome.

Definition

AI shopping agent readiness is a store's ability to be enumerated, parsed and confidently recommended by an autonomous shopping agent, which depends on crawl access, server-rendered product structured data, current availability and pricing, and machine-readable shipping and return terms.

signals decide the recommendation: access, schema, policy, preference

11%

of non-Shopify stores in our sample had readable product schema

30s

to scan a store and see which of the four is failing

What an agent reads, in order

An agent handling "find me X under $Y" does roughly the same five things every time. Each step can end your candidacy silently — there is no error message and no signal in your analytics.

1. Resolve access: Fetch robots.txt, check its own user-agent against the rules. A disallow ends the process here and you are never considered.
2. Enumerate the catalogue: Read the sitemap, or fall back to on-site links. If the sitemap lists only collections and blog posts, the agent has no products to consider.
3. Parse candidates: Fetch product pages, extract JSON-LD first, then microdata, then Open Graph. Fields it cannot extract are treated as unknown, not as favourable.
4. Resolve objections: Look for delivery window, return terms and warranty. Unknown terms are a risk factor weighed against the competitor whose terms were explicit.
5. Justify a choice: Produce a recommendation it can explain. Products with thin data lose to products with complete data even at a worse price, because the agent can defend the complete one.

The five silent failures

None of these show up in a human QA pass, and none produce a support ticket. All five are common in the stores we scan.

Failure	What the agent sees	Fix
Client-rendered JSON-LD	A product page with no product on it	Emit Product JSON-LD server-side in the initial HTML
Sitemap without product URLs	A store with no catalogue	Publish a product sitemap; verify it resolves and lists product URLs
Bot rule blocking retrieval agents	A store that refused the request	Allow OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-User, Googlebot explicitly
Availability never updated	A source that recommends out-of-stock products	Drive availability from live inventory, not a hardcoded InStock
Returns only in prose	An unresolved risk in the comparison	Add hasMerchantReturnPolicy with window, fees and method

Failure modes ranked by how often we see them

Fix order that actually changes the score

Readiness work compounds badly when done out of order. Nothing downstream matters if the agent cannot enumerate your catalogue, so the sequence is fixed.

In practice a store moving from the 25th to the 75th percentile of our benchmark does steps one through four; step five is what separates a good score from winning the comparison.

Access: Unblock retrieval agents. Publish a resolvable product sitemap.
Presence: Server-render Product JSON-LD on every product page.
Correctness: Make price, currency and availability reflect live state.
Completeness: Add brand, identifier, images, description, aggregate rating.
Objection handling: Add machine-readable return and shipping terms to the offer.
Preference: Run the head-to-head against a real competitor and fix what loses.

Frequently asked questions

How do AI shopping agents find products?: Most start from a crawl or a search index rather than your homepage: robots.txt to establish access, sitemap.xml to enumerate product URLs, then the product pages themselves to extract structured data. Some also consult retail feeds, but feeds do not replace an unreadable product page.
Do I need llms.txt for AI shopping agents?: It is a useful signal, not a requirement. llms.txt tells an agent where the meaningful parts of your store are and which policies apply; it does not compensate for missing Product schema. Roughly half of the stores in our sample serve one, mostly because their platform generates it automatically.
Will blocking AI crawlers protect my content?: It removes you from consideration. Retrieval crawlers such as OAI-SearchBot, ChatGPT-User, PerplexityBot and Claude-User are the ones that surface a store to a buyer in the moment — they are distinct from training crawlers, and blocking both together is the most common accidental self-exclusion we find.
What is the single highest-impact fix?: Server-rendered Product JSON-LD with price, currency and availability. Two-thirds of the stores we scanned failed here, and every downstream signal — comparison, policy weighting, preference — depends on it existing.
How do I know if my store is ready?: Run the free scan. It fetches your robots.txt, sitemap, a sample of product pages and your policy pages, then reports which of the five steps an agent would fail at and what to change.

Find out which step an agent fails at

The scan walks the same five steps an AI shopping agent does and tells you where it stops.