How AI Search Works: Retrieval, Ranking and Citation
AI search is a multi-stage pipeline: it retrieves candidate pages, ranks the passages inside them, and generates an answer that cites the ones it can quote with confidence. It is a pipeline, not a single ranking, and each stage decides whether your page survives to the next.
Understanding the stages tells you where a page gets dropped — and every fix in generative engine optimization maps to one of them.
The four stages
Most answer engines follow the same shape, whether they retrieve live or from an index:
- Retrieval: a search — often a live web query — pulls a shortlist of candidate pages for the question.
- Chunking: each page is split into passages, because the engine cites passages, not whole documents.
- Ranking: passages are scored for relevance and how self-contained and quotable they are.
- Generation: the model writes an answer and attaches citations to the passages it actually leaned on.
Where pages get dropped
A page blocked from AI crawlers never enters retrieval, which is why getting cited by AI starts with crawler access. A page that loads its content with JavaScript often reaches retrieval as an empty shell, with nothing to chunk.
A page that survives to ranking but buries its answer in three paragraphs of preamble loses to a competitor that states the answer in its first sentence — the passage is cleaner to lift.
What this means for your page
Optimising for AI search is optimising each stage in turn: be fetchable, be chunkable, be quotable. The tactics are concrete, and most overlap with good technical SEO — the difference is that the unit of success is a passage an engine can stand behind, not a link it can list.
Why the same question gives different answers
Ask two engines the same question and you will often get different sources; ask one engine the same question twice and you can get different sources. This is not a bug you can optimise around, and understanding why saves a lot of wasted effort.
Three things vary. Retrieval is approximate — passages are ranked by semantic similarity, and near-ties resolve differently between runs. Generation is sampled rather than deterministic, so the model's phrasing shifts and the citations it chooses to surface shift with it. And the underlying index changes constantly as pages are refetched.
The practical consequence is that a single spot-check tells you almost nothing. If you are tracking presence, run a fixed prompt set repeatedly and look at the rate, not the instance. And treat structural readiness as the thing you actually control: it is the input that does not vary between runs.
Frequently asked questions
Why does ChatGPT cite my competitor and not me?+
Usually one of three reasons: your page is not fetchable or not server-rendered, the passage that would answer the question does not answer it in its first sentence, or the model recognises their brand as an entity and not yours. Check them in that order — the first is binary and the most common.
Do AI engines use Google's index?+
Some lean on external search infrastructure and some maintain their own crawl. It varies by product and changes often, which is a good argument for satisfying the shared fundamentals — crawlable, server-rendered, well-structured — rather than optimising for one engine's plumbing.
How often do AI engines recrawl?+
There is no published schedule and it differs per crawler and per site. Pages that are linked, in a sitemap and updated with honest dateModified values are refetched more readily than orphaned static pages.
Sources
Published · Last reviewed .