How search engines work

In one line

Search engines work in three steps: crawling (discovering pages), indexing (storing and understanding them), and ranking (ordering them for each query).

The three steps

  1. 1Crawl: bots follow links and sitemaps to find pages
  2. 2Index: engines analyse and store pages they judge worth keeping
  3. 3Rank: for each query, they order indexed pages by relevance, quality and authority

Why it matters

Each step is a gate: a page that can't be crawled can't be indexed, and a page that isn't indexed can't rank. Technical SEO clears the first two gates so your content gets a chance at the third.

Frequently asked

What's the difference between crawling and indexing?

Crawling is finding a page; indexing is deciding it's worth storing and understanding it. Both must happen before ranking.

Why isn't my page ranking?

Check the gates in order: is it crawlable, is it indexed, and only then, is it relevant and authoritative enough?

Put this into practice

Grade your site against our 250-point AEO/GEO checklist and see exactly what to fix, free, no signup.

Run a free audit Free tools

Related

What is indexing?
Getting stored in the search database, the prerequisite to ranking at all.
What is crawl budget?
How much of your site engines will crawl, and when it actually matters.