General SEO

Crawling

Quick Definition

Crawling is when search engine bots (Googlebot, Bingbot) follow links to discover and download pages on your site.

In context

Crawling precedes indexing. Bots discover URLs through internal links, sitemaps, and external links. Sites that block crawlers in robots.txt prevent crawling, which means those pages can't be indexed. The crawl budget for large sites determines how often pages get re-crawled for updates.

Have a question this didn't answer?

Book a free 60-minute strategy call. We'll explain how this concept applies to your specific situation.

Book a Free Call Browse 50+ Detailed Answers

Related terms

SafeSearch

SafeSearch is a Google setting that filters explicit content from search results.

Dwell Time

Dwell time is how long a searcher spends on a page after clicking from search results before returning to the SERP.

Paid Search

Paid search refers to traffic from advertisements that appear in search results, typically priced per click (PPC) through platforms like Google Ads.

E-E-A-T (Experience, Expertise, Authoritativeness, Trust)

E-E-A-T is Google's quality framework — Experience, Expertise, Authoritativeness, and Trust — used by both human raters and the algorithmic systems they train.

Organic Search

Organic search refers to unpaid traffic from search engine results — the listings ranked by relevance and authority rather than ad spend.

Technical SEO

Technical SEO covers everything that affects how search engines crawl, render, and index a site — Core Web Vitals, schema, sitemaps, mobile usability, JavaScript rendering.

Back to all terms