robots.txt Template for Canadian Ecommerce (2026)

The template approach in one paragraph

Most Canadian ecommerce robots.txt files are either too aggressive (blocking everything that produces a parameter, killing 30–60% of indexable pages) or too lax (allowing everything, creating massive duplicate-content from faceted nav). The right approach is targeted: block internal search, cart, account, checkout, and admin paths; allow everything else; let the sitemap tell Google what to prioritize. Below: copy-paste templates for Shopify, WooCommerce, Magento, and custom builds, plus an annotation explaining every line.

robots.txt principles for ecommerce

• Block, don't noindex, in robots.txt

  robots.txt tells crawlers not to fetch a URL. It can't tell them not to index a URL they reach via other links. Use noindex meta tags for the latter.
• Be surgical, not aggressive

  Blocking entire directories often catches valuable pages by accident. Block specific path patterns; allow the rest.
• Always include sitemap reference

  Sitemap directive tells crawlers where to find your sitemap. Always at the bottom of robots.txt.
• Test changes before deploying

  Google Search Console robots.txt Tester catches syntax errors and shows you which URLs are blocked. Use it.
• Don't try to hide content

  robots.txt is publicly readable. Anything you 'hide' is visible to anyone who looks. Use noindex + auth for content that should be private.

Shopify template

Shopify ships a default robots.txt that's reasonable for most stores. Since 2021, Shopify allows you to customize it via robots.txt.liquid in your theme. Recommended template:

User-agent: *

# Block internal search and filter URLs
Disallow: /search
Disallow: /*?q=
Disallow: /*?ls=
Disallow: /*&filter.
Disallow: /*?filter.
Disallow: /*?sort_by=

# Block cart/account/checkout
Disallow: /cart
Disallow: /orders
Disallow: /account
Disallow: /checkouts/
Disallow: /apple-pay/
Disallow: /policies/

# Block draft and hidden content
Disallow: /products/*?variant=
Disallow: /collections/all
Disallow: /collections/vendors
Disallow: /collections/types

# Allow CSS/JS for rendering
Allow: /*.css$
Allow: /*.js$

Sitemap: https://yourdomain.ca/sitemap.xml

WooCommerce template

WordPress + WooCommerce default robots.txt is minimal. The template below covers the patterns specific to WooCommerce.

User-agent: *

# WordPress admin and includes
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/

# WooCommerce cart, checkout, account
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/

# Add-to-cart URLs
Disallow: /*?add-to-cart=
Disallow: /*?wc-ajax=

# Internal search
Disallow: /?s=
Disallow: /search/

# WooCommerce attribute filters (block parameter combinations)
Disallow: /*?orderby=
Disallow: /*?filter_

# Theme files for rendering
Allow: /wp-content/uploads/
Allow: /wp-content/themes/

Sitemap: https://yourdomain.ca/sitemap_index.xml

Magento template

Magento generates many parameter-based URLs by default. The template below covers the most common patterns.

User-agent: *

# Magento system paths
Disallow: /app/
Disallow: /var/
Disallow: /lib/
Disallow: /pub/static/
Disallow: /admin/
Disallow: /index.php/

# Cart, checkout, account
Disallow: /checkout/
Disallow: /customer/
Disallow: /wishlist/
Disallow: /catalog/product_compare/

# Catalog filtering and sorting
Disallow: /catalogsearch/
Disallow: /*?p=
Disallow: /*?dir=
Disallow: /*?order=
Disallow: /*?limit=
Disallow: /*?mode=

# Sessions and tracking
Disallow: /*?SID=
Disallow: /*?___store=

Sitemap: https://yourdomain.ca/sitemap.xml

Custom build template

For custom-built ecommerce on Next.js, Remix, or similar. Adapt to your specific URL conventions.

User-agent: *

# Internal search and filter
Disallow: /search
Disallow: /*?q=
Disallow: /*?sort=
Disallow: /*?filter=

# Cart, checkout, account
Disallow: /cart
Disallow: /checkout
Disallow: /account
Disallow: /api/

# Admin / staging
Disallow: /admin/
Disallow: /studio/
Disallow: /preview/

# Allow Next.js static assets
Allow: /_next/static/
Allow: /_next/image

Sitemap: https://yourdomain.ca/sitemap.xml

Common mistakes to avoid

• Disallow: / on staging that gets copied to production

  The single most common catastrophic robots.txt mistake. Staging environment blocks all crawlers, then the staging file goes live with production. Result: zero indexation. Always sanity-check robots.txt on launch day.
• Blocking CSS and JavaScript files

  Google needs to render your page to understand it. Blocking /css/ or /js/ folders prevents rendering and damages indexation. Always allow CSS/JS.
• Blocking image directories

  Image search is non-trivial traffic for ecommerce. Blocking /images/ or /uploads/ kills image indexing. Don't.
• Using robots.txt to hide content from competitors

  robots.txt is public. Listing your secret URL paths in robots.txt actively advertises them to anyone who looks.
• Wildcard mistakes

  Disallow: /*? blocks every URL with any parameter — including pagination, sort, filter. Catches more than intended. Be specific (Disallow: /*?sort=).
• Forgetting the sitemap directive

  Cheap, high-leverage. Always include.
• Not testing after deploy

  Google Search Console → robots.txt Tester. Run after every change.

Frequently Asked Questions

Implementation questions from Canadian operators.