Tag

#Robots Txt

10 posts tagged Robots Txt: Shopify SEO and GEO guides, playbooks and research from the Nivk.com team.

Safe crawl: allow AI citations, block AI training | Perplexity · Nivk.com
Technical GEO

Safe crawl: allow AI citations, block AI training

Blocking all AI bots deletes you from ChatGPT answers; allowing everything donates your catalog to model training with nothing back. The middle path exists: bot-by-bot policy that welcomes citation crawlers and declines training-only ones. Here is the exact configuration.

Lawrence Dauchy··5 min
Crawlbeheer en noindex voor AI-zoeken op Shopify | Perplexity · Nivk.com
Technical GEO

Crawlbeheer en noindex voor AI-zoeken op Shopify

AI-crawlers hebben een beperkt budget. Zo houd je dunne filter- en zoekpagina's buiten de crawl, zodat je waardevolle pagina's worden opgehaald en geciteerd.

Lawrence Dauchy··4 min
Should You Block or Allow AI Crawlers on Shopify? | Grok · Nivk.com
Technical GEO

Should You Block or Allow AI Crawlers on Shopify?

A clear robots.txt decision guide for Shopify stores: which AI crawlers to allow, which to consider blocking, and why most stores want to be cited in AI search.

Lawrence Dauchy··7 min
Keep Private Wholesale Pages Out of AI Answers | Grok · Nivk.com
Compliance & Trust

Keep Private Wholesale Pages Out of AI Answers

Stop AI engines from surfacing private wholesale catalogs, drafts, and gated B2B pricing while keeping your public Shopify store fully citable. The layered fix.

Lawrence Dauchy··6 min
AI-crawlers toelaten in robots.txt op Shopify | Claude · Nivk.com
Technical GEO

AI-crawlers toelaten in robots.txt op Shopify

GPTBot blokkeren kost je zichtbaarheid in ChatGPT-zoeken. Zo laat je de juiste AI-crawlers toe in je Shopify-robots.txt en blijf je vindbaar.

Lawrence Dauchy··4 min