# These are paid-traffic landing pages. They are PUBLIC (customers arrive from # ads and must be able to open them) but must NEVER appear in organic search # results, and must not feed AI training/scraping corpora. # # How non-ranking is guaranteed: every page serves "noindex" (as a # and an X-Robots-Tag response header). Search engines are therefore deliberately # ALLOWED to crawl here — a crawler must fetch a page to see its noindex and drop # it. Blocking search crawlers in robots.txt would BACKFIRE: an uncrawlable URL # can still surface as a bare link in results because the noindex is never read. User-agent: * Allow: / # AI crawlers/trainers have no "noindex" equivalent, so robots.txt is the only # lever for them: keep them out entirely. This list is ILLUSTRATIVE, not # authoritative — verify current user-agent tokens against each vendor's docs. User-agent: GPTBot Disallow: / User-agent: OAI-SearchBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Google-Extended Disallow: / User-agent: PerplexityBot Disallow: / User-agent: CCBot Disallow: / User-agent: Bytespider Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: meta-externalagent Disallow: /