User-agent: GPTBot Allow: / User-agent: PerplexityBot Allow: / User-agent: ClaudeBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: OAI-SearchBot Allow: / User-agent: Google-Extended Allow: / User-agent: CCBot Allow: / # Aggressive SEO / marketing scrapers — no crawl budget for them. User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: Bytespider Disallow: / User-agent: PetalBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: Barkrowler Disallow: / User-agent: serpstatbot Disallow: / User-agent: * Allow: / Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php # Forum crawl traps: user profiles, tag archives and their paginated # sub-views carry no SEO value and explode the crawl surface (incl. the # translated /xx/ language variants generated by PageTrans). Disallow: /forums/users/ Disallow: /forums/topic-tag/ Disallow: /forums/reply/ Disallow: /*/forums/users/ Disallow: /*/forums/topic-tag/ Disallow: /*/forums/reply/ Disallow: /forums/*/replies/ Disallow: /forums/*/favorites/ Disallow: /forums/*/engagements/ Disallow: /forums/*/subscriptions/ Disallow: /forums/*/topics/ # PageTrans variant query parameter — duplicate content. Disallow: /*?v= Sitemap: https://opensprinklershop.de/sitemap.xml Sitemap: https://opensprinklershop.de/product-sitemap.xml