# OAI ChatGPT Search crawler User-agent: OAI-SearchBot Allow: / Allow: /sitemap.xml* Disallow: /api/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /login/ Disallow: /auth/ # OpenAI GPTBot crawler User-agent: GPTBot Allow: / Allow: /sitemap.xml* Disallow: /api/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /login/ Disallow: /auth/ # Anthropic crawlers User-agent: ClaudeBot Allow: / Allow: /sitemap.xml* Disallow: /api/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /login/ Disallow: /auth/ User-agent: Claude-SearchBot Allow: / Allow: /sitemap.xml* Disallow: /api/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /login/ Disallow: /auth/ # Perplexity crawler User-agent: PerplexityBot Allow: / Allow: /sitemap.xml* Disallow: /api/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /login/ Disallow: /auth/ # Common AI data crawler User-agent: CCBot Allow: / Allow: /sitemap.xml* Disallow: /api/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /login/ Disallow: /auth/ # Default rules User-agent: * Allow: /sitemap.xml* Disallow: /api/ Disallow: /admin/ Disallow: /dashboard/ Disallow: /login/ Disallow: /auth/ # Sitemap, Host Sitemap: https://www.riverbank.eu/sitemap.xml Host: www.riverbank.eu