A detailed report on whether AI assistants can access and read the site, including specific issues with robots.txt rules, CDN bot protection, and missing content.
The agent uses list_ai_crawlers to understand which crawlers impact visibility, then uses check_site to perform a live audit. It verifies if robots.txt allows key crawlers like OAI-SearchBot, if the server serves content correctly (bypassing CDN blocks), and if the HTML contains readable text. It checks for noindex/nosnippet tags and validates structured data.
- Category: Research and knowledge
- Effect: Read-only
- MCP servers: Agentwatch MCP (https://koskamo.com/explore/agentwatch-mcp)
- Source: Agentwatch — see your site the way an AI assistant sees it (https://comiza.lighting/agentwatch/)
- Source quote: "check_site returns the verdict, the findings with their fixes, and a link to the readable report."
- Playbook page: https://koskamo.com/explore/use-cases/audit-ai-crawler-visibility-and-robots-txt-compliance
Prompt: Check the AI visibility for https://example.com. First, list the relevant AI crawlers. Then, run a full site check to identify any issues with robots.txt, server responses, or content readability that would prevent AI assistants from citing the site.