Audit website for AI crawler readiness
A detailed report on how public AI crawlers interact with the target site, including blocked policies and actionable fixes that require no purchase.
How it works
Read-onlyResearch and knowledge
The agent uses the audit_public_website tool to check public identity, offer evidence, and discovery files. It then uses check_ai_crawler_policy to evaluate homepage policies for eight major AI crawler tokens. Finally, it synthesizes these findings into a summary of blocked/partial access and suggests free fixes, ensuring no purchase tools are invoked.
Prompt
Paste this into an agent that has the servers below connected. Adjust the names and dates to your own.
Prompt for Audit website for AI crawler readiness
Audit https://example.com for AI-agent readiness. Summarize the blocked and partial findings, then tell me which fixes I can make without buying anything.MCP servers it needs
Source
The vendor page this playbook is drawn from, and the passage that supports it.
Your agent can audit a public website, inspect eight AI crawler policies, and compare three clearly priced offers—including recurring Crawler Watch monitoring—without credentials, write access, or purchasing power.Read-only website audit MCP server — ActableSite