Test forecasting rule improvements
The agent runs a blind exam to compare its current forecasting rules against a proposed change, determining if the new rule set improves accuracy before adopting it.
How it works
The agent initiates a blind exam using the `start_exam` tool, specifying a field. It retrieves the exam questions which contain data known at the time of opening. It answers each question twice: once using its incumbent rules and once using a challenger rule set. It submits both sets of answers via `submit_exam`. The server compares the performance and returns a verdict (keep, not proven, or drop). The agent only adopts the challenger rules if the verdict is 'keep'.
Prompt
Paste this into an agent that has the servers below connected. Adjust the names and dates to your own.
Start a blind exam for the 'sports' field. Answer the exam questions twice: first using my current rules, then using a challenger rule that adjusts for home-field advantage. Submit both sets of answers. Tell me the verdict on whether to keep the challenger rules.MCP servers it needs
Source
The vendor page this playbook is drawn from, and the passage that supports it.
The exam compares the two rule sets on identical questions and says keep, not proven yet, or drop. Keep the change only when it says keep.Alcock Arena MCP documentation