Koskamo

Web Search API Eval MCP

A read-only MCP server that provides access to a benchmark of web search APIs including You.com, Tavily, Exa, and Parallel. It allows AI agents to execute agent queries and retrieve results for comparison and analysis without requiring authentication or storing data.

Endpoint and access

No sign-in to connect and list tools. Individual tools may still need a vendor account, credentials, or permissions; check the documented prerequisites.

Web Search API Eval MCP logo, by psinglapCommunity listingStreamable HTTPNo sign-inEndpoint checked 2026-11-01Published by psinglap
MCP endpoint (Streamable HTTP)
https://websearch-api-eval.psberkeleymba.workers.dev/mcp
Vendor documentation

What you can do with it

  • Benchmark web search API results
  • Execute agent queries on search engines
  • Compare search results from multiple providers

Source and maintenance

Community listingRegistry 1.1.0
mcp-registry: io.github.psinglap/websearch-api-eval@1.1.0Source repositoryProject website
Maintenance checked

Protocol and capabilities

Server
websearch-api-eval 1.1.0
MCP protocol
2025-06-18
Tools
11 advertised
Resources
Not advertised
Resource templates
Not advertised
Prompts
Not advertised
Server-advertised capabilities
{
  "tools": {
    "listChanged": true
  }
}

Server-provided instructions

Untrusted server output shown for inspection. It is not Koskamo guidance and is excluded from knowledge search.

Published benchmark of four web search APIs for AI agents (You.com, Tavily, Exa, Parallel): 30 agent-style queries across GTM, insurance and finance, 3 runs each, run on 2026-10-08. Use get_overall_results for the headline and overall scorecard, get_scorecard or compare_providers for numbers, recommend_provider for 'which API should I use', get_question for a specific query, get_charts for the published graphs, get_methodology for how it was measured, and how_to_run_locally to rerun it. Results are a fixed snapshot; always mention the run date and link the site.

Tools

11 advertised

get_overall_results — Overall results
Advertised by the server during the last successful anonymous check.
Advertised read-only

Headline findings and the overall scorecard across all 30 queries (90 calls per API): accuracy, fully correct, precision@5, freshness, empty responses, failed calls, latency p50/p95, cost per correct answer. Also the per-use-case summary, which API fits which agent, and developer activation time.

get_overall_results input schema
{
  "type": "object",
  "properties": {}
}
get_overall_results annotations
{
  "title": "Overall results",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
get_scorecard — Scorecard
Advertised by the server during the last successful anonymous check.
Advertised read-only

Full metric scorecard for one use case (gtm, insurance, finance) or 'overall'. Metrics: search-only latency p50/p95 (server and client), calls under 1 s, accuracy, freshness (% in window, newest relevant result age), content in one call, tokens, search cost.

get_scorecard input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "required": [
    "use_case"
  ],
  "properties": {
    "use_case": {
      "enum": [
        "overall",
        "gtm",
        "insurance",
        "finance"
      ],
      "type": "string",
      "description": "overall, gtm, insurance or finance"
    }
  }
}
get_scorecard annotations
{
  "title": "Scorecard",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
compare_providers — Compare providers
Advertised by the server during the last successful anonymous check.
Advertised read-only

Compare two or more of You.com, Tavily, Exa and Parallel side by side, overall or within one use case, optionally filtered to metrics whose name contains a keyword (e.g. 'latency', 'accuracy', 'fresh', 'cost', 'empty').

compare_providers input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "required": [
    "providers"
  ],
  "properties": {
    "use_case": {
      "enum": [
        "overall",
        "gtm",
        "insurance",
        "finance"
      ],
      "type": "string",
      "default": "overall",
      "description": "overall, gtm, insurance or finance"
    },
    "providers": {
      "type": "array",
      "items": {
        "enum": [
          "You.com",
          "Tavily",
          "Exa",
          "Parallel"
        ],
        "type": "string"
      },
      "minItems": 2,
      "description": "Two or more of: You.com, Tavily, Exa, Parallel"
    },
    "metric_filter": {
      "type": "string",
      "description": "Optional keyword to keep only matching metrics"
    }
  }
}
compare_providers annotations
{
  "title": "Compare providers",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
recommend_provider — Recommend a provider
Advertised by the server during the last successful anonymous check.
Advertised read-only

Which web search API fits an agent, based on this benchmark: live voice/chat turns, deep research, semantic or long-tail queries, docs lookups, or a specific use case (GTM, insurance, finance). Returns the guidance with the supporting numbers.

recommend_provider input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "properties": {
    "agent_need": {
      "type": "string",
      "description": "What the agent does, e.g. 'voice agent answering in under a second' or 'finance research'"
    }
  }
}
recommend_provider annotations
{
  "title": "Recommend a provider",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
list_questions — List questions
Advertised by the server during the last successful anonymous check.
Advertised read-only

The 30 benchmark questions (G1-G10 GTM, I1-I10 insurance, F1-F10 finance): ID, what the agent is asked, the search call and freshness.

list_questions input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "properties": {
    "use_case": {
      "enum": [
        "all",
        "gtm",
        "insurance",
        "finance"
      ],
      "type": "string",
      "default": "all",
      "description": "all, gtm, insurance or finance"
    }
  }
}
list_questions annotations
{
  "title": "List questions",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
get_question — Get a question
Advertised by the server during the last successful anonymous check.
Advertised read-only

One benchmark question by ID (e.g. G3, I4, F5): the task, search call, parameters, answer key with sources, and what each API returned with its accuracy score (0-2, mean of 3 runs).

get_question input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "required": [
    "id"
  ],
  "properties": {
    "id": {
      "type": "string",
      "description": "Question ID, e.g. G3, I4 or F5"
    }
  }
}
get_question annotations
{
  "title": "Get a question",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
get_charts — Charts
Advertised by the server during the last successful anonymous check.
Advertised read-only

The published graphs: overall scorecard, key metrics and per-query grids for GTM, insurance and finance, and developer activation. Without chart_id, lists them with titles and image links; with chart_id, also returns the image.

get_charts input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "properties": {
    "chart_id": {
      "type": "string",
      "description": "e.g. overall_scorecard, finance_key_metrics, gtm_query_grid, time_steps_activation"
    },
    "use_case": {
      "enum": [
        "all",
        "gtm",
        "insurance",
        "finance",
        "setup"
      ],
      "type": "string",
      "default": "all",
      "description": "Filter the list"
    }
  }
}
get_charts annotations
{
  "title": "Charts",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
get_methodology — Methodology
Advertised by the server during the last successful anonymous check.
Advertised read-only

How the benchmark was run and graded: query design, identical request parameters, 3 runs per query, latency timing, SimpleQA and FreshQA grading, the 10 metrics and the standards they follow, and limits.

get_methodology input schema
{
  "type": "object",
  "properties": {}
}
get_methodology annotations
{
  "title": "Methodology",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
how_to_run_locally — Run it yourself
Advertised by the server during the last successful anonymous check.
Advertised read-only

How to rerun this benchmark on your own machine with your own API keys, including on your own queries. Nothing runs on this server.

how_to_run_locally input schema
{
  "type": "object",
  "properties": {}
}
how_to_run_locally annotations
{
  "title": "Run it yourself",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
search — Search the benchmark
Advertised by the server during the last successful anonymous check.
Advertised read-only

Search the Web Search API Eval benchmark (results, questions, methodology, charts, how to run). Returns matching document IDs, titles and URLs.

search input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "required": [
    "query"
  ],
  "properties": {
    "query": {
      "type": "string",
      "description": "What to look for"
    }
  }
}
search annotations
{
  "title": "Search the benchmark",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
fetch — Fetch a benchmark document
Advertised by the server during the last successful anonymous check.
Advertised read-only

Full text of one benchmark document by its ID (for example question-F5, scorecard-finance, overall, methodology).

fetch input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "required": [
    "id"
  ],
  "properties": {
    "id": {
      "type": "string",
      "description": "Document ID from search"
    }
  }
}
fetch annotations
{
  "title": "Fetch a benchmark document",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}

Prompts

Not advertised

No prompt definitions available.

Resources

Not advertised

No resource definitions available.

30-day availability

Aggregated from read-only remote checks; this is not a tool-execution or security guarantee.

Successful checks
1 of 1 (100%)
Average latency
214 ms
95th percentile latency
214 ms

Maintenance history

Public check and source-refresh events. Editorial and ownership records remain private.

  • · probe · passed · 214 ms · metadata changed · 11 tool(s)

Connect from your client

Each snippet registers the same endpoint; pick the client you use.

This server is operated by the named third party. Listing and endpoint checks are not an endorsement or a security audit. Review the operator's permissions, terms, and data handling before connecting. How listings are checked and corrected.

Claude Code

Run this once in a terminal. It registers the remote server. Use --scope project to write .mcp.json for the repo.
Claude Code configuration
claude mcp add --transport http web-search-api-eval-mcp-workers https://websearch-api-eval.psberkeleymba.workers.dev/mcp

Codex

Run this once. Codex stores MCP servers in ~/.codex/config.toml (or .codex/config.toml in a trusted project).
Codex configuration
codex mcp add web-search-api-eval-mcp-workers --url https://websearch-api-eval.psberkeleymba.workers.dev/mcp

Cursor

Merge this server into .cursor/mcp.json in your project, then enable web-search-api-eval-mcp-workers in your MCP settings.
Cursor configuration
{
  "mcpServers": {
    "web-search-api-eval-mcp-workers": {
      "url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
    }
  }
}

Cline

Merge this into cline_mcp_settings.json (Cline → MCP Servers → Configure).
Cline configuration
{
  "mcpServers": {
    "web-search-api-eval-mcp-workers": {
      "type": "streamableHttp",
      "url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp",
      "disabled": false
    }
  }
}

VS Code

Merge this server into .vscode/mcp.json. Use MCP: List Servers to start web-search-api-eval-mcp-workers and inspect its tools.
VS Code configuration
{
  "servers": {
    "web-search-api-eval-mcp-workers": {
      "type": "http",
      "url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
    }
  }
}

Gemini CLI

Run this once. It writes .gemini/settings.json in the current project; pass --scope user for ~/.gemini/settings.json.
Gemini CLI configuration
gemini mcp add --transport http web-search-api-eval-mcp-workers https://websearch-api-eval.psberkeleymba.workers.dev/mcp

Claude Desktop

Merge this into claude_desktop_config.json (macOS: ~/Library/Application Support/Claude/, Windows: %APPDATA%\Claude\), then restart Claude Desktop.
Claude Desktop configuration
{
  "mcpServers": {
    "web-search-api-eval-mcp-workers": {
      "url": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
    }
  }
}

Windsurf

Merge this into ~/.codeium/windsurf/mcp_config.json, then refresh MCP in Cascade. Windsurf remote servers use serverUrl, not url.
Windsurf configuration
{
  "mcpServers": {
    "web-search-api-eval-mcp-workers": {
      "serverUrl": "https://websearch-api-eval.psberkeleymba.workers.dev/mcp"
    }
  }
}

Let your agent find its tools

Find servers like Web Search API Eval MCP from your agent. Add Koskamo once to search the index for endpoints, access requirements, and client setup. Public discovery needs no key.

Connect Koskamo Explore workflows
See all 3,975