retriEVAL MCP

A local-first LLM evaluation server that scores AI outputs for faithfulness, relevancy, and hallucination. It supports various judge models including local Ollama instances, tracks run history via a dashboard, and integrates with MCP clients like Cursor and Claude Desktop to facilitate RAG and generation quality assessment.

Visit hcarrillo001

Package and access

Local package with source-attributed installation metadata. Koskamo does not execute packages during review.

Community listingLocal stdioSource reviewed; not executedPublished by hcarrillo001

Vendor documentation

What you can do with it

Score RAG outputs by stage to distinguish retrieval misses from generation misses
Track evaluation history and view charts on a local dashboard
Run CI gates using fast Jev judge for quick pass/fail on claims
Author custom metrics in plain English for specific evaluation needs

Source and maintenance

Community listingRegistry 1.0.0

mcp-registry: io.github.hcarrillo001/retrieval-mcp@1.0.0 Source repository Project website

Maintenance checked: Oct 31, 2026, 10:59 PM UTC

Packages and installation

Source-provided package data. Koskamo has not executed or live-probed this package.

ghcr.io/hcarrillo001/retrieval-mcp:1.0.0

oci

ghcr.io/hcarrillo001/retrieval-mcp:1.0.0 package definition

{
  "transport": "stdio",
  "identifier": "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0",
  "registryType": "oci",
  "environmentVariables": [
    {
      "name": "ANTHROPIC_API_KEY",
      "description": "Judge model API key. Not needed if you set RETRIEVAL_JUDGE_BACKEND=ollama to run the judge locally."
    },
    {
      "name": "RETRIEVAL_JUDGE_BACKEND",
      "default": "anthropic",
      "description": "Which judge to score with: anthropic (default), openai, or ollama for a fully local judge."
    },
    {
      "name": "RETRIEVAL_JUDGE_MODEL",
      "description": "Judge model id, e.g. claude-sonnet-4-6 or qwen2.5:14b for Ollama."
    },
    {
      "name": "RETRIEVAL_BUDGET_USD",
      "description": "Spend cap for judge calls. Unset means no cap."
    }
  ]
}

Tools

Reported by the attributed source; not independently observed by Koskamo.

No tool definitions available.

Connect from your client

Each snippet configures the same source-provided package; replace placeholders before use.

This server is operated by the named third party. Listing and endpoint checks are not an endorsement or a security audit. Review the operator's permissions, terms, and data handling before connecting. How listings are checked and corrected.

Cursor

Merge this local stdio configuration, replace every placeholder environment value, and restart Cursor. Koskamo has not executed this package.

Cursor configuration

{
  "mcpServers": {
    "retrieval-mcp": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "-e",
        "ANTHROPIC_API_KEY=<YOUR_ANTHROPIC_API_KEY>",
        "-e",
        "RETRIEVAL_JUDGE_BACKEND=<YOUR_RETRIEVAL_JUDGE_BACKEND>",
        "-e",
        "RETRIEVAL_JUDGE_MODEL=<YOUR_RETRIEVAL_JUDGE_MODEL>",
        "-e",
        "RETRIEVAL_BUDGET_USD=<YOUR_RETRIEVAL_BUDGET_USD>",
        "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
      ],
      "env": {
        "ANTHROPIC_API_KEY": "<YOUR_ANTHROPIC_API_KEY>",
        "RETRIEVAL_JUDGE_BACKEND": "<YOUR_RETRIEVAL_JUDGE_BACKEND>",
        "RETRIEVAL_JUDGE_MODEL": "<YOUR_RETRIEVAL_JUDGE_MODEL>",
        "RETRIEVAL_BUDGET_USD": "<YOUR_RETRIEVAL_BUDGET_USD>"
      }
    }
  }
}

Cline

Merge this local stdio configuration, replace every placeholder environment value, and restart Cline. Koskamo has not executed this package.

Cline configuration

{
  "mcpServers": {
    "retrieval-mcp": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "-e",
        "ANTHROPIC_API_KEY=<YOUR_ANTHROPIC_API_KEY>",
        "-e",
        "RETRIEVAL_JUDGE_BACKEND=<YOUR_RETRIEVAL_JUDGE_BACKEND>",
        "-e",
        "RETRIEVAL_JUDGE_MODEL=<YOUR_RETRIEVAL_JUDGE_MODEL>",
        "-e",
        "RETRIEVAL_BUDGET_USD=<YOUR_RETRIEVAL_BUDGET_USD>",
        "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
      ],
      "env": {
        "ANTHROPIC_API_KEY": "<YOUR_ANTHROPIC_API_KEY>",
        "RETRIEVAL_JUDGE_BACKEND": "<YOUR_RETRIEVAL_JUDGE_BACKEND>",
        "RETRIEVAL_JUDGE_MODEL": "<YOUR_RETRIEVAL_JUDGE_MODEL>",
        "RETRIEVAL_BUDGET_USD": "<YOUR_RETRIEVAL_BUDGET_USD>"
      }
    }
  }
}

VS Code

Merge this local stdio configuration, replace every placeholder environment value, and restart VS Code. Koskamo has not executed this package.

VS Code configuration

{
  "servers": {
    "retrieval-mcp": {
      "type": "stdio",
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "-e",
        "ANTHROPIC_API_KEY=<YOUR_ANTHROPIC_API_KEY>",
        "-e",
        "RETRIEVAL_JUDGE_BACKEND=<YOUR_RETRIEVAL_JUDGE_BACKEND>",
        "-e",
        "RETRIEVAL_JUDGE_MODEL=<YOUR_RETRIEVAL_JUDGE_MODEL>",
        "-e",
        "RETRIEVAL_BUDGET_USD=<YOUR_RETRIEVAL_BUDGET_USD>",
        "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
      ],
      "env": {
        "ANTHROPIC_API_KEY": "<YOUR_ANTHROPIC_API_KEY>",
        "RETRIEVAL_JUDGE_BACKEND": "<YOUR_RETRIEVAL_JUDGE_BACKEND>",
        "RETRIEVAL_JUDGE_MODEL": "<YOUR_RETRIEVAL_JUDGE_MODEL>",
        "RETRIEVAL_BUDGET_USD": "<YOUR_RETRIEVAL_BUDGET_USD>"
      }
    }
  }
}

Claude Desktop

Merge this local stdio configuration, replace every placeholder environment value, and restart Claude Desktop. Koskamo has not executed this package.

Claude Desktop configuration

{
  "mcpServers": {
    "retrieval-mcp": {
      "command": "docker",
      "args": [
        "run",
        "-i",
        "--rm",
        "-e",
        "ANTHROPIC_API_KEY=<YOUR_ANTHROPIC_API_KEY>",
        "-e",
        "RETRIEVAL_JUDGE_BACKEND=<YOUR_RETRIEVAL_JUDGE_BACKEND>",
        "-e",
        "RETRIEVAL_JUDGE_MODEL=<YOUR_RETRIEVAL_JUDGE_MODEL>",
        "-e",
        "RETRIEVAL_BUDGET_USD=<YOUR_RETRIEVAL_BUDGET_USD>",
        "ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
      ],
      "env": {
        "ANTHROPIC_API_KEY": "<YOUR_ANTHROPIC_API_KEY>",
        "RETRIEVAL_JUDGE_BACKEND": "<YOUR_RETRIEVAL_JUDGE_BACKEND>",
        "RETRIEVAL_JUDGE_MODEL": "<YOUR_RETRIEVAL_JUDGE_MODEL>",
        "RETRIEVAL_BUDGET_USD": "<YOUR_RETRIEVAL_BUDGET_USD>"
      }
    }
  }
}

Let your agent find its tools

Find servers like retriEVAL MCP from your agent. Add Koskamo once to search the index for endpoints, access requirements, and client setup. Public discovery needs no key.

Connect Koskamo Explore workflows

See all 4,424