Koskamo

Nonobench MCP

An open-source MCP server providing access to the Nonobench leaderboard, which benchmarks how well large language models solve nonogram (picross) puzzles. It allows agents to retrieve model rankings, compare specific models, fetch puzzle clues and solutions, and check grid answers against benchmarks.

Visit Maurice Kleine

Endpoint and access

No sign-in to connect and list tools. Individual tools may still need a vendor account, credentials, or permissions; check the documented prerequisites.

Nonobench MCP logo, by Maurice KleineCommunity listingStreamable HTTPNo sign-inEndpoint checked 2026-11-06Published by Maurice Kleine
MCP endpoint (Streamable HTTP)
https://www.nonobench.com/mcp
Vendor documentation

What its tools do

11 tools, grouped by the hints the server advertises. Hints are self-declared, not verified behavior.

Read-only (11)

11 of 11 have a description · 11 of 11 declare behavior hints · 9 of 9 with parameters document them all

Checks and signals

Each line is separate evidence, not a trust score.

  • Passed Endpoint answered the live check ·
  • Availability 1 of 1 checks passed in 30 days

What you can do with it

  • Retrieve the current leaderboard of LLMs solving nonogram puzzles
  • Compare performance metrics between specific model variants
  • Fetch puzzle clues and verify grid solutions programmatically
  • List available puzzle sizes and model families in the benchmark

Source and maintenance

Community listingRegistry 1.1.0
mcp-registry: io.github.mauricekleine/nonobench@1.1.0Source repositoryProject website
Maintenance checked

Protocol and capabilities

Server
nonobench 1.1.0
MCP protocol negotiated in check
2025-11-25
Tools
11 advertised
Resources
Not advertised
Resource templates
Not advertised
Prompts
Not advertised
Server-advertised capabilities
{
  "tools": {
    "listChanged": false
  }
}

Server-provided instructions

Untrusted server output shown for inspection. It is not Koskamo guidance and is excluded from knowledge search.

Nonobench measures how well LLMs solve nonogram (picross) puzzles: 40 puzzles across 5x5, 10x10, 15x15, 20x20 grids. Standard is 5x5, 10x10 and 15x15, and overall accuracy covers those 30 puzzles. Hard mode is ten 20x20 puzzles, reported only per size (size "20x20"). Accuracy is the share of puzzles where the model's grid satisfies every row and column clue. Results last updated 2026-11-04T06:08:16.701Z.

Tools

11 advertised

get_leaderboard — Get leaderboard
Advertised by the server during the last successful anonymous check.
Advertised read-only

Models ranked by accuracy; defaults to all effort levels for compatibility.

get_leaderboard input schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "properties": {
    "size": {
      "enum": [
        "5x5",
        "10x10",
        "15x15",
        "20x20"
      ],
      "type": "string",
      "description": "Grid size to filter on"
    },
    "effort": {
      "type": "string",
      "description": "best, all (default), or one effort level; empty means all"
    },
    "family": {
      "type": "string",
      "description": "Comma-separated family ids; empty means no filter"
    },
    "version": {
      "type": "string",
      "description": "Comma-separated benchmark versions: 1.0, 1.1, 1.2; empty means all"
    },
    "provider": {
      "type": "string",
      "description": "Comma-separated provider ids; empty means no filter"
    },
    "reasoning": {
      "type": "boolean",
      "description": "Only reasoning (true) or non-reasoning (false) variants"
    },
    "min_correct": {
      "type": "integer",
      "maximum": 9007199254740991,
      "minimum": 0,
      "description": "Minimum puzzles solved in the selected tier; default 0 includes unsolved variants"
    },
    "open_weights": {
      "type": "boolean",
      "description": "Only open-weight (true) or closed (false) models"
    }
  }
}
get_leaderboard output schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "updatedAt",
    "size",
    "models"
  ],
  "properties": {
    "size": {
      "type": "string"
    },
    "models": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [
          "model",
          "displayName",
          "family",
          "effort",
          "provider",
          "reasoning",
          "accuracy",
          "rank",
          "correct",
          "total",
          "totalCostUsd"
        ],
        "properties": {
          "rank": {
            "type": "number"
          },
          "model": {
            "type": "string",
            "description": "Model variant id, e.g. claude-opus-5.5-high"
          },
          "total": {
            "type": "number"
          },
          "effort": {
            "type": [
              "string",
              "null"
            ]
          },
          "family": {
            "type": "string"
          },
          "correct": {
            "type": "number"
          },
          "accuracy": {
            "type": "number",
            "description": "Percentage of puzzles solved"
          },
          "provider": {
            "type": [
              "string",
              "null"
            ]
          },
          "reasoning": {
            "type": "boolean"
          },
          "displayName": {
            "type": "string"
          },
          "totalCostUsd": {
            "type": "number"
          }
        },
        "additionalProperties": {}
      }
    },
    "updatedAt": {
      "type": "string"
    }
  },
  "additionalProperties": false
}
get_leaderboard annotations
{
  "title": "Get leaderboard",
  "readOnlyHint": true,
  "openWorldHint": false
}
list_providers — List providers
Advertised by the server during the last successful anonymous check.
Advertised read-only

Provider ids, names, families and variant counts.

list_providers input schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "properties": {}
}
list_providers output schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "providers"
  ],
  "properties": {
    "providers": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [
          "id",
          "name",
          "variantCount",
          "families"
        ],
        "properties": {
          "id": {
            "type": "string"
          },
          "name": {
            "type": "string"
          },
          "families": {
            "type": "array",
            "items": {
              "type": "string"
            }
          },
          "variantCount": {
            "type": "number"
          }
        },
        "additionalProperties": {}
      }
    }
  },
  "additionalProperties": false
}
list_providers annotations
{
  "title": "List providers",
  "readOnlyHint": true,
  "openWorldHint": false
}
list_families — List families
Advertised by the server during the last successful anonymous check.
Advertised read-only

Model families, available efforts and best variants.

list_families input schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "properties": {}
}
list_families output schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "families"
  ],
  "properties": {
    "families": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [
          "family",
          "displayName",
          "provider",
          "bestVariant",
          "efforts"
        ],
        "properties": {
          "family": {
            "type": "string"
          },
          "efforts": {
            "type": "array",
            "items": {
              "type": "string"
            }
          },
          "provider": {
            "type": [
              "string",
              "null"
            ]
          },
          "bestVariant": {
            "type": "string"
          },
          "displayName": {
            "type": "string"
          }
        },
        "additionalProperties": {}
      }
    }
  },
  "additionalProperties": false
}
list_families annotations
{
  "title": "List families",
  "readOnlyHint": true,
  "openWorldHint": false
}
compare_models — Compare models
Advertised by the server during the last successful anonymous check.
Advertised read-only

Side-by-side core overall and per-size accuracy, cost, latency and token results for model or family names.

compare_models input schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "models"
  ],
  "properties": {
    "models": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "maxItems": 20,
      "minItems": 2,
      "description": "2 to 20 model variant ids or family names, e.g. claude-opus-5.5 or gpt-6-astra-xhigh"
    }
  }
}
compare_models output schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "models"
  ],
  "properties": {
    "models": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [
          "model",
          "displayName",
          "family",
          "effort",
          "provider",
          "reasoning",
          "accuracy",
          "correct",
          "total",
          "failedRuns",
          "bySize"
        ],
        "properties": {
          "model": {
            "type": "string",
            "description": "Model variant id, e.g. claude-opus-5.5-high"
          },
          "total": {
            "type": "number"
          },
          "bySize": {
            "type": "array",
            "items": {
              "type": "object",
              "required": [
                "size"
              ],
              "properties": {
                "size": {
                  "type": "string"
                }
              },
              "additionalProperties": {}
            }
          },
          "effort": {
            "type": [
              "string",
              "null"
            ]
          },
          "family": {
            "type": "string"
          },
          "correct": {
            "type": "number"
          },
          "accuracy": {
            "type": "number",
            "description": "Percentage of puzzles solved"
          },
          "provider": {
            "type": [
              "string",
              "null"
            ]
          },
          "reasoning": {
            "type": "boolean"
          },
          "failedRuns": {
            "type": "number"
          },
          "displayName": {
            "type": "string"
          }
        },
        "additionalProperties": {}
      }
    }
  },
  "additionalProperties": false
}
compare_models annotations
{
  "title": "Compare models",
  "readOnlyHint": true,
  "openWorldHint": false
}
get_model_results — Get model results
Advertised by the server during the last successful anonymous check.
Advertised read-only

Accuracy, cost, latency and token use for one model, broken down by grid size.

get_model_results input schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "model"
  ],
  "properties": {
    "model": {
      "type": "string",
      "description": "Model name as listed on the leaderboard, e.g. gpt-5.4-xhigh"
    }
  }
}
get_model_results output schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "model",
    "displayName",
    "family",
    "effort",
    "provider",
    "reasoning",
    "accuracy",
    "correct",
    "total",
    "failedRuns",
    "bySize"
  ],
  "properties": {
    "model": {
      "type": "string",
      "description": "Model variant id, e.g. claude-opus-5.5-high"
    },
    "total": {
      "type": "number"
    },
    "bySize": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [
          "size"
        ],
        "properties": {
          "size": {
            "type": "string"
          }
        },
        "additionalProperties": {}
      }
    },
    "effort": {
      "type": [
        "string",
        "null"
      ]
    },
    "family": {
      "type": "string"
    },
    "correct": {
      "type": "number"
    },
    "accuracy": {
      "type": "number",
      "description": "Percentage of puzzles solved"
    },
    "provider": {
      "type": [
        "string",
        "null"
      ]
    },
    "reasoning": {
      "type": "boolean"
    },
    "failedRuns": {
      "type": "number"
    },
    "displayName": {
      "type": "string"
    }
  },
  "additionalProperties": {}
}
get_model_results annotations
{
  "title": "Get model results",
  "readOnlyHint": true,
  "openWorldHint": false
}
list_puzzles — List puzzles
Advertised by the server during the last successful anonymous check.
Advertised read-only

The benchmark puzzles with their ids and row/column clues.

list_puzzles input schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "properties": {
    "size": {
      "enum": [
        "5x5",
        "10x10",
        "15x15",
        "20x20"
      ],
      "type": "string",
      "description": "Grid size to filter on"
    }
  }
}
list_puzzles output schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "puzzles"
  ],
  "properties": {
    "puzzles": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [
          "id",
          "index",
          "size",
          "width",
          "height",
          "rowClues",
          "columnClues",
          "url"
        ],
        "properties": {
          "id": {
            "type": "string"
          },
          "url": {
            "type": "string"
          },
          "size": {
            "type": "string"
          },
          "index": {
            "type": "number"
          },
          "width": {
            "type": "number"
          },
          "height": {
            "type": "number"
          },
          "rowClues": {
            "type": "array",
            "items": {
              "type": "array",
              "items": {
                "type": "number"
              }
            }
          },
          "columnClues": {
            "type": "array",
            "items": {
              "type": "array",
              "items": {
                "type": "number"
              }
            }
          }
        },
        "additionalProperties": {}
      }
    }
  },
  "additionalProperties": false
}
list_puzzles annotations
{
  "title": "List puzzles",
  "readOnlyHint": true,
  "openWorldHint": false
}
get_puzzle — Get puzzle
Advertised by the server during the last successful anonymous check.
Advertised read-only

One puzzle, including the clue text models were prompted with. The reference solution is only included on request; some puzzles have several valid solutions.

get_puzzle input schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "id"
  ],
  "properties": {
    "id": {
      "type": "string",
      "description": "Puzzle id from list_puzzles"
    },
    "include_solution": {
      "type": "boolean",
      "description": "Include a reference solution"
    }
  }
}
get_puzzle output schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "id",
    "index",
    "size",
    "width",
    "height",
    "rowClues",
    "columnClues",
    "url",
    "prompt"
  ],
  "properties": {
    "id": {
      "type": "string"
    },
    "url": {
      "type": "string"
    },
    "size": {
      "type": "string"
    },
    "index": {
      "type": "number"
    },
    "width": {
      "type": "number"
    },
    "height": {
      "type": "number"
    },
    "prompt": {
      "type": "string",
      "description": "Clue text as models received it"
    },
    "rowClues": {
      "type": "array",
      "items": {
        "type": "array",
        "items": {
          "type": "number"
        }
      }
    },
    "columnClues": {
      "type": "array",
      "items": {
        "type": "array",
        "items": {
          "type": "number"
        }
      }
    },
    "referenceSolution": {
      "type": "string"
    }
  },
  "additionalProperties": {}
}
get_puzzle annotations
{
  "title": "Get puzzle",
  "readOnlyHint": true,
  "openWorldHint": false
}
check_solution — Check solution
Advertised by the server during the last successful anonymous check.
Advertised read-only

Check a grid against a puzzle's clues, using the same rule as the benchmark grader. Reports which rows and columns do not match.

check_solution input schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "id",
    "grid"
  ],
  "properties": {
    "id": {
      "type": "string",
      "description": "Puzzle id from list_puzzles"
    },
    "grid": {
      "type": "string",
      "description": "Row-major string of 0 (empty) and 1 (filled), width × height characters"
    }
  }
}
check_solution output schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "correct",
    "rowViolations",
    "columnViolations"
  ],
  "properties": {
    "error": {
      "type": "string"
    },
    "correct": {
      "type": "boolean"
    },
    "rowViolations": {
      "type": "array",
      "items": {}
    },
    "columnViolations": {
      "type": "array",
      "items": {}
    }
  },
  "additionalProperties": {}
}
check_solution annotations
{
  "title": "Check solution",
  "readOnlyHint": true,
  "openWorldHint": false
}
get_puzzle_results — Get puzzle results
Advertised by the server during the last successful anonymous check.
Advertised read-only

Per-model outcomes for one puzzle. Answers are omitted unless requested.

get_puzzle_results input schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "id"
  ],
  "properties": {
    "id": {
      "type": "string",
      "description": "Puzzle id from list_puzzles"
    },
    "effort": {
      "type": "string",
      "description": "best, all (default), or one effort level"
    },
    "family": {
      "type": "string",
      "description": "Comma-separated family ids; empty means no filter"
    },
    "provider": {
      "type": "string",
      "description": "Comma-separated provider ids; empty means no filter"
    },
    "reasoning": {
      "type": "boolean",
      "description": "Only reasoning (true) or non-reasoning (false) variants"
    },
    "open_weights": {
      "type": "boolean",
      "description": "Only open-weight (true) or closed (false) models"
    },
    "include_answers": {
      "type": "boolean",
      "description": "Include each model's answer grid"
    }
  }
}
get_puzzle_results output schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "updatedAt",
    "puzzleId",
    "index",
    "size",
    "attempts",
    "solved",
    "runs"
  ],
  "properties": {
    "runs": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [
          "model",
          "correct",
          "status",
          "displayName"
        ],
        "properties": {
          "model": {
            "type": "string"
          },
          "answer": {
            "type": [
              "string",
              "null"
            ]
          },
          "status": {
            "type": "string"
          },
          "correct": {
            "type": "boolean"
          },
          "displayName": {
            "type": "string"
          }
        },
        "additionalProperties": {}
      }
    },
    "size": {
      "type": "string"
    },
    "index": {
      "type": "number"
    },
    "solved": {
      "type": "number"
    },
    "attempts": {
      "type": "number"
    },
    "puzzleId": {
      "type": "string"
    },
    "updatedAt": {
      "type": "string"
    }
  },
  "additionalProperties": false
}
get_puzzle_results annotations
{
  "title": "Get puzzle results",
  "readOnlyHint": true,
  "openWorldHint": false
}
get_model_puzzles — Get model puzzles
Advertised by the server during the last successful anonymous check.
Advertised read-only

Which puzzles one model solved, missed, timed out on, or has not run.

get_model_puzzles input schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "model"
  ],
  "properties": {
    "model": {
      "type": "string",
      "description": "Model variant id as listed on the leaderboard, e.g. claude-opus-5.5-high"
    }
  }
}
get_model_puzzles output schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "model",
    "displayName",
    "solved",
    "attempted",
    "puzzles"
  ],
  "properties": {
    "model": {
      "type": "string"
    },
    "solved": {
      "type": "number"
    },
    "puzzles": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [
          "id",
          "index",
          "size",
          "state"
        ],
        "properties": {
          "id": {
            "type": "string"
          },
          "size": {
            "type": "string"
          },
          "index": {
            "type": "number"
          },
          "state": {
            "enum": [
              "solved",
              "wrong",
              "cut-off",
              "not-run"
            ],
            "type": "string"
          }
        },
        "additionalProperties": {}
      }
    },
    "attempted": {
      "type": "number"
    },
    "displayName": {
      "type": "string"
    }
  },
  "additionalProperties": false
}
get_model_puzzles annotations
{
  "title": "Get model puzzles",
  "readOnlyHint": true,
  "openWorldHint": false
}
list_runs — List runs
Advertised by the server during the last successful anonymous check.
Advertised read-only

Individual benchmark runs (one model on one puzzle), optionally with the raw prompt and model output.

list_runs input schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "properties": {
    "size": {
      "enum": [
        "5x5",
        "10x10",
        "15x15",
        "20x20"
      ],
      "type": "string",
      "description": "Grid size to filter on"
    },
    "limit": {
      "type": "integer",
      "maximum": 500,
      "minimum": 1,
      "description": "Default 100"
    },
    "model": {
      "type": "string",
      "description": "Only runs of this model variant id"
    },
    "offset": {
      "type": "integer",
      "maximum": 9007199254740991,
      "minimum": 0,
      "description": "Runs to skip, for paging; default 0"
    },
    "puzzle_id": {
      "type": "string",
      "description": "Only runs on this puzzle id from list_puzzles"
    },
    "include_output": {
      "type": "boolean",
      "description": "Include raw prompt and model output (large)"
    }
  }
}
list_runs output schema
{
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "required": [
    "total",
    "limit",
    "offset",
    "runs"
  ],
  "properties": {
    "runs": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [
          "model",
          "puzzleId",
          "size",
          "correct",
          "status"
        ],
        "properties": {
          "size": {
            "type": "string"
          },
          "model": {
            "type": "string"
          },
          "status": {
            "type": "string"
          },
          "correct": {
            "type": "boolean"
          },
          "puzzleId": {
            "type": "string"
          }
        },
        "additionalProperties": {}
      }
    },
    "limit": {
      "type": "number"
    },
    "total": {
      "type": "number"
    },
    "offset": {
      "type": "number"
    }
  },
  "additionalProperties": false
}
list_runs annotations
{
  "title": "List runs",
  "readOnlyHint": true,
  "openWorldHint": false
}

Prompts

Not advertised

No prompt definitions available.

Resources

Not advertised

No resource definitions available.

90-day uptime

A lightweight connection check about every six hours, sampled on hosts with many listings. It shows the server answered, not that its tools work.

Too few checks for a percentage yet (1 of 1 answered).

  1. 2026-11-07: 1 of 1 checks answered

30-day availability

Aggregated from read-only remote checks; this is not a tool-execution or security guarantee.

Successful checks
1 of 1 (100%)
Average latency
282 ms
95th percentile latency
282 ms

Maintenance history

Public check and source-refresh events. Editorial and ownership records remain private.

  • · probe · passed · 282 ms · metadata changed · 11 tool(s)

Connect from your client

Each snippet registers the same endpoint; pick the client you use.

This server is operated by the named third party. Listing and endpoint checks are not an endorsement or a security audit. Review the operator's permissions, terms, and data handling before connecting. How listings are checked and corrected.

Claude Code

Run this once in a terminal. It registers the remote server. Use --scope project to write .mcp.json for the repo.
Claude Code configuration
claude mcp add --transport http nonobench-mcp https://www.nonobench.com/mcp

Codex

Run this once. Codex stores MCP servers in ~/.codex/config.toml (or .codex/config.toml in a trusted project).
Codex configuration
codex mcp add nonobench-mcp --url https://www.nonobench.com/mcp

Cursor

Merge this server into .cursor/mcp.json in your project, then enable nonobench-mcp in your MCP settings.
Cursor configuration
{
  "mcpServers": {
    "nonobench-mcp": {
      "url": "https://www.nonobench.com/mcp"
    }
  }
}

Cline

Merge this into cline_mcp_settings.json (Cline → MCP Servers → Configure).
Cline configuration
{
  "mcpServers": {
    "nonobench-mcp": {
      "type": "streamableHttp",
      "url": "https://www.nonobench.com/mcp",
      "disabled": false
    }
  }
}

VS Code

Merge this server into .vscode/mcp.json. Use MCP: List Servers to start nonobench-mcp and inspect its tools.
VS Code configuration
{
  "servers": {
    "nonobench-mcp": {
      "type": "http",
      "url": "https://www.nonobench.com/mcp"
    }
  }
}

Gemini CLI

Run this once. It writes .gemini/settings.json in the current project; pass --scope user for ~/.gemini/settings.json.
Gemini CLI configuration
gemini mcp add --transport http nonobench-mcp https://www.nonobench.com/mcp

Claude Desktop

Open Claude Settings > Connectors > Add custom connector, enter this server URL and enable it for the conversation. This is a manual remote-connector handoff, not a desktop JSON configuration.
Claude Desktop configuration
https://www.nonobench.com/mcp

Windsurf

Merge this into ~/.codeium/windsurf/mcp_config.json, then refresh MCP in Cascade. Windsurf remote servers use serverUrl, not url.
Windsurf configuration
{
  "mcpServers": {
    "nonobench-mcp": {
      "serverUrl": "https://www.nonobench.com/mcp"
    }
  }
}

Let your agent find its tools

Find servers like Nonobench MCP from your agent. Add Koskamo once to search the index for endpoints, access requirements, and client setup. Public discovery needs no key.

Connect Koskamo Explore workflows
See all 5,158