Koskamo

A/B Test Sample Size Calculator MCP

An MCP server that provides tools for calculating A/B test sample sizes, estimating test duration, performing statistical significance tests (z-test, chi-square, Fisher exact), and checking peeking risks. It supports multiple variants, unequal splits, and connects via Streamable HTTP or local stdio.

Endpoint and access

No sign-in to connect and list tools. Individual tools may still need a vendor account, credentials, or permissions; check the documented prerequisites.

A/B Test Sample Size Calculator MCP logo, by theluckystrikeCommunity listingStreamable HTTPNo sign-inEndpoint checked 2026-11-02Published by theluckystrike
MCP endpoint (Streamable HTTP)
https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp
Vendor documentation

What you can do with it

  • Calculate sample size for two proportions with baseline and MDE
  • Estimate test duration based on daily traffic and variant share
  • Perform two-proportion z-test and chi-square significance tests
  • Check false positive rate from peeking with stopping boundaries

Source and maintenance

Community listingRegistry 1.0.0
mcp-registry: io.github.theluckystrike/ab-test-sample-size-calculator-mcp@1.0.0Source repositoryProject website
Maintenance checked

Protocol and capabilities

Server
ab-test-sample-size-calculator-mcp 1.0.0
MCP protocol
2025-06-18
Tools
5 advertised
Resources
Not advertised
Resource templates
Not advertised
Prompts
Not advertised
Server-advertised capabilities
{
  "tools": {
    "listChanged": true
  }
}

Server-provided instructions

Untrusted server output shown for inspection. It is not Koskamo guidance and is excluded from knowledge search.

Exact A/B test statistics: sample size for two proportions, test duration, two-proportion z-test and chi-square, Fisher exact and Fisher-Freeman-Halton r x c tests, and the false positive cost of peeking. Use these tools instead of mental arithmetic: inverse normal quantiles, exact hypergeometric sums and sequential boundaries are easy to get wrong by hand. Rates are proportions (4.2% = 0.042).

Tools

5 advertised

calculate_sample_size — A/B test sample size calculator
Advertised by the server during the last successful anonymous check.
Advertised read-only

Exact sample size per variant for an A/B test on a conversion rate (two independent proportions): baseline rate, minimum detectable effect (relative or absolute), alpha, power, one or two-sided, several variants with optional Bonferroni, and unequal allocation. Uses the pooled/unpooled normal formula (same as R power.prop.test) and also reports the figure Evan Miller's calculator gives, so the two can be reconciled.

calculate_sample_size input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "required": [
    "baseline_rate",
    "mde"
  ],
  "properties": {
    "mde": {
      "type": "number",
      "description": "Minimum detectable effect. Relative by default (0.10 = a 10% lift); set mde_type \"absolute\" for percentage points as a proportion (0.01 = 1 point).",
      "exclusiveMinimum": 0
    },
    "alpha": {
      "type": "number",
      "default": 0.05,
      "description": "Significance level, default 0.05.",
      "exclusiveMaximum": 0.5,
      "exclusiveMinimum": 0
    },
    "power": {
      "type": "number",
      "default": 0.8,
      "description": "Statistical power, default 0.80.",
      "exclusiveMaximum": 1,
      "exclusiveMinimum": 0.5
    },
    "sided": {
      "enum": [
        "two",
        "one"
      ],
      "type": "string",
      "default": "two",
      "description": "two-sided (default) or one-sided test."
    },
    "mde_type": {
      "enum": [
        "relative",
        "absolute"
      ],
      "type": "string",
      "default": "relative",
      "description": "relative (default) or absolute."
    },
    "variants": {
      "type": "integer",
      "default": 2,
      "maximum": 20,
      "minimum": 2,
      "description": "Number of arms including control. 2 = A/B, 3 = A/B/C."
    },
    "bonferroni": {
      "type": "boolean",
      "default": false,
      "description": "Split alpha across the variant-vs-control comparisons (Bonferroni)."
    },
    "baseline_rate": {
      "type": "number",
      "description": "Control conversion rate as a proportion, for example 0.042 for 4.2%.",
      "exclusiveMaximum": 1,
      "exclusiveMinimum": 0
    },
    "allocation_ratio": {
      "type": "number",
      "default": 1,
      "maximum": 10,
      "description": "Users in each variant per user in control. 1 = even split.",
      "exclusiveMinimum": 0
    }
  },
  "additionalProperties": false
}
calculate_sample_size annotations
{
  "title": "Sample size",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
estimate_test_duration — A/B test duration estimator
Advertised by the server during the last successful anonymous check.
Advertised read-only

How many days an A/B test must run: from a known sample size per variant, or from baseline rate and MDE (computed as in calculate_sample_size), plus daily visitors and the share of traffic in the test. Returns exact days and the recommended whole weeks (to cover weekday effects).

estimate_test_duration input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "required": [
    "daily_visitors"
  ],
  "properties": {
    "mde": {
      "type": "number",
      "description": "Minimum detectable effect (relative by default).",
      "exclusiveMinimum": 0
    },
    "alpha": {
      "type": "number",
      "default": 0.05,
      "description": "Significance level, default 0.05.",
      "exclusiveMaximum": 0.5,
      "exclusiveMinimum": 0
    },
    "power": {
      "type": "number",
      "default": 0.8,
      "exclusiveMaximum": 1,
      "exclusiveMinimum": 0.5
    },
    "mde_type": {
      "enum": [
        "relative",
        "absolute"
      ],
      "type": "string",
      "default": "relative"
    },
    "variants": {
      "type": "integer",
      "default": 2,
      "maximum": 20,
      "minimum": 2,
      "description": "Number of arms including control."
    },
    "baseline_rate": {
      "type": "number",
      "description": "Control conversion rate, for computing the sample size.",
      "exclusiveMaximum": 1,
      "exclusiveMinimum": 0
    },
    "traffic_share": {
      "type": "number",
      "default": 1,
      "maximum": 1,
      "description": "Fraction of those visitors entered into the test, default 1.",
      "exclusiveMinimum": 0
    },
    "daily_visitors": {
      "type": "number",
      "description": "Eligible visitors per day across the whole site or page.",
      "exclusiveMinimum": 0
    },
    "sample_size_per_variant": {
      "type": "integer",
      "description": "Known users per arm. If omitted, give baseline_rate and mde.",
      "exclusiveMinimum": 0
    }
  },
  "additionalProperties": false
}
estimate_test_duration annotations
{
  "title": "Test duration",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
test_ab_significance — A/B test significance calculator
Advertised by the server during the last successful anonymous check.
Advertised read-only

Is an A/B test result statistically significant? From visitors and conversions per group: two-proportion z-test (pooled), one or two-sided p-value, Wald confidence interval for the difference, relative lift, Pearson chi-square with and without Yates correction (R prop.test), Fisher exact p for the same 2x2 table, and a sample ratio mismatch (SRM) check against the planned split.

test_ab_significance input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "required": [
    "control_visitors",
    "control_conversions",
    "variant_visitors",
    "variant_conversions"
  ],
  "properties": {
    "alpha": {
      "type": "number",
      "default": 0.05,
      "description": "Significance level, default 0.05.",
      "exclusiveMaximum": 0.5,
      "exclusiveMinimum": 0
    },
    "alternative": {
      "enum": [
        "two-sided",
        "greater",
        "less"
      ],
      "type": "string",
      "default": "two-sided",
      "description": "greater = variant converts better than control."
    },
    "expected_split": {
      "type": "number",
      "default": 0.5,
      "description": "Planned share of traffic in control, for the SRM check. Default 0.5.",
      "exclusiveMaximum": 1,
      "exclusiveMinimum": 0
    },
    "control_visitors": {
      "type": "integer",
      "exclusiveMinimum": 0
    },
    "variant_visitors": {
      "type": "integer",
      "exclusiveMinimum": 0
    },
    "control_conversions": {
      "type": "integer",
      "minimum": 0
    },
    "variant_conversions": {
      "type": "integer",
      "minimum": 0
    }
  },
  "additionalProperties": false
}
test_ab_significance annotations
{
  "title": "Significance test",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
fisher_exact_test — Fisher exact and Fisher-Freeman-Halton test
Advertised by the server during the last successful anonymous check.
Advertised read-only

Exact p-value for a contingency table of counts. 2x2: Fisher exact test (two-sided, greater or less, as R fisher.test). Larger r x c tables (for example 3 variants x converted/not, or 4 x 4 survey tables): Fisher-Freeman-Halton exact test by a network algorithm. Also returns Pearson chi-square and how many cells have expected counts below 5, so you can see when the chi-square approximation is unsafe.

fisher_exact_test input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "required": [
    "table"
  ],
  "properties": {
    "table": {
      "type": "array",
      "items": {
        "type": "array",
        "items": {
          "type": "integer",
          "minimum": 0
        },
        "maxItems": 12,
        "minItems": 2
      },
      "maxItems": 12,
      "minItems": 2,
      "description": "Rows of counts, for example [[12, 488], [25, 475]] (rows = groups, columns = outcomes)."
    },
    "alternative": {
      "enum": [
        "two-sided",
        "greater",
        "less"
      ],
      "type": "string",
      "default": "two-sided",
      "description": "One-sided options apply to 2x2 only; greater = odds ratio above 1."
    }
  },
  "additionalProperties": false
}
fisher_exact_test annotations
{
  "title": "Exact test",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}
check_peeking_risk — A/B test peeking and sequential testing risk
Advertised by the server during the last successful anonymous check.
Advertised read-only

What peeking does to an A/B test: the real false positive rate if you check a fixed-horizon test K times and stop at the first p < alpha (for example about 14% at 5 looks with alpha 0.05), and the Pocock and O'Brien-Fleming group sequential boundaries (z and nominal p per look) that keep the overall rate at alpha, for up to 10 equally spaced looks.

check_peeking_risk input schema
{
  "type": "object",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "required": [
    "looks"
  ],
  "properties": {
    "alpha": {
      "type": "number",
      "default": 0.05,
      "description": "Significance level, default 0.05.",
      "exclusiveMaximum": 0.5,
      "exclusiveMinimum": 0
    },
    "looks": {
      "type": "integer",
      "maximum": 50,
      "minimum": 1,
      "description": "How many times the result is checked, including the final look. Equally spaced."
    }
  },
  "additionalProperties": false
}
check_peeking_risk annotations
{
  "title": "Peeking risk",
  "readOnlyHint": true,
  "openWorldHint": false,
  "idempotentHint": true,
  "destructiveHint": false
}

Prompts

Not advertised

No prompt definitions available.

Resources

Not advertised

No resource definitions available.

30-day availability

Aggregated from read-only remote checks; this is not a tool-execution or security guarantee.

Successful checks
1 of 1 (100%)
Average latency
527 ms
95th percentile latency
527 ms

Maintenance history

Public check and source-refresh events. Editorial and ownership records remain private.

  • · probe · passed · 527 ms · metadata changed · 5 tool(s)

Connect from your client

Each snippet registers the same endpoint; pick the client you use.

This server is operated by the named third party. Listing and endpoint checks are not an endorsement or a security audit. Review the operator's permissions, terms, and data handling before connecting. How listings are checked and corrected.

Claude Code

Run this once in a terminal. It registers the remote server. Use --scope project to write .mcp.json for the repo.
Claude Code configuration
claude mcp add --transport http a-b-test-sample-size-calculator-mcp https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp

Codex

Run this once. Codex stores MCP servers in ~/.codex/config.toml (or .codex/config.toml in a trusted project).
Codex configuration
codex mcp add a-b-test-sample-size-calculator-mcp --url https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp

Cursor

Merge this server into .cursor/mcp.json in your project, then enable a-b-test-sample-size-calculator-mcp in your MCP settings.
Cursor configuration
{
  "mcpServers": {
    "a-b-test-sample-size-calculator-mcp": {
      "url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp"
    }
  }
}

Cline

Merge this into cline_mcp_settings.json (Cline → MCP Servers → Configure).
Cline configuration
{
  "mcpServers": {
    "a-b-test-sample-size-calculator-mcp": {
      "type": "streamableHttp",
      "url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp",
      "disabled": false
    }
  }
}

VS Code

Merge this server into .vscode/mcp.json. Use MCP: List Servers to start a-b-test-sample-size-calculator-mcp and inspect its tools.
VS Code configuration
{
  "servers": {
    "a-b-test-sample-size-calculator-mcp": {
      "type": "http",
      "url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp"
    }
  }
}

Gemini CLI

Run this once. It writes .gemini/settings.json in the current project; pass --scope user for ~/.gemini/settings.json.
Gemini CLI configuration
gemini mcp add --transport http a-b-test-sample-size-calculator-mcp https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp

Claude Desktop

Merge this into claude_desktop_config.json (macOS: ~/Library/Application Support/Claude/, Windows: %APPDATA%\Claude\), then restart Claude Desktop.
Claude Desktop configuration
{
  "mcpServers": {
    "a-b-test-sample-size-calculator-mcp": {
      "url": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp"
    }
  }
}

Windsurf

Merge this into ~/.codeium/windsurf/mcp_config.json, then refresh MCP in Cascade. Windsurf remote servers use serverUrl, not url.
Windsurf configuration
{
  "mcpServers": {
    "a-b-test-sample-size-calculator-mcp": {
      "serverUrl": "https://ab-test-sample-size-calculator-mcp.lipmichal.workers.dev/mcp"
    }
  }
}

Let your agent find its tools

Find servers like A/B Test Sample Size Calculator MCP from your agent. Add Koskamo once to search the index for endpoints, access requirements, and client setup. Public discovery needs no key.

Connect Koskamo Explore workflows
See all 4,784