Octocrawl MCP

Connects AI agents to Octocrawl for web scraping, mapping site URLs, and running crawl or batch jobs. It supports hosted access via HTTP or local execution via npx, providing clean Markdown and structured JSON with Evidence Records for verification.

Visit Octocrawl

Package and access

Local package with source-attributed installation metadata. Koskamo does not execute packages during review.

Community listingLocal stdioSource reviewed; not executedPublished by Octocrawl

You need:An account key enables browser-backed scraping and the Amazon.sg scrape_product toolWithout a key, hosted access reads over HTTP within a daily allowance

Vendor documentation

What you can do with it

Scrape public web pages into clean Markdown and JSON
Map site URLs to discover all pages on a website
Run batch jobs to process up to 1,000 URLs
Monitor documentation pages for changes via webhooks

Source and maintenance

Community listingRegistry 0.7.0

mcp-registry: io.github.77777R7/octocrawl@0.7.0 Source repository Project website

Maintenance checked: Oct 31, 2026, 10:59 PM UTC

Packages and installation

Source-provided package data. Koskamo has not executed or live-probed this package.

@octocrawl/mcp 0.7.0

npm

@octocrawl/mcp package definition

{
  "version": "0.7.0",
  "transport": "stdio",
  "identifier": "@octocrawl/mcp",
  "registryType": "npm",
  "registryBaseUrl": "https://registry.npmjs.org",
  "environmentVariables": [
    {
      "name": "W2L_API_URL",
      "description": "The Octocrawl API the server calls: npx octocrawl serve on this computer (default http://127.0.0.1:8787), or https://api.octocrawl.dev"
    },
    {
      "name": "W2L_API_TOKEN",
      "description": "A bearer token for a hosted or self-hosted API; none for a local one"
    },
    {
      "name": "W2L_MCP_TOOLSETS",
      "description": "The tool groups to list, comma-separated: core, crawl, batch, products, logins, monitor, delivery, or all (default all); the same as --toolsets"
    },
    {
      "name": "W2L_MCP_READ_ONLY",
      "description": "1 lists read-only tools alone, with a scrape that does not act as the person; the same as --read-only"
    }
  ]
}

Tools

Reported by the attributed source; not independently observed by Koskamo.

No tool definitions available.

Connect from your client

Each snippet configures the same source-provided package; replace placeholders before use.

This server is operated by the named third party. Listing and endpoint checks are not an endorsement or a security audit. Review the operator's permissions, terms, and data handling before connecting. How listings are checked and corrected.

Cursor

Merge this local stdio configuration, replace every placeholder environment value, and restart Cursor. Koskamo has not executed this package.

Cursor configuration

{
  "mcpServers": {
    "octocrawl-mcp-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@octocrawl/mcp@0.7.0"
      ],
      "env": {
        "W2L_API_URL": "<YOUR_W2L_API_URL>",
        "W2L_API_TOKEN": "<YOUR_W2L_API_TOKEN>",
        "W2L_MCP_TOOLSETS": "<YOUR_W2L_MCP_TOOLSETS>",
        "W2L_MCP_READ_ONLY": "<YOUR_W2L_MCP_READ_ONLY>"
      }
    }
  }
}

Cline

Merge this local stdio configuration, replace every placeholder environment value, and restart Cline. Koskamo has not executed this package.

Cline configuration

{
  "mcpServers": {
    "octocrawl-mcp-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@octocrawl/mcp@0.7.0"
      ],
      "env": {
        "W2L_API_URL": "<YOUR_W2L_API_URL>",
        "W2L_API_TOKEN": "<YOUR_W2L_API_TOKEN>",
        "W2L_MCP_TOOLSETS": "<YOUR_W2L_MCP_TOOLSETS>",
        "W2L_MCP_READ_ONLY": "<YOUR_W2L_MCP_READ_ONLY>"
      }
    }
  }
}

VS Code

Merge this local stdio configuration, replace every placeholder environment value, and restart VS Code. Koskamo has not executed this package.

VS Code configuration

{
  "servers": {
    "octocrawl-mcp-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@octocrawl/mcp@0.7.0"
      ],
      "env": {
        "W2L_API_URL": "<YOUR_W2L_API_URL>",
        "W2L_API_TOKEN": "<YOUR_W2L_API_TOKEN>",
        "W2L_MCP_TOOLSETS": "<YOUR_W2L_MCP_TOOLSETS>",
        "W2L_MCP_READ_ONLY": "<YOUR_W2L_MCP_READ_ONLY>"
      }
    }
  }
}

Claude Desktop

Merge this local stdio configuration, replace every placeholder environment value, and restart Claude Desktop. Koskamo has not executed this package.

Claude Desktop configuration

{
  "mcpServers": {
    "octocrawl-mcp-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@octocrawl/mcp@0.7.0"
      ],
      "env": {
        "W2L_API_URL": "<YOUR_W2L_API_URL>",
        "W2L_API_TOKEN": "<YOUR_W2L_API_TOKEN>",
        "W2L_MCP_TOOLSETS": "<YOUR_W2L_MCP_TOOLSETS>",
        "W2L_MCP_READ_ONLY": "<YOUR_W2L_MCP_READ_ONLY>"
      }
    }
  }
}

Let your agent find its tools

Find servers like Octocrawl MCP from your agent. Add Koskamo once to search the index for endpoints, access requirements, and client setup. Public discovery needs no key.

Connect Koskamo Explore workflows

See all 3,189