Smart Crawler MCP MCP server

Smart Crawler is an agent-native web data collection engine that provides structured, real-time data via MCP. It features adaptive crawling, anti-blocking resilience, and supports scraping from Shopify, Vue SPAs, Nuxt SSR, and marketplaces like Wayfair. The server exposes tools for competitor analysis, VOC sentiment, and product data retrieval.

Endpoint and access

API key: create a key with the vendor first; your client sends it as an authorization header on every request.

API keyEndpoint checked 2026-10-05Operated by Smart Crawler

MCP endpoint (Streamable HTTP)

https://smartcrawler.io/mcp/

Vendor documentation

What you can do with it

Search for competitor products across multiple marketplaces
Retrieve detailed product information and pricing
Fetch Amazon reviews and Voice of Customer reports
List active promotions and track inventory changes

Connect from your client

Each snippet registers the same endpoint; pick the client you use.

This server is operated by the named third party. Listing and endpoint checks are not an endorsement or a security audit. Review the operator's permissions, terms, and data handling before connecting. How listings are checked and corrected.

Claude Code

Run this once in a terminal. It registers a Streamable HTTP server. Use --scope project to write .mcp.json for the repo.

Claude Code configuration

claude mcp add --transport http smart-crawler-mcp https://smartcrawler.io/mcp/

Codex

Run this once. Codex stores MCP servers in ~/.codex/config.toml (or .codex/config.toml in a trusted project).

Codex configuration

codex mcp add smart-crawler-mcp --url https://smartcrawler.io/mcp/

Cursor

Merge this server into .cursor/mcp.json in your project, then enable smart-crawler-mcp in your MCP settings.

Cursor configuration

{
  "mcpServers": {
    "smart-crawler-mcp": {
      "url": "https://smartcrawler.io/mcp/"
    }
  }
}

Cline

Merge this into cline_mcp_settings.json (Cline → MCP Servers → Configure). The type must be streamableHttp for a remote HTTP server.

Cline configuration

{
  "mcpServers": {
    "smart-crawler-mcp": {
      "type": "streamableHttp",
      "url": "https://smartcrawler.io/mcp/",
      "disabled": false
    }
  }
}

VS Code

Merge this server into .vscode/mcp.json. Use MCP: List Servers to start smart-crawler-mcp and inspect its tools.

VS Code configuration

{
  "servers": {
    "smart-crawler-mcp": {
      "type": "http",
      "url": "https://smartcrawler.io/mcp/"
    }
  }
}

Gemini CLI

Run this once. It writes .gemini/settings.json in the current project; pass --scope user for ~/.gemini/settings.json.

Gemini CLI configuration

gemini mcp add --transport http smart-crawler-mcp https://smartcrawler.io/mcp/

Claude Desktop

Merge this into claude_desktop_config.json (macOS: ~/Library/Application Support/Claude/, Windows: %APPDATA%\Claude\), then restart Claude Desktop.

Claude Desktop configuration

{
  "mcpServers": {
    "smart-crawler-mcp": {
      "url": "https://smartcrawler.io/mcp/"
    }
  }
}

Windsurf

Merge this into ~/.codeium/windsurf/mcp_config.json, then refresh MCP in Cascade. Windsurf remote servers use serverUrl, not url.

Windsurf configuration

{
  "mcpServers": {
    "smart-crawler-mcp": {
      "serverUrl": "https://smartcrawler.io/mcp/"
    }
  }
}

See all 1,583

Do you run Smart Crawler MCP?

Prove ownership with a DNS record, a well-known file, or your Official MCP Registry entry, and your edits publish immediately. Proofs are re-checked weekly; admins can still hide or remove a listing.

Claim this listing