Field note / tutorial

How to set up an MCP server for web scraping

Connect an MCP-compatible client to the Purify MCP binary using the same configuration contract shown in the dashboard.

tutorialMCPgetting-started

Publication record: capabilities, examples, and pricing reflect the date above. See the current product and pricing pages for the live offering.

Model Context Protocol gives compatible clients a standard way to call external tools. With Purify's MCP server, an agent can request scrape, batch, crawl, map, and structured-extract operations without each client implementing the REST calls itself.

This guide uses the current binary-based configuration. Older examples that referenced an npm package named purify-mcp are obsolete; that package name is not the supported installation path.

Prerequisites

You need:

  • an MCP-compatible client such as Claude Desktop or Cursor;
  • a local purify-mcp executable built or obtained from the Purify project;
  • the Purify API URL you want the connector to call;
  • an API key when the target API requires authentication.

The signed-in Purify dashboard generates a configuration with your current hosted API URL and key. For self-hosting, point the same variable at the base URL of your own instance.

Obtain the MCP binary

Use the current project source or release instructions. When building from this repository, the backend Makefile provides the MCP build target and writes the executable into the backend binary directory.

After obtaining the binary, note its absolute path. The examples below use /path/to/purify-mcp as a placeholder; replace it with the real executable path on your machine.

Claude Desktop configuration

Open the Claude Desktop MCP configuration file and add:

{
  "mcpServers": {
    "purify": {
      "command": "/path/to/purify-mcp",
      "env": {
        "PURIFY_API_URL": "https://purify.verifly.pro",
        "PURIFY_API_KEY": "YOUR_API_KEY"
      }
    }
  }
}

On macOS, Claude Desktop commonly reads its configuration from ~/Library/Application Support/Claude/claude_desktop_config.json. Check the current client documentation if that location has changed.

Cursor configuration

For Cursor, use the same server definition in .cursor/mcp.json:

{
  "mcpServers": {
    "purify": {
      "command": "/path/to/purify-mcp",
      "env": {
        "PURIFY_API_URL": "https://purify.verifly.pro",
        "PURIFY_API_KEY": "YOUR_API_KEY"
      }
    }
  }
}

The important environment variable is PURIFY_API_URL; copy its name exactly from the current dashboard configuration.

Self-hosted API

Keep the same MCP executable and point PURIFY_API_URL at your own Purify base URL:

{
  "mcpServers": {
    "purify-local": {
      "command": "/path/to/purify-mcp",
      "env": {
        "PURIFY_API_URL": "http://localhost:8080",
        "PURIFY_API_KEY": "YOUR_LOCAL_KEY_IF_CONFIGURED"
      }
    }
  }
}

Do not include PURIFY_API_KEY if your local instance is intentionally configured without API-key authentication.

Restart and verify

Restart the MCP client after saving valid JSON. Confirm that the Purify server appears in the client's tool list, then try a public URL you are authorized to access.

The current Purify MCP binary exposes these tools:

  • scrape_url — clean one page;
  • batch_scrape — process a set of URLs;
  • crawl_site — crawl within a site scope;
  • map_site — discover URLs;
  • extract_data — apply a requested schema using BYOK extraction.

Tool availability should still be checked against the binary version you deploy.

What happens under the hood

The connector converts the MCP tool call into the corresponding Purify API request. For scrape operations, the API's current response contract uses content and nested diagnostic objects:

  • tokens.original_estimate;
  • tokens.cleaned_estimate;
  • tokens.savings_percent;
  • timing.total_ms;
  • timing.navigation_ms;
  • timing.cleaning_ms.

Use the active nested contract above when updating older client examples.

Troubleshooting

The process does not start. Verify that command is an absolute path to an executable binary and that your operating system can run it.

The server appears but requests fail. Confirm PURIFY_API_URL, API-key validity, and network access from the client process. The value is the API base URL, not an individual endpoint path.

The JSON configuration is rejected. Remove comments and trailing commas, then validate the file as JSON.

A page cannot be read. Login walls, CAPTCHAs, access policy, unsupported content, and highly dynamic applications can all prevent useful extraction. The agent should report the failure rather than infer page contents.

Responsible use

Only access sources you are authorized to process. Respect applicable law, site terms, access controls, and the Purify Acceptable Use Policy.

Next steps

Put clean web data into your next system.

Start on the hosted service or inspect the open-source core first.

Try Purify →