Self-hosted MCP fetcher for agent web research and extraction
master-fetch, by Dondai1234, is a Model Context Protocol server that gives AI agents local access to live web content for model context and research. It fetches pages and returns cleaned text and search results while handling JavaScript-heavy sites and anti-bot walls. The app highlights zero-config, keyless operation and on-device document processing. Developers and AI power users gain a private, self-hosted retrieval pipeline for localization, documentation ingestion, and agent-driven research workflows.
What tasks can you actually use it for?
master-fetch serves as a retrieval engine that supplies readable source material to downstream models, focused on ai-text-localization and research tasks. Typical outcomes include extracting documentation for translation, scraping web pages for localized strings, and mapping site content for model prompts. Practical task types:
- Fetching and cleaning HTML for model input
- Site-wide crawling via sitemaps
- Converting PDFs and images into text for ingestion
How reliable and clean are the fetched outputs?
The tool produces stripped, model-ready text by using Trafilatura and lxml to remove ads and boilerplate, and it can output Markdown or plain text suitable for prompt context. For image-heavy documents it applies a local ONNX-based OCR pipeline to extract characters. The project also includes a local neural reranking step that reorders search results, which helps prioritize the most relevant retrieved pages for agent consumption.
What inputs and operational limits should you expect?
Inputs accepted include URLs, site sitemaps for deep crawling, and uploaded PDFs or images for OCR. Installation requires a Python 3.10+ environment and the package (pip install hound-mcp), and stealth fetching can optionally use a local Chrome or Chromium binary. The server integrates with MCP hosts such as Claude Desktop, Cursor, and OpenCode, and its search operates without external API keys by relying on local scraping and routing logic.
Does it fit into developer workflows without heavy overhead?
The design targets developers and AI power users who can self-host and integrate an MCP endpoint into agent stacks. The developer implemented automatic escalation between HTTP, browser rendering, and stealth modes to increase successful retrievals from guarded sites, and the community notes its auto-escalation strengths. Because the service runs entirely locally, teams that prioritize data custody can incorporate it into existing in-house pipelines for model context generation.
Who should adopt it and who should look elsewhere
master-fetch is a developer-oriented choice for teams that build agent-driven research or localization pipelines and can operate a self-hosted server. It suits groups that prioritize control over fetched material and can maintain a Python-based service. Organizations without in-house engineering capacity or those that prefer a managed, turnkey scraping service should consider alternatives that remove the hosting responsibility.
