Web Search Research x402 offerings

Clean, LLM-ready markdown from any URL. JavaScript rendering included. Feed web content directly into agent context without parsing HTML.

robots.txt and AI crawler policy check. Tells an agent whether a website permits crawling and whether it blocks AI bots such as GPTBot, ClaudeBot, Google-Extended, PerplexityBot and CCBot, before scraping, RAG ingestion, training data collection or archiving. Parses robots.txt and returns a crawl-policy tier with per-bot allow and disallow rules and sitemap URLs.

Read a website's /llms.txt (the standard that tells LLMs what a site is about) as structured data: title, summary, sections with links and descriptions, and whether llms-full.txt exists.

Read a website's /llms.txt (the standard that tells LLMs what a site is about) as structured data: title, summary, sections with links and descriptions, and whether llms-full.txt exists.

All links on a page classified as internal or external, with anchor text, nofollow/sponsored, new-tab flags and the most linked external domains. For SEO, crawling and audits.

Extractive summary without AI: picks the most representative sentences (term frequency and position scoring) and returns them in original order, with the reduction percentage. Deterministic and insta…

SEO and sharing metadata of a page: title and description with length, canonical, robots, language, hreflang, Open Graph, Twitter Card, JSON-LD types, favicon, H1/H2, plus warnings (missing tags, noi…

Summary of a Wikipedia article in any language: title, short description, lead paragraph, image, coordinates, link and Wikidata id. If the title is not exact it returns the closest article, flagged a…

List the URLs of a site's XML sitemap (found via robots.txt if needed, following sitemap indexes and .gz files) with last-modified dates, optionally filtered by text. Discover a site's pages for craw…

SEO and sharing metadata of a page: title and description with length, canonical, robots, language, hreflang, Open Graph, Twitter Card, JSON-LD types, favicon, H1/H2, plus warnings (missing tags, noi…

Parse a site's robots.txt: whether a user-agent may crawl a path, which AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot...) are blocked, and declared sitemaps. Check before your…

All links on a page classified as internal or external, with anchor text, nofollow/sponsored, new-tab flags and the most linked external domains. For SEO, crawling and audits.

List the URLs of a site's XML sitemap (found via robots.txt if needed, following sitemap indexes and .gz files) with last-modified dates, optionally filtered by text. Discover a site's pages for craw…

Detect the technologies behind a website: CMS (WordPress, Shopify...), frameworks (Next.js, React...), server and CDN, analytics, payments, chat, cookie manager, captcha and fonts, from HTML and head…

Summary of a Wikipedia article in any language: title, short description, lead paragraph, image, coordinates, link and Wikidata id. If the title is not exact it returns the closest article, flagged a…

Extractive summary without AI: picks the most representative sentences (term frequency and position scoring) and returns them in original order, with the reduction percentage. Deterministic and insta…

Fetch a public URL and return cleaned markdown

WEB SEARCH for keyless agents — POST {query} and get LLM-ready results: title, url, extracted content snippet, relevance score, plus a direct answer synthesized from live results. Options: max_results (1-10, default 5), topic (general|news). No API key, no account — the x402 payment IS the auth. Query up to 400 chars.

Grounded answer with inline [n] citations: searches the web, fetches sources, and answers strictly from them.

Capture high-quality screenshots of any webpage using headless browser.

Real-time news search via Google News. Returns up to 10 ranked articles with snippet, source, date, and thumbnail.

Multi-step research in one call: plans sub-questions, searches each, fetches and dedupes sources, and synthesizes an answer with inline [n] citations, key findings, and gaps. Tiers: quick = 1 search …

Real-time web search powered by Google. Returns ranked results with titles, URLs, and snippets. Set includeContent to also fetch the top result pages as markdown in the same call.

Fetch webpage with full JavaScript rendering using headless browser. Perfect for SPAs, React/Vue apps, and dynamic content that requires JS execution.