Api Endpoint

web.openverbs.com

Fetch a site's robots.txt and evaluate crawl permissions: given a URL (plus optional extra paths) and a user-agent, return whether each path is allowed or disallowed, the rule that matched, the user-…

Open interactive test console

Offering

Provider: web.openverbs.com
Method: GET
Offering type: Api Endpoint
Transport type: http
Category: Web Search Research
Delivery: Not provided
Pricing model: Not provided
x402 version: 1

Endpoint context

URL: https://web.openverbs.com/v1/robots
MIME type: Not provided
First seen: 2026-10-16T02:29:43.002Z
Last seen: 2026-10-25T22:55:32.800Z
Tags: None supplied

Website

Payment pricing

x402 payment terms are the authoritative price for calling this endpoint.

Payment option 1: 0.006 USDC
Atomic price: 6000 · Network: eip155:8453 · Asset: USDC · Scheme: exact

Raw x402 terms

[
  {
    "scheme": "exact",
    "network": "eip155:8453",
    "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
    "amount": "6000",
    "payTo": "0xb3f4de103F34b99f0Da19B4F16D6B53a14C2194A"
  }
]

Observed health

Status: Healthy
Latest latency: 169 ms
Average latency: 255 ms
HTTP status: 402
Last checked: 2026-11-04T05:02:00.618Z
Reachability: 100% over 8 checks

Latency is measured to the first response headers. A reachable result does not verify response accuracy or merchant identity.

Input schema

{}

Output schema

null