Index onlineMerchant portal
Api Endpoint

api.x402node.dev

Fetch and parse robots.txt for a domain. Returns parsed allow/disallow rules per user-agent, sitemap URLs, crawl-delay directives, and raw content. Built for AI agents planning ethical web crawling and SEO audit. Use ?domain=example.com. Accepts payment on Base or Solana — either network works.

Open interactive test console

Offering

Provider
api.x402node.dev
Method
GET
Offering type
Api Endpoint
Transport type
http
Category
General Utilities
Delivery
Not provided
Pricing model
Not provided
x402 version
2

Endpoint context

URL
https://api.x402node.dev/web/robots
MIME type
application/json
First seen
2026-10-16T02:09:24.457Z
Last seen
Tags
None supplied

Website

Payment pricing

x402 payment terms are the authoritative price for calling this endpoint.

Payment option 1
0.005 USDC
Atomic price: 5000 · Network: eip155:8453 · Asset: USDC · Scheme: exact
Payment option 2
0.005 USDC
Atomic price: 5000 · Network: solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp · Asset: USDC · Scheme: exact
Raw x402 terms
[
  {
    "scheme": "exact",
    "network": "eip155:8453",
    "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
    "amount": "5000",
    "payTo": "0x4466d4A84b7c49a6A094ec6eef4a0712D6dd125e"
  },
  {
    "scheme": "exact",
    "network": "solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp",
    "asset": "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
    "amount": "5000",
    "payTo": "9urRvUx69HqAopg1JGV9P9wR7JdKoztzdkuNyJ8Eo1up"
  }
]

Observed health

Status
Healthy
Latest latency
185 ms
Average latency
435 ms
HTTP status
402
Last checked
Reachability
100% over 12 checks

Latency is measured to the first response headers. A reachable result does not verify response accuracy or merchant identity.

Input schema

{
  "type": "object",
  "properties": {
    "type": {
      "type": "string",
      "const": "http"
    },
    "method": {
      "type": "string",
      "enum": [
        "GET"
      ]
    },
    "queryParams": {
      "type": "object",
      "properties": {
        "domain": {
          "type": "string",
          "description": "Domain name (optional)"
        }
      }
    }
  },
  "required": [
    "type",
    "method"
  ],
  "additionalProperties": false
}

Output schema

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "input": {
      "type": "object",
      "properties": {
        "type": {
          "type": "string",
          "const": "http"
        },
        "method": {
          "type": "string",
          "enum": [
            "GET"
          ]
        },
        "queryParams": {
          "type": "object",
          "properties": {
            "domain": {
              "type": "string",
              "description": "Domain name (optional)"
            }
          }
        }
      },
      "required": [
        "type",
        "method"
      ],
      "additionalProperties": false
    }
  },
  "required": [
    "input"
  ]
}