Index onlineMerchant portal
Api Endpoint

Website Crawler to Markdown

Website crawler: read a whole website or one section of it (for example every page under /docs) as clean Markdown: starts at a URL, follows same-site links breadth-first and returns up to 20 pages with title and description. Use for documentation, product catalogs or company sites. For one page use web-scrape, for a known list of URLs web-read-batch (up to 5) or web-scrape-batch (up to 20). Charged only if at least one page was read.

Open interactive test console

Offering

Provider
agentsvc.io
Method
POST
Offering type
Api Endpoint
Transport type
http
Category
Web Search Research
Delivery
Not provided
Pricing model
Not provided
x402 version
2

Endpoint context

URL
https://agentsvc.io/api/v1/proxy/site-crawl
MIME type
application/json
First seen
2026-10-16T01:50:32.458Z
Last seen
Tags
web-crawler, site-crawl, website-to-markdown, crawl, markdown

Website

Payment pricing

x402 payment terms are the authoritative price for calling this endpoint.

Payment option 1
0.005 USDC
Atomic price: 5000 · Network: eip155:8453 · Asset: USDC · Scheme: exact
Payment option 2
0.005 USDC
Atomic price: 5000 · Network: solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp · Asset: USDC · Scheme: exact
Payment option 3
0.005 USDC
Atomic price: 5000 · Network: eip155:137 · Asset: USDC · Scheme: exact
Payment option 4
0.005 USDC
Atomic price: 5000 · Network: eip155:42161 · Asset: USDC · Scheme: exact
Raw x402 terms
[
  {
    "scheme": "exact",
    "network": "eip155:8453",
    "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
    "amount": "5000",
    "payTo": "0x622efA6Ceb83a5b6EdBC931332f4db68B7f74F27"
  },
  {
    "scheme": "exact",
    "network": "solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp",
    "asset": "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
    "amount": "5000",
    "payTo": "5JzPS5qpEwUM1vTC9GG7uqvwJdHzbEwfoPHf2Ki977tg"
  },
  {
    "scheme": "exact",
    "network": "eip155:137",
    "asset": "0x3c499c542cEF5E3811e1192ce70d8cC03d5c3359",
    "amount": "5000",
    "payTo": "0x622efA6Ceb83a5b6EdBC931332f4db68B7f74F27"
  },
  {
    "scheme": "exact",
    "network": "eip155:42161",
    "asset": "0xaf88d065e77c8cC2239327C5EDb3A432268e5831",
    "amount": "5000",
    "payTo": "0x622efA6Ceb83a5b6EdBC931332f4db68B7f74F27"
  }
]

Observed health

Status
Healthy
Latest latency
319 ms
Average latency
984 ms
HTTP status
204
Last checked
Reachability
100% over 14 checks

Latency is measured to the first response headers. A reachable result does not verify response accuracy or merchant identity.

Input schema

{
  "type": "object",
  "properties": {
    "type": {
      "type": "string",
      "const": "http"
    },
    "method": {
      "type": "string",
      "enum": [
        "POST",
        "PUT",
        "PATCH"
      ]
    },
    "bodyType": {
      "type": "string",
      "enum": [
        "json",
        "form-data",
        "text"
      ]
    },
    "body": {
      "properties": {
        "url": {
          "type": "string"
        },
        "max_pages": {
          "type": "integer",
          "minimum": 1,
          "maximum": 20,
          "default": 5
        },
        "max_chars": {
          "type": "integer",
          "minimum": 500,
          "maximum": 20000,
          "default": 4000
        },
        "path_prefix": {
          "type": "string",
          "description": "Only follow links whose path starts with this, e.g. /docs"
        }
      },
      "required": [
        "url"
      ]
    }
  },
  "required": [
    "type",
    "method",
    "bodyType",
    "body"
  ],
  "additionalProperties": false
}

Output schema

{
  "type": "object",
  "properties": {
    "type": {
      "type": "string"
    },
    "example": {
      "type": "object",
      "required": [
        "success",
        "data"
      ],
      "properties": {
        "success": {
          "type": "boolean"
        },
        "data": {
          "type": "object",
          "properties": {
            "site": {
              "type": "string"
            },
            "pages_crawled": {
              "type": "integer"
            },
            "ok_count": {
              "type": "integer"
            },
            "skipped_non_html": {
              "type": "integer"
            },
            "discovered_urls": {
              "type": "integer"
            },
            "pages": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "url": {
                    "type": "string"
                  },
                  "ok": {
                    "type": "boolean"
                  },
                  "status": {
                    "type": "integer"
                  },
                  "title": {
                    "type": "string"
                  },
                  "description": {
                    "type": "string"
                  },
                  "markdown": {
                    "type": "string"
                  },
                  "word_count": {
                    "type": "integer"
                  }
                }
              }
            },
            "took_ms": {
              "type": "integer"
            },
            "crawled_at": {
              "type": "string"
            }
          },
          "required": [
            "site",
            "pages_crawled",
            "ok_count",
            "pages"
          ]
        }
      }
    }
  },
  "required": [
    "type"
  ]
}