Index onlineMerchant portal
Api Endpoint

AveDaris Agent Utilities

Read a site's robots.txt: send a domain or URL and get allow/disallow rules per user agent and the sitemap URLs. Use it to check whether a crawl is permitted before fetching pages. $0.003 USDC per call on Base.

Open interactive test console

Offering

Provider
api.avedaris.com
Method
POST
Offering type
Api Endpoint
Transport type
http
Category
Api
Delivery
Not provided
Pricing model
Not provided
x402 version
2

Endpoint context

URL
https://api.avedaris.com/v1/paid/robots-inspect
MIME type
application/json
First seen
2026-10-16T03:04:46.000Z
Last seen
Tags
web-intelligence, robots, inspect, agent-tools

Website

Payment pricing

x402 payment terms are the authoritative price for calling this endpoint.

Payment option 1
0.003 USDC
Atomic price: 3000 · Network: eip155:8453 · Asset: USDC · Scheme: exact
Raw x402 terms
[
  {
    "scheme": "exact",
    "network": "eip155:8453",
    "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
    "amount": "3000",
    "payTo": "0x6c10344d3f96511438458e07a6f2be568a76dfff"
  }
]

Observed health

Status
Healthy
Latest latency
267 ms
Average latency
806 ms
HTTP status
204
Last checked
Reachability
100% over 12 checks

Latency is measured to the first response headers. A reachable result does not verify response accuracy or merchant identity.

Input schema

{
  "type": "object",
  "properties": {
    "type": {
      "type": "string",
      "const": "http"
    },
    "method": {
      "type": "string",
      "enum": [
        "POST"
      ]
    },
    "bodyType": {
      "type": "string",
      "enum": [
        "json",
        "form-data",
        "text"
      ]
    },
    "body": {
      "type": "object",
      "additionalProperties": false,
      "required": [
        "url"
      ],
      "properties": {
        "url": {
          "type": "string",
          "minLength": 8,
          "maxLength": 8192,
          "description": "Public http(s) URL on the site to check; the site's /robots.txt is fetched and parsed into crawler groups and sitemaps. 8 to 8,192 characters."
        }
      }
    }
  },
  "required": [
    "type",
    "method",
    "bodyType",
    "body"
  ],
  "additionalProperties": false
}

Output schema

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "properties": {
    "input": {
      "additionalProperties": false,
      "properties": {
        "body": {
          "additionalProperties": false,
          "properties": {
            "url": {
              "description": "Public http(s) URL on the site to check; the site's /robots.txt is fetched and parsed into crawler groups and sitemaps. 8 to 8,192 characters.",
              "maxLength": 8192,
              "minLength": 8,
              "type": "string"
            }
          },
          "required": [
            "url"
          ],
          "type": "object"
        },
        "bodyType": {
          "enum": [
            "json",
            "form-data",
            "text"
          ],
          "type": "string"
        },
        "method": {
          "enum": [
            "POST"
          ],
          "type": "string"
        },
        "type": {
          "const": "http",
          "type": "string"
        }
      },
      "required": [
        "type",
        "method",
        "bodyType",
        "body"
      ],
      "type": "object"
    },
    "output": {
      "properties": {
        "example": {
          "properties": {
            "result": {
              "description": "Output of robots.inspect. The fields below are the documented core; some responses add further detail fields.",
              "properties": {
                "groups": {
                  "description": "Parsed rule groups, each with userAgents, allow and disallow path lists. Empty when no robots.txt.",
                  "type": "array"
                },
                "robotsUrl": {
                  "description": "The robots.txt URL that was fetched (site origin + /robots.txt).",
                  "type": "string"
                },
                "sitemaps": {
                  "description": "Sitemap URLs declared in robots.txt.",
                  "type": "array"
                },
                "status": {
                  "description": "HTTP status of the robots.txt fetch; 404 means the site has none.",
                  "type": "integer"
                }
              },
              "required": [
                "robotsUrl",
                "status",
                "groups",
                "sitemaps"
              ],
              "type": "object"
            }
          },
          "required": [
            "result"
          ],
          "type": "object"
        },
        "type": {
          "type": "string"
        }
      },
      "required": [
        "type"
      ],
      "type": "object"
    }
  },
  "required": [
    "input"
  ],
  "type": "object"
}