Api Endpoint

api.trustsource.cc

robots.txt and AI crawler policy check. Tells an agent whether a website permits crawling and whether it blocks AI bots such as GPTBot, ClaudeBot, Google-Extended, PerplexityBot and CCBot, before scraping, RAG ingestion, training data collection or archiving. Parses robots.txt and returns a crawl-policy tier with per-bot allow and disallow rules and sitemap URLs.

Open interactive test console

Offering

Provider: api.trustsource.cc
Method: GET
Offering type: Api Endpoint
Transport type: http
Category: Web Search Research
Delivery: Not provided
Pricing model: Not provided
x402 version: 2

Endpoint context

URL: https://api.trustsource.cc/robots
MIME type: application/json
First seen: 2026-10-16T01:59:37.852Z
Last seen: 2026-11-01T23:01:24.556Z
Tags: None supplied

Website

Payment pricing

x402 payment terms are the authoritative price for calling this endpoint.

Payment option 1: 0.002 USDC
Atomic price: 2000 · Network: eip155:8453 · Asset: USDC · Scheme: exact

Raw x402 terms

[
  {
    "scheme": "exact",
    "network": "eip155:8453",
    "asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
    "amount": "2000",
    "payTo": "0xC1FE5Ce97fB6038527D9660cF290b716a264e874"
  }
]

Observed health

Status: Healthy
Latest latency: 130 ms
Average latency: 564 ms
HTTP status: 402
Last checked: 2026-11-01T23:01:24.552Z
Reachability: 100% over 14 checks

Latency is measured to the first response headers. A reachable result does not verify response accuracy or merchant identity.

Input schema

{
  "type": "object",
  "properties": {
    "type": {
      "type": "string",
      "const": "http"
    },
    "method": {
      "type": "string",
      "enum": [
        "GET"
      ]
    },
    "queryParams": {
      "type": "object",
      "properties": {
        "domain": {
          "type": "string",
          "description": "Domain whose robots.txt to fetch and parse (e.g. example.com); https is tried first, then http."
        }
      },
      "required": [
        "domain"
      ]
    }
  },
  "required": [
    "type",
    "method"
  ],
  "additionalProperties": false
}

Output schema

{
  "type": "object",
  "properties": {
    "type": {
      "type": "string"
    },
    "example": {
      "type": "object"
    }
  },
  "required": [
    "type"
  ]
}