llms.txt
+92 −48
https://agdata.shveik.dev/llms.txt
− - MPP is Base only. Solana is taken on every route except unblock, twitter, reddit, youtube-transcript, screenshot, linkedin-jobs, amazon-reviews, web-search, x-replies, youtube-comments, tiktok-comments and instagram-comments: pay those on Base or Polygon (their 402 has no Solana offer).− - That quote is what you sign (x402 `exact` and MPP `charge` are fixed-amount). If fewer results come back, the per-result price of the missing ones is refunded on-chain to the paying address, usually within minutes of the response; refunds too small to be worth a transfer are not sent. If the scrape finds nothing, you pay only the fixed per-call fee: every result is refunded (except on unblock, instagram, google-maps, tiktok, linkedin, youtube, twitter, reddit, amazon, google-search, google-map− - A field an endpoint does not know is ignored on instagram, google-maps, tiktok, linkedin, youtube, twitter, reddit, amazon and google-search and refused on unblock, google-maps-reviews, youtube-transcript, screenshot, linkedin-jobs, indeed-jobs, google-news, amazon-reviews, web-search, x-replies, youtube-comments, tiktok-comments, instagram-comments and search-plus.− - Errors the service itself produces are JSON: `{"error": code, "message": text}`. An invalid request body gets the 400 `invalid_request` once a payment is attached; without a payment it gets the 402 quote for 1 result with the header `X-Agdata-Invalid-Request: 1`, which is not worth signing: read `error` in it.− - instagram, reddit, youtube, linkedin and amazon records carry a `normalized` object with the same keys whichever upstream provider served the request (null where a provider has no value), plus a `source` field. The provider's own fields are kept alongside it.− - Who is saying what (people & social): Posts, profiles and search results from the big social platforms. Replies under a post on X. Comments under a YouTube video. Comments under a TikTok video. Comments under an Instagram post. Endpoints: `instagram`, `tiktok`, `reddit`, `twitter`, `youtube`, `linkedin`, `x-replies`, `youtube-comments`, `tiktok-comments`, `instagram-comments`.− - What customers say (places & reviews): Google Maps places by search and location. Reviews of a place. Reviews of an Amazon product. Endpoints: `google-maps`, `google-maps-reviews`, `amazon-reviews`.− - What is hiring, what is selling (jobs & products): LinkedIn job listings. Indeed job listings with salary and employer details. Amazon product search. Endpoints: `linkedin-jobs`, `indeed-jobs`, `amazon`.− - Need a residential IP of your own? agproxy sells residential proxy traffic by the gigabyte and a raw fetch from a residential address, paid the same way. Site: https://agproxy.shveik.dev, docs: https://agproxy.shveik.dev/llms.txt− - POST `/x402/screenshot` (or `/mpp/screenshot`) with `{"url":"https://example.com/","format":"jpeg"}`. `format` is `jpeg` (the default), `png`, `webp` or `pdf`. Optional: `fullPage`, `device` (`desktop`, `desktop_hd`, `mobile` or `tablet`), `width`, `height`, `delayMs`, `colorScheme` (`light` or `dark`), `selector` (capture only that element) and `waitForSelector`, both plain CSS selectors. Cookies, headers and proxies are not accepted.− - A call can take up to about 60 s: keep the connection open. Retrying with the same payment returns the stored capture for 24 hours. It is not offered on Solana: pay on Base or Polygon.− - The answer is a JSON array with one record: `videoId`, `url`, `title`, `channelName`, `channelUrl`, `durationSeconds`, `publishedAt`, `viewCount`, `language`, `availableLanguages`, `subtitleType`, `format`, `transcript`, `segmentCount` and `truncated`. With `json`, `transcript` is a list of `{start, duration, text}` (seconds); with `text`, `srt` or `vtt` it is one string. Only published subtitles are returned, there is no speech-to-text. `any` takes the English track when the video has one, el− - A call can take up to about 60 s: keep the connection open. Retrying with the same payment returns the stored first answer, in the format of the first call. It is not offered on Solana: pay on Base or Polygon.− - POST `/x402/google-maps-reviews` (or `/mpp/google-maps-reviews`) with `{"place":"ChIJN1t_tDeuEmsRUsoyG83frY4","limit":10}`. `place` is a Google place_id (27 characters starting with `ChIJ` or `GhIJ`) or a Google Maps place URL (`https://www.google.com/maps/place/...` or `https://www.google.com/maps?cid=...`; a URL with only the place's name and no place id is best effort and can end as not fetched, which is not charged). `limit` is 1 to 50 (default 10), `sort` is `newest` (the default), `relev− - The quote is for the `limit` you ask for; if fewer reviews exist, the missing ones are refunded like any other results. A place with no reviews, or an unknown place, is never charged: `502 {"error":"not_fetched",...,"charged":false}`.− - The answer is a JSON array with one record per job: `id`, `url`, `title`, `companyName`, `companyUrl`, `location`, `postedDate`, `postedTimeAgo`, `contractType`, `experienceLevel`, `workType`, `sector`, `salary`, `applicants`, `applyType`, `applyUrl`, `description` and `descriptionTruncated` (null where a listing has no value). Recruiter names are never returned. Nothing found, or a source that returned no data, is not charged: the answer is 502 not_fetched, said plainly, and the payment is no− - The answer is a JSON array with one record per job: `id`, `url`, `applyUrl`, `title`, `publishedAt`, `expired`, `language`, `location` (`city`, `state`, `country`, `postalCode`), `employer` (`name`, `url`, `website`, `industry`, `employeeCount`, `rating`, `ratingsCount`), `salary` (`min`, `max`, `unit`, `currency`, or null), `jobTypes`, `benefits`, `description` and `descriptionTruncated`. Nothing found, or a source that returned no data, is not charged: the answer is 502 not_fetched, said pla− - The answer is a JSON array with one record per article: `title`, `url`, `source`, `publishedAt` and `description`. `url` is the article's Google News link, which opens the publisher's page; the article's text is not included. Nothing found, or a source that returned no data, is not charged: the answer is 502 not_fetched, said plainly, and the payment is not settled.− - The quote is for the `limit` you ask for; if fewer reviews exist, the missing ones are refunded like any other results. A product with no reviews, or an unknown one, is never charged: `502 {"error":"not_fetched",...,"charged":false}`. A call can take up to about 15 s. It is not offered on Solana: pay on Base or Polygon.− - The answer is a JSON array with one record per reply: `id`, `url`, `text` (cut at 4,000 characters, with `textTruncated`), `createdAt`, `lang`, `likeCount`, `replyCount`, `repostCount`, `quoteCount`, `viewCount`, `inReplyToId` and `author` (`username`, `name`, `verified`). The post itself is not in the array. Nothing found, or a source that returned no data, is not charged: the answer is 502 not_fetched, said plainly, and the payment is not settled. Fewer replies than asked for refunds the mis− - The answer is a JSON array with one record per comment: `id`, `text` (cut at 2,000 characters, with `textTruncated`), `postedAt`, `likeCount`, `replyCount`, `isReply`, `byOwner`, `ownerLiked` and `pinned` (null where the platform does not say). The commenter's name, handle and photo are never returned. Nothing found, or a source that returned no data, is not charged: the answer is 502 not_fetched, said plainly, and the payment is not settled. Fewer comments than asked for refunds the missing o− - The answer is a JSON array with one record per result: `position`, `title`, `url`, `description` (a query-relevant excerpt, cut at 2,000 characters), `type`, `category` and `date` (null where there is none). Nothing found, or a source that returned no data, is not charged: the answer is 502 not_fetched, said plainly, and the payment is not settled. Fewer results than asked for refunds the missing ones.− - The answer is a JSON array with one record per page that was read: `url`, `title`, `description`, `language`, `content` (the page as Markdown, cut at 20,000 characters) and `contentTruncated`. A result whose page could not be fetched is dropped and its price refunded; Nothing found, or a source that returned no data, is not charged: the answer is 502 not_fetched, said plainly, and the payment is not settled.− - A call takes about 20 to 30 s. It is not offered on Solana: pay on Base or Polygon.− - [POST /x402/unblock](https://agdata.shveik.dev/x402/unblock): Unblocker: any URL to clean Markdown. Example body: `{"format":"markdown","url":"https://example.com/"}`− - [POST /x402/instagram](https://agdata.shveik.dev/x402/instagram): Instagram posts by profile, tag, search. Example body: `{"limit":5,"mode":"profile","query":"natgeo"}`− - [POST /x402/google-maps](https://agdata.shveik.dev/x402/google-maps): Google Maps places and businesses. Example body: `{"limit":5,"location":"Prague","query":"coffee shops"}`− - [POST /x402/tiktok](https://agdata.shveik.dev/x402/tiktok): TikTok videos by user, hashtag, search. Example body: `{"limit":5,"mode":"profile","query":"nasa"}`− - [POST /x402/linkedin](https://agdata.shveik.dev/x402/linkedin): LinkedIn profile lookup. Example body: `{"query":"jane-example"}`− - [POST /x402/youtube](https://agdata.shveik.dev/x402/youtube): YouTube video search and channel videos. Example body: `{"limit":5,"mode":"search","query":"web scraping tutorials"}`− - [POST /x402/twitter](https://agdata.shveik.dev/x402/twitter): X (Twitter) tweets by handle or search. Example body: `{"limit":5,"mode":"profile","query":"elonmusk"}`− - [POST /x402/reddit](https://agdata.shveik.dev/x402/reddit): Reddit posts by subreddit, user, search. Example body: `{"limit":5,"mode":"hashtag","query":"programming"}`− - [POST /x402/amazon](https://agdata.shveik.dev/x402/amazon): Amazon product search. Example body: `{"limit":5,"query":"wireless keyboard"}`− - [POST /x402/google-search](https://agdata.shveik.dev/x402/google-search): Google Search results (SERP). Example body: `{"limit":5,"query":"best CRM tools"}`− - [POST /x402/google-maps-reviews](https://agdata.shveik.dev/x402/google-maps-reviews): Google Maps reviews of a place. Example body: `{"limit":10,"place":"ChIJN1t_tDeuEmsRUsoyG83frY4"}`− - [POST /x402/youtube-transcript](https://agdata.shveik.dev/x402/youtube-transcript): YouTube transcript of a video. Example body: `{"format":"json","lang":"en","video":"dQw4w9WgXcQ"}`− - [POST /x402/screenshot](https://agdata.shveik.dev/x402/screenshot): Website screenshot or PDF of any URL. Example body: `{"format":"jpeg","url":"https://example.com/"}`− - [POST /x402/linkedin-jobs](https://agdata.shveik.dev/x402/linkedin-jobs): LinkedIn job listings for a search. Example body: `{"limit":10,"location":"Prague","query":"data engineer"}`− - [POST /x402/indeed-jobs](https://agdata.shveik.dev/x402/indeed-jobs): Indeed job listings for a search. Example body: `{"country":"us","limit":10,"location":"Austin, TX","query":"nurse"}`− - [POST /x402/google-news](https://agdata.shveik.dev/x402/google-news): Google News articles for a search. Example body: `{"limit":10,"query":"openai"}`+ > Preferred location: /agents.md (this text is also served at /llms.txt).+ ## When to use this service+ - Use it when an agent needs structured public web data.+ - Pick the endpoint matching the source and data type, read its request shape below, send a JSON POST; `/openapi.json` is the contract.+ - Not for private or login-protected content, browsing without a data task, or workloads needing a coverage commitment; sources are best-effort and failed calls are not charged.+ - MPP is Base only. Solana is taken on every route except unblock, twitter, reddit, youtube-transcript, screenshot, linkedin-jobs, shopify-products, amazon-reviews, web-search, x-replies, youtube-comments, tiktok-comments, instagram-comments, ebay-sold, meta-ads, linkedin-profile-email and linkedin-posts: pay those on Base or Polygon (their 402 has no Solana offer).+ - You sign that quote (x402 `exact` and MPP `charge` are fixed-amount). Missing results are refunded on-chain to the paying address within minutes (refunds too small to send are skipped). If the scrape finds nothing, every result is refunded and you pay only the fixed per-call fee (except on unblock, instagram, google-maps, tiktok, linkedin, youtube, twitter, reddit, amazon, google-search, google-maps-reviews, youtube-transcript, screenshot, linkedin-jobs, indeed-jobs, shopify-products, google-n+ - A field an endpoint does not know is ignored on instagram, google-maps, tiktok, linkedin, youtube, twitter, reddit, amazon and google-search and refused on unblock, google-maps-reviews, youtube-transcript, screenshot, linkedin-jobs, indeed-jobs, shopify-products, google-news, amazon-reviews, google-play-reviews, linkedin-company, web-search, x-replies, youtube-comments, tiktok-comments, instagram-comments, ebay-sold, meta-ads, linkedin-profile-email, linkedin-posts and search-plus.+ - Service errors are JSON `{"error": code, "message": text}`. An invalid body gets 400 `invalid_request` once paid; unpaid, it gets a 402 quote for 1 result with `X-Agdata-Invalid-Request: 1`, not worth signing: read `error`.+ - instagram, youtube, linkedin and amazon records carry a `normalized` object with the same keys whichever upstream served it (null where it has no value), plus `source`; the upstream's own fields stay.+ - reddit and tiktok return compact records by default (reddit: posts only, at most `limit`). Send `"raw": true` to get the provider's own records instead (more fields; for tiktok some links in them expire within hours).+ - Who is saying what (people & social): Posts, profiles and search results from the big social platforms. LinkedIn company details from page URLs. One LinkedIn profile with its verified email. Posts of a LinkedIn profile or company page. Replies under a post on X. Comments under a YouTube video. Comments under a TikTok video. Comments under an Instagram post. Endpoints: `instagram`, `tiktok`, `reddit`, `twitter`, `youtube`, `linkedin`, `linkedin-company`, `linkedin-profile-email`, `linkedin-post+ - What customers say (places & reviews): Google Maps places by search and location. Reviews of a place. Reviews of an Amazon product. Reviews of an Android app on Google Play. Endpoints: `google-maps`, `google-maps-reviews`, `amazon-reviews`, `google-play-reviews`.+ - What is hiring, what is selling (jobs & products): LinkedIn job listings. Indeed job listings with salary and employer details. Amazon product search. eBay sold listings, for resale price comps. Public products of a Shopify store, with prices and variants. Public ads from the Meta ad library, for competitor research. Endpoints: `linkedin-jobs`, `indeed-jobs`, `amazon`, `ebay-sold`, `shopify-products`, `meta-ads`.+ ## Pay with a wallet CLI+ No x402 client? The open-source agpay wallet CLI (https://github.com/shveik-dev/agpay) pays x402 routes within limits you set: `go install github.com/shveik-dev/agpay/cmd/agpay@latest`, `agpay init --generate-passphrase-file ~/.agpay.pass`, then for example `agpay pay https://agdata.shveik.dev/x402/web-search --data '{"query":"lithium prices"}' --max 1`. `agpay mcp` serves MCP.+ ## Limits and reliability+ No uptime SLA; service and upstream availability are best-effort. A quote is shown before payment, and work starts only after settlement; a request refused before settlement is not charged. Refunds for partial results, orders and deals follow the docs. Paid calls have an upstream timeout, usually 120 seconds (shorter on some routes); some routes also cap in-flight work per host. Undeliverable results are not charged and the missing portion of a short result is refunded. Free GET docs, guides, pr+ ## Reference documents+ - [API reference](/docs.md)+ - [OpenAPI](/openapi.json)+ - Need a residential IP of your own? agproxy sells residential proxy traffic by the gigabyte and a raw fetch from a residential address, paid the same way. Site: https://agproxy.shveik.dev, docs: https://agproxy.shveik.dev/agents.md+ - POST `/x402/screenshot` (or `/mpp/screenshot`) with `{"url":"https://example.com/","format":"jpeg"}`. `format` is `jpeg` (the default), `png`, `webp` or `pdf`. Optional: `fullPage`, `device` (`desktop`, `desktop_hd`, `mobile` or `tablet`), `width`, `height`, `delayMs`, `colorScheme` (`light` or `dark`), `selector` (capture one element) and `waitForSelector`, both CSS selectors. Cookies, headers and proxies are refused.+ - A call can take up to about 60 s: keep the connection open. Retrying with the same payment returns the stored capture for 24 hours.+ - The answer is a JSON array with one record: `videoId`, `url`, `title`, `channelName`, `durationSeconds`, `publishedAt`, `language`, `availableLanguages`, `subtitleType`, `format`, `transcript` and `truncated`. With `json`, `transcript` is a list of `{start, duration, text}` (seconds); with `text`, `srt` or `vtt` it is one string. Only published subtitles are returned, there is no speech-to-text. `any` takes English when present, else the first track.+ - A call can take up to about 60 s: keep the connection open. Retrying with the same payment returns the stored first answer, in the format of the first call.+ - POST `/x402/google-maps-reviews` (or `/mpp/google-maps-reviews`) with `{"place":"ChIJN1t_tDeuEmsRUsoyG83frY4","limit":10}`. `place` is a Google place_id (27 characters starting with `ChIJ` or `GhIJ`) or a Google Maps place URL (`https://www.google.com/maps/place/...` or `https://www.google.com/maps?cid=...`; a URL without a place id is best effort: otherwise not fetched, not charged). `limit` is 1 to 50 (default 10), `sort` is `newest` (the default), `relevant`, `highest` or `lowest`, `languag+ - The quote is for the `limit` you ask for; missing reviews are refunded. A place with no reviews, or an unknown place, is never charged: `502 {"error":"not_fetched",...,"charged":false}`.+ - The answer is a JSON array with one record per job: `id`, `url`, `title`, `companyName`, `companyUrl`, `location`, `postedDate`, `postedTimeAgo`, `contractType`, `experienceLevel`, `workType`, `sector`, `salary`, `applicants`, `applyType`, `applyUrl`, `description` and `descriptionTruncated` (null where a listing has no value). Recruiter names are never returned. Nothing found is not charged: `502 not_fetched`, and the payment is not settled.+ - The answer is a JSON array with one record per job: `id`, `url`, `applyUrl`, `title`, `publishedAt`, `expired`, `language`, `location` (`city`, `state`, `country`, `postalCode`), `employer` (`name`, `url`, `website`, `industry`, `employeeCount`, `rating`, `ratingsCount`), `salary` (`min`, `max`, `unit`, `currency`, or null), `jobTypes`, `benefits`, `description` and `descriptionTruncated`. Nothing found is not charged: `502 not_fetched`, and the payment is not settled.+ ## Company details from LinkedIn+ - POST `/x402/linkedin-company` (or `/mpp/linkedin-company`) with `{"companies":["https://www.linkedin.com/company/stripe/"]}`. `companies` is 1 to 10 LinkedIn company page URLs (`https://www.linkedin.com/company/<name>/`). No search by name; priced per company.+ - The answer is a JSON array with one record per company: `name`, `universalName`, `linkedinUrl`, `website`, `employeeCount`, `followerCount`, `foundedOn`, `tagline`, `description` (cut at 3,000 characters), `industries`, `specialities`, `headquarters`, `locations`, `companyType` and `pageVerified`. No people are returned.+ - Companies not found are refunded; none found is never charged (`502 not_fetched`).+ ## LinkedIn profile with verified email+ - POST `/x402/linkedin-profile-email` (or `/mpp/linkedin-profile-email`) with `{"url":"https://www.linkedin.com/in/jane-example/"}`. One profile URL per call. The answer is the profile with its verified work email (`emails`). No verified email is `502 not_fetched`, never charged. Not offered on Solana.+ ## LinkedIn posts+ - POST `/x402/linkedin-posts` (or `/mpp/linkedin-posts`) with `{"url":"https://www.linkedin.com/company/stripe/","limit":5}`. `url` is a profile or company page URL; `limit` is 1 to 20 (default 5). Priced per post; no posts is `502 not_fetched`, never charged. Not offered on Solana.+ ## Shopify store products+ - POST `/x402/shopify-products` (or `/mpp/shopify-products`) with `{"store":"example.com","limit":20}`. `store` is the store's domain (no port, path or IP address); `limit` is 1 to 250 (default 20).
Showing the first 40 of each.