# Distill HTML to Markdown API ## Overview Turn any web page or raw HTML into clean Markdown for LLMs and RAG: main content only, with title, author, date, language and token counts. Respects robots.txt. Free plan on RapidAPI. ## Authentication Base URL: `https://distill-html-to-markdown-api.p.rapidapi.com` All requests go through RapidAPI. Send these two headers with every request: ``` X-RapidAPI-Key: YOUR_RAPIDAPI_KEY X-RapidAPI-Host: distill-html-to-markdown-api.p.rapidapi.com ``` ## Endpoints ### GET /v1/markdown Fetches the page (robots.txt first, redirects followed and checked hop by hop), extracts the main content and returns Markdown with metadata. | param | type | default | meaning | |---|---|---|---| | url | string | — | Page to fetch: http/https, ports 80/443, a hostname (not an IP), no credentials. A bare domain gets https://. Required for GET. | | include_links | bool | true | Keep [text](url) links in the Markdown and return the links list. false gives plain text and links: null. | | include_images | bool | false | Keep images as ![alt](src), with absolute URLs. | | include_tables | bool | true | Keep tables as GitHub-flavoured Markdown tables. | | max_chars | int | none | 100 to 2,000,000. Cut the Markdown at a paragraph (or word) boundary, close an open code fence, set truncated: true. | Request ``` curl -G "https://distill-html-to-markdown-api.p.rapidapi.com/v1/markdown" \ --data-urlencode "url=https://example.com/" \ -H "X-RapidAPI-Key: YOUR_RAPIDAPI_KEY" \ -H "X-RapidAPI-Host: distill-html-to-markdown-api.p.rapidapi.com" ``` Response · 200 (real) ``` { "url": "https://example.com/", "final_url": "https://example.com/", "status": "ok", "detail": null, "http_status": 200, "title": "Example Domain", "author": null, "published_date": null, "language": "en", "site_name": "example.com", "description": null, "markdown": "This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.", "word_count": 25, "char_count": 156, "approx_tokens": 39, "truncated": false, "links": [], "cached": false, "took_ms": 99 } ``` With options ``` curl -G "https://distill-html-to-markdown-api.p.rapidapi.com/v1/markdown" \ --data-urlencode "url=https://en.wikipedia.org/wiki/Markdown" \ -d include_links=false -d max_chars=4000 \ -H "X-RapidAPI-Key: YOUR_RAPIDAPI_KEY" \ -H "X-RapidAPI-Host: distill-html-to-markdown-api.p.rapidapi.com" ``` ### POST /v1/markdown JSON body. Use it for long URLs, or to convert HTML you already have. | field | meaning | |---|---| | url | Same as the query parameter. Send url or html, not both. | | html | Raw HTML you already have, up to 5 MB. Nothing is fetched. A bare fragment (

…

) works too. | | base_url | With html only: the page's address, used to make relative links and images absolute. | | include_links, include_images, include_tables, max_chars | Same as for GET. | URL ``` curl -X POST "https://distill-html-to-markdown-api.p.rapidapi.com/v1/markdown" \ -H "X-RapidAPI-Key: YOUR_RAPIDAPI_KEY" \ -H "X-RapidAPI-Host: distill-html-to-markdown-api.p.rapidapi.com" \ -H "Content-Type: application/json" \ -d '{"url": "https://docs.python.org/3/library/json.html", "include_images": true}' ``` Raw HTML ``` curl -X POST "https://distill-html-to-markdown-api.p.rapidapi.com/v1/markdown" \ -H "X-RapidAPI-Key: YOUR_RAPIDAPI_KEY" \ -H "X-RapidAPI-Host: distill-html-to-markdown-api.p.rapidapi.com" \ -H "Content-Type: application/json" \ -d @page.json # {"html": "<html>…</html>", "base_url": "https://blog.acme.example/release-2"} ``` page.json → html (made-up blog post) ``` <html lang="en"> <head> <title>Release 2.0 | Acme Blog</title> <meta name="author" content="Ana Lopes"> <meta property="article:published_time" content="2026-10-01"> </head> <body>

We use cookies. <button>Accept</button>

Release 2.0

Imports are now twice as fast. See the upgrade guide.

Upgrade

Stop the worker
Install 2.0

pip install acme==2.0

</body> </html> ``` Response · 200 (real) ``` { "url": "https://blog.acme.example/release-2", "final_url": null, "status": "ok", "detail": null, "http_status": null, "title": "Release 2.0", "author": "Ana Lopes", "published_date": "2026-10-01", "language": "en", "site_name": "Acme Blog", "description": null, "markdown": "# Release 2.0\n\nImports are now **twice as fast**. See the [upgrade guide](https://blog.acme.example/docs/upgrade).\n\n## Upgrade\n\n1. Stop the worker\n2. Install 2.0\n\n```bash\npip install acme==2.0\n```", "word_count": 28, "char_count": 196, "approx_tokens": 49, "truncated": false, "links": [ { "text": "upgrade guide", "url": "https://blog.acme.example/docs/upgrade" } ], "cached": false, "took_ms": 108 } ``` ## Response fields Every 200 response has all of these fields; unknown values are `null`. | field | meaning | |---|---| | url | The URL you asked for, normalised; base_url (or null) for raw HTML. | | final_url | URL after redirects; null for raw HTML. | | status | ok, no_content, blocked, robots_txt, opted_out, not_html, http_error or unreachable (see below). | | detail | Plain-English reason when status isn't ok, e.g. robots.txt disallows this path for *. | | http_status | HTTP code of the site's last response (null when there wasn't one). | | title, author, site_name, description | From the page's metadata (Open Graph, meta tags, JSON-LD) and <title>. | | published_date | YYYY-MM-DD from page metadata only; no guessing from the text. Often null. | | language | From <html lang> or Content-Language, normalised like en-US. | | markdown | The main content as GitHub-flavoured Markdown. Empty unless status is ok. | | word_count, char_count | Of the returned Markdown (link URLs aren't counted as words). | | approx_tokens | char_count / 4, rounded up. An estimate: real tokenizers differ by model and language. | | truncated | true when max_chars cut the Markdown. | | links | [{text, url}] for every http(s) link in the main content, absolute and de-duplicated; null when include_links=false. | | cached | true when served from the 1-hour cache. | | took_ms | Time spent on our side. | ## Statuses A page we can't or may not read is still a **200**. Branch on `status`; `markdown` is empty for everything except `ok`, even if the refusal page had text. Refusals are cached for 10 minutes; `unreachable` isn't cached. | status | when | |---|---| | ok | Content extracted. | | no_content | The page loaded but has no main content we could find, often because it's built by JavaScript. | | blocked | The site's bot protection refused us: HTTP 401, 403 or 429, or a 202 with an empty body (AWS WAF challenge). We don't retry or disguise ourselves. | | robots_txt | robots.txt disallows the path for DistillBot, for *, or for a named AI crawler; or robots.txt answered with a server error (RFC 9309: assume disallowed). | | opted_out | The page has a noai directive (X-Robots-Tag, <meta name="robots"> or <meta name="distillbot">). Applies to raw HTML too. | | not_html | The URL serves something else: PDF, image, JSON… | | http_error | 404, 500 and other errors, or more than 5 redirects. http_status has the code. | | unreachable | DNS failure, connection error, a timeout (10 s per request, 20 s in total), or a host that resolves to a private, loopback or link-local address. | robots_txt · 200 (real) ``` { "url": "https://docs.python.org/2.7/library/json.html", "final_url": "https://docs.python.org/2.7/library/json.html", "status": "robots_txt", "detail": "robots.txt disallows this path for *", "http_status": null, "title": null, "author": null, "published_date": null, "language": null, "site_name": null, "description": null, "markdown": "", "word_count": 0, "char_count": 0, "approx_tokens": 0, "truncated": false, "links": null, "cached": false, "took_ms": 0 } ``` unreachable: localtest.me resolves to 127.0.0.1 (real, trimmed) ``` { "url": "http://localtest.me/", "final_url": "http://localtest.me/", "status": "unreachable", "detail": "could not connect to the site", "http_status": null, "markdown": "", "word_count": 0, "char_count": 0, "approx_tokens": 0, "truncated": false, "links": null, "cached": false, "took_ms": 2 } ``` ## What is kept - **Kept:** headings (`#` to `######`), paragraphs, bold/italic, inline code, ordered and bullet lists, blockquotes, GFM tables, links, images (opt-in), and fenced code blocks with the language from `language-*`, `lang-*`, Sphinx `highlight-*` or `data-lang`. - **Dropped:** navigation, menus, headers and footers, sidebars, cookie and newsletter banners, ads, comments, scripts and styles, heading permalinks (¶) and `[1]`-style footnote markers. - Links and images are absolute, resolved against the final URL, `<base href>` or `base_url`. - Pages are read up to 5 MB. JavaScript isn't run. - Extraction is heuristic (trafilatura). On very short pages it can miss a headline or a link; on unusual layouts it can keep a stray line. ## Errors | code | when | body | |---|---|---| | 401 | Missing or invalid credentials. | `{"detail":"Unauthorized"}` | | 422 | Bad input: missing url, not http(s), an IP address, a port other than 80/443, both or neither of url/html, empty or oversized html, max_chars out of range. | `{"detail":"…"}` | | 429 | This site was fetched 10 times in the last minute (all users together), or your RapidAPI plan's rate or quota was reached. | Retry-After: 60, or RapidAPI's body + x-ratelimit-* headers | Bad URL ``` HTTP/1.1 422 Unprocessable Content { "detail": "url must use a hostname, not an IP address" } ``` Bad POST body ``` HTTP/1.1 422 Unprocessable Content { "detail": [ { "loc": ["body"], "msg": "send exactly one of url or html", "type": "value_error" } ] } ``` Per-site limit ``` HTTP/1.1 429 Too Many Requests Retry-After: 60 { "detail": "example.com was fetched too often in the last minute; try again shortly" } ``` ## Rate limits and plan quotas From the [pricing page](https://distill.hybridforgelab.com/subscribe.html): | Plan | Price | Includes | |---|---|---| | Basic | $0 / month | 15 requests / mo; 1 req/s; Hard limit, no overage | | Pro | $9 / month | 10,000 requests / mo; 5 req/s; + $0.002 per extra request | | Ultra | $29 / month | 50,000 requests / mo; 10 req/s; + $0.001 per extra request | | Mega | $99 / month | 250,000 requests / mo; 20 req/s; + $0.0006 per extra request | ## Links - Live demo: https://distill.hybridforgelab.com/#try - Integration guide: https://distill.hybridforgelab.com/integration.html - Full API reference: https://distill.hybridforgelab.com/api.html - Contact: hybridforgelab@gmail.com