Changelog

Unreleased

Changed

• Deliberate states no longer cost points (scoring model 3). noindex, nofollow, none, nosnippet and noimageindex pages, pages canonicalised to another page, mismatched canonicals, single internal redirects and repeated H2 headings are now Notes: still reported, never scored. A site with a few intentional noindex pages was told to undo them and lost up to a few points. Repeated H2s are often the template's own section headings, such as "Frequently asked questions", so change one only if it is meant to be unique to the page. Scores of crawls stored earlier are not recomputed; raptor_compare warns when it compares crawls scored by different models.
• Every finding in crawl.json carries its id, copy, priority and rank. Each issue gains check_id (a stable id such as content.exact_duplicate), title, why, fix, priority, area and rank, and summary gains findings_by_priority and ranking_model. The new fields are additive, and crawls stored earlier still load.
• description and guidance keep their meanings, but their words now come from the check catalogue. description says what Raptor found, and guidance is why it matters followed by how to fix it.
• AI Crawlers Blocked names the crawlers in its description. The blocked crawlers' robots.txt names now appear in the finding's description and in culprits, and no longer in guidance.
• A crawl stopped early says so. crawl_metadata.interrupted appears, set to true, only on a crawl that was stopped before it finished, and the pages fetched up to then are analysed. The crawler also reports which phase it is in (start URL, robots.txt, pages, sitemaps, analysis) as it runs.
• docs/checks.md is generated from the check catalogue. It lists every check by area, including the five directive checks the old page left out, each with its id, priority, and plain what, why and how. scoring.md now states the ranking rule and the impact-to-priority table.

• New HTML report. The report opens on what to fix: the grade and one plain sentence, then Fix first, up to six cards, each with what Raptor found, why it matters, how to fix it and an example page. Below that are every finding in one list (filter by priority, reach, area or text), every crawled page in a sortable table with a side panel for each page, a site structure tree, and search across findings and pages (Ctrl-K or ⌘K). Links such as #finding=… and #page=… open the report on one finding or page. It works in light and dark, on a phone, with scripts off (the summary and every finding still read), and it makes no network requests: its own Content-Security-Policy lets only its own scripts run. An area whose checks did not all run says how many were not checked instead of "Clear", and a printed report lists what was not checked and why. See docs/report.md.
• Every crawled page is in the report. The 2,000-page cap is gone: a 20,000-page crawl makes a report of about 6 MB that scrolls smoothly. Each finding lists every affected page; only the per-page details stop at 100, and the report says when they do. Copy URLs and Download CSV include every address.
• Report bugs fixed. The report opened on whichever check came first in Raptor's list ("URL"); it now opens on the top fix. Paths kept only the part before ?, so eight /blog?category=… pages showed as one; each keeps its query string. Why and how to fix showed the wrong text for some checks. Clicking a URL in a finding now opens that page's details. Paths on sites served from a non-default port no longer carry the host and port.
• Charts that said nothing are gone. The score gauge, donut and bar charts are replaced by counts in findings, the "How is this calculated?" breakdown and plain sentences such as "All 125 pages returned 200 OK."
• Copy fix brief and CSV export. Each finding copies a short brief (what, why, how and the affected URLs) to paste into a ticket or a message to a developer; Copy all fixes copies the whole list. Affected pages download as CSV, with cells that would start a spreadsheet formula made safe. Save as PDF prints the summary.
• Smaller. The stored 125-page crawl that made a 1,352,521-byte report now makes one of 491,582 bytes, 36% of the old size, while carrying every page.

• New terminal output. raptor crawl leads with the grade and a Fix first list of the five most important fixes, in plain words with their reach, points, an example page and a stable check id, then the files it wrote and what to do next. In a terminal it opens with the /// header, shows one live progress region (phase, counts, status tallies, rate, the page in flight) and collapses each finished phase into one line. Through a pipe, in CI or for an AI agent it prints plain key: value lines instead: no colour, no \r, full URLs and absolute paths, verdict first and files last. --format rich|plain and --color auto|always|never choose, and NO_COLOR, FORCE_COLOR and CLICOLOR are honoured. The old Health, Response and Issues block is gone. See Output.
• --json prints a small JSON summary of a crawl (raptor.crawl-summary/1) or an inspect (raptor.inspect-summary/1) on stdout, with the exit code inside it.
• Exit codes mean something. 3 now also covers a crawl that ran but could not be graded (a bot wall, or an app shell with no readable content), 4 means a quality gate failed, and 130 means you pressed Ctrl-C. See Exit codes.
• Quality gates. --min-score N and --fail-on fix-now|high|medium|low fail a crawl with exit 4, so CI can fail a build when a site gets worse.
• Ctrl-C saves what was crawled. The first press stops the crawl, analyses the pages fetched so far, writes the files and exits 130; a press during --pagespeed skips the tests, and one that lands once the crawl is being analysed or saved lets it finish whole; a second press quits at once.
• raptor inspect speaks like a crawl. It lists the page's findings most important first, in the catalogue's words with Notes last, says first when the page is gone, and exits 3 when the page cannot be reached.
• A home screen. raptor on its own shows the commands to start with and your last five crawls, read from $RAPTOR_HOME/history.jsonl, instead of a box of server jobs. It never touches the jobs database, and in a pipe it prints the help.
• serve --help no longer prints the API key that RAPTOR_API_KEY or $RAPTOR_HOME/.env holds.
• -v now means Raptor's own debug log; -vv logs every library, and -q prints only errors. RUST_LOG is honoured.
• Errors read the same everywhere: one error: line, its cause and a hint. raptor crawl example.com says to add https:// (exit 2) instead of failing mid-crawl.
• Removed: raptor version's component lines and emoji; it prints the version and the build's features.

• MCP tools now answer in plain words. Every tool's text block is a written digest: the verdict first, then the ranked fixes in the check catalogue's words, capped lists with their true totals, the crawl's limits and the next call to make. It replaces the one-line key=value pointer, so a host that passes the model only text no longer leaves it guessing. raptor_summary, raptor_findings and raptor_crawl carry the same text in structuredContent.digest, and raptor_summary adds fix_first, findings_by_priority, limits, report_path and more beside its existing keys. Clients older than protocol 2025-06-18 still get the JSON as text. See docs/mcp.md.
• raptor_findings lists every finding on a crawl, ranked, filterable by priority and area, and with a check_id returns one finding in full with a markdown fix brief ready to paste into a task.
• raptor_cancel replaces raptor_status {action: "cancel"}, and it really stops a running crawl. A cancelled crawl stops fetching, and the pages it already fetched are analysed and saved, so raptor_summary and raptor_findings work on it and say it was interrupted. While those pages are saved, raptor_status reports stopping: true. A crawl another Raptor process runs cannot be stopped from here; until that process saves it, raptor_status reports saving_elsewhere: true and says no pages are saved yet. The old call now returns "cancel moved to raptor_cancel: call raptor_cancel with this job_id", and raptor_status is read-only. REST DELETE /api/v1/jobs/{id} stops a running crawl the same way.
• Every tool has a title and read-only and destructive hints, so hosts no longer prompt on every status poll or treat a crawl as destructive.
• Panels in hosts that support MCP Apps. In Claude Desktop, VS Code, Cursor and Goose, raptor_summary shows a site health card, raptor_findings a findings list that expands to a filterable table with fix briefs, raptor_inspect a page check, and raptor_crawl live crawl progress. Each is a self-contained page at ui://raptor/v{version}/{panel}.html that makes no network request and follows the host's theme. Claude Code shows the same facts as text.
• Live progress in raptor_status. A running job's progress adds status tallies, failed fetches, the current URL, the phase in flight in the CLI's words, and a bar that never promises more than the page limit. A finished job's result_summary adds its verdict, what to fix and how it ended, and a failed job's holds the reason.
• Job errors say what happened. A failed job says why it failed instead of asking you to wait for it, a running one says to poll raptor_status, and a job cancelled before it saved anything says so.
• raptor_compare says when a crawl is partial. Comparing against a crawl that was cancelled or stopped at its page limit sets comparable: false and names the partial side, since findings on pages it never fetched would otherwise read as fixed or new.
• raptor_inspect reports the true size of a page's title, description and H1 list in title_chars, meta_description_chars and h1_count, and the page panel reads its own capped view, page_panel. The existing title, meta_description and h1s keep their full values.
• raptor_query titles and counts match the report. Each issue gains check_id, title, count, unit, priority and priority_label, the same title and shown count the report, the CLI and raptor_findings print; affected_page_count and affected_url_count keep their meanings.
• raptor_screenshot is listed only in builds that can run it, and raptor_inspect offers its rendered and compare modes only there too.
• MCP mode's stderr is plain text, with no colour codes and no REST API-key warning.
• Reports served by the REST API load their embedded fonts (font-src data: in the report's security policy; scripts stay blocked).

Added

• Duplicate Content names the copies, and AI Crawlers Blocked lists the crawlers. A Duplicate Content row said only "Exact duplicate content (3 pages share hash 7fc3c72348cb)", a fragment of a content hash nobody can look up. Each row now reads "Same content as" followed by the other pages with the same content, and its culprits list names them too, up to 10 (the detail says how many more there are). AI Crawlers Blocked in robots.txt now lists the blocked crawlers' robots.txt names (such as GPTBot) in the finding's culprits; its guidance text still names them as well. See docs/checks.md and Robots / AI.
• JavaScript findings name their scripts, hosts and limits. Render-blocking scripts, Excessive JavaScript and Heavy third-party scripts said only how many scripts a page had ("1 render-blocking script(s)", "22 script(s), 0 KB inline"). Each row's detail now names them, and so does its new culprits list: the render-blocking scripts, the external scripts on a page over the 20-script limit, and the third-party hosts. Script addresses are resolved against the page, so /assets/app.js reads the same on every page that loads it, while pages[].scripts[].src keeps the address as written. Excessive JavaScript also says which limit tripped. Over the script limit it splits the count into external and inline scripts ("39 scripts, 14 external and 25 inline (limit 20)"), so the listed scripts add up to the external figure. Over the inline limit it gives the size of the largest block and whether it sits in <head> or <body>. A row lists at most 10 culprits, and a culprit longer than 300 bytes is cut short. See docs/checks.md.
• raptor_compare says when two crawls are not like-for-like. Every comparison reports scoring_model ({baseline, current}) and comparable. When the crawls were scored by different models, such as a crawl stored before this release (model 1) against a new one (model 2), comparability_warning says the score and issue changes include changes in how Raptor measures, not only changes to the site, so model changes are not presented as fixes. See docs/mcp.md.
• raptor_summary reports findings_count and pages_with_findings, and raptor_query reports affected_page_count and culprits. findings_count counts the checks that fired, pages_with_findings the distinct crawled pages they list, and affected_page_count the distinct URLs among one finding's URL rows (each URL once; not resolved through redirects). All three are computed from the crawl itself, so a crawl stored before this release reports them too, except that pages_with_findings reads null for a stored crawl with no page data. issue_breakdown and affected_url_count keep counting URL rows. culprits appears on an issue, or on one of its affected_urls[], only when the finding names offending values. See docs/mcp.md.
• The crawl summary records findings_count, pages_with_findings and scoring_model. findings_count counts the checks that fired and pages_with_findings the distinct crawled pages they list, beside total_issues, which still counts URL rows. scoring_model names the analysis-and-scoring model that produced the score: 2 from this release, 1 for a stored crawl without the field, and 0 for a summary that was never scored. See docs/scoring.md.

Fixed

• Next.js page data no longer counts as JavaScript. Next.js App Router sites carry each page's data in dozens of small inline self.__next_f.push(…) blocks. Excessive JavaScript counted each one as a script and added its size toward the inline limit, so every page of ravanasolutions.com read "68 scripts" for 14 real ones, and the finding led the site's report on all 125 pages. These blocks are now left out of both limits, which is expected to clear the finding from that site. Ordinary inline scripts still count. Each script in pages[].scripts[] records is_data_payload (present only when true), and the CSV export's script_count column still counts every script. See docs/checks.md.

• Next.js fill images are no longer reported as missing dimensions. Missing Image Dimensions listed every image without width and height, including the ones Next.js writes for <Image fill>. Those are positioned absolute, so they cannot push content around when they load, and fill mode cannot take a width and height, so the advice could not be followed. On ravanasolutions.com 86 of the 87 pages listed were there only because of them. An image positioned absolute or fixed in its own style attribute is now skipped, which is expected to leave that site with 1 page, whose SVG figures really have no size. Each image in pages[].images[] records is_out_of_flow (present only when true). See docs/checks.md.

• An encoded space in a query value is no longer a URL with spaces. URLs with Spaces matched %20 anywhere in the URL, so /blog?category=Web%20Development, where %20 is the normal way to write a space in a query value, was reported as a warning. The check now looks at the path only: /spaced%20page.html is still reported. See docs/checks.md.

• A site served from a local host is no longer flagged for linking to itself. Every page of a site served from localhost or 127.0.0.1, such as a development server being audited, was listed under Localhost Links for its own internal links. Localhost Links now skips pages that are themselves served from a local host. A page on a public host that links to localhost is still reported. See docs/checks.md.

• A site served from a local host is no longer judged on HTTPS. Every page of a development server on localhost or a loopback address, served over plain http, was reported under HTTP Pages (Not Secure), a page defect that cut the health score sharply, although no one can get a certificate for 127.0.0.1. HTTP Pages (Not Secure) now skips pages served from a local host, so a clean site audited locally can grade A. A public page served over http is still reported. See docs/checks.md.

• Next.js polyfills are no longer reported as render-blocking scripts. Next.js loads its polyfills with <script src=… noModule> in <head>, a fallback that current browsers skip without downloading, yet every page of every Next.js site was listed under Render-blocking scripts. A nomodule script is no longer render-blocking, and each script in pages[].scripts[] records is_nomodule (present only when true). See docs/checks.md.

• Third-party scripts are classified by where the browser loads them from. A protocol-relative src (//cdn.example.net/x.js) was counted as first-party, so a page loading 11 vendor scripts that way never raised Heavy third-party scripts. A src is now resolved against the page's <base href> when it has one, as a browser resolves it, so a relative src under a CDN <base> is third-party, and so is a script on the page's own host at another port. pages[].scripts[].src still holds the address as written.

• The CSV's render_blocking_scripts column now matches the Render-blocking scripts check. It also counted inline scripts, scripts in <body> and nomodule polyfills, so one page read 3 in the spreadsheet and 1 in the report. Both use the same rule now.

• raptor_compare no longer merges checks that share a name across categories. Findings were matched by check name and page, and duplicate is both a Page Titles and a Meta Description check, so a new duplicate description on a page that already had a duplicate title was not reported, and fixing both counted as one resolved issue credited to one category. Findings are now matched by category, check and page, and each regressions[] entry adds new_findings ({category, check} pairs) beside new_checks.

• raptor_query with category: "javascript" now finds the JavaScript checks. The schema advertised javascript, but the issues read java_script, so the filter matched nothing and an agent was told the site had no JavaScript findings.

• raptor_summary's worst_pages[].issue_count counts the findings on a page, not its URL rows. On a crawl stored before checks listed each URL once, Duplicate H2 could list one page several times, so a page with 10 findings read as 18, and the list ranked pages by how verbose one check was. Pages with the same count are now listed in URL order, so two calls on one crawl list the same pages; before, the order of tied pages could change from call to call.

• Duplicate H2 Headings and Non-Indexable in Sitemap now list each URL once (Duplicate H2 listed 307 rows for 113 pages). Scores count each page once. summary.total_issues and the severity totals keep counting URL rows, and now count each URL once per check.

• Pages canonicalised to another URL are no longer reported as duplicates (exact content, title, description, H1, H2). On ravanasolutions.com, all 13 rows of Duplicate Content, Duplicate Titles, Duplicate Descriptions and Duplicate H1 were canonicalised variants and their base pages, and those checks no longer fire there. Duplicate H2 Headings listed 113 pages; excluding canonicalised variants brings it to about 100 pages. A canonical that redirects back to its own page now counts as self when the crawl followed that redirect before it found a direct link to the page. A page reached through a redirect is now compared for duplicate content instead of being skipped.

• A page whose <head> contains invalid markup is no longer reported as having no title. Anything the HTML spec disallows in the head — a tracking pixel <img>, a stray text node, a <div> from a mis-templated partial — closes the head early, and the <title> after it is parsed into <body>. Title selection was scoped to head title, so those pages were reported as Missing Title while their markup plainly carried one. Found on a real site, where three pages were flagged and all three had titles. The title is now found wherever the parser put it, inline SVG accessibility labels are still excluded, and the misplaced title is reported as what it actually is. This also activates page_titles::outside_head, a check that could never fire before: it requires a title to be present in order to report it as misplaced, and the scoping guaranteed there wasn't one.

• raptor_summary reported affected_urls as a bare count while raptor_query uses that name for the list of affected URLs. One field name with two types across two tools in the same server read as an empty list to a caller that had seen the other, sending it looking for the URLs in the per-page data instead. The summary field is now affected_url_count, matching the name raptor_query already uses for the count beside its list.

• A seed that redirects no longer collapses the crawl to one page. Pages are recorded at the URL that finally served them, but the crawl scope was still established from the URL the user typed. Seeding https://www.example.com/ on a site that canonicalises to the apex — or http:// on a site that redirects to https:// — classified every discovered link as external, and the audit ended after one page with a passing grade, exit 0, and no warning. The seed's own redirect is now resolved first and the destination is treated as the site under audit.

• The page a seed redirects to is no longer dropped from the report. It deduped against a visited key seeded from its own destination, which removed the homepage and left everything it links to reported as an orphan.

• <template> contents are no longer counted as body text. The markup inside a <template> is inert — the browser never renders it — so counting it inflated word counts (a page with five visible words reported 21), pushed soft-404 pages past the length guard that suppresses the check, and polluted both the duplicate-content hash and the readability score.

• Links that redirect into a 404 are reported broken again. Page identity moved to the URL that finally served a page, but the link joins still compared against the URL as written in the href, so a link to /moved that 301s to a missing page matched nothing and was dropped. On a site whose only bad link was of that shape, the critical "Broken Internal Links" issue did not appear at all — the worst possible outcome on a recently-migrated site, which is exactly when an audit is ordered.

• Redirect destinations are no longer reported as orphan pages, and the link graph keeps those edges. A page linked only through a redirect was reported as having no inbound links, and raptor link printed the homepage as "(2 in / 0 out)" — inverted. The dropped edges corrupted total_edges, avg_out_degree and every PageRank value; on one fixture the graph went from 2 edges and 2 components to 8 edges and 1.

• The "3xx Redirects" list names the URL that redirects, not its destination. It reported the destination — a URL returning 200 — as the redirect, so the guidance told the user to repoint links at the address that was already correct, while the URL that actually needed changing appeared in no issue and no CSV column. Redirect chains now also print their final destination instead of ending in a dangling arrow, and a 4xx/5xx reached through a redirect names the link that has to change.

• The HTML report no longer prints impossible affected-URL counts. It summed issue rows instead of counting distinct URLs, so a 10-page crawl claimed "14 URLs affected" — and at client scale a 2,600-page crawl claimed 13,000. JSON, CSV, CLI and MCP were all correct; the HTML was the outlier.

• Site-level findings are no longer hidden from the HTML. Issues that legitimately have no URL list (no sitemap found, AI crawlers blocked) were filtered out, so the category rendered a green dot and "All checks passed — impressive!" while the severity bar beside it counted the issue. They now render with a Site-wide marker, in both the category panel and the Overview's top-issues list.

• 404, 500 and 403 pages are no longer labelled Indexable in the CSV, the HTML badge, the structure detail panel and raptor inspect, which printed "Indexable yes" directly above "Status 404". All four now use the same shared rule the sitemap generator uses, so the artifacts agree.

• raptor sitemap no longer writes a schema-invalid empty <urlset> and reports success. It called the unguarded single-file helper, so a crawl with no pages array (--summary-only) or nothing indexable produced a file xmllint --schema rejects and Search Console reads as empty — while printing ✓ Wrote … and exiting 0. It now uses the guarded generator, writes nothing when there is nothing to write, and exits non-zero saying which of the two causes applied. Crawls past the 50,000-URL protocol cap now emit a <sitemapindex> plus chunk files instead of one oversized file.

• URL fragments no longer become page identity. /docs/#toc was recorded, and emitted into <loc>, as a distinct page. It also produced false findings on the fragment's own text: "URL contains spaces" for a %20 after the #, a url_length longer than the URL, and "has parameters" on a page with no query string.

• googlebot-news no longer suppresses general Search checks. Google scopes that token to News, but a <meta name="googlebot-news" content="noindex"> marked the page non-indexable site-wide: it dropped out of the generated sitemap and silently stopped being eligible for around twenty gated checks, so duplicate titles involving it simply stopped firing. Scoped directives are now kept but bucketed separately, matching how X-Robots-Tag was already handled.

• <meta name="robots" content="none"> now stops link discovery. Google documents none as shorthand for noindex, nofollow, but only the literal nofollow token was matched, so the crawler kept following links the site had asked it not to.

• alt="" is no longer reported as a missing alt attribute. It is the spec-correct way to mark an image decorative, so flagging it told authors to undo the right thing and made screen-reader output worse if followed. Only a genuinely absent attribute now counts as missing, and the distinction survives into the JSON instead of being flattened to null.

• A second Raptor process no longer kills the first one's running jobs. Startup recovery marked every in-flight job failed with "Server restarted during execution" while the crawl was still going, then blocked the real completion — so completed_at landed 25 seconds before the crawl actually finished, next to a fully populated result summary. Attaching an MCP client to the default ~/.raptor, which is the documented Claude Desktop setup, did this to a running REST crawl, and two concurrent Claude sessions did it to each other. Jobs now carry an owner and a heartbeat, and are reclaimed only when their owner is genuinely gone. Trade-off: a crashed process's jobs are reclaimed after 120–180 seconds rather than instantly at the next startup.

• Crawl settings are no longer silently replaced by defaults. max_pages: 3 was honoured but max_pages: 3.0 was discarded — and 3.0 is valid against the tool's own published "type": "integer" schema. The same applied to delay_ms and concurrency, so an operator who asked for 3 pages at one request per second could get up to 500 pages at four per second against a client's production site. Integral floats are now accepted and anything genuinely out of range is rejected with a message naming the parameter.

• Unknown or misplaced arguments are rejected instead of ignored. REST took crawl settings under options, so {"type":"crawl","url":"…","max_pages":3} returned 201 and crawled to the 500-page default; MCP behaved the same way for a misspelled maxpages. Both now refuse the call and name the offending argument alongside the accepted set, which is derived from each tool's own schema so it cannot drift.

• Job types that cannot execute are refused. inspect, extract and screenshot were accepted with a 201 and then sat queued forever; because REST and MCP share one queue cap, fifty of them wedged job creation on both transports until the process was restarted.

• A rejected webhook_url no longer strands a queued job. The job row and its output directory were committed before validation, and the error path returned before the caller ever received an id — so each retry of a mistyped webhook burned a queue slot permanently. Validation now happens before anything is written, and user errors return 4xx with the reason instead of a retryable 500.

• MCP resources/read is bounded. See the note above; it previously returned the whole artifact in a single line.

• Webhooks fire on every terminal state. A cancelled job delivered nothing at all on the REST path, and on the MCP path delivered {"event":"job.completed","status":"cancelled"} — burning the one-shot on an event that contradicted itself. job.cancelled and job.queued were accepted at registration but emitted by nothing, and a job reclaimed after a crash left its one-shot stranded forever.

• An unreachable host is no longer reported as "blocked by robots.txt". robots.txt now fails closed only when it answers 5xx or 429. A robots.txt that cannot be fetched at all (DNS, refused, TLS, timeout) no longer blocks the crawl, so the start page decides whether the site is reachable. This deliberately departs from RFC 9309 §2.3.1.4, and the robots line says that Google would treat the site as blocked. A missing robots.txt is no longer a warning.

• A dead start URL no longer yields site findings, and the crawl JSON names the cause. A host that never answered still got sitemap discovery, so the report said "No XML sitemap found" about a site nothing was read from, and the JSON never said why nothing was crawled. Sitemap discovery now runs only when at least one page answered below HTTP 400, and crawl_metadata.seed_error records why the start page got no response (dns, tls, timeout, refused, …), with the first 100 failed fetches in crawl_metadata.fetch_errors.

• Per-URL failures name their cause once. A failed URL was logged as "Failed to fetch X: Failed to fetch X" and shown as "✗ X — Failed to fetch X", naming the URL twice and the cause never. Both now name the cause, as in "connection refused (…)", and sitemap fetch errors carry it the same way. The log line moves from WARN to INFO (--verbose), since the progress output already shows it.

• Stopping at --max-pages is recorded. A crawl that hit the page limit looked identical to one that finished, and the URLs it left behind were discarded uncounted. crawl_metadata now carries stopped_at_page_limit and urls_not_crawled, which counts only URLs the crawl would still have fetched (within --depth and allowed by robots.txt). A limit equal to the site's size reads as complete.

• An unreachable, robots-blocked or all-HTTP-error crawl no longer prints "✓ Crawl complete". A dead host printed "✓ Crawl complete — 0 pages", and a site that answered 403 to everything printed a summary reading "Health – (0/100)", both exiting 0. A ✗ line now names the cause in place of the summary (on stderr under -q), only the JSON is written, and the run exits 3. A report or CSV that an earlier run left at the same paths is removed, so it cannot pass for this run's audit.

• The page-limit stop is called by its name. A crawl that stopped at --max-pages with URLs left behind printed "✓ Crawl complete". It now prints "⚠ Stopped at the page limit" with the number of URLs found but not crawled.

• The robots line and per-URL error lines appear when output is piped. They were printed through the progress bar, which hides itself whenever stderr is not a terminal, so agents, CI and | tee never saw them. They now go straight to stderr when there is no bar.

• No ANSI codes go into pipes. The log wrote colour escape codes into piped stderr, corrupting logs and agent transcripts. Log colour now needs stderr to be a terminal and honours NO_COLOR, and the progress lines take their colour from stderr, not stdout.

• The JavaScript-rendering hint names a backend that exists. The CLI hint, the HTML report and the server's backend error told users to re-run with --backend cloudflare, which 0.3.0 removed. They now name --backend playwright, and the server and any build without that backend (the release binaries among them) also give the cargo build --release --features playwright command that provides it. -q no longer prints the hint.

• Server and MCP jobs that reach no page, or whose every page answered with an HTTP error, end failed with the cause, not completed. A crawl of a host that did not resolve was marked completed with grade "–", so raptor_status told an agent the audit had finished. The job's error now names the cause, such as "Could not reach … DNS lookup failed". A crawl that fails outright keeps the reason in error too, where it used to store only the outermost context ("Invalid seed URL: not a url", with the reason dropped).

• Tied categories in the crawl summary are listed in a stable order. Categories with the same critical, warning and affected-URL counts came out in a random order, so two runs of the same crawl could list a different top three. Ties are now broken by category name.

Security

• raptor serve silently ignored RAPTOR_API_KEY. The documented environment variable was never read by the CLI — only by the standalone raptor-server binary — so RAPTOR_API_KEY=… raptor serve started a fully unauthenticated server while the operator believed it was protected. --api-key worked and masked the gap.

Changed

• Reports can carry your agency's brand. raptor crawl --brand <FILE> reads a brand.toml with your agency name, logo, accent colour, contact line, voice and mascot, and passes it to the HTML report. RAPTOR_BRAND, or brand.toml in Raptor's home folder, does the same without the flag, and --no-brand turns it off. A file Raptor cannot use stops the crawl before it fetches anything, with exit code 1 and an error naming the file and the key. An accent too pale or too dark to read keeps Raptor green in that theme and prints a warning with the measured contrast. raptor serve reads the same file when it starts, and keeps the Raptor brand, rather than stopping, if the file is refused. See docs/report.md.
• Health scores move for the same site (scoring model 2). Pages canonicalised elsewhere no longer count as duplicates, coverage counts pages instead of rows, noModule polyfills are no longer render-blocking, an encoded space in a query value no longer counts as a space in the URL, pages served from a local host are no longer judged on HTTPS or listed under Localhost Links, absolutely positioned images are no longer missing dimensions, and Next.js page data no longer counts as JavaScript. Two changes can lower a score: a duplicate page reached through a redirect is now reported, and third-party scripts loaded by a protocol-relative src, through a CDN <base href> or from another port can now raise Heavy third-party scripts. Each change is described under Fixed above. On ravanasolutions.com the score moves from 83/B to 93/A, measured on the same 125 pages before and after these changes, except the last two, to Missing Image Dimensions and Excessive JavaScript, which were not re-measured and are expected to raise it further. Stored crawls are not re-scored. raptor_compare flags comparisons across models.
• Cloudflare is now opt-in. It reaches a third-party API and needs credentials, so it is a feature flag like playwright rather than part of every build. The default build contains no Cloudflare code at all. Two modules were misplaced under it and are now where they belong: static single-page inspect (which needs no rendering service) and the security-header HEAD requests (which use the local fetcher). Both consequently work in the default build for the first time, and their tests now run there too. --features cloudflare restores screenshot, extract, and inspect --rendered / --compare.
• The CLI and server previously depended on raptor-core without default-features = false, so the cloudflare feature was compiled in regardless of what was selected — a --no-default-features build still carried all of it while refusing to run the commands.
• raptor crawl exits 3 when there is nothing to grade (start URL unreachable, robots.txt disallows it, or every page answered with an HTTP error). Scripts that read exit 0 from such runs now get 3.

A second audit pass (see AUDIT.md) found two P0s and fourteen P1s; all are fixed.

Security

• robots.txt SSRF. robots.txt was fetched by its own HTTP client that neither disabled redirects nor re-applied the SSRF guard per hop, so a site whose /robots.txt returned a 302 to http://169.254.169.254/… pulled cloud metadata into crawl.json and the report. It now goes through the shared guarded Fetcher.
• DNS-rebinding TOCTOU. The guard resolved and vetted a host, then reqwest re-resolved it at connect time, so the address checked was never the address used. A GuardedResolver is now installed on the crawl and webhook clients, vetting the addresses the connector dials.
• The PageSpeed API key no longer reaches logs: it travels in the query string and reqwest's error Display includes the full URL, which was printed on any transport failure.
• Linux builds link rustls instead of system OpenSSL (the published artifact previously failed to start without a compatible libssl.so.3). CI actions are pinned by commit SHA, contents: write is scoped to the release job, and builds use --locked.

Changed

• MCP requests are now handled concurrently. A slow raptor_inspect or raptor_screenshot previously blocked ping and every other call for the duration. Responses may now arrive out of request order (legal JSON-RPC — the id correlates); each response is still written as one complete newline-delimited line, stdout stays pure JSON-RPC, and shutdown drains outstanding handlers rather than dropping their replies.
• Each analyzer check is now capped at 60% of its category's severity allowance before the category cap applies, so one high-frequency check can no longer mask every other check in its category.
• Health-score deductions are weighted by the share of the site affected. "15 pages are missing a meta description" previously cost the same on a 15-page site (every page broken) as on a 1000-page one (1.5% broken). A proportional term now reaches the full allowance only when the whole site is affected, while a bounded absolute term keeps a single serious finding from rounding to nothing at scale — one broken page costs the same whether the crawl is 10 pages or 100,000. Site-level findings keep their flat weight.
• notifications/cancelled now actually cancels. In-flight work is aborted, the cancelled request emits no response (per spec), initialize is never cancellable, and a cancellation for an unknown or already-completed id is a silent no-op.

Fixed

• --output <name> without a .json suffix destroyed the crawl JSON — the HTML report was written to the same path under default flags.
• A cancelled job could resurrect itself: the Running transition was unguarded, so cancelling a queued job was silently reversed and the full crawl ran anyway.
• --concurrency 0 hung forever (zero-permit semaphore); it is now rejected, and configs arriving from REST/MCP are sanitized.
• The visited set is claimed atomically, so a URL discovered by two pages at once is no longer crawled twice and double-counted.
• robots.txt is fetched with the seed's scheme — an http-only host previously failed the fetch and fell through to disallow-all, yielding a silent 0-page crawl.
• Response bodies stream against a byte budget rather than being fully buffered before the 10 MB cap is checked; fetch_with_redirects no longer issues one request past its cap.
• Analysis false positives: duplicate/length checks are gated to indexable HTML, so noindex pages no longer generate "Duplicate Titles"/"Duplicate Descriptions"/"Duplicate H1" against indexable ones (a page excluded from the index cannot cause a duplicate-content problem); hreflang and canonical joins normalize both sides; only executable scripts count toward "Excessive JavaScript" (JSON-LD and __NEXT_DATA__ no longer do); self-links no longer mask orphan pages; single-hop redirects are detected from the redirect chain; "No Sitemap Found" only fires when discovery actually ran; console errors are reported once; 401/403/429 are Warnings rather than Critical; X-Robots-Tag values such as max-snippet:50 keep their value.
• The grade verdict is composition-aware — an F with zero critical issues no longer reads "Critical".
• MCP/REST: page_data paginates against its own total (a filter matching no issues reported total_pages: 0 while pages existed); malformed JSON returns -32700 instead of hanging the client; batches are handled; the transport size limit binds while reading; the queue cap applies to MCP; REST validates backend and pre-checks targets as MCP already did.
• Sitemap data is retained on CrawlResult, so the server executor's unconditional re-pipeline (and the CLI's PageSpeed re-run) no longer discards sitemap coverage analysis from every REST/MCP crawl.
• The job queue cap is enforced in a single transaction rather than as a count-then-insert race, and a refused job no longer leaves an output directory behind.
• The SSRF guard decodes IPv4 tunnelled inside IPv6 (6to4, Teredo, NAT64), closing a path where a literal like 2002:7f00:1:: reached loopback through a relay.
• Duplicate-content grouping excludes noindex pages, matching the title/description/H1 checks.
• Report: the per-page issue map is built once instead of rescanned for every page (quadratic on large crawls); the structure tree is depth-capped against stack overflow; a 0-page crawl no longer renders a fabricated all-green chart; crawl_date is a checked prefix, not a panicking byte slice.

Removed

• crawl --cf-account-id/--cf-api-token, which have had no effect since 0.3.0; passing them is now a usage error (exit 2).