proofpath

proofpath v0.4.12

Don't guess.
Show the evidence.

proofpath checks whether the sources behind a claim actually say what the claim says. Point it at a paper or a draft; it verifies every citation on three levels and shows you the passage behind each verdict.

uv tool install proofpath

Runs locally and free. No API key, no Docker, no server. Windows, Linux, macOS.

A large raven, hunched on cobblestones, staring straight at the viewer. Etching.
Félix Bracquemond, Le Corbeau, 1854. The raven that stares back.

A run, as it happens.

proofpath — draft-live.md

Three questions, asked in order.

Most tools stop at the first. The third is the point.

Does the source exist?

A raven in flight, wings spread, cropped from Doré's Nevermore plate.

Huginn takes the reference as written and asks five indexes whether they have ever seen it.

A reference is resolved against Crossref, Semantic Scholar, arXiv, Open Library and OpenAlex. A DOI that resolves, an arXiv id that matches, a title and author list that line up: that is a RESOLVED record.

Several plausible records and no clear winner is AMBIGUOUS, and every candidate is listed. Only a reference that none of the indexes has seen is called a GHOST REFERENCE — and even then, the report names the indexes it asked before saying so.

asksCrossrefSemantic ScholararXivOpen LibraryOpenAlex

RESOLVEDAMBIGUOUSGHOST REFERENCE

A raven perched on a marble bust of Pallas, a halo of faces behind it. Wood engraving.
L41[7] Marchetti, L. R., Osei, K. & Lind…GHOST REFERENCE

arxiv and openlibrary consulted before the ghost call

Is it still valid?

A raven in flight, wings spread, cropped from Doré's Nevermore plate.

Huginn checks whether the record has since been retracted, corrected or withdrawn.

A paper that exists can still have been pulled. Every resolved record is checked against the Retraction Watch data that Crossref carries, and against OpenAlex.

A retraction is its own finding, not a footnote: RETRACTED is reported on the line where the citation sits, with the notice that says why.

asksCrossref (Retraction Watch data)OpenAlex

VALIDRETRACTED

A tall double door standing ajar, a figure half visible in the gap, light pouring through. Wood engraving.
L18[3] Wakefield, A. J. et al. Ileal-lymphoid…RETRACTED

retraction notice 2010-02-02, The Lancet, via Crossref

An illustrative line; the run above had no retractions.

Does it support the claim?

A raven gliding beneath the moon, cropped from Doré's frontispiece to The Raven.

Muninn reads the source, finds the passage closest to the sentence you wrote, and brings it back — word for word.

The sentence that carries the citation is compared with the text of the source itself. The nearest passages are retrieved, a numeric rule runs first — if the claim says 21 days and the source says 3.5, no model is asked — and an entailment model judges the rest, on your machine.

SUPPORTED and NOT SUPPORTED always show the passage they rest on. When the source neither supports nor contradicts, the finding is NEI, and it says so instead of guessing. Most tools stop at step 1. This step is the point.

asksfull text or abstractretrievala numeric rulean entailment model

SUPPORTEDNOT SUPPORTEDNEILOW CONFIDENCE (abstract only)

A man reading at a table by lamplight, a child asleep against him. Wood engraving.
L6[1] Vaswani, A., Shazeer, N., Parmar,…NOT SUPPORTEDhigh

numeric mismatch: claim says 21 days, source says 3.5 days

you“On the WMT 2014 English-to-French translation task the Transformer reached…”

source“On the WMT 2014 English-to-French translation task, our model establishes …” ⧉

What it reads.

Most of it needs no key at all. What cannot be read is said out loud, never skipped.

Reads it

No key, no account.

  • Your filesPDF, Word (.docx), Markdown, .txt
  • PapersDOI, arXiv, books
  • Web pagesNews, blogs, Wikipedia, any page
  • PostsBluesky, Hacker News, Lobste.rs, Mastodon, Lemmy

Needs one thing from you

Free, set up once.

  • Reddita free Reddit app of your own
  • LLM second opiniona free Groq key, or Ollama with no key
  • Sites that block robotsyour OK for a 280 MB browser, asked first
  • Claims with no sourcesearched on the web: a free Tavily key, or your own SearXNG

Can't read, and says so

Never skipped quietly.

  • Scanned PDFsno OCR yet
  • Old formats.doc, .odt, .rtf: save as .docx or PDF
  • Paywallsreported as unread
  • X / Twitterpaste the text instead

Six things it will never do.

These are not preferences. Breaking one breaks the reason the tool exists.

A raven flying out from behind a heavy curtain that a figure is pulling aside; a banner reads Nevermore. Wood engraving.
Gustave Doré, Nevermore, 1884. The raven leaves; the curtain stays.
A raven gliding beneath the moon, cropped from Doré's frontispiece to The Raven.

Muninn, back with nothing. He reports an empty beak, not an empty world.

  1. No passage, no verdict.

    A SUPPORTED or NOT SUPPORTED verdict always carries the quoted sentence it rests on. If the passage cannot be shown, the verdict is not given.

    Muninn shows you what he read.

  2. Absence of evidence is not evidence of absence.

    Unreachable, blocked, paywalled and ambiguous are separate, reported states. None of them is quietly folded into a verdict. The full list is below.

    A raven that comes back empty says "nothing found", not "nothing exists".

  3. A real reference is never called a ghost.

    When the indexes disagree or the match is uncertain, the finding is AMBIGUOUS. GHOST REFERENCE is reserved for a record no index has ever seen. The false-ghost rate is a release gate.

    Huginn doubts before he accuses.

  4. No prompt without a terminal.

    In CI or behind a pipe, a permission set to ask is treated as deny, and the report says so. A job never hangs waiting for a human who is not there.

    Piped means quiet.

  5. Nothing large is installed without asking.

    The browser fetcher and the GPU extra are opt-in. The base install stays small enough to put in a pipeline without thinking about it.

    The base install is the whole install.

  6. Every report states its own coverage.

    How much was read as full text, how much as abstract only, how much not at all — and why. A run that read little cannot look like a run that read everything.

    A clean report on 40% coverage is not a clean report.

    62% full text21% abstract only17% unverifiedblocked: 3credentials missing: 1

    coverage is weak: unread sources may hold more, so this is a lower bound

rule two, spelled out: every one of these is printed as itself, never folded into a verdict

LOW CONFIDENCE (abstract only)
full text unavailable, abstract used
UNVERIFIED (blocked)
403 or bot protection
UNVERIFIED (blocked, robots.txt)
the site's robots.txt disallows the fetch
UNVERIFIED (blocked, browser not permitted)
steps 1–2 blocked and the browser consent was denied or impossible
UNVERIFIED (unreachable)
dead link, Wayback miss
UNVERIFIED (reached, no text extracted)
200 answered, nothing readable came back
UNVERIFIED (network not permitted)
permissions.network = deny
UNVERIFIED (provider unavailable)
API down, rate limited after backoff, or answering with a page instead of a record
UNVERIFIED (credentials missing)
the platform reads only with a credential this machine lacks (Reddit: REDDIT_CLIENT_ID, REDDIT_CLIENT_SECRET) — nobody was asked, so it is not unreachable
UNVERIFIED (not in bibliographic indexes)
web page, blog, report — the indexes do not cover it, so absence proves nothing
AMBIGUOUS
several plausible records, all listed
NEI
the source was read and neither supports nor contradicts
PARAGRAPH-SCOPED
the citation covers a paragraph; each sentence is judged separately
FOUND BY PROOFPATH (not cited by the author)
evidence search (experimental): proofpath found this page itself, for a text that cited nothing; the verdict still quotes its passage
NO EVIDENCE FOUND (searched)
the search ran and no page it found confirmed or contradicted the claim; never REFUTED
UNVERIFIED (search unavailable)
the search provider did not answer
UNVERIFIED (language not supported)
the claim is not in English and no judge translated it; the models read English only

A post is not a source.

What it checks is whether the links inside the post back what it says. The post's own words never count as evidence for themselves.

A social post asserts something and links to where it came from. What proofpath checks is whether the links inside it back what it says: every sentence of the post against the pages it points to, and the post's own words are never allowed to stand as their own evidence. Any other address is a page, read as the document itself: fetched up the same ladder as its sources, cut into paragraphs, each sentence checked against the pages that paragraph links to. A page that prints a reference list is paired by citation number, like a paper.

proofpath check --url https://bsky.app/profile/bsky.app/post/3movpwtbjgs2d # a post: every link inside it is a source
proofpath check --url https://news.ycombinator.com/item?id=8863 # a story's URL, and the links in a comment
proofpath check --url https://lemmy.world/post/1 # the same on Lobste.rs and the Lemmy instances it knows
proofpath check - # paste the text of a post that cannot be read
proofpath check --url https://en.wikipedia.org/wiki/AlphaFold # any other address is a page, read as the document itself
Blueskyreadpublic.api.bsky.app, no account. Link cards, rich-text links and one level of quoted post.
Hacker NewsreadThe official Firebase API, no account. A story's URL and the links in a comment.
Lobste.rsreadThe .json twin of every story and comment page, no account. A story's URL, the links in its text and in a comment.
Redditcredentials missingWith a free app you register yourself: REDDIT_CLIENT_ID and REDDIT_CLIENT_SECRET in .env. Without them no request is made: the post is UNVERIFIED (credentials missing), and the run names both variables.
MastodonblockedBest effort, per instance. An instance that wants a login says so, and that answer is reported as UNVERIFIED (blocked) — not as a missing post.
LemmyreadBest effort, per instance, through the /api/v3 every instance serves without a login. Known by its host — lemmy.* and the big instances — never by /post/<n> alone, which is any blog's address.
Xpaste the textCannot be read at all. The run says so and asks you to paste the text; the links inside it are then verified as usual, and with evidence search set up a text with no link is searched (experimental).
A crowd of winged figures gathered close around a man slumped in his chair, a skeleton at his feet. Wood engraving.
Gustave Doré, The Raven, 1884. The room fills with voices. None of them is the source.

A platform that will not answer is its own state. No request made is not a request that failed.

Measured, not claimed.

No number is quoted without the run that produced it. The row in the box is the one to read before pointing this at the news.

  1. 0.0 %false-ghost rate

    Reference resolution

    hand-built ghost set, 274 references99.1 % ghost recall

    the run

  2. 0.609accuracy · 0.597 macro-F1

    Retrieval and entailment

    SciFact dev, 340 pairsagainst a 0.406 trivial baseline

    the run

  3. 0.697macro-F1 · 0.697 accuracy

    Retrieval and entailment, accurate model (opt-in)

    SciFact dev, 340 pairs · 643 MB, two passages per claimagainst 0.580 macro-F1 for the default model in the same run, with cuts fitted on SciFact train

    dev both chose this model and scores it, so 0.697 is optimistic · not better on news: AVeriTeC readable 0.475 against 0.537 for the default model at k=2 — for scientific sources, not news

    the run

  4. 135 / 340pairs above the decision threshold

    Most verdicts are NEI

    SciFact dev · NEI is "not enough information"the other 60 % are reported as NEI, not guessed

    tiers read off the same sweep: decide 0.45 · medium 0.458 · high 0.999 (21 in-sample verdicts) — in practice two tiers: near-certain, and asserted at all

    the run

  5. 0.99numeric · 0.940 author-year

    Citation pairing

    61 and 55 hand-built passages

    the run

  6. 1.35 ssecond run · 79.3 s cold

    Speed

    1-page draft, 7 references, Apple Siliconthe second run asks the network for nothing; a 19-page arXiv PDF with 68 references: 12 m 57 s, then 3 m 42 s

    the run

  7. 72 %full text

    Source access

    50 DOIsa real biomedical paper in the live runs reached 33 % full text

    72% full text18% abstract only10% unverified
    the run
A small bird high over an open sea, above a sphinx and a child on a cliff. Wood-engraved vignette.
Gustave Doré, The Raven, 1884. One bird over the open sea.

0.3713-way accuracy

End to end on real web claims

AVeriTeC dev, 100 claims, browser allowedagainst a 0.708 majority baseline — still worse than always guessing "refuted"

Coverage is no longer the loss: 165 of 200 sources were read. 36 of 89 claims had a readable source and still came back NEI, and 11 went the wrong way. The models were calibrated on scientific abstracts; a fact-check page is a different object. With evidence search and the judge writing the queries it scores 0.483 (14 claims fixed, 4 broken, p = 0.031), the one gain beyond the noise — but 25 of 89 claims come back the wrong way there, 20 of them false claims called supported (likely cause, not yet measured: search finds pages that repeat the claim).

Pointed at a news claim on the open web, it is a coverage report with a weak verdict attached. Read this row first.

the run

Where it stops, today.

A tool that measures its own coverage also has to say what it does not read yet.

  1. Superscript citations only when the PDF marks them.

    A PDF that draws them as ordinary digits loses them, and km² can be read as [2].

  2. Three sentences is not a source.

    When only an abstract is reachable the verdict is labelled LOW CONFIDENCE (abstract only) and counts as a finding.

  3. An unnumbered two-column bibliography is cut at line breaks.

    RoBERTa's ACL list came out as 103 entries for about 50, half with no author, so author-year pairing resolved 15 claims where a rejoined list would give 65. The largest open item.

  4. Without a References heading, only the last block is read.

    A paged document is read by its shape — the last contiguous run of numbered paragraphs — which recovers that block alone (AlphaFold: 17 of 84).

  5. The 0 % false-ghost rate is paid for in recall.

    A fabricated entry with no year between the names and the title is reported as not in bibliographic indexes, not as a ghost, and one proceedings-volume record accepted a fabricated paper cited into it (0.9 %).

    the run

  6. A cold run of a long bibliography is serial.

    The providers are asked one reference at a time (129 references: minutes). Only the TUI runs several documents’ network stages at once.

  7. A refused browser install is reported, not hidden.

    The source is UNVERIFIED (blocked, browser not permitted) with the install log, never silently counted as unreachable.

  8. Coverage is not perfectly reproducible.

    Two runs minutes apart can read a different number of sources, depending on which providers answered.

The same engine behind a pipe.

A report, a JSON document, a SARIF log — and an exit code you can gate on.

uv tool install proofpath

Python 3.10+. The first run downloads about 250 MB of ONNX models once; the [gpu] extra addstorch for CUDA, MPS and CPU, in that order. Nothing else is installed without asking.

proofpath check paper.pdf # report → report.md, exit 0/1/2
proofpath check draft.md --format json | jq '.coverage' # machine-readable
proofpath check draft.md --format sarif --out draft.sarif # any SARIF 2.1.0 viewer
proofpath -q check - < draft.md # stdin; findings and coverage only

the exit code is the interface — no text parsing needed to gate a job

0clean
1findings — every UNVERIFIED and LOW CONFIDENCE counts
2the run itself failed

--format sarif writes a SARIF 2.1.0 log: one result per finding on its line, the quoted passage in the message, the exact honesty state inproperties, and the run's coverage in the run's properties, so a log opened without the terminal still says how much was read. Any SARIF viewer reads it, including VS Code's.

Piped or in CI there is no prompt: a permission set to ask is treated asdeny and reported. A job never waits on a question nobody will answer.

A veiled figure with a scythe seated on the moon among clouds, a raven flying beneath. Wood engraving.
Gustave Doré, frontispiece to The Raven, 1884.

Everything it read is in one file.

One plain SQLite file, no server, no extension. Raw publisher text expires after seven days; a verdict keeps the passage it quotes; a provider outage is never stored. Open proofpath cache path in DB Browser for SQLite, TablePlus or DBeaver — plain tables. proofpath resolve REF and proofpath fetch URL|DOI run either half on its own.

proofpath cache # where it is, what it holds
proofpath cache ls # sources, chunk/verdict counts, text expiry
proofpath cache show <id> # one source's chunks and verdicts
proofpath cache clear --expired

since v0.2 the resolution, the retraction check, the text, the chunks and the verdicts are all kept — the 1.35 s in the ledger is the second run of the draft in the demo

resolutions
30 days
retraction hit
30 days
retraction miss
7 days
fetched text
7 days
chunks, verdicts
while the text is unchanged

A second opinion, if you ask for one.

Everything above runs on your machine, and the default run makes zero LLM calls. Two flags add a model at the end, and only there:--judge asks it about the low-confidence verdicts alone, and its answer is printed beside the local verdict, never in place of it. --summarize is one call over the finished report, labelled model-written. If the provider hits its limit or stops answering, the run switches at once to a local qwen3.5:9b, if it is installed, and says so; the next run tries the provider first again. The SARIF log never carries either.

proofpath check paper.pdf --judge # a second opinion on the low-confidence verdicts
proofpath check paper.pdf --summarize # one model-written paragraph over the finished report
proofpath config check # proves the key works before you spend a run on it

from the live run of 2026-09-15: the local verdict stands, the judge's line sits under it

error[not-supported]: claim is not supported by the cited source  (confidence: low)
   = judge (groq openai/gpt-oss-120b): NEI — Passage states "SciPy provides fundamental algorithms for scientific computing" but does not mention SciPy building on anything.