Measured against Citoid on random samples: same coverage, nothing invented

An endpoint that turns a URL into a citation sounds like a solved problem. It is not: half the sources a student would actually cite refuse to answer a server at all. Here is which half, how fast the other half answers, and the number that decides whether the answers can be trusted.

2 August 2026 · 18 works drawn at random, plus a thematic run on 12 sources · raw data

What was measured

The extract_citation tool on this site's MCP endpoint is given nothing but an address. It fetches the page, reads whatever citation metadata the page declares about itself — Highwire citation_* tags, Dublin Core, PRISM, schema.org, OpenGraph — and returns a structured record with RIS and BibTeX. No citation service is queried, so nothing learns what is being read.

The comparison that decides it

A hit rate on its own says nothing. Citoid — the Wikimedia service that turns a URL into a citation, built on the Zotero translators — was given the same 26 addresses in the same order.

This endpointCitoid
Complete citations1111
Any record returned1114
References built from a block page02
Median seconds0.681.92
Slowest single call10.029.2
100 %
accuracy, ours
79 %
accuracy, Citoid
61 %
coverage, both

The earlier measurement was wrong, and how

A first run used 26 sources chosen by hand and reported parity. Nine works drawn at random from OpenAlex reported 4 correct against Citoid’s 7. The list had been assembled by the person doing the measuring, from sources already known to work. A list assembled that way measures the list. Everything below uses random draws.

The same coverage, nothing invented, in a third of the time. Citoid returns a record more often — twenty against thirteen — but that is where the second row matters: two of those extra records are references to a bot wall. For EconStor and SSOAR, Citoid returned Making sure you’re not a bot! as the title of the work. Formatted, complete-looking, and worthless.

That is the whole argument for refusing to answer. A reference that looks right and is not costs more than a gap does: the gap is visible while you write, the wrong entry is discovered by whoever marks the work.

It runs both ways. Citoid reaches three publishers that block us — MDPI, PeerJ and the OECD — and those are real losses, not rounding. We reach two it misses, Zenodo and Wikipedia. On coverage the two are close; on what they do when coverage fails, they are not.

By field

FieldReturnedSources that answeredSources that refused
Medizin2/3PMC, BMC Public H.BMJ Open (http-403)
Psychologie2/3PLOS ONE, Springer ASBFrontiers (http-404)
Informatik1/2arXivPeerJ CS (http-403)
Biologie1/1bioRxiv
Naturwiss.1/2Sci ReportsMDPI (http-403)
Wirtschaft0/2EconStor (block-page), OECD (http-403)
Sozialwiss.0/2SSOAR (block-page), GESIS (http-403)
Geisteswiss.0/1Persée (http-404)
Recht2/2EUR-Lex DSGVO, RIS StGB
Statistik2/2Statistik AT, Destatis
Repositorium1/2ZenodoOSF (block-page)
Presse0/1Guardian (http-404)
Enzyklopädie1/1Wikipedia DE
Verlag/Wall0/2ScienceDirect (http-403), Wiley (http-403)

The number that matters more than the hit rate

Zero false alarms. Every one of the 13 refusals was re-fetched from an unrelated network with a browser user-agent. All thirteen were real: same 403, same 404, same bot wall. And in the other direction, all 13 records that were returned carried every required field — authors, year, container, identifier — and were classified into the right kind of source, from journal article through preprint to legal text.

This matters more than coverage because of what the failure mode would be. A reader that invents a reference from an error page produces a bibliography entry that looks correct and is not. Half an answer is usable; a confident wrong answer is not.

Speed

Seconds
Endpoint alone, no page fetched0.15
Median, all calls0.35
Median, successful lookup0.74
Median, refusal0.22
95th percentile3.46
Slowest single call12.52

Roughly six tenths of a second is the fetch; the rest is the endpoint. A refusal costs a third of a success, because a blocked page is small. The slowest call — 12.52 s, an Austrian consolidated statute — is a page of several hundred kilobytes, and the time is spent transferring it, not parsing it. Two runs of the same address differed by up to 176 %: this measures the target site, not the endpoint.

Two things that went wrong while measuring

The first run returned HTTP 403 for all 26 sources, each in 0.13 s — far too fast for a real fetch, which is how it was caught. The cause was the measuring script, not the pipeline: Python's urllib sends Python-urllib/3.x, and Cloudflare's bot protection rejects exactly that signature. requests, node-fetch, Go's client, a browser and an MCP client all pass. Worth knowing for anyone connecting to this endpoint from a script.

The second was subtler. Fetched pages were cached at the edge for five minutes. A bot wall answers with HTTP 200, so it cached like any page — and a freshly deployed detection for bot walls then ran against the stored response and could not take effect at all. Caching for this tool is now off. A citation is worth an extra fetch.

What this says about the two ways to save a source

One source in this set makes the point cleanly. SSOAR, a German social-science repository, serves the real page to a desktop browser and a bot wall to Cloudflare worker addresses. The endpoint cannot read it. A browser extension running in the reader's own session can, because it is not a third party — it is the reader.

So the two are not competitors. The endpoint is fast, needs no installation and works for open sources; it fails at exactly the publishers whose material sits behind a login a student already has. The extension covers those and cannot be called from a script. Anyone building a citation workflow needs both, and should know which one is answering.

Questions

Why not resolve the DOI through Crossref when the page refuses?

It would raise coverage. It would also tell Crossref which paper is being read, every time. For the sources measured here it would have added nothing to the thirteen that worked — they already declare the full record — and the thirteen that failed mostly failed before a DOI was ever found. The trade was not worth it.

Is 50 % good?

It is the honest number for a server-side reader in August 2026, and it will get worse rather than better as publishers tighten bot defences. Which is the argument for reading a page where the reader already is, rather than asking a server to do it from somewhere else.