Performance¶

Status: measurements. Written 2026-08-03.

Written for anyone choosing settings on their own hardware and wanting the measured cost rather than a guess. Assumed: CONFIG.md for what each setting does. Not covered here: the parallelism design behind these numbers, which is PARALLELISM.md, written for someone changing it.

What each setting in CONFIG.md costs, measured rather than estimated. CONFIG.md says what a setting does and what values it takes; this says what it costs, so neither document has to carry both jobs.

Read the numbers with the machine in mind¶

Every figure below is one machine's, and yours will differ. They are here to give you ratios and orders of magnitude -- "OCR roughly halves throughput", "the parse is CPU-bound, not GPU-bound" -- not absolute times to plan against. Where a figure only makes sense against the hardware, the hardware is named.

Two reference machines are used throughout. This is their full specification; README.md has the sizing guidance that follows from it.

Name used below	What it is
the small machine	4 cores, 9.7 GB RAM (~3 GB actually free), no GPU
the multi-GPU machine	96 logical cores (48 available to the process), 251 GB RAM, 4x NVIDIA A40 46 GB, driver 555.42.02, CUDA 12.5. Verified 2026-07-30

The corpus is this project's own bibliography: 501 PDFs, 13,400 pages, 1.54 GB, median 16 pages, with one 675-page book that is 5% of all pages by itself. Software: docling 2.117.0, torch 2.7.1+cu126, Python 3.12.3.

Reproduce any of it with the harness in bench/ -- see bench/README.md.

Install-time costs and traps¶

Two costs land before any setting below matters, and both are paid at install time. Neither is a knob you tune -- they are the two ways the install comes out wrong.

No GPU, disk tight -- several GB of CUDA you will never use. pip/Poetry's default torch wheel pulls a full set of nvidia-* CUDA packages whether or not a GPU is present. That is most of what makes the venv 6.0 GB. On a CPU-only host, install torch from the CPU-only wheel index before running the installer:

.venv-full/bin/pip install torch --index-url https://download.pytorch.org/whl/cpu
bash scripts/install_full_pipeline.sh python-deps

GPU present, but torch.cuda.is_available() is False -- silently CPU-only. This is the failure mode that costs the entire 4.70x below while looking like a working install. scripts/install_full_pipeline.sh's ensure_gpu_torch exists to catch it: it reads the driver's supported CUDA ceiling from nvidia-smi and reinstalls torch from a matching CUDA-tagged wheel index. It runs on every python-deps/dev-deps install, is idempotent, and is safe to re-run by hand. Check with:

.venv-full/bin/python -c "import torch; print(torch.__version__, torch.cuda.is_available())"

It was verified end to end on the multi-GPU machine, whose driver caps at CUDA 12.5 while the Poetry-resolved wheel wanted CUDA 13 -- exactly the mismatch that runs CPU-only without complaining. If it still reports False after a python-deps install, the driver may predate every wheel tag the script knows; that function's own comments have the manual fallback.

`[parser].backend` -- pdftotext or docling¶

Measured on 5 real bibliography PDFs, cold (no caching -- pdf_text.py does not cache, so these are extraction times, not sync's steady state, which skips PDFs whose bytes have not changed).

Backend	Total, 5 PDFs	Words extracted	Ratio
`pdftotext`	1.43s	68,888	1x
`docling` (OCR on, i.e. before OCR defaulted off)	60.77s	69,565	~42x

Per document the ratio ranged ~18x-102x, tracking document length loosely at best -- so budget against the total, not the best case.

Fidelity is close. docling's word counts stay within ~3.5% of pdftotext's on every one of the five. The reason to pick docling is structure (reading order, sections, tables), not word recovery -- and the reason to pick pdftotext is speed plus page boundaries, which docling's output does not have. PDF-PARSER.md has the full comparison, including the two backends that were evaluated and removed.

That ~42x figure is enough to choose a backend and useless for planning a run, which is what the rest of this document is for.

`[parser].ocr` -- the largest single lever, and a trade¶

OCR's cost is not a single number. It grows with worker count, because OCR is CPU-bound and therefore competes with the parallelism you added. Measured 2026-08-04, end to end over the whole 501-PDF corpus:

Workers	OCR off	OCR on	Cost of OCR
1	3330.4s	6941.4s	2.08x
12	310.2s	1213.9s	3.91x
24	237.6s	1139.0s	4.79x

Equivalently, from the other side -- turning OCR on roughly halves how well the pipeline parallelises:

	Speedup, 1 -> 24 workers
OCR off	14.02x
OCR on	6.09x

At 24 workers with OCR on, 93% of the available CPU is busy -- the one configuration measured where this machine is genuinely full. docling's OCR runs on the CPU (RapidOCR on onnxruntime), which is why.

And it cannot be moved to the GPU by configuration alone, which is worth knowing before you go looking for the setting. Two things are in the way, either of which is enough on its own:

The onnxruntime wheel this project installs is the CPU build. onnxruntime.get_available_providers() returns ['AzureExecutionProvider', 'CPUExecutionProvider'] -- no CUDAExecutionProvider to select.
docling's RapidOcrModel sets use_cuda on the paddle and torch engine configs but not on the onnxruntime one, so the default backend never asks for CUDA even where it is available.

So [parser].ocr = true is a CPU cost that the device a worker is given does not touch. Measured here on one PDF, one worker, cuda:0: 5.31s with OCR on against 1.13s with it off -- OCR is 79% of the wall clock, all of it on the CPU.

That 79% is also the size of the prize, and it is not small. Getting OCR onto a card would mean RapidOcrOptions(backend="torch") -- the one backend docling wires use_cuda into -- plus a config key to select it. Do not size that work against the 1.79x above: that figure is what the GPU is worth while OCR stays on the CPU, not a ceiling on moving OCR itself. The stages that already run on a GPU go 4.70x faster there (above), and OCR is the larger share of an OCR-enabled run, so the plausible gain is a good deal more than 1.79x. Measure it before believing any particular number, including this reasoning.

An earlier figure of 2.46x appears in older documents and in bench/RESULTS.md. It came from a 16-PDF serial sample and is a reasonable estimate of the serial cost (measured: 2.08x); it is not the cost you will pay on a parallel run.

It is not free, and this is the part to read twice. OCR only runs on bitmap regions, so what it recovers is text stored in the PDF as an image rather than as characters. Turning it off changed the extracted text of 8 of those 16 documents:

Mostly publisher furniture (IEEEAccess, DTU Library) and figure sub-captions. Losing that is an improvement.
But one document lost 10.1% of its characters, including two complete tables embedded as images, and another lost a paragraph of body prose set in a graphical text box.

So false suits born-digital papers. Set true for scans, since with OCR off docling extracts almost nothing from one, or where tables-as-images matter more than parse time. The parse-quality guard will not catch a wrong choice here: it looks for run-together words, not for content that never arrived.

GPU vs CPU -- it depends entirely on OCR¶

With OCR on, 1.79x:

	s/page	Extrapolated to the corpus
One process, one GPU, OCR on	0.43	~1.6 hours
One process, CPU only, OCR on	1.37	~5.1 hours
Like for like, same 6 PDFs		1.79x

During that run the GPU averaged ~7% SM utilisation and 1.7 GB of 46 GB, while the process held ~300% CPU -- three of the 48 available cores.

With OCR off -- the default -- 4.70x, measured 2026-08-05 over 100 documents / 2,529 pages, serial, one process, converters warmed so model loading is excluded, the same PDFs through both devices:

Documents	Pages	GPU	CPU	Aggregate
10	75	0.222 s/page	0.975 s/page	4.39x
25	394	0.174 s/page	0.948 s/page	5.43x
50	798	0.211 s/page	0.985 s/page	4.67x
100	2,529	0.205 s/page	0.965 s/page	4.70x

Per document: median 4.31x, quartiles 3.46x and 6.56x, range 1.74x to 17.35x. Only one document of the hundred came in under 2x. The benefit grows with document size -- 3.49x aggregate under 10 pages against 5.78x at 30 pages or more -- because layout and table inference are the stages that scale with page count while the fixed per-document costs do not.

The two figures are not in conflict, and the difference is the point. OCR runs on the CPU either way, so it adds the same seconds to both sides of the comparison and drags the ratio toward 1. It is not that the GPU does less when OCR is on; it is that the run contains much more work the GPU cannot touch. Read 1.79x as "what a GPU is worth on an OCR run" and 4.70x as "what it is worth on a default run" -- and note that the default is the one most people will measure.

docling is still CPU-bound overall (PDF backend, layout post-processing, and OCR when enabled), one worker still leaves ~93% of a card idle, and the numbers below are still why the parallelism work went after CPU-level document concurrency first. A GPU is worth having; it is not what makes a corpus parse fast.

`[parser].workers` -- document-level parallelism¶

The largest lever on a multi-core machine, and the code currently caps it well below where the curve flattens.

Measured 2026-08-04 with the real python -m src.corpus sync over the whole 501-PDF corpus (13,400 pages), docling, OCR off, 4 GPUs. Each row is one run from an empty ledger; all reported 501 parsed, 0 failed:

Workers	Wall clock	Speedup	Efficiency
1	3330.4s	1.00x	--
4	799.2s	4.17x	104%
8	428.6s	7.77x	97%
12	310.2s	10.74x	89%	<- the most `worker_ceiling()` allows
16	268.1s	12.42x	78%
24	235.0s	14.17x	59%	median of 3
32	223.4s	14.91x	47%	median of 3
48	221.4s	15.04x	31%	median of 3

Scaling holds far better than previously documented. 97% at 8 workers, 89% at 12. The 104% at 4 is not an error: one worker does not saturate the threads it is given, so serial is a slightly unfair denominator.
The knee is somewhere past 24, not at 12. worker_ceiling() caps at allowed_cpus // 4, which is 12 here. Running 32 is ~1.4x faster -- available today only by changing that constant.
The curve plateaus from 32 to 48; it does not reverse. Medians of three runs put them 0.9% apart (223.4s vs 221.4s), while the spread within the 32-worker configuration alone was 86.8s. An earlier single-run pass had 32 beating 48 and read that as a knee; with repeats the ordering flips. Anything past ~32 is flat, and single runs cannot resolve it.
So the useful statement is "the divisor should be much smaller than 4", not "it should be 1.5". Any specific value here is arithmetic from one machine and one corpus, on a plateau, and a CPU-only machine would likely want a different one.
Asking for 16 on this machine resolves to 12 and takes 315.9s. The clamp is doing exactly what it documents, and costing 1.43x.

Why the cap is too low: the constant models a worker as occupying 4 CPUs. It does not. Measured CPU busy, against the 48 available:

Run	CPUs busy	of the 48 allowed
16 workers	18.7	39%
32 workers	~34	~70%
24 workers, OCR on	44.6	93%

At 32 workers -- well past the point the code will go -- the CPU is still only ~70% busy. With OCR off a worker uses closer to one CPU than four. (The 32-worker figure landed at 71% in one sweep and 70% in another; a run-to-run point is worth about a percentage point, not a decimal.)

These CPU figures are host-wide. sweep_sync.py samples /proc/stat, which counts every process on the machine, and expresses it against the 48 CPUs this process may use. On an otherwise-idle machine that is the run; on a busy one it is an upper bound on what the run used, and can exceed 100%. Treat them as "the machine was this busy", not "the parse used this much" -- and note that the conclusion below rests on the trend across configurations, not the absolute level.

What flattens the curve past ~24 workers¶

Timing each run's phases separates the candidates:

Workers	Startup (to 1st document)	Tail (after last)	CPU busy
24	18.6s — 7.9% of the run	4.9s — 2.1%	56%
32	21.8s — 8.9%	5.9s — 2.6%	70%
48	28.5s — 12.7%	7.9s — 3.6%	78%

Startup is a growing tax, not a fixed one. Every worker pays its own ~8.5s model load, so standing the pool up costs more the bigger the pool: 7.9% of the run at 24 workers, 12.7% at 48. (The column is time to the first completion, so it also contains the fastest document's parse -- an upper bound on startup rather than a measurement of it. The growth is the startup part: one document's parse does not get slower because the pool got bigger.)
The CPU is heading for saturation, 56% to 78% across the same range. Read alone, "70% busy at 32" suggests headroom; read against 56% at 24 and 78% at 48, it is becoming the limit.
The long-document tail is not the story — 5-8s throughout, under 4%.

Neither cost alone explains the plateau; together they account for it, and both worsen with every worker added.

Corpus size still decides whether raising this is worth anything. Over 8 documents, 4 workers gave 1.90x and 8 gave none at all -- 34.6s / 18.3s / 19.3s. Each worker pays its own ~8.5s model load, so the benefit is proportional to how much work there is to amortise it over. That is why the resolved count is also capped by the number of documents needing a parse.

Multi-GPU -- nothing to configure¶

With docling and more than one worker, each worker process claims one CUDA device round-robin. This is not automatic in docling: its AcceleratorDevice.AUTO resolves to cuda:0 in every process, so without an explicit per-worker device every worker piles onto card 0 while the rest idle.

Measured over the whole 501-PDF corpus (2026-08-04, OCR off):

Workers	1 GPU	2 GPUs	4 GPUs	1->2	2->4
12	518.4s	339.7s	310.2s	1.53x	1.10x
24	535.8s	298.6s	237.6s	1.79x	1.26x

The second card is worth far more than the third and fourth.
GPUs matter more at higher worker counts, since more workers share each card.
At 24 workers, one GPU is slower than at 12 (535.8s vs 518.4s): piling more workers onto a single card is counterproductive.

The 2026-08-02 run of the same 12-worker configuration measured 528.0s and 326.2s -- within 2-5% of the figures above, on a different day and a rebuilt venv.

Restrict which cards are used with CUDA_VISIBLE_DEVICES; there is no separate setting, and the pool only ever sees what that leaves visible.

And it is worth nothing on a small corpus. The same change measured over a 60-document subset showed no difference at all (122.4s at 4 workers, 123.0s at 12), with all four GPUs busy and the CPU ~85% idle. Per-worker startup dominates at that size.

`[parser].start_method` -- per-worker startup¶

A cold docling worker needs about 8.5s before it produces its first page on the multi-GPU machine:

Stage	Time
`import torch`	1.16s
`import docling`	2.08s
Build the `DocumentConverter`	0.13s
First `convert()` -- docling loads its models here	5.17s
Total before the first parsed page	8.5s
A later `convert()`, models warm	0.33s

Only the ~3.2s of imports can be shared between processes; the ~5s model load lives on the converter instance, in whichever process built it. So forkserver -- which imports torch and docling once in a helper process that every worker is forked from -- can address at most that 3.2s.

And sharing the import, on its own, is worth nothing. Workers import concurrently, so on a host with spare CPUs that cost was already overlapped. Measured head to head over 8 documents at 4 workers: spawn 22.9s, forkserver 22.4s.

The saving is in when the preload runs. sync starts the forkserver before reading the bibliography, so the import happens during the ~2.5s that takes rather than blocking pool construction afterwards -- four live workers ready at 4.40s instead of 6.90s.

End to end on the real sync, medians of three runs, fresh output directory each time:

Documents	Workers	`spawn`	`forkserver`	Saving
8	1 (serial)	46.2s	--	--
8	4	23.1s	21.8s	1.3s (5.6%)
8	8	22.9s	20.7s	2.2s (9.6%)
60	1 (serial)	383.2s	--	--
60	4	103.6s	101.9s	1.7s (1.6%)
60	12	80.8s	78.8s	2.0s (2.5%)

Run-to-run spread was 0.3-1.0s, so the effect clears the noise everywhere -- and it is the same effect everywhere: a roughly constant 1.3-2.2s off pool startup. What changes is how much of the run that is: 9.6% of an 8-document run, 2.5% of a 60-document one, well under 1% of the full corpus.

So this does not make a bulk parse meaningfully faster, and was never going to. It helps the case where startup is the run: a handful of documents.

`[parser].document_timeout` -- what a safe value looks like¶

Not a performance knob so much as a knob whose value has to be chosen from performance. Any threshold has to clear the slowest document you legitimately have. In this corpus that is a 675-page book which takes 246s on its own, so a value that is safe here may not be safe on a corpus with a longer document. Measure before setting it.

[parser].stall_timeout's 1800s default is 7x that figure, chosen loose on purpose: it is meant to catch a run that will never finish, not to police a slow one.

`[enrich].docling_images` -- disk, and a full re-parse¶

Two costs, both worth knowing before turning it on:

It invalidates the whole docling cache, so the next run re-parses every PDF from scratch. That re-parse is the point rather than a bug -- the existing .md files genuinely have no figure references in them.
The PNGs are real disk: a 17-page paper produced 13 of them. docling_image_scale = 2.0 is roughly 144 DPI, enough to read a figure back without storing print-resolution files.

`[source_pdfs] dir` -- retired¶

Nothing left to measure: the enrichment corpus is now the bibliography alone, so there is no second source to deduplicate against. A config file still naming the key is ignored. src/enrich/corpus.py has the reasoning, and the retired duplicate-check timings stand in bench/RESULTS.md.

Where it all ended up¶

Measured end to end on 2026-08-04, rather than extrapolated:

Change	Kind	Full 501-PDF corpus
Baseline: serial, OCR on	--	1h 56m
OCR off	not parallelism	55m 30s
12 workers, 4 GPUs (today's cap)	CPU + GPU	5m 10s
32 workers, 4 GPUs (needs the clamp raised)	CPU	3m 43s

22x with the shipped defaults, 31x with the clamp raised. Earlier editions of this table read ~1.6 h -> ~39 min -> 8.8 min -> 5m26s; the first two were extrapolations that ran low, so the improvement was understated.

22x with the shipped defaults, and the GPU work is the smallest contribution. The largest is a boolean. PARALLELISM.md describes the machinery that produces these numbers; bench/RESULTS.md carries the measurements themselves, including the conclusions later ones overturned.

What a drift sweep costs¶

Everything above is the corpus layer: sync and the enrichment stages, where a run is measured in minutes. python -m src.draft dossier status --all sits in the drafting layer and is measured in seconds. It is worth pricing here for one reason: it is meant to be run after every sync, so "cheap enough to be habitual" is a requirement rather than a nicety.

The sweep builds a BM25 index in memory and discards it, rather than calling src.retrieval.search() -- which would take a write connection to the ledger and rewrite content/retrieval_index.json every time an inspection ran. DRAFT-ITERATION.md is the argument; this is the price.

Multi-GPU machine, bare python (no venv -- src.dossier is stdlib-only), no GPU involved. Medians of 5 runs over this project's own corpus: 646 ledger rows, 47.4 MB of parsed text. The drift scan never opens a PDF, so what it costs depends on the parsed text and the row count, not on the PDFs behind them.

Dossiers swept	Cold (no index cache)	Warm (cache from a prior `search()`)
1	2.032s	0.218s
10	2.036s	0.257s
50	2.227s	0.436s

Fifty dossiers cost 0.19s more than one. The corpus tokenization is paid once for the whole sweep, not once per dossier -- at ~4ms marginal cost each, the sweep is dominated by corpus size and effectively indifferent to how many drafts you have. Dossiers that logged no retrieval queries never build an index at all: 0.040s for 50.

That marginal cost is measured within a run, where the two numbers share a process and a page cache. Cold time drifts ~5% between runs, so differencing two separate runs of this table would not reproduce it; three runs each produce a paired delta of 0.19-0.24s. bench/RESULTS.md has the spread.

Cold cost is linear in corpus size (a generated 2000-document corpus: 5.857s), and a warm cache is 5.1-9.3x faster than a cold one. So the honest summary is not "free" but "about two seconds after a sync, and it does not grow with your drafts". bench/RESULTS.md has the full run, the synthetic scaling cross-check, and what the measurement excludes.

What raising the worker count costs in reproducibility¶

Worth pricing alongside the speedups above, because it is the one cost of parallelism not measured in seconds: the more workers, the less reproducible the parse. Docling groups dense reference blocks into elements differently under contention. That changes both the parsed text and the passage spans quoted from it.

At the scale this matters: comparing runs over all 501 documents, ~1.4% of documents come back with different text and ~1.0% with a different quotable passage. Two runs of the same configuration are not exempt, at roughly a third of that rate. Serial parsing has not been observed to vary.

The full contract is ARCHITECTURE.md's "What is reproducible, and what is not", artifact by artifact, and bench/RESULTS.md's 2026-08-07 section has the measurement. Both are stated once, there.

Two corrections to figures that once stood in this document, kept because the retracted versions were quoted elsewhere. The earlier "6 files differ, under 0.06%" understated the effect by counting bytes rather than passages. And "repeating a run at the same worker count reproduces exactly" is false: 5 of 572 same-configuration comparisons differ in bytes, and 2 in passage text.

Performance¶

Read the numbers with the machine in mind¶

Install-time costs and traps¶

[parser].backend -- pdftotext or docling¶

[parser].ocr -- the largest single lever, and a trade¶