Configuration¶
Status: reference. Written 2026-08-03.
Written for anyone setting this pipeline up on their own machine. Assumed: CLI.md for the commands these settings change. Not covered here: what each setting costs, which is measured in PERFORMANCE.md, and why the defaults are what they are, which is DESIGN.md's job.
Every setting, what values it accepts, and what it defaults to.
What each setting costs lives in PERFORMANCE.md, so this document can stay a reference rather than an argument.
Table of contents¶
- How configuration is loaded
- A minimal config.toml
- Every setting
- Paths
[render]-- citation style[style]-- prose conformance and the acronym vocabulary[parser]-- PDF text extraction[logging]-- the pipeline log file[provenance]-- citation-support bands[enrich]-- the optional enrichment layer- How values are parsed
- Notes on individual settings
- Choosing an embedding model
How configuration is loaded¶
config.toml is not in the repository -- you create it, once:
1 | |
src/config.py reads it at import time and fails with that exact
command if it is missing, rather than silently falling back to the
example: a machine quietly running settings its owner never chose is a
worse failure than one that refuses to start.
- The file is read once, at import, into plain module-level
constants. They are fixed for the life of the process; editing
config.tomlmid-run changes nothing until the next run. - Every setting can be overridden per-run by an environment variable
of the same name, without editing the file:
BIB_FILE=/path/to/other.bib python -m src.corpus sync. The environment always wins. - Set
CONFIG_PATHto keep the file elsewhere:CONFIG_PATH=/etc/research/config.toml python -m src.corpus sync. - Every key is optional. Anything absent falls back to the default in the tables below, so your file only needs the settings you want to change.
config.toml.example is the tracked template, carrying the same settings
with longer commentary. This document is the authoritative list of
accepted values.
A minimal config.toml¶
Because every key is optional, the smallest valid file is an empty
file -- that runs the whole pipeline on defaults, expecting your
bibliography at papers/bibliography.bib.
The smallest useful file names the one thing that genuinely varies between machines:
1 2 3 | |
A realistic small config, opting into the slower/higher-fidelity parser and some parallelism:
1 2 3 4 5 6 | |
Every setting¶
Paths resolve relative to the repository root; an absolute path is used as given.
Paths¶
| Key | Env var | Accepts | Default |
|---|---|---|---|
[bib] path |
BIB_FILE |
path | papers/bibliography.bib |
[bib] collections_field |
BIB_COLLECTIONS_FIELD |
field name | groups |
[content] dir |
CONTENT_DIR |
path | content |
[bib] path-- the BibTeX exportsrc/bib_reader.pyparses. The only source of citekeys; nothing in the pipeline invents or renames one. Gitignored per-host data.[bib] collections_field-- which BibTeX field carries Zotero collection membership, read bysrc/bib_collections.pyand used bypython -m src.corpus ledger --collectionandpython -m src.draft retrieve search --collectionto scope a draft to a curated subset of the library.groupsis JabRef's field and what Better BibTeX writes under Export JabRef-specific fields. Zotero's own BibTeX exporter drops collections, so on a plain export the field is simply absent, nothing is in any collection, and no command changes behaviour -- see ZOTERO.md for how to keep them and what that costs. Lower-cased on read, since BibTeX field names are case-insensitive.[content] dir-- everything every layer writes:sync'sledger.sqliteandparsed/, the drafting layer'sdrafts/,dossiers/andrendered/, the review layer'sreview/, anddocling/,chroma/andtopics.jsonfrom the enrichment stages. One exception lives here too, hand-edited rather than pipeline-written:verbatim_allowlist.toml, the per-host boilerplate allowlistsrc.review verbatim scanconsults -- see PLAGIARISM.md. Fixed at<dir>/verbatim_allowlist.toml, not independently configurable, same as the enrichment caches below it. It is also what every tier-1 command that takes a path will accept:citation_gate,referencesandrender_outputeach refuse a path that resolves outside it, and so do all three ofsrc.review's aids.ledgeris the one that takes no path argument at all -- its CLI only ever addresses rows by citekey or status -- so the rule applies to it vacuously rather than needing a check. This one directory is then the whole record of the work, and a copy of it is complete.
The enrichment layer's artefacts keep the names above --
content/docling/, content/chroma/, content/topics.json, and every
DOCLING_* environment variable -- and are not derived from the [enrich]
table's name. Renaming them would invalidate work already on disk for no
conceptual gain.
There is no key for "extra PDFs to enrich": the enrichment layer indexes
the bibliography and nothing else, so everything it can retrieve is
something a draft may cite. To add a paper, catalogue it in your
reference manager, re-export, and re-run sync -- see
ZOTERO.md.
[render] -- citation style¶
Used only by src/render_output.py, never by sync or the
citation gate.
| Key | Env var | Accepts | Default |
|---|---|---|---|
csl |
CSL_STYLE |
path | assets/csl/ieee.csl |
collapse_citations |
RENDER_COLLAPSE_CITATIONS |
boolean | true |
csl-- the CSL style pandoc's--citeprocformats citations and the bibliography with. The IEEE style ships with this repo (vendored, not fetched: rendering has to work with no network, and a style that changed underneath a draft would renumber one already reviewed). Point it at any other.cslfile to use a different style; the CSL project publishes several thousand. Without this, pandoc falls back to Chicago author-date. Relative to the repo root here, as with every other path setting;render_output's--cslflag additionally accepts a path relative to the current directory, since that is what a shell-typed path means.collapse_citations-- whether a run of consecutive numbers collapses:[3]โ[6]rather than[3], [4], [5], [6]. The IEEE Reference Guide's own examples use the collapsed form, but upstreamieee.csldoes not produce it, sorender_output.pyinjects the one CSL attribute that does (collapse="citation-number") into a temp copy of the style. Setfalseto render whatever the style on disk says, unmodified. A style that already setscollapseitself is never overridden. Seeassets/csl/README.md.
[style] -- prose conformance and the acronym vocabulary¶
vale_config and language are used only by python -m src.draft
style, which is a review aid: it exits 0 whatever it finds, and
nothing in this pipeline blocks on it. acronyms, below, is the one key
in this section that command does not read -- it is read directly by the
five genre-writing skills at drafting time (docs/GENRE.md), not by
src.draft style.
| Key | Env var | Accepts | Default |
|---|---|---|---|
vale_config |
VALE_CONFIG |
path | assets/vale/vale.ini |
language |
STYLE_LANGUAGE |
BCP-47 tag | unset |
acronyms |
ACRONYMS |
path | assets/style/acronyms.toml |
vale_config-- the Vale configuration and rule package a draft's prose is checked against. Vendored for the same two reasonscslis: the check has to work with no network, and a rule set that changed underneath a draft would report a document that had already been reviewed. Point it at your own house style to override what ships;assets/vale/README.mddocuments each rule, which section of WRITING-STANDARDS.md it implements, and the word pairs it deliberately leaves out.- The
valebinary itself is not configured here -- it is looked up onPATH. Without it the command reports missing-binary and every other command is unaffected, the same bargainrendermakes with pandoc.bash scripts/install_full_pipeline.sh os-depsinstalls the pinned version. language-- a fallback dialect for a draft whose dossier records none. A fallback, never an override: thelanguage:line in a draft's ownscope.mdwins, because a thesis at an Indian university and an IEEE submission legitimately differ, and only the per-draft record knows which this is. The report names which source a dialect came from, so a draft checked against this one never reads like a draft that declared its own.- The order is:
--languageon the command line, then the draft'sscope.md, then this key. With none of the three set, the command measures the draft both ways and proposes a tag with thedossier set-languagecommand that would record it -- it never writes one itself. acronyms-- a genre skill's acronym vocabulary at step 0, read alongside the dialect.assets/style/acronyms.tomlis the vendored floor (PDF,CPU,URL,API,HTML) and always loads; point this at your own file to merge your field's vocabulary over it -- your definition wins if you redefine one of the vendored five, and every vendored entry you don't redefine still applies. Copyassets/style/acronyms.toml.exampletocontent/acronyms.toml(gitignored, per-host, the same footing asconfig.tomlitself) and point this key there -- not back atassets/style/acronyms.toml, which is the vendored file this one merges over, not a template to edit in place.assets/style/README.mdhas the file's shape and provenance.
python -m src.draft dossier acronyms-suggest <draft> proposes new
entries for it from a draft's glossary and its own prose (a term
coined and expanded inline but never glossaried is exactly the lapse
this catches) without writing anything; add
--apply to write the proposed entries to your file (creating it if
it doesn't exist yet), merged without duplicating what is already
there. --apply refuses if this key is unset, rather than writing
into the vendored floor. python -m src.draft style separately
reports when a draft's own glossary has drifted from the current
vocabulary (docs/WRITING-STANDARDS.md ยง9); draft-reviser's
acronym-realignment mode fixes what that reports.
[parser] -- PDF text extraction¶
| Key | Env var | Accepts | Default |
|---|---|---|---|
backend |
PARSER |
"pdftotext" | "docling" |
"pdftotext" |
ocr |
PARSER_OCR |
boolean | false |
workers |
PARSER_WORKERS |
positive integer, or "auto" |
1 |
start_method |
PARSER_START_METHOD |
"auto" | "forkserver" | "spawn" |
"auto" |
document_timeout |
PARSER_DOCUMENT_TIMEOUT |
positive number of seconds, or "off" |
"off" |
stall_timeout |
PARSER_STALL_TIMEOUT |
positive number of seconds, or "off" |
1800 |
long_word_chars |
PARSE_LONG_WORD_CHARS |
number, used as an integer | 20 |
long_word_ratio |
PARSE_LONG_WORD_RATIO |
number, a fraction 0.0-1.0 | 0.01 |
min_tokens |
PARSE_MIN_TOKENS |
number, used as an integer | 200 |
The values in full:
backend--"pdftotext"needs thepdftotextbinary onPATHand no Python package;"docling"needs theenrichdependency group. Any other value is rejected, naming the valid ones. See notes.ocr-- onlydoclinghas an OCR stage;pdftotextignores this.workers--1takes a strictly serial path: no pool, no subprocesses, nothing about a run changes. An integer above 1, or"auto", opts into a worker pool. The resolved count is clamped rather than obeyed blindly -- see notes.0, negative numbers, andtrue/falseare rejected at load.start_method-- consulted only whenworkers > 1andbackend = "docling"; nothing else here uses a process pool."auto"--forkserverwhere the platform has it (Linux, macOS),spawnwhere it does not (Windows)."forkserver"-- one helper process imports torch and docling; every worker is forked from it."spawn"-- a fresh interpreter per worker, importing everything itself."fork"is not accepted -- see notes.document_timeout/stall_timeout-- a positive number of seconds, or one of"off","none","false", or an empty string, all meaning "no limit".0and negative numbers are rejected rather than read as "off", because "zero seconds" is the opposite of what someone writing it means. Integers and floats both work (stall_timeout = 90.5is valid).long_word_chars/min_tokens-- any number; the fractional part is truncated.long_word_ratio-- a fraction between 0.0 and 1.0. The range is not enforced, so a value above 1.0 loads fine and disables the warning, since no document can exceed it.
[logging] -- the pipeline log file¶
| Key | Env var | Accepts | Default |
|---|---|---|---|
level |
LOGGING_LEVEL |
"DEBUG" | "INFO" | "WARNING" | "ERROR" | "CRITICAL" |
"INFO" |
level-- how much the pipeline writes tologs/pipeline.log(rotated at 5 MB, 5 backups kept -- fixed in code, not configurable). Case-insensitive; any other value is rejected, naming the valid ones. Only affects the file: terminal output is the same regardless of this setting. This is the only[logging]key -- rotation size and backup count haven't needed to vary per host. See CLI.md's "Running sync on a schedule".
One file, shared. Both python -m src.corpus sync and
src/enrich/__main__.py write here, and each line names its source
(src.sync, src.enrich, src.enrich.docling_parse), so
grep 'src\.sync' logs/pipeline.log recovers a per-command view. The
file is shared rather than split per command because that is what
makes it safe: a rotating file can only have one writer process at a
time, and these two already exclude each other through the pipeline
write lock. Commands that don't take that lock -- src.draft (all six
drafting-layer CLIs: gate, dossier, retrieve, references, render) --
write to stdout only and are not logged.
The log file's own location, logs/ beside the repo root, has no
config.toml key -- but does still honor a LOGS_DIR environment
variable, the same escape hatch every path in this file gets, for a
script (or a test) that needs it somewhere else.
[provenance] -- citation-support bands¶
| Key | Env var | Accepts | Default |
|---|---|---|---|
weak_score |
PROVENANCE_WEAK_SCORE |
number, a fraction 0.0-1.0 | 0.20 |
good_score |
PROVENANCE_GOOD_SCORE |
number, a fraction 0.0-1.0 | 0.50 |
src/review/citation_provenance.py (the review layer) bands the fraction of a
citing sentence's
distinctive words found in the best-matching source passage. Below
weak_score a finding reads "no support found", which means go look at
this one first -- never "this citation is wrong". At or above
good_score it is banded "supported".
Round numbers on purpose: the report sets a reading order for a human,
not a pass/fail line, so tuning them precisely would be false precision.
Neither is range-checked, and nothing enforces
weak_score < good_score.
[enrich] -- the optional enrichment layer¶
Used only by src/enrich/* (the enrich dependency group), never by
sync or the citation gate.
| Key | Env var | Accepts | Default in code | In config.toml.example |
|---|---|---|---|---|
embedding_model |
EMBEDDING_MODEL |
a sentence-transformers model id | sentence-transformers/all-MiniLM-L6-v2 |
sentence-transformers/all-mpnet-base-v2 |
docling_images |
DOCLING_IMAGES |
boolean | false |
false |
docling_image_scale |
DOCLING_IMAGE_SCALE |
number | 2.0 |
2.0 |
The two embedding_model columns differ on purpose, and the
distinction matters. The code's fallback is the smaller, faster
MiniLM -- what you get if the key is absent. The shipped example sets the
larger, more accurate mpnet, so anyone who copied config.toml.example
is running mpnet. Check your own file rather than assuming either. See
Choosing an embedding model.
How values are parsed¶
Worth knowing, because two of these will surprise you.
Booleans from the environment. In TOML, write true / false. From
an environment variable, only 1, true, yes, on
(case-insensitive) mean true -- anything else is false, including
typos. That is deliberate: bool("false") is True in Python, so
without it every documented way of turning a setting off via the
environment would silently turn it on.
A wrong TOML type falls back silently. String and boolean settings
return their default when the value has the wrong type, rather than
raising. So ocr = "true" -- a string, not a boolean -- is read as the
default false, with no complaint. Quote strings; don't quote booleans
or numbers.
Four settings are validated at load and fail loudly: workers,
start_method, document_timeout, stall_timeout. A bad value raises
immediately, naming the key, the environment variable, and what was
expected -- rather than surfacing much later as a nonsense pool size or a
strange timeout.
Notes on individual settings¶
backend: pdftotext or docling¶
src/pdf_text.py dispatches through a table, so adding a backend is one
function plus one entry -- and two candidates were added and later
removed through that same seam.
| Backend | Dependency | Page boundaries? | Quotable passages? | Speed |
|---|---|---|---|---|
pdftotext (default) |
poppler-utils on PATH |
Yes -- form feeds between pages | No -- reading order is lost | Fastest |
docling |
docling, enrich group |
Yes -- form feeds between pages | Yes -- writes a passage sidecar | ~42x slower; see PERFORMANCE.md |
Page boundaries are not cosmetic, which is why both backends now keep
them. src/review/verbatim_check.py reports which PDF page a verbatim
run came from by splitting on those form feeds. Before docling asked
for them, a citekey parsed that way reported pdf p.1 for every hit,
regardless of where the text sat.
What separates the two backends now is quoting, not paging.
pdftotext -layout preserves a page's visual arrangement rather than its
reading order, so an excerpt cut from it can splice two columns together.
The passage ladder therefore refuses to quote from it, and reports a page
number instead.
docling resolves reading order, and the corpus layer keeps that
resolution as content/parsed/<citekey>.passages.json. Choosing it here
therefore buys real quotable passages without running the enrichment
layer at all.
The mechanism is in CITATION-PROVENANCE.md, and the ladder it feeds is in LADDERS.md.
PDF-PARSER.md has the full fidelity comparison.
Setting backend = "docling" does not fold docling into the
enrichment layer, and does not make src/enrich/docling_parse.py redundant.
They are two consumers of the same library, with different scopes:
src/pdf_text.py(the corpus layer, onsync) extracts plain text per citekey intocontent/parsed/<citekey>.txtfor BM25 retrieval, plus the passage sidecar beside it. docling here is a higher-fidelity substitute forpdftotext's job.src/enrich/docling_parse.py(the enrichment layer, opt-in) produces structured Markdown for the whole corpus intocontent/docling/, feeding the embedding and topic stages that need real reading order and section boundaries. It always uses docling regardless of this setting, over exactly the same ledger documents -- it has no corpus of its own.
They no longer duplicate the parse, though. When this setting is
docling, the enrichment stage adopts the corpus layer's output for a
citekey instead of parsing the PDF a second time -- a file copy in place
of 6.65s per document.
It falls back to a real parse in three cases. First, for a citekey the
corpus layer wrote no text for -- a PDF whose parse failed at sync time,
say. Second, for a run with [enrich].docling_images on, because the
corpus layer writes no figure bitmaps to adopt. Third, when the artefacts
are older than the PDF, which means the PDF has been replaced since the
corpus
layer read it. The dependency only ever runs that way round: the
enrichment layer reads the corpus layer's files, and the corpus layer is
not shaped by this at all.
workers, and how it is clamped¶
The resolved count is the smallest of three ceilings, never below 1:
- what you asked for,
- what the machine can sustain,
- how many documents actually need parsing.
The third matters more than it looks: standing up 12 docling workers to parse 3 documents pays 12 model loads to save two documents' work.
"What the machine can sustain" counts the CPUs this process may run
on -- not the machine's total, which on a shared or containerised
machine can be far larger. For docling that count is divided by 4:
| CPUs available to the process | 4 | 8 | 16 | 48 |
|---|---|---|---|---|
workers = "auto" resolves to |
1 | 2 | 4 | 12 |
That divisor of 4 is measurably too conservative. It models a docling worker as occupying about 4 CPUs, and a full-corpus sweep does not support that. At 32 workers the CPU is only ~70% busy, and 32 workers run 1.41x faster than the 12 this table allows. 48 workers are no worse.
The honest reading is "much smaller than 4" rather than a specific replacement. Changing it is a behaviour change and has not been made -- see PERFORMANCE.md.
So a four-core desktop resolves to 2, and asking for 15 there still gets 2 -- clamped and said out loud on stderr, rather than silently obeyed (which thrashes) or silently ignored. docling's own internal thread count is divided down to match, so workers x threads still fits.
Two things the clamp cannot see: a cgroup CPU quota
(docker --cpus=2) throttles without changing which CPUs are permitted,
and RAM is not considered at all. Set an explicit number on either.
Each backend gets the concurrency it can use: processes for docling
(in-process, holds the GIL), threads for pdftotext (an external
subprocess that releases it). Ledger writes always stay on the main
process -- sqlite has a single writer -- and results are reported in
bibliography order regardless of which worker finished first, so two
identical runs still print identically.
Using more than one GPU¶
Nothing to configure. With docling and more than one worker, each
worker claims one CUDA device round-robin -- docling's own
AcceleratorDevice.AUTO resolves to cuda:0 in every process, so
without this every worker would pile onto card 0 while the rest idle.
Restrict which cards are used with CUDA_VISIBLE_DEVICES; the pool only
ever sees what that leaves visible. Figures in
PERFORMANCE.md.
A card someone else is already using is skipped, and the run says so on stderr:
1 2 | |
That threshold is a worker's ~1.7 GiB of models plus its CUDA context. The check matters more than it sounds: a worker that cannot get device memory fails a document in seconds where a working one takes minutes, and the pool hands the next document to whichever worker is free first โ so without this, the broken workers take most of the corpus. If every card is busy the run parses on the CPU (slower, but it finishes), and a worker that runs out of device memory during a run falls back to the CPU for its remaining documents rather than failing them.
Nothing to configure here either โ but if you would rather wait for a card than parse on the CPU, the warning is your cue to stop and re-run later.
Why fork is not an option¶
Not an oversight. By the time the pool is built, the process holds two
live sqlite connections -- the run lock and the ledger -- and SQLite's
own documentation says not to carry an open connection across fork().
forkserver starts its server as a fresh interpreter, so workers inherit
the preloaded modules and nothing else. It also measured no slower
than plain fork, so nothing is given up.
Both available start methods re-import the calling program's __main__
in each worker, so a script of your own driving sync or parse_corpus
must guard its top level with if __name__ == "__main__":.
The two timeouts are not the same guard¶
document_timeoutbounds one document, and the two backends enforce it with unequal strength. Forpdftotextit is a subprocess timeout -- a real kill, the one case where a wedged parse can actually be stopped. Fordoclingit is that library's own check between pipeline stages: it bounds a pathologically slow document but will not interrupt a hang inside a single stage.stall_timeoutbounds a parallel run, by asking whether any document at all has completed recently. With several workers, completions arrive constantly, so total silence across the whole pool separates a hung worker from a merely slow document far better than any per-document number could. It applies only whenworkers > 1; a serial run has no pool to go silent.
stall_timeout is on by default, unlike most safety valves here, because
the failure it catches is one a user actually hit: a run that never
finishes. A false positive is cheap -- outstanding documents are marked
failed and retried next run, not lost -- and a warning is printed at half
the budget before anything is given up on.
Either way a timed-out document is reported as a failure, not silently truncated -- but what happens next differs, because the two guards blame different things:
- A
document_timeoutcasualty is named on its own line insync's summary (WARNING: 2 document(s) hit the 120.0s [parser].document_timeout and were not parsed: <citekeys>), and is not retried. The limit that expired is a setting, so the next run under the same setting would spend the same minutes to reach the same answer. Raisedocument_timeoutand re-run with--reparse. - A
stall_timeoutcasualty is retried, automatically: those documents were never given a fair attempt, so they are recorded as transient failures and come back next run.
Choosing a safe value means knowing your slowest legitimate document; see PERFORMANCE.md.
The parse-quality guard¶
sync warns when an implausible share of a freshly extracted document's
words are unusually long -- the signature of a backend that has lost the
spaces between words. That is easy to miss by eye and expensive
downstream: src/retrieval.py tokenises on runs of [a-z0-9], so two
words that lost the space between them become a single token and neither
one matches a query for it any more.
long_word_chars, long_word_ratio and min_tokens are its thresholds.
It is a warning, never a failure -- the text is still usable, and an
unusual corpus could trip it legitimately. It will not catch a bad ocr
choice: it looks for run-together words, not for content that never
arrived.
docling_images¶
Extracts figure bitmaps into content/docling/<doc>_artifacts/, plus a
<doc>.figures.json giving each figure's page, caption, and the exact
string to cite it by.
Those images are a reading aid for checking a draft against its sources. They are never inserted into a draft: a figure's copyright is not the paper's citekey to grant. See DEVELOPER.md.
Changing this invalidates the whole docling cache, so the next run re-parses the corpus from scratch. Costs in PERFORMANCE.md.
Choosing an embedding model¶
src/enrich/embed_index.py calls
SentenceTransformer(config.EMBEDDING_MODEL).encode(...)
symmetrically -- the same call embeds a 200-word document chunk
(40-word overlap) and a search query, with no prefix or instruction text
added on either side.
That one fact decides which models are drop-in and which are not.
Drop-in¶
| Model | Dimensions | Relative cost | Best for | Tradeoff |
|---|---|---|---|---|
sentence-transformers/all-MiniLM-L6-v2 (code default) |
384 | Lowest -- ~22M params, fast even on CPU | Small corpora, quick iteration, CPU-only machines | Least semantic nuance of the three; general-purpose training data, nothing science-specific |
sentence-transformers/all-mpnet-base-v2 (example default) |
768 | ~4-5x MiniLM -- comfortable on a GPU, noticeably slower CPU-only | Meaningfully better general-purpose semantic quality | More RAM/VRAM and slower indexing/search, for a gain that may not matter at a small corpus size |
sentence-transformers/multi-qa-mpnet-base-dot-v1 |
768 | Same class as all-mpnet-base-v2 |
Trained specifically on short-query-vs-long-passage retrieval -- the closest match to what search() actually does |
Slightly weaker on generic sentence similarity outside retrieval |
What each one is:
all-MiniLM-L6-v2-- a 6-layer transformer distilled from a larger model, then fine-tuned on roughly a billion general sentence pairs with contrastive learning, so semantically similar sentences land close together. That symmetric objective is exactly what the prefix-freeencode()call needs, which is why it is the code default. ~22M parameters, fast on CPU alone.all-mpnet-base-v2-- the same recipe on a larger 12-layer MPNet backbone (~109M parameters). Generally the strongest all-around sentence-transformers model for semantic similarity with no domain specialisation. The extra quality costs roughly 4-5x the compute and doubles the vector dimensionality, which doubles per-chunk storage in Chroma and slows similarity search somewhat.multi-qa-mpnet-base-dot-v1-- the same MPNet backbone, fine-tuned on ~215M question/answer and query/passage pairs for dot-product retrieval rather than generic similarity. Conceptually the closest match to a short query against a longer chunk, and it needs no prefix, so it stays a clean drop-in. Same cost profile asall-mpnet-base-v2.
Not without a code change first¶
allenai/specter/specter2-- a SciBERT-based model trained on scientific title+abstract pairs using citation graphs as the signal. It is built for whole-paper similarity, not passage retrieval, and expects a specific input shape (title[SEP]abstract) that does not match the arbitrary 200-word body chunks this pipeline produces. Using it well means feeding it titles and abstracts -- a real code change -- and even then it answers "which papers are alike", a different question than "which chunk answers this query".- BAAI
bge-*andintfloat e5-*families -- strong on public retrieval benchmarks, but they expect literal"query: "/"passage: "prefixes baked into the input so the model knows which role each side plays. Nothing here adds one. Feeding either in as-is will not error -- it will silently underperform, which is the worst failure mode of the set. Adopting one means pairing it with matching prefix-handling code, not a config-only swap.
Switching¶
Edit [enrich].embedding_model, or set EMBEDDING_MODEL=... for a single
run, then rebuild the index:
1 | |
The model downloads on first use (needs network), and Chroma's existing collection is not re-embedded automatically -- switch only when you are prepared to rebuild the index.