db-src-load-sentence-document-join
The groupant beside tools/db/src/load.rs in the Mova repository: what the code computes, what it never does, how to go back from its outputs, its invariants, cost and checks.
Facts
Implementation material. A Mova code model describes this code at one source revision: partial models, finite checks and their stated scope. It is not a reviewed general exposition, and its labels are shown as written, never raised.
Components
| component | label | where |
|---|---|---|
| approximant | partial | approximant |
| ast | partial | ../syntax.json |
| cost | partial | invariant-and-cost |
| descriptant | present | descriptant |
| invariant | partial | invariant-and-cost |
| tests | bounded checks partial scope | tests |
Obernants
The typed relations around the forward model: the behaviour complement (what the code never does), the reverse reconstruction (every input compatible with an output) and specialised views.
| id | type | label | where |
|---|---|---|---|
behavior_complement | behavior_complement | partial source derived model | behavior_complement |
reverse_reconstruction | reverse_reconstruction | partial source derived model | reverse_reconstruction |
match_mode_partition | match_mode_partition | partial source derived model | match_mode_partition |
document_policy_join | document_policy_join | partial source derived model | document_policy_join |
paragraph_counter | paragraph_counter | partial source derived model | paragraph_counter |
metadata_overwrite | metadata_overwrite | partial source derived model | metadata_overwrite |
Model text
descriptant
Prior sentence match and new document join
Filter st.sents[sent_id] by exact text. Auto/Existing with exactly one candidate reuse sid/doc; ties fail, Existing missing fails; otherwise derive document key, reuse/create DocCur, advance paragraph on marker and ord/sid, stage sentence.
Prospective epoch โ 2026-10-31
Full source was reviewed before edits; external prospective.md records API states, failure order and predicted checks. Production prefix remains byte-identical; test additions are isolated cfg(test)/test items.
Observed epoch โ 2026-10-31
2026-10-31 wave31: 16 selected target/name cases passed twice debug and twice release with identical sorted outcomes; six compiled source-behavior mutants failed assertions, exact bytes restored and positive checks passed. Evidence scope is only the named helpers/fixtures. External receipt directory: $MOVA_DATA/runs/octopus-wave31-store-2026-10-31. No DuckDB integration, production loader/corpus, training or GPU run.
Retained limitations
No SQL evaluation was substituted with a mock process. Missing DuckDB is a dependency gap, not passing database evidence. Partial syntax has no macro expansion/name resolution/types; Rust helpers and fixtures do not prove all models.
approximant
Prior sentence match and new document join forward relation
Filter st.sents[sent_id] by exact text. Auto/Existing with exactly one candidate reuse sid/doc; ties fail, Existing missing fails; otherwise derive document key, reuse/create DocCur, advance paragraph on marker and ord/sid, stage sentence.
Domain
Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.
Approximation limits
Categorical parsing, exact identity, transactional effects and resource errors have no useful percentage approximation metric here. No executable surrogate is claimed. Future profiles, if justified, may use only 1%, 2%, 5%, 10% with a defined metric/reference/domain.
models
Prior sentence match and new document join typed model catalog
Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.
Observation is one call/trace in the declared scope. Operators below are partial source-derived relations; complement is a proposed inner exclusion relation, reverse an outer compatible-antecedent relation retaining ambiguity. No universal proof follows from finite checks. Each oracle is the relevant named helper check below when it reaches the behavior; remaining SQL/effect cases are source review or proposed real-CLI checks.
behavior_complement
Auto/Existing cannot pick the first of two exact candidates. Existing cannot create a missing sentence. New cannot silently reuse an already stored doc key.
Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.
Evidence: see tests.md; unexecuted database branches remain conditional.
reverse_reconstruction
Given reused (sid,doc), compatible states have exactly one same-id/same-text candidate with that pair; differing-text candidates are unconstrained. New outputs permit many prior maxima/document metadata states consistent with assigned IDs.
Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.
Evidence: see tests.md; unexecuted database branches remain conditional.
match_mode_partition
New bypasses matching cases despite candidate count; Auto zero enters creation; Existing zero fails. Snapshot state excludes newly staged rows.
Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.
Evidence: see tests.md; unexecuted database branches remain conditional.
document_policy_join
NewDoc selects cur_doc or file_doc_key; IdPrefix strips last colon segment; File always file_doc_key. These key policies can merge different sentence IDs.
Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.
Evidence: see tests.md; unexecuted database branches remain conditional.
paragraph_counter
newpar/newpar id increments para before ord and sentence staging; absent marker with para0 writes NULL, not numeric0.
Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.
Evidence: see tests.md; unexecuted database branches remain conditional.
metadata_overwrite
NewDoc metadata JSON insert overwrites duplicate keys, but raw comments are later separately preserved by annotation staging. Metadata map alone is not lossless.
Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.
Evidence: see tests.md; unexecuted database branches remain conditional.
tests
Selected checks
2026-10-31 wave31: 16 selected target/name cases passed twice debug and twice release with identical sorted outcomes; six compiled source-behavior mutants failed assertions, exact bytes restored and positive checks passed. Evidence scope is only the named helpers/fixtures. External receipt directory: $MOVA_DATA/runs/octopus-wave31-store-2026-10-31. No DuckDB integration, production loader/corpus, training or GPU run.
Named target-scoped checks for parent source: ["load::wave31::option_and_comment_representation","load::tests::comments_and_dates","load::teacher_provenance_tests::codex_runs_are_not_labelled_as_legacy_claude"]
Native commands and numeric status/log/outcomes live in external native-commands.json and receipts. Integration fixture checks use actual en::conllu parsing only. Legacy prag_layer_and_negative_controls, qa_table_loader_and_negative_controls and lossless_round_trip_and_negative_controls were inspected but not executed: real DuckDB unavailable. A fragment's listed parent tests may only cover part of its relation; SQL/query/transaction/IO-fault claims remain source-derived.
Negative controls
Six source-behavior mutations: quote doubling, TSV NULL rejection, QA positivity, prag bit truth, MWT counter, sentence boundary predicate. Each compiled, failed an assertion, restored exact original bytes and passed the selected positive. No assertion-only mutations and no surviving behavioral mutants.
invariant-and-cost
Prior sentence match and new document join invariants and costs
Invariant
Reused sentence doc comes from found candidate and is independent of current newdoc key. New sentence increments created exactly once; only new sentences update local DocCur counters.
Cost
Per sentence O(candidate vector + comments + metadata bytes) expected map work. Total suffix probing occurs outside this fragment; counters require nonoverflow.
Symbolic bounds are source-derived, not measured wall-clock speed or universal complexity proof. Allocation failures, integer exhaustion and concurrency are outside finite evidence.
This page in Markdown โ https://md.groupants.elamur.ai/free/en/mova/tools/db/src/load.rs.groupant/sentence-document-join.groupant/