Elamur GroupantsโŒ•
โ˜ฐ

db-src-load-sentence-document-join

The groupant beside tools/db/src/load.rs in the Mova repository: what the code computes, what it never does, how to go back from its outputs, its invariants, cost and checks.

Facts

Implementation material. A Mova code model describes this code at one source revision: partial models, finite checks and their stated scope. It is not a reviewed general exposition, and its labels are shown as written, never raised.

Components

componentlabelwhere
approximantpartialapproximant
astpartial../syntax.json
costpartialinvariant-and-cost
descriptantpresentdescriptant
invariantpartialinvariant-and-cost
testsbounded checks partial scopetests

Obernants

The typed relations around the forward model: the behaviour complement (what the code never does), the reverse reconstruction (every input compatible with an output) and specialised views.

idtypelabelwhere
behavior_complementbehavior_complementpartial source derived modelbehavior_complement
reverse_reconstructionreverse_reconstructionpartial source derived modelreverse_reconstruction
match_mode_partitionmatch_mode_partitionpartial source derived modelmatch_mode_partition
document_policy_joindocument_policy_joinpartial source derived modeldocument_policy_join
paragraph_counterparagraph_counterpartial source derived modelparagraph_counter
metadata_overwritemetadata_overwritepartial source derived modelmetadata_overwrite

Model text

descriptant

Prior sentence match and new document join

Filter st.sents[sent_id] by exact text. Auto/Existing with exactly one candidate reuse sid/doc; ties fail, Existing missing fails; otherwise derive document key, reuse/create DocCur, advance paragraph on marker and ord/sid, stage sentence.

Prospective epoch โ€” 2026-10-31

Full source was reviewed before edits; external prospective.md records API states, failure order and predicted checks. Production prefix remains byte-identical; test additions are isolated cfg(test)/test items.

Observed epoch โ€” 2026-10-31

2026-10-31 wave31: 16 selected target/name cases passed twice debug and twice release with identical sorted outcomes; six compiled source-behavior mutants failed assertions, exact bytes restored and positive checks passed. Evidence scope is only the named helpers/fixtures. External receipt directory: $MOVA_DATA/runs/octopus-wave31-store-2026-10-31. No DuckDB integration, production loader/corpus, training or GPU run.

Retained limitations

No SQL evaluation was substituted with a mock process. Missing DuckDB is a dependency gap, not passing database evidence. Partial syntax has no macro expansion/name resolution/types; Rust helpers and fixtures do not prove all models.

approximant

Prior sentence match and new document join forward relation

Filter st.sents[sent_id] by exact text. Auto/Existing with exactly one candidate reuse sid/doc; ties fail, Existing missing fails; otherwise derive document key, reuse/create DocCur, advance paragraph on marker and ord/sid, stage sentence.

Domain

Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.

Approximation limits

Categorical parsing, exact identity, transactional effects and resource errors have no useful percentage approximation metric here. No executable surrogate is claimed. Future profiles, if justified, may use only 1%, 2%, 5%, 10% with a defined metric/reference/domain.

models

Prior sentence match and new document join typed model catalog

Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.

Observation is one call/trace in the declared scope. Operators below are partial source-derived relations; complement is a proposed inner exclusion relation, reverse an outer compatible-antecedent relation retaining ambiguity. No universal proof follows from finite checks. Each oracle is the relevant named helper check below when it reaches the behavior; remaining SQL/effect cases are source review or proposed real-CLI checks.

behavior_complement

Auto/Existing cannot pick the first of two exact candidates. Existing cannot create a missing sentence. New cannot silently reuse an already stored doc key.

Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.

Evidence: see tests.md; unexecuted database branches remain conditional.

reverse_reconstruction

Given reused (sid,doc), compatible states have exactly one same-id/same-text candidate with that pair; differing-text candidates are unconstrained. New outputs permit many prior maxima/document metadata states consistent with assigned IDs.

Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.

Evidence: see tests.md; unexecuted database branches remain conditional.

match_mode_partition

New bypasses matching cases despite candidate count; Auto zero enters creation; Existing zero fails. Snapshot state excludes newly staged rows.

Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.

Evidence: see tests.md; unexecuted database branches remain conditional.

document_policy_join

NewDoc selects cur_doc or file_doc_key; IdPrefix strips last colon segment; File always file_doc_key. These key policies can merge different sentence IDs.

Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.

Evidence: see tests.md; unexecuted database branches remain conditional.

paragraph_counter

newpar/newpar id increments para before ord and sentence staging; absent marker with para0 writes NULL, not numeric0.

Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.

Evidence: see tests.md; unexecuted database branches remain conditional.

metadata_overwrite

NewDoc metadata JSON insert overwrites duplicate keys, but raw comments are later separately preserved by annotation staging. Metadata map alone is not lossless.

Domain/assumptions: Finite valid-UTF8 inputs, representable counters/IDs, available memory; filesystem fixtures confined to owned temporary root. SQL semantics are conditional and unexecuted. Subject is the source and SHA256 bound in manifest; fragment applies only to declared byte range.

Evidence: see tests.md; unexecuted database branches remain conditional.

tests

Selected checks

2026-10-31 wave31: 16 selected target/name cases passed twice debug and twice release with identical sorted outcomes; six compiled source-behavior mutants failed assertions, exact bytes restored and positive checks passed. Evidence scope is only the named helpers/fixtures. External receipt directory: $MOVA_DATA/runs/octopus-wave31-store-2026-10-31. No DuckDB integration, production loader/corpus, training or GPU run.

Named target-scoped checks for parent source: ["load::wave31::option_and_comment_representation","load::tests::comments_and_dates","load::teacher_provenance_tests::codex_runs_are_not_labelled_as_legacy_claude"]

Native commands and numeric status/log/outcomes live in external native-commands.json and receipts. Integration fixture checks use actual en::conllu parsing only. Legacy prag_layer_and_negative_controls, qa_table_loader_and_negative_controls and lossless_round_trip_and_negative_controls were inspected but not executed: real DuckDB unavailable. A fragment's listed parent tests may only cover part of its relation; SQL/query/transaction/IO-fault claims remain source-derived.

Negative controls

Six source-behavior mutations: quote doubling, TSV NULL rejection, QA positivity, prag bit truth, MWT counter, sentence boundary predicate. Each compiled, failed an assertion, restored exact original bytes and passed the selected positive. No assertion-only mutations and no surviving behavioral mutants.

invariant-and-cost

Prior sentence match and new document join invariants and costs

Invariant

Reused sentence doc comes from found candidate and is independent of current newdoc key. New sentence increments created exactly once; only new sentences update local DocCur counters.

Cost

Per sentence O(candidate vector + comments + metadata bytes) expected map work. Total suffix probing occurs outside this fragment; counters require nonoverflow.

Symbolic bounds are source-derived, not measured wall-clock speed or universal complexity proof. Allocation failures, integer exhaustion and concurrency are outside finite evidence.

This page in Markdown โ†’ https://md.groupants.elamur.ai/free/en/mova/tools/db/src/load.rs.groupant/sentence-document-join.groupant/