Elamur Groupantsgparse-model-attention
The groupant beside gparse/src/model.rs in the Mova repository: what the code computes, what it never does, how to go back from its outputs, its invariants, cost and checks.
Facts
Implementation material. A Mova code model describes this code at one source revision: partial models, finite checks and their stated scope. It is not a reviewed general exposition, and its labels are shown as written, never raised.
Components
| component | label | where |
|---|---|---|
| approximant | partial | approximant |
| ast | partial | ../syntax.json |
| cost | symbolic source derived | invariant |
| descriptant | present | descriptant |
| invariant | partial | invariant |
| tests | partial bounded checks | tests |
Obernants
The typed relations around the forward model: the behaviour complement (what the code never does), the reverse reconstruction (every input compatible with an output) and specialised views.
| id | type | label | where |
|---|---|---|---|
behavior_complement | behavior_complement | partial source derived model | behavior_complement |
reverse_reconstruction | reverse_reconstruction | partial source derived model | reverse_reconstruction |
gradient_dispatch | gradient_dispatch | partial source derived model | gradient_dispatch |
mask_bias | mask_bias | partial source derived model | mask_bias |
residual_state | residual_state | partial source derived model | residual_state |
dimension_domain | dimension_domain | partial source derived model | dimension_domain |
Model text
descriptant
Source-bound description
Embedding sum and pre-LN transformer dispatch attention softmax by (fused,train), then biaffine masked arcs.
Epoch 2026-10-31
Prospective model was recorded before edits. Added tests use bounded synthetic fixtures; production algorithms unchanged. Coverage is partial, not proof. No trained corpus, live teacher or throughput experiment was executed.
approximant
Forward approximant
Embedding sum and pre-LN transformer dispatch attention softmax by (fused,train), then biaffine masked arcs.
Domain: valid source-specific dimensions and resources stated in invariant; no automatic production substitution. Numerical observations are finite checks, not an end-to-end relative bound. Executable Rust surrogate: unsupported gap; existing exact tests are not surrogates. Future metric-specific caps may be 1/2/5/10%, never above 10%.
invariant
Invariant and cost
Fused training uses p*(g-dot) backward; inference fused path has no Q/K backward. Valid B,T,heads dimensions required.
Cost
O(BTd+L*(BTd²+BT²d+BTdf)+BTda+BTa²+BT²a); attention probabilities occupy BHT², with retained graphs in training. BTd embedding/final-LN work remains at L=0.
Allocation, finite arithmetic, platform dimensions and external failures are not universally certified.
models
Typed model catalog
behavior_complement
Public train fused cannot invoke upstream inference-only softmax.
reverse_reconstruction
Categorical outputs cannot recover encoder or logits; encoder itself admits embedding-sum parameter symmetries.
gradient_dispatch
(true,true)=custom differentiable;(true,false)=upstream fused;(false,_)=reference.
mask_bias
keys-1 multiplied1e9 on attention/head columns; finite negative bias not hard exclusion for arbitrary huge logits.
residual_state
Pre-LN attention and FF outputs add to h; final LN precedes head projections.
dimension_domain
heads nonzero,d divisible heads,position/feature indices valid; no centralized config validation.
tests
Checks and limits
Selected parent checks relevant to this fragment (not exhaustive): model::tests::embeddings_get_gradients; model::fused::which_ops_backprop; activation_configuration_roundtrips; CUDA gradient/GELU/synthetic optimizer checks; CPU prediction
Selected commands run twice debug and twice release; exact receipts and source-to-test mapping retained in wave30 gparse external run directory. Tests do not certify allocation failure, all NaN/overflow inputs, trained-model accuracy or corpus timing. Compiled mutants cover selected boundaries; per-source coverage is recorded in the external map, not presumed.
This page in Markdown → https://md.groupants.elamur.ai/free/en/mova/gparse/src/model.rs.groupant/attention.groupant/