📐fish-swarm-library

The groupant beside tools/fish-swarm/src/lib.rs in the Mova repository: what the code computes, what it never does, how to go back from its outputs, its invariants, cost and checks.

Facts

Components

Obernants

The typed relations around the forward model: the behaviour complement (what the code never does), the reverse reconstruction (every input compatible with an output) and specialised views.

Model text

domain

One trusted cooperative local filesystem, exclusive flock owner, immutable configuration C, finite JSONL queue J, canonical regular source/task/profile/executable pins, and a conforming annotation executor. Selected observations are reservations, accepted PID, child exit, receipt identities and scheduling boundaries. Model is source-derived metadata reproduction, not a model of language understanding. Clock is Unix seconds; timeout uses monotonic elapsed time. No numerical approximation or percentage accuracy claim is introduced.

forward

Let R denote retained job directories, T valid bound outer terminals, D the exclusive deadline, t the current wall clock, and A an executor invocation. First inspect every retained directory: any element outside T yields uncertain regardless of STOP, PAUSE or t. Otherwise STOP, t>=D or PAUSE produce their boundary observation without A. Verify immutable queue identity, reject duplicate job IDs, skip matched terminals, and choose the first fresh item. Check task/profile/source/binary hashes and native task id. Recheck boundaries; create exclusive directory and fsynced reservation before copying task/profile. Recheck boundaries again. A newly crossed boundary produces a not-started terminal. Otherwise open distinct stdout/stderr, spawn executor with D and timeout, and record accepted PID. Poll exit and heartbeat. Validate exit plus pinned executor receipt; record review-only terminal or failure. No queue item is resubmitted.

The vidtvorant is this finite metadata state machine. It predicts whether an executor start is admitted; it deliberately leaves executor content opaque. Executor inner timeout is part of the interface contract. Outer cleanup grace is timeout+10seconds, followed by process-group signals. Separately grouped descendants and a crashing scheduler require explicit uncertainty analysis.

invariant

For every A, exclusive directory and durable reservation precede spawn. At most one owner holds the cooperative lease. Existing uncertain directories cannot be automatically reused, even after queue removal or STOP. Config raw hash fixes all settings; queue raw hash fixes ordering and provenance. Source commitment says these exact bytes were inspected, not that the inputs are scientifically sufficient. A successful outer receipt always preserves training_authorized=false and semantic_correctness=not established.

cost

For n queue entries, r retained directories, total bytes B of metadata, retained task/profile copies and selected pinned inputs, one step costs O(n+r+B) time and O(n+B) memory, plus executor latency. Repeated full queue scans can produce O(n^2) cumulative metadata work; no throughput improvement is claimed. Files are bounded individually to64MiB; this does not bound total queue-referenced bytes or total campaign disk growth. Active polls25ms and heartbeat5s are operational intervals, not scheduling guarantees.

behavior-complement

Type behavior_complement. In the declared transition domain, admitted starts exclude expired/STOP/PAUSE boundaries, mismatched pins, duplicate IDs, occupied fresh-output paths and unresolved retained history. The excluded-effect relation is A intersect (boundary union identity_failure union uncertainty)=empty. Counterexample to universal no-effects-on-expiry: status metadata is written, and a boundary crossed after reservation leaves a not-started terminal. Finite native controls and source inspection support this partial relation; no crash-atomic process theorem is established.

reverse-reconstruction

Type reverse_reconstruction. A valid terminal reconstructs compatible config/job commitments, selected task/profile hashes, exit, retained output hashes and review-only classification. Its preimage contains multiple internal annotation histories and possible semantic errors. Reservation without terminal has preimages not-started, active, failed, or finished-before-receipt; no unique restart action follows. Counterexample to retry-on-missing-receipt: the external call may already have consumed quota and produced a result. Finite orphan fixture demonstrates nonreplay, not unique history recovery.

deadline-drain

Type temporal_boundary. Domain is accepted outer calls under a stable exclusive deadline. Let s be pre-reservation check and e child completion. New reservation requires s<D; e may exceed D for an already accepted bounded call. A subsequent step with t>=D cannot start another call. Counterexample to e<=D: the sleep fixture completes after the synthetic clock advances past D. Evidence separates synthetic deterministic clock observations from the measured monotonic stalled-executor fixture. Clock rollback and system scheduling are assumptions/limits; this is not a hard real-time proof.

uncertainty-precedence

Type error_partition. Partition retained state into fresh, valid terminal, invalid/absent terminal and invalid commitment. Only fresh can be considered for start; invalid/absent terminal yields uncertain before any purported quiescent boundary. Counterexample discovered in review: checking STOP first could report model-call false while an orphan remained alive. The source now scans and validates history first; the regression adds PAUSE, STOP and expiry to an orphan case. This is a concrete scheduler safety correction, independent of model semantics.

receipt-evidence-join

Type independent_evidence_join. Review eligibility is exit0 AND valid receipt schema AND exact task.id/receipt.task_id AND receipt.task_file_sha256=raw task pin AND receipt.profile_file_sha256=raw profile pin AND wellformed receipt.task_sha256 AND review_only AND not training_authorized AND not replay_authorized AND model exit0 AND explicit started/reaped/group-absent evidence AND declared executor status. Raw file and typed Task commitments are distinct observations. This generic scheduler checks typed digest shape; it does not independently reproduce native typed serialization. Counterexample: exit0 plus empty receipt, wrong raw hash or malformed typed digest is rejected even with accepted status. Accepted transport neither proves syntax/pragmatics nor authorizes training.

pause-observation

Type observation_exclusion. A fresh paused snapshot with active_model_call=false and no_unresolved_reservations=true excludes an accepted unfinished call in this owner's known retained history, under the cooperative executor contract. PAUSE-file existence alone excludes nothing about an earlier in-flight call. Active snapshots are true; uncertain is null. Heartbeat freshness and lease are additional observations, not inferred from labels. A dead supervisor may leave active descendants; missing heartbeat never reconstructs death.

checks

Thirteen focused Rust fake-executor tests cover STOP/PAUSE/expiry, boundary rechecks, completion across deadline, no repeated exhausted job, lease/config mismatch, orphan nonreplay before all boundaries, failed/malformed/wrong-identity receipt, changed pins/duplicates, and bounded hung-executor cleanup. Fake binary is compiled locally and never calls a model. These are finite evidence, not Lean proof, semantic review or training evidence.

epoch-memory

Independent compatibility review exposed uncertainty-before-boundary and native Task.id versus receipt.task_id. Both were corrected before deployment. Keep those distinctions when changing scheduling or integrating priority observers.

representation-hash-split

Type representation_boundary. Domain: bounded raw task file B accepted by the native executor as typed Task T. Define F=SHA256(B), H=SHA256(native compact serialization(T)). The queue pins F; receipt.task_file_sha256 must equal F. receipt.task_sha256 carries H and must be a valid digest, but the generic supervisor does not independently derive T or H. Counterexample to F=H: pretty-printed JSON parses to the same Task yet includes different whitespace. The positive fixture deliberately makes F!=H, then verifies both retained commitments. Wrong F and malformed H are rejected. Finite serialization fixture and protocol inspection, not a cryptographic collision proof or native-schema theorem.

process-group-uncertainty

Type process_liveness_exclusion. Domain: outer executor process group G and a separately grouped descendant C. Observing wait(executor) after signal(G) excludes a live direct executor, but does not exclude live(C). Therefore executor_timeout and executor_failed outer records are not admitted terminal histories, even when a receipt file exists. Every later step, including after restart, STOP and expiry, reports uncertain/null with no_unresolved_reservations=false until independently reviewed outside this automatic path. Counterexample: the fake executor starts a separate-group sleep child; the outer timeout kills its own group, while kill(child,0) succeeds and the observed /proc process state is non-zombie. The fixture kills that harmless child explicitly after verifying uncertainty. This finite process observation establishes the concrete counterexample; arbitrary escaped descendants remain unknown.

known-closed-quarantine

Type error_partition. Domain: a pinned executor's bounded receipt, copied task/profile, outer child completion and retained reservation. Let B bind task id, raw task/profile hashes, typed task digest shape, review-only/no-training/no-replay flags and successful model exit. Let C be explicit model_process_started=true, model_process_reaped=true and model_process_group_absent=true. Define Q=B AND C AND native status=invalid_response AND NOT outer_timeout. Q admits only a durable quarantined_review_only record, never a successful annotation, gold label or retry. After Q the scheduler may choose the next fresh distinct queue entry; the same reserved entry remains skipped. If C is false/null/missing, raw pins differ, receipt is absent, native status is timeout or outer timeout occurred, Q is false and uncertainty persists even after STOP/expiry. Counterexample to stopping forever on every bad answer: the two-job fake exits1 for a known-closed invalid draft, then processes a second task exactly once. Counterexample to trusting a status string alone: false/null group proof prevents quarantine. Retained audit verifies inner receipt hash, exact copied bytes and reservation/outer/inner bindings; a changed proof with refreshed digest is still rejected. Thirteen finite local tests support these partitions; no semantic truth or whole-process-tree sandbox proof is inferred. The absent-group observation concerns the declared native group at its recorded instant, not arbitrary descendants that escape it.

read-only-history-audit

Domain: an existing immutable Config, its parsed prepared queue and retained job tree. Observation: audit_history returns a no-unresolved flag, source-target counts and retained receipt pins without acquiring leases, writing files or launching work. For every retained directory require the current queue job identity, config identity and the existing closed_history predicate (or an explicitly not-started/spawn-failed boundary). A foreign directory, missing receipt, false/null closure, changed copied task/profile or unknown status excludes no_unresolved_reservations=true. A queued job with no reservation is pending, not an unknown paid attempt. Source counts come from each pinned Task.target_ids rather than job counts. Thus a closed four-target review adds four validated targets, while a two-target known-closed invalid reply adds two quarantined targets without declaring them reviewed. Reverse observation is nonunique: the same complete history audit cannot recover linguistic truth or remote billing events. Counterexamples: an orphan beside otherwise completed jobs blocks the audit; an all-deferred empty page has no unknown attempts but zero reviewed targets. Cost is O(queue/task/retained-receipt bytes), bounded per page by the paging caller. Evidence: eight paging fake-Rust controls exercise the audit, and all thirteen existing swarm tests remain green; no formal proof or live-provider check.

provider-busy-requeue

Type error_partition (2026-11-04). Domain: one accepted executor call whose child exited nonzero, not timed out, with an inner receipt satisfying busy_bound: the same task/profile/task-id bindings as closed_bound, status=provider_busy, a nonempty provider_busy_message, a nonzero model exit and explicit started/reaped/group-absent evidence. busy_history additionally requires the outer record to carry status provider_busy, transport_eligible=false, no reply.json and raw.stdout bytes equal to the inner stdout hash. Such an attempt is "provider busy, nothing done": the attempt directory is renamed from jobs/<id> to busy/<id>/<k> (k=01,02,...), a line is appended to provider-busy.jsonl, and the job is fresh again. With f_k the finish time of busy attempt k and b=(60,120,240) seconds, the next start of that job requires t>=f_k+b_k; while t<f_k+b_k the step reports provider_busy_backoff, and after the fourth busy attempt (k>|b|) it reports provider_busy_held with automatic_retry=false until operator review. Neither state is uncertain, a usage limit or a terminal outcome; no_unresolved_reservations stays true. Behavior complement: a timeout, a nonzero exit without busy_bound (usage limit, quota, auth, partial output), a busy-looking attempt with a reply, a tampered retained busy attempt, a noncontiguous attempt sequence or a foreign busy directory never requeue and keep the existing uncertain path. Reverse reconstruction: a retained busy attempt reconstructs the bound task/profile, the exact refusal message and the raw Codex event bytes; it cannot exclude a provider-side bill or prove that no remote work was done beyond the executor's classification. Crash recovery: a closed busy attempt still under jobs/ is retired at the next step; busy/ is authoritative, the journal is a readable index that may miss a line after a crash. Cost per step adds O(a) metadata reads for a retained busy attempts of the chosen job. Evidence: two fake-executor tests (backoff 61/181/421 then hold at the fourth refusal; one refusal then success; red controls for busy-with-reply, ordinary failure and tampered raw stdout). Fifteen swarm tests pass; no live-provider replay was performed.

read-only-history-audit-busy

The audit also checks every busy/<id> directory against the current queue, config and busy_history and reports provider_busy_attempts; an unverifiable or foreign busy attempt yields no_unresolved_reservations=false. Busy attempts do not count as terminal; their jobs remain pending targets until a later closed terminal.

This page in Markdown → https://md.groupants.elamur.ai/free/en/mova/tools/fish-swarm/src/lib.rs.groupant/