NeuroTwin Pair-Operator Historical Ablation Design

Status: superseded architecture branch; retained as NFC baseline/ablation Scope: Algonauts/CNeuroMod fMRI-first historical architecture experiment Claim scope: no model superiority claim until real A100 evidence exists

Pair-Operator is no longer the main NeuroTwin architecture. The main experimental architecture is NeuroTwin NFC, the Neural Field Compiler. This file is retained to document the historical Pair-Operator branch and to define the baseline/ablation role it plays inside NFC.

Problem Statement

Naturalistic brain activity is not a flat token stream. It is a time-evolving relational system over parcels, sensors, or regions, driven by stimulus streams and constrained by evidence quality. The historical Pair-Operator experiment tests whether explicit low-rank parcel-pair state improves fMRI tasks under leakage-audited splits.

The north-star benchmark is Algonauts/CNeuroMod fMRI first, with three tasks:

1. future_state_forecasting

2. masked_neural_reconstruction

3. stimulus_to_fmri_response

The allowed future paper claim is narrow:

Pair-Operator improves at least one core fMRI/stimulus task under leakage-audited splits versus strong local baselines.

No current local evidence supports model superiority, SOTA, clinical use, first brain foundation model, first multimodal brain model, or first stimulus-to-brain claims.

Prior Art Table

System	Relevant idea	Constraint on NeuroTwin claims
TRIBE v2	Tri-modal video/audio/language modeling for fMRI prediction.	Stimulus-to-fMRI is crowded; NeuroTwin must not claim first stimulus-to-brain. Use tribe_style_clean_room only as a local approximation unless exact upstream reproduction is added.
BrainVista	Naturalistic fMRI next-token prediction with history/stimulus masking and long-horizon rollout framing.	BrainVista-style rollout is a baseline lane, not empty territory. NeuroTwin uses brainvista_style as a causal local approximation, not exact BrainVista.
Brain-OF	fMRI/EEG/MEG foundation-style model with any-resolution sampling and sparse MoE.	NeuroTwin must not claim first multimodal brain model or uncontested foundation-model novelty.
BrainOmni	EEG/MEG tokenizer and sensor-aware modeling.	NeuroTwin must not claim first sensor-aware neural model.
DeepSeek-V3	Efficient conditional computation via sparse MoE and latent attention.	Efficiency and conditional computation matter; Pair-Operator should expose low-rank and block controls instead of dense all-pair memory.
AlphaFold-style Pairformer idea	Separate node and pair states plus confidence-style outputs can be powerful in relational domains.	Pair-Operator borrows the relational-state lesson, not protein-domain claims or exact Pairformer mechanics.

What Pair-Operator Adds

NeuroTwin Pair-Operator adds an executable, leakage-gated fMRI architecture lane with:

• node states z_i(t) for parcel activity/state

• low-rank pair states P_ij(t) represented by learned left/right factors

• a pair update path that actively changes node updates and predictions

• causal stimulus alignment for stimulus-to-fMRI tasks

• per-region/per-time uncertainty outputs and optional pair uncertainty

• explicit evidence gates for leakage, stimulus provenance, non-finite failures, baseline ranking, and final claim eligibility

This is not a generic model zoo. The experiment is useful only if the pair path changes outcomes under real, audited fMRI splits.

Architecture Diagram

Prepared neural windows
  ├─ fMRI parcel windows: x_fmri[t, i]
  ├─ optional stimulus features: s[t, k]
  └─ metadata: subject/session/site/dataset/split/stimulus provenance

Node encoder
  x_source[t, *] -> projected parcel values -> node states z_i(t)

Stimulus alignment adapter
  causal lag/HRF-style stimulus drive -> additive node drive

Low-rank pair state
  left factors  L_i,r
  right factors R_j,r
  optional top-k sparsity over R
  optional network block gates over L
  P_ij is implicit: softmax((L_i · R_j) / sqrt(rank))

Pair-to-node operator
  pair_context_i(t) = sum_r L_i,r * sum_j R_j,r * z_j(t)
  z_i(t) <- z_i(t) + MLP(pair_context_i(t))

Multi-timescale temporal operator
  per-parcel GRU/SSM/Transformer path over time
  fMRI HRF context path

Task heads
  future_state_forecasting
  masked_neural_reconstruction
  stimulus_to_fmri_response

Uncertainty and evidence
  per-region/time uncertainty
  optional pair uncertainty
  evidence_gate.json only after explicit finalization

State Definitions

Node state:

z_i(t) ∈ R^d

i indexes an fMRI parcel or sensor. t indexes the prepared window time step. The implementation builds node states by projecting the source modality onto the target parcel axis, encoding scalar parcel values, adding optional causal stimulus drive, and then applying pair and temporal updates.

Pair state:

P_ij(t) ≈ f(L_i, R_j, z_j(t))

The implementation does not store a dense learned N x N parameter table. It stores low-rank factors:

L ∈ R^(N x r)
R ∈ R^(N x r)

The active pair context is computed as:

summary_r(t) = sum_j R_j,r z_j(t)
context_i(t) = sum_r L_i,r summary_r(t)

This keeps the node update path O(N * r * d) instead of O(N^2 * d). Dense pair confidence is materialized only for small parcel counts. For 1000 fMRI parcels, pair_confidence is compact with shape (2, 1000, rank).

Low-Rank Controls

Required controls are now explicit in config:

• use_pair_state: disables the pair-to-node update path

• pair_rank: low-rank factor width

• pair_top_k: optional rank-wise top-k sparsity on right factors

• network_blocks: optional network/block gates over left factors

• pair_confidence_max_parcels: maximum dense confidence map size

• use_pair_uncertainty: optional pair uncertainty output

The no-pair ablation keeps the same node/temporal path and disables the active pair update. Runtime estimates mark it as disabled_low_rank_parameters_present because the current module still allocates pair parameters for config symmetry.

Multi-Timescale Operator

The current temporal path is intentionally small:

• per-parcel recurrent/SSM fallback or Transformer path over window time

• fMRI HRF context adapter

• refinement head for prediction updates

The A100 run should not tune performance locally. The only local requirement is that the path is shape-safe, finite, and ablation-ready.

Stimulus Alignment Adapter

Stimulus-to-fMRI is evidence-gated. The local BrainVista-style approximation is:

history fMRI + optional lagged stimulus features -> future fMRI

Default stimulus features are history-only:

• stimulus_lag_steps=1

• include_current_stimulus=false

• hrf_lag_steps=2

Tests verify that future and current stimulus values do not affect the current BrainVista-style feature row by default.

Uncertainty Head

Pair-Operator emits:

• uncertainty: per-time/per-region uncertainty with prediction shape

• pair_uncertainty: optional compact or dense pair uncertainty

• uncertainty_calibration.csv: task-level mean uncertainty and error/uncertainty correlation when metrics are finite

Uncertainty metrics are diagnostic. They do not make a run claim-eligible by themselves.

Evidence Gate

Stimulus claim eligibility is impossible through transcript hashes, synthetic embeddings, or self-attested embedding hashes. For claim_eligible=true, every used stimulus feature window must provide:

• stimulus_feature_source

• stimulus_feature_modalities

• stimulus_feature_hash

• stimulus_feature_status in a real/precomputed class

• local source artifact path, manifest, or file:// URI

• computed source artifact hash equal to declared hash

If any required item is missing, the task is plumbing-only and claim_eligible=false with exact failure reasons.

Training writes provisional evidence only:

• evidence_gate_provisional.json

• diagnostic_report_provisional.md

Final evidence is written only by explicit finalization:

PYTHONPATH=src python3 -m neurotwin.cli report evidence-gate --run-dir /abs/run

report and report model-card are read-only and must not mutate summary.json. Baseline ranking validity comes from structured prepared_baseline_suite.json, not CSV string scanning.

Non-Finite Safety

Training now fails closed for non-finite behavior:

• non-finite loss is detected

• non-finite gradient norm is detected

• optimizer step is skipped on non-finite loss or gradient

• default clipping is max_grad_norm=1.0

• task quarantine forces scientific_claim_allowed=false

• aggregate metrics ignore NaN values but list quarantined tasks

• best finite checkpoint rollback is recorded separately from final metrics

• final reports distinguish completed training, task quarantine, and claim disallowance

Baselines

Executable local baselines:

• train_mean

• persistence

• linear_ridge

• autoregressive_ridge

• neurotwin

• tribe_style_clean_room

• brainvista_style

• pair_operator

• pair_operator_no_pair_state

Non-executable catalog baselines remain labeled as upstream/contextual when no local runner exists. Do not describe tribe_style_clean_room as exact TRIBE v2. Do not describe brainvista_style as exact BrainVista.

Ablation Matrix

Variant	Purpose	Expected artifact
current_neurotwin	Existing translator baseline.	baseline ranking row
brainvista_style	Causal history/stimulus fMRI approximation.	baseline ranking row
pair_operator_no_pair_state	Node and temporal path without pair update.	pair_operator_ablation.csv
pair_operator_low_rank_pair_state	Pair update on, uncertainty/refinement off.	pair_operator_ablation.csv
pair_operator_pair_state_stimulus_adapter	Pair update plus stimulus adapter.	pair_operator_ablation.csv
pair_operator_pair_state_uncertainty	Pair update plus uncertainty diagnostics.	uncertainty_calibration.csv
pair_operator_full	Full candidate.	all final sidecars

Configs

Required configs are present:

• configs/train/algonauts_pair_operator_debug.yaml

• configs/train/algonauts_pair_operator_full.yaml

• configs/train/algonauts_pair_operator_ablation_array.yaml

• configs/train/algonauts_pair_operator_no_pair.yaml

• configs/train/algonauts_brainvista_style_debug.yaml

• configs/train/algonauts_current_neurotwin_debug.yaml

Each includes explicit dataset placeholders, run_id, max_grad_norm, bf16, evidence gate settings, require_real_stimulus, and transcript-hash denial.

A100 Memory Estimates

Dry-run estimates from the local config resolver:

Config	Params	Activation MB	Optimizer MB	Checkpoint MB	Pair state MB	1x A100 MB	6x DDP per GPU MB
debug	3,861,872	8.000	29.464	7.366	0.061	44.830	44.830
full	3,849,584	16.000	29.370	7.342	0.061	52.712	52.712
no-pair	3,849,584	8.000	29.370	7.342	0.061	44.712	44.712

These are planning estimates only, not measured A100 memory. They exclude data loader overhead, CUDA workspaces, and framework fragmentation.

Exact Command Helper

The print-only helper emits the 1x debug, 6x full, and one-GPU-per-ablation commands without submitting jobs:

PREPARED_ROOT=/abs/prepared/algonauts_cneuromod \
RUN_ROOT=/abs/persistent/neurotwin_pair_operator \
CONFIG_ROOT=/abs/persistent/neurotwin_pair_operator/configs \
PHASE1_EVAL_DIR=/abs/persistent/neurotwin_pair_operator/eval/phase1_paper_mode \
bash scripts/print_pair_operator_a100_commands.sh

The launchers require materialized absolute config paths and persistent absolute RUN_ROOT. The full 6-GPU path does not run paper-mode inside the full allocation unless A100_RUN_PAPER_MODE_IN_FULL=1.

Failure Modes

• Pair state does not beat the no-pair ablation on any audited fMRI task.

• BrainVista-style local approximation beats Pair-Operator on rollout.

• Stimulus evidence remains plumbing-only because source artifacts cannot be verified.

• A required task is quarantined due to non-finite loss, gradient, or metric.

• Structured baseline suite is missing or contains only unavailable rows.

• Pair confidence becomes uninterpretable or unstable under 1000 parcels.

• Uncertainty is finite but uncalibrated or anticorrelated with error.

• The full DDP run diverges although the 1x debug run passes.

Paper Claim Criteria

An architecture-paper claim is allowed only if all are true:

1. Real prepared Algonauts/CNeuroMod manifests are used.

2. Splits are leakage-audited and subject/site/dataset policy is claim-eligible.

3. Stimulus features are source-artifact verified when stimulus claims are made.

4. Required tasks complete without quarantine.

5. Structured baseline rankings exist for the required tasks.

6. Pair-Operator beats strong baselines or its no-pair ablation on at least one core fMRI/stimulus task under the audited split.

7. Model card reports split policy, leakage audit, stimulus evidence, identity risk, uncertainty diagnostics, and final evidence gate.

8. Claims avoid first/SOTA/clinical language.

Kill Criteria

Kill or demote the architecture paper if any are true:

• no-pair ablation matches or beats Pair-Operator across core tasks

• Pair-Operator only wins under invalid or ambiguous splits

• stimulus-to-fMRI evidence depends on transcript hashes, synthetic embeddings, or self-attested embedding hashes

• any required task is quarantined in the final run

• final evidence gate fails

• baseline ranking is unavailable or CSV-only

• pair state causes unacceptable memory/runtime overhead at 1000 parcels

• the best story is reproducibility/leakage hygiene rather than architecture

If killed, fall back to the Track A reproducibility paper: leakage demos, model-card reporting, identity risk, and executable claim gates.