Deterministic Research Pipelines: Seeds, Fingerprints and Typed Configuration

Why determinism is a research requirement

A research result is a claim about a computation. If the computation cannot be repeated, the claim cannot be checked. In practice, three things break repeatability: unrecorded parameters, random number generators that depend on global state, and intermediate artifacts that are overwritten. This article describes a design that avoids all three with modest overhead.

Principle 1: configuration is a typed, immutable object

Every experiment is described by one configuration object. Keep it typed, validated and frozen once a run starts. Scattered command-line flags and module-level constants are where results go to become unreproducible.

export interface ExperimentConfig {
  readonly name: string;
  readonly seed: number;
  readonly nObservations: number;
  readonly nFeatures: number;
  readonly processVariance: number;
  readonly observationVariance: number;
}

export const baseline: Readonly<ExperimentConfig> = Object.freeze({
  name: "kalman-sweep",
  seed: 20260827,
  nObservations: 2_000,
  nFeatures: 1,
  processVariance: 0.01,
  observationVariance: 1.0,
});

Principle 2: derive every random stream from the seed

Global random state makes results depend on execution order. Instead, create a generator for each logical component, derived from one root seed. NumPy's SeedSequence does this cleanly: each component gets a reproducible, statistically independent stream.

import numpy as np

def fold_rng(root_seed: int, fold: int) -> np.random.Generator:
    """Independent, reproducible generator for one cross-validation fold."""
    seq = np.random.SeedSequence(entropy=root_seed, spawn_key=(fold,))
    return np.random.default_rng(seq)

rng_fold_0 = fold_rng(20260827, 0)
rng_fold_1 = fold_rng(20260827, 1)  # unaffected by how many draws fold 0 consumed

Adding a new fold, or reordering folds, no longer changes the numbers in the other folds.

Principle 3: fingerprint the configuration

A short, stable hash of the canonical configuration serves as an identifier for every artifact. If two runs share a fingerprint, they ran the same experiment; if they differ, the cache must not return the old result.

import hashlib
import json

def config_fingerprint(cfg: dict) -> str:
    """Stable identifier for a configuration. Key order does not affect the result."""
    canonical = json.dumps(cfg, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(canonical.encode("utf-8")).hexdigest()[:16]

print(config_fingerprint({"seed": 20260827, "q": 0.01, "r": 1.0}))

Store artifacts under their fingerprint, for example artifacts/kalman-sweep/3f9a1c0b7e2d4a58/metrics.json. Results are then immutable: a new configuration produces a new path, and nothing is overwritten silently.

Principle 4: make the environment part of the record

Log the exact library versions, the commit hash of the code, and the hardware used for timing measurements. A benchmark number without its environment is an anecdote.

python -c "import numpy, sklearn; print(numpy.__version__, sklearn.__version__)"
git rev-parse --short HEAD

Verification

A pipeline is deterministic when running it twice produces byte-identical artifacts. Make that a test. Run the experiment, hash the output, run it again in a clean environment, and compare hashes. Failures usually reveal an unseeded generator or a timestamp leaking into an output file.

Determinism does not make a method correct. It makes the method's behavior checkable, and that is the precondition for every other form of validation.

@misc{research2026deterministic,
  author       = {{Valence Quantitative Research}},
  title        = {Deterministic Research Pipelines: Seeds, Fingerprints and Typed Configuration},
  howpublished = {Valence Quantitative Research LP},
  year         = {2026},
  url          = {https://readscripts.online/blog/deterministic-research-pipelines/}
}