FFCapital

The content on this page was written by AI under human supervision.

FFCapital recovers exact results from the saved state of a FireFly rational-function reconstruction that stopped before it finished, for example because the reduction job ran out of memory or the machine went down. The input is the ff_save/ directory that Kira with FireFly writes while reducing a family of Feynman integrals. The output is the exact rational function for every coefficient that had already finished, each checked against an independent value stored in the same directory. FFCapital only reads what is already on disk and never resumes or advances the reduction.

What it does

In an IBP reductionintegration-by-parts identities relate the many integrals of a Feynman family to a small basis; solving that large linear system is the reduction each coefficient is a rational function of the spacetime dimension $d$ and the kinematic variables. Kira with FireFly finds it by sampling the linear system modulo large primes and reconstructing the exact function from the samples. FireFly checkpoints each function in ff_save/states/<n>_<tag>.gz; a state file with is_done equal to 1 already holds the complete exact numerator and denominator coefficients. FireFly also writes ff_save/validation.gz: one probe point and the value of every function there, computed from the linear system directly rather than from the reconstruction. Both files survive a crash.

FFCapital rebuilds $N/D$ for each finished function and accepts it only if $N/D$ at the probe point reproduces the stored value modulo the FireFly prime. The prime is detected from FireFly's table of 63-bit primes and must come out the same for a sample of up to eight finished functions; otherwise the run stops with an error instead of guessing. A function that fails is marked in the output and named on standard error, the others are unaffected, and the exit code is nonzero.

The package has three scripts, and the shape of the saved run decides which one to use. harvest_ffsave_eta.py handles the one-variable case: $d$ was fixed to a rational number and one kinematic variable, eta, remains. harvest_ffsave_2var.py handles a save in $(d,\eta)$, optionally with earlier snapshot copies. It also works out which (target integral, master integral) coefficient each FireFly function index is, and certifies that assignment by exact polynomial cross-multiplication against at least two finished reductions of the same system at fixed rational $d$. recon_symbolic_d.py goes the other way: from one-variable extractions of many runs at different rational $d$ ("slices") it fits the $d$-dependence of every coefficient exactly. It uses four more slices than interpolation needs, checks every slice left out of the fit, refits without each fit slice in turn, and compares against a separate multi-prime FireFly state that never enters a fit. Equality is always tested as equality of rational functions, $P\,D_s = \lambda\,Q\,N_s$ with one overall scale $\lambda$, because a slice whose numerator and denominator share a factor stores a reduced but correct function.

All arithmetic on the value path is exact, with no floating-point numbers anywhere. The per-function check compares a function with its own run's probe value, so a run whose linear system was set up wrongly can still pass; only the two-variable and cross-slice scripts test agreement across runs. Do not read a save that a solver process is still writing (FireFly rotates the state files), and do not use the scripts on states with more than two variables. Resuming an unfinished run is FireFly's job; the AMFlow wrapper covers resumable runs.

Examples

Inventory a stopped one-variable run, then extract what finished. From the usage text of harvest_ffsave_eta.py:

harvest_ffsave_eta.py census  --ffsave <dir> --out census.jsonl [--workers 16]
harvest_ffsave_eta.py extract --ffsave <dir> --out funcs.jsonl
                              (--fns 0,17,26683 | --all-done) [--workers 16]
                              [--no-validate]

census writes one JSON line per state file: function index fn, FireFly tag, the done flag, np (how many primes had been combined), and degrees degN, degD. extract writes one line per requested function with N and D as maps from the power of eta to an exact string "p/q", plus a validated field naming the prime index used. A function that fails gets "validated": false with the reason, standard error lists the failing indices, and the exit code is 2. --no-validate skips the check and is only for a quick look.

Recover and assign a two-variable save. From the usage text of harvest_ffsave_2var.py:

harvest_ffsave_2var.py salvage \
  --primary <ff_save dir> [--extra label=<ff_save dir> ...] \
  --slice label=<kira_target.m>=<d_num>/<d_den> ... (>=2) \
  [--census <ffsave_degree_census json>]   (degree stats for RESIDUAL_TODO) \
  [--residual-target label=<target file> ...] (staged slices to shrink) \
  --bank <outdir> [--family myfam] [--workers 24] \
  [--min-slice-pass 2] [--kill-rate 0.001]

Each --slice is a finished Kira reduction of the same system at $d =$ d_num/d_den, exported with kira2math; at least two are required. The output directory receives funcs_2var.jsonl.gz (exact numerator and denominator, assigned slot, checks passed), extracted_<family>.m in kira2math format, RESIDUAL_TODO.json and residual_targets/<label>.target listing what still has to be reduced, QUARANTINE.jsonl with one typed record per rejected function, and EXTRACTION_RECEIPT.json summarizing the run. If hard failures exceed --kill-rate (default 0.1%) of the finished functions, the save is treated as contaminated: the recovered reduction is not written, EXTRACTION_RECEIPT.json records the verdict CONTAMINATED_STOP, and the exit code is 3.

Reconstruct the $d$-dependence from many fixed-$d$ runs. From the package guide:

recon_symbolic_d.py control|index|probe3|run --outdir <dir> [--coverage ...]
  [--wave1 ...] [--bank-states ...] [--ff-helper <ReconstHelper.cpp>] [--p2-file ...]
  [--config <json>] [--base <root>] [--skip-banked <prior>]

--wave1 is a directory of per-slice extract outputs named node_eps_<a>_<b>_harvest.jsonl, meaning $d = 4 - 2a/b$ (a _HELDOUT suffix marks a slice used only for checking). --coverage gives each function's $d$-degrees and slice list, --bank-states is the ff_save/states directory of the reference multi-prime run, and --ff-helper is FireFly's ReconstHelper.cpp, read for the prime table. Each path may come from a flag, the --config JSON, or --base; a needed path left unset stops the run with CONFIG: <NAME> unset. Run control first and again after any code change; it needs only --coverage, --ff-helper and --outdir. It slices a known synthetic function at the run's own $d$-grid (the slice names in the coverage file) and pushes it through the same code. Exact recovery is required and each of several deliberately corrupted inputs must fail loudly; the result goes to control_result.json and the exit code is 0 only on PASS. A grid with fewer than 16 slices is too small for the synthetic function and is refused with a CONTROL: message. Then index writes slice_index.json, and run writes shard_NNN.jsonl with one line per function. Each line holds integer $d$-coefficient lists per power of eta in N and D, the slices used, and a status of BANKED when every check passed, or the failed check's name (FIT_FAIL, HELDOUT_FAIL, ...).

Routines

harvest_ffsave_eta.py census — inventory every state file of a one-variable save: done flag, primes combined, degrees.
harvest_ffsave_eta.py extract — exact coefficients for the listed (--fns) or all finished (--all-done) functions, each checked against validation.gz; exit 2 on any mismatch.
harvest_ffsave_2var.py salvage — extract, check, assign to (target, master) slots and certify a $(d,\eta)$ save plus optional snapshots; writes the recovered reduction, the residual to-do list and a quarantine file; exit 3 on CONTAMINATED_STOP.
recon_symbolic_d.py control — synthetic-truth self-check of the cross-slice reconstruction on the run's own slice grid, with negative controls; needs no ff_save/ data and must pass before run.
recon_symbolic_d.py index — build the byte-offset index of the per-slice files (slice_index.json).
recon_symbolic_d.py run — reconstruct the exact $(d,\eta)$ functions in parallel (--workers, --fns, --limit, --no-loo, --skip-banked).
recon_symbolic_d.py probe3 — diagnostic: for a few functions, reports which coefficient FireFly normalized to 1 on each slice, then runs the full reconstruction on them.
selftest.py — --help on both harvest scripts, the named CONFIG refusal of recon_symbolic_d.py control without inputs, then the real control self-check on three generated slice grids (two must pass, a too-small grid must be refused; skipped by name if python-flint is absent). Exits nonzero by design, because the harvest checks need your own ff_save/.

Requirements and source

Python 3; the two harvest scripts use only the standard library, recon_symbolic_d.py also needs python-flint. Every script needs a user-supplied ff_save/ directory from a Kira + FireFly run (only the control self-check runs without one); no sample save is included. Self-test: python3 selftest.py in the package directory. It prints smoke ok for the two --help checks and the CONFIG refusal, then runs the synthetic control self-check of recon_symbolic_d.py for real on generated slice grids with a generated prime-table fixture, printing control ok per grid (or a named skip if python-flint is missing). It then exits nonzero on purpose, its last line saying the harvest checks need your own ff_save/ data. The state format is FireFly's (Klappert and Lange, arXiv:1904.00009; Klappert, Klein and Lange, arXiv:2004.01463); the reductions come from Kira (arXiv:2008.06494). Related: ffsave_degree_census.py on the Kira, FireFly and Fermat page reads the same states for degree statistics, and Numkin sets up the fixed-$d$ slice runs that recon_symbolic_d.py combines. Code: tools/ffcapital/ in BootLoops' bootloops-dev repository (GitHub organization BootLoops-ai), released under the MIT license.

← back to the tools index