Gatekeeper

The content on this page was written by AI under human supervision.

Gatekeeper checks the numerical values of Feynman integrals that go into a fit for an exact formula, and then checks the fitted formula. It scans the value files for duplicated, empty or unreadable data, then accepts a fitted expression only if every value, left out in turn, is reproduced by a fit on the others to a set number of digits. It also carries a parser for AMFlow result files and two small probes, one to test a quadrature rule before a long run and one to choose the rule for a square-root integrand. It is Python on the standard library and mpmath.

What it does

A family of Feynman integrals reduces to a finite set of master integralsthe basis integrals that every integral in the family can be written in terms of; find them and the family is solved. One way to find a master exactly is a value fit. At each order in the dimensional-regularization parameter $\varepsilon$, write it as a combination of known functions with unknown coefficients, solve for them from numerical values at several kinematic points, and recognize them as rationals with PSLQan integer-relation algorithm: given high-precision numbers it finds small integers combining them to zero, which turns a floating-point coefficient into an exact fraction. The master values come from an independent reference evaluator such as AMFlow, one JSON file per master per point. Two failures look like success. The same numbers stored under two master labels, or all-zero coefficients from an evaluator that returned nothing, make the fit report masters that were never computed. Separately, a small residual at the fitting points says nothing about other points.

The integrity scan, gatekeeper.oracle_integrity, reads every *.json under the directories you name, strips the identity fields (tag, k, indices, master, name), and hashes the sorted remainder with SHA-256. Different labels sharing a hash form a duplicate group; files whose coefficients are empty or all below --zero-tol (default $10^{-80}$), and files that do not parse, are listed too. Two file forms are read: the tagged one-record-per-file form, and AMFlow's own solve_integrals output, where every result entry counts as one record tagged <file tag>#<indices>. Coefficient strings go through amf_ball, so Arb balls such as [0.25 +/- 1e-115] parse and the zero test is applied to the midpoint (calling float() on such a string fails or misreads it as zero). A ball whose radius exceeds its midpoint is listed separately under consistent_with_zero and does not change the verdict unless --strict-zero-balls is given. It writes integrity_report.json with a verdict of CLEAN or QUARANTINE_NEEDED and exits with status 0 or 2 to match. With --quarantine it also moves the offenders (all but the first file of each duplicate group, plus the zero and unreadable ones) into a sibling <dir>_QUARANTINE/ folder. It is pure hashing and takes seconds.

The leave-one-out certifier, gatekeeper.heldout_cv, takes each master at one order and drops the flagged files. For every point $P_k$ it fits the coefficients by least squares on the other $N-1$ points, predicts the value at $P_k$, and records $-\log_{10}\lvert\text{predicted}-\text{actual}\rvert$. The master is marked certified only if the smallest of these numbers reaches the threshold (30 digits by default), so the verdict is set by the worst left-out point rather than the average. Only then are the coefficients converted to rationals. For an ill-conditioned basis matrix, auto_condition applies LLLLenstra–Lenstra–Lovász lattice reduction, which finds short integer combinations of a set of vectors; here it exposes exact integer relations among the basis functions and a better-conditioned integer change of basis to the columns and drops exact dependencies among the basis functions (typically shuffle relations among iterated integrals). pslq_with_lll then finds the integer relations where plain PSLQ stalls. A certified verdict is numerical evidence and carries no proof. It is only as strong as the independence of the reference values from the fitted functions, and on unscanned files it means nothing.

The probes subpackage tests a quadrature rule cheaply before a long high-precision run. In a product quadrature each one-dimensional rule has its own level (node count). quad_probe raises one level with the others fixed, so that their errors cancel in consecutive differences, and reports the digits gained per level by that rule alone. Below five it prints METHOD-MISMATCHED, which means the rule is wrong for the integrand and more nodes will not help. ell_subst builds the matching rule for $\int_a^b h(K)\prod_i P_i(K)^{-1/2}\,dK$ with $a$, $b$ simple roots of the polynomial product: Gauss–Chebyshev when no other real root is near the interval, a Jacobi $\mathrm{sn}$ substitution when one real root sits just outside each end, a Jacobi $\mathrm{sn}^2$ substitution when a single real root sits just outside one end only, and kind='unsupported' with a reason otherwise (an interior root, a repeated endpoint root, or two nearby roots on the same side).

Of the smaller parts, amf_laurent parses one entry of an AMFlow solve_integrals result into a dictionary from $\varepsilon$-order to value, reading each coefficient's order as the absolute power (adding leading_order on top double-counts; it warns when the two disagree), and amf_ball / amf_arb read AMFlow's Arb ball strings; the value-file index and the per-order comparison script are described under Routines. Gatekeeper computes no values itself, and its loaders expect files with invariants s and t, an identity field, and a coeffs map from order to {"re": ..., "im": ...} strings.

Examples

Run the self-tests. From the repository's tools/ directory,

python3 gatekeeper/test_gatekeeper.py

prints (T4's diagnostic lines omitted; tests T5b–T12 abridged to ...)

T1 integrity-dedup: PASS  (dup tags=['m00', 'm02'], zero=m04)
T2 heldout-CV: PASS  (min=89.7d  mean=90.0d  rats=['1', '3'])
T3 Lyndon/Witt: PASS  (w3 over 2 letters: 2 Lyndon words; full shuffle basis dim=8, |det T|=144)
T4 ill-cond LLL (before/after via heldout_certify): PASS
T5 pslq_with_lll: PASS  (method=lll, rel=[1, -7, 2, -11, 5, -3])
...
ALL TESTS PASS

T1 writes five small value files, one a copy of another under a different label and one all zero, and requires the scan to flag exactly those two. T2 writes six files from $\zeta_2\log(-s)+3\log^2(-t)$, fits them in the basis $[\zeta_2\log(-s),\,\log^2(-t)]$ with a 40-digit threshold, and requires certification with rationals ['1', '3']. T4 requires LLL conditioning to recover exact integer coefficients from a basis matrix with condition number near $10^{62}$, where plain least squares plus PSLQ finds nothing. T5b checks the AMFlow order convention. T6–T9 check the Arb ball forms: verbatim nonzero balls from fixtures/gatekeeper/ are not flagged, six ways of writing zero are, a ball wider than its midpoint goes to consistent_with_zero, and the command line returns the matching exit codes. T10–T12 read the solve_integrals file form.

Scan a set of value files, then certify a fit. With value files under mdata/ and mdata_nt/ and the basis functions evaluated at the same points in basis_values.json:

python -m gatekeeper.oracle_integrity mdata/ mdata_nt/ \
       --report integrity_report.json [--quarantine]
python -m gatekeeper.heldout_cv \
       --oracle mdata/ mdata_nt/ \
       --basis basis_values.json --weight 2 --gate 30 \
       --integrity integrity_report.json --report cv_report.json

The first command prints the counts of good, duplicate, suspicious-zero, consistent-with-zero and unreadable files with the verdict; resolve or quarantine what it lists before going on. The second excludes the flagged files and runs the leave-one-out fit at order 2. It prints one line per master (CERT or blank, points used, minimum and mean digits, condition number, rationals) and writes cv_report.json with a summary block and the per-master records. basis_values.json is {"names": [...], "points": {"-3|-5": {"B0": "<digits>", ...}, ...}} keyed by "s|t", or the same per order under "weights": {"2": {...}}. From Python:

from gatekeeper import scan_integrity, heldout_certify, load_oracle_values

rep = scan_integrity(["mdata/", "mdata_nt/"])
excl = rep.quarantine_set()                       # files to drop from a fit

vals, kin, _ = load_oracle_values(["mdata/"], exclude_files=excl)
res = heldout_certify(vals, kin, weight=2,
                      basis_fn=lambda pk,s,t: [...], basis_names=[...],
                      gate=30, dps=80,
                      n_duplicates_excluded=len(excl))

res maps each master tag to a CVResult; read .certified, .min, .heldout_digits and .pslq_rationals.

Choose a quadrature rule. In the acceptance tests, $1/\sqrt{(1-x^2)(4-x^2)}$ on $[-1,1]$ has a root at each end and one just outside each end. select therefore returns the Jacobi-sn rule with modulus $1/4$, and plan.integrate() equals the complete elliptic integral $K(1/4)$. With $x^2+4$ in place of $4-x^2$ only the endpoint roots remain, the rule is Gauss–Chebyshev, and plan3.integrate(mp.exp) matches adaptive quadrature to more than 50 digits at 96 nodes.

import mpmath as mp, sympy as sp
from gatekeeper.probes import select
x = sp.symbols('x'); mp.mp.dps = 80
plan = select([1 - x**2, 4 - x**2], -1, 1, N=64, dps=70)      # plan.kind == 'jacobi-sn'
plan3 = select([1 - x**2, x**2 + 4], -1, 1, N=96, dps=70)    # plan3.kind == 'chebyshev'

Each plan carries nodes, weights and a report; for a configuration the selector does not cover, plan.kind is 'unsupported' and plan.report['reason'] says why.

Routines

Command-line entry points

Integrity scan (gatekeeper.oracle_integrity)

Leave-one-out certification (gatekeeper.heldout_cv)

AMFlow result parsing (gatekeeper.amf_result)

Quadrature probes (gatekeeper.probes)

Used on this site

Requirements and source

Python 3 with mpmath; sympy for probes/ell_subst.py and the tests; fpylll optional (fast LLL, pure Python otherwise). Self-tests: python3 tools/gatekeeper/test_gatekeeper.py and python3 tools/gatekeeper/probes/ell_subst_test.py (its T4 uses a reference script that is not included and is skipped unless ELL_KLINE_DIR is set). ctrl_per_order.py --selftest and registry --selftest need your own data (CTRL_TOWERS, CTRL_ORACLE, ORACLE_REG_ROOTS) and skip or fail without it. The code is tools/gatekeeper/ in BootLoops' bootloops-dev repository (GitHub organization BootLoops-ai), released under the MIT license. Nestor's nestor.probe is this quad_probe; the PSLQ and LLL page covers the integer-relation step. Two related tools live in other packages: the singularity-subtracted dispersion quadrature is Nestor's nestor.dispersion module, and the $\varepsilon\to0$ extrapolator is Wayfinder's wayfinder.epslimit.

← back to the tools index