Gatekeeper

Gatekeeper is a Python package that checks the numerical values of Feynman integrals that go into a fit for an exact formula, and then checks the fitted formula. It scans the value files for duplicated, empty or unreadable data, and accepts a fit only if each value, left out in turn, is reproduced from the others to a set number of digits. The fit it checks is the method of Barrera, Dersy, Husain, Schwartz and Zhang (arXiv:2507.17815).

What it does

A family of Feynman integrals reduces to a finite set of master integralsthe basis integrals that every integral in the family can be written in terms of; find them and the family is solved. One way to find a master exactly is a value fit. At each order in the dimensional-regularization parameter $\varepsilon$, write it as a combination of known functions with unknown coefficients and solve for them from numerical values at several kinematic points. Then recognize the coefficients as rationals with PSLQan integer-relation algorithm: given high-precision numbers it finds small integers combining them to zero, which turns a floating-point coefficient into an exact fraction. The master values come from a reference evaluator such as AMFlow, one JSON file per master per point.

Two failures look like success. The same numbers stored under two master labels, or all-zero coefficients from an evaluator that returned nothing, make the fit report masters that were never computed. Separately, a small residual at the fitting points says nothing about other points.

The integrity scan, gatekeeper.oracle_integrity, reads every *.json under the directories you name.

The leave-one-out certifier, gatekeeper.heldout_cv, takes each master at one order and drops the flagged files.

The probes, gatekeeper.probes, test a quadrature rule (the nodes and weights of a numerical integration) before a long high-precision run.

Other real roots of the productRule
None near the intervalGauss–Chebyshev
One just outside each endA Jacobi $\mathrm{sn}$ substitution
A single one just outside one end onlyA Jacobi $\mathrm{sn}^2$ substitution
An interior root, a repeated endpoint root, or two nearby roots on the same sideNone: kind='unsupported' with a reason

The parser, gatekeeper.amf_result, reads AMFlow result files. amf_laurent parses one entry of a solve_integrals result into a dictionary from $\varepsilon$-order to value. It reads each coefficient's order as the absolute power, because adding leading_order on top double-counts, and it warns when the two disagree. amf_ball and amf_arb read the Arb ball strings.

Gatekeeper has these limits:

Nestor's nestor.probe is this quad_probe, and the PSLQ and LLL page covers the integer-relation step. Two related tools live in other packages: the singularity-subtracted dispersion quadrature is Nestor's nestor.dispersion module, and the $\varepsilon\to0$ extrapolator is Wayfinder's wayfinder.epslimit.

Examples

Run the self-tests. With Python, mpmath and sympy installed (see Requirements), run from the repository's tools/ directory:

python3 gatekeeper/test_gatekeeper.py

It prints (T4's diagnostic lines omitted; tests T5b–T12 abridged to ...):

T1 integrity-dedup: PASS  (dup tags=['m00', 'm02'], zero=m04)
T2 heldout-CV: PASS  (min=89.7d  mean=90.0d  rats=['1', '3'])
T3 Lyndon/Witt: PASS  (w3 over 2 letters: 2 Lyndon words; full shuffle basis dim=8, |det T|=144)
T4 ill-cond LLL (before/after via heldout_certify): PASS
T5 pslq_with_lll: PASS  (method=lll, rel=[1, -7, 2, -11, 5, -3])
...
ALL TESTS PASS

T1 writes five small value files, one a copy of another under a different label and one all zero, and requires the scan to flag exactly those two. T2 writes six files from $\zeta_2\log(-s)+3\log^2(-t)$, fits them in the basis $[\zeta_2\log(-s),\,\log^2(-t)]$ with a 40-digit threshold, and requires certification with rationals ['1', '3']. The other tests cover the conditioning step, the AMFlow order convention, the Arb ball forms and the solve_integrals file form.

Scan a set of value files, then certify a fit. With value files under mdata/ and mdata_nt/ and the basis functions evaluated at the same points in basis_values.json:

python -m gatekeeper.oracle_integrity mdata/ mdata_nt/ \
       --report integrity_report.json [--quarantine]
python -m gatekeeper.heldout_cv \
       --oracle mdata/ mdata_nt/ \
       --basis basis_values.json --weight 2 --gate 30 \
       --integrity integrity_report.json --report cv_report.json

The first command prints the counts of good, duplicate, suspicious-zero, consistent-with-zero and unreadable files with the verdict. Resolve or quarantine what it lists before going on. The second excludes the flagged files, runs the leave-one-out fit at order 2, and prints one line per master: CERT or blank, points used, minimum and mean digits, condition number, rationals. It writes cv_report.json with a summary block and the per-master records.

basis_values.json is {"names": [...], "points": {"-3|-5": {"B0": "<digits>", ...}, ...}} keyed by "s|t", or the same per order under "weights": {"2": {...}}.

The same from Python.

from gatekeeper import scan_integrity, heldout_certify, load_oracle_values

rep = scan_integrity(["mdata/", "mdata_nt/"])
excl = rep.quarantine_set()                       # files to drop from a fit

vals, kin, _ = load_oracle_values(["mdata/"], exclude_files=excl)
res = heldout_certify(vals, kin, weight=2,
                      basis_fn=lambda pk,s,t: [...], basis_names=[...],
                      gate=30, dps=80,
                      n_duplicates_excluded=len(excl))

res maps each master tag to a CVResult. Read .certified, .min, .heldout_digits and .pslq_rationals.

Choose a quadrature rule. Two integrands on $[-1,1]$ from the acceptance tests:

import mpmath as mp, sympy as sp
from gatekeeper.probes import select
x = sp.symbols('x'); mp.mp.dps = 80
plan = select([1 - x**2, 4 - x**2], -1, 1, N=64, dps=70)      # plan.kind == 'jacobi-sn'
plan3 = select([1 - x**2, x**2 + 4], -1, 1, N=96, dps=70)    # plan3.kind == 'chebyshev'

$1/\sqrt{(1-x^2)(4-x^2)}$ has a root at each end and one just outside each end. select therefore returns the Jacobi-sn rule with modulus $1/4$, and plan.integrate() equals the complete elliptic integral $K(1/4)$. With $x^2+4$ in place of $4-x^2$ only the endpoint roots remain, the rule is Gauss–Chebyshev, and plan3.integrate(mp.exp) matches adaptive quadrature to more than 50 digits at 96 nodes.

Each plan has nodes, weights and a report. For a configuration the selector does not cover, plan.kind is 'unsupported' and plan.report['reason'] says why.

Routines

The python -m commands run from the repository's tools/ directory. Paths that begin with tools/ are relative to the repository root.

CommandWhat it does
python -m gatekeeper.oracle_integrity DIR ... [--report FILE] [--quarantine] [--zero-tol X] [--strict-zero-balls]The integrity scan; exit status 0 if clean, 2 otherwise
python -m gatekeeper.heldout_cv --oracle DIR ... --basis FILE --weight W [--gate D] [--integrity FILE]Leave-one-out certification at one order; scans first if no integrity report is given
python -m gatekeeper.probes.quad_probe --workdir DIR --module MOD --func FN --base DPS=26,LEVEL_K=4 --chains LEVEL_K [--span 2]Convergence probe, one rule at a time; exit status 1 on METHOD-MISMATCHED
python -m gatekeeper.registry with --rebuild, --point s,t --masters lo..hi or --dupesIndexes the stored value files, lists the masters that have values at a point, or lists duplicate files
python3 tools/gatekeeper/ctrl_per_order.py --towers FILE --oracle FILE [--masters lo:hi]Digits of agreement per master and order between differential-equation boundary values and reference values
tools/quad_probe.py, tools/ell_subst.py, tools/oracle_registry.pyFlat-path scripts that forward to the package

quad_probe passes the levels to your module as environment variables, and the registry takes its roots from ORACLE_REG_ROOTS. ctrl_per_order.py reports ALIAS-LIMITED with the highest usable order when the digits lost per order match $\lvert\log_{10}\varepsilon_0\rvert$ (the eps0_log10 value stored in the towers file), the pattern of series truncation rather than insufficient precision.

These names import from gatekeeper:

FunctionWhat it returns
scan_integrity(dirs, zero_tol, strict_zero_balls)An IntegrityReport: the lists of flagged files and the verdict; .quarantine_set() lists the files to exclude
load_oracle_values(dirs, exclude_files)values[tag][point][order] and kinematics[point], with each point keyed by s and t joined by a vertical bar
heldout_certify(values, kinematics, *, weight, basis_fn, basis_names, gate, dps){tag: CVResult}, each with certified, min, mean, heldout_digits, pslq_rationals, cond_before, cond_after, pslq_method and note
lyndon_basis(alphabet, weight)The well-conditioned Lyndon-word basis of the shuffle algebra at that weight, with its rational transition matrix
auto_condition(basis_matrix, target_cond)The LLL column reduction of an ill-conditioned basis matrix
pslq_with_lll(values, prec, max_coeff_bits)(relation, method): an integer relation by LLL, with PSLQ as fallback
amf_laurent(result_entry){absolute_order: mpc} from one solve_integrals entry
amf_arb(v), amf_ball(v)The midpoint of an Arb ball string at the caller's precision; the midpoint and radius as decimal strings

These import from gatekeeper.probes:

FunctionWhat it returns
probe(evaluator, base_levels, chains, span)Per rule: levels, differences, digits per level, and the at_noise_floor and mismatched flags
EnvEvaluator(workdir, module, func)An evaluator that runs module.func in parallel subprocesses with the levels as environment variables
select(factors, a, b, N, dps)A SubstPlan with kind, nodes, weights, m, report, .integrate(h) and .log_frame()
cubic_under_sqrt_K(a3, a2, a1, a0)$\int dz/\sqrt{\text{cubic}}$ between its two lowest real roots, in closed form through $K(m)$

.log_frame() splits off endpoint logarithms. It is defined for the Chebyshev and sn kinds only and raises for jacobi-sn2. The exported names are listed in __init__.py and probes/__init__.py, and the full argument lists are in the docstrings.

Requirements

Source and references

The code is tools/gatekeeper/ in the BootLoops repository, released under the MIT license.

ReferenceWhat it supplies
O. Barrera, A. Dersy, R. Husain, M. D. Schwartz and X. Zhang, "Analytic regression of Feynman integrals from high-precision numerical sampling", arXiv:2507.17815The value fit that the certifier checks
H. R. P. Ferguson, D. H. Bailey and S. Arno, "Analysis of PSLQ", Math. Comp. 68 (1999) 351The PSLQ algorithm
F. Johansson and the mpmath contributors, mpmathPSLQ as called here, and the arbitrary-precision arithmetic
A. K. Lenstra, H. W. Lenstra Jr. and L. Lovász, Math. Ann. 261 (1982) 515; the FPLLL development team, fplllLLL lattice reduction, run through fpylll when it is installed
X. Liu and Y.-Q. Ma, AMFlow, arXiv:2201.11669; the AMFlow.cpp contributors, AMFlow.cppThe result files that the scan and the parser read
Duval's algorithm; Radford's theoremThe Lyndon words and the Lyndon basis of the shuffle algebra in lyndon_basis
Gradshteyn and Ryzhik, formula 3.131The closed form in cubic_under_sqrt_K and the substitution of the $\mathrm{sn}^2$ rule

← back to the tools index