V&V in Campaigns
pyFrost-GM is a Python reimplementation of the LoKI-GM global model. It is verified and validated through a series of campaigns. Each campaign is a complete, reproducible pressure sweep against a fixed validation target, and each one widens what is covered: more of the codebase, more of the physics, more independent solvers. The residuals a campaign leaves behind define the scope of the next one.
This campaign approach is not specific to global models: it can be applied to any open-source scientific codebase. pyFrost-GM is the worked example.
The Vision
A simulation result on its own answers nothing. Asking why it differs from a measurement only makes sense in context: which reference it is compared against, which solver produced it, what the code was assumed to do, and what has already been ruled out. The campaigns build that context, one layer at a time.
pyFrost-GM is a Python reimplementation of LoKI-GM, IST Lisbon's global model for low-temperature plasma chemistry. It is built to be extended: the global-model solver does not depend on which Boltzmann solver computes the electron energy distribution, the chemistry is read from input files rather than written into the code, and every validation sweep is scripted and reproducible. Each campaign uses that design to add one thing at a time — a new solver, a new physical coupling, a new reference — and to measure what changes.
How a Campaign Works
What Each Campaign Covers
| Campaign 1 | Campaign 2 | Campaign 3 (planned) | |
|---|---|---|---|
| Validation target | Dias 2023: 7 figures (E/N, ne, Tg, O(³P), O₂(a¹Δg), O(³P) pathways, VDF). Alves 2026: ne | Same targets, plus solver-vs-solver comparison | Nitrogen discharges; the remaining oxygen items (below) |
| Boltzmann (EEDF) solver | LoKI-B (two-term) | LoKI-B, MultiBolt N=2, MultiBolt N=10 | LoKI-B and MultiBolt, with rotational (CAR) losses added to MultiBolt |
| Physics coupling | EEDF computed without excited-state feedback | Electronic and vibrational populations fed back into the EEDF; V–T/V–V rates refreshed at each restart | Surface chemistry (wall recombination and quenching) |
| Codebase coverage | Sweep harness, one verification script per figure | Backend-agnostic EEDF interface; MultiBolt given exactly LoKI-B’s Boltzmann problem; rate coefficients integrated over the EEDF as in LoKI-GM; 4 coupling defects fixed; 4.4× faster pressure points | LoKI-GM (MATLAB) baseline rerun at the paper’s conditions |
| Headline result | E/N mean error 1.5 %; ne mean error 1.3 % | Mean deviation from the paper (LoKI-B): E/N 0.9 %, Tg 0.3 %, O(³P) 1.8 %, O₂(a¹Δg) 1.8 %. MultiBolt N=2 within 0.5 % of LoKI-B; N=10 1.7–4.3 % less field than N=2 | Planned |
Campaign Overview
Campaign 1: Baseline Validation
COMPLETEDThe first full sweep with the LoKI-B solver. E/N, ne, Tg and O₂(a¹Δg) reproduced the reference to within a few percent on average. Atomic oxygen O(³P) came out 9–20 % low at every pressure, and that residual was carried forward.
View Campaign 1 Report → | Methodology →Campaign 2: Two Boltzmann Solvers
COMPLETEDA second, independent Boltzmann solver (MultiBolt, N=2 and N=10) ran alongside LoKI-B, with excited-state populations fed back into the EEDF. pyFrost-GM reproduces the paper to a mean of 1–2 % with LoKI-B, MultiBolt N=2 agrees with LoKI-B to within 0.5 %, and N=10 isolates the multi-term effect: 1.7–4.3 % less field. A pressure point runs in 7.9 minutes at 1 Torr, quicker than the MATLAB reference code.
View Campaign 2 Report → | Methodology →Campaign 3: Nitrogen and Surface Chemistry
PLANNEDThe four hypotheses first planned here (cross-section inputs, V–T rate temperature, reaction binding, wall loss) were tested and resolved within Campaign 2. Campaign 3 extends validation to nitrogen and surface chemistry, and closes the remaining oxygen items: the deviation at 0.19 Torr and a LoKI-GM baseline at the paper’s conditions.
View Campaign 3 Plan →Tracked Residuals
Known differences from the reference, followed across campaigns.
| Residual | Campaign 1 | Campaign 2 | Campaign 3 |
|---|---|---|---|
| O(³P) density low | Open: 9–20 % low | Closed: mean 1.8 %, within 5.4 % (LoKI-B) | — |
| O₂(X, v=1) population | Within 7 % | Closed: v=1/v=0 within 12 % (V–T rates now refreshed at each restart) | — |
| MultiBolt vs LoKI-B E/N | — | Closed: N=2 within 0.5 % of LoKI-B (MultiBolt now solves LoKI-B’s Boltzmann problem); N=10 1.7–4.3 % below N=2 | — |
| O₂(a¹Δg) with MultiBolt | — | Closed: three reactions had no rate on the MultiBolt path; now mean 1.8 % (N=2), 2.5 % (N=10) | — |
| O(³P) pathways: R6/R7 crossing | Close: 0.95 Torr (paper ~1.45) | Closed: 1.46 Torr (paper 1.4), counted in O atoms as the paper does | — |
| Lowest pressure (0.19 Torr) | — | Open: O₂(a¹Δg) 14 % low, E/N 3.8 % high, with every solver | To investigate |
Candidate Future Campaigns
Beyond Campaign 3, these are directions pyFrost-GM can grow in. None is scheduled; each would follow the same four steps, against its own validation target.
Interested in one of these for your own work? Get in touch.
Need a cloud-ready, validated translation of your scientific code?
Whether it's investigating specific plasma chemistry questions or adopting these AI/V&V methodologies for your own tools, we can help build your auditable, highly deployable architecture.
Contact Us for a Consultation