mirror of
https://github.com/leanprover/lean4.git
synced 2026-03-17 10:24:07 +00:00
49715fe63c
The tests need to run with certain environment variables set that only cmake really knows and that differ between stages. Cmake could just set the variables directly when running the tests and benchmarks, but that would leave no good way to manually run a single benchmark. So cmake generates some stage-specific scripts instead that set the required environment variables. Previously, those scripts were sourced directly by the individual `run_*` scripts, so the env scripts of different stages would overwrite each other. This PR changes the setup so they can instead be generated next to each other. This also simplifies the `run_*` scripts themselves a bit, and makes `tests/bench/build` less of a hack.
30 lines
971 B
Bash
30 lines
971 B
Bash
rm -f measurements.jsonl
|
|
|
|
# Build dependencies first so their compilation isn't measured.
|
|
lake build Cases Driver Baseline
|
|
|
|
# Run benchmarks single-threaded for reproducible measurements.
|
|
LEAN_NUM_THREADS=1 lake build VCGenBench 2>&1 | tee vcgen.out
|
|
LEAN_NUM_THREADS=1 lake build BaselineBench 2>&1 | tee -a vcgen.out
|
|
|
|
# Parse lines like:
|
|
# AddSubCancel(1000): 528 ms, 1 VCs by grind: 245 ms, kernel: 446 ms
|
|
# into JSONL measurements.
|
|
python3 -c "
|
|
import json, re, sys
|
|
|
|
for line in open('vcgen.out'):
|
|
m = re.search(r'(\w+)\((\d+)\):\s+(\d+) ms.*kernel:\s+(\d+) ms', line)
|
|
if not m:
|
|
continue
|
|
case, n, vcgen_ms, kernel_ms = m.group(1), m.group(2), m.group(3), m.group(4)
|
|
for phase, val in [('vcgen', vcgen_ms), ('kernel', kernel_ms)]:
|
|
print(json.dumps({
|
|
'metric': f'mvcgen/sym/{case}/{n}/{phase}//wall-clock',
|
|
'value': int(val) / 1000,
|
|
'unit': 's'
|
|
}))
|
|
" >> measurements.jsonl
|
|
|
|
rm -f vcgen.out
|