This PR fixes some process signals that were incorrectly numbered.
From what I can tell, the code used signals and signal numbers for
Alpha/SPARC, not x86/ARM. The test was also broken and always green,
hiding the mistake.
The tests need to run with certain environment variables set that only
cmake really knows and that differ between stages. Cmake could just set
the variables directly when running the tests and benchmarks, but that
would leave no good way to manually run a single benchmark. So cmake
generates some stage-specific scripts instead that set the required
environment variables.
Previously, those scripts were sourced directly by the individual
`run_*` scripts, so the env scripts of different stages would overwrite
each other. This PR changes the setup so they can instead be generated
next to each other. This also simplifies the `run_*` scripts themselves
a bit, and makes `tests/bench/build` less of a hack.
This PR extracts the example programs from the sym mvcgen benchmarks
into
shared `Cases.*` modules so that both benchmarks and a new fast test
suite
can reuse them. It also renames `vcgen_deep_add_sub_cancel` to
`vcgen_add_sub_cancel_deep` for consistency.
The test suite (`test_vcgen.lean`) runs all cases at n=10, completing in
~2s vs minutes for the full benchmarks. It is wired up as a `lake test`
driver and integrated with the lean4 test/bench infrastructure via
`run_test`/`run_bench` scripts registered in `CMakeLists.txt`.
Benchmark output now uses aligned `CaseName(n):` labels. The `run_bench`
script extracts per-case vcgen and kernel timings into
`measurements.jsonl`.
Benchmarks run single-threaded (`LEAN_NUM_THREADS=1`) for
reproducibility.
`vcgen_get_throw_set` is excluded from benchmarks due to pathological
`instantiateMVars` behavior.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This PR adds a `lake-ci` label that enables the full Lake test suite in
CI,
avoiding the need to temporarily commit and revert changes to
`tests/CMakeLists.txt`. The `lake-ci` label implies `release-ci` (check
level
3), so all release platforms are also tested.
Motivated by
https://github.com/leanprover/lean4/pull/12540#issuecomment-4000081071
where @tydeu requested running `release-ci` with Lake tests enabled,
which
previously required temporarily uncommenting a line in
`tests/CMakeLists.txt`.
Users can add it via a PR comment containing `lake-ci` on its own line,
or by
adding the label manually. CI automatically restarts when the label is
added.
Implementation:
- `ci.yml`: detect `lake-ci` label, set check level 3, pass
`-DLAKE_CI=ON` to cmake
- `tests/CMakeLists.txt`: `option(LAKE_CI ...)` conditionally enables
full `tests/lake/tests/` glob
- `restart-on-label.yml`: restart CI on `lake-ci` label
- `labels-from-comments.yml`: support `lake-ci` comment
🤖 Prepared with Claude Code
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This PR migrates most remaining tests to the new test suite. It also
completes the migration of directories like `tests/lean/run`, meaning
that PRs trying to add tests to those old directories will now fail.
MacOS uses a very old version of bash where `"${FOO[@]}"` fails if `set
-u` is enabled and `FOO` is undefined. Newer versions of bash expand
this to zero arguments instead.
Also, `lint.py` used the shebang `#!/usr/bin/env python` instead of
`python3`, which fails on some systems.
In CI, all macos tests run on nscloud runners. Presumably, they have
installed newer versions of various software, hence this didn't break in
CI.
The linter was running in parallel with other tests, which were creating
and deleting files. Since the linter was iterating over some files and
directories at the time, it crashed.
This PR sets up the new integrated test/bench suite. It then migrates
all benchmarks and some related tests to the new suite. There's also
some documentation and some linting.
For now, a lot of the old tests are left alone so this PR doesn't become
even larger than it already is. Eventually, all tests should be migrated
to the new suite though so there isn't a confusing mix of two systems.
In preparation for adding the bench suite to the cmake-based test suite,
this PR moves test-related cmake code into the `tests` directory.
It also fixes a warning by removing an obsolete bit of the cmake code.
