Skip to content

Testing

The suites in the table are registered with ctest; the few that are not are run by hand, as the sections below say. A test that is built but not registered never runs, in CI or locally: after adding one, check that ctest --test-dir build -N lists it.

Suite Source ctest entries Registered when
Library unit tests the files of TEST_SOURCES in tests/ one per test case BUILD_TESTING (on by default)
Component unit tests components/<name>/.../tests/ wmtk_test_<name>, one per component BUILD_TESTING and WMTK_COMPONENT_TESTS (both on by default)
Memory route tests/test_component_io.cpp one per test case as the component tests
Integration tests tests/integration_tests.cpp, configs in data/integration_tests Integration_Tests WMTK_BUILD_INTEGRATION_TESTS=ON
Python tests/python/ python_memory_io, python_tracking, python_tracking_correspondences the Python module is built and its interpreter has NumPy; the last in non-Debug builds only
Documentation examples docs/examples/ doc_example_cpp_<name>, doc_examples_python_json WMTK_BUILD_DOC_EXAMPLES=ON, non-Debug builds only

Library unit tests

wmtk_tests is one Catch2 executable built from the files listed in TEST_SOURCES in tests/CMakeLists.txt: a new test file goes into that list. catch_discover_tests registers every test case as a ctest entry of its own, under its name. The tests link wmtk::data, so they read the data repository through WMTK_DATA_DIR:

const std::string root(WMTK_DATA_DIR);

Small meshes built in code are in src/wmtk/utils/examples/ (TriMesh_examples.hpp: single_triangle(), two_triangles(), ...).

-DWMTK_DEBUG_BRUTE_FORCE_OPS=ON compiles a check into TriMesh's split and collapse that compares each result with one rebuilt from the whole mesh. It makes every operation linear in the mesh size: for tests on small meshes only.

Component unit tests

Each component's tests/CMakeLists.txt calls add_component_test(<name>) (components/cmake/add_component_test.cmake), which builds wmtk_test_<name> and registers it as one ctest entry:

add_test(NAME ${COMPONENT_TEST_NAME} COMMAND ${COMPONENT_TEST_NAME} --allow-running-no-tests --durations yes)

--durations yes prints the time of every case, which is how a case too slow for the Debug jobs shows up in the CI log. To run one case, run the executable with its name or a tag:

ctest --test-dir build -R wmtk_test_qslim --output-on-failure
build/components/qslim/wmtk/components/qslim/tests/wmtk_test_qslim "[boundary]"

wmtk_component_io_tests (tests/test_component_io.cpp, built from components/CMakeLists.txt) tests the memory route: components called on named meshes, and the memory and file routes giving the same result. Each component's cases are compiled only when the component is (#ifdef WMTK_TEST_<name>).

Integration tests

With -DWMTK_BUILD_INTEGRATION_TESTS=ON, wmtk_integration_tests (tests/integration_tests.cpp) runs every config listed in data/integration_tests/integration_tests.json, the way wmtk_app does (the JSON, input_dir set to its directory, the component from the generated map):

for (const auto& input_file : input_files) {
    const path& f = integration_tests_dir / input_file;
    CHECK(fs::exists(f));
    CHECK_NOTHROW(wmtk_wrapper(f));
}

That is the whole check: a config passes if it runs without throwing. No output is compared with a stored one. A config therefore asks its component to throw when the result is wrong: throw_on_fail (the output misses what the component promises: the target vertex count, the stop energy), max_expected_iterations in tetwild and triwild (more optimization iterations than expected), throw_on_nonconvergence in topological_offset. All the configs run in one ctest entry, Integration_Tests, with a timeout of 7200 s.

The manifold_extraction configs have a case of their own, manifold-extraction-models, which reads data/integration_tests/manifold_extraction_models.json. It is not hidden, so it is a ctest entry of its own and runs in CI; to run it alone:

build/tests/wmtk_integration_tests "[manifold]"      # manifold_extraction_models.json

The other cases in the file are hidden ([.]): Catch2 does not run them unless asked, and catch_discover_tests does not register them. Each reads its own list from data/integration_tests:

build/tests/wmtk_integration_tests "[challenging]"   # challenging_low_stop_energy_models.json: minutes to hours each
build/tests/wmtk_integration_tests "[offset]"        # topological_offset_models.json

A new config goes into the data repository and its list, and the data pin moves to the data2 commit that has it (writing a component).

Python tests

The tests in tests/python run the bindings. Four are registered in tests/CMakeLists.txt, when the module is built (WMTK_PYBIND, on by default) and the interpreter CMake found imports NumPy:

ctest entry File What it runs
python_memory_io test_memory_io.py the memory route through the bindings, NumPy only
python_tracking test_tracking.py histories as bytes and files, wm.Tracking, wm.CurveTracking, wm.VolumeTracking
python_pipeline test_pipeline.py, test_gui.py, test_viewer.py wildmeshing.pipeline, wildmeshing.gui and wildmeshing.viewer, with pytest; registered when the interpreter has pytest; the last two skip without polyscope, whose mock backend opens no window
python_tracking_correspondences test_tracking_correspondences.py the tracking tutorial's pipeline (docs/examples/python/tracking_correspondences.py): the dense bunny remeshed, tetrahedralized, and every vertex mapped through both histories and back; its counts are ranges, since a run with threads changes slightly from one run to the next; non-Debug builds only, timeout 1800 s

The others are run by hand or elsewhere:

  • test_docs_examples.py: the documentation's examples, below.
  • test_smoke.py: imports the module and its error paths; the wheel builds run it with test_memory_io.py, test_tracking.py and the three of python_pipeline on every wheel (test-command in pyproject.toml).
  • test_integration.py: the integration configs through the bindings, compared byte for byte with wmtk_app's output when the config is serial (python -m pytest tests/python; see its README.md).
  • test_face_tags.py, test_memory_route.py: the tagged components' face, edge and point tags, and their memory route against the file route; three PolyFEM-reader tests need gmsh.

tests/python/polyfem holds the tests of wildmeshing.polyfem_ops. pytest collects it only when the polyfem extras (gmsh, scipy, h5py) are installed, and its tests that run PolyFEM skip without POLYFEM_BIN.

Documentation examples

With -DWMTK_BUILD_DOC_EXAMPLES=ON, every docs/examples/cpp/<name>.cpp is built as wmtk_doc_example_<name>, linked with every component, and in a non-Debug build registered as doc_example_cpp_<name>; doc_examples_python_json (tests/python/test_docs_examples.py) runs the Python scripts and, through wmtk_app -j, the JSON files. They check that each example runs without an error. Writing documentation has the conventions.

Slow cases

Debug builds run 10 to over 100 times slower than Release. A case that takes more than a fraction of a second in Debug is tagged release_only: hidden in Debug builds, run as usual in Release. The pattern, at the top of the test file:

// Cases too slow for the Debug CI jobs, Windows Debug above all: Release builds only.
#ifdef NDEBUG
const std::string release_only = "";
#else
const std::string release_only = "[.]";
#endif

and on each slow case:

TEST_CASE("sec-open-surface-boundary-is-preserved", "[test_sec][boundary]" + release_only)

Commit e8494ffb4f introduced it for the 95 cases over 0.3 s in a macOS Debug build or 5 s on Windows Debug, taking a macOS Debug ctest to 27.5 s; tetwild's and simwild's whole runs were already hidden in Debug. A case that should not run anywhere by default, a benchmark on a large model say, gets a hidden tag of its own ([.slow]), and the reason in a comment.

Running the tests locally

cmake -B build -DCMAKE_BUILD_TYPE=Release -DWMTK_BUILD_INTEGRATION_TESTS=ON
cmake --build build -j
ctest --test-dir build -N                                   # what is registered
ctest --test-dir build -E Integration_Tests --output-on-failure
ctest --test-dir build -R Integration_Tests --output-on-failure

The first build clones the data repository into data/.

In CI

.github/workflows/continuous.yml runs on every pull request and every push to main, in nine jobs:

Job (check name) Runner Configs
Linux ubuntu-latest Debug, Release
Linux arm64 ubuntu-24.04-arm Debug, Release
Linux per-input orientation ubuntu-latest Release
macos-latest macos-latest Debug, Release
Windows 2025 (VS 2026) windows-2025-vs2026 Debug, Release

A check is named after its job and config, as in Linux (Release). A new push to a pull request cancels the run of the previous one.

Each job configures with CMAKE_BUILD_TYPE set to its config, builds with as many jobs as the runner has cores (make; Ninja on Windows) behind a compiler cache (ccache; sccache on Windows), and runs ctest --verbose --output-on-failure, one test per core (CTEST_PARALLEL_LEVEL: 4, 3 on macOS). So a test that writes files must write them in a directory of its own, or under names no other test uses; the long ones carry a COST, so they start first. The Debug builds use -g1 (-g makes the cache several times larger), and the Windows builds no debug information at all.

The compiler cache is restored from the newest one of the same runner and config, and saved right after the build, whether the build or the tests pass or not; ccache is first trimmed to the entries the build used, and sccache is capped at 1 GB. The Windows jobs fail when the compile commands do not go through sccache. So:

  • every job runs the library, component and memory-route tests, with the release_only cases in the Release jobs only;
  • the Release jobs add -DWMTK_BUILD_INTEGRATION_TESTS=ON and -DWMTK_BUILD_DOC_EXAMPLES=ON, and run every listed integration config and the documentation's C++ examples;
  • the Linux jobs install python3-numpy, so they run the registered Python tests; the Linux arm64 (Release) job also runs the Python and JSON examples (Linux (Release) leaves them out with ctest -E: they were its longest test). The macOS and Windows jobs run the Python tests and examples if the interpreter CMake finds has NumPy.

When the tests fail on Linux or macOS, a last step re-runs the component test executables under gdb or lldb and prints the backtrace of every thread, for a crash that leaves only a signal in the log. A manual run of the workflow (workflow_dispatch) with the debug_ssh input opens an SSH session on a Linux or macOS runner when a step fails.

Two other workflows run on every pull request: pre-commit.yml checks the formatting (contributing), and docs.yml checks the specs (gen_reference.py --check) and builds the site with --strict (writing documentation).