Code testing

To ensure the integrity of chombo-discharge, we include tests in

$DISCHARGE_HOME/Exec/Tests for a functional test suite.
$DISCHARGE_HOME/Exec/Convergence for a code verification tests.

Test suite

To make sure chombo-discharge compiles and runs as expected, we include a test suite that runs functional tests of many chombo-discharge components. The tests are defined in $DISCHARGE_HOME/Exec/Tests, and are organized by application type.

Running the test suite

To do a clean compile and run of all tests, navigate to $DISCHARGE_HOME/Exec/Tests and execute the following:

python3 tests.py --compile --clean --silent --no_exec -cores X

where X is the number of cores to use when compiling. By default, this will compile without MPI and HDF5 support.

Advanced configuration

The following options are available for running the various tests:

--compile Compile all tests.
--clean Do a clean recompilation.
--silent Turn off terminal output.
--benchmark Generate benchmark files.
--no_exec Compile, but do not run the test.
--compare Run, and compare with benchmark files.
-dim <number> Run only the specified 2D or 3D tests.
-mpi <true/false> Use MPI or not.
-hdf <true/false> Use HDF5 or not.
-cores <number> Run with specified number of cores.
-suites <string> Run a specific application test suite.
-tests <string Run a specific test.

For example, to compile with MPI and HDF5 enabled:

python3 tests.py --compile --clean --silent --no_exec -mpi=true -hdf=true -cores 8

If one only wants to compile a specified test suite:

python3 tests.py --compile --clean --silent --no_exec -mpi=true -hdf=true -cores 8 -suites Electrostatics

One can also restrict the tests to specific dimensionality, e.g.

python3 tests.py --compile --clean --silent --no_exec -mpi=true -hdf=true -cores 8 -suites Electrostatics -dim=2

Using benchmark files

The test suite can generate benchmark files which can later be compared against new test suite output files. This is often a good idea if one wants to ensure that modifications to the chombo-discharge source code does not unintentionally change the output of computer simulations. In this case one can run the test suite and generate benchmark files before adding changes to chombo-discharge. Once the code development is completed, the benchmark files can later be bit-wise (using h5diff) compared against the results of a later test suite.

This consists of the following steps:

Before making changes to chombo-discharge, generate benchmark files with

python3 tests.py --compile --clean --silent --benchmark -mpi=true -hdf=true -cores 8

Make the required changes to the chombo-discharge code.

Run the test suite again, and compare benchmark and output files as follows:

python3 tests.py --compile --clean --silent --compare -mpi=true -hdf=true -cores 8

When running the tests this way, the output files are bit-wise compared and a warning is issued if the files do not exactly match.

Automated testing

On GitHub, the test suite is integrated with GitHub actions and are automatically run when opening a pull request for review. In general, all tests must pass before a pull request can be merged. The test status can be observed either in the pull request, or at https://github.com/chombo-discharge/chombo-discharge/actions. The automated tests run chombo-discharge with DEBUG=TRUE and OPT=FALSE in order to catch assertion errors or other places where the code might break. They usually take around 1 hour to complete.

The automated tests will clone, build, and run the chombo-discharge test suite for various configurations:

Parallel and serial.
With or without HDF5.
In 2D and 3D.

The tests are run with the following compiler suites:

GNU.
Intel oneAPI.

Convergence testing

To ensure that the various components in chombo-discharge converge at desired truncation order, many modules are equipped with their own convergence tests. These are located in $DISCHARGE_HOME/Exec/Convergence. The tests are too extensive to include in continuous integration, and they must be run locally like a regular chombo-discharge application. Our approach for convergence testing is found in Verification and validation.

Performance profiling

There are two ways to run performance profiling of chombo-discharge:

A posteriori profiling using Chombo macros. Most routines in chombo-discharge use these macros and they will compute the wall clock time spent in each routine.

To enable these timers, set CH_TIMER=1 in the shell where you run your application. E.g,
```
export CH_TIMER=1
```
The Chombo will not only compute the time spent in each function, but also figure out the parent-child relation between function, and present the output as a hierarchical structure. This is often useful when optimizing functions at the development stage.

Warning

Chombo’s timers are not meant to use with many time steps. For efficient use, it is best to use it for a single time step.
In-place profiling using the chombo-discharge Timer class (see https://chombo-discharge.github.io/chombo-discharge/doxygen/html/classTimer.html for the C++ API).

Some classes in chombo-discharge use the Timer class, and can be used for run-time evaluations of function costs and load imbalance.

Warning

The Timer class incurs large performance penalties at high concurrencies (1K CPU cores and above).