Code testing

To ensure the integrity of chombo-discharge, we include tests in

$DISCHARGE_HOME/Exec/Tests for a functional test suite.
$DISCHARGE_HOME/Exec/Convergence for a code verification tests.

Test suite

To make sure chombo-discharge compiles and runs as expected, we include a test suite that runs functional tests of many chombo-discharge components. The tests are defined in $DISCHARGE_HOME/Exec/Tests, and are organized by application type.

Running the test suite

To do a clean compile and run of all tests, navigate to $DISCHARGE_HOME/Exec/Tests and execute the following:

python3 tests.py --compile --clean --silent --no_exec -cores X

where X is the number of cores to use when compiling. By default, this will compile without MPI and HDF5 support.

Advanced configuration

The following options are available for running the various tests:

--compile Compile all tests.
--clean Do a clean recompilation.
--silent Turn off terminal output.
--benchmark Generate benchmark files.
--no_exec Compile, but do not run the test.
--compare Run, and compare with benchmark files.
-dim <number> Run only the specified 2D or 3D tests.
-mpi <true/false> Use MPI or not.
-hdf <true/false> Use HDF5 or not.
-cores <number> Run with specified number of cores.
-suites <string> Run a specific application test suite.
-tests <string Run a specific test.

For example, to compile with MPI and HDF5 enabled:

python3 tests.py --compile --clean --silent --no_exec -mpi=true -hdf=true -cores 8

If one only wants to compile a specified test suite:

python3 tests.py --compile --clean --silent --no_exec -mpi=true -hdf=true -cores 8 -suites Electrostatics

One can also restrict the tests to specific dimensionality, e.g.

python3 tests.py --compile --clean --silent --no_exec -mpi=true -hdf=true -cores 8 -suites Electrostatics -dim=2

Using benchmark files

The test suite can generate benchmark files which can later be compared against new test suite output files. This is often a good idea if one wants to ensure that changes the chombo-discharge code does not unintentionally change simulations. In this case one can run the test suite and generate benchmark files before adding changes to chombo-discharge. Once the code development is completed, the benchmark files can later be bit-wise (using h5diff) compared against the results of a later test suite.

This consists of the following steps:

Before making changes to chombo-discharge, generate benchmark files with

python3 tests.py --compile --clean --silent --benchmark -mpi=true -hdf=true -cores X

Make the required changes to the chombo-discharge code.

Run the test suite again, and compare benchmark and output files as follows:

python3 tests.py --compile --clean --silent --compare -mpi=true -hdf=true -cores X

When running the tests this way, the output files are bit-wise compared and a warning is issued if the files do not exactly match.

Automated testing

On GitHub, the test suite is integrated with GitHub actions and are automatically run when opening a pull request for review. In general, all tests must pass before a pull request can be merged. The test status can be observed either in the pull request, or at https://github.com/chombo-discharge/chombo-discharge/actions. The automated tests run chombo-discharge with DEBUG=TRUE and OPT=FALSE in order to catch assertion errors or other code breaks. They usually take 1-2 hours to complete.

The automated tests will clone, build, and run the chombo-discharge test suite for various configurations:

Parallel and serial.
With or without HDF5.
In 2D and 3D.

The tests are run with the following compiler suites:

GNU.
Intel oneAPI.

Convergence testing

To ensure that the various components in chombo-discharge converge at desired truncation order, many modules are equipped with their own convergence tests. These are located in $DISCHARGE_HOME/Exec/Convergence. The tests are too extensive to include in continuous integration, and they must be run locally like a regular chombo-discharge application. Our approach for convergence testing is found in Verification and validation.

Performance profiling

There are two ways to run performance profiling of chombo-discharge:

A posteriori profiling using Chombo macros. Most routines in chombo-discharge use these macros and they will compute the wall clock time spent in each routine.

To enable these timers, set CH_TIMER=1 in the shell where you run your application. E.g,
```
export CH_TIMER=1
```
Warning

Chombo’s timers are not meant to use with many time steps. For efficient use, it is best to use it for a single time step.
In-place profiling using the chombo-discharge Timer class. Some classes in chombo-discharge use the Timer class, which also computes on-the-fly calculations of potential load imbalance.

Warning

The Timer class incurs large performance penalties at high concurrencies (1K CPU cores and above).