How do you test preCICE and its official adapters?
Can I trust you?
When refering to preCICE, one may mean the coupling library, or the combination of coupling library and the official preCICE adapters for various well-known solvers. Testing happens in various components all the time and we regularly validate new scenarios against published literature.
We test the components at various points:
- We use the Boost Test library to create unit and integration tests for the coupling library. You can run the testing suite as described in our wiki.
- Every commit and pull request automatically triggers:
- Travis CI, which builds preCICE with different compilers and optional dependencies. It also runs our testing suite.
- Codecov, which checks if we add tests for any new lines of code, enforcing that the code remains tested.
- Various code quality checkers, such as lgtm, Codacy, and CodeFactor. Using these tools, we get independent feedback on applying good software engineering practices, aiming for sustainable code development.
- Most of our official adapters do not currently have unit or integration tests, but every change in them triggers an extensive system test suite on Travis CI. These tests are based on Docker and can also be run locally (systemtests repository). By default, every result file is compared with a reference and values should not differ more than a predefined threshold. The complete suite is also triggered every night.
- Before every major or feature release, we manually execute selected / additional tests, based on our tutorials or other reference cases.
- We also test some of our language bindings (e.g. Python) on Travis CI with every commit.
- The coupling library has been validated with established benchmarks by its developers in published literature or by its users in various examples. The official adapters are, in many cases, results of student theses, which also validated the code in various scenarios. For this, have a look into our literature guide and the documentation of each adapter.
In any case, you can see the details and testing status of each component in its README file or other relevant documentation. See, example, the preCICE README.md.
In any case, please pay attention to the “no warranty” and “no liability” clauses of the LGPL3 license and always apply good scientific practices.