Highlights of the new preCICE release v2.1

Do you wonder what is actually new in preCICE v2.1? :thinking:
Let’s have a look together at the highlights.

1. Usability

1.1 Reworked all error messages and added new ones

We completely reworked all error messages to make then more useful. Let’s look at one example.

If a certain data name was unknown, we previously had:

Data with name "Forces" is not defined on mesh with ID 1.

You might wonder: which mesh had ID 1? Now, you get the full information:

Data with name "Forces" is not defined on mesh "FluidMesh". 
Please add <use-data name="Forces"/> under <mesh name="FluidMesh/>.

We really like it when software watches our steps and let us know how to fix mistakes, so we brought this to preCICE as well!

You can now find such error messages across all the preCICE features, also in places where before information was really missing. For example, when configuring preCICE on a supercomputer that requires a special network interface, one would get an error message with available options:

Cannot find network interface "loopy". Available interfaces are: lo enp0s31f6 br-3d4e767967a7 docker0 vethb6dcb86 Please check "network" attributes in your configuration file.

The default is now also system-specific, increasing cross-platform compatibility.

1.2 Improved and extended convergence measure logging

Maybe you wondered in the past why the limit in

relative convergence measure: two-norm diff = 2.6036e-08, relative limit = 1.0005e-08, conv = true

changes in every iteration. Well, it was the relative limit, but – admitted – probably not very helpful.
With v2.1, you now get:

relative convergence measure: relative two-norm diff = 2.6023e-05, limit = 1e-05, normalization = 0.00100051, conv = false

So, the relative norm, the fixed limit, and the normalization factor. Even better, these numbers now coincide with what you have in preCICE-PARTICIPANT-convergence.log.

An example: For this configuration:

<relative-convergence-measure data="Pressure" mesh="Fluid_Nodes" limit="1e-5"/>
<absolute-convergence-measure data="CrossSectionLength" mesh="Structure_Nodes" limit="1e-5"/>
<residual-relative-convergence-measure data="CrossSectionLength" mesh="Structure_Nodes" limit="1e-3"/>

you get:

TimeWindow  Iteration  ResRel(Pressure)  ResAbs(CrossSectionLength)  ResDrop(CrossSectionLength)  
1  1  1.0000000000000000  0.0000015151688798  1.0000000000000000  
1  2  0.0092426077637776  0.0000014861770983  0.9808656435350955  
1  3  0.8894974714023499  0.0000000036011503  0.0023767319918400  
1  4  0.0013652823318746  0.0000000000147103  0.0000097086996980  
1  5  0.0000009272349971  0.0000000000000499  0.0000000329390337

While we were already tweaking the convergence file, we also improved the iterations file. You now see how much columns are used in the quasi-Newton acceleration, how many are filtered out, and how many are dropped because they went out of scope (i.e. beyond max-used-iterations and time-windows-reused) – very helpful information if you want to study how well the quasi-Newton acceleration works.

TimeWindow  TotalIterations  Iterations  Convergence  QNColumns  DeletedQNColumns  DroppedQNColumns  
1  5  5  1  4  0  0  
2  9  4  1  6  1  0  
3  12  3  1  7  1  0  
4  15  3  1  9  0  0  
5  18  3  1  10  1  0  
6  21  3  1  11  1  0  
7  24  3  1  12  1  0  
8  27  3  1  13  1  0  
9  30  3  1  13  2  0  
10  33  3  1  14  1  0  
11  36  3  1  15  1  0  
12  39  3  1  15  1  1  
13  42  3  1  15  1  1  

As a last small extension, you can now mark a convergence measure as strict:

<relative-convergence-measure data="Pressure" mesh="Fluid_Nodes" limit="1e-5" strict="true"/>

Then, this convergence measure needs to be fulfilled, meaning: if it does not converge within the maximum allowed iterations, the coupled simulation stops prematurely. A nice feature for large parameter studies.

1.3 More

  • write-data now always resets data to zero in advance(). This will make adapter bugs more apparent.
  • We removed the summary of event timings that were printed when preCICE finalizes and preCICE now writes it to the file precice-PARTICIPANT-events-summary.log instead.

2. New Features

2.1 Using quads for projection

We finally filled setMeshQuad and setMeshQuadWithEdges with meaningful life :tada: .

Actually, we don’t create a quad, but we create two triangles and use those for all our projection methods. For that, we split the quad at the shorter of both diagonals. This gives smaller errors for nearest-project mappings. Good to know, the triangulation is then also visible in the mesh export.

Caution: Please only define planar quads, of course. Otherwise, preCICE complains.

We are already testing this feature with CalculiX.

2.2 A gather-scatter QR-decomposition for RBF mapping

For a consistent RBF mapping from mesh A to mesh B, we build up an interpolant

\sum_{i=1}^{n^A} \gamma_i \phi(\|x-x_i^A\|_2) + q(x)\, ,

where:

  • x_i^A are the vertex coordinates of mesh A,
  • n^A is the number of vertices of mesh A,
  • \phi is the radial basis function,
  • and q(x) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 a linear regularization polynomial.

The interpolation condition

s(x_i^A)=v_i^A \;\forall i=1...n^A\;,

together with a regularlization condition

\sum_{i=1}^{n^A} \gamma_i p(x_i^A) = 0

for every constant and linear polynomial p, leads to a linear equation system. v_i^A are the (scalar) data values on mesh A we want to map.

preCICE now offers two variants how to solve this linear system:

  • GMRES, implemented with PETSc,
  • QR decomposition in initialization, with a triangular solve in every mapping step, implemented with Eigen.

Previously, for a parallel participant, we only supported GMRES, which made PETSc a necessary build dependency if you wanted to use RBF. Now, we also implemented a gather-scatter QR decomposition. This means, during initialization the mesh is gathered on the master rank, where we then compute the QR decomposition. In every mapping step, all data values are gathered on the master rank, followed by the triangular solve, and a scattering of the mapped values. Obviously, these operations are very costly for large meshes or many parallel ranks. For small to medium cases, however, these costs are very tolerable. The benefit is: PETSc becomes an optional dependency even if you want to use RBF mapping. You only need it if you want to handle large cases.

But which variant does preCICE now use? It’s simple:

  • If you build with PETSc, GMRES is used by default. You can still enforce a QR decomposition:
<mapping:rbf-compact-tps-c2 support-radius="0.05" constraint="consistent" 
      direction="read" from="MeshA" to="MeshB" use-qr-decomposition="true"/>
  • If you build without PETSc, QR decomposition is used by default.

In the long run, we want to replace the gather-scatter approach with an actual parallel sparse decomposition.

2.3 Sorted out actions timings and summation action

Actions allow to easily modify coupling data at run time. You don’t know about this feature yet? Have a look at the actions documentation. An action can be applied at different timings. Those were, however, a bit mixed up in the past (are they applied before or after the data mapping?). We tried to clarify them now:

  • write-mapping-prior and write-mapping-post: directly before or after each time the write mappings are applied
  • read-mapping-prior and read-mapping-post: directly before or after each time the read mappings are applied
  • on-time-window-complete-post: after the coupling in a complete time window is converged

We kept the old timings for backwards compatibility, but recommend to switch to the new ones.

Do you want to try that right now? Well, we also have a neat new action you can try it with: a summation-action, which can sum up coupling data, for example forces from different participants:

<participant name="Solid">
  <use-mesh name="SolidMesh" provide="yes"/>
  <read-data name="Forces" mesh="SolidMesh"/> 
  <action:summation timing="on-exchange-post" mesh="SolidMesh">
    <source-data name="FluidForces"/>
    <source-data name="ParticleForces"/>
    <target-data name="Forces"/>
  </action:summation> 
</participant>

<participant name="Fluid">
  <use-mesh name="FluidMesh" provide="yes"/>
  <use-mesh name="SolidMesh" from="Solid"/>
  <mapping:nearest-neighbor direction="write" from="FluidMesh" to="SolidMesh" constraint="conservative"/>
  <write-data name="FluidForces" mesh="FluidMesh"/>
</participant>

<participant name="ParticleCode">
  <use-mesh name="ParticleMesh" provide="yes"/>
  <use-mesh name="SolidMesh" from="Solid"/>
  <mapping:nearest-neighbor direction="write" from="ParticleMesh" to="SolidMesh" constraint="conservative"/>
  <write-data name="ParticleForces" mesh="ParticleMesh"/>
</participant>

3. Maintainability

3.1 Tests

The most important change up front: When you now run the tests with:

make test

it should always work. If not, please let us know!
No more excuses from our side like “this is a known problem, please ignore this failure”. :no_mouth:

How did we do that? Well, we switch off tests that are known to not work, for instance MPI Ports if you use OpenMPI.

What was actually far more work for us was our complete rework of the testing fixtures. Defining new integration tests is now really easy. Look at this small test that couples a parallel participant on 3 MPI ranks with a serial participant:

BOOST_AUTO_TEST_CASE(MasterSockets)
{
  PRECICE_TEST("ParallelSolver"_on(3_ranks), "SerialSolver"_on(1_rank));
  std::string configFilename = _pathToTests + "master-sockets.xml";
  std::string myMeshName;
  if (context.isNamed("ParallelSolver")) {
    myMeshName = "ParallelMesh";
  } else {
    myMeshName = "SerialMesh";
  }
  SolverInterface interface(context.name, configFilename, context.rank, context.size);
  int             meshID      = interface.getMeshID(myMeshName);
  double          position[2] = {0, 0};
  interface.setMeshVertex(meshID, position);
  interface.initialize();
  interface.advance(1.0);
  interface.finalize();
}

Another small improvement: we now also automatically test the solverdummies (and they now also exchange data):

      Start 19: precice.solverdummy.build.cpp
19/27 Test #19: precice.solverdummy.build.cpp .............   Passed    1.05 sec
      Start 20: precice.solverdummy.build.c
20/27 Test #20: precice.solverdummy.build.c ...............   Passed    0.57 sec
      Start 21: precice.solverdummy.build.fortran
21/27 Test #21: precice.solverdummy.build.fortran .........   Passed    0.60 sec
      Start 22: precice.solverdummy.run.cpp-cpp
22/27 Test #22: precice.solverdummy.run.cpp-cpp ...........   Passed    0.51 sec
      Start 23: precice.solverdummy.run.c-c
23/27 Test #23: precice.solverdummy.run.c-c ...............   Passed    0.51 sec
      Start 24: precice.solverdummy.run.fortran-fortran
24/27 Test #24: precice.solverdummy.run.fortran-fortran ...   Passed    0.54 sec
      Start 25: precice.solverdummy.run.cpp-c
25/27 Test #25: precice.solverdummy.run.cpp-c .............   Passed    0.52 sec
      Start 26: precice.solverdummy.run.cpp-fortran
26/27 Test #26: precice.solverdummy.run.cpp-fortran .......   Passed    0.51 sec
      Start 27: precice.solverdummy.run.c-fortran
27/27 Test #27: precice.solverdummy.run.c-fortran .........   Passed    0.52 sec

3.2 Even more maintainability improvements

  • We fixed the life cycle of SolverInterface, both in the C++ API, as well as in the Fortran and C bindings. finalize() is now called automatically in the destructor if you forgot to call it.
  • We narrowed consistently reliable PETSc version down to 3.12. We increased the minimum required PETSc version to v3.12. (Ubuntu 20.04 LTS uses 3.12.4 by the way, so another reason to upgrade!)
  • We refactored the cplscheme packages, which significantly increases future maintainability.
  • We added support for Boost 1.73.0. The baseline remains 1.65.1.

… and we should also mention that we fixed quite some bugs, especially in the communication establishment.

What do you say? Do you like v2.1? Please let us know here.

2 Likes