How to do the coupling coding if one of the participants doesn't decompose the coupling interface

From Benjamin’s dissertation, we have:
“As the coupling interface is a lower-dimensional manifold of the complete simulation domain”.

“It is challenging, since both participants possess, in general, different domain decompositions, which then also entail different decompositions of the coupling interface. Thus, for every rank, we need to identify which part of the coupling mesh it has to communicate to whom.”

Let me put down some language to start from:

  1. There are 2 things from the above quote: different domains and the coupling interface.
  2. Let’s consider the 3D case.
  3. According to Benjamin’s dissertation, the domain and its interface will be decomposed together. For a typical 3D FSI problem, for instance the perpendicular flap problem, for the solid part, this means the interior discretization, such as hexahedron elements, and the exterior discretization, such as the quadrilateral elements (which are the sides of the hexahedrons), are usually decomposed together into the same domain.
  4. The surface pressure/force exerted on these exterior quadrilaterals from the fluidSolver will become the external nodal force and drive the deformation of the whole solid.

My question is that if the pressure or the external nodal force calculation is not decomposed together with the interior hexa, but instead is done only on rank 0 process for solidSolver, how should I do the coding and let preCICE know?

  1. Because the coupling interface is not decomposed, there is no need to “identify which part of the coupling mesh it has to communicate to whom.” For the solidSolver, the requirement is to ensure only rank 0 holds all the external forces. So the following item 2 and item 3.
  2. If rank.eq.0, then everything (on the coupling interface) exchanges on rank 0:
    call precicef_create(solidSolverName, configFile, rank=0, commsize=10)
    call precicef_set_vertices(the-coupling-interface-nodes)
  3. If then return. (basically do nothing since these non-root processes don’t participate the external nodal forces calculation which are on the coupling interface).
  4. By using m2n:sockets, how can I ensure preCICE sends all mapped pressure/force from the fluidSolver side to solidSolver rank 0? And the preCICE (linked with solidSolver) does all communication, mapping and coupling check only on rank 0 (in the other direction, the displacement will be collected onto rank 0 and send to preCICE then to fluidSolver).

In summary, my main point is: the coupling interface is not decomposed on the solidSolver and the external force is required to be put on rank 0 so all communications to outside of the solidSolver need to be done on rank 0 (while the interior is decomposed into multiple domains when mpirun, but it is not relevant because for FSI, preCICE is only responsible for the displacement and the force exchanges on the coupling interface.)

related to the topic:

In this case, only rank 0 of solidSolver needs to define the mesh and read/write data to it. Other ranks don’t need to define a mesh. preCICE handles this for you.

This is pretty common in practise.
As an example we couple two individual pipes using two solvers green and blue, then the partitioning can easily result in a single rank holding the entire coupling mesh of a solver.
In the image below each solver partitions the domain among 4 ranks. Only rank 0 of blue and rank 3 of green provide the coupling mesh and exchange data.

I hope that clarifies things.
Best regards