How to debug the OpenFOAM adapter?

Hello preCICE community,

I am opening this topic in the hopes of following:

  • It works as guidance for future developers working on OF adapters
  • Getting help on debugging OpenFOAM adapter or OF related things

My System:

Kubuntu 20.04 LTS
preCICE v2.3.0, compiled in Debug mode
OpenFOAM v7 (Optimised version),
OpenFOAM v7 adapter compiled in Debug mode
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)

My usual way of debugging things related to OpenFOAM

As only a common user of OpenFOAM, I am not very well versed in debugging OpenFOAM or OpenFOAM libraries. But following some general tips, I am able to make do.

  • Here is a good post on CFD online forum with the banana trick. Good when you are only dealing with the OpenFOAM case files
  • The Info << "Does this print in the terminal way" << endl;, where we put print statements. Good if you are messing with solvers
  • or DEBUG(adapterInfo("message goes here"));, in case of preCICE OpenFOAM adapters (I think this needs ADAPTER_PREP_FLAGS = -DADAPTER_DEBUG_MODE).
  • Coupled OF simulation: I only run the Fluid participant and wait before I run the other participant. Open the openfoam-adapter project in QT, Debug > Start Debugging > Attach to Running application > Choose the solver running for Fluid participant. After this, I run the 2nd participant, and usually, I can get the debugger to stop on Adapter.C or Interface.C. (I have not been able to enter into modules).
  • After sourcing OpenFOAM environment set WM_COMPILE_OPTION=Debug (this gives some extra terminal info), but will help with QT debug option above.
  • When modifying solver or adapter, we can get same result as previous point by modifying first line in Make/options from EXE_INC = \ to EXE_INC = -O0 -fdefault-inline -ggdb3 -DFULLDEBUG \

With the methods mentioned above, most user-related issues are solved.

Issues:

I am hoping to get some thoughts/hints/opinions on the following issue I personally face. In my particular case, I have added extra modules to the OpenFOAM adapter and I want to debug the modules.

  • With the methods mentioned above, I am not able to start debugging the OpenFOAM adapter, before it starts running, as I am merely attaching the debugger to a running application. So I am only able to debug from the point I attach to the application.
  • How can we access data values (fields values) that are exchanged by the OpenFOAM adapter? Currently, I see what fields are exchanged but data values are inaccessible. Do we need to compile OpenFOAM with Debug mode?

With this post, I am hoping to hear from OpenFOAM adapter developers how they handle debugging the OpenFOAM adapter and their experience with debugging.

3 Likes

Did you try using gdb? You could compile your code with debugging flags (OpenFOAM and also the adapter) and then run something like gdb OPENFOAMEXECUTABLE to attach the debugger to the application. The executable would then be started from within gdb. This way you would be able to debug the code from the start. gdb might not be easy to get into, but it is a very powerful debugging tool.

I must admit that I don’t use the OpenFOAM adapter nor OpenFOAM so I do not know if there is any problem with this.

Hi,

A great tool to painlessly debug adapters is the export functionality of preCICE.

To enable this, simply define <export:vtk /> in the configuration of the participant of interest.
It will then export all locally handled meshes for all time-windows.
This export includes the vertex positions, defined vertex data, and connectivity for all locally handled Meshes. Note that connectivity is only available for meshes where the mapping requires it (nearest neighbour mapping / scaled-consistent mappings).

The upcoming release will greatly extend exporters by adding more exporters:

  • VTU only (VTK exports legacy in serial and VTU in parallel)
  • VTP
  • CSV (no connectivity, easy scriptable)

You can already try these features by using the develop version of preCICE.

1 Like

Thank you @ajaust. Yes, we can use gdb xxxFoam to run the solver in debug mode in the terminal. And this is a really good idea to stop the simulation at the initial steps.

The issue I face here is that I cannot run the OF solver from the global directory. The runFluid script has a command

$solver -case Fluid

I did try gdb $solver -case Fluid, but I get an error gdb: unrecognized option '-case'.
So I modified my case to run the participants from individual directories.

I made the following modification to precice-config.xml:

<!--     <m2n:sockets from="Fluid" to="XDEM"    enforce-gather-scatter="1"/> --> // the ususal to run from global directory 
<m2n:sockets from="Fluid" to="XDEM"    enforce-gather-scatter="1"   exchange-directory=".."/> // to run from local participant directory

Then I made necessary modifications to run scripts as well, but my simulation is stuck at

preCICE: Setting up master communication to coupling partner/s

I believe after this issue is solved then we can use the sugeestion from @ajaust

Does it work with this two step approach:

  1. Attach gdb to your solver

    gdb $solver
    

    This will put you in gdb while gdb is ready to debug $solver.

  2. Start the debugging with

    run -case Fluid
    

    Now gdb should run the solver and pass the options provided, i.e., -case Fluid, to the $solver.

The way you adapted your case should also work. I assume that something is wrong with the exchange directory. Make sure that .. actually points to the same directory and is write/readable for each participant. You could also try to explicitly give the full path to some exchange directory.

preCICE: Setting up master communication to coupling partner/s

See also Help! The participants are not finding each other!

First of all, some general comments:

  • Debugging partitioned simulations is anyway complicated. Going with a debugger (such as gdb, as mentioned above, or any other debugger/interface you want) is always a good idea, but you will need more results-oriented simulation. In that case, remember that the adapter needs to be compiled with debug symbols (-g flag, you can add it, e.g., in the Make/options file next to the include paths, so that every object file is built with debug symbols enabled. I guess this should be enough.). You may also want to have a debug build of OpenFOAM, to get more insight there.
  • Exports really help when debugging any partitioned simulation, as mentioned above.
  • You may also want to work with the simplest coupling and mapping available (explicit coupling, nearest-neighbor mapping, uni-directional coupling), without subcycling (set the solver time step size to be equal to the coupling time window size).
  • You can set the results writing period of OpenFOAM to be the same as the solver time step size.

On more specific parts of your questions:

The adapter does not print these, you need to use the preCICE exports.

As mentioned above, you can start an application with additional options from inside gdb. See also this discussion.

This is now going to a different direction, which is not needed, but did you also start your solid solver from inside its own subdirectory? Remember that the exchange-directory is applied to both participants.

1 Like