TL;DR
Running the quickstart tutorial (OpenFOAM adapter + solid-cpp) on a regular desktop/laptop, both participants can hang forever right at startup, before any preCICE coupling code runs, with no error message at all. Root cause: OpenFOAM and the solid-cpp example both link against system OpenMPI, and MPI_Init runs a hardware-topology survey via hwloc, which includes an optional plugin that tries to detect GPUs by connecting to the X11 display server. On my machine that specific connection attempt never completes and never times out, it just blocks indefinitely. Since this happens inside MPI_Init, it happens even for a single-process, non-parallel run, and it happens for both participants independently (OpenFOAM via its Pstream library, and the plain C++ solid-cpp solver via libprecice itself), so from the outside it looks exactly like a coupling/networking problem.
Workaround: set export HWLOC_COMPONENTS=-gl before launching either solver. This disables just the GPU-via-X11 detection plugin and has no other effect. I’ve now run the full tutorial successfully with it set.
Environment
-
OS: Ubuntu (apt-based install), desktop session with a normal X11/XWayland display
-
OpenFOAM: v2606 (OpenCFD/openfoam.com, via the official apt repo)
-
openfoam-adapter: built from the develop branch (needed for v2606 compatibility; the tagged v1.3.1 release predates it)
-
preCICE: 3.4.1 (system package)
-
MPI: system OpenMPI (the sys-openmpi OpenFOAM is configured against)
-
GPU: AMD, with ROCm/HSA runtime libraries present on the system
Symptom
Following the quickstart docs exactly:
Terminal 1:
cd tutorials/quickstart/fluid-openfoam && ./run.sh
Terminal 2:
cd tutorials/quickstart/solid-cpp && ./run.sh
Both processes start and print normal early output (OpenFOAM’s mesh/field setup on the fluid side, nothing at all on the solid side — more on that below), then just sit there. No further output, no error, no CPU usage climbing, process never exits. precice-run/ is never created, so it’s not even a stale-state issue, the participants never get far enough to start any coupling handshake at all.
The complete absence of output from solid-cpp is a red herring worth calling out explicitly: its very first line (“Rigid body: starting…”) is normally printed before preCICE is touched at all, but since its run.sh pipes stdout through tee for logging, a program that doesn’t explicitly flush can sit there producing zero visible output even while it’s actually progressing (or, as it turns out here, not progressing at all). Don’t assume “no output” means “hasn’t started”, it can equally mean “started, but stdout is buffered.”
Diagnosis
Since there was nothing to go on from the logs, I attached to the hung processes directly:
ps aux | grep -E 'pimpleFoam|rigid_body_solver'
sudo gdb -p <PID> --batch -ex "thread apply all bt" -ex detach
Both processes’ main threads showed an identical call chain (fluid side shown, solid side is the same minus the OpenFOAM-specific frames):
Foam::Time::run()
└─ preciceAdapterFunctionObject::preciceAdapterFunctionObject(…)
└─ Foam::UPstream::initNull()
└─ PMPI_Init_thread() [OpenMPI]
└─ ompi_mpi_init() → mca_base_framework_open() → opal_hwloc_base_get_topology()
└─ hwloc_topology_load()
└─ hwloc_gl.so: ???
└─ XOpenDisplay() [libX11]
└─ _XConnectXCB() → xcb_connect_to_display_with_auth_info()
└─ connect() ← blocked here, forever
For solid-cpp, the same chain starts from precice::Participant::Participant(…) directly (its very first line), which itself calls MPI_Init internally.
I also independently confirmed with strace -e trace=network and a firewall log check that this isn’t a networking/firewall issue on my end, no connection attempts, blocked or otherwise, show up anywhere at the OS level related to this call; it’s purely the hwloc_gl → XOpenDisplay path that never returns.
Root cause
hwloc’s optional GL component tries to enumerate GPUs attached to a display as part of building its hardware topology, which OpenMPI runs unconditionally during MPI_Init (this is a documented, known quirk of hwloc’s GL backend, see open-mpi/hwloc#483 and open-mpi/hwloc#55, where it’s shown to at least print spurious errors on some systems when XOpenDisplay() fails). On my machine, rather than failing fast, that connection attempt simply never returns. Since both OpenFOAM (via Pstream) and libprecice itself call MPI_Init even for a fully serial, non-parallel run, this bites both participants independently, with no relation to preCICE’s own coupling/networking logic at all.
Workaround
export HWLOC_COMPONENTS=-gl
set in every terminal/shell that launches either participant, before running ./run.sh. This disables only the GPU-via-display detection plugin, it doesn’t affect MPI’s actual job (process communication) or anything about OpenFOAM/preCICE’s own computation, so it’s safe to leave on permanently (e.g. in ~/.bashrc).
One dead end worth mentioning so others don’t waste time on it: unset DISPLAY did not fix this for me, even though it seems like the more obvious fix. HWLOC_COMPONENTS=-gl was the only thing that worked.
The text above was written with AI, but the headaches were felt by a human being.