Hi everyone,
I am trying to run the FSI3 Benchmark with openfoam and dealii in parallel. I am using the original configuration of the tutorial case and have no problems when running it with the run.sh files. But as soon as I am trying to put openfoam on 10 or 20 MPI the process aborts after setting up the communication.
---[precice] e[0m Setting up master communication to coupling partner/s
--------------------------------------------------------------------------
An MPI process has executed an operation involving a call to the
"fork()" system call to create a child process. Open MPI is currently
operating in a condition that could result in memory corruption or
other system errors; your MPI job may hang, crash, or produce silent
data corruption. The use of fork() (or system() or other calls that
create child processes) is strongly discouraged.
The process that invoked fork was:
Local host: master (PID 183045)
MPI_COMM_WORLD rank: 7
If you are *absolutely sure* that your application will successfully
and correctly survive a call to fork(), you may disable this warning
by setting the mpi_warn_on_fork MCA parameter to 0.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 8 in communicator MPI_COMM_WORLD
with errorcode 1.
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
[9] [3] [7] #0[8] #0 #0 #0 Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&) at ??:?
[8] #1 Foam::sigSegv::sigHandler(int) at ??:?
at ??:?
at ??:?
[3] #1 [9] #1 [7] #1 Foam::sigSegv::sigHandler(int)Foam::sigSegv::sigHandler(int)Foam::sigSegv::sigHandler(int) at ??:?
[8] #2 ? at ??:?
[7] #2 ? at ??:?
[9] #2 ? at ??:?
[3] #2 ? in "/usr/lib64/libc.so.6"
[8] #3 preciceAdapter::Interface::configureMesh(Foam::fvMesh const&) in "/usr/lib64/libc.so.6" in "/usr/lib64/libc.so.6"
[9] # in "/usr/lib64/libc.so.6"
3 preciceAdapter::Interface::configureMesh(Foam::fvMesh const&)[3] #3 preciceAdapter::Interface::configureMesh(Foam::fvMesh const&)[7] #3 preciceAdapter::Interface::configureMesh(Foam::fvMesh const&) at ??:?
[8] #4 preciceAdapter::Interface::Interface(precice::SolverInterface&, Foam::fvMesh const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, bool) at ??:?
at ??:?
[7] #4 at ??:?
[9] #4 preciceAdapter::Interface::Interface(precice::SolverInterface&, Foam::fvMesh const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, bool)[3] #4 preciceAdapter::Interface::Interface(precice::SolverInterface&, Foam::fvMesh const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, bool)preciceAdapter::Interface::Interface(precice::SolverInterface&, Foam::fvMesh const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, bool) at ??:?
[8] #5 preciceAdapter::Adapter::configure() at ??:?
at ??:?
[7] #5 preciceAdapter::Adapter::configure() at ??:?
[3] #5 preciceAdapter::Adapter::configure()[9] #5 preciceAdapter::Adapter::configure() at ??:?
[8] #6 Foam::functionObjects::preciceAdapterFunctionObject::read(Foam::dictionary const&) at ??:?
[3] #6 Foam::functionObjects::preciceAdapterFunctionObject::read(Foam::dictionary const&) at ??:?
at ??:?
[9] #6 Foam::functionObjects::preciceAdapterFunctionObject::read(Foam::dictionary const&)[7] #6 Foam::functionObjects::preciceAdapterFunctionObject::read(Foam::dictionary const&) at ??:?
[8] #7 Foam::functionObjects::preciceAdapterFunctionObject::preciceAdapterFunctionObject(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
at ??:?
[3] #7 Foam::functionObjects::preciceAdapterFunctionObject::preciceAdapterFunctionObject(Foam::word const&, Foam::Time const&, Foam::dictionary const&)[9] #7 Foam::functionObjects::preciceAdapterFunctionObject::preciceAdapterFunctionObject(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[7] #7 Foam::functionObjects::preciceAdapterFunctionObject::preciceAdapterFunctionObject(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[8] #8 Foam::functionObject::adddictionaryConstructorToTable<Foam::functionObjects::preciceAdapterFunctionObject>::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[9] #8 Foam::functionObject::adddictionaryConstructorToTable<Foam::functionObjects::preciceAdapterFunctionObject>::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[3] #8 at ??:?
Foam::functionObject::adddictionaryConstructorToTable<Foam::functionObjects::preciceAdapterFunctionObject>::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&)[7] #8 Foam::functionObject::adddictionaryConstructorToTable<Foam::functionObjects::preciceAdapterFunctionObject>::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[8] #9 Foam::functionObject::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
at ??:?
at ??:?
[3] #9 [7] #9 [9] #9 Foam::functionObject::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&)Foam::functionObject::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&)Foam::functionObject::New(Foam::word const&, Foam::Time const&, Foam::dictionary const&) at ??:?
[8] #10 Foam::functionObjectList::read() at ??:?
at ??:?
at ??:?
[9] #10 [3] #10 [7] #10 Foam::functionObjectList::read()Foam::functionObjectList::read()Foam::functionObjectList::read() at ??:?
[8] #11 Foam::Time::run() const at ??:?
at ??:?
at ??:?
[9] #11 [7] #11 [3] #11 Foam::Time::run() constFoam::Time::run() constFoam::Time::run() const at ??:?
[8] #12 at ??:?
[7] #12 at ??:?
[3] #12 at ??:?
[9] #12 ???? at ??:?
at ??:?
at ??:?
at ??:?
[8] #13 __libc_start_main[7] #13 __libc_start_main[3] #13 __libc_start_main[9] #13 __libc_start_main in "/usr/lib64/libc.so.6"
[8] #14 in "/usr/lib64/libc.so.6"
in "/usr/lib64/libc.so.6"
in "/usr/lib64/libc.so.6"
[9] #14 [7] #14 [3] #14 ???? at ??:?
at ??:?
at ??:?
at ??:?
[master:183036] 3 more processes have sent help message help-mpi-runtime.txt / mpi_init:warn-fork
[master:183036] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
[master:183036] 3 more processes have sent help message help-mpi-api.txt / mpi-abort
I used the same setup with openfoam on 20 MPI and CalculiX, it worked without problems. Therefore, I think it must have something to do with precice or dealii.
Has anyone experienced this before?
Regards,
Alina