Influence of the number of processors in cylinderFlap tutorial

Official adapters and tutorials
, ,
1

Hello everybody,
I am new to preCICE and just installed it. My configuration is :

  • OpenFoam V19-12
  • Petsc 3.13.2 compiled from source
  • OpenMPI 4.0.4
  • boost 1.72.0

I tested the FSI/cylinderFlap/OpenFOAM-CalculiX but I have some issue according to the number of processor :

  • Serial computation : OK
  • 2 processors : OK
  • >2 processors : KO

The error is always the same :

---[precice] ^[[0m Mapping Displacements0 consistent from Calculix_Mesh (ID 2) to Fluid-Mesh-Nodes (ID 1) for dimension 0) with polynomial set to separate
    KSP Object: QR Solver 3 MPI processes
      type: lsqr
    	standard error not computed
    	using inexact matrix norm
      maximum iterations=10000, initial guess is zero
      tolerances:  relative=1e-05, absolute=1e-50, divergence=10000.
      left preconditioning
      using UNPRECONDITIONED norm type for convergence test
    PC Object: 3 MPI processes
      type: none
      linear system matrix = precond matrix:
      Mat Object: Q 3 MPI processes
    	type: mpidense
    	rows=290, cols=4
    	total: nonzeros=1160, allocated nonzeros=1160
    	total number of mallocs used during MatSetValues calls=0
    ---[precice] ^[[31mERROR: ^[[0m Polynomial QR linear system has not converged. Try to fix axis-aligned mapping setups by marking perpendicular axis as dead.
    terminate called after throwing an instance of 'boost::wrapexcept<boost::log::v2_mt_posix::system_error>'
      what():  Failed to set TLS value: Invalid argument

Have you ever heard/seen such issue ?
Thank you

2

This looks like a dependency issue, which we should look at.

For me it currently works with older versions (e.g. with 4 Fluid processes, OpenFOAM v1912, PETSc 3.9.1, OpenMPI 1.10.2, Boost 1.67, CalculiX 2.15, preCICE v2.0.2). Please don’t start trying older dependencies already, you don’t need to go so much back in time.

A few questions:

  • Which CalculiX version are you using? I already had a similar issue with 2.16 (but not 2.15), already in serial. However, it only appears in some systems and we don’t yet understand when.
  • Do you get the same issue with the tutorials/FSI/flap_perp/OpenFOAM-CalculiX case?

Could you also check how the domain is decomposed? I assume that in some cases where the flap is split amond different processes, the default RBF mapping starts behaving weird in your case.

Can you try setting setting your mapping to mapping:rbf-compact-tps-c2 support-radius="0.01"?

3

Hello,

Thank you for your help. Please find my answers to your questions:

  • we are using CalculIX 2.16,
  • for the flap_perp tutorial, it is worst as the computation stops immediatly with the same error message, even for the serial computation,
  • I did not check the domain decomposition yet,
  • Setting mapping to mapping:rbf-compact-tps-c2 support-radius="0.01" worked for // computation of cylinder case but at the end, the compuuation stops with the following message:

Newton-Raphson iterative procedure is active
Nonlinear geometric effects are taken into account
*ERROR: solution seems to diverge; please try automatic incrementation; program stops best solution and residuals are in the frd file

4

I keep thinking that this is another instance of the problem I observed with CalculiX 2.16. Could you please downgrade to CalculiX 2.15 (and the adapter from the branch v2.15_preCICE2.x) and try again? If you don’t get any problem, then it would be very helpful to document your findings in the issue mentioned above.

5

I was trying the flap_perp tutorial and I ran into the same issue.

I got it to run correctly using CalculiX 2.15 and the adapter as suggested in the previous comment.

1 Like