CMake does not find PETSc

(Discussion moved from Cannot find -lpthreads, as it starts a new topic)

(Edit by moderators: discussion continued with a solution here: See more about this here: CMake does not find PETSc reloaded)

Thanks! The problem was fixed.

But a new problem comes. The compilation with cmake shows that Petsc could not be found. PETSC_DIR and PETSC_ARH had been set in ~/.bashrc. I don’t understand ’ Check that the compiler is set to the compiler wrapper provided by your MPI distribution (e.g. with CXX=mpicxx cmake [options] ).’ Should it be cmake CXX=mpicxx to compile or add CXX=mpicxx in some files?

According to the online instruction ( Building: Using CMake · precice/precice Wiki · GitHub), this problem has been fixed since preCICE v1.5.0. The version I am compiling is 1.6.1

The error message is shown below:

CMake Error at /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  PETSc could not be found.  Be sure to set PETSC_DIR and PETSC_ARCH.
  (missing: PETSC_EXECUTABLE_RUNS) (found suitable version "3.12.1", minimum
  required is "3.6")
Call Stack (most recent call first):
  /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindPETSc.cmake:345 (find_package_handle_standard_args)
  CMakeLists.txt:153 (find_package)


-- Configuring incomplete, errors occurred!

Hi, @HDY! I moved your post here, as it is a separate, interesting topic on its own.

It should be CXX=mpicxx cmake [whatever options].

Why like this? CMake (and Make) use the CXX environment variable to know which compiler to run, but this variable is usually set to something else for your user environment. By prepending cmake (or any other command) with variable=value (or env variable=value) you set that variable to another value only for this time you run this command.

If you wrote it like cmake CXX=mpicxx, this would mean that CXX is an option for CMake. This is not the case here, it is a different mechanism.

1 Like

You can also use a CMake option to explicitly set the compiler. This verbose version can be handy in scripts where you would like to pack everything into one single cmake command.

cmake -DCMAKE_CXX_COMPILER=mpicxx [other options]

2 Likes

Many thanks for your prompt help, MakisH & fsimonis!

Sorry that perhaps I was misleading. In CMakeError.log, it still shows “/usr/bin/ld: cannot find -lpthreads”. Besides, the Ubuntu version is 19 (not 18)

I tried to re-install libboost, cmake, Petsc and preCICE. The compilation steps are shown below.

==================================================

sudo apt-get install libboost-all-dev

**Reading package lists... Done**
**Building dependency tree       **
**Reading state information... Done**
**libboost-all-dev is already the newest version (1.67.0.1).**
**sh: 0: getcwd() failed: No such file or directory**
**0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.**
**sh: 0: getcwd() failed: No such file or directory**

==================================================

sudo apt-get -y install cmake

**Reading package lists... Done**
**Building dependency tree       **
**Reading state information... Done**
**cmake is already the newest version (3.13.4-1).**
**sh: 0: getcwd() failed: No such file or directory**
**0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.**
**sh: 0: getcwd() failed: No such file or directory**
which cmake

**sh: 0: getcwd() failed: No such file or directory**
**/usr/bin/cmake**
cd /usr/bin
cmake --version

**cmake version 3.13.4**
**CMake suite maintained and supported by Kitware (kitware.com/cmake).**

===================================================================

Petsc is installed using the commands below.

sudo ./configure --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-mpich
sudo make all test

Last several lines at the end of the compilation log info printed on the screen.

**CLINKER arch-linux2-c-debug/lib/libpetsc.so.3.12.1**
**make[2]: Leaving directory '/scratch/hd/sw/petsc-3.12.1'**
**make[1]: Leaving directory '/scratch/hd/sw/petsc-3.12.1'**
**Now to check if the libraries are working do:**
**make PETSC_DIR=/scratch/hd/sw/petsc-3.12.1 PETSC_ARCH=arch-linux2-c-debug check**
**=========================================**
**Running test examples to verify correct installation**
****Using PETSC_DIR=/scratch/hd/sw/petsc-3.12.1 and PETSC_ARCH=arch-linux2-c-debug****
**C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process**
**C/C++ example src/snes/examples/tutorials/ex19 run successfully with 2 MPI processes**
**Fortran example src/snes/examples/tutorials/ex5f run successfully with 1 MPI process**
**Completed test examples**

===================================================================

In the file ~/.bashrc, add the following lines.

export PETSC_DIR=/scratch/hd/sw/petsc-3.12.1
export PETSC_ARCH=arch-linux2-c-debug

source ~/.bashrc

====================================================================

rm precice-1.6.1 -rf
tar -xzvf v1.6.1.tar.gz
cd precice-1.6.1
mkdir build && cd build
sudo cmake  -DCMAKE_CXX_COMPILER=mpicxx -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo ..

See the compilation error printed on the screen at the bottom.

======================================================================

CMakeError.log and CMakeOutput.log are saved in the links below.

https://drive.google.com/open?id=1ZALOuPIQNyz0IZGJ0BVm9mGaPky8R1L9

======================================================================

The same error message is shown when either of the two below methods is used.

  1. sudo CXX=mpicxx cmake -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
  2. sudo cmake -DCMAKE_CXX_COMPILER=mpicxx -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
**-- The CXX compiler identification is GNU 8.3.0**
**-- Check for working CXX compiler: /usr/local/bin/mpicxx**
**-- Check for working CXX compiler: /usr/local/bin/mpicxx -- works**
**-- Detecting CXX compiler ABI info**
**-- Detecting CXX compiler ABI info - done**
**-- Detecting CXX compile features**
**-- Detecting CXX compile features - done**
**-- --------------------**
**-- == CONFIGURATION**
**-- Library version to build: 1.6.1**
**-- Build configuration: RelWithDebInfo**
**-- Build shared libraries: ON**
**-- Target system: Linux-5.0.0-32-generic**
**-- Host system: Linux-5.0.0-32-generic**
**-- Library architecture: x86_64-linux-gnu**
**-- CXX compiler: /usr/local/bin/mpicxx**
**-- CXX compiler flags: <empty>**
**-- CXX linker: /usr/bin/ld**
**-- Install prefix: /usr/local**
**-- Source directory: /scratch/hd/sw/precice-1.6.1**
**-- Binary directory: /scratch/hd/sw/precice-1.6.1/build**
**-- --------------------**
**-- Build with MPI: ON**
**-- Build with PETSC: ON**
**-- Build with PYTHON: ON**
**-- Configure package generation: ON**
**-- Install tests/testfiles: ON**
**-- Additional CTest MPI Flags: <empty>**
**-- --------------------**
**-- Looking for C++ include pthread.h**
**-- Looking for C++ include pthread.h - found**
**-- Looking for pthread_create**
**-- Looking for pthread_create - not found**
**-- Looking for pthread_create in pthreads**
**-- Looking for pthread_create in pthreads - not found**
**-- Looking for pthread_create in pthread**
**-- Looking for pthread_create in pthread - found**
**-- Found Threads: TRUE  **
**-- Boost version: 1.67.0**
**-- Found the following Boost libraries:**
**--   filesystem**
**--   log**
**--   log_setup**
**--   program_options**
**--   system**
**--   thread**
**--   unit_test_framework**
**--   date_time**
**--   regex**
**--   chrono**
**--   atomic**
**-- Found Eigen3: /usr/include/eigen3 (Required is at least version "3.2") **
**-- Validating Eigen**
**-- Validating Eigen - success**
**-- Found LibXml2: /usr/lib/x86_64-linux-gnu/libxml2.so (found version "2.9.4") **
**-- Validating LibXML2**
**-- Validating LibXML2 - success**
**-- Validating JSON**
**-- Validating JSON - success**
**-- Validating Prettyprint**
**-- Validating Prettyprint - success**
**-- Found MPI_CXX: /usr/local/bin/mpicxx (found version "3.1") **
**-- Found MPI: TRUE (found version "3.1")  **
**CMake Error at /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137 (message):**
**PETSc could not be found.  Be sure to set PETSC_DIR and PETSC_ARCH.**
**(missing: PETSC_INCLUDES PETSC_LIBRARIES PETSC_EXECUTABLE_RUNS) (Required**
**is at least version "3.6")**
**Call Stack (most recent call first):**
**/usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)**
**cmake/modules/FindPETSc.cmake:345 (find_package_handle_standard_args)**
**CMakeLists.txt:148 (find_package)**


**-- Configuring incomplete, errors occurred!**

Perhaps my problem on cmake is very difficult to tackle.

I just succeeded in using Scons.

Are you by any chance on a cluster? (I just saw a /scratch/ path in your CMake error log) In that case, the best would be to use Spack to get preCICE, as it will install every minor dependency from scratch, in the way it should be.

Please don’t rely on SCons, we will remove it in preCICE v2.0 (very soon) and our adapters are not anymore trying to make is easy to install them if you use SCons.

Your setup and your use of cmake looks fine.

Can you please run the following command and attach the resulting petscfiles.txt

find $PETSC_DIR/$PETSC_ARCH > petscfiles.txt

Also, try to run cmake without sudo. It should not need the elevated privileges.

Finally, please remove the entire build directory to make sure that the cache is not messing with subsequent runs.

I am not using a cluster. It is only a personal habit to set a director named /scratch. :sweat_smile:

The file is saved in the link below.

Hello again,

Although I am not compiling on a cluster but a PC, I tried Spack. An error was found as ‘ModuleNotFoundError: No module named ‘_ssl’’. See below.

source spack/share/spack/setup-env.sh

Traceback (most recent call last):
** File “/scratch/hd/sw/spack/bin/spack”, line 63, in **
** import spack.main # noqa**
** File “/scratch/hd/sw/spack/lib/spack/spack/main.py”, line 29, in **
** import spack.architecture**
** File “/scratch/hd/sw/spack/lib/spack/spack/architecture.py”, line 69, in **
** import spack.compiler**
** File “/scratch/hd/sw/spack/lib/spack/spack/compiler.py”, line 19, in **
** import spack.spec**
** File “/scratch/hd/sw/spack/lib/spack/spack/spec.py”, line 106, in **
** import spack.repo**
** File “/scratch/hd/sw/spack/lib/spack/spack/repo.py”, line 36, in **
** import spack.caches**
** File “/scratch/hd/sw/spack/lib/spack/spack/caches.py”, line 15, in **
** import spack.fetch_strategy**
** File “/scratch/hd/sw/spack/lib/spack/spack/fetch_strategy.py”, line 44, in **
** import spack.util.web as web_util**
** File “/scratch/hd/sw/spack/lib/spack/spack/util/web.py”, line 14, in **
** import ssl**
** File “/home/hd/local/lib/python3.6/ssl.py”, line 101, in **
** import _ssl # if we can’t import it, let the error propagate**
ModuleNotFoundError: No module named '_ssl’

I see that this is a known problem with Spack and some Python versions (e.g. 3.6). Let’s drop the Spack approach for now. Try what @fsimonis suggested above.

Even though you solved your instance of the problem, this would help us better understand it and solve it. So, thanks for the help! :grin:

Hi fsimonis,

Any feedback on petscfiles.txt from me?

The method – compiling without sudo and remove the build directory to remove – does not work, unfortunately.

Sorry that I missunderstood @fismonis idea! I thought that he would give me feedback after reading the file pescfiles.txt.

No, the problem does not solved.

BTW, why will you remove SCons from the future release of preCICE? For my present situation at least, SCons is simple and can compile the code, while cmake doest not work with unknown reasons.

Regarding your petscfiles.txt, I see that you don’t have any CMake-related files (e.g. $PETSC_DIR/$PETSC_ARCH/lib/petsc/conf/PETScBuildInternal.cmake). How did you configure the PETSc build if not with CMake?

Here is how it looks for me: petscfiles.txt (588.0 KB)

Well, ideally CMake should always work and that’s why we are trying to understand and fix this problem. :slight_smile:

SCons also had several (much bigger) issues for us and it is very difficult to maintain two build systems, especially since we try to maintain not only the main library, but a larger ecosystem of adapters. Both CMake and SCons require some workarounds for some dependencies, but CMake has a much wider community, adoption, and feature set. It is e.g. due to CMake that we are able to easily create Debian packages. We got the feeling over the past two years that with SCons you can easily do the basics, but it gets quite complicated if you want to do more than that.

Thanks! Now I understand why CMake will be used in the future.

Petsc was compiled using the following commands:

sudo ./configure --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-mpich
sudo make all test

I found the reason.

The files are missing in the latest version of Petsc 3.12.1, but they are found in 3.11.4.

But the problem is still there. Although Petsc has no problem with MPI, preCICE cannot pass the parallel test. See below.

======================================================================
Petsc:
make PETSC_DIR=/scratch/hd/sw/petsc-3.11.4 PETSC_ARCH=arch-linux2-c-debug check
=========================================
Running test examples to verify correct installation
Using PETSC_DIR=/scratch/hd/sw/petsc-3.11.4 and PETSC_ARCH=arch-linux2-c-debug
C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process
C/C++ example src/snes/examples/tutorials/ex19 run successfully with 2 MPI processes
Fortran example src/snes/examples/tutorials/ex5f run successfully with 1 MPI process

======================================================================
preCICE:
– Performing Test MULTIPASS_TEST_1_petsc_works_minimal
– Performing Test MULTIPASS_TEST_1_petsc_works_minimal - Failed
– Performing Test MULTIPASS_TEST_2_petsc_works_allincludes
– Performing Test MULTIPASS_TEST_2_petsc_works_allincludes - Failed
– Performing Test MULTIPASS_TEST_3_petsc_works_alllibraries
– Performing Test MULTIPASS_TEST_3_petsc_works_alllibraries - Failed
– Performing Test MULTIPASS_TEST_4_petsc_works_all
– Performing Test MULTIPASS_TEST_4_petsc_works_all - Failed
– PETSc could not be used, maybe the install is broken.
CMake Error at /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
** PETSc could not be found. Be sure to set PETSC_DIR and PETSC_ARCH.**
** (missing: PETSC_EXECUTABLE_RUNS) (found suitable version “3.11.4”, minimum**
** required is “3.6”)**
Call Stack (most recent call first):
** /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)**
** cmake/modules/FindPETSc.cmake:345 (find_package_handle_standard_args)**
** CMakeLists.txt:148 (find_package)**

– Configuring incomplete, errors occurred!

In the preCICE build directory, there should be a folder called CMakeFiles containing the file CMakeError.log.
This log should contain the failing checks including the source code, the full compiler invocations and the error message of the execution.

Can you please have a look at it? If the error is not obvious, feel free to upload it here.

As a new user, I am not allowed to upload files. A strange thing is that I am also not allowed to post links, which I did a couple of days ago…

Please find CMakeError.log and CMakeOutput.log in the links below, but please add https before the links. (edit by MakisH: fixed settings)

https://drive.google.com/file/d/1oRxS0UrcAzOIx6fUGUllkAm2UE53e0Eb/view?usp=sharing

https://drive.google.com/file/d/1OyhrW3E3m5l0vHOMkQQJ46TKd_TybaE4/view?usp=sharing

Discourse thought that this was a spam, because all the links were pointing to the same domain. I found the problematic setting and fixed it. Thanks for pointing this out!

I also found and fixed (I hope) the setting that new users could not attach files.

Please keep sending us feedback! :slight_smile:

The files expose the following problems:

  • The FindPETSc module tries to locate petsc using the PETSC_DIR=/scratch/hd/sw/petsc-3.6.4/ and the PETSC_ARCH=arch-linux2.
    This is most likely a cached setting from a previous run. You have to clear the cache order for the module to use the current PETSC_DIR. The easiest way is to remove the entire precice build directory.
  • A more critical problem, is that you are mixing Intel MPI and Open MPI leading to linker errors of the type:
    /usr/bin/ld: /..../libmpifort.so.12: undefined reference to 'MPIR_CommGetAttr_fort'
    To solve this, I kindly forward you to the installation instructions of petsc.