Software Engineering Blog

Tipps und Tricks aus dem Leben eines Systemadministrators.

Using CMake to resolve dependencies

von

Why you should use CMake to resolve transitive dependencies

Currently I am working on a C/C++ library for a German research project in the area of HPC (dash-project.org).
The library consists of two parts, a communication backend written in C (called DART) and a C++ library providing distributed data structures and algorithms (DASH).
DART itself can be implemented for various communication conduits like MPI, SHMEM or CUDA, while DASH communicates with DART independently of the backend.
Furthermore DART provides mechanisms to detect the topology of the interconnect, as well as host characteristics like the number of CPUs or NUMA domains.
DART is shipped as a library to link against, while DASH is almost "header only". Just this to the setup.

If you are not familiar with CMake, read this tutorial first. I will present some advanced techniques which require a sound knowledge of the concepts CMake is based on.

Handling Dependencies

The problems begins when linking the application against DART, because the topology detection uses many third party libraries like hwloc, PAPI, libNUMA, etc.
These libraries can only be linked dynamically using shared objects due to various reasons. Hence, the application has to know all internal dependencies of DART, including linker flags.
This is definitely a pain in the as and not the way to go! Whenever DART changes its dependencies even without changing DASH's public interface, the developer has to adjust his makefile.
Another aspect is that - as a developer of an application - I am not in charge of tracking the internals of a third party library.

The situation gets even worth: As DASH is almost header-only, headers of third party libraries are included in the cpp files. Hence, the compiler has to know the include paths of them.

Dependencies of the DASH project

Using CMake

This is the point where CMake kicks in: As a developer of a library it is good practice to provide a CMake configuration script (see here how to write one) placed in the installation directory.
Now the app developer can easily use "find_package" and all transitive dependencies are resolved automatically.

Including DASH in Your Application

I will show the necessary steps using the DASH example. At first to the easy part: Including DASH in your application:

find_package("DASH-MPI" REQUIRED HINTS $ENV{HOME}/opt/dash-0.3.0)

set( CMAKE_CXX_FLAGS  "${CMAKE_CXX_FLAGS} ${DASH_CXX_FLAGS}" )        
                                                                     
## set dependencies and build options                                           
add_executable(MyProject ${SOURCES})                                      
target_link_libraries(MyProject ${DASH_LIBRARIES})

DASH CMakeLists.txt

# [...]
# cmake packaging                                                             
include(CMakePackageConfigHelpers)                                            
                                                                            
target_include_directories("${DASH_LIBRARY}"                                  
                          PUBLIC $<INSTALL_INTERFACE:include>               
                          PUBLIC ${ADDITIONAL_INCLUDES})                    
                                                                            
# Install library                                                             
install(TARGETS ${DASH_LIBRARY}                                               
      	DESTINATION lib                                                       
      	EXPORT "${DASH_LIBRARY}-targets")                                     
                                                                            
# exports                                                                     
install(EXPORT "${DASH_LIBRARY}-targets"                                            
		DESTINATION "${CMAKE_INSTALL_PREFIX}/cmake")                                

# generate project config file
configure_package_config_file(                                                
  "dash-config.cmake.in"                                                    
  "${DASH_LIBRARY}-config.cmake"                                            
  INSTALL_DESTINATION "${CMAKE_INSTALL_PREFIX}/cmake"                       
  PATH_VARS CMAKE_INSTALL_PREFIX)                                           
                                                                            
# install custom config                                                       
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/${DASH_LIBRARY}-config.cmake"      
      DESTINATION "${CMAKE_INSTALL_PREFIX}/cmake")
# [...]

dash-config.cmake.in

# - Config file for the dash package
# - provides compiler flags of dash installation
# - as well as all transitive dependencies
#
# - Automatically locates DART-BASE
# - and the choosen DART implementation

@PACKAGE_INIT@

set(DASH_VERSION_MAJOR "@DASH_VERSION_MAJOR@")
set(DASH_VERSION_MINOR "@DASH_VERSION_MINOR@")
set(DASH_VERSION_PATCH "@DASH_VERSION_PATCH@")

set(DASH_LIBRARY "@DASH_LIBRARY@")
set(DASH_LIBRARIES ${DASH_LIBRARY} "dart-@dart_variant@")
set(DASH_CXX_FLAGS "@VARIANT_ADDITIONAL_COMPILE_FLAGS@ @CMAKE_CXX_FLAGS_RELEASE@")
set(DASH_INSTALL_PREFIX "@CMAKE_INSTALL_PREFIX@")

# find DART package
find_package(DART-@DART_VARIANT@ REQUIRED HINTS "${DASH_INSTALL_PREFIX}/cmake")

#include exportet targets
include("${DASH_INSTALL_PREFIX}/cmake/${DASH_LIBRARY}-targets.cmake")

Notes

All placeholders (denoted by "@") in the project config file will be replaced by the correspondent CMake variables. To keep the install directory clean, all .cmake files are placed in the "cmake" sub directory.

Alternatives

There are some common but (at least in my own opinion) really bad solutions for this problem:

  1. generate a shell script that has to be sourced (UPC++)
  2. printing all compiler flags before the compilation of the library. In fact, this is always useful for debugging, but not to tackle this problem.

Sources and further information

  1. How to create a ProjectConfig CMake file
  2. CMake find_package (CMake 3.6.1 documentation)
  3. DASH Project
  4. UPC++ at Bitbucket

Zurück

Kommentare

Einen Kommentar schreiben

Was ist die Summe aus 2 und 5?

Ähnliche Beiträge

von

We reverse engineer a Dotnet Monitor in Windbg to see how it is internally implemented.

von

As SSDs are getting cheaper, low HDD / SSD ratios of 10/1 or better become an option. This article describes how to tune bcache for this scenario from an empirical perspective.