Skip to content

CTest/CDash without CMake

In general, four files are needed to test and submit to a CDash dashboard:

  1. CTestConfig.cmake
  2. CTestCustom.cmake
  3. CTestTestfile.cmake

These files can be generated by hand. The detailed contents and format of the above files is beyond the scope of this documentation but can be found here.

NERSC provides a Python package PyCTest to streamline the generation of the above files. This package enables easily wrapping any build system into a CDash submission without CMake and generate tests. For example:

  • Makefiles
    • pyctest.pyctest.CONFIGURE_COMMAND = "./configure"
    • pyctest.pyctest.BUILD_COMMAND = "make -j8"
  • Python Project
    • pyctest.pyctest.BUILD_COMMAND = "python setup.py install"

Using PyCTest

Install pyctest from Anaconda or PyPi

  • conda install -c conda-forge pyctest
  • python -m pip install -v pyctest

PyCTest does not simply implement routines for writing the necessary files, instead the package creates direct bindings to the CMake/CTest source code. The Anaconda distributions are pre-compiled and recommended. The PyPi distribution is a source distribution that will require building CMake from scratch -- which can take a significant amount of time depending on your system specifications.

PyCTest Driver Example

Write a primary driver script for the project, e.g. pyctest-runner.py, and place in the top-level directory of the project.

#!/usr/bin/env python

import os, sys, platform
import pyctest.pyctest as pyctest
import pyctest.helpers as helpers

cwd = os.getcwd()
# process standard arguments
args = helpers.ArgumentParser("PyCTest", cwd, cwd, vcs_type="git",
                              build_type="MinSizeRel").parse_args()
# base web address of dashboard
pyctest.DROP_SITE = "cdash.nersc.gov"
# custom setup.py command (runs CMake)
pyctest.CONFIGURE_COMMAND = "python setup.py configure"
# build and install
pyctest.BUILD_COMMAND = "python setup.py install"
# create test
test_dir = os.path.join(pyctest.SOURCE_DIRECTORY, "examples", "Basic")
pyctest.test(name="basic", cmd=["python", "basic.py", "--", "-VV"],
             properties={ "WORKING_DIRECTORY" : test_dir })
# run stages
pyctest.run()