unit testing - CMake - run test as part of the build process and capture stdout output to file -

we have several unit tests run part of our build process.

to achieve have helper script creates custom command runs test, , if successful, creates file "test_name.passed".

i add custom target "test_name.run" depends on "test_name.passed".

the idea if "test_name.passed" doesn't exist or older "test_name", custom command run.

builds continue run custom command until test passes. once passes, subsequent builds won't call custom command, test won't run when doesn't need to.

so far works described

here script:

# create command runs test , creates sentinel file if passes add_custom_command(     output  ${test_name}.passed     command $<target_file:${test_name}>     command ${cmake_command} -e touch ${test_name}.passed     depends ${test_name}     )  # create test.run module depends on test.passed add_custom_target(${test_name}.run         depends ${test_name}.passed     ) 

the problem - noise on stdout

the problem our tests log tonne of information stdout, , makes noisy build.

i'm trying capture stdout file, , in event of failure, display test output.

my first attempt try bash shell scripting syntax - capture stdout file , when exit status error, cat file.

add_custom_command(     output  ${test_name}.passed     command $<target_file:${test_name}> > ${test_name}.output || cat ${test_name}.output     command ${cmake_command} -e touch ${test_name}.passed     depends ${test_name}     ) 

this doesn't work, if test fails getting sentinal "test_name.passed" file created, means next time try build thinks test passed.

possible sub-standard fix

by integrating ctest, can run each test through ctest , use command-line option --output-on-failure

add_custom_command(     output  ${test_name}.passed     command ctest --build-config $<configuration> --tests-regex ${test_name} --output-on-failure     command ${cmake_command} -e touch ${test_name}.passed     depends ${test_name}     ) 

the problems two-fold.

1. it vastly increases build time. each , every test has executed through separate ctest process, registered test names parsed against regex, etc. number of individual tests have time adds considerably.
2. ctest outputs lot of noise default. specifying --quiet flag suppresses --output-on-failure flag, can either have noisy output or no output - there no way failures.

question

is there way achieve want?

ie:

• run test manually (ie: not through ctest)
• capture output file
• only output file in event of test exit status indicating failure.
• touch sentinel file in event of test exit status indicating success.

bonus points cross-platform method, if has linux only, it.

the problem there no standard way redirect output command invoked through add_custom_command file. cmake command execute_process have capability.

thus possible solution run test executable configured cmake script run cmake custom command. following code sketches necessary steps:

in cmakelists.txt adds test, configure cmake script template:

configure_file("test_runner.cmake.in" "test_runner_${test_name}.cmake" @only) 

then add custom command invoke script @ build time:

add_custom_command(     output  ${test_name}.passed     command ${cmake_command} -p "test_runner_${test_name}.cmake" $<target_file:${test_name}>     depends ${test_name}     working_directory "${cmake_current_binary_dir}") 

the actual path of test executable passed script argument via generator expression.

the template test runner script test_runner.cmake.in runs test executable execute_process , redirects error output log file:

set (_testexecutable "${cmake_argv3}") execute_process(command ${_testexecutable} error_file "@test_name@.output" result_variable _testresult) if (_testresult)     file(remove "@test_name@.passed")     file(read "@test_name@.output" _contents)     message (status "${_contents}") else()     file(write "@test_name@.passed" "") endif() 

if test fails, script removes sentinel file , outputs error log. if test succeeds, script creates sentinel file.