Autotests

Source code validation is a critical step in any software library, particularly for verifying the portability of code to different processors and platforms. Packaged with liquid are a number of automatic test scripts to validate the correctness of the source code. The test scripts are located under each module's tests directory and take the form of a C source file. The testing framework operates similarly to cppunit and cxxtest , however it is written in C. The generator script scripts/autoscript parses these header files looking for the key " void autotest_ " which corresponds to a specific test. The script generates the header file autotest_include.h which includes all the modules' test headers as well as several organizing structures for keeping track of which tests have passed or failed. The result is an executable file, xautotest , which can be run to validate the functional correctness of liquid on your target platform.

Macros

Each module contains a number of autotest scripts which use pre-processor macros for asserting the functional correctness of the source code.

  • CONTEND_EQUALITY \((x,y)\) asserts that \(x==y\) and fails if false.
  • CONTEND_INEQUALITY \((x,y)\) asserts that \(x\) differs from \(y\) .
  • CONTEND_GREATER_THAN \((x,y)\) asserts that \(x \gt y\) .
  • CONTEND_LESS_THAN \((x,y)\) asserts that \(x \lt y\) .
  • CONTEND_DELTA \((x,y,\Delta)\) asserts that \(|x-y| \lt \Delta\)
  • CONTEND_EXPRESSION \((expr)\) asserts that some expression is true.
  • CONTEND_SAME_DATA \((ptrA,ptrB,n)\) asserts that each of \(n\) byte values in the arrays referenced by \(ptrA\) and \(ptrB\) are equal.
  • AUTOTEST_PASS \(()\) passes unconditionally.
  • AUTOTEST_FAIL \((string)\) prints \(string\) and fails unconditionally.
  • AUTOTEST_WARN \((string)\) simply prints a warning.

The autotest program will keep track of which tests elicit warnings and add them to the list of unstable tests.

Here are some examples:

  • CONTEND_EQUALITY (1,1) will pass
  • CONTEND_EQUALITY (1,2) will fail

Running the autotests

The result is an executable file named xautotest which has several options for running. These options may be viewed with either the -h or -u flags (for help/usage information).


$ ./xautotest -h
Usage: xautotest [OPTION]
Execute autotest scripts for liquid-dsp library.
  -h,-u         display this help and exit
  -t[ID]        run specific test
  -p[ID]        run specific package
  -L            lists all scripts
  -l            lists all packages
  -x            stop on fail
  -s[STRING]    run all tests matching search string
  -v            verbose
  -q            quiet

Simply running the program without any arguments executes all the tests and displays the results to the screen. This is the default response of the target make check .

Autotest Examples

Run all autotests matching the string " firfilt ":


$ ./xautotest -s firfilt
40: firfilt_xxxf
40: firfilt_xxxf:
    203 :   PASS    passed    8 /    8 checks (100.0%) : firfilt_rrrf_data_h4x8
    204 :   PASS    passed   16 /   16 checks (100.0%) : firfilt_rrrf_data_h7x16
    205 :   PASS    passed   32 /   32 checks (100.0%) : firfilt_rrrf_data_h13x32
    206 :   PASS    passed   64 /   64 checks (100.0%) : firfilt_rrrf_data_h23x64
    207 :   PASS    passed   16 /   16 checks (100.0%) : firfilt_crcf_data_h4x8
    208 :   PASS    passed   32 /   32 checks (100.0%) : firfilt_crcf_data_h7x16
    209 :   PASS    passed   64 /   64 checks (100.0%) : firfilt_crcf_data_h13x32
    210 :   PASS    passed  128 /  128 checks (100.0%) : firfilt_crcf_data_h23x64
    211 :   PASS    passed   16 /   16 checks (100.0%) : firfilt_cccf_data_h4x8
    212 :   PASS    passed   32 /   32 checks (100.0%) : firfilt_cccf_data_h7x16
    213 :   PASS    passed   64 /   64 checks (100.0%) : firfilt_cccf_data_h13x32
    214 :   PASS    passed  128 /  128 checks (100.0%) : firfilt_cccf_data_h23x64

==================================
 PASSED ALL 600 CHECKS
==================================

Run test 405 :


$ ./xautotest -t 405
    405 :   PASS    passed   16 /   16 checks (100.0%) : demodsoft_apsk8
==================================
 PASSED ALL 16 CHECKS
==================================