Inspection and Testing

Diomidis Spinellis
Department of Management Science and Technology
Athens University of Economics and Business
Athens, Greece
dds@aueb.gr

Example: Processor Testing

Intel 8088 processor register test routine from IBM's 1981 PC BIOS. The carry flag is used as the loop index.
8088 Processor Test

Static Verification

Examples: Characteristics:

Identified Issues

Some FindBugs Issues

(FindBugs reports more than 200 issues.)

A FindBugs Run

A FindBugs run

GCC Best Practices

C Compilation Example

main(char *argv[])
{
        int a, b, c;

        a = b;
        printf("%g\n", a);
}
Plain compile:
$ gcc t.c
$
Compile with warnings:
$ gcc -O4 -Wall -Werror t.c
cc1: warnings being treated as errors
t.c:9: warning: return-type defaults to `int'
t.c:9: warning: first argument of `main' should be `int'
t.c:9: warning: `main' takes only zero or two arguments
t.c: In function `main':
t.c:13: warning: implicit declaration of function `printf'
t.c:13: warning: double format, different type arg (arg 2)
t.c:10: warning: unused variable `c'
t.c:14: warning: control reaches end of non-void function
t.c:10: warning: `b' might be used uninitialized in this function

Lint Best Practices

Tracing Tools

InterfaceTool
Operating Systemstrace
Libraryltrace
Networktcpdump
Resource Snapshotlsof

Using Tracing Tools

[sl]trace Tips and Tricks

Example: System Call Tracing

Which shared libraries is dot loading?
$ strace dot </dev/null 2>&1 | fgrep .so | fgrep -v ENOENT
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/usr/lib/libfreetype.so.6", O_RDONLY) = 3
open("/usr/lib/libpng.so.2", O_RDONLY)  = 3
open("/usr/lib/libjpeg.so.62", O_RDONLY) = 3
open("/usr/lib/libz.so.1", O_RDONLY)    = 3
open("/lib/libm.so.6", O_RDONLY)        = 3
open("/lib/libc.so.6", O_RDONLY)        = 3

Example: Library Call Tracing

How does the whoami command obtain its data?
$ ltrace whoami
[...]
geteuid()                                         = 1000
getpwuid(1000, 0x40013678, 0xbffffcac, 0x08048cff, 0x4012ee48) = 0x401310f0
puts("dds"dds
)                                       = 4
exit(0 <unfinished ...>

Unit Testing

JUnit Example

Test Class and Fields

public class RationalTest {
    private Rational r12;
    private Rational r13;
    private Rational r56;
}

Initialization Code

    @Before public void setUp() {
        r12 = new Rational(12);
        r13 = new Rational(13);
        r56 = new Rational(56);
    }

A Test Method

    @Test public void simpleAdd() {
        Rational result = r12.add(r13);
        assertTrue(result.equals(r56));
    }

Run the Class's Test Methods

    public static void main (String... args) {
        junit.textui.TestRunner.run(RationalTest.class);
    }

Textual Regression Testing

Example: Testing the Sort Program

#!/bin/sh
FILES=`cd input; echo *`

# Prime test data
if [ "$1" = "-p" ] ; then
        for in $FILES ; do
                sort input/$i >old/$i
        done
fi

# Compare old with new result
for in $FILES ; do
        sort input/$i >new/$i
        if diff old/$i new/$i ; then
                echo "OK $i"
        else
                echo "FAIL $i"
                exit 1
        fi
done

Test Coverage in C/C++ Code

To investigate test coverage, you can use basic block profiling. Example:
 83.33% of 650 source lines executed in file hilbert.c

Example: Test coverage of Perl's source code versus the number of executed test cases

Test coverage of Perl's source code versus the number of executed test cases

Further Reading