Run time code coverage using Emma

Emma is a free java code coverage tool which measures and reports code coverage of java based products. The most important and usable factor of Emma is, its ability to measure code coverage during the application runtime. At WSO2, we have been searching for a tool to get QA test coverage of our middleware products. Automated QA tests are executed against the binary products. Therefore, source level unit test coverage does not help to identify the figures of QA test coverage. Emma was the best tool which satisfied our requirements.

The procedure is quite simple. Suppose you are going to get the coverage of a binary which obviously contains hundred of jars. First, you need to prepare a list of jars which requires to be instrumented. The list can be prepared easily in *nix environments as follows.

xargs -n 1 $JAVA_HOME/jre/bin/java -cp emma.jar emma instr -m overwrite -cp < jarlist.txt

This inserts tracking code to the jars listed in jarlist.txt.

Now, you should restart your application and Emma starts to gather coverage data. If you have an automated functional test suite, run it over the instrumented binary or test the product manually.

After completing the testing, stop your application. Now, we need to generate the coverage report. Issue the following command.

java -cp emma.jar emma report -r html -in coverage.em,coverage.ec

This generates an html report as shown below.





Currently, we are in the process of developing a selenium based automation framework and we use this test coverage mechanism to find out the components which require more tests.