Good question if you really don’t know what it means :) I’ve more than 7 years of software development experience and to be honest, I’ve never seen a company who uses code coverage as a metric unfortunately. We’re developing a lot of beautiful software and in most cases we’re ending up with lots of code messes :( You can argue on why it happens like this, but I’m not gonna go into that today. Let’s get back to the main topic:

What is Code Coverage

Code coverage is any metric of completeness with respect to a test selection criterion. Or you can say, it basically shows which part of your code get executed and which part don’t. So roughly speaking, you have a code coverage tool that monitors your code and when you finish monitoring your code, this tool generates a code coverage report. In this report you see so many numbers, but most importantly you see which lines get executed and which lines don’t. And if you sum the total number of lines which were executed you would find the executed lines total. With this number you can easily find your code coverage percentage by dividing it by the total number of lines which could have been executed. And if you reach 100%, it means that you can be proud of your team :) Practically reaching this number is pretty much a dream since there are a lot of constraints that ties you to the business side of this world. I would say, a real good agile team would hit somewhere close to 75-80%.

How Do You Use This Tool?

I think usage of a code coverage tool cannot be just by itself and plain application code. You probably wanna work with some kind of automated test suite beside the actual code. By this way, you can try to cover all your code by your tests (unit, integration, performance, smoke, fit, etc). You might quickly think that “well, in this case if I have a lot of test cases then I can hit 100%” :S Hmm, in theory it might seem like that, and even though covering it by all over tests sometimes gives the 100%, but this number might be quite deceptive. And don’t forget your code is most probably not a plain “Hello World” application :)

Deception

I believe, testing is based on samples. You cannot cover all possible situations in a practical world. I think so many people still thinks that if their unit tests all pass with 100% code coverage, they are 100% covered. In fact, this is not true. Code coverage can only tell you how much of your code is being tested. It cannot tell you anything more than that. Let’s speak on an example:

If you look at the above code, you’ll quickly notice that compiler will do a shortcut, and won’t check the “checkMeIAmAlsoImportant()” method. If you have a test case that only covers the first part of the code, you’ll hit 100% code coverage, but actually you are not hitting 100%, ‘coz you still have to cover this logic. Example might not be 100% accurate, however 100% code coverage doesn’t mean that all your conditional paths are covered. You have to upgrade your coverage level from Basic Coverage to Condition Coverage.

Is condition coverage enough? Hmm, not quite. Let’s say that you are using condition coverage and you hit 100%, you are probably still looking at a lot of possible bugs, since line numbers don’t cover all your application state (if you just think about the fragile part of your code like bitwise operations, regex, stack/heap, tree traversals, recursives, etc). To cover all of these things we have to upgrade to Functional Coverage that tracks which states have been hit. To do this, you would probably create a random stimulator (more info) which attempts to call your function with random inputs.

Conclusion

Even with functional coverage, it’s impossible get to 100% for virtually any function. Since bug-free code is pretty much impossible :) There will be always some corner cases. To reduce your risk and to reach 100%, you might want to look at more about code coveraging techniques like Cross Coverage, Transition Coverage, External Coverage etc..

Unit testing and code coverage are only tools. They just show/provide us how to increase the quality of our software, but 100% code coverage obviously does not mean 100% code quality. There are so many other tools and techniques in the software world that aims same outcome. I wish we would have been a little bit more sensible about code coveraging, testing, better design and maybe some refactoring. Nobody can be perfect, we all know this. But we can think better, design better, and we can always try to use baby steps!

And one little tiny note about my blog posts: these are all my humble opinions, or the opinions that I agree with other people. Everybody welcomes to criticize them. In the end, I’m still trying to develop myself.