Code coverage is it good or bad?

Code Coverage!

One question I have been pondering over for a while. We often focus on high code coverage and use it as guide for measuring the quality of our code, but is it appropriate or the right thing?

Recently I saw infoq video Beauty Is in the Eye of the Beholder, so here are my thoughts.


Its bad because its percentage of your lines of code covered by tests. But how can that be bad. Like any metric developers/development managers/team leader often find it easier it as a measure of code quality. Now there is nothing wrong with that if that meant that the quality of tests you were writing were excellent. Now let say if you as team agree that the tests are doing their job why do you measure some thing you already know, right. So often its used by teams where the quality does not meet their expectation. If the code is not meeting your expectation how will a measure which is absolute help you improve it. In reality it won’t. What will happen often team will focus on getting a very high coverage figure. Once you have achieved the high figure it will feel like a big stepping stone achieved. In some case this may be right, but it does not guarantee that you have

  • You have not over engineered your code to achieve the high figure you were aiming for.
  • What about coupling and cohesion? Still may be awful in your code but your test stats will be green.
  • There is good chance you may written tests for the sake of high coverage figure.
  • When it come to changing your code you are likely have with brittle tests which need multiple changes every time you change your code.

Now it does not feel very good when on hand you have high coverage, but brittle tests.

The real issue here is that fact that even if you have excellent code coverage its does not guarantee that you would have written maintainable code.

In a mature team, where there is common understanding of quality & maintainability, the most important measure is the gut feel, every time you have make a small change to the code the number tests you have to change, how brittle are the tests and more importantly how confident that the the change will work without and side-effect.

I am not proposing that don’t measure your code coverage. What I am proposing is that only use the measure if it really helps your team develop quality code and release value early. One final thought, your best feedback is the rate of change and time to test & release a small change though to production and getting early feedback from your real testers i.e. the users of the system.

More to come on this …




About Gautam pachnanda

Gautam Pachnanda Agile Delivery Leader I am a Delivery Leader & a Proficient software engineer and leader with over 15 years commercial experience and a track record of delivering quality solutions. Having been involved in all aspects of software development on various roles, I am now specialising in Technical leadership, Software Architecture, Object Oriented Design and Implementation. I am also a keen proponent of agile methodology and have been involved in various agile projects in my current role. I have successfully led teams of varying sizes and accomplished in all areas relating to the development, delivery, innovation, quality, user-centric and practical solutions. I am a dedicated and enthusiastic with a wide range of experience in traditional and agile software delivery. I am currently leading a talent team of IT professional covering the breadth of Development, Quality Assurance & Business Analysis in Kanban/XP environment in new Growth area Razsor of Trader Media Group. Over the years I had opportunities and have build a solid understanding of business demands and the requirements expected to form a successful delivery team. Educated to Masters degree level in Information Technology, I believe in continuous improvement and effective delivery. I enjoy being able to inspire, motivate and lead teams to deliver software that exceeds expectation of its users and sellers and love the environment and team I work with.
This entry was posted in Code Quaility. Bookmark the permalink.