Independent Refactoring is Irresponsible

Both Michael Feathers and Rachel Davies recently recently wrote about attempts to make refactoring a more explicit step in the development process by adding a particular refactoring task to the board.

It got me thinking about my latest project, a green-field application where, in the last week, I think we almost tripled our estimated velocity. You might think that we gamed the estimates, bloating figures to make us look good, but we did not. We still used relative complexity to estimate essential work that must be done. We did learn a little bit more than the week before, but the biggest change was actually some preparatory refactoring. Before I explain why it worked, I’m going to take a slight detour.

I remember working with one client who spun up a “refactoring team”. It sounded great at the outset – a legacy application that had plenty of code as they knew the development team cut corners to meet their milestones. Rather than completely halt new development, they split out a small team who would refactor mercilessly. This team spent one whole month renaming classes, adding tests, adding patterns here and there, fighting the big cyclomatic complexity numbers and then claiming victory on their poor opponent (the codebase). The result after the refactoring team… new feature development slowed down even more.

When investigating why feature development slowed down, we discovered the following:

• New, and unfamiliar designs – The refactoring team did some wonderful work cleaning up certain parts of the codebase. THey made some places consistent, introduced a few patterns to tackle common smells and honestly helped reduce the codebase. What they neglected to do was to inform the other developers of the new design, where they now needed to look for the same functionality and the intention behind the newly named classes. Instead, the developers working on new functionality struggled to find the huge, very finely commented code they were familiar with and then when they tried to apply their old technique for fixes, failed to do.
• Immediately irrelevant refactoring – With most codebases, there are parts that change a lot, parts that change a bit, and parts that almost never change. In my experience (disclaimer: not researched) those parts that change a lot and are highly complex end up as a huge source of bugs. Those parts that don’t change can remain ugly and still be perfectly fine. In the case of this client, a lot of effort spent refactoring ended up in areas where new functionality wasn’t being added.
• A divide in cultures – I heard a few snarky comments during this time from the new feature development team about the refactoring team, basically implying most of them to be developer-divas whilst they had to do all the grunt work. The result… by outsourcing refactoring, the new feature team basically cared less about the codebase and I’m sure they weren’t helping the clean up with the code they added.

Image taken from Will Lion’s Flickr stream under the Creative Commons licence

My reading on lean thinking taught me that you need to Build Quality In and that separately quality from the product ends up costlier and results in poorer quality.

The answer… is actually quite clear in Martin Fowler’s book on Refactoring:

Refactor because you want to do something else (i.e. add function, or fix a bug)

I return to my current situation. We achieved the tripling in velocity because we spent time thinking about why adding a new feature was so cumbersome. I’ll admit that I spent a lot of time (almost a day) trying to add part of a new feature, attempting a few refactors and rolling back when they did not work. I was trying to get a feel for what steps we did most often, and attempted several approaches (most failed!) to make it simpler, clearer and added the least amount of code. We did settle on some patterns and we realised its benefits almost immediately – adding a new feature that previously took us a day to implement now took us only an hour or two with tests.

I find that sometimes the most satisfying part of software development is actually reshaping existing code so that the addition of a new feature is just a single method call, or just a single instance of a class. Unfortunately I don’t see this often enough.