Category: Agile

The Retrospective Handbook – Now in Print

Last year, I announced the digital version of The Retrospective Handbook being released. As much as I feel digital books are important, I am one of those people who like reading using a physical copy of a book. It’s great for you, and it’s also a great way to give one away. And now you can too!

The Retrospective Handbook

The print copy of the book is now available via Amazon (all the links are below). Buy one for you, your team or as a gift today.

Independent Refactoring is Irresponsible

Both Michael Feathers and Rachel Davies recently recently wrote about attempts to make refactoring a more explicit step in the development process by adding a particular refactoring task to the board.

It got me thinking about my latest project, a green-field application where, in the last week, I think we almost tripled our estimated velocity. You might think that we gamed the estimates, bloating figures to make us look good, but we did not. We still used relative complexity to estimate essential work that must be done. We did learn a little bit more than the week before, but the biggest change was actually some preparatory refactoring. Before I explain why it worked, I’m going to take a slight detour.

I remember working with one client who spun up a “refactoring team”. It sounded great at the outset – a legacy application that had plenty of code as they knew the development team cut corners to meet their milestones. Rather than completely halt new development, they split out a small team who would refactor mercilessly. This team spent one whole month renaming classes, adding tests, adding patterns here and there, fighting the big cyclomatic complexity numbers and then claiming victory on their poor opponent (the codebase). The result after the refactoring team… new feature development slowed down even more.

When investigating why feature development slowed down, we discovered the following:

  • New, and unfamiliar designs – The refactoring team did some wonderful work cleaning up certain parts of the codebase. THey made some places consistent, introduced a few patterns to tackle common smells and honestly helped reduce the codebase. What they neglected to do was to inform the other developers of the new design, where they now needed to look for the same functionality and the intention behind the newly named classes. Instead, the developers working on new functionality struggled to find the huge, very finely commented code they were familiar with and then when they tried to apply their old technique for fixes, failed to do.
  • Immediately irrelevant refactoring – With most codebases, there are parts that change a lot, parts that change a bit, and parts that almost never change. In my experience (disclaimer: not researched) those parts that change a lot and are highly complex end up as a huge source of bugs. Those parts that don’t change can remain ugly and still be perfectly fine. In the case of this client, a lot of effort spent refactoring ended up in areas where new functionality wasn’t being added.
  • A divide in cultures – I heard a few snarky comments during this time from the new feature development team about the refactoring team, basically implying most of them to be developer-divas whilst they had to do all the grunt work. The result… by outsourcing refactoring, the new feature team basically cared less about the codebase and I’m sure they weren’t helping the clean up with the code they added.

Litter
Image taken from Will Lion’s Flickr stream under the Creative Commons licence

My reading on lean thinking taught me that you need to Build Quality In and that separately quality from the product ends up costlier and results in poorer quality.

The answer… is actually quite clear in Martin Fowler’s book on Refactoring:

Refactor because you want to do something else (i.e. add function, or fix a bug)

I return to my current situation. We achieved the tripling in velocity because we spent time thinking about why adding a new feature was so cumbersome. I’ll admit that I spent a lot of time (almost a day) trying to add part of a new feature, attempting a few refactors and rolling back when they did not work. I was trying to get a feel for what steps we did most often, and attempted several approaches (most failed!) to make it simpler, clearer and added the least amount of code. We did settle on some patterns and we realised its benefits almost immediately – adding a new feature that previously took us a day to implement now took us only an hour or two with tests.

I find that sometimes the most satisfying part of software development is actually reshaping existing code so that the addition of a new feature is just a single method call, or just a single instance of a class. Unfortunately I don’t see this often enough.

Reflecting on Feature Leads

Last year, I wrote about trialling the idea of Feature Leads. I think the idea worked out and I would encourage more teams to adopt this approach. It helped devolve some of the responsibility and made the work more engaging for developers. Looking back at the list of things to consider, I would now add more items.

What is missing list?

  • New environment needs? – Do we require new environments to support business stakeholders in their own testing, or do we overload an existing environment? If we rely on external dependencies, can they support the number of environments that we need.
  • Identify external dependencies – If we are working with external vendors, we need to probably be a bit more upfront in working out when key dates are so that we can co-ordinate
  • Has the business made any external/internal commitments – As much as teams get frustrated by arbitrary dates set by the business, it’s useful to know if a) any have already been set, or b) business stakeholders want to communicate dates because that means you need to manage expectations and ensure that those commitments are balanced with other priorities going on.
  • Is the solution simple, but evolvable – Does the approach make any anticipated work harder than it needs to be? Does it balance out time to market? Can we go for an even more lightweight solution and substitue a more complex one later if needed?
  • Do we need to build anything for the feature? – Is software even needed, or can some lightweight business process take care of the need? If we build this, how long will be used, and therefore how much effort in maintaining it/adding automated tests around it?

Looking back at the list of responsibilities, I think these elements help add to a standard list of what things to consider when designing any sort of software solution, and not just the building of it, but thinking about the long term effects of it (who uses it, who’s going to run it, who’s going to maintain it).

Managing Ruby Development Environments

One of the principles I like is being able to set up new development environments very quickly. The java space offers many libraries for managing your environment that means that each project works in a separate space. In contrast, in the .Net space an anti-pattern is one that often requires many installs to your GAC (Global Assembly Cache) often through the use of a “mouse-driven”-only installer.

Fortunately the ruby community offers a number of tools for managing both versions of ruby and the libraries that you use. The ones that I will often reach for, now include:

  • RVM – Ruby Version Manager. Allows you to have different versions of ruby, and to quickly switch between them
  • Bundler – Management of gems.

The ultimate acceptance test for this, is can developers simply “check-out” and go. The lead time to setup a new development environment should be very quick.

Note that there is now a competing tool for managing environments called RBenv although integrated tool support (like in RubyMine) is only starting to come through

Taming the Hippo (CMS) Beast

I eluded in a previous post our struggles dealing with the HippoCMS platform. It wasn’t our preferred path, but a choice handed down from above. Enough said about that. It’s useful to understand a little bit about the environment we were using it.

I believe the pressure to choose a CMS came from a deadline that required some choices about platform choice to be made in an organisation. At this time, the extent to what the actual product was unknown. Our experience working with other clients is that you should generally work out what you want to do before you pick a product, or the platform will often dictate and limit your ability to do things. My colleague Erik Dörnenburg has been writing more about this recently.

The premises of a CMS are alluring for organisations. We have content… therefore we need a content management system. The thought ensues, “Surely we can just buy one off the shelf.” Whether or not you should use a CMS is for another blog post, and you can read some of Martin Fowler’s thoughts on the subject here.

We wanted to protect our client’s ability to evolve their website beyond the restrictions of their CMS, so we architected a system where content managed in a CMS would sit behind a content service, and a separate part of the stack focused on the rendering side. It looks a little like this:

ContentService

The issues that we faced with HippoCMS included:

A small community based on the Java Content Repository

Hippo is based on the Java Content Repository (JCR) API, a specification for standardising the storage and access of content. Even as I write this blog, putting “JCR” or “Java Content Repository” I am forced to link to the wikipedia page because I spent three minutes trying to find the official Java site (it looks like the official site is hosted by Adobe here). If the standard is small, the community surrounding the products are naturally going to be smaller. Unlike users of spring, putting a stacktrace into google will generally show the sourcecode of the file rather than how someone got over it. I’d be happy living on the bleeding edge… if the technology was actually pretty decent.

Unfortunately a lot of the gripes I write about are the fact that the product itself is based on the the JCR specification. Some simple examples include:

  • A proprietary query syntax – You query the JCR with an xpath-like query language. It’s actually less useful than xpath, such as not implementing all functions available in xpath and some weird quirks
  • Connecting to the repository via two mechanisms – Either RMI (yuck! and inefficient) or in memory. This automatically limits your deployment options to the application container model. Forget fast feedback loops of changing, starting a java process and then retesting.

Hippo CMS UI generates a huge number of exceptions
One reason Hippo was selected was for the perceived separability of the CMS editor and the website component (referred to as the Hippo Site Toolkit). We didn’t want to tightly couple the publishing/rendering side to the same technology stack as the underlying CMS. Hippo allows you to do this by having separately deployed artefacts in the application container. Unfortunately, the Wicket-based UI (maybe because we used it without the Hippo Site Toolkit) generates exceptions like nobody’s business. We spent some effort trying to understand the exceptions and fix them, but there were frankly too many to mention.

Poor taxonomy plugin implementation
One of the reasons Hippo was allegedly picked was for the taxonomy plugin. Unfortunately this gave us no world of pain both in usability and in terms of maintaining it. In terms of the specific issues we faced with the maintenance included the multi-language support (it didn’t allow that) and then just simply getting it deployed without issues.

CMS UI lack of responsiveness
Our client’s usage of the site wasn’t very big. Less than 300 articles and, at the peak, about 10 concurrent users. Let’s just say that even with three people, the UI was sluggish and unresponsive. We tried some of the suggestions on this page, but it’s a bit of a worry that it can’t responsively support more than one user out of the box with standard configuration.

Configuration inside the JCR
Most of our projects take a pretty standard approach to implementing Continuous Delivery. We want to easily source control configuration, and script deployments so that releases into different environments are repeatable, reliable, rapid and consistent. Unfortunately a lot of the configuration for new document type involves “switching a flag to capture changes”, playing around with the UI for a new document type” and then exporting a bunch of XML that you must then load with some very proprietary APIs.

After several iterations, we were able to streamline this process as best we could but that took some time (I’m guessing about a developer two weeks full time).

Lack of testability
We spent quite a bit of effort trying to work out the best automated testing strategy. Some of the developers first tried replicating the JCR structure the UI would recreate but then I pointed out that would give us no feedback of if Hippo changed the way did its mapping. We ended up with some integration tests that drove the wicket-based UI (with a wonderfully consistent but horrid set of generated IDs) and then poked our content service for expected results.

A pair of developers worked out a great strategy for dealing with this, working out the dynamically generated APIs and driving the UI via Selenium Webdriver to generate the data we would query inside the proprietary XML-based data store.

Lack of real clustering
In “enterprise” mode, you can opt to pay for clustering support although it’s a little bit strange because you aren’t recommended to upgrade a single node within a cluster when other nodes are connected to the same datastore (in case the shared state is corrupted). This kind of makes seamless upgrades without complicated DB mirror/restore and switcheroo really difficult. We ended up architecting the system for a degraded service using caches on the content service as a compromise to the “clustered” CMS.

Summary
As much as I wish success for the Hippo group, I think many of the problems are around its inherent basis on the JCR. I do think that there are a couple more things that could be done to make life easier for developers including increasing the amount of documentation and thinking about how to better streamline automated, frequent deployments around the CMS.

Looking back at a year with a client

Over the last twelve months, I’ve worked with a client to rebuild a digital platform and a team to deliver it. It’s now time for us to leave, and a perfect time to reflect on how things went. The digital platform rebuild was part of a greater transformation programme that also involved the entire business changing alongside at almost all levels of people in the organisation. The programme also outlined, before we arrived, outlined a complete change in all technology platforms as well (CRM, CMS, website) to be rebuilt for a more integrated and holistic service offering.

Our part in this program turned into building the new web digital platform, working against a very high level roadmap, and a hard marketing deadline. We ended up building the site using Ruby on Rails serving content driven by a 3rd party decisioning platform (much like Amazon recommendations) guided by the business vision of better tailored content for end users. We didn’t have much input into the final choice of several products. I’m very proud of the end result, particularly given the tense and short-timed framed environment in which we worked. Here are some examples of constraints we worked with:

  • 4 Product Owners over the span of 11 months – From January this year, through to the end of October, the business was onto its fourth Product Owner for the digital platform. Building a consistent product is pretty much nigh impossible with changing product hands, and trying to bridge work from one Product Owner to the next was definitely a challenge.
  • Constant churn in the business – The 4 product owners is one instance, but we would often be working with people in the business to work out what should be done, only to find that the following week they were no longer with the business.
  • 3 Design Agencies engaged resulting in “reskinning” approved by the board before the 6 month public launch – We saw several “design changes” done by firms well stocked with people capable of generating beautifully-rendered PDFs that were signed off. However often these would imply new functional work, or be impractical to the web medium.
  • Marketing deadlines announced well before the development team had been engaged - A common pattern in the business was marketing launching a press release announcing a date, well before the people involved in delivering it were made aware, or even consulted on it.
  • PM Explosion – At one point, it felt like we had more Project Managers on the group planning out work with budgets and timelines that would be approved well before the development team had been approached.

Even with these constraints we’ve been able to deploy to production 37 times in the last three months and more since the original MVP go-live in July. Part of what I’m particularly proud of is the team where we were able to achieve the following:

  • Building an Evolvable Architecture – We questioned the original choice and need for a CMS but with a constraint that a decision had been made on buying these tools, we architected a solution that would hide the implementation details of the CMS via a content service. With our TW experience and pain dealing with CMSes that are shadowed by business need, we wanted something that would not constrain what the business could achieve (hence the decoupling). We even had a chance to prove this out when the business requirements quickly hit the limit of the CMS’s built in categorisation module.
  • Responding to Change – The business roadmaps seems to change on a daily basis, and our team was able to quickly tack to accommodate these business changes. We changed the team structure as the team size increased, changed the team structure as we went live, and again as people in the business changed. Whilst our process felt similar, it would look nothing like a textbook XP, Scrum or Kanban process.
  • Improving the Process – Our team has been constantly trying to change the process not only internally to the development team, but also helping people in the business find ways of improving their own way of working. Progress has been slow as the change that starts falters as people leave. Retrospectives have been a key tool but also has the ability for the team to feel empowered with recommending and pursuing improvements they see fit.
  • Setting an example of transparency – Showcases are key to the business, and we would offer fortnightly showcases to the features built to the entire organisation. Huge numbers of people came along and I found it fascinating that it was one place where people had an opportunity to talk across silos. This sometimes slowed down our ability to show what we had actually done, but I felt exposed missing communication structures that people still needed.

At a technical level, I’m really proud of some of the principles I wanted to achieve at the start and that the team lived throughout (I’d love to hear what their experience is). Some of these include:

  • Fast developer setup – Getting started on each new machine should be fast without complicated installation processes
  • Developers rotating through operations – There’s nothing like supporting the code you wrote to help developers understand the importance of logging, test cases that are missed and just experiencing what production support is like
  • DevOps culture – Everyone on the team is capable of navigating puppet, knowing where to look for configuration changes and ensuring that applications are configurable enough to be deployed without special builds across environments.
  • Continuous Delivery – Our second product owner (the first transitioned out the day we went live) actually asked for us to release less often (i.e. it is a business decision to go-live) so that they could work with the rest of the business to ensure they had their non-IT dependencies in place.
  • Devolved Authority to Feature Leads – I blogged previously about Feature Leads who could help shape the technical solution and drive the knowledge for the project.
  • Metrics Driven Requirements – Though not completely successful, we were able to stop the business from implementing some feature by showing them production metrics. In this case, we were able to avoid building a complex search algorithm to show that we could achieve the same result by adding to a list of synonyms on search.
  • Everyone grows – If I look back at the project, I think everyone on the team has experienced and grown a significant amount in different ways. I think we struck a good balance between being able to work towards individuals goals and find ways they could help the project at the same time.

Other particular things I’m proud of the team:

  • Taming the Hippo - Worthy of its own post, Hippo CMS has been one of the least developer friendly tools I’ve had to deal with for some time. The team managed to work out how to run an effective functional test around its poor UI as well as deploy and upgrade the beast in different environments without the 12 step manual process outlined on their wiki.
  • Rapid team size – Management wanted the entire team to start at the same time. Even being able to push back, we ended up with a very aggressive ramp up and we still managed to deliver well.
  • Diverse but co-operative – We had something like 17 people and 14 different nationalities and it’s one of the strongest teams I’ve seen who were able to work through their differences and forge ahead.

Things that I would like to be different:

  • Find a way to code a lot more – Due to the circumstances, many elements drew me away from coding. At best, I can remember pairing with someone for at most two days a week (for a short time) and I would like to find a way to do that more.
  • Implement more validated learning – Although dependent on a product owner willing to do this, I would have liked to work more on trying to build and experiment a lot more.
  • Have a stronger relationship with decision makers with authority – I believe development teams work best when they are connected to people who can make decisions, not just organisational proxies who provide answers. Unfortunately I felt most of this cascaded very far up the organisation and due to the constant flux and organisational change, this wasn’t possible in the timeframe. I’m hopeful that as the business matures and more permanent people find their place, this will be more possible.

Agile Testing Days Summary

My last speaking conference for the year was the Agile Testing Days held in Potsdam on the outskirts of Berlin. In terms of conference history, this was its fourth year of running and attracts a lot of the big names in agile testing circles, as one would expect from the name. For me, as a test-infected developer , I found it fascinating to see what concerns the testing audience and I felt many common themes around whether or not testing was important, the difference between an agile tester and a normal tester, and of course a focus on collaboration and working iwth other roles.

They had three keynotes a day, a pretty overwhelming number considering that there were multiple tracks and sessions. I don’t think I would do much justice trying to summarise them all but I’ll share a number of highlights that I took away. Jurgen Appelo spoke about the stories behind his books. He’s an entertaining speaker, very well prepared and I feel very in agreement with his “all models are broken but some are useful” approach to collecting different software methodologies and giving a go at lots of different things. Just as the community is continuing to expand beyond just software development, his focus is also pointing upwards into the upper hierarchies of management – to those with the money, budget and decisioning making authority. He’s invented something he’s calling the “management workout” although I shudder at this after seeing it associated (read: abused) by a very hierarchical organisation very much into command and control. Ask me about it one day if you see me in person.

I also appreciated Gojko’s keynote presentation challenging existing thoughts on collecting information, metrics and that all we are doing is improving the process of software development, not necessarily the product. His argument feels very similar to the lean start up movement that why bother putting quality around features we aren’t even sure are useful by anyone, or not validated. He quoted a number of clients he’s seen who throw away automated testing suite in favour of measuring impact on business metrics and trends – using that to work out regressions. It’s an interesting approach that requires a) businesses to actually identify their real business metrics (very rare in my opinion) and b) link features to these metrics and c) measure them very carefully for changes. I guess this also goes along with his new book on Impact Mapping to work out where to put the effort.

He also criticised the testing quadrants, argued that we’re collecting the wrong data, points out that exploratory testing is still testing the process (not the product) and that most organisations are missing feedback once they go live. He also came up with his own adaptation on the Maslo’s hierarchy of needs in terms of software. It starts off with building deployable/functional software, then it should meet performance/security needs, then be usable, followed by useful and then followed by successful. He also recommended a book new to me, called “The Learning Alliance: Systems Thinking in Human Resource Development”

I ran my session, TDDing Javascript Front Ends shortly after a BDD session that complemented the idea very well. The previous session focused on why/what and not the how, where mine was a good depth into how you do it from a hands on practitioner point of view. I had a number of good questions and people after the talk that gave me some great feedback and encouraged me to do more. The only shame was that the session was limited to 45minutes and the tutorial that I run with is normally achievable. I look forward to people taking the online tutorial (found here) and then passing in even more feedback.