thekua.com@work

One of my biggest annoyances when it comes to testing is when anyone reaches for a mock out of habit. The “purists” that I prefer to call “zealots”, are overwhelming in numbers, particularly in the TDD community (you do realise you can test drive your code without mocks?) Often I see teams use excuses like, “I only want to test a single responsibility at a time.” It sounds reasonable, yet it’s generally the start of something more insidious, one where suddenly mocks become the hammer and people want to apply it to everything. I sense it through signals like when people say “I need to mock a class”, or “I need to mock a call to a superclass”. Alternatively it’s quite obvious when the number of “stub” or “ignored” calls outnumber the “mocked” calls by an order of magnitude.

Please don’t confuse my stance with “classical state based testing purists” who don’t believe in mocking. Though Martin Fowler describes two, apparently opposing, styles to testing, Classical and Mockist Testing, I don’t see them as mutually exclusive options, rather two ends of a sliding scale. I typical gauge risk as a factor for determining which approach to take. I believe in using the right tool for the right job (easy to say, harder to do right), and believe in using mocking to give me the best balance of feedback when things break, enough confidence to refactor with safety, and as a tool for driving out a better design.

Image of broken glass taken from Bern@t’s flickr stream under the creative commons licence

Even though I’ve never worked with Steve or Nat, the maintainers of JMock, I believe my views align quite strongly with theirs. When I used to be on the JMock mailing list, it fascinated me to see how many of their responses focused on better programming techniques rather than caving into demands for new features. JMock is highly opinionated software and I agree with them that you don’t want to make some things too easy, particularly those tasks that lend themselves to poor design.

Tests that are difficult to write, maintain, or understand are often a huge symptom for code that is equally poorly designed. That’s why that even though Mockito is a welcome addition to the testing toolkit, I’m equally frightened by its ability to silence the screams of test code executing poorly designed production code. The dogma to test a single aspect of every class in pure isolation often leads to excessively brittle, noisy and hard to maintain test suites. Worse yet, because the interactions between objects have been effectively declared frozen by handfuls of micro-tests, any redesign incurs the additional effort of rewriting all the “mock” tests. Thank goodness sometimes we have acceptance tests. Since the first time something is written is often not the best possible design, writing excessively fine grained tests puts a much larger burden on developers who need to refine it in the future.

So what is a better approach?
Interaction testing is a great technique, with mocks a welcome addition to a developer’s toolkit. Unfortunately it’s difficult to master all aspects including the syntax of a mocking framework, listening to the tests, and responding to appropriately with refactoring or redesign. I reserve the use of mocks for three distinct purposes:

1. Exploring potential interactions - I like to use mocks to help me understand what my consumers are trying to do, and what their ideal interactions are. I try to experiment with a couple of different approaches, different names, different signatures to understand what this object should be. Mocks don’t prevent me from doing this, though it’s my current preferred tool for this task.
2. Isolating well known boundaries - If I need to depend on external systems, it’s best to isolate them from the rest of the application under a well defined contract. For some dependencies, this may take some time to develop, establish, stabilise and verify (for which I prefer to use classical state based testing). Once I am confident that this interface is unlikely to change, then I’m happy to move to interaction testing for these external systems.
3. Testing the boundaries of a group of collaborators - It’s often a group of closely collaborating objects to provide any useful behaviour to a system. I err towards using classical testing to test the objects in that closely collaborating set, and defer to mocking at the boundary of these collaborators.

Comments(1)

A long time ago, I wrote about the differences between the types of tests I see, yet a lot of people don’t appreciate where acceptance tests fit in. I won’t be the first to admit that at first glance, automated acceptance tests seem to have lots of problems. Teams constantly complain about the build time, the maintenance burden they bring, and the difficulty of writing them in the first place. Developers specifically complain about duplicating their effort at different levels, needing to write assertions more than once (at a unit or integration level), and that they don’t get too much value from from them.

I wish everyone would realise that tests are an investment, a form of insurance against risk (the system not working), and most people don’t even know what risk level they are willing to take. That’s why, on experimental code (i.e. a spike), I don’t believe in doing test driven development. I’ve been fortunate enough, or is that learn some lessons from, seeing both extremes.

Maintaining a system only with acceptance tests (end to end)
I worked on one project where the architect banned unit tests, and we were only allowed integration and acceptance tests. He believed (rightly so) that acceptance tests let us change the design of the code without breaking functionality. His (implicit) choice of insurance was a long term one - ensure the state of the system constantly works over time, even with major architectural redesign. During my time on this project, we even redesigned some major functionality to make the system more understandable and maintainable. I doubt that without the acceptance tests, we would have had the confidence to move so quickly. The downside to this style of testing is that the code-test feedback was extremely slow. It was difficult to get any confidence that a small change was going to work. It felt so frustrating to move at such a slow pace without any faster levels of feedback.

Scenario driven acceptance tests (opposed to the less maintainable, story-based acceptance tests) also provide better communication for end users of the system. I’ve often used them as a tool for aiding communication with end users or customer stakeholders to get a better understanding about what it is they think the system should be doing. It’s rare that you achieve the same with unit or integration tests because they tell you more how a particular aspect is implemented, and rarely lacks the system context acceptance tests have.

Maintaining a system only with unit tests
On another project, I saw a heavy use of mocks, and unit tests. All the developers moved really fast, enjoyed refactoring their code, yet on this particular project, I saw more and more issues where basic problems meant that starting up the application failed because all those tiny, well refactored objects just didn’t play well together. Some integration tests caught some of these, but I felt like this project could have benefited from at least a small set of acceptance tests to prevent the tester continuously deploying a broken application despite a set of passing unit tests.

What is your risk profile when it comes to testing?
I think every team developer must understand that different types of test give us different levels of feedback (see the testing aspect to the V-Model), and each has a different level of cost determined by constraints of technology and tools. You’re completely wrong if you declare all automated acceptance tests bad, or all unit tests are awful. Instead you want to choose the right balance of tests (the insurance) that match system’s constraints for its entire lifetime. For some projects, it may make more sense to invest more time in acceptance tests because the risk of repeated mistakes is significantly costly. For others, the cost of manual testing mixed with the right set of unit and integration tests may make more sense. Appreciate the different levels of feedback tests bring, and understand the different levels of confidence over the lifetime of the system, not just the time you spend on the project.

Comments(0)

Projects where the team directly translates story-level acceptance criteria into new automated test cases set themselves up for a maintenance nightmare. It seems like an old approach (I’m thinking WinRunner-like record-play back scripts), although at least the teams probably feel the pain faster. Unfortunately not many teams seem to know what to do. It sounds exactly like the scenarios that my colleagues, Phillip and Sarah are experiencing or experienced recently.

Diagnosing this style of testing is easy. If I see the story number or reference in the title of the test class or test case name, chances are, your team experiences automated story-based acceptance tests.

Unfortunately the downfall to this approach has more to do with the nature of stories than it does with the nature of acceptance tests (something I’ll get to later). As I like to say, stories represent the system in a certain snapshot of time. The same thing that lets us deliver incremental value in small chunks just doesn’t scale if you don’t consolidate the new behaviour of the system, with its already existing behaviour. For developers, the best analogy is like having two test classes for a particular class, one that reflected the behaviours and responsibilities of the system at the start, and one that represents the new behaviours and responsibilities of the system right now. You wouldn’t do this at a class level, so why should you do it at the system level?

Avoid temporal coupling in the design of your tests. The same poor design principle of relating chunks of code together simply because someone asked for them at the same time, also apply to how you manage your test cases. In terms of automated story-based acceptance tests, avoid spreading similar tests around the system just because they were needed at different times.

What is a better way? A suggested antidote…

On my current project, I have been lucky enough to apply these concepts early to our acceptance test suites. Our standard is to group tests, not based on time, but on similar sets of functionality. When picking up new stories, we see if any existing tests need to change, before adding new ones. The groupings in our system are based on the system level features, allowing us to reflect the current state of the system as succinctly as possible.

Comments(7)

I’m a big believer that if software projects aren’t set up correctly at the beginning, it potentially puts a lot of drag on the team going forward. This doesn’t mean planning everything meticulously before implementing it. Rather, its mean experimenting and evaluating whether the tools you choose give you the things you need for your project. When I set up new development projects, here’s some guidelines I try to follow:

• Check out and go - Contributors should not need to install 10 applications, four plugins and configure special settings in order to start developing. I’ve successfully tried techniques like checking in your build tool (like ant) alongside your code, with windows and unix executables (go.bat, go.sh) that enable people to get consistent feedback quickly. On one project, we had all of the tools checked into a different subversion module and we ran a configuration script that set up the environment variables we needed to start developing. Eclipse allows you to bundle up all your plugins you need. If you don’t want to check things into source control, put it onto a standard share accessible to everyone. If you do need to install specific applications, list the details so they can run through it like a checklist (although prefer this as a last option).
• IDE (un)bound - If you need to open your IDE to assemble your final executable, you’re far away from the ideal situation. It’s important for me that you can build your application in both your IDE and from the command line. Doing this early in the project helps raise issues about what can and can’t be done with your toolset.
• IDE friendly - Having said that, it’s important that developers can debug easily, and so ensure that your code works with the environment and tools of your choice
• Determine the testing strategy - Work with your QA to understand what things you will test, and to automate or try to automate. Determine what tools they might need to properly test your application.
• Great example test cases - Trying to automate unit and acceptance tests is important to try before you have your final application (as the design may change to accommodate better testing). Your scripts should look neat and concise. If you have 100 lines of setup, 100 lines of test, imagine if you then need to write 1000 more of these and the nightmare your code will be to maintain.
• Have a consistent code style - Strive for collective ownership. Having code that looks completely different from each other will detract from this. Define code format style early (where curly braces go, etc). Spend more time discussing the more interesting standards that may actually have an impact on your project (when, where and what to log, how to handle exceptions).

Comments(0)

I will confess at the outset of this review that I did not buy the Agile Adoption Patterns book, rather it was sent to me by the author (Amr Elssamadisy) whom I’d met a few years ago at XP2006. I intended for my review to be as independent as possible but wanted any readers to recognise the potential for bias given the context. The history of the book is an interesting one, combining the author’s own experiences with that of many agile practitioners derived from workshops that he ran at many agile/xp conferences over the course of several years. The benefits for the reader is that it captures the lessons learned of a collective crowd from a number of different contexts, something difficult for a single person to achieve on their own. One side effect of this is that the practices are a little self selecting with many of the practices biased towards XP and Scrum practices (and less of those originating from FDD, DSDM or lean).

I ended up reading this book twice before writing this, although the first time I simply flipped through it lightly to get a feel for the book. I remember distinctly thinking, ‘Yet another book on agile practices? Do I really want to spend more time reading through it?’ The second time, with much more time on a train, I’m glad I did as the author spends a lot of effort setting the context for how to go about using the book.

Given the content, I do feel the title of the book slightly misleading as it talks less about patterns in which people successfully (or unsuccessfully) adopted agile as a whole. Though a good book at what it does, I think it would have been better called, “A framework for adopting agile practices” or “Patterns of agile practice adoption” given its emphasis on the practices and less so on the principles behind them. For me, I find it’s important people recognise the difference between practices (good for beginners during the Shu learning phase) and principles and values (better for the Ha and Ri learning phases) and I’m a little disappointed that little of this seemed emphasised. Of course, this book is not targeted at seasoned agile practitioners, so it fits the audience. I think it bothers me that people thinking that adopting practices alone automatically makes them agile especially when you consider sustainable agile in organisations.

Structurally, the author lays the book out well, setting the context and history of the book and the intended audience. Although he mentions it’s not useful for advanced practitioners, I do think there is value as a means of diagnosing smells in practices for teams that already implement the individual practices. To get the most of out, I would recommend they simply jump to the practices they’re using to see if they recognise any signs of the smells described.

I like the format and I do believe this book contains a lot of useful information although suffers from a small set of problems. One of them is that it suffers the same problem many other pattern books do - it’s a great reference over time, yet a little too repetitive to read in one sitting. Another is that I think this is a great reference for people starting, yet leaves a gap about what to do when you get some of the practices implemented. Often I find some practices are great intermediate practices (iterations and time boxing as a way of developing rhythm) yet doesn’t detail what people should do to ensure agile is sustainable in the long term (dropping practices, and developing their own). My only other issue, as minor as it feels, is that the aesthetics of the book feel really dated, with many of the diagrams and the cover feeling like a textbook I would have used for university.

On the plus side, the author presents a simple, clear and step framework for people to follow when looking at a wealth of agile practices. It’s not prescriptive, and gives a balanced perspective by describing the benefits and the possible less desirable side effects of an individual practice and highlighting the impact of wrongly applying a practice in the wrong context. This is a great addition to the large body of literature on agile that will prove useful for early adopters.

Comments(1)

I remember reading about the Cause, Made Visible and Not Related story on Esther Derby’s blog a while back. My biggest takeaway was that retrospectives aren’t normally the Cause of issues, instead creating Visibility into the issues already present. People constantly surprise me when they say that don’t like retrospectives because it doesn’t fix their issues. Guess what? Simply holding a retrospective won’t magically fix all your issues because it isn’t the Cause of them. Yet how do you go about fixing your issues if you don’t take the time to identify what issues you have, what impact they’re having and how you’re going to fix the true Cause?

Image taken from Dr Pat’s Flickr photostream under the Creative Commons licence.

Comments(0)

I’ve previously written an article for InfoQ about how agile coaches act when working with teams. For this particular entry, I want to investigate what jobs or roles you might consider trying to either focus on part-time or try full-time to develop the habits and skills necessary to be an even more effective agile coach.

At the XP2008 conference I spoke with both Liz Sedley and Rachel Davies, two other agile coaches, who ran a workshop about agile coaching. We talked about the different skills an agile coach might need and some of the duties they might perform. We talked about overlaps with other jobs and concluded that an agile coach may do some training, yet only being a trainer doesn’t make you an agile coach (there’s more to it). Below is a diagram that hopefully makes it clear some of the responsibilities that overlap with a number of other roles.

As you can see in the diagram above, an agile coach may do many of the things you see full time facilitators, full time trainers, and full time coaches do. Yet doing each of these roles by themselves without the real experience garnered from agile projects does not make them an agile coach.

Using this as a model for career development

Even though I just put this diagram together to help others visualise this model, this is exactly what I’ve been using to further improve my skills as an agile coach. More recently, I’ve been in the role of a full time coach and trainer both internally and externally, and especially fortunate to work closely with other full time trainers to benefit from their experience. That experience has given me a better understanding of how people learn and a broader set of techniques to draw upon when helping others understand the concepts and principles.

Earlier to that, my focus had been towards better facilitation practice, reading books about good facilitation skills, and eagerly applying this during the projects that I’ve worked on. This has been particularly useful in executing the Retrospective practice. Beyond that, I’ve been lucky enough to work on many different agile projects in a development role, benefiting from others’ experiences through observation.

I’d be keen to hear what other full time or part time roles other agile coaches have attempted for short amounts of time in order to develop the skills that will make them a better agile coach. Please leave a comment if you have a suggestion.

Comments(1)

George Ambler’s blog entry titled, “Agility Means Simple Things Done Well, Not Complex Things Done Fast“.

Comments(0)

Not much to update other than I’ve returned from Canada and have been busy running a couple of presentations around the place, most recently heading up to Manchester for a night to help Alex Scordellis out with the Manchester Geek Night he’s running regularly. I ran a session called The Agile Primer, essentially an introduction to agile methods. We had a great turn out and I particularly enjoyed the discussion that ensued both during the session and at the pub afterwards. It still seems that, at least in that one pub up north, they shut relatively early. My only regret about the trip was that I didn’t get to stay longer and see Manchester because the weather was so nice.

Yesterday I helped out an analyst group with their Master Class on Agile. Liz Keogh and I pair-presented on ThoughtWorks’ experiences and lessons learned with distributed development, titling the session Distributed Agile: An Oxymoron?. We then followed this up with the XP Lego Game that helped people keep awake after plenty of talking and a heavy lunch.

All the slides and resources, you can continue to find on my page of presentations and papers.

Comments(0)

It’s easy to execute a particular practice. The difficult part is executing the practice well. This applies with any trade, any profession and anything we do. That’s why we have professional sports people, professional tradespeople, and professions in general. It takes a long time to get good at anything.

One thing I’ve learned from observing great leaders and less than great leaders is that those that make a big difference follow the words they say. Less than great leaders talk in one way, yet act completely incongruently with what they said. Some leaders don’t realise this, believing that what they say is enough to motivate people. The result is that people get confused, defaulting to the behaviour they see, not the words that are spoken.

The lesson here: It’s important understand what people hear because that can be different, and to ensure that what you do follows what you say if you really want the message to get across.

Comments(0)