thekua.com@work

This year’s XP2010 conference was held in the great Northern parts of Norway, up in Trondheim, only about 200km south from my first XP conference (XP2006) I attended several years ago. I always find this series of conferences interesting, partly because there is always the blend between the academic (research papers and experience reports) and the industrial side as well as the infusion of the hosting country’s culture.

This year, the conference saw 425 attendees (50% of them Norwegian) across 8 different tracks and covering four days of a wide variety of conference mixtures including Lightning Talks, Panels, Workshops, Tutorials, and Open Space events.

The first of the two keynotes, run by Scott Page was my favourite – a lecturer and researched on complexity in different environments. He talked about the importance of diversity and its impact on building better solutions as well as a provable formula calculating what the wisdom of the crowds was. Hopefully they’ll be putting up his slides somewhere as a result.

The second of the keynotes, ran by David Anderson did a great job of condensing down a long talk into a thought-provoking discussion on the work that he’s been doing on building a Kanban movement in the places he’s been working as well as a discussion of his recently released book about Kanban. He had some nice visualisations on the slides and even finished slightly earlier with plenty of times for questions.

Overall the conference seemed to have a nice blend of both process and technical topics, although there were a few complaints about specific discussions around “XP” itself although I think plenty of discussions about specific practices quite useful within any agile context.

I ended up much busier than I thought I would be, playing host for two of the session talks, helping out with a Pecha Kucha (more on that later) session and running a workshop on my own.

Venue, organisation and efficiency wise, the organisers need to be congratulated on a job that will be hard to surpass. Everything seemed to run smoothly including an amazing and difficult-to-forget Conference Banquet involving interesting local foods (smoked reindeer heart, moose and local seafood), a pair of comedic jazz academics, Keep of Kalessin playing some Extreme Metal, all inside Trondheim’s amazing Student Society venue.

The strangest part of the conference for me was participating in a “What’s in my agile suitcase?” Pecha Kucha run by Martin Heider and Bernd Schiffer. You can read more about the format here and it was great to be one of the five other prepared Pecha Kucha’s including Rachel Davies, Mary Poppendieck, Joshua Kierevsky and Jeff Patton. I found the diversity of styles and approaches fascinating as well as the specific things that people “packed” in their suitcases. Being the first time format all the speakers found it a fascinating format made thrilling by the short-time and the fact you don’t have control over the timing of the slides. If I were to do this differently, I’d definitely try to focus on a smaller range of topics (or just one).

My week ended (as did the conference) with my final workshop called “Building the Testing Pipeline.” I’d specifically targeted people who’d been working with automated tests for some time and I ended up with a surprisingly full room. I’d run this previously at ACCU with a slightly different audience. We had some great brainstorming sessions (I’ll be putting that up soon) and hopefully more people came away thinking more consciously about the balance of tests they have and whether or not they have the correct balance that maximises their confidence, and minimises their feedback cycle.

I’m also very proud that the experience report that I shepherded won best paper (experience report), and I finally got to meet the author, Jørn Ola Birkeland of Bekk Consulting, in person to congratulate him on the day.

Thanks to all the organisers, participants, and passionate people that made the conference so much fun and success. Wonderful to reconnect with old friends and make many new ones.

Comments(4)

I’ve been fortunate enough to spend my last week (most of it holidays) in Australia at ThoughtWorks’ Melbourne office. The last time that I visited here was when I first joined over six years ago.

One of the things I love about visiting our other offices is to see how consistently strong the company culture is. We have so many unique individuals yet there is some common bond that easily makes us identify with each other. I feel it’s part of the passion that we all share, and our ability to have fun all the time. In fact, I’ve never laughed so hard this last week for some time now.

I’m proud to be part of this family and will definitely miss all my Australia colleagues.

Comments(1)

The pace of the conference and the speakers drinks event on Thursday night meant that I missed the first session on the final day. It’s a shame since I think Joe Armstrong would have been good to listen to. I look forward to reading anyone’s report of that session in the blogosphere.

Track: The Concurrency Challenge – Session: Embracing Concurrency at Scale

I really appreciated this speaker for being very pragmatic and conscious about helping people think more explicitly about tradeoffs. He encouraged people to actively think about what solution is most appropriate considering there is always some trade off.

This session resonated another one of those ideas that we use “models” all the time and that models are inherently limited by simplifying things. As the speaker said, “There are many models in science, all of them wrong but some of them useful.” He used time as an example in that there are many ways of looking at it although we have a tendency to want to structure it linearly. “When you admit time is a difficult problem, you will have problems with multiple actors”.

He talked about the ideas that we like convenient mechanisms (such as distributed transactions) because they’re easy to grok and we’re comfortable with those ideas but because they have implicit state, you’re throwing away the value of distributed systems. He went on to talk about how we’re taught about the goodness of ACID properties (particularly around database-centric applications). The alternative he presents is BASE (Basic, Availability, Soft State, Eventual Consistency). There’s apparently a paper on this found here (ACM login required). He referenced Pat Helland’s alternative definition of the acronym, Associative, Commutative, Idempotent and Distributed.

Once again, he emphasises the importance of making the decision on alternatives an explicit one and there is a real tradeoff. This was yet another time someone talked about the CAP theorem that describes three different system properties but you can only guarantee two of them. It’s great as a reminder and highly recommend you read about it as a refresher.

He talks about the world being Eventually Consistent and that “We shouldn’t try to model a world more perfect than it actually is”. He gave a number of examples of using this in practice, and there are ways to make sure that you try to do this. One such example was a Shopping Cart situation where a service should tell the “stock to decrement by 1″ instead of “reduce stock from 99 to 98″. I think this is just another example of good OO (tell, don’t ask).

Track: Solution – Session: Scenario Driven Development

Our own Ben Butler Cole ran this session that attracted a huge audience considering it was running in the Solution Track. Ben attempted to help people understand how to use Scenario based testing instead of alternatives such as exhaustive acceptance testing or story based acceptance testing as the regression suite.

I think the audience were largely sceptical but Ben’s views align very similarly to the things I’ve seen on projects, particularly when doing development using stories. One of the misunderstandings about automated testing and stories is that everyone always views stories as additive. For me, you not only have stories that add functionality, you also have stories that change functionality (replace) and stories that also cause functionality to disappear. I tend to think the set of tests also goes hand in hand – if you’re deleting functionality, you should be deleting existing tests as well, or if you’re changing tests, you should be changing existing tests.

I think part of the problem is that stories are kind of fractal in nature and the scenarios that Ben talked about were like at an epic level (maybe even a use case level). What is interesting is how you go about enhancing, maintaining that scenario over time and how the team works around those scenarios – something we unfortunately only got a taster for in the hour.

I heard a lot of people still come away with lots of interesting insights and I would highly recommend everyone go and listen to Ben speak even if it’s just to hear him present. Totally worth it.

Track: The Concurrency Challenge – Session: Modelling Concurrency with Actors in Java – Lessons learned from Erjang

This session was fairly packed out, I guess given the hype of people working with or wanting to work with Erlang. This presenter’s take was interesting because in order to better understand concurrency and Erlang, this guy decided to implement an interpreter on top of the JVM (instead of simply playing around with the language).

The premise behind his talk was that Object Orientation as a paradigm brought with it plenty of productivity and that Actor-based modelling was going to be the thing to give us the next boost. Whilst I admire his premise, I don’t necessarily agree. I don’t think we’re all that good (as an industry) as implementing Object Orientation. I do think things like the JVM have helped raise the abstraction level so that we don’t need to think as much about the lower level memory management (although you have a memory leak in these languages if you’re not careful).

He spent a long time fascinating all the language geeks with his ability to run a simply Erlang program on top of his project, Erjang (Erlang on the JVM) but I really wanted to hear about the lessons learned from programming instead of the technical details of writing an interpreter on the JVM, something he spent a majority of the time on. I don’t think there were very many interesting bits other than designing parts of a system with different ideas. As he said, “Imagine treating parts of system as different actors with different responsibilities?” to which I was thinking, “That’s what good Object Orientation is about. Treating objects playing different roles with the right set of (small) responsibilities”. Nothing new unfortunately.

Track: How do you want to test that? – Session: Performance Testing at the Edge
This session brought a lot of mixed reactions. I agree that the start of it seemed a little bit commercial, talking about their product and the complexities of performance testing. I like discussing the importance of performance testing for applications because it’s one of those things that are often neglected and really not treated very well. This presenter eventually got around to talking about some of these aspects, but like many of the talks still ended up being fairly late in the talk.

I think that this speaker had a lot of great things to say and although much of it was very common sense, didn’t give people an impression about how they could go about doing it well. They had some interesting approaches to monitoring performance such as running it on the same set of dedicated hardware (that wasn’t even a replica of production). This has a trade off of being able to compare results over time which they had some interesting graphs, although it is at a cost of being a proper representative sample of what the software might actually run on.

I found that some of the good things he talked about were similar to the talk that Alistair and I had, although it still sounded like the performance testing team was a predominantly split responsibility that worked orthogonally to the actual development team. It also sounds like there was a big cultural barrier that they have been trying to break down by getting developers to write performance tests and have some more ownership.

The crowd also had a lot of scepticism about convincing a business to invest in the automation of performance testing.

Comments(0)

Keynote: Ralph Johnson on Living and working with aging software
The final keynote of the conference kicked off the second day of QCon and presented to us by Ralph Johnson of the Gang of Four fame and whose students contributed to Fowler’s timeless Refactoring book.

Ralph was asking about the number of people who work on systems they created (versus systems that other created) and then argued about the meaning of maintenance programmer arguing for software evolutionists instead. Whilst I agree that no one likes to be labelled a maintenance programmer, changing the name doesn’t really matter.

I like the different perspectives that Ralph brought. He talked about Software Capital as about truly understanding how the software works. Code is one example of this, but actually there are many other things that go into understanding how the software works. He argues that the right sort of documentation (and other artefacts) that helps understand how the software works is another part of Software Capital. As a result, you also need the people around the software to help keep the story alive.

My observations of various organisations also backs up some of the things that he says such as, the larger a system gets, the more likely you need some form of documentation and social structure around the software.

I think a lot of people didn’t like Ralph’s keynote because he talked about refactoring as if it was a new idea and, unfortunately this audience is one that has been exposed to these ideas for a long time.

Track: Beyond Tools and Technologies
I spent the rest of my time on the track that I was speaking at later that day. The first talk of the day was Martin Fowler whose talk, “What are we programming for?” brought the tiny room to a stand still. The room was so full (although it was a small room) that they had to turn people away.

Martin used the talk to highlight the importance of going beyond acting like hired guns and actually thinking about how to use the skills for something good. He asked people to consider the essence of the firms that they work for and whether or not they are benefiting humanity.

Rebecca Parsons, ThoughtWorks’ CTO then talked about the importance of team diversity in her talk, “How to avoid ‘We never even thought of that’!” She talked about the role that diversity played in creating innovations in science and how too much of the “sameness” creates problems with creating too many assumptions. I like the idea of a balancing tradeoffs of not enough diversity leading to problems in teams and too much diversity to the point that the team don’t share the same vocabulary.

After the lunch break, I ran my session, “Building the next generation of Technical Leaders“. I was very happy with the turnout and had some great feedback from the participants. It’s an issues I think is sorely neglected in our industry and has a huge impact on the team effectiveness.

The next session was run by our Director of Social Engagement (Jeff Wishnie) and Merrick Schaefer of Unicef discussing the importance that technology has on developing countries and the significant impact it can have on the lives. They talked about the Rapid FTR project that is being developed to help capture information about children who’ve been separated from their parents with hope of quickly reconnecting them, and at worst, giving them legal status by registering them with an agency such as Unicef.

The final session of the day rounded out with Mick Blowfield with a presentation titled, “IT in a Low Carbon Future“.

Whilst there were many other potentially interesting sessions, I was really happy to hear about the issues that go beyond just Tools and Technology.

Comments(0)

When you introduce agile methods to very traditional organisations there’s a side effect most people underestimate. This is, I believe, where the first line in the manifesto is most appropriate:

Individuals and interactions over processes and tools

Traditional organisations are geared for efficiency – keeping individuals busy to the point where you lose overall effectiveness if you ever need to synchronise between parts (which you inevitably do for software these days). Calling a meeting is frowned upon as it takes up lots of time, so out of necessity, most communication occurs over email or IM. As Alistair Cockburn pointed out a long time ago, these are pretty ineffective channels of communication. Relying solely on these types of channels only turns to a build up of frustration as issues go unheard by the right set of ears. In traditional environments, these often turn into build ups set to explode at the most random of moments.

Image from Nic Jacka’s flickr stream under the Creative Commons licence

Perhaps you’ve seen it? Think about when one person hijacks a meeting to highlight an issue (typically important one) not being addressed, or when someone spontaneously shouts at someone for no apparent reason. It might even be as bad as someone having a temper tantrum, literally throwing things across the room. Traditional environments often do not provide mechanisms as outlets for this.

Therefore, when introducing practices such as daily stand ups, planning meetings and retrospectives, the first few times you run them, expect an avalanche of unresolved issues and emotions to ensue. This is fine. Expect this from some of the quieter members as you give them explicit permission to talk about important things that need resolution where traditional structures have not given them that permission to talk about them.

Treat it as you would the opening of a previously shaken soft drink bottle for the first time. You’ll have to be ready to clean up some of the spilt liquid, but you don’t have to worry about picking up broken glass. Now that the bottle’s open, you can work to address the real issue at hand.

Comments(2)

I’ve been quiet on the blogging front because I’ve been busy doing other sorts of writing. One of my focal points is around leadership, both in teams in general, and around agile situations. I think it’s important that if you’re leading, you need to be seen to demonstrate the values and the suggestions that you speak of.

Wonderful, you have great intentions behind what you do. It’s even more powerful if you demonstrate the behaviours that you speak of. I think part of leading is knowing when to act. If you’re simply preaching at people, I don’t think you will be very effective as a leader.

Comments(2)

Businesses need to be comfortable that not all projects are going to succeed. Out of a portfolio of projects, people need to be comfortable that these projects will not achieve what they originally set out to do. Don’t expect that, even with agile methods and values guiding your teams, these projects will achieve what they set out to do.

Some projects aren’t set up for success. Poor organisational governance, leading to unrealistic expectations established from the outset often set up a project for a real death march. Or perhaps projects spun out of the whims of one set of people only to not understand what organisational capabilities they really have. Sometimes it comes down running a project with the wrong mix of skills and without a support structure in place that creates a time bomb waiting to explode.

Don’t forget that agile and lean methodologies cannot guarantee success. At the heart of it, agile will surface problems and make them visible. Organisations need to be prepared to confront the truth (many are not ready for this level of transparency) and support changes that will make it better.

For projects destined to fail, your best result is often to Fail Fast, take those lessons and then do something differently. Avoid the situation where a project sucks up resources that could be more effectively allocated. And don’t forget, just because a project didn’t achieve what it set out to do, it’s only a true failure if you don’t learn anything from it.

Comments(5)

Last week reminded me how hard communication is to get right. Last week reminded me of how important it is to be visible with information as early as possible. Last week reminded me of what happens with the people involved don’t have access to information early.

Successful teams applying agile principles quickly involve those impacted by the situation, equipping them with as much information as early as possible. These teams call upon agile practices such as daily stand up meetings, retrospectives, and frequent showcases to achieve this. Better and earlier access to information helps all parties involved come up with more options. More options creates more opportunities to have better conversations, and more opportunities to collaborate to meet everyone’s needs, and ultimately end up with a solution that everyone is more likely to feel committed to.

Compare this to those teams who hoard information, selecting and filtering the information others hear. Filtering and transforming information limits the number of options, often adding additional stress because the team now how to come up with the perfect option. Even with contributions from others, the pool of options will often be tainted by solutions not entirely appropriate or relevant. More importantly, if the people affected by a decision aren’t involved, they will end up less committed to the solution and often, cause more problems because of resentment.

No one likes to be handed decisions. That’s why the Agile Manifesto emphasises “Individuals and Interactions”, and a key principle of Lean Thinking is “Respect for People”.

The moral? Remember to involve the appropriate people in the decision making process as early as possible. Even if you suspect there is only going to be one solution, be transparent with the information you do have in the hope you may end up with more options, or at least, the outcome is no surprise.

Comments(2)

Thoughts about the second day
Ivar Jacobson gave an equally uninspiring keynote, talking about “What they don’t teach you about software at school: Be Smart!” In it, he focuses on the fact that it’s not only enough to try to “Be Agile”, but you need to “Be Smart”. Unfortunately many of the items that he talked about didn’t really seem to be any different from “Being Agile” and that he had obviously targeted the wrong audience.

I’m glad the conference ran with an open space session, this year introduced by Willem Van Den Ende and Lasse Koskela. I suggested a topic titled, “The Kanban, Lean, Agile Divide” and I felt we had the best quality discussions during this open space. I wanted to answer a number of questions including:

• Did people know about the Kanban Community?
• How active were people in that community?
• Did people perceive a divide?
• And if so, why?

Out of around the 12 people participating, probably only two or three hadn’t heard about the community, and only one or two people were active in participating (in the mailing list portions). We had lots of discussion as to what people considered as Kanban as well.

First page of notes from the open space session

We noted down the mailing list, available on Yahoo Groups: Kanban-Dev and several “published” resources including the paper, Kanban Vs Scrum, and the book, Scrumban.

When asked about what Kanban was the participants, here’s what we recorded:

• Elements of one piece flow (particularly limits to WIP or Work In Progress). Translated into practical terms, it meant focusing on one story at a time (as a team or as a pair)
• Kanban was an alternative scheduling technique
• Unprescriptive, or less prescriptive (relying on other practices to help out)
• Constantly evolving
• “Good for maintenance”
• Most people considered Kanban a “Practice more than a methodology”, though there was general agreement that some people are trying to turn it into more than that
• Focused on delivering “Minimal Marketable Features”
• Quote from Tom Poppendieck, “Scheduling and Workflow are orthogonal”. We expanded upon this for a bit and a more concrete example is you can use different scheduling techniques such as Kanban or Timeboxes with the same workflow such as a simple, “To Do, In Progress, Done” workflow.

I recorded that some people were still doing sprint/iteration planning whilst using Kanban though I don’t remember if we got into the details.

I also recorded two concerns, such as knowing, “When do we get stuff done?” and a very “Loose Terminology” leading to lots of confusion, particularly the overloaded associate with Kanban (as a practice) and Kanban (as a methodology).

When asking the question, “Is there a kanban divide?” some answers included, there is maybe a divide in a community sense, and when probing further, when interpreting Kanban as a practice, not a methodology there wasn’t however there was definitely a sense when treating Kanban as a methodology and not as the practice with a feeling that some people in the community wanted to make money.

Last page of notes from the open space session

We asked why the community divide and understandably, there was no real community around to openly discuss Kanban (as a practice) and felt similar to those that had been around during the XP early days. There didn’t seem to be any general list for “agile practices” with extremeprogramming and scrum-dev mailing lists. It’s also not the first time practices have had their own list, like the pairprogramming or the test-driven-development lists. We did note down that there were several potential ones the community could have joined including lean-agile, lean-dev instead of creating their own kanban-dev.

In the afternoon, I went along to the panel on Lean Software Development although I seemed to come away with the same questions and less answers. One of my questions about budgeting was answered by the next morning’s keynote, while another focusing around the lack of discussion around practices seemed to be dismissed by Mary Poppendieck as, “I don’t believe in practices”, maybe best surmised as, “I don’t know” (which I would have also been happy to take).

The evening was an impromptu beach BBQ held by the awesome folks at Agical. Whilst a small company, I’m glad to see them still as passionate and having grown year upon year with more enthusiasm about a huge variety of topics. The BBQ was amazing and I know that everyone had a great time.

More to come…

Comments(0)

Steven List asks the question, Are Retrospectives an Anti-pattern? Of course, retrospectives are a topic close to my heart so I naturally wanted to share my view of them. The conversation apparently started on the Kanban Development mailing list and Steven’s post already captures some great discussion. I won’t repeat it here, but I find the dialogue echoing the same sentiments about other agile practices and whether or not they’re useful. For me, it’s too extremist and not particularly helpful. They make it sound like you need to choose from two positions: Either you run retrospectives, or you don’t.

I think the more interesting question is, “When are retrospectives most useful?” To help explain my thoughts, I’ve put together the following: A Model for understanding Retrospective Impact (click on it for a slightly bigger view).

What is a retrospective?
Being specific and clear about this term is important in the discussion about whether or not retrospectives are useful. Some people’s understanding of a retrospective can differ from other people’s depending on each of their experiences. The conversations on the Kanban list seem to imply retrospectives are solely an iteration focused ritual (and of course, there is no such concept of a Scrum or XP iteration in Kanban). Though I appreciate Kerth’s original definition, I think in the agile community, Derby and Larsen’s definition better captures its essence:

A special meeting where the team gathers after completing an increment of work to inspect and adapt their methods and teamwork.

I like this definition best because its ties the concept of improvement (inspect and adapt) based on a shared understanding (the team) according to some unstated time period (an increment of work, noting it’s not specifically tied to an iteration).

What is impact?
I think it’s also important to define the term impact. One of the definitions by the Oxford Dictionary seems most appropriate:

a marked effect or influence

I want to emphasis that you can have a high impact retrospective without necessarily taking a lot of time, and that the opposite is entirely possible too (a long retrospective with a low impact). Note that I’m not going to discuss how to run retrospectives that have the highest impact (a totally separate topic worth its own set of posts).

Effect of team dynamics
The state of the team has an enormous influence on the impact of retrospectives. A high performing team will naturally have better communication amongst its members, and are more likely to fix things that get in the way of their goal. There will naturally be better and more frequent individual improvements, identified and implemented more frequently without the call for a specific meeting. This doesn’t mean that retrospectives are never a practice they call upon. Rather, it’s a practice that has less impact as there are many other items in the toolkit that get called upon. Sometimes it’s important for the group to get together, establish a common understanding and to inspect and adapt noting there may not be any specific regularity to these particular meetings.

In contrast, a dysfunctional team are less likely to call upon individual improvement practices, and thus, retrospectives play a more important role, providing an explicit opportunity to improve. I use the term dysfunctional in this sense to capture a broad category including newly formed teams, or simply groups of people. This does not necessarily mean the team cannot advance, nor that it hasn’t had a chance to advance, just simply the state it exists in.

Environmental Effects
I would classify a majority of environments as chaotic. In chaotic environments, improvement isn’t second nature. Through practices that start with good intentions such as auditing (“I want to have confidence you are doing what you say you are doing”), people in these environments become afraid to suggest changes, and are often fearful of attempting anything different because they’ve been punished for a failed experiment in the past. In these chaotic environments, retrospectives have a significant impact by providing an explicitly safe environment for teams to make, commit to, and hopefully be supported in making changes. Retrospectives have a higher impact in these environments because the majority of other activities doesn’t encourage learning, innovation and improvement.

In contrast, a nurturing environment welcomes fail fast innovation, rewards learning from failed experiments and supports continuous improvement in an explicit, non document centric approach. In these environments, the outcomes from implementing improvements and passing on tacit learnings is more important that meeting audits and compliance. Retrospectives have less impact in this environment because individuals are more likely to suggest and implement improvements without the need for a special meeting, and without the need for an entire team. Once again, it doesn’t necessarily imply that retrospectives are not useful (some issues require a shared understanding across the entire team) or that they are regularly scheduled events.

Avoid deciding between whether or not you run retrospectives. Consider when is it best for you to run a retrospectives instead.
Understand what situation you find yourself in using the model above and ask yourself whether or not retrospectives have the potential for the most impact (dysfunctional teams and chaotic environments), or where energy is better spent pursuing other activities because retrospectives have less, but not zero, impact (high performing teams and nurturing environments).

In the next set of posts, I hope to describe different situations I’ve seen first hand and what impact retrospectives had in relation to them.

Leave a comment and let me know what you think.

Comments(3)