To add to the mix, I had the opportunity to present my session on Building the Next Generation of Technical Leaders here. This is the first conference I’ve been to where the majority of the session were not in English. This made me think a lot about how memes spread, and how quickly this affects how adaptable a community is.

I think it is wonderful for this conference to invite speakers from non-French speaking backgrounds, as I hope that it helped seed some more ideas into a community where translating text into a local language hinders the uptake of new ideas. I know that it is much more difficult for people to understand a language other than their native tongue and I can only admire those French people willing to strike up a conversation with me during the conference where my study of the French language is what tourist books teach.

The conference was very well run and although I would like to comment much more on the presentations since most of them were in French. If you understand French, and you find yourself near Paris, then I think it’s an excellent one to attend.

Anyway, after sometime, here’s the plugin configuration I used to do the download. Note that we’re using Maven 2.1.0 (and it could have changed in the latest version).

The following plugin will download configuration.jar and artifact.jar from the web directory http://host/pathToArtifact and drop it into a folder called target/downloadedFiles. This does this at the start of the maven lifecycle (note the phase it is attached to below).

<plugin>
	<groupId>org.codehaus.mojo</groupId>
	<artifactId>wagon-maven-plugin</artifactId>
	<version>1.0-beta-3</version>
	<executions>
		<execution>
			<phase>validate</phase>
			<goals>
				<goal>download</goal>
			</goals>
			<configuration>
				<url>http://host/pathToArtifact</url>
				<includes>
					configuration.jar, artifact.jar
				</includes>
				<toDir>target/downloadedFiles</toDir>
			</configuration>
		</execution>
	</executions>
</plugin>

Some limitations is that it won’t fail if it can download some of the artifacts and not others.

Another tool I stumbled across was winstone (Hudson uses this beast). Wrapping this up seemed pretty easy enough. Note that the configuration below is used to start a war that won’t have dynamic JSP compilation. If you do, you can add a useJasper=true configuration and add the appropriate maven dependencies for jasper to ensure they’re available.

The result of this plugin will effectively build an executable jar, that you simply start up.

Need to add these dependencies:

Given that your project is a war package as well

<plugin>
	<groupId>net.sf.alchim</groupId>
	<artifactId>winstone-maven-plugin</artifactId>
	<executions>
		<execution>
			<goals>
				<goal>embed</goal>
			</goals>
			<phase>package</phase>
		</execution>
	</executions>
	<configuration>
		<filename>executableWebApplication.jar</filename>
	</configuration>
</plugin>

You will find this artifact built in your standard target directly.

I’ve rebuilt the code here:

Source:

package com.thekua.examples;

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;

public class TraceBackProxy implements InvocationHandler {
    public static interface CallingMethodListener {
        void notify(String method);
    }

    private final Object wrapped;
    private final CallingMethodListener listener;

    private TraceBackProxy(Object wrapped, CallingMethodListener context) {
        this.wrapped = wrapped;
        this.listener = context;
    }

    public static Object wrap(Object target, CallingMethodListener context) {
        Class targetClass = target.getClass();
        return Proxy.newProxyInstance(targetClass.getClassLoader(),
                targetClass.getInterfaces(), new TraceBackProxy(target, context));
    }

    @Override
    public Object invoke(Object proxy, Method method, Object[] arguments) throws Throwable {
        String callingMethod = findCallingMethod(method);
        listener.notify(callingMethod);
        return method.invoke(wrapped, arguments);
    }

    private String findCallingMethod(Method method) {
        try {
            throw new RuntimeException();
        } catch(RuntimeException e) {
            StackTraceElement[] elements = e.getStackTrace();
            int callingMethodIndex = findIndexOfMethod(elements, method) + 1; // caller is next one down in stack
            return elements[callingMethodIndex].getMethodName();
        }
    }

    private int findIndexOfMethod(StackTraceElement[] elements, Method method) {
        for (int i = 0; i < elements.length; i++) {
            StackTraceElement current = elements[i];
            // does not cope with overloaded or duplicate method names
            if (current.getMethodName().equals(method.getName())) {
                return i;
            }
        }
        throw new IllegalStateException("Something went wrong and couldn't find method in stacktrace");
    }
}

Test:

package com.thekua.examples;

import org.junit.Test;

import static org.hamcrest.core.Is.is;
import static org.hamcrest.core.IsEqual.equalTo;
import static org.junit.Assert.assertThat;

public class TraceBackProxyTest {

    public static interface SomeRole {
        void doStuff();
    }

    public static class TestSubject implements SomeRole {
        public boolean called;

        @Override
        public void doStuff() {
            called = true;
        }
    }

    @Test
    public void shouldStillDelegate() {
        TestSubject target = new TestSubject();
        SomeRole action = (SomeRole)TraceBackProxy.wrap(target, new TestOnlyListener());

        action.doStuff();

        assertThat(target.called, is(true));
    }

    public static class TestOnlyListener implements TraceBackProxy.CallingMethodListener {
        String lastCalledMethod;

        @Override
        public void notify(String method) {
            lastCalledMethod = method;
        }
    }

    @Test
    public void shouldFindCallingMethod() {
        TestOnlyListener listener = new TestOnlyListener();
        SomeRole action = (SomeRole) TraceBackProxy.wrap(new TestSubject(), listener);

        action.doStuff();

        assertThat(listener.lastCalledMethod, equalTo("shouldFindCallingMethod"));
    }
}

Note that your mileage may vary since it probably won’t work when you have duplicate method names across classes, or overloaded methods on the same. It proved useful for me and hope it helps you.

Make the static instance explicit by introduce a static instance encapsulating any static state and delegate items to it. Use the instance.

Imagine you had this sort of code (below) and there’s dozens of use throughout a codebase. Static imports make it easy for people to simply increment a count, but in order to use the Counter, you have no assurance nothing else will try to use it.

public class Counter {
  static int totalCount;

  public static void count() {
    totalCount++;
  }
}

The following transformations will help you keep existing clients happy:

• Introduce new method (instance level)
• Introduce global instance
• Delegate static method to new method (instance level)
• Turn static state into instance state

The final result is below (note that this code is not thread-safe and not designed for concurrent programs, but is there to demonstrate the result of the refactoring steps):

public class Counter {
   private static final Counter INSTANCE = new Counter();

   private int totalCount;

   public static void count() {
     INSTANCE.recordCount();
   }

   public void recordCount() {
     totalCount++;
   }
}

Now we had a bit more of a complex example, but this refactoring helped us move to the right direction of using instances where possible, instead of sharing global state.

1. Those who like it; and
2. Those who loathe it

I try to be objective as much as possible about tools, so let’s explore what Maven is and what it’s good for.

What’s Maven all about?
Maven has two major responsibilities (with a handful of minor ones). These two are:

1. Library dependency management – Anyone who’s worked on a java project will know the large number of libraries you might end up with. Part of this is the JDKs inability to solve some of the simplest programming problems (solved with Apache Commons, various logging frameworks, etc). Maven was one of the first successful attempts at fixing this, focusing on applications on what version of libraries they’d like with its dependencies automatically resolved.
2. Build lifecycle tool – Maven provides a standard set of fixed build lifecyles with hooks for each step in the lifecycle should you need to extend it.

Why was Maven developed?
To better understand Maven’s design, let’s look at the context developers built it in. Maven sprung forth out of the growing complexities in the Java Apache community. New Apache projects started off almost all the time and there was no “standard” way of building, or creating them. More than that, many Apache projects built on the work of other projects and managing the versioning of libraries started to become an onerous task.

Each Apache project characteristically depended on several other projects although generated its own single distributable (i.e. jar, war, etc).

Maven helped simplify these by assuming a default project structure, a default build lifecycle common to all many Apache projects.

Where is Maven most useful?
If you’re on a project that is similar to a single Apache project (a single distributable with a single set of unit tests) then you’ve hit Maven’s sweet spot.

Where does Maven start to break?
Here are some of the situations I’ve faced where Maven is not the best tool for the job:

• When one team is producing a number of artefacts, you find yourself needing to work around the Maven structure, often with resorting to “Pom Inheritence”. You have to do ugly things like maven installs before hand.
• If you need to do anything more complex than the standard build, then you start to obfuscate the build process. Strange things like different processes running under different profiles (whether or not the build is running on a build server or not) can be very confusing for lots of people.
• Maven adds more complexity to your development environment as you rely on either a public artefact server, or need to set up an internal one. Whilst this seems simple, it is yet another environmental factor that affects how you develop, and its external nature adds variability to projects that often cause false negative build failures.

What other situations would your recommend using Maven or not?

Track: The Concurrency Challenge – Session: Embracing Concurrency at Scale

I really appreciated this speaker for being very pragmatic and conscious about helping people think more explicitly about tradeoffs. He encouraged people to actively think about what solution is most appropriate considering there is always some trade off.

This session resonated another one of those ideas that we use “models” all the time and that models are inherently limited by simplifying things. As the speaker said, “There are many models in science, all of them wrong but some of them useful.” He used time as an example in that there are many ways of looking at it although we have a tendency to want to structure it linearly. “When you admit time is a difficult problem, you will have problems with multiple actors”.

He talked about the ideas that we like convenient mechanisms (such as distributed transactions) because they’re easy to grok and we’re comfortable with those ideas but because they have implicit state, you’re throwing away the value of distributed systems. He went on to talk about how we’re taught about the goodness of ACID properties (particularly around database-centric applications). The alternative he presents is BASE (Basic, Availability, Soft State, Eventual Consistency). There’s apparently a paper on this found here (ACM login required). He referenced Pat Helland’s alternative definition of the acronym, Associative, Commutative, Idempotent and Distributed.

Once again, he emphasises the importance of making the decision on alternatives an explicit one and there is a real tradeoff. This was yet another time someone talked about the CAP theorem that describes three different system properties but you can only guarantee two of them. It’s great as a reminder and highly recommend you read about it as a refresher.

He talks about the world being Eventually Consistent and that “We shouldn’t try to model a world more perfect than it actually is”. He gave a number of examples of using this in practice, and there are ways to make sure that you try to do this. One such example was a Shopping Cart situation where a service should tell the “stock to decrement by 1″ instead of “reduce stock from 99 to 98″. I think this is just another example of good OO (tell, don’t ask).

Track: Solution – Session: Scenario Driven Development

Our own Ben Butler Cole ran this session that attracted a huge audience considering it was running in the Solution Track. Ben attempted to help people understand how to use Scenario based testing instead of alternatives such as exhaustive acceptance testing or story based acceptance testing as the regression suite.

I think the audience were largely sceptical but Ben’s views align very similarly to the things I’ve seen on projects, particularly when doing development using stories. One of the misunderstandings about automated testing and stories is that everyone always views stories as additive. For me, you not only have stories that add functionality, you also have stories that change functionality (replace) and stories that also cause functionality to disappear. I tend to think the set of tests also goes hand in hand – if you’re deleting functionality, you should be deleting existing tests as well, or if you’re changing tests, you should be changing existing tests.

I think part of the problem is that stories are kind of fractal in nature and the scenarios that Ben talked about were like at an epic level (maybe even a use case level). What is interesting is how you go about enhancing, maintaining that scenario over time and how the team works around those scenarios – something we unfortunately only got a taster for in the hour.

I heard a lot of people still come away with lots of interesting insights and I would highly recommend everyone go and listen to Ben speak even if it’s just to hear him present. Totally worth it.

Track: The Concurrency Challenge – Session: Modelling Concurrency with Actors in Java – Lessons learned from Erjang

This session was fairly packed out, I guess given the hype of people working with or wanting to work with Erlang. This presenter’s take was interesting because in order to better understand concurrency and Erlang, this guy decided to implement an interpreter on top of the JVM (instead of simply playing around with the language).

The premise behind his talk was that Object Orientation as a paradigm brought with it plenty of productivity and that Actor-based modelling was going to be the thing to give us the next boost. Whilst I admire his premise, I don’t necessarily agree. I don’t think we’re all that good (as an industry) as implementing Object Orientation. I do think things like the JVM have helped raise the abstraction level so that we don’t need to think as much about the lower level memory management (although you have a memory leak in these languages if you’re not careful).

He spent a long time fascinating all the language geeks with his ability to run a simply Erlang program on top of his project, Erjang (Erlang on the JVM) but I really wanted to hear about the lessons learned from programming instead of the technical details of writing an interpreter on the JVM, something he spent a majority of the time on. I don’t think there were very many interesting bits other than designing parts of a system with different ideas. As he said, “Imagine treating parts of system as different actors with different responsibilities?” to which I was thinking, “That’s what good Object Orientation is about. Treating objects playing different roles with the right set of (small) responsibilities”. Nothing new unfortunately.

Track: How do you want to test that? – Session: Performance Testing at the Edge
This session brought a lot of mixed reactions. I agree that the start of it seemed a little bit commercial, talking about their product and the complexities of performance testing. I like discussing the importance of performance testing for applications because it’s one of those things that are often neglected and really not treated very well. This presenter eventually got around to talking about some of these aspects, but like many of the talks still ended up being fairly late in the talk.

I think that this speaker had a lot of great things to say and although much of it was very common sense, didn’t give people an impression about how they could go about doing it well. They had some interesting approaches to monitoring performance such as running it on the same set of dedicated hardware (that wasn’t even a replica of production). This has a trade off of being able to compare results over time which they had some interesting graphs, although it is at a cost of being a proper representative sample of what the software might actually run on.

I found that some of the good things he talked about were similar to the talk that Alistair and I had, although it still sounded like the performance testing team was a predominantly split responsibility that worked orthogonally to the actual development team. It also sounds like there was a big cultural barrier that they have been trying to break down by getting developers to write performance tests and have some more ownership.

The crowd also had a lot of scepticism about convincing a business to invest in the automation of performance testing.

Ralph was asking about the number of people who work on systems they created (versus systems that other created) and then argued about the meaning of maintenance programmer arguing for software evolutionists instead. Whilst I agree that no one likes to be labelled a maintenance programmer, changing the name doesn’t really matter.

I like the different perspectives that Ralph brought. He talked about Software Capital as about truly understanding how the software works. Code is one example of this, but actually there are many other things that go into understanding how the software works. He argues that the right sort of documentation (and other artefacts) that helps understand how the software works is another part of Software Capital. As a result, you also need the people around the software to help keep the story alive.

My observations of various organisations also backs up some of the things that he says such as, the larger a system gets, the more likely you need some form of documentation and social structure around the software.

I think a lot of people didn’t like Ralph’s keynote because he talked about refactoring as if it was a new idea and, unfortunately this audience is one that has been exposed to these ideas for a long time.

Track: Beyond Tools and Technologies
I spent the rest of my time on the track that I was speaking at later that day. The first talk of the day was Martin Fowler whose talk, “What are we programming for?” brought the tiny room to a stand still. The room was so full (although it was a small room) that they had to turn people away.

Martin used the talk to highlight the importance of going beyond acting like hired guns and actually thinking about how to use the skills for something good. He asked people to consider the essence of the firms that they work for and whether or not they are benefiting humanity.

Rebecca Parsons, ThoughtWorks’ CTO then talked about the importance of team diversity in her talk, “How to avoid ‘We never even thought of that’!” She talked about the role that diversity played in creating innovations in science and how too much of the “sameness” creates problems with creating too many assumptions. I like the idea of a balancing tradeoffs of not enough diversity leading to problems in teams and too much diversity to the point that the team don’t share the same vocabulary.

After the lunch break, I ran my session, “Building the next generation of Technical Leaders“. I was very happy with the turnout and had some great feedback from the participants. It’s an issues I think is sorely neglected in our industry and has a huge impact on the team effectiveness.

The next session was run by our Director of Social Engagement (Jeff Wishnie) and Merrick Schaefer of Unicef discussing the importance that technology has on developing countries and the significant impact it can have on the lives. They talked about the Rapid FTR project that is being developed to help capture information about children who’ve been separated from their parents with hope of quickly reconnecting them, and at worst, giving them legal status by registering them with an agency such as Unicef.

The final session of the day rounded out with Mick Blowfield with a presentation titled, “IT in a Low Carbon Future“.

Whilst there were many other potentially interesting sessions, I was really happy to hear about the issues that go beyond just Tools and Technology.

Talking to most of the attendees, it also attracts a noticeably different side with the majority of people already in architect or senior developer roles with a relatively smaller proportion of people in other ancillary roles such as PMs, testers or BAs.

Feedback that works
I like the feedback note for the conference, asking people tweet, email or talk to people at the conference. I think some of the feedback mechanisms even worked as we saw a lack of caffeine/water during the breaks on the first day change for the other remaining days.

The Keynote
The first day’s keynote started with the evangelical Uncle Bob Martin. He’s a great speaker and channels a lot of opinion towards the crowd wandering back and forth on stage. I saw a version of this at the Agile 2009 conference. I’m glad he adjusted it somewhat more appropriately for this conference. He talked about some good principles that should be followed after a few bumpy technical glitches. He is all about the drama (which I guess is part and parcel of a keynote) such as showing us a video that scrolled through a single java class for something like two to three minutes backed up by a song from the movie from 2001 (thanks Ola!) that almost sounded like the flight of the bumblebee. The point of this showmanship was to demonstrate how we can sense bad code visually without even looking at the specifics. He went through a number of his mantras such as “The Boy Scout” rule, leaving things better than how you found them, or that functions should be no more than four or five lines long.

Whilst I don’t think I learned anything new, Uncle Bob Martin was entertaining and had good things to say, despite how preacher-like he comes across.

Track: Architecture’s You’ve Always Wondered About – Session: BWin – P5 a future proof poker platform
The rest of the day was split into six different tracks. I went to the first of these and was truly disappointed. The two presenters came from BWin, apparently one of the largest poker and gaming sites in Europe. They spent half an hour talking about their problems of performance (many of the -iliites) and where the history of their application came from. They even had to include a three minute video from their marketing folks. They finally flashed up a screenshot of their architecture diagram which I think everyone had come to hear about it. Unfortunately they moved onto the next screen very quickly and said that they couldn’t actually talk about their architecture. How disappointing! Indeed this might have been an “Architecture You’ve Always Wondered About”, and after this session was one to still, indeed, wonder about it.

They proceeded to pick a single example about two processes that concurrently wrote and read from the same database table and their architectural change simply transitioning it to a disk persistent queue and the reporting service reading from that queue and writing to its own database. Neither very profound, or a very interesting tidbit that failed to give the audience any more insight into any interesting part of how they architected their system.

I found it interesting they spent a long time (something like 250 man years) rewriting this version of their architecture so I asked the question about how they went about moving from their old to their new platform at which they pretty much described a big bang approach – neither iterative or incremental. Pretty disappointing.

My lesson learned from this: Regardless of how pretty your slides are, make sure your abstract matches what you’re going to actually talk about.

Track: Dev and Ops: A Single Team – Session: Devs and Ops Cooperation when the Worst Happens
My next session was much more fulfilling with Michale Nygard (author of Release It) describing some of the stories that were the basis for the book but wouldn’t have been publishable. Michael is really down to earth and happy to chat about many of the details behind the book. It’s really obvious that he is also a very pragmatic person, understanding that models are just that and recommending people look into the details because there is often something much more complex underneath.

Many of the things that Michael talked about also resonated very strongly with ideas that I’ve seen on applications. Many of them included focusing on ensuring documentation artefacts should be tested by those who will consume them. As he put it, “UML is virtually useless to the deployment and operations team”. Other gems included, “From a high enough view, everything starts to look the same”

His stories also reinforced the importance of having a ubiquitous language will all members of the company in the shared domain. One of his most successful rescue missions dealing with a production problem only worked because the architect (on the dev side) and Michael (on the ops side) understood enough to really communicate the real problems. Michael also kept (understandably) hammering home the point of lack of information on both sides really hurts the problem solving process.

A lot of Michael’s stories also described their successes working around the existing (organisational) systems in order to deal with the problems at hand. To me, this was just another great example of organisations that structure themselves for efficiency over effectiveness.

I like the way that Michael finished off his talk as well, focusing on the fact that you’re not just trying to build software, that you need to care about the organisation (and I would also argue, the systems and procedures in that organisation) that support the software. Design them for effectiveness, not efficiency.

Architectures you’ve always wondered about: Facebook
The next session I attended was by Aditya Agarwal (one of the engineering directors of Facebook). He shared some great insights and lessons learned into what things Facebook did that got them to where they were. Admittedly they had some strange choices like writing something that converted PHP into compiled C++ and using MySQL is a very “reliable bitstore”.

They have a lot of great things to say, like being very proud of the low number of engineers ratio to actual users. Like Alistair, I’m puzzled why they’re secretive of the exact number of their employees.

Aditya has some good things to say and although he means well, I don’t support everything that he says. it’s obvious a lot of what he says is based on a start up culture. Things like, “don’t worry about it until it becomes a problem” encourages the recklessness we see in many software companies. Same thing about the choice of programming language such as “every programming language will have problems at our scale”.

On the plus side, I was pleased to hear them encouraging the innovative engineering culture that I don’t see in many organisations.

Track: Non relational DBs and Web Oriented Data – Session: Social networks and the richness of data: getting it done with
After going through my notes the most interesting tidbit was about using a partitioning mechanism to deal with data updates based on the number of connections. There was plenty of buzzwords (both internal and external) that I think I had a hard time fitting it all together. Some things I’m going to look at include:

• Bit versus bloom filters – Not familiar with this term or how to use it.
• Tools – RESTeasy (I now see it’s a JBoss project). Redis and Voldemort – something they used heavily all over the place.

That’s about all I got out of this session though I’m not sure if it was about the lack of familiarity with the tools, or the excessive use of acronyms and unfamiliar terms or just plain tiredness.

Track: Functional Programming – Session: Demystifying Monads
Josh Graham did a great job stepping into fill in for his ill wife talking through some of the different ideas behind monads. I haven’t done functional programming since university and I’ll be the first to admit Monad’s are still a mystery to me but I came away with some key learnings.

Firstly, to understand monads properly, you really just have to see several examples in action. There’s a high barrier to entry because it means:

1. Learning a new language
2. Learning a new development environment
3. Playing with plenty of examples

When I find some time, I’ll do those. In the meantime, I understand a bit more about the properties of a monad, and that they can be a powerful abstraction for functional languages.

Final keynote – Dan Ingalls: Forty Years of Fun with Computers
For being so late in the day, this was a very entertaining keynote. Just like any modern day small talk person, this entire demonstration took place in the Squeak development environment. Dan demonstrated some interesting and interactive elements.

I had a few key takeaways including:

• We’re really bad at learning the lessons from the past. Partially because we don’t have enough storytellers and that our generation doesn’t listen very well.
• I’m going to read this article at some point.