Failing to upgrade to iPhone 2.2 firmware

In trying to do the right thing and keep the iPhone up to date, I got the unfortunate “something went wrong” dialog. I’m sure all the fan boys will ask why I bother if I have a windows machine, but that’s not the point of this post.

In trying to fix this, I have now discovered the big difference between Apple’s “reset” (think reboot) and “restore” (reset to factory defaults or wipe and reboot) with “restore” only available through the iTunes interface. For some reason, when I used to own an iPod three years ago, I thought there was a hard reset that would effectively “restore”. Unfortunately no more.

A few searches revealed nothing that worked for. I made sure I had the latest drivers installed, uninstalled and reinstalled iTunes (restarting windows). When all other posts seemed to suggest I would find an Apple Mobile Device under Device Manager, all I could find under the USB devices was an Apple Recovery (iBoot) USB device. I followed the advice from a number of sources including Apple support, all suggesting I should unplug, restart iTunes, replug in the iPhone and it should all worked.

Eventually, a combination between restarting windows, starting iTunes, plugging in the iPhone, restarting iTunes (note that last step) seemed to finally pick up the fact that I had an iBrick. So I sit here now while I wait for it to sync to my last backup, fingers crossed for the next time I update. I’m hoping someone else benefits from this post in case they also have trouble upgrading to the 2.2 firmware update.

Validating JVM Arguments in Code

I tried looking around on the Internet for a way from code, to get the arguments passed to the JVM itself, rather than environment properties, system properties, and program arguments. As of Java 5, apparently it got a whole lot easy in code than before with the java.lang.management.ManagementFactory.

Testing to see if you have a particular JVM argument is easy with a function like:

public boolean hasJvmArgument(String argument) {
  List arguments = ManagementFactory.getRuntimeMXBean().getInputArguments();
  return arguments.contains(argument);
}

Why would you want to do that?
For some systems, it’s critical the system operates under a specially configured environment. In our current system, for example, we have a health check that runs a series of tests to validate those assumptions still hold true. To achieve our performance targets, it’s integral our application starts with the correct JVM settings, and even with automation in place, we potentially have mistakes. Fortunately with the code above, we can simply add a JvmHasCorrectSettingsCheck to guarantee the application never gets deployed under the wrong circumstances.

Thanks to Saager Mhatre for pointing me in the right direction!

Agile North

On a Thursday in November the University of Central Lancashire held a gathering of the most passionate people around the agile space in the north. The number of attendees really surprised me at around one hundred, and seemed to indicate the northern IT industry is in a pretty healthy state. Based on the people that I talked to, it seems that only a small minority (i.e. 10% seemed to be thinking about adopting agile. A majority, probably around 75% seemed to be doing agile from a practices sense, although I got a sense most of them were still getting the hang of it.

A common theme for this conference seemed to be the focus on the more humane aspects rather than the technical challenges faced by teams. Roger Marlow talked about the need to cross pollinate the lessons learned from the social sciences while Chris Matts talked about the need to heal the relationships of delivery teams and the business, or more specifically the BA crowd.

The first keynote was, to be honest quite disappointing with the emphasis on why agile is good. In case the presenter hadn’t realised, a lot of people were already using agile practices, and the issue is actually how most people move beyond just the practices to the principles.

We hoped to address some of this by holding a session on acceptance testing intended to connect people with new ideas. I will be first to admit it fell a little short, with an overrun session effectively cutting into a third of our planned time and the expected audience more than double of what we had planned for. Still, I think there were some good topics and hopefully the write up proves useful for some people. I’ll get it posted to Agile North as soon as I can.

Other observations included many more people calling themselves agile coaches. It also struck me about how a lot of the talks were much less technical than other conferences I’d been to. I also learned that even with a great message some presenters had, delivery still matters and death by powerpoint still seems to be a popular method of execution. For those thinking about doing just presentations, there’s a really good website called Presentation Zen I think everyone should read. Either way, the bombardment of thirty (subtlety) different graphs reminded me of the excruciating pain I hear about agile projects that fail to practice continuous improvement… iterating in smaller batches, but missing the whole point of feedback and adapting to change.

How do I tell if a team is agile?

Vivek asks what are the signs you use to tell if a team is agile? I prefer not to count the practices they might be using because that is almost too easy (to get wrong).

Example: I remember talking to a person about their team who claimed they did lot of automated testing. They apparently had tests but upon further questioning I found many of their tests didn’t work and hadn’t worked for quite some time. I continued to ask questions about their practices and they mostly seemed excessively ritualistic without the team benefiting from them.

Ticking off a checklist about practices is easy to do, yet hard to do well. I prefer to ask questions with a less explicit goal in mind. I care more about how people do what they do rather than what they are doing right now.

I prefer to look for consistent behaviours across what they are doing that tell me:

  • they care about what they’re doing
  • they understand why they are doing what they are doing, and actively question when it’s not clear to them
  • even better is that they can explain to people why they do what they do as a team and what value each part of their process has (I dare you to try that with your current team)
  • they help each other out even if it’s beyond their normal roles because they don’t believe that people fit in boxes perfectly. And most importantly;
  • they’re constantly trying to improve the environment they are in.

What other ones do peope look for? I’m sure there are plenty more.

Application logging principles

A friend asked me if I had any good links about logging in applications and I realised that I haven’t blogged about it. I think logging is an important part to any good application, so I thought I’d emphasis some principles I tend to follow when applying logging.

Logging cannot be considered in isolation of the exception handling strategies

Often I hear about teams who think that logging is a completely isolated concern for a system and exception handling is another. Though they seem like it, systems where exception handling strategies that don’t consider logging strategies and vice versa end up in a very confused state.

As a rule of thumb, my default position on exception handling looks like this:

  • If you can handle an exception, recover from it and continue
  • If you can’t handle an exception, but you can translate it, then do so and continue.
  • If you can’t handle an exception and you won’t translate it, then propagate it to someone who can.

Weaving in logging, it becomes a little bit more complicated.

  • If you’re going to handle an exception, you may choose to log the details. Ideally you might choose to log them at a finer level if it’s not a critical exception.
  • If you translate an exception and still propogate that, log the exception only if you aren’t going to propogate it (previous rule applies here to what level).
  • If you can’t handle an exception and you won’t translate it, don’t bother logging it. Ensure that someone higher up is going to apply the first two rules. There’s nothing more horrid than seeing multiple stack traces, reporting the same problem and there’s too much noise for the signal to be of any use.

Defend the user against stack traces

Applying the previous set of guidelines, then I always try to have one place in my application (at the outer most level possible) to prevent stack traces from appearing out of nowhere to the user. You might still choose to display it to a user for them to submit it back to a support team, or to simply log it to a file to be sent off. What should be avoided at all costs is crashing the application in spectacular fashion with only a stack trace as evidence.

Handling this at the outer most layer, hopefully from a single location, gives a few more benefits. You might choose to log additional information about the application, or render it in a different manner that proves useful for fixing problems.

Think of what would information would help you at 3am in the morning

The reason we log things is to have additional information about what is going on, when things aren’t going as expected. I like to think about what it would be like, to be called at 3am, only to diagnose an issue.

This is particularly difficult to do on iterative projects because you need to balance out enough logging with the cost of changing that logging. I try to add logging to parts of code that are fairly stable and whose implementation patterns will unlikely change.

If you TDD your code, and you reach for the debugger, you need to add some logging

TDD reduces the need to debug, however it never removes the need entirely. Debugging is a useful tool during development that lets us discover information our logs fail to tell us. Unfortunately it’s rare that we can apply this technique when our code is in production, therefore notice what information helps you when you debug and add its equivalent to a log somewhere.

If information logged is part of a fundamental business process, add some tests

I don’t necessarily test all log statements I add to code, however the cost of adding tests around some log statements if often worth it, especially if it’s critical information supporting a business process.

The pattern I use to test loggers is:

  1. Get a reference to the logger under test
  2. Save the logging level for that logger (to restore after the test runs)
  3. Turn up the logging level to the level you expect
  4. Execute the code that should trigger the logging
  5. Verify the correct message and information was logged
  6. (After the test – i.e. teardown) restore the logging level to the original value

Organise sets of information into groups and associate those groups with different log levels

Logging everything to a single level can often be too overwhelming. The trick to getting the right level of logging and information is to have a clear, shared understanding with the team about what information should be logged at what level.

Performance tuning garbage collection

Our current project has a heavy emphasis on throughput and latency, and our team spent a good amount of time building up the infrastructure to be able to performance test our application. We’re currently in the middle of performance tuning, and got to a point where garbage collection is a significant enough problem for us to start looking at options. I’ve never really had to do this low level tuning before and it’s amazing to see how many options you can tweak. Here’s a few links I’ve found that have been helpful. Please leave a comment if you think there’s more that would be of use and I’ll update the list.

Please note that we’re using Sun’s JVM in production, and hence performance tuning against that JVM.

Scriptable vs Visual?

Most programs tend to follow two schools of philosophy in their design, one that I’m going to call ’scriptable’, the other, ‘visual’. Those in the ’scriptable’ camp tend to come from a unix background where being able to use pipes to direct input and output into different programs provides a simple, yet very powerful paradigm. Those found in the ‘visual’ camp tend to come from a windows background where almost everything has a GUI, and people mostly use a mouse to execute the functions.

Of coures there are different advantages and disadvantages to both approaches.

For example, discovering functionality in a ’scriptable’ program is often not obvious simply by executing it. Without the program providing well written, detailed documentation and addtional examples, an end user can spend considerable time discovering how to do something trivial, rather than simply do it. Remembering special codes or parmaters, or the order it expects them in can be considered especially tediuous. Of course, the benefits of the having this interface allows the program to be automated as part of a larger job. The power of this approach is particularly useful for system administrators who may need to do something as part of a common job, or for developers who don’t want to reinvent the wheel.

The ‘visual’ approach, on the other hand, requires less documentation as long as the interface is easy to use and intuitive to the user (although this is often not the case). For people from a less technical background, using a ‘visual’ manner for discovering behaviour is a powerful paradigm and sometimes the user’s only option given their individual set of knowledge. Unfortunately, using this approach in isolation also means excluding the possibility of automating any functionality provided (or at least making it significantly more difficult).

I see people unconsciously make the choice between these two modes all the time. If I was to generalise, Java and Rubyists tend to gravitate towards the ’scriptable’ interfaces (providing libraries, jars) while a lot of .Net people tend to gravitate towards the ‘visual’ (providing exes).

My preference for writing applications obviously leans towards a mixed approach where I would always want to prioritise the ’scriptable’ interface above the ‘visual’ one. My thinking here is that the ’scriptable’ interface should give enough separation between the functionality and the way an end user interacts with it that the ‘visual’ interface should be easy to drive out. In my experience, doing it the other way around leaves people coupling a lot more of their functionality into the graphical user interface, often making the ’scriptable’ interface more difficult to extract.

Don’t get me wrong, as an end user, I almost always want a user interface for things that I’m going to do very infrequently. The developer in me also wants assurance that, in case that infrequent becomes frequent, I have the easy option of automating it later.

Death by a thousand differences

One of the most common smells I see on projects is the desire to do things slightly differently for no good reason. Triggered by many different reasons such as new people joining a project, or when new functionality is being added, or when bugs need to be fixed. Each individual difference isn’t a problem on its own, rather it’s their accumulative effect that has a significant negative impact on projects that force developers to continually context switch when trying to match their mental model against the code or to navigate their way around a new area in the system. I find the death by a thousand differences is especially a significant problem on iterative projects where change happens rapidly such as the project Frankie describes.

Code isn’t the only thing that suffers from this condition as well with many unwarranted differences built into configuration and build and deployment scripts (ant, nant and msbuild).

What to do about it?
In many ways Rails had a very strong opinion that I agree with, following convention over configuration, with a core principle that every difference should exist for a good reason. Projects need code standards to help maintain consistency, and needs to be updated to reflect the current state of the system. New team members need to understand what the standard is, why it exists and the whole team needs to be vigilant against the additional context-switch each new difference adds.

Sometimes differences must exist, but always apply the 5 whys to question if the difference is really needed. Sometimes they exist as a means to an end (a transition to a better implementation pattern) yet projects must be wary that not too many differences exist at the same time without a good reason.