thekua.com@work

Projects where the team directly translates story-level acceptance criteria into new automated test cases set themselves up for a maintenance nightmare. It seems like an old approach (I’m thinking WinRunner-like record-play back scripts), although at least the teams probably feel the pain faster. Unfortunately not many teams seem to know what to do. It sounds exactly like the scenarios that my colleagues, Phillip and Sarah are experiencing or experienced recently.

Diagnosing this style of testing is easy. If I see the story number or reference in the title of the test class or test case name, chances are, your team experiences automated story-based acceptance tests.

Unfortunately the downfall to this approach has more to do with the nature of stories than it does with the nature of acceptance tests (something I’ll get to later). As I like to say, stories represent the system in a certain snapshot of time. The same thing that lets us deliver incremental value in small chunks just doesn’t scale if you don’t consolidate the new behaviour of the system, with its already existing behaviour. For developers, the best analogy is like having two test classes for a particular class, one that reflected the behaviours and responsibilities of the system at the start, and one that represents the new behaviours and responsibilities of the system right now. You wouldn’t do this at a class level, so why should you do it at the system level?

Avoid temporal coupling in the design of your tests. The same poor design principle of relating chunks of code together simply because someone asked for them at the same time, also apply to how you manage your test cases. In terms of automated story-based acceptance tests, avoid spreading similar tests around the system just because they were needed at different times.

What is a better way? A suggested antidote…

On my current project, I have been lucky enough to apply these concepts early to our acceptance test suites. Our standard is to group tests, not based on time, but on similar sets of functionality. When picking up new stories, we see if any existing tests need to change, before adding new ones. The groupings in our system are based on the system level features, allowing us to reflect the current state of the system as succinctly as possible.

Comments(9)

Writing low quality software is easy. Achieving high quality software is much harder. Leadership and safety are essential factors in achieving and maintaining high quality.

Comments(0)

After lots of research, I’m in favour of the Virtual Disk Driver program called VDK available from http://chitchat.at.infoseek.co.jp/vmware/vdk.html and distributed under the GPL licence. Here’s some of the features that attracted me to it:

• VDK doesn’t require running an MSI for installation – In keeping with the principles of starting a new project (checkout and go), I can simply include this file as part of a code repostiory, and know that I’m not forcing every other developer yet another manual step. Although you do need to install VDK, it’s a command line registration that is as simple as: vdk.exe install.
• VDK is fully executable via the command line – This allows me to wrap this in the language of my choice to help automate environment setup as needed for a particular test to run.
• VDK supports VMDK (VMWare Disk) images – Using other tools to generate the files that I want, I can easily use VDK to mount them to a particular drive letter.
• VDK supports mounting with different options – Including read-only, read-write, and write-block mode.
• VDK has good documentation – I found it really easy to understand what commands to execute to install, remove, mount, unmount, all with different options because both the command line (vdk.exe help) and the readme.txt had plenty of information and examples. It also helps that it follows conventions with other command line programs (following the DOS conventions of parameters with slashes)
• VDK is realiable – I did plenty of different tests mounting and unmounting and it just keeps going (though your Windows Explorer may need a refresh (F5) to keep up).

Using a VMWare Disk Image created by QEMU as discussed previously, I can now create a new virtual disk mounted in windows simply by using the following commands:

vdk install
vdk open 0 floppy_disk.vmdk /P:0 /RW /L:Y
format /fs:fat y:

See the image below:

Comments(1)

If you can’t tell from the latest series of posts, I’ve been working very closely with the file system, especially exceptional circumstances when certain conditions go wrong. I’ve found it really interesting to see how to automate some of the test scenarios since I’ve never done anything like this before. One of the things I’ve been looking at closely are RamDisk utilities for windows, especially for things I can manipulate programmatically.

Most of the utilities on the net seem really robust, with the most favoured being the freeware RamDisk that comes from MyDigitalLife. There are also plenty of other commercial alternatives. The freeware RamDisk has a simple user interface that makes it really easy to manually mount a drive though their instructions for command line execution must less user-friendly, with few examples available, and what help document there is very unintuitive. Even obvious attempts with class command line options (slashes, dashes, double dashes, colon-separate name-value pairs) failed and I ran out of patience.

Unfortunately most other RamDisk tools for windows are all GUI-based and, in general, cost some money to use.

Comments(2)

From the book, The Wisdom of the Flying Pig by Jack Hayhow:

“There was a time when the term manager implied someone who controlled. Today the manager must facilitate”. – Page 14

Comments(0)

Our current development team is split, some working on windows machines, others on macs, with continuous integration tests running against both windows and linux environments. We’ve had a need to run operating specific JUnit4 tests and I got a little tired putting guard clauses into different @Before, @After and @Test methods to prevent a particular block of code running.

Using the new org.junit.runner.Runner annotation, and SystemUtils from commons-lang, here is the result:

package com.thekua.java.junit4.examples;

import org.apache.commons.lang.SystemUtils;
import org.junit.internal.runners.InitializationError;
import org.junit.internal.runners.JUnit4ClassRunner;
import org.junit.runner.notification.RunNotifier;

public class RunOnlyOnWindows extends JUnit4ClassRunner {

    public RunOnlyOnWindows(Class klass) throws InitializationError {
        super(klass);
    }

    @Override
    public void run(RunNotifier notifier) {
        if (SystemUtils.IS_OS_WINDOWS) {
            super.run(notifier);
        }
    }
}

Our test classes, then look like this:

...
@RunWith(value=RunOnlyOnWindows.class)
public class WindowsOnlySpecificFooBarUnitTest {
    ...
}

Of course, you can use any other particular variations like SystemUtils.IS_OS_UNIX, or SystemUtils.IS_OS_MAC but we haven’t needed to yet.

Of course, this is easily turned into some sort of conditional runner abstract class but at least you get the basic idea.

Comments(1)

File.setReadonly(true) doesn’t actually work on windows (at least on the JDK that we’re working on). According to the bug report filed here, it is just setting the DOS flag that prevents it from being deleted just not being written to.

Meanwhile, assuming you have mounted a read only disk partition in windows to, say X:, new File(“X:/”).canWrite() returns true when it really should be returning false. I’m still trying to find the bug reported to sun for this, or will later update this entry to include it. It seems to persist when running against both jdk1.5.0_15 and jdk1.6.0_07.

Comments(4)

The current versions of both VMWare Fusion and VMWare Player limit the minimum capacity of virtual disks (*.vmdk) to 100MB (or 0.1GB). Even their command line utility for windows (vmware-vdiskmanager.exe) describes this limit despite the file format allowing other file sizes to be specified (see screenshot below)

If you want to create a smaller disk, then I suggest downloading the QEMU program that allows you to create VMWare compatible disk images. When you install QEMU (at least for windows), you should find a qemu-img.exe file in their folder that gives you the ability to create your disk image. Here’s the command we used to create a 1MB VMware compatible image.

qemu-img create -f vmdk smallfile.vmdk 1M

Comments(0)

This is probably something that old timers in Java will probably know, so I’m posting this more for my reference. Trying to execute:

Runtime.getRuntime().exec("dir");

results in the following stack trace.

java.io.IOException: CreateProcess: dir error=2
  at java.lang.ProcessImpl.create(Native Method)
  at java.lang.ProcessImpl.(ProcessImpl.java:81)
  at java.lang.ProcessImpl.start(ProcessImpl.java:30)
  at java.lang.ProcessBuilder.start(ProcessBuilder.java:451)
  at java.lang.Runtime.exec(Runtime.java:591)
  at java.lang.Runtime.exec(Runtime.java:429)
  at java.lang.Runtime.exec(Runtime.java:326)

After doing some reading

error=2

apparently means file not found. The fix for something like this is to first pass everything to the windows command line shell (cmd.exe on windows xp). This seems to do the job better:

Runtime.getRuntime().exec("cmd /c dir");

The slash-C means “Carries out the command specified by string and then terminates”.

Comments(3)

I’m a big believer that if software projects aren’t set up correctly at the beginning, it potentially puts a lot of drag on the team going forward. This doesn’t mean planning everything meticulously before implementing it. Rather, its mean experimenting and evaluating whether the tools you choose give you the things you need for your project. When I set up new development projects, here’s some guidelines I try to follow:

• Check out and go – Contributors should not need to install 10 applications, four plugins and configure special settings in order to start developing. I’ve successfully tried techniques like checking in your build tool (like ant) alongside your code, with windows and unix executables (go.bat, go.sh) that enable people to get consistent feedback quickly. On one project, we had all of the tools checked into a different subversion module and we ran a configuration script that set up the environment variables we needed to start developing. Eclipse allows you to bundle up all your plugins you need. If you don’t want to check things into source control, put it onto a standard share accessible to everyone. If you do need to install specific applications, list the details so they can run through it like a checklist (although prefer this as a last option).
• IDE (un)bound – If you need to open your IDE to assemble your final executable, you’re far away from the ideal situation. It’s important for me that you can build your application in both your IDE and from the command line. Doing this early in the project helps raise issues about what can and can’t be done with your toolset.
• IDE friendly – Having said that, it’s important that developers can debug easily, and so ensure that your code works with the environment and tools of your choice
• Determine the testing strategy – Work with your QA to understand what things you will test, and to automate or try to automate. Determine what tools they might need to properly test your application.
• Great example test cases – Trying to automate unit and acceptance tests is important to try before you have your final application (as the design may change to accommodate better testing). Your scripts should look neat and concise. If you have 100 lines of setup, 100 lines of test, imagine if you then need to write 1000 more of these and the nightmare your code will be to maintain.
• Have a consistent code style – Strive for collective ownership. Having code that looks completely different from each other will detract from this. Define code format style early (where curly braces go, etc). Spend more time discussing the more interesting standards that may actually have an impact on your project (when, where and what to log, how to handle exceptions).

Comments(0)