Even though JaQu comes bundled with H2, it is not specific to that database. I have tested the following code example successfully with an Apache Derby database.

JaQu provides a fluent interface (or internal DSL) for building SQL queries in Java. Before you start writing queries, you need to create a class for each table, with a public field for each column. This is how such a class looks for my Robots table:

public class Robot implements Table {

    public Integer id;
    public String name;
    public Date dateOfBirth;

    @Override
    public void define() {
        tableName("Robots");
        primaryKey(id);
    }
}

Using this table class, my example query can be expressed as follows:

public void printClassicRobots() {
    QuotedTimestamp dobThreshold =
        QuotedTimestamp.from(
                new GregorianCalendar(1980, 0, 1));
    Db db = Db.open(Database.URL, Database.USER,
            Database.PASSWORD);
    Robot r = new Robot();
    List<Robot> robots =
        db.from(r).
        where(r.dateOfBirth).smaller(dobThreshold).
        select();
    for (Robot robot : robots) {
        System.out.format("%08d: %s\n",
                robot.id, robot.name);
    }
}

In order to get the date inserted correctly into the query, I had to define the following class, which encloses the string representation of the time stamp in single quotes:

public class QuotedTimestamp extends Timestamp {

    public static QuotedTimestamp from(
            Calendar calendar) {
        return new QuotedTimestamp(
                calendar.getTimeInMillis());
    }

    public QuotedTimestamp(long time) {
        super(time);
    }

    @Override
    public String toString() {
        return "'" + super.toString() + "'";
    }
}

It is obviously still early days for JaQu, and I’m sure these kinds of issues will be resolved in the future, maybe by using prepared statements internally rather than converting everything to pure SQL strings. I definitely like the powerful API provided by JaQu, which supports much more complex queries than the one shown above. Please take a look at this page for more code examples, including joins, grouping etc.

Microsoft has addressed this issue quite elegantly with the introduction of LINQ to SQL in the .NET Framework 3.5. Although there is nothing equivalent in Java, I recently came across some promising efforts to improve language integration by providing fluent interfaces or other lightweight wrappers around JDBC and SQL.

Standard JDBC Example

Before we dive into these newer approaches, consider the following example using traditional JDBC (which prints a list of robots that were “born” before 1980):

public void printClassicRobots() {
    Calendar dobThreshold =
        new GregorianCalendar(1980, 0, 1);
    PreparedStatement statement = null;
    try {
        Connection connection = getConnection();
        statement = connection.prepareStatement(
                "SELECT ID, Name" +
                " FROM Robots" +
                " WHERE DateOfBirth < ?");
        statement.setDate(1, new Date(
                dobThreshold.getTimeInMillis()));

        ResultSet rs = statement.executeQuery();
        while (rs.next()) {
            System.out.format("%08d: %s\n",
                    rs.getInt(1),
                    rs.getString(2));
        }
    } catch (SQLException e) {
        e.printStackTrace();
    } finally {
        cleanup(statement);
    }
}

JEQUEL

The JEQUEL (Java Embedded QUEry Language) project by Michael Hunger provides an internal DSL (domain specific language) for building SQL statements. Using JEQUEL, the above example can be rewritten as follows:

public static class Robots extends BaseTable<Robots>{

    public final Field<Integer> id = integer();
    public final Field<String> name = string();
    public final Field<Date> dateOfBirth = date();

    {
        initFields();
    }
}

public void printClassicRobots() {
    Calendar dobThreshold =
        new GregorianCalendar(1980, 0, 1);
    Robots robots = new Robots();
    Sql query =
        Select(robots.id, robots.name)
        .from(robots)
        .where(robots.dateOfBirth.lt(named("dob")))
        .toSql();
    query.executeOn(getDataSource())
        .withParams("dob", dobThreshold)
        .handleValues(new ValueRowHandler() {
            public void handleValue(int id,
                    String name) {
                System.out.format("%08d: %s\n",
                        id, name);
            }
    });
}

After defining a query, it is executed on a DataSource, and its result set is processed. One way (among others) of doing this is by providing a callback object with a handleValue method whose parameters correspond to the columns in the result set. Unfortunately you don’t find out until run time if the method signature doesn’t match the result set. Overall, I think JEQUEL looks like a very elegant solution, and I will be looking into it in more detail.

Quaere

The Quaere project by Anders Norås aims to provide query capabilities similar to LINQ in Java. Just like LINQ itself, Quaere is not limited to database access, but provides a general DSL for querying various types of data structures and data sources. The current implementation supports in-memory arrays and collections, as well as JPA entities (Java Persistence API). There is currently no support for SQL queries, so it doesn’t quite fit into the category of lightweight alternatives to ORM tools (since it requires one in the form of JPA). But it is certainly a project worth mentioning, and it should be possible to add support for SQL in the future.

EoD SQL

Early beta versions of JDK 6 contained an EoD (ease of development) feature as part of JDBC 4.0, using annotations to define SQL statements. This feature was later removed from JDK 6, and so far it has not come back (it is not included in JDK 7 as of build 28). However, the same API was reimplemented (with some additional features) in the EoD SQL project. Using this library, the above example can be rewritten as follows:

public static class Robot {
    public int id;
    public String name;
}

public static interface RobotDAI extends BaseQuery {

    @Select("SELECT ID, Name FROM Robots " +
            "WHERE DateOfBirth < ?{1}")
    public DataSet<Robot> getRobotsOlderThan(
            final Date date) throws SQLException;
}

public void printClassicRobots() {
    Calendar dobThreshold =
        new GregorianCalendar(1980, 0, 1);
    try {
        RobotDAI query = QueryTool.getQuery(
                getConnection(), RobotDAI.class);
        DataSet<Robot> robots =
            query.getRobotsOlderThan(
                    dobThreshold.getTime());
        for (Robot robot : robots) {
            System.out.format("%08d: %s\n",
                    robot.id, robot.name);
        }
    } catch (SQLException e) {
        e.printStackTrace();
    }
}

SQL statements are specified in annotations on the methods of a data access interface (RobotDAI above). An object implementing this interface is automatically generated by the library, and calling the annotated methods on that object causes the corresponding SQL statements to be executed. The query results are returned conveniently as custom data objects rather than just a ResultSet. Unfortunately, the SQL statements themselves still need to be specified as strings, without any compile-time checking or IDE support.

FEST-SQL?

In their article on InfoQ about internal DSLs in Java, Alex Ruiz (of FEST fame) and Jeff Bay showed an example of a DSL for building SQL statements and hinted that this would be released as an open source project. I’m not sure what they will call it, maybe “FEST-SQL” (although its usefulness should not be limited to testing)? Anyway, it certainly seems like a promising project.

Conclusion

Summing up, there is considerable interest and ongoing efforts to provide improvements over traditional JDBC for those situations where a full-blown ORM solution may not be the right tool for the job. I would definitely like to see these efforts continue and gain wider adoption. Thanks to all involved for providing these innovative tools!

Defining a class

Although Python is not specifically an object-oriented language, it does support OOP to a certain extent. So I created a couple of classes to represent the tic-tac-toe game. Let’s start with a class for the grid-shaped game board:

class Board:
    def __init__(self, size, symbol1, symbol2):
        self.size = size
        self.symbol1 = symbol1
        self.symbol2 = symbol2
        self.grid=[size * [' '] for i in range(size)]

Indentation

One thing I had to get used to with Python is the significance of code formatting. While it’s common for scripting languages to attach meaning to line breaks, I have never come across a language where even the level of indentation affects the semantics of the code. In the example above, the first line starts the definition of the Board class, and all subsequent lines with a higher indentation level are treated as part of the class definition. The second line defines the __init__ method, which becomes part of the Board class since it is indented. The subsequent lines are even more indented, which means that they form the body of the __init__ method.

Methods

Methods and functions are defined using the def keyword. The special name __init__ identifies the constructor, which is called when an instance of the class is created. There is no special keyword to refer to the current instance within a method (like the keyword this in Java, C# and C++). Instead, the instance is passed to the method as its first parameter, which by convention is called self. Function parameters and variables are untyped and can be assigned values of any type at runtime. However, the above code was written with the intention that size is an integer specifying the width and height of the grid, while symbol1 and symbol2 should be single-character strings containing the symbols used by the two players. So for the standard tic-tac-toe game, these parameters would be set to 3, ‘O’ and ‘X’.

Working with lists

The last line in the code snippet above initializes a data structure to represent the grid for the tic-tac-toe game. It makes use of several Python features for working with lists. List literals are written as comma-separated values enclosed in square brackets. The expression [' '] creates a list that contains a space character as its only element. Lists can be multiplied with integers, so if size is 3, the expression size * [' '] is equivalent to [' ',' ',' '] (a list with 3 elements, each of which is a single space character). This gives us one row for our grid. The next step is to put several such rows into another list to represent the whole grid. This is done using a Python feature called list comprehensions:

[size * [' '] for i in range(size)]

Within the outer pair of brackets, the expression before the for keyword is evaluated once for each iteration defined by the for clause, and the results from all iterations are collected in a new list. The range function returns a list containing the numbers from 0 to size - 1, and the expression for i in range(size) defines an iteration over these numbers. So the above expression creates a list with size elements, each of which is another list containing size individual space characters.

Object equality and ordering

So far our game board contains an empty grid (filled with spaces). Before we add methods to manipulate the board, let’s define a class to represent a move that a player can make in the game. A move is defined by the row index and column index of the grid cell that the player fills with his or her symbol:

class Move:
    def __init__(self, row, column):
        self.row = row
        self.column = column

    def __cmp__(self, other):
        if self.column == other.column:
            return self.row - other.row
        else:
            return self.column - other.column

In addition to a constructor identified by the special name __init__, this class contains another special method named __cmp__. This method is called whenever instances of this class need to be compared for ordering or equality. In Java terms, implementing this method is equivalent to overriding the Object.equals method and implementing the Comparable.compareTo method in a consistent way. Just like the latter, this method should return a negative integer, zero, or a positive integer depending on whether the object should be considered less than, equal to, or greater than the other object, respectively. For classes that represent value objects, it usually makes sense to implement this method, so that things like set membership work as expected.

The body of the __cmp__ method above shows how the if statement looks in Python. Again, it is important to indent the statements in the if and else clauses properly.

Working with class instances

With the Move class defined as above, we can now add the following methods to the Board class:

    def possibleMoves(self):
        moves = []
        for row in range(self.size):
            for column in range(self.size):
                if self.grid[row][column] == ' ':
                    moves.append(Move(row, column))
        return moves

    def applyMove(self, move, symbol):
        self.grid[move.row][move.column] = symbol

    def undoMove(self, move, symbol):
        self.grid[move.row][move.column] = ' '

The possibleMoves method returns a list of Move objects representing all the grid positions that are available for the next move. It iterates over all grid positions using two nested for loops, which work in a similar way to the enhanced for loop in Java and the foreach loop in C#.

A list element is accessed by writing its index in brackets. Since we represent our grid as a list of lists, the element at a specific row and column is accessed with the expression self.grid[row][column], in the same way as elements of two-dimensional arrays are accessed in Java. If the element is empty (represented by a space character), a corresponding Move instance is created and appended to the list of possible moves. An instance of a class is created by simply writing the name of the class followed by the argument list for its constructor (like in Java and C#, but without the new keyword).

Duck typing

The applyMove method simply stores the given symbol in the grid cell identified by the given Move instance. Actually, since method parameters are untyped, there is nothing that defines that the move parameter must be an instance of the Move class. In fact, it doesn’t need to be. It could be any object that has members named row and column containing integer values. This is an example for the duck typing that Python supports.

Generator expressions

Now we have a game board that accepts moves from the players, but we still need a way to determine when the game is over and who the winner is. So let’s first add a method to the Board class to determine whether the given symbol occupies a complete row, column or diagonal and therefore wins the game:

    def isWinner(self, symbol):
        # check for completely filled rows or columns
        n = self.size
        for i in range(n):
            if (n == self.grid[i].count(symbol)
            or n == sum(1 for j in range(n)
                        if self.grid[j][i]==symbol)):
                return True
        # check diagonals
        return (n == sum(1 for i in range(n)
                         if self.grid[i][i]==symbol)
        or n == sum(1 for i in range(n)
                    if self.grid[i][n-1-i]==symbol))

To determine whether row number i is filled with the given symbol, we use the expression self.grid[i] to get the list corresponding to this row and then call the count method to determine the number of elements in this list that are equal to the given symbol. Unfortunately we cannot use the same approach to check for complete columns, since each column’s elements are spread out across multiple lists. Of course we could just use a nested for loop to iterate over a column’s elements in these different lists. But I decided to make use of a nice Python feature called generator expressions instead. These are similar to the list comprehensions mentioned above, but instead of creating a complete list of values produced by an expression, they produce the values incrementally as they are processed by the surrounding expression. The following generator expression produces a sequence in which the value 1 is repeated as many times as the symbol appears in column i:

(1 for j in range(n) if self.grid[j][i]==symbol)

Applying the built-in sum function to this expression then yields the number of occurrences of the symbol in the column. Similar expressions are also used to check whether the diagonals are completely filled with the symbol.

Invoking methods

Finally, the last method in the Board class determines whether the game is over by checking if any of the two players has won or if no more move is possible because the board is filled completely:

    def isGameOver(self):
        return (self.isWinner(self.symbol1)
                or self.isWinner(self.symbol2)
                or self.possibleMoves() == [])

Note how methods are invoked using dot notation, with the instance (in this case self) before the method name, even though in the method definition it appears as the first parameter after the method name. In general, a method defined as

def method(self, param1, param2, ...)

is invoked like this:

obj.method(arg1, arg2, ...)

Inheritance

The Board class is now complete in that it encapsulates the rules of the game and the effects that individual moves have on the state of the game. What we need now is something that determines what move to make in a given state. That’s the job of the players, so let’s define a class hierarchy of players reflecting different strategies for playing the game. Here is the Player class containing common functionality and the RandomPlayer class that determines its next move by randomly choosing one of the possible moves:

import random

class Player:
    def __init__(self, symbol):
        self.symbol = symbol

    def __str__(self):
        return self.symbol

class RandomPlayer(Player):
    def nextMove(self, game):
        moves = game.board.possibleMoves()
        return moves[random.randrange(len(moves))]

The Player class keeps track of the player’s symbol (‘O’ or ‘X’). In a static language, this class would also contain an abstract declaration of the nextMove method, which must be implemented by subclasses and returns the player’s next move based on the given state of the game. But in Python with its dynamic typing, there is no need (or support) for abstract methods. We just need to remember to define such a method in every subclass, without being forced to do so by the compiler.

The Player class contains another special method named __str__. This method corresponds to the toString method in Java, or ToString in C#. It is called when a string representation of an object is needed, for example when using the print statement.

The RandomPlayer class is derived from the Player class. Unlike in Java, C# and C++, constructors in Python are inherited by derived classes. So even though the above definition of the RandomPlayer class contains no __init__ method, an instance of RandomPlayer can be created using an expression like RandomPlayer('X'), which calls the __init__ method defined in the Player class, causing the argument to the stored in the symbol member variable.

Modules

The import statement in the first line of the code snippet above loads the random module, a standard Python module providing pseudo-random number generator functions. Modules are basically Python source files. Unlike the import statement in Java (which only imports the names of classes so that they can be referred to by their simple names rather than their fully-qualified names), the import statement in Python actually loads and executes the code from the module (note: modules mostly contain class and function definitions, and executing them only creates and adds class/function objects to the symbol table, but does not execute the code inside the functions). Names from an imported module still need to be prefixed with the module name: in the last line of the code snippet above, random.randrange refers to the randrange function defined in the random module.

The nextMove method in the RandomPlayer class above expects as its parameter a game object, which should be an instance of the Game class:

class Game:
    def __init__(self, size, player1, player2):
        self.board = Board(size, player1.symbol,
                           player2.symbol)
        self.active = player1
        self.other = player2
        self.winner = None
        self.observers = []

    def run(self):
        self.notifyObservers()
        while not self.board.isGameOver():
            self.applyMove(
                self.active.nextMove(self))

    def applyMove(self, move):
        self.board.applyMove(move,
                             self.active.symbol)
        if self.board.isWinner(self.active.symbol):
            self.winner = self.active
        self.active,self.other=self.other,self.active
        self.notifyObservers()

    def notifyObservers(self):
        for observer in self.observers:
            observer(self)

    def addObserver(self, observer):
        self.observers.append(observer)

    def removeObserver(self, observer):
        self.observers.remove(observer)

The run method contains the main loop of the game, which keeps asking the active player for the next move until the game is over. For each move, it calls the applyMove method, which applies the move to the board and keeps track of the winner and the active player. It uses a multiple assignment to swap the active player with the other player.

Functions

The Game class uses the observer pattern to notify interested parties about changes to the game state. The observers list contains function objects, which are invoked by the notifyObservers method whenever the game state changes. Observer function objects can be added and removed by calling the methods addObserver and removeObserver, respectively. Let’s define an observer function that displays the current board on the console:

import string

def printBoard(game):
    n = game.board.size
    rowChars = string.digits[1:n+1]
    columnChars = string.ascii_lowercase[:n]
    print '   ' + '   '.join(columnChars)
    print ' ' + '-' * (n * 4 + 1)
    for i, row in enumerate(game.board.grid):
        print rowChars[i] + '|',
        print ' | '.join(row),
        print '|' + rowChars[i]
        print ' ' + '-' * (n * 4 + 1)
    print '   ' + '   '.join(columnChars)
    if game.board.isGameOver():
        print "Game over!"
        if game.winner is None:
            print "No winner!"
        else:
            print "The winner is", game.winner

Working with strings

The printBoard function uses string operations and the print statement to create a visual representation of the game board. Using slice notation and some predefined string constants, the expressions string.digits[1:n+1] and string.ascii_lowercase[:n] evaluate to the digits 1 to n and the first n letters in the alphabet, respectively. These characters are then used to identify the rows and columns of the board, as in algebraic chess notation. The built-in join method is used to concatenate the elements of a row with the given separator. The for loop in the printBoard function makes use of the built-in enumerate function which returns a tuple of index and value for each element in a sequence.

Here is a function that runs a game with the printBoard function as an observer, causing the current board to be printed to the console after every move:

def runGame(game):
    game.addObserver(printBoard)
    game.run()

Now we can let two instances of RandomPlayer play against each other and follow their moves on the console:

runGame(Game(3,RandomPlayer('O'),RandomPlayer('X')))

Exceptions

Watching the computer play against itself may get a bit boring, so let’s define another player class that lets the user choose the next move by typing the desired coordinates into the console:

import string

class ConsolePlayer(Player):
    class InputError(Exception):
        def __init__(self, message):
            self.message = message

    def nextMove(self, game):
        while True:
            input = raw_input("Your move please, " +
                      "player " + self.symbol + ": ")
            try:
                m = self.parseMove(input.strip(),
                                   game.board);
                if m in game.board.possibleMoves():
                    return m
                else:
                    print "Position already taken!"
            except self.InputError, e:
                print e.message

    def parseMove(self, input, board):
        if len(input) != 2:
            raise self.InputError("Please enter " +
                    "exactly two characters " +
                    "(without spaces)")
        n = board.size
        column = input[0].lower()
        columnChars = string.ascii_lowercase[:n]
        if column not in columnChars:
            raise self.InputError("First character" +
                    " must be one of these: " +
                    columnChars)
        row = input[1]
        rowChars = ''.join(string.digits[1:n+1])
        if row not in rowChars:
            raise self.InputError(
                    "Second character must be " +
                    "one of these: " + rowChars)
        return Move(rowChars.index(row),
                    columnChars.index(column))

The ConsolePlayer class is declared as a subclass of the Player class defined earlier, and it contains a nested definition of another class, InputError, which is derived from the built-in Exception class. This exception is used internally by ConsolePlayer to handle errors in the user input.

Recall that the Game class calls a player’s nextMove method whenever it is that player’s turn. The ConsolePlayer‘s implementation of the nextMove method uses the built-in function raw_input to let the user enter the desired grid position. The parseMove method is then called to interpret the input. It uses string manipulation functions to map an input string like “b3″ (letter for a column, digit for a row) to the corresponding column and row indices and returns an instance of the Move class. If the input string is not in the expected format, an InputError is raised. This exception is then caught in the nextMove method by the try/except block surrounding the call to parseMove. So the exception handling in Python seems quite similar to Java, just the syntax is slightly different. And of course there is no such thing as checked exceptions in a dynamic language.

Now we can let the user play against the computer by using a ConsolePlayer instance and a RandomPlayer instance:

runGame(Game(3,ConsolePlayer('O'),RandomPlayer('X')))

OK, I think this was enough Python for today. You can get the full source code for this example from my Subversion repository or download it as a zip file. It was developed using Eclipse with the PyDev plugin. Of course there are a lot of interesting features in Python that I didn’t mention or haven’t even discovered yet. I am going to continue exploring the Python language and the available libraries, especially Google App Engine. I am planning to turn the tic-tac-toe game into a web application using Google App Engine and blog about my experience, so please stay tuned.

Bonus

In case you still want to see more Python code, here is another automated player that makes a stronger opponent than the RandomPlayer. It uses a brute force approach to find the best possible move by recursively evaluating all paths through the game. Caution: While it takes only about 1 second for this algorithm to find the best move on a 3×3 board, it would take something in the order of 1 year on a 4×4 board, because the number of possible paths grows exponentially with the number of grid elements.

class BruteForcePlayer(Player):
    def nextMove(self, game):
        bestMove = self.findBestMove(game.board,
                     self.symbol, game.other.symbol)
        return bestMove[0]

    def findBestMove(self, board, active, other):
        if (board.isGameOver()):
            if (board.isWinner(active)):
                return (None, 1)
            elif (board.isWinner(other)):
                return (None, 0)
            else:
                return (None, 0.5)
        else:
            bestMove = (None, -1)
            for move in board.possibleMoves():
                board.applyMove(move, active)
                score = 1 - self.findBestMove(board,
                                other, active)[1]
                board.undoMove(move, active)
                if score > bestMove[1]:
                    bestMove = (move, score)
                if bestMove[1] == 1:
                    break
            return bestMove

iContract uses the special Javadoc tags @pre and @post to specify a method’s preconditions and postconditions (see my earlier post for an introduction to these terms). There is also an @invariant tag that can be used on the class level to specify a common condition that must hold before and after any of the class’s methods are executed.

Let’s See some Code

The following example code is based on the swarm robotics example described in my previous post. iContract does not support the new language features of Java 5, so I adapted my example to Java 1.4 syntax. I am only going to include some code snippets here to show how to use iContract, but you can get the full source code for this example from my Subversion repository or download it as a zip file. All you need to run this example is JDK 1.4 (or higher) and Ant.

Simple Conditions

The hasFeature method in the Robot interface is used to check whether a robot has a specific feature. It expects its argument to be one of the predefined constants in the FEATURES array. Using the Arrays.binarySearch method (which returns a non-negative result if the value is found in the array), this precondition can be specified as follows:

    /**
     * @pre java.util.Arrays.binarySearch(FEATURES,
     *      feature) >= 0
     */
    public boolean hasFeature(int feature);

A Robot may be part of a Swarm, which is returned by the getSwarm method. The return value must be either null (if the robot does not belong to a swarm), or it must be a Swarm instance whose members (given by its getRobots method) include this robot. This postcondition can be specified as follows:

    /**
     * @post return == null ||
     *       return.getRobots().contains(this)
     */
    public Swarm getSwarm();

Implications

The postcondition of the getSwarm method can be expressed in another way: If this method returns a non-null value, then it must be a swarm that includes this robot among its members. This kind of condition can be expressed in iContract using the implies operator. With this operator, the above condition could be rewritten as follows:

return != null implies return.getRobots().contains(this)

The implies operator is also used in the precondition of the setSwarm method. This precondition is supposed to prevent people from calling setSwarm directly when they want to add a robot to a swarm. Instead, they should call the addRobot method in the Swarm class, which adds the robot to its list of members and then calls Robot.setSwarm.

    /**
     * @pre swarm != null implies
     *      swarm.getRobots().contains(this)
     * @post getSwarm() == swarm
     */
    public void setSwarm(Swarm swarm);

Pre-State Values

For methods that modify an object’s state, it often makes sense to specify the postcondition in terms of how the new state relates to the state before method execution (the pre-state). To express this kind of postcondition, iContract supports the @pre operator, which can be appended to an expression and represents the expression’s value before method execution. For example, after executing the addRobot method in the Swarm class, the swarm’s collection of members should contain the newly added robot, as well as all the robots it already contained before (but no others). Using the @pre operator to express these postconditions, the method would look like this:

    /**
     * @pre robot.getSwarm() == null
     * @post getRobots().contains(robot)
     * @post getRobots().containsAll(getRobots()@pre)
     * @post getRobots().size() ==
     *       1 + getRobots().size()@pre
     */
    public void addRobot(Robot robot) {
        robots.add(robot);
        robot.setSwarm(this);
    }

Quantifiers

There are two ways to tell that a given Robot instance belongs to a given Swarm instance: the robot’s getSwarm method returns the swarm, and the robot is contained in the collection returned by the swarm’s getRobots method. To ensure that the two sides of this relationship are always consistent, we can define an invariant for the Swarm class using the forall quantifier. This quantifier specifies a condition (after the vertical bar) that must be satisfied for all elements in a collection:

/**
 * @invariant forall Robot r in getRobots() |
 *            r.getSwarm() == this
 */
public class Swarm {
    // ...
}

Besides the forall quantifier, iContract supports the exists quantifier, which only requires that at least one element of a collection satisfies the given condition. By nesting the exists quantifier n times, you can specify that at least n elements of a collection must satisfy the condition. This approach is used in the precondition for the Robot‘s getLocation method, which specifies that in order to determine its location, a robot must either have a GPS device or be part of a swarm in which at least 3 other robots have GPS devices, from which it could triangulate its location:

    /**
     * @pre !hasFeature(FEATURE_GPS) implies
     *      exists Robot a in getSwarm().enumRobots()|
     *      exists Robot b in getSwarm().enumRobots()|
     *      exists Robot c in getSwarm().enumRobots()|
     *      a != b && a != c && b != c &&
     *      a.hasFeature(FEATURE_GPS) &&
     *      b.hasFeature(FEATURE_GPS) &&
     *      c.hasFeature(FEATURE_GPS)
     * @post return != null
     */
    public Point2D getLocation();

To make this work, I had to add the enumRobots method to the Swarm class, which returns the swarm’s robots as a java.util.Enumeration. For some reason, iContract wouldn’t accept this condition when using the getRobots method instead of enumRobots.

Runtime Checking

Let’s see what happens when a contract is violated at runtime, for example when we try to directly call Robot.setSwarm instead of going through Swarm.addRobot:

    public static void main(String[] args) {
        Worker worker = new Worker();
        Swarm swarm = new Swarm();
        worker.setSwarm(swarm);
    }

Note: The Worker class implements the Robot interface. As you would expect (based on the Liskov substitution principle), iContract automatically applies the contracts defined in the Robot interface to the Worker class.

In order to enable runtime checking of the contracts, the source code needs to be instrumented with the iContract tool (see the build.xml file for details on how it is invoked). This step results in a new set of source files with embedded code for checking the contracts. When compiling and running these instrumented source files, a RuntimeException is thrown at the beginning of the setSwarm method, indicating that its precondition was violated.

Contract Documentation

Contract specifications can be included in the Javadoc documentation by using iDoclet. However, this only seems to work with JDK 1.3, not with any newer version.

Discussion

What I like about iContract is the declarative language with its logic operators, similar to OCL. This kind of language should be more suitable for static analysis techniques than other contract tools that are based on more general scripting languages. However, I’m not aware of any existing static analysis tools for iContract. There is also nothing to keep you from using arbitrary Java methods in your contract specifications, including methods with side effects.

There are some drawbacks inherent in iContract’s approach of embedding contract specifications in Javadoc comments. First of all, it means that any tool operating on contracts needs to have access to the source code. But contract specifications should really be included in the byte code of a class, since they are part of its public interface. In a perfect world, I would imagine that third-party jar files come with embedded contract specifications that could be seamlessly included in any sort of runtime or static contract checking that you apply to your project, along with the contracts in your own code.

Another problem with specifying contracts within comments is that you don’t get IDE support (such as code completion, syntax checking, refactoring) when writing contracts. Of course this problem could be addressed by special IDE plug-ins, but I’m not aware of any such plug-ins for iContract.

As mentioned above, runtime checking of contracts requires source code instrumentation as a preprocessing step before compilation. This means that, unlike with standard Java assertions, contract checking cannot be enabled or disabled at runtime.

Unfortunately, iContract proved unsuitable for my professional work, mainly because it does not support Java 5 syntax and is no longer actively developed.

Overall, my impression of iContract is that it was an early implementation of design by contract for Java, making best use of what was available at the time. But from today’s perspective, iContract’s approach seems somewhat outdated, and I think that some of its limitations could be overcome by leveraging more recent developments, such as annotations, aspect-oriented programming, the Java Scripting API and instrumentation support. In fact, there are several newer tools based on these technologies, and I will be reviewing some of them in upcoming posts.

Disclaimer: This example is only for illustration and not meant to be useful for any real robotics application.

The following UML class diagram (created with NetBeans) shows a simple object model for robots and swarms. Each Robot may or may not belong to a Swarm, and a swarm can consist of any number of robots. The AbstractRobot class implements some common functionality defined by the Robot interface and serves as the base class for two concrete robot implementations: Navigator and Worker.

The Navigator is equipped with a GPS system that enables it to determine its own location. The Worker‘s unique feature is its robot arm that allows it to pick up and manipulate objects. So each of these specialized robots has only limited functionality, but when working together in a swarm they are much more powerful. For example, consider the task of picking up an object from a specific location and dropping it off somewhere else. Neither one of these two robot types could perform this task by itself, because the navigator cannot pick up objects, and the worker does not know how to get to the object’s location. But if they work together, the navigator can lead the way to the object, and the worker can then pick up the object.

Let’s take a look at the source code for the Robot interface.

public interface Robot {

    public static final int FEATURE_GPS = 1;
    public static final int FEATURE_ARM = 2;

    public static final int[] FEATURES = new int[] {
        FEATURE_GPS, FEATURE_ARM
    };

    /**
     * Checks whether this robot has the given
     * feature.
     *
     * @param feature   must be one of the values in
     *                  {@link #FEATURES}
     */
    public boolean hasFeature(int feature);

    /**
     * Gets the swarm to which this robot belongs.
     *
     * @return must be either null or a swarm whose
     *         members (see {@link Swarm#getRobots()})
     *         include this robot
     */
    public Swarm getSwarm();

    /**
     * Sets the swarm to which this robot belongs.
     * This method should only be called from the
     * {@link Swarm} class.
     *
     * @param swarm if this is not null, its
     *              collection of members (see
     *              {@link Swarm#getRobots()})
     *              must already contain this robot
     */
    public void setSwarm(Swarm swarm);

    /**
     * Gets the robot's location. This method may only
     * be called if this robot either has a GPS device
     * (see {@link #FEATURE_GPS}) or belongs to a
     * swarm where at least 3 robots have GPS.
     *
     * @return  the robot's location (not null)
     */
    public Point2D getLocation();
}

The getLocation method deserves an explanation. If the robot is equipped with GPS, it can get its location easily. But even if it doesn’t have GPS, it might be able to determine its location with the help of other robots. It could measure the time it takes for a signal to travel between itself and another robot, and use this information to compute its distance from that robot. If there are at least 3 robots with GPS in the swarm, the robots without GPS can triangulate their positions by measuring their distances from these 3 robots. This is why the getLocation method requires that the robot either has GPS or belongs to a swarm where at least 3 robots have GPS.

The Swarm class is basically just a collection of robots. Here is the source code:

/**
 * Swarm of {@link Robot}s.
 *
 * All member robots (see {@link #getRobots()}) must
 * "know" that they belong to this swarm (that is,
 * their {@link Robot#getSwarm()} method must return
 * this swarm).
 */
public class Swarm {

    private Collection<Robot> robots =
        new ArrayList<Robot>();

    /**
     * Gets the collection of robots that are members
     * of this swarm.
     *
     * @return  not null
     */
    public Collection<Robot> getRobots() {
        return new ArrayList<Robot>(robots);
    }

    /**
     * Adds the given robot to the swarm.
     *
     * @param   robot   the robot to add; must not
     *                  already belong to a swarm
     */
    public void addRobot(Robot robot) {
        robots.add(robot);
        robot.setSwarm(this);
    }

    /**
     * Removes the given robot from the swarm.
     *
     * @param   robot   the robot to remove; must be
     *                  a member of this swarm (see
     *                  {@link #getRobots()})
     */
    public void removeRobot(Robot robot) {
        robots.remove(robot);
        robot.setSwarm(null);
    }
}

The AbstractRobot class contains straightforward implementations of the methods defined by the Robot interface, except for the getLocation method, which is implemented differently in the Navigator and Worker classes (using GPS and triangulation, respectively). You can get the full source code for this example from my Subversion repository or download it as a zip file. All you need to run this example is JDK 5 (or higher) and Ant.

In upcoming posts I will evaluate various tools for creating formal contract specifications to replace the informal contracts that I wrote in the JavaDoc comments above.

Clearly specifying such contracts is essential, especially for published interfaces. It allows developers of client code to make sure they use the method correctly, and it tells them what assumptions they can make about the result and/or side effect of the method. When something goes wrong, the contract helps find the cause of the problem. If the precondition was not satisfied, then there must be a bug in the calling code. If the precondition was satisfied, but the postcondition was not, then there must be a bug within the method.

So I guess what I’m saying is that documenting your contracts is important. That’s pretty much common practice, and many people would say it’s a no-brainer. So let’s go a step further. In fact, relying only on documentation to specify contracts has some limitations. First of all, contracts specified in a natural language (such as English) can be ambiguous, and different people may interpret them differently. It’s also easy to forget to update the documentation when changing the code. And a lot of methods do not even get documented in the first place, especially if they are not part of a published interface. But I think that being clear about contracts is also important among internal classes and non-published interfaces. Finally, the biggest limitation seems to be that adherence to these contracts can only be checked manually. In order to use the contracts for locating a bug, you first need to know that the bug exists. Only after learning about a bug and knowing how to reproduce it, you can step through the code and look for a contract violation.

To overcome these limitations, you need a formal, unambiguous way of specifying contracts, and an automated way of checking them. That’s exactly what the design by contract methodology promises. Bertrand Meyer introduced this concept with his programming language Eiffel back in 1986. Eiffel has built-in support for specifying formal contracts as part of the program code. But what if you can’t use Eiffel? Interestingly, in over 20 years design by contract still hasn’t made its way into most mainstream languages.

For lack of language support, I haven’t used “real” design by contract in a serious project. Instead, I have used regular assertions (using the assert keyword in Java or the corresponding macro in C/C++) to verify my contracts. I have found assertions very helpful in debugging and catching bugs early. But since these assertions are just statements within the method implementation, they cannot really be used to specify contracts, but only to verify them. So I still have to specify the contracts separately in the documentation in order to make them explicit and visible. But this means that the contract is represented twice, violating the DRY principle.

I think that design by contract done right can really help achieve software sustainability. So I’ve been experimenting with some external tools that add support for design by contract to languages like Java, and I will be writing about my experiences with these tools. In evaluating these tools, I’m particularly interested in their support for:

• Generating documentation from contracts
• Run-time verification of contracts
• Static analysis of contracts to generate unit tests or even prove correctness

Stay tuned for some hands-on code examples using design by contract.

One thing that has always bothered me about JDBC is the disconnect between the SQL command strings and the surrounding Java code. Consider the following example (which prints a list of robots that were “born” before 1980):

    public void printClassicRobots() {
        Calendar dobThreshold =
            new GregorianCalendar(1980, 0, 1);
        PreparedStatement statement = null;
        try {
            Connection connection = getConnection();
            statement = connection.prepareStatement(
                    "SELECT ID, Name" +
                    " FROM Robots" +
                    " WHERE DateOfBirth < ?");
            statement.setDate(1, new Date(
                    dobThreshold.getTimeInMillis()));

            ResultSet rs = statement.executeQuery();
            while (rs.next()) {
                System.out.format("%08d: %s\n",
                        rs.getInt(1),
                        rs.getString(2));
            }
        } catch (SQLException e) {
            e.printStackTrace();
        } finally {
            cleanup(statement);
        }
    }

What I don’t like about this kind of code:

1. Since the SQL command is just another string, the compiler does not know anything about its meaning and cannot check if its syntax is correct or if the number and types of columns in its result set are consistent with the subsequent method calls that extract data from the result set.
2. Different database systems use different variants of SQL, making it quite challenging to write portable SQL statements in all but the simplest cases.
3. Refactoring the database schema can be very tedious and error-prone if you have to resort to a text search to find all the places where a certain table or column is used in your code.

In C# 3.0, all of these issues have been addressed through the introduction of LINQ to SQL. But Java developers are out of luck and probably will be for some time, since adding something comparable to Java would require similarly wide-ranging language changes as were necessary to make this work in C#. So while I can’t provide a complete solution to these problems in Java, there are some simple steps I found useful in addressing at least some of them.

Making table references explicit

As a first step, let’s add a class to represent the Robots table, with a string constant for the table name and one for each column name:

public class Robots {

    public static final String TABLE = "Robots";

    public static final String ID = "ID";

    public static final String NAME = "Name";

    public static final String DOB = "DateOfBirth";
}

We can then change the query method from above as follows:

    public void printClassicRobots() {
        Calendar dobThreshold =
            new GregorianCalendar(1980, 0, 1);
        PreparedStatement statement = null;
        try {
            Connection connection = getConnection();
            statement = connection.prepareStatement(
                    "SELECT " + Robots.ID +
                    ", " + Robots.NAME +
                    " FROM " + Robots.TABLE +
                    " WHERE " + Robots.DOB + " < ?");
            statement.setDate(1, new Date(
                    dobThreshold.getTimeInMillis()));

            ResultSet rs = statement.executeQuery();
            while (rs.next()) {
                System.out.format("%08d: %s\n",
                        rs.getInt(1),
                        rs.getString(2));
            }
        } catch (SQLException e) {
            e.printStackTrace();
        } finally {
            cleanup(statement);
        }
    }

The main advantage of this approach is that it makes table references explicit. If string constants are defined in this way for all table and column names and used consistently in all SQL statements, then it becomes easy to find all the places in your code where a given table or column is used, simply by using the “Find References” feature of your IDE. This can be helpful when refactoring the database schema.

Another advantage of representing database artifacts as explicit Java artifacts is that these Java artifacts provide a good place for documenting the database schema. Each table’s or column’s meaning can be documented by attaching a JavaDoc comment to the corresponding class or field declaration.

But wait, there’s more!

Adding table definitions

Now you might say that using string constants is pretty obvious and really no big deal, and I would agree with you. So let’s go a step further. If we’re going to create a separate class for each table, we might as well use that class for a little more than just to hold a couple of string constants. In addition to declaring the names of tables and columns that are referenced in the program, we can include the definitions of these tables and columns.

So I created two annotations, @Table and @Column, to hold some meta-data about tables and columns, for example the data type and size of a column. The class for the Robots table now looks like this:

@Table(name = "Robots")
public class Robots {

    @Column(name = "ID", index = 0,
            type = Types.INTEGER, primaryKey = true)
    public final String id;

    @Column(name = "Name", index = 1,
            type = Types.VARCHAR, size = 100)
    public final String name;

    @Column(name = "DateOfBirth", index = 2,
            type = Types.DATE)
    public final String dob;

    public Robots(String name, String[] columns) {
        this.tableName = name;
        this.id = columns[0];
        this.name = columns[1];
        this.dob = columns[2];
    }

    @Override
    public String toString() {
        return tableName;
    }

    private final String tableName;
}

I am not going to go into details about the individual elements in these annotations, but you get the idea. The point is that we now have a class that encapsulates the definition of a specific table. The class contains the information necessary for creating this table in a database. It is actually pretty straightforward to write some general code that takes this kind of class as input, uses reflection to read the annotations from it and then calls DdlUtils to create the actual database table.

Another thing I changed from the first version of the Robots class is that the fields for the columns are no longer static. In this way, these fields can be initialized at run time, for example to account for naming differences between multiple existing databases, or to automatically quote the names if they conflict with reserved words in the database.

Using this new version of the Robots class, the query method looks like this:

    public void printClassicRobots() {
        Calendar dobThreshold =
            new GregorianCalendar(1980, 0, 1);
        PreparedStatement statement = null;
        try {
            Connection connection = getConnection();
            Robots robots = getRobotsTable();
            statement = connection.prepareStatement(
                    "SELECT " + robots.id +
                    ", " + robots.name +
                    " FROM " + robots +
                    " WHERE " + robots.dob + " < ?");
            statement.setDate(1, new Date(
                    dobThreshold.getTimeInMillis()));

            ResultSet rs = statement.executeQuery();
            while (rs.next()) {
                System.out.format("%08d: %s\n",
                        rs.getInt(1),
                        rs.getString(2));
            }
        } catch (SQLException e) {
            e.printStackTrace();
        } finally {
            cleanup(statement);
        }
    }

The method now requires an instance of the Robots class, since its fields are no longer static. This instance could be created as part of some initialization code when the database connection is established. At this point it could also be checked whether the existing tables in the connected database are consistent with the definitions in the corresponding Java table classes, and if necessary, tables could be created or altered automatically using tools like DdlUtils as mentioned above.

I like to think of table classes like the Robots class as defining the interface between the Java code and the database. Here you can define explicitly the assumptions you make troughout your program about the database schema. These assumptions can be verified or enforced early in your program’s execution (by checking and/or creating the database schema), and from that point on you shouldn’t have to worry about getting exceptions at random locations in your code because some column doesn’t exist (unless someone changes the schema at runtime).

I hope you enjoyed my first real blog post, and I am looking forward to your comments. I would be interested in your experiences in this area, and in any ideas on how to solve the remaining problems that I mentioned in the beginning.

Bonus

In case you were wondering, this is what the above method prints on my example database:

00000027: R2-D2
00000018: C-3PO
00000011: Maria

You can read more about these and other famous robots at the Robot Hall of Fame.

One of my goals in my daily work is to develop sustainable software: software that can be maintained at a high level of quality in the long term while frequently adapting to changing requirements. All too often, development focuses on getting stuff working for the next deadline, while the long-term vision and consistency of the design is neglected. Over time, the code becomes increasingly complex and hard to maintain, and the quality deteriorates.

Of course these problems are not new, and there are ways to help prevent them, for example using agile methodologies and practices such as unit testing and refactoring. I am planning to blog about my experiences in putting some of these techniques into practice, and there will probably also be insights from everyday problem solving, as well as random geek stuff…

Stay tuned!