A potted history of JAOO 2009

A couple of weeks ago I attend the JAOO 2009 conference in Brisbane. What follows is a biased, incomplete and probably misleading account of my impression of the two days.

Keynote

I always assumed conference keynotes were meant to be broad, sweeping and inspiring. This one was narrow, technical and delivered in a mind-numbing monotone. Maybe it’s just the way they do things now?

Introduction to Objective-C

This was clearly targeted at people who have no exposure to Objective-C, but rather than just being a dry survey of the language syntax and libraries, Glenn Vanderburg provided a nice historical overview of the Objective-C and its heritage.

My take away: Objective-C is basically SmallTalk, and SmallTalk is basically Lisp.

Google App Engine: Building an App the Google Way

Pamela got rave reviews in Sydney, and she’s certainly an entertaining speaker. If you’d never heard of GAE, or never looked at its capabilities, this would have been a very good introduction. I’ve built a couple of small GAE apps though (in Python), and other than seeing the Java version of some of the APIs, this talk really told me nothing new.

1,001 Iterations: Product Design, Illustrated

This was a recounting of the process Avi Bryant went through taking a new idea from its inception through many refinements to a polished product.

Perhaps most the interesting part for me was Avi’s assessment of the relative strengths of the various languages he ended up using to implement the product:

• Squeak - for “thinking” in (i.e. the interesting problems and their solutions)

• Java - for nuts-and-bolts computing (crunching numbers)

• Ruby - for interfacing with external libraries and APIs (e.g. twitter)

• JavaScript - for interacting with the user

I’m not sure it’s always a good idea to mix so many technologies in the one product, but it certainly makes some sense to not get hung up on the One True Language, and just use each where they’re best suited.

Speeding Ducks

Avi again. Much more technical this time. Avi’s main point: Ruby really is slow, but there’s no reason it has to be.

He began with an interesting history of Java’s Hotspot VM, which was based on technology developed for SmallTalk and Self in the 1980s. But Google’s V8 was built by three people in about 3 months–surely we can do the same for ruby!

At the end of the talk, Avi was challenged by Joshua Bloch. Josh disputed Avi’s claim that because V8 was built in three months, all optimising “hotspot” VMs should be easy to build. Java’s current VM has been constantly improved over many years, and solves many non-trivial problems.

Of course, this sort of interaction between notable figures in our industry is exactly why you go to conferences like JAOO.

Hey You! Get On To My Cloud! - Application Development in the Clouds

Dave Thomas gave us some thought provoking ideas about current languages development platforms. Is JavaScript the way of the future? I’m not so sure, but I think one of Dave’s main points is worth paying attention to: functional programming is the way forward if we want to improve the speed with which we can build software.

Atlassian

Mike Cannon-Brookes gave us a bit of background of Atlassian’s history (they’ve gone from two people and one product to nearly 100 engineers and ten products in eight years), then listed what he thought were the ten key practices that have made them successful. I’ll excerpt just the ones I think are worth talking about, and add my thoughts (not necessarily agreeing with Mike):

• Agile - it’s the principles that are important, not any particular methodology or set of tools

• Code review - there’s plenty of hard evidence that code review/inspection is one of the best ways to reduce the number of defects in software. Of course, pairing is the ultimate form of code review.

• Optimise tests - the main goal: get feedback to developers as fast as possible. Some of the things Atlassian do to achieve this include selectively running only tests that could possibly be affected by a code change (by doing static analysis on coverage), and splitting functional tests into parallel builds.

  This is a common problem–functional test suites that take so long to test an application that the pipeline from code check-in to the “you broke the build!” feedback can be hours. Atlassian’s solution is to split the tests into chunks that run in a maximum of ten minutes, and have enough build agents to run all the chunks in parallel.

• Put everything in a wiki. Yeah, they would say that, wouldn’t they? :-)

• “Dev speed posse” - Atlassian have a small team that spend a fixed amount of time every week just focusing on removing things that slow down development. This is a great idea (although not one that’s unique to Atlassian), and something more organisations should consider. One of the more interesting goals they have is that the “checkout loop” (the time it takes a developer to go from a clean machine to having a checked out app running locally and ready to work on) should be no more than ten minutes. How many large development shops can achieve that?

Josh Bloch - Effective Java

This was basically a summary of some of the new things in the second edition of Effective Java. About a third of the talk was all about generics. Good grief. Surely someone has noticed by now that this has all gone horribly wrong.

“Concurrency is hard” - even if you use the right APIs (for example, always use ConcurrentHashMap not Collections.synchronized*()) it’s still easy to get it wrong. Read Brian Goetz’s Java Concurrency in Practice.

And finally: Serializable is bad, since it allows objects to be created without using constructors. This can lead to invariants and other assumptions being violated. Josh says to use serialization proxies instead.

Doug Crockford on JavaScript

One of the classic Doug Crockford JavaScript talks. Probably nothing new if you’d listened to his talks from Yahoo’s YUI Theater, but still well worth spending 45 minutes listening to in person.

Some of Doug’s comments, observations and tips:

• JavaScript has widest range of user programming skills of any language, from computers scientist to cut-and-pasters

• JavaScript has many influences, including: Self (prototypes, dynamic typing), Scheme (lambda, loose typing), Java (syntax), Perl (regexps)

• it is commonly being used as a functional language–you’ll write better JavaScript if you think functionally

• eval is the most misused feature - just don’t do it!

• always use ===. You’ll be tempted to use == instead, but it’s broken–it causes type coercion, which leads to unexpected and buggy results

• manage the divide between client and server (don’t recreate the server in the browser)

Software Visualization and Model Generation

Eric Doernenburg is a consultant at ThoughtWorks, and I’d heard him talk before about some of the cool code visualisation tools he’s put together. The basic idea is that by visualising certain attributes of a code base, it’s much easier to focus on the trouble spots without getting lost in the detail of thousands of lines of code.

Interestingly, Eric uses both common tools (e.g. CheckStyle) and the more exotic (CodeCrawler, CodeCity). Those last two are more or less self-contained, but Eric does really cool things with CheckStyle and Graphviz, and a bit of XSL to glue them together. As a general approach, use whatever analysis tool is closest to what you need, then map the output into a format your visualisation tool can read.

Smart Software with F#

An overview of, and small sample app, in F#, Microsoft’s functional language for the CLR. The main message:

• F# is great for data-intensive applications

• smart algorithms are (relatively) easy in F#

Both of which apply to any functional language of course.

You try to give Microsoft people the benefit of the doubt… but Joel Pobar, despite obviously being very knowledgeable about F# and functional programming, still managed a couple of clangers. Most egregious: he called python an “elementary imperative language”. Fair enough if your background is Visual Basic and you’d never heard of functional programming… but this guy is the F# expert.

Anyway, it was good to see a bit of F# in action. If it gets more people thinking about functional programming, great. But it doesn’t offer anything you can’t get in Clojure, SmallTalk, Scheme, etc., unless you’re stuck in the Microsoft ecosystem.

Overall, a great couple of days. I learnt new things and expanded my mind about things I already knew. I hope to go again next year, and hopefully it will come to Melbourne!

A new start

I have a new job, and contrary to what I’ve said previously, it’s at a consulting company. With a differnce.

Cogent is a consulting company, but with ambitions to be a product company. In fact, I’ve spent my first few weeks here working on our first publicly available product, Runway. (Runway is a task management app that supports the principals of Getting Things Done®.)

I’m really excited to be here. Cogent has an explicit goal of treating its employees and its customers humanely. It’s very open and free of bureaucratic nonsense. And it’s full of really smart people–although the average probably just went down bit… :-)

Emacs full-screen shortcut

When you’re writing or coding, you want to remove as many distractions as possible. In addition to obvious things like shutting down you email, IM and twitter clients, it can be helpful to put your editor in full-screen mode. This way, the editor is the only thing visible, so your attention isn’t drawn to menu bars, flashing notifications or bouncing dock icons.

To create a shortcut for fullscreen mode in emacs, put this in your ~/.emacs file:

(defun toggle-fullscreen ()
  (interactive)
  (set-frame-parameter nil 'fullscreen (if (frame-parameter nil 'fullscreen)
                                           nil
                                           'fullboth)))
(global-set-key [(meta return)] 'toggle-fullscreen)

Now pressing M-return (usually alt + return on Windows/Linux or `⌘

• return` on a Mac) will toggle Emacs between normal and full screen mode.

Thanks to Vebjorn Ljosa in this thread for this code snippet.

Spaces becomes usable in OS X 10.5.3

Spaces was one of the most anticipated features in Leopard, at least for Unix/X11 refugees like myself. X has had virtual desktops for decades, but users of “mainstream” desktop operating systems (i.e. Windows and Mac OS X) have had to rely on third-party utilities to get the same functionality.

In the case of OS X, Leopard was set to change that with Spaces. Unfortunately, the implementation was broken in such a way as to make it incredibly frustrating to use the way I’m used to using X11. I typically have Terminal and Safari (and often Emacs) windows open on multiple desktops. But on a desktop dedicated to a particular task, I want to be able to ⌘-⇥ (command-tab) between application windows on that desktop. Prior to 10.5.3, this would invariably do precisely the opposite of what I wanted, and flip to another desktop that had a window of that application open. This resulted in Spaces being about 5% as useful as X11 for serious keyboard-oriented work.

(For what it’s worth, this whole thing is mostly an issue because of the distinction OS X makes between apps and windows of apps–in X11, alt-tab usually cycles between all windows equally, regardless of what application they belong to. On OS X however, command-tab cycles between applications–⌘-` can be used to cycle between windows of an application.)

But good news! The recent 10.5.3 update to Mac OS X fixes it! Contrary to what Gruber says:

[Y]ou shouldn’t notice any changes, because the default behavior remains the same in 10.5.3

the default behaviour has changed: command-tabbing between applications now stays on the same desktop if the target application has a window there, and jumps to another desktop otherwise.

This is just about perfect. I actually like the jump-to-desktop behaviour for applications that aren’t on multiple desktops (e.g. iTunes), but now the default is to stay on-desktop for apps that are. (I still think I’d be slighly more comfortable if OS X behaved the same way as X11, and treated all windows as equal–but that could be Just What I’m Used To.)

Thanks Apple!

Using git on subversion projects

Despite all the noise lately about distributed version control systems, the chances are any given project you want to work on today will be using Subversion. But that’s OK, you can still get the benefit of all the advanced features of git by using it as a “front end” to subversion.

Before I get into the “how”, why would you want to do this?

The most obvious benefits are having a full local history, and cheap local branching. It’s trivial in git to create branches for features you’re working on, and then easily switch between them. Say you’re working on a feature for the next release, and an urgent bug for 1.0 comes in. Simply:

$ git commit -m "work in progress"
$ git checkout --track -b fix-urgent-bug-1234 release-1.0

...hack hack...

$ git commit -m "fixed bug #1234"
$ git checkout cool-feature-foo

and continue where you left off.

There’s also a bunch of other neat stuff in git that I miss whenever I have to use something else (keep in mind that I’m no svn guru, so there may be similar things in svn if you look hard enough. But I very much doubt they’re as fast). git grep for rapidly searching source trees. gitk for visualising branches and interactively searching for commit messages and changes. Local commits. Oh, and everything is much faster.

OK, on to the how.

We start by checking out the svn repo:

$ git svn clone -s http://svn.example.com/svn/cool-project

The -s switch means “standard layout”, i.e. the recommended subversion usage of trunk/branches/tags. If your project doesn’t follow this convention, you can specify the names of the subdirectories used:

$ git svn clone --trunk=MAIN --branches=branches --tags=releases \
    http://svn.example.com/svn/cool-project

There are lots of other options to clone that can help if you have a really non-standard repo to work with. Check the init command in man git-svn(1).

You should now have the HEAD of trunk in a directory called “cool-protect”. (You can specify a different target directory name by appending it to the git svn clone command.)

The full power of git is now at your command! You can grep the source tree:

$ git grep '^class Model('
django/db/models/base.py:class Model(object):
tests/modeltests/invalid_models/models.py:class Model(models.Model):

Find the git commit corresponding to a subversion revision:

$ git svn find-rev r1234
c5dfec042453672a27fd19ff81131edd01145584

$ git show c5dfec0
commit c5dfec042453672a27fd19ff81131edd01145584
Author: Michael Rowe <mrowe@mojain.com>
Date:   Sat Feb 16 10:14:57 2008 +1100
...

And interrogate the full history of the repo:

$ cd ~/src/django
$ git log '@{3 weeks ago}' -1
commit 696a3322d6709ebffcc436eb6188ea4d769ebfc5
Author: mtredinnick <mtredinnick@bcc190cf-cafb-0310-a4f2-bffc1f526a37>
Date:   Mon Feb 4 04:57:56 2008 +0000

    Fixed a simple TODO item in one error path of the "extends" tag.

In the time we’ve been playing with this, maybe some changes have been committed upstream. To make sure our local repository is up to date, we rebase:

$ git svn rebase

You could also just use git svn fetch to fetch the upstream changes into the repo without rebasing your working tree. In general, I would avoid this unless you know what you are doing, since it can make things complicated when you go to merge and push your changes upstream. If you are working on the main trunk of the svn repo, rebase is almost always what you want.

So now we have an up to date checkout, lets get to work! As you work, add files to git’s “index” and commit to the repo. Commits in git are fast, and should be used almost as frequently as saving a file in your editor. You can always consolidate these “micro-commits” into larger feature or bug fix commits later.

...hack hack...

$ git add src/module.py src/other.py
$ git commit -m "I did stuff"

and repeat.

Note for subversion users: you have to tell git about every file you change, even if it’s not a new file. Details of git usage are beyond the scope of this article (there are some excellent starting points), but be aware that you have to git add each file you want included in a commit.

(Note for lazy git users: This can be combined into a single command for existing files:

$ git commit -m "I did stuff" src/module.py src/other.py

but I tend to prefer the two-step approach for anything but the most trivial changes.)

As you work, you can periodically sync with the upstream subversion repo to get other people’s changes:

$ git svn rebase

This won’t work if you have any local uncommitted changes. However, you can “stash” them away temporarily (in git 1.5.3 and later):

$ git stash
$ git svn rebase
$ git stash apply

In any case, as mentioned above, you want to commit locally as often as possible.

When you have finished work on a feature or bug fix that you want to push back to the subversion repository, make sure all your changes are committed locally to git (git status), then review what you’ve done:

$ git log origin/trunk (by default, or whatever svn branch you're on)
$ git diff origin/trunk

Finally, when you are happy with the work you’ve done and are ready to push it up to subversion:

$ git svn dcommit

This will create individual svn check ins for each git commit since the last upstream revision. If you want to combine local commits into one large svn check in (e.g. because you followed my advice above and made frequent local commits), the interactive rebase command will help:

$ git rebase --interactive origin/trunk

Interactive rebase opens an editor with a list of all the commits since the revision you specify (remotes/trunk in our example).

pick d79a908 A small change to a file
pick c5dfec0 An unrelated change
pick db0346b Fix typo in hello

To combine the typo fix into the first commit, move its line directly below the line for the first commit and change “pick” to “squash”:

pick d79a908 A small change to a file
squash db0346b Fix typo in hello
pick c5dfec0 An unrelated change

The result will be two commits (d79a908 and c5dfec0), with d79a908 incorporating the changes from db0346b. You can do this for multiple consecutive lines if you want to combine many commits into one. See man git-rebase(1) for full details.

Now use git svn dcommit as above to push the revised commits upstream.

We’ve been working on a single branch so far, but one of the big benefits of using git is the cheap branching. Lets start work on a new experimental feature:

$ git checkout -b my-wacky-feature

The -b switch means create a new branch. Without that, git checkout switches to an existing branch.

...hack hack...

$ git add ...
$ git commit ...

At any time, we can commit locally and switch to another branch:

$ git checkout other-thing-to-work-on

...hack hack...

$ git add ...
$ git commit ...

then switch back and continue where we were:

$ git checkout my-wacky-feature

All of the commands we’ve discussed operate on the current branch (unless you specify otherwise). So you can grep for strings, get change logs and diffs and view visual history all in the context of the branch. You can also diff the current branch with another. To get a diff from release-1.0 to the current working tree (on branch fix-urgent-bug-1234):

$ git checkout fix-urgent-bug-1234
$ git diff release-1.0

Or to get diffs between arbitrary branches and revisions (without having to checkout either branch):

$ git diff release-1.0..my-wacky-feature 

See man git-diff(1) for all the options to diff.

git svn dcommit will only push changes on the current branch up to the subversion repository, so you can clean up and consolidate your commits using rebase, then push them back to subversion when they’re ready.

I hope this quick introduction has whet your appetite for combining the power of git with the ubiquity of subversion. There is much more to git (we haven’t touched on merging at all), and once you’ve dipped your feet in, I recommend reading the intros and man pages at the git site.

Please let me know if you have any suggestions or notice any errors.

Aperture 2

I’m a little disappointed that Apple are charging AUD129 for the upgrade to Aperture 2. Sure, there are a bunch of new, and very attractive, features that would otherwise make paying for the upgrade acceptable, but it seems a bit rich given that the full retail price has dropped from USD300 to USD200. Effectively, people who bought 1.x are getting hit twice.

But I guess it doesn’t bother me enough to stop me buying the upgrade…

PHP namespace

This comment from the Drupal Theme developer’s guide is an example of why whatever your question, PHP is not the answer:

An important note- when developing a theme using any of the methods described here, you must be sure that the name of the theme is not the same as the name of any module being used on the site because the function names may collide and your site may no longer function correctly.

New home for a blog

My blog has moved to a dedicated new home: http://www.mikerowecode.com/

All appropriate redirects are in place, but please check your feed reader to be sure. I’ve done a far-reaching survey of a wide range of users and clients–ok, well, actually myself and one friend, both using NetNewsWire–and it seems that it works fine when the feed is accessed directly, but if you have it syncing via NewsGator it doesn’t correctly propagate the new feed URL. It does follow the redirect to get the feed content, but doesn’t to push the changed URL back to the client. I’d be interested to hear about experiences with other readers.

A word about what’s behind curtain

The new site is built from text files using the blosxom publishing system. The text files are formated using John Gruber’s Markdown, with punctuation fixed by his SmartyPants.

I use a number of plugins for blosxom to get things working the way I want. These include archives and recententries to provide the navigation options in the sidebar, entries_index to maintain article time stamps and atomfeed to produce, er, an atom feed. :)

Blosxom runs in “static” mode to generate the site locally, and then I rsync it to my web server, where it’s served as static HTML.

Why blosxom?

It probably seems like a strange choice, when there are so many “advanced” alternatives such as Drupal (which was my previous system), WordPress, MovableType, Blogger, etc., etc. But a couple of things convinced me that blosxom was the way to go.

First, my needs are minimal. I just want to publish the stuff I write with the minimum of fuss and overhead. I wanted a publishing system that would get out of the way.

Second, there is something very appealing about keeping things in plain text. I can write in emacs (which is of course the One True Editor), manage changes with git, search with grep (or spotlight). The directory layout is the same on my hard disk as on the public server. There’s no database to worry about backing up.

Finally, since I’m serving static HTML, in the (admittedly far-fetched) event that this site becomes wildly popular and sees huge amounts of traffic, scaling will be trivial. :)

Reviewboard git mirror

For some months now, I’ve been maintaining a git mirror of the Reviewboard project’s svn repository. The git-svn tool works really well for this, except for one small wrinkle: the reviewboard projects uses svn:external to include an external module, djblets, and git-svn provides no transparent way to support this.

For now, I manage this manually. When ever I notice an update to djblets (which are thankfully rather rare), I use the following process to merge the changes into a branch (with-djblets) in the git repo:

$ cd ~/src/djblets
$ git svn rebase
$ git log -1 | grep -v '^commit' > /tmp/djblets.log

Note: change “1” to whatever number of commits have happened in djblets since the last time I did this. The grep command removes the git-specific “commit” lines from the log, which won’t be interesting enough to include in the commit message below.

$ cd ~/src/reviewboard-with-djblets
$ git status # make sure working dir is clean
$ cp -rp ~/src/djblets/* reviewboards/djblets/

At this point, I do a git status and manual sanity check to make sure the changes I’m about to commit here match the incoming change to djblets.

$ git add <files that are changed/new>
$ git commit -F /tmp/djblets.log
$ git push public-repo with-djblets

Done! Simple, no? Well, no… This process has a number of problems, the main one of which is it’s manual, and I have to do it. I’m hoping that I’ll be able to bend git-submodule to my will enough to take care of this.

Job search update

It’s been a while coming, but here is a quick update on my job search:

Whether it was my letter to recruiters, or just dumb luck, I ended up finding and accepting a pretty good contract job back in November. A product company, smart people, great relaxed environment. More or less everything on my list. Even a kick-ass coffee machine in the office. As expected, it was a smaller “boutique” recruiter that came through.

I’ve had a happy and productive couple of months.

Then this week, the company was bought by Microsoft and my contract terminated early. *sigh* More job search news to come, I guess.