For those that don't know SQLAlchemy-migrate allows you to version control database changes and easily upgrade and downgrade a database. Basically you write a python script with two functions: upgrade and downgrade. You test the script against the database, commit it to the SQLAlchemy-migrate version repository (not to be confused with your source control mechanism). Finally you upgrade your development database.

When it the time comes to deploy the new application you simply ask sqlalchemy-migrate to upgrade your database to the current version. sqlalchemy-migrate reads the current version of your schema from the database (via a custom table it inserts) and proceeds to upgrade your schema by running each upgrade script in turn.

Often you want your upgrades and downgrades to run within a transaction. Not because you expect them to fail but because while writing them you don't wont to leave the database partially upgraded or downgraded if your script fails. To do this I wrote a transaction decorator. Here is a template for an upgrade script:

#!/usr/bin/env python
# encoding: utf-8
 
from datetime import datetime
from sqlalchemy import *
from migrate import *
from migrate.changeset import *
 
metadata = MetaData(migrate_engine)
 
def transaction(f, *args, **kwargs):
    def wrapper(*args, **kwargs):
        connection = migrate_engine.connect()
        transaction = connection.begin()
        try:
            result = f(*args, **kwargs)
            transaction.commit()
            return result
        except:
            transaction.rollback()
            raise
        finally:
            connection.close()
 
    wrapper.__name__ = f.__name__
    wrapper.__dict__ = f.__dict__
    wrapper.__doc__ = f.__doc__
    return wrapper
 
@transaction
def upgrade():
    pass
 
@transaction
def downgrade():
    pass
 

I fill in the upgrade and downgrade functions and I'm done
I include the decorator in every script as it's good practice to make your scripts as independent as possible. If you imported it from a module you may improve it in the future and inadvertently break your old downgrade scripts.

I hope this helps you version control your database schema and data.

#if defined(__FreeBSD__) || (defined(_WIN32) && !defined(__GNUC__)) || defined(__sun__)
/* what's this, we have no round function either? */
static double round(double num)
{
  return (num >= 0) ? floor(num + 0.5) : ceil(num - 0.5);
}
#endif
 

However 'round' is defined in FreeBSD and has been since FreeBSD 5.3 (according to the manual page). The fix is simple, just remove the 'defined(__FreeBSD__) ||' part of the '#if' and you should be fine. Now you can 'easy_install .' psycopg2.

PS: I'd tried to raise a ticket but http://www.initd.org/ trac seems to have been down for ages.

What gets started is handled by a program called SystemStarter. On boot it looks through /Library/StartupItems for folders. Inside those folders it looks for a script with the same name as the folder and a plist called StartupParameters.plist.

So as super user:

cd /Library/StartupItems/
mkdir PostgreSQL
touch PostgreSQL/PostgreSQL
touch PostgreSQL/StartupParameters.plist
 

Now we need to fill in the details. In PostgreSQL/PostgreSQL enter:

#!/bin/sh

sudo -u postgres /usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data -l /usr/local/pgsql/data/logfile start

This is a long line, but it's basically running postgres (via pg_ctl) as the user postgres. You'll need to change this to the whatever user you run postgres as and where ever your install of postgres is. I've used the standard names and directories so it's likely this will work for you too.

Now lets look at the plist. This file contains information for SystemStarter about when to start postgres, what postgres needs and provides and any messages to print to log files:

 
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist SYSTEM "file://localhost/System/Library/DTDs/PropertyList.dtd">
<plist version="0.9">
<dict>
        <key>Description</key>
        <string>PostgreSQL</string>
        <key>Messages</key>
        <dict>
                <key>start</key>
                <string>Starting PostgreSQL</string>
                <key>stop</key>
                <string>Stopping PostgreSQL</string>
        </dict>
        <key>OrderPreference</key>
        <string>None</string>
        <key>Provides</key>
        <array>
                <string>PostgreSQL</string>
        </array>
</dict>
</plist>
 

You can test your new settings by typing sudo SystemStarter -n -D, fix any error messages, reboot and you should have postgres all the time

I use twitter every day to send out really quite boring messages. "I've had some chocolate" and "I like lossless encoding" are all stunning examples of how boring my twitters are. Sometimes they don't appear on twitter for hours... occasionally they don't appear at all.

As an engineer I think I have a fair idea of how twitter works BUT I have a code base full of my own problems. I tend to spend my time thinking about those rather then fixing twitters by proxy.

So I'm asking the internet, can you all just shut up about fixing twitter. I am sure they are aware of their deficiencies and how it would be better if they didn't do X or did do Y. Let's leave it to them to solve and if they don't I'm sure one of the genius twitter fixing bloggers can make his own distributed erlang über resilient systems that only blogger can create.

Here endeth the rant!

Putting yourself in the place of a normal user is, basically, impossible but every programmer/ui designer attempts to do this EVERY day.

Why am I talking about this? Well my dad's hotmail account was hacked over the weekend and of course I was called to look in to it. Of course my first thought was that he had a weak password. So I ask him "How good is your password, does it contain numbers and puntuation?". He paused and said "No, it's a word, they asked for a pass WORD".

Now it's easy to think that this is my dad's fault. But when you think about it, he did exactly what was asked of him. He entered a pass WORD a single word. A word from the dictionary, that's where words come from for normal people.

No one programmer or designer has failed him. An entire industry asks to world for passwords. To the industry, this is normal. It knows what it means by password but to normal people it means a word.

With this in mind I've raised a ticket for New Metal Army to change the word password to pass phrase (after asking my dad what he thought it meant of course).

Check out the video!

You can read more here

The next day (determined to get somewhere) I run VMWare Fusion, go in to IE and after a million click sounds I start downloading the Mac version of Flash CS3. So I need Windows to download the Mac version...grr

Being a good netizen I decide to file a ticket against the problem:

EUROPEAN ENGLISH! There is no such thing as European English. There is English and American English. One of them is correct, the other is a phonetically correct abomination.

I didn't realise my 30 day trial would be such a trial.

Well this scientist has an excellent story about how HE was expelled from the queue for the movie... but there is an awesome twist to the tale.

http://scienceblogs.com/pharyngula/2008/03/expelled.php

Subversion

First and foremost, subversion is a source control tool but it's also a safety net. I can't see how you can call yourself a developer if you don't have source control. From day one I got myself an account on Webfaction set up subversion and started versioning everything. They allow https connection so everything is safe and because I don't live in their data center it's off site as well

Trac

Again on Webfaction, I installed Trac. Trac is a project management webtool. It hooks in to svn so you can see repository changes as a timeline and see checkins as colored diffs. It also has a ticket system and wiki. I found the wiki to be invaluable. It allowed me to track my thoughts in a 'low barrier to entry' fashion. It's so easy to add a wiki page, there really isn't any excuse for not doing it! The ticket/task tracking is basic but fine for small teams. There is a simple notion of a milestone and a release but the metrics for measuring progress just aren't there. For me though, this was fine. Web projects are relatively simple and single man projects don't require metrics really.

TextMate

TextMate is a text editor primarily aimed at programmers but useful for anyone who needs to edit structured documents where the content is more important then the layout. I first ran in to TextMate while I was evaluating Ruby. The Ruby community seems to be dominated by macbook pro users sitting in Starbucks, bashing code in to TextMate and buzzing on coffee. Although I decided not to use Ruby I did stick with TextMate. It really is the premier text editor for OS X. It's the first editor that has replaced emacs for me. I didn't use any extensions for TextMate but I did use PyChecker and JSLint with it to check my python and Javascript code on save.

CSSEdit

CSSEdit is an editor devoted to CSS. The cool thing is that it has live editting. So as you edit the CSS file for a site, it changes the preview of the site. This is quite frankly brilliant and very cheap compared to the time it saved me.
Combined with YUI's CSSReset I can proudly say I 'ported' New Metal Army from Firefox to Safari, IE and Opera in a few days with minimal difficulty. CSSReset should be the basis for ALL website development and Yahoo should get a fucking medal for creating it. Cheers guys

Pixelmator

Pixelmator is a nice image editor the supports PSD's and layers. I got it as part of a MacHeist bundle and just started using it rather then paying hundreds of quid for Photoshop. Because it supports PSD's I can still work with artists who quite rightly want to use Photoshop.

VMWare Fusion

Working on a mac makes it hard to get Internet Explorer working. I don't want or need a PC so VMWare comes to the rescue. VMWare is an OS virtualisation package that basically allows me to run various versions on Windows, Linux and BSD in a window. So I can run IE 5.5, 6.0 and 7 in separate virtual instances and check compatibility across IE5.5+, Safari, FireFox 2+ and Opera 9+. Since I'm a one man operation time consuming and boring tasks like this need to be made as painless as possible... or they won't get done.

Firebug

Firebug is an extension for Firefox that allows you to examine pages and page elements to see how their layout has been calculated but it also allows you to debug Javascript inside the browser. It far exceeds IE's rather forced script debugger compatibility or .NET integration. This tool saved me months of debugging and more importantly sped up the learning curve for web development.

Wingware IDE

There were times when I needed to debug fiddly python code and the python command line debugger just wasn't cutting it. I turned to Wingware's IDE for debugging and it served me very well. I only used it a few times though.

NeoOffice

Neooffice is a Mac port of Open Office. It's a good replacement for Claris or MS Office. I only needed to write a few letters to lawyers and such but it's good to not have to pay £500 for the privilege. I did use it a lot for opening up 50Mb csv files and it coped quite well with it.

Python

Well this is pretty obvious, TurboGears uses Python so I am forced to work with it. Well luckily python is one of the best languages to do anything in. With a vast array of modules it's really quick to create scripts to process data for your website. As an affiliate for many music companies (iTunes, HMV, Play, Amazon, ...) I get access to huge XML and CSV files with their latest prices, stock and shipping information. Because of python's flexibility it is easy to merge this data in to New Metal Army.

TurboGears

TurboGears is a no brainer here. It's the web framework New Metal Army sits upon. For me though TG was more then a framework. TG comes with a great community bundled for free. I learned a lot just reading the mailing list. I also learned a lot by reading the code for TurboGears itself. It's well written and cleanly constructed. There is some advanced python on there but I don't think anything is overly complicated. Python is a great language to read and I think more programmers need to learn that they CAN dive in to most code and quickly work out what is going on.

PostgreSQL

Postgres has always been my database of choice. Unfortunately I have no real reason for this other then it's always covered my needs and it is standards compliant so I expect it to cover my needs for the foreseeable future. TurboGears uses an ORM mapper (SQLAlchemy) between it and the database so I could swap Postgres for MSSQL, MySQL, Firebird, Oracle or a number of others. Postgres serves New Metal Army VERY well.

It did however appear in Google Reader. Stou from http://www.siafoo.net/ noticed the spam and kindly emailed me.

It is completely my fault. I've known that my install of Wordpress was quite old for sometime. This was the kick in the arse I needed to upgrade I guess. So if you are looking at this in a browser you will have noticed it's got the basic WP theme. I'll be bringing my unique design skills back to the internet soon.

Once again thanks to Stou for being a good internet citizen.

edit: you may see a lot of drug adverts on the site for a few days till google sorts itself out!