Archive for January, 2008

FreeBSD 6.3 and Turbogears

Saturday, January 19th, 2008

I upgraded a test server to FreeBSD 6.3 (released a few days ago) and all was working well apart from my TurboGears app. I run a TurboGears instance behind mod_wsgi and it wouldn't start. Here is the error I got in http_errors.log

[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149] mod_wsgi (pid=1292): Exception occurred within WSGI script '/home/m/release1.0/apache/turbogears.wsgi'.
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149] Traceback (most recent call last):
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149]   File "/home/m/release1.0/apache/turbogears.wsgi", line 67, in <module>
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149]     import turbogears
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149] ImportError: No module named turbogears

That's odd. I've not uninstalled TurboGears and my background processes that #import TurboGears still work. Infact if I go to the python command line and type #import TurboGears it all works... bugger.

To complicate matters (in this case) I use a workingenv to contain a very specific version of TurboGears and all of it's dependencies. In order for the wsgi script to access the sandbox environment I use an excellent script which tweaks the runtime environment to include the paths in a working env. My first though is that something here had gone wrong. So I turned to prints and some basic error capture.

# Load all distributions into the working set.
from pkg_resources import working_set, Environment
 
env = Environment(root)
env.scan()
 
distributions, errors = working_set.find_plugins(env)
for dist in distributions:
    working_set.add(dist)

Printing out errors revealed:

errors:
{Amara 1.2.0.2 (/usr/home/m/tgenv1_0_32/lib/python2.5/Amara-1.2.0.2-py2.5.egg):
   DistributionNotFound(Requirement.parse('4Suite-XML>=1.0.2'),),
TGCaptcha 0.11 (/usr/home/m/tgenv1_0_32/lib/python2.5/TGCaptcha-0.11-py2.5.egg):
   DistributionNotFound(Requirement.parse('pycrypto>=2.0.1'),)}

Well I hadn't uninstalled those packages and I'm pretty sure that freebsd-update hadn't uninstalled them so where the hell have they gone! Looking in the workingenv sandbox package directory

ls -la /usr/home/m/tgenv1_0_32/lib/python2.5
4Suite_XML-1.0.2-py2.5-freebsd-6.2-RELEASE-i386.egg
Amara-1.2.0.2-py2.5.egg
BeautifulSoup-3.0.5-py2.5.egg
Cheetah-2.0.1-py2.5-freebsd-6.2-RELEASE-i386.egg
Cheetah-2.0rc8-py2.5-freebsd-6.2-RELEASE-i386.egg
CherryPy-2.2.1-py2.5.egg
...
PasteScript-1.3.6-py2.5.egg
PyProtocols-1.0a0dev_r2302-py2.5-freebsd-6.2-RELEASE-i386.egg
Routes-1.7.1-py2.5.egg
RuleDispatch-0.5a0.dev_r2306-py2.5-freebsd-6.2-RELEASE-i386.egg
SQLAlchemy-0.3.10-py2.5.egg
...
moved_aside_site.py
psycopg2-2.0.6-py2.5-freebsd-6.2-RELEASE-i386.egg
pycrypto-2.0.1-py2.5-freebsd-6.2-RELEASE-i386.egg
python_dateutil-1.3-py2.5.egg
...
setuptools.pth
simplejson-1.7.3-py2.5-freebsd-6.2-RELEASE-i386.egg
...
 

BUGGER, there are packages in there with the OS version number in that need to be updated:

easy_install -U amara
easy_install -U pycrypto
easy_install -U psycopg2
...

fixed all the problems and finally the site is up again :) So I've fixed the problem but I don't know why my other processes and the python command line worked. If anyone knows, I love to know too. Cheers.

Posted in FreeBSD, Python, TurboGears | No Comments »

Buying Photoshop is more expensive then downloading it… no really

Thursday, January 3rd, 2008

I've been using GIMP for image editing but to be honest it's just not cutting it especially when it comes to EPS files. I love GIMP and I respect all the hard work people have put in to it. I also respect all the hard work people have put in to Photoshop.

I don't need the box or manuals and I want photoshop now (I don't want to have to 'think' about the money I'm spending).. Brilliant! they have a download option. But hang on..

Picture 1
Picture 2
wtf. It's more expensive to DOWNLOAD Photoshop CS3 then it is for someone somewhere to pop photoshop in another cardboard box and post 2Kg of paper and DVD I will NEVER look at. I've been looking at the two screen grabs and I just can't believe it!

Posted in Rant, Software | 1 Comment »

Python Style Plugins Made Easy

Wednesday, January 2nd, 2008


Sometimes you need to write code that loads python at runtime. Plugin architectures are a good example of this. Plugins allow extensibility but more importantly (for me at least) they enforce a strict API. Anyway, I've written this code a few times so I thought I'd modularize it.

The specific bit of code I am going to post is the python code to look for and load a series of python plugins. Plugins (in this case) are just classes that are subclasses of the Plugin base class. This plugin base class dictates the API that all plugins must implement. Here is an example plugin abstract base class

class Plugin(object):
    def setup(self):
        """called before the plugin is asked to do anything"""
        raise NotImplementedError
 
    def teardown(self):
        """called to allow the plugin to free anything"""
        raise NotImplementedError
 
    def domagic(self):
        """do whatever it is the plugin does"""
        raise NotImplementedError
 

Here the is code to look for and return a list of classes that can be instantiated:

def find_subclasses(path, cls):
    """
    Find all subclass of cls in py files located below path
    (does look in sub directories)
 
    @param path: the path to the top level folder to walk
    @type path: str
    @param cls: the base class that all subclasses should inherit from
    @type cls: class
    @rtype: list
    @return: a list if classes that are subclasses of cls
    """
 
    subclasses=[]
 
    def look_for_subclass(modulename):
        log.debug("searching %s" % (modulename))
        module=__import__(modulename)
 
        #walk the dictionaries to get to the last one
        d=module.__dict__
        for m in modulename.split('.')[1:]:
            d=d[m].__dict__
 
        #look through this dictionary for things
        #that are subclass of Job
        #but are not Job itself
        for key, entry in d.items():
            if key == cls.__name__:
                continue
 
            try:
                if issubclass(entry, cls):
                    log.debug("Found subclass: "+key)
                    subclasses.append(entry)
            except TypeError:
                #this happens when a non-type is passed in to issubclass. We
                #don't care as it can't be a subclass of Job if it isn't a
                #type
                continue
 
    for root, dirs, files in os.walk(path):
        for name in files:
            if name.endswith(".py") and not name.startswith("__"):
                path = os.path.join(root, name)
                modulename = path.rsplit('.', 1)[0].replace('/', '.')
                look_for_subclass(modulename)
 
    return subclasses
 

and here is how you would call it:

classes = find_subclasses("./pluginsfolder/", Plugin)
 
#lets create an instance of the first class
inst = classes[0]()
 

So there you go, create folder of python files with classes in them that subclass the Plugin class and you are away.

Posted in Python | 7 Comments »