LuckyDonkey

I upgraded a test server to FreeBSD 6.3 (released a few days ago) and all was working well apart from my TurboGears app. I run a TurboGears instance behind mod_wsgi and it wouldn't start. Here is the error I got in http_errors.log

[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149] mod_wsgi (pid=1292): Exception occurred within WSGI script '/home/m/release1.0/apache/turbogears.wsgi'.
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149] Traceback (most recent call last):
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149]   File "/home/m/release1.0/apache/turbogears.wsgi", line 67, in <module>
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149]     import turbogears
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149] ImportError: No module named turbogears

That's odd. I've not uninstalled TurboGears and my background processes that #import TurboGears still work. Infact if I go to the python command line and type #import TurboGears it all works... bugger.

To complicate matters (in this case) I use a workingenv to contain a very specific version of TurboGears and all of it's dependencies. In order for the wsgi script to access the sandbox environment I use an excellent script which tweaks the runtime environment to include the paths in a working env. My first though is that something here had gone wrong. So I turned to prints and some basic error capture.

# Load all distributions into the working set.
from pkg_resources import working_set, Environment
 
env = Environment(root)
env.scan()
 
distributions, errors = working_set.find_plugins(env)
for dist in distributions:
    working_set.add(dist)

Printing out errors revealed:

errors:
{Amara 1.2.0.2 (/usr/home/m/tgenv1_0_32/lib/python2.5/Amara-1.2.0.2-py2.5.egg):
   DistributionNotFound(Requirement.parse('4Suite-XML>=1.0.2'),),
TGCaptcha 0.11 (/usr/home/m/tgenv1_0_32/lib/python2.5/TGCaptcha-0.11-py2.5.egg):
   DistributionNotFound(Requirement.parse('pycrypto>=2.0.1'),)}

Well I hadn't uninstalled those packages and I'm pretty sure that freebsd-update hadn't uninstalled them so where the hell have they gone! Looking in the workingenv sandbox package directory

ls -la /usr/home/m/tgenv1_0_32/lib/python2.5
4Suite_XML-1.0.2-py2.5-freebsd-6.2-RELEASE-i386.egg
Amara-1.2.0.2-py2.5.egg
BeautifulSoup-3.0.5-py2.5.egg
Cheetah-2.0.1-py2.5-freebsd-6.2-RELEASE-i386.egg
Cheetah-2.0rc8-py2.5-freebsd-6.2-RELEASE-i386.egg
CherryPy-2.2.1-py2.5.egg
...
PasteScript-1.3.6-py2.5.egg
PyProtocols-1.0a0dev_r2302-py2.5-freebsd-6.2-RELEASE-i386.egg
Routes-1.7.1-py2.5.egg
RuleDispatch-0.5a0.dev_r2306-py2.5-freebsd-6.2-RELEASE-i386.egg
SQLAlchemy-0.3.10-py2.5.egg
...
moved_aside_site.py
psycopg2-2.0.6-py2.5-freebsd-6.2-RELEASE-i386.egg
pycrypto-2.0.1-py2.5-freebsd-6.2-RELEASE-i386.egg
python_dateutil-1.3-py2.5.egg
...
setuptools.pth
simplejson-1.7.3-py2.5-freebsd-6.2-RELEASE-i386.egg
...
 

BUGGER, there are packages in there with the OS version number in that need to be updated:

easy_install -U amara
easy_install -U pycrypto
easy_install -U psycopg2
...

fixed all the problems and finally the site is up again So I've fixed the problem but I don't know why my other processes and the python command line worked. If anyone knows, I love to know too. Cheers.

I've been doing a lot of TurboGears work recently. TurboGears is an excellent framework and Python is my favorite language so I can't complain. Well of course I can!

One of the great things about python is easy_install which (as it's name suggests) installs things really easily. Want to install postgres drivers for python? Well just type

easy_install psycopg2

What could be easier then that! Well the problem is that you end up with a drive littered with old versions of modules. You'd really like to clean them up but your not sure if they are need, who put them there and sometimes what the module even is.

Step forward WorkingEnv.

WorkingEnv creates a directory structure and a source shell file for a little mini setuptools environment.

dazza@macpro:~/>workingenv tgenv
dazza@macpro:~/>source tgenv/bin/activate
(tgenv)dazza@macpro:~/>easy_install TurboGears
...

There you go... TurboGears installs, you're not shitting all over your drive and if it all goes tits up you can delete it all and start again. Note the way my shell prompt has changed to tell me I'm living the dream No more complaining!

You can also setup your private working environment to match any setup an ISP may have as well which is a must for easy pain free deployment.

Anyway, I wish I'd know about this a few years ago. Enjoy.

update
cavorite added two excellent links to his reddit comment about this article that I thought I'd share here:

• Cheeseshop page
• Authors (Ian Bicking's) Page

Thanks cavorite