Beep My Stuff

Beep My Stuff is a web site that makes it FREE and easy to create an online library of your Books, Movies, Video Games and Music. Check it out, all feedback is much appreciated.

It's built with Turbogears, SQLAlchemy, Genshi and countless other python modules. It's all sitting on Python, FreeBSD and PostgreSQL.

Just wanted to say thanks to all the pythonista's out there that helped me get this far

#if defined(__FreeBSD__) || (defined(_WIN32) && !defined(__GNUC__)) || defined(__sun__)
/* what's this, we have no round function either? */
static double round(double num)
{
  return (num >= 0) ? floor(num + 0.5) : ceil(num - 0.5);
}
#endif
 

However 'round' is defined in FreeBSD and has been since FreeBSD 5.3 (according to the manual page). The fix is simple, just remove the 'defined(__FreeBSD__) ||' part of the '#if' and you should be fine. Now you can 'easy_install .' psycopg2.

PS: I'd tried to raise a ticket but http://www.initd.org/ trac seems to have been down for ages.

[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149] mod_wsgi (pid=1292): Exception occurred within WSGI script '/home/m/release1.0/apache/turbogears.wsgi'.
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149] Traceback (most recent call last):
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149]   File "/home/m/release1.0/apache/turbogears.wsgi", line 67, in <module>
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149]     import turbogears
[Sat Jan 19 11:32:42 2008] [error] [client 207.155.93.149] ImportError: No module named turbogears

That's odd. I've not uninstalled TurboGears and my background processes that #import TurboGears still work. Infact if I go to the python command line and type #import TurboGears it all works... bugger.

To complicate matters (in this case) I use a workingenv to contain a very specific version of TurboGears and all of it's dependencies. In order for the wsgi script to access the sandbox environment I use an excellent script which tweaks the runtime environment to include the paths in a working env. My first though is that something here had gone wrong. So I turned to prints and some basic error capture.

# Load all distributions into the working set.
from pkg_resources import working_set, Environment
 
env = Environment(root)
env.scan()
 
distributions, errors = working_set.find_plugins(env)
for dist in distributions:
    working_set.add(dist)

Printing out errors revealed:

errors:
{Amara 1.2.0.2 (/usr/home/m/tgenv1_0_32/lib/python2.5/Amara-1.2.0.2-py2.5.egg):
   DistributionNotFound(Requirement.parse('4Suite-XML>=1.0.2'),),
TGCaptcha 0.11 (/usr/home/m/tgenv1_0_32/lib/python2.5/TGCaptcha-0.11-py2.5.egg):
   DistributionNotFound(Requirement.parse('pycrypto>=2.0.1'),)}

Well I hadn't uninstalled those packages and I'm pretty sure that freebsd-update hadn't uninstalled them so where the hell have they gone! Looking in the workingenv sandbox package directory

ls -la /usr/home/m/tgenv1_0_32/lib/python2.5
4Suite_XML-1.0.2-py2.5-freebsd-6.2-RELEASE-i386.egg
Amara-1.2.0.2-py2.5.egg
BeautifulSoup-3.0.5-py2.5.egg
Cheetah-2.0.1-py2.5-freebsd-6.2-RELEASE-i386.egg
Cheetah-2.0rc8-py2.5-freebsd-6.2-RELEASE-i386.egg
CherryPy-2.2.1-py2.5.egg
...
PasteScript-1.3.6-py2.5.egg
PyProtocols-1.0a0dev_r2302-py2.5-freebsd-6.2-RELEASE-i386.egg
Routes-1.7.1-py2.5.egg
RuleDispatch-0.5a0.dev_r2306-py2.5-freebsd-6.2-RELEASE-i386.egg
SQLAlchemy-0.3.10-py2.5.egg
...
moved_aside_site.py
psycopg2-2.0.6-py2.5-freebsd-6.2-RELEASE-i386.egg
pycrypto-2.0.1-py2.5-freebsd-6.2-RELEASE-i386.egg
python_dateutil-1.3-py2.5.egg
...
setuptools.pth
simplejson-1.7.3-py2.5-freebsd-6.2-RELEASE-i386.egg
...
 

BUGGER, there are packages in there with the OS version number in that need to be updated:

easy_install -U amara
easy_install -U pycrypto
easy_install -U psycopg2
...

fixed all the problems and finally the site is up again So I've fixed the problem but I don't know why my other processes and the python command line worked. If anyone knows, I love to know too. Cheers.

I'm not having a go at the maintainers of the project. They are always extra helpful on the mailing list and to be fair if it really bothers me that much why don't I just fix it myself. I'm sad to say I can't be bothered, I've got more important things to do and I'm sure the maintainers have made the same decision.

Anyway, today I ran across a new one and couldn't find a solution via google so I'm offering up this snippet to the search engine in the sky.

After installing most of TurboGears I went to install Routes. I used easy_install but it didn't work.

 
easy_install "Routes >= 1.7"
Searching for Routes>=1.7
Reading http://pypi.python.org/simple/Routes/
Reading http://routes.groovie.org/
Best match: Routes 1.7.1
Downloading http://pypi.python.org/packages/source/R/Routes/
  Routes-1.7.1.tar.gz#md5=89cdc61ae803dc151c338ddd85551478
Processing Routes-1.7.1.tar.gz
Running Routes-1.7.1/setup.py -q bdist_egg --dist-dir
  /tmp/easy_install-6ZoHjg/Routes-1.7.1/egg-dist-tmp-GXYrHm
Traceback (most recent call last):
 
...
 
  File "/usr/local/tgmetalinfo/tgenv/lib/python2.5/
   setuptools-0.6c7-py2.5.egg/setuptools/sandbox.py", line 29, in <lambda>
    {'__file__':setup_script, '__name__':'__main__'}
  File "setup.py", line 1, in <module>
ImportError: No module named ez_setup
 

The first thing that struck me is that ez_setup is part of the setup tools package so how can it not be imported. To complicate things more I'm using workingenv which dicks about with your PYTHONPATH to create a sandbox.

After poking about on the easy_install site I found the answer in the docs. I created a file (in the current working directory) like this:

 
[easy_install]
 
# set the default location to install packages
install_dir = CWD/ENVIRONMENT FOLDER/lib/python2.5
 

where CWD and ENVIRONMENT FOLDER are replaced with your workingenv library path. Now the install works.

I think all of this has something to so with workingenv not creating a site-packages folder inside lib/python2.5 and something somewhere expecting it. Anyway, it all works for me now. Hope this helps.

PS: I know working env is deprecated in favour of virtualenv but I ran in to the same problem with virtualenv

I'll admit it's an expensive way for me to be lazy. I could test regular expression in the python command line. I'm just to lazy Anyway, to get to the point. There is a good google tech talk by one of the authors of VMWare Fusion. He goes in to the ins and outs of OSX development touching on the 'Apple creates API's for it's own needs' which I think most OSX developers know. For some reason embedding is disabled so here is the link.

Well one of the engineers from Sun that gave birth to DTrace has done a google tech talk about DTrace and it's very interesting indeed. Highlights for me were the amazing instrumentation of Python which I will definitely use. He also muses on the lack of support for DTrace on linux and tries to gather a posse of developers from google to kick it off, porting DTrace to Windows and how he can't find a good book on programming.

--- Makefile.orig       Fri Jan  5 02:35:36 2007
+++ Makefile    Fri Jan  5 02:47:45 2007
@@ -40,7 +40,8 @@
                MYSQL           "Enable MYSQL support" off \
                OPENLDAP        "Enable LDAP support" off \
                OPENSSL         "Enable SSL support" on \
-               VALGRIND        "Enable valgrind support" off
+               VALGRIND        "Enable valgrind support" off \
+               WEBDAV          "Enable WebDAV support" off

 .include <bsd.port.pre.mk>

@@ -107,6 +108,12 @@
 RUN_DEPENDS+=          valgrind:${PORTSDIR}/devel/valgrind
 CONFIGURE_ARGS+=       --with-valgrind
 .endif
+
+.if defined(WITH_WEBDAV)
+LIB_DEPENDS+=           libuuid.1:${PORTSDIR}/misc/e2fsprogs-libuuid
+CONFIGURE_ARGS+=        --with-webdav-props --with-webdav-locks
+.endif
+

 SUB_LIST+=             REQUIRE="${_REQUIRE}"

run

make config install clean

select WebDAV and sit back and watch. There is one glitch though. If you already have e2fsprogs-libuuid installed it may fail (I'm pretty sure I haven't patched the file correctly). Just make uninstall e2fsprogs-libuuid and run make install on lighttp again and it should all work. Now you've got lighttp with WebDAV on freebsd.

I've emailed the port maintainer to offer the patch. Ill update this if something changes.

Update: changed the diff format to unified

Edit syslog.conf

I'm using bonobo.local as my syslog server. Start by editting /etc/syslog.conf. At the first line that isn't a comment (first line that doesn't start with a #) insert +<servername><return> so for me I added +bonobo.local

#$FreeBSD: src/etc/syslog.conf
+bonobo.local
*.err;kern.debug;auth.notice;mail.crit          /dev/console
*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err   /var/log/messages
security.*                                      /var/log/security
auth.info;authpriv.info                         /var/log/auth.log
mail.info                                       /var/log/maillog
... <snip> ...
!ppp
*.*                                             /var/log/ppp.log

then at the bottom of the file add the name of the client machine, for me this was hypnotoad.local:

!*
+hypnotoad.local
*.*                                             /var/log/hypnotoad.log

there are some funny characters in here that complicate matters.

!* resets the target. A few lines up the default syslog.conf has the line !ppp this tells the syslog daemon that all the following rules only apply to ppp messages. Putting !* tells the syslog daemon that all the following rules match all messages.

+hypnotoad.local tells the syslog daemon that the following rules relate to the machine mentioned.

*.* <tab> <filename> tells the syslog daemon to log anything from the afore mentioned machine to the logfile 'filename'

For security reasons the syslog daemon won't start appending (logging) to a file that DOESN'T exist when it starts so you must create the files before restarting the syslog daemon.

sudo touch /etc/log/hypnotoad.log

Edit /etc/rc.conf

edit /etc/rc.conf and append the line

syslogd_flags=""

This looks odd. Why deliberately pass in empty flags? Because /etc/defaults/rc.conf has

syslogd_flags="-s"

The -s option stops syslog from listening for external logging messages. This caught me out first time I tried to do this

Now restart syslog

sudo /etc/rc.d/syslogd restart

you should see a few lines about syslog stopping and starting again.

Setting up the clients

On each client make a backup of the original /etc/syslog.conf file and edit the original to include a single line

#$FreeBSD: src/etc/syslog.conf
*.*     @bonobo.local

replacing bonobo.local with the name / ip address of your logging server. restart your syslog server:

sudo /etc/rc.d/syslogd restart

try out the logging with

logger "hello from my syslog client"

the message "hello from my syslog client" should appear in the syslog servers log file for the machine you are changing.

It's missing the per port break down and the connection list that IPTraf has. But to be honest all I really want is ascii graphs . It is easier to use in my opinion because it just does what I want.. draws a cool graph and lets me know something is happening when I worried that I fucked something up again .

And you though this was going to be a late night rant about a fictional soft drink