Friday, February 22, 2008

Dell Vostro 400 - Ubuntu/Kubuntu Linux

I was able to get a Dell Vostro 400 to work with Kubuntu 8.4 Alpha 4 LiveCD. I used the installer and it resized the Windows XP partition just fine... What I did to get this to work is written below:

Okay, SATA cdroms and Linux don't seem to be mixing very well... at least not with Ubuntu/Kubuntu 7.10. If you pop in one of the live cds, it will likely give you an IO Error when you select any item on the live cd boot menu.

As it turns out, that problem is fixed in Ubuntu/Kubuntu 8.4... but it's not out yet being Feb 22nd (Release date scheduled for April). But, the live cd works with 8.4 Alpha 4 (looks like Alpha 5 came out today), which can be found here: https://wiki.kubuntu.org/HardyHeron/.

When I booted the live cd, the screen went blank... The next time I booted off the live cd, I choose F6, and removed "quiet splash" from the kernel arguments.

At some point you might run into an error like:
Feb 22 21:55:19 localhost ata2: failed to recover some devices, retrying in 5 secs
If this happens add the kernel argument irqpoll and it should bypass/fix the problem (not sure which; it just worked).

To add the irqpoll option in the live cd:
  1. Select F6
  2. Remove "quite splash"
  3. Add "irqpoll"
To add the irqpoll option to grub (once installed):
  1. Replace # defoptions=quiet splash with # defoptions=irqpoll
I am in a bit of a hurry during this post today, so if you have any questions or would like me to expand on something, just post a comment.

Labels: , , ,

Thursday, January 31, 2008

Django App + mod_fcgid + Apache 2 Setup on Mac OS X & Debian/Ubuntu Linux

Update 2008Feb11:

From rather obscure comment from 'apt-cache show python-flup' on a Debian system, I discovered that flup has been superseded by http://www.modwsgi.org/. It is probably worth checking out mod_wsgi instead of mod_fcgid for Python web applications. Based on the documentation for mod_wsgi integration with Django, I am planning on switching. I will post my results in a future post.

Required Software

mod_fcgid

Download Source and Install

For reference, here is the mod_fcgid INSTALL.txt... see my Mac OS X notes below it for changes required to get it working on Mac OS X 10.5 (Leopard):

NOTE: This module is for Apache2 ONLY

UNIX
It's tested on my RedHat8 and Solaris. But it should work on other *NIX platform.
NOTE: This module MUST run on share memory supported system

1. If your Apache2 installation isn't in /usr/local/apache2, please edit Makefile and correct it
2. cd $mod_fcgid_dir
3. make //in Mac you need Xcode tools, optional install on Install CD
4. make install
5. add the following line in httpd.conf

LoadModule fcgid_module modules/mod_fcgid.so

MAC OS X Notes

  • In Mac OS X Leopard, the Makefile needs to be changed from /usr/local/apache2 to /usr/share/httpd, then steps 3 and 4 above should work.
  • Step 4 requires: sudo make install
  • Step 5, you need to add the following to /etc/apache2/httpd.conf: LoadModule fcgid_module libexec/apache2/mod_fcgid.so
  • IMPORTANT: apache 2 on Mac OS X Leopard is 64-bit and by default, the make file does not build the 64-bit module. To do this, you need to add the following to the Makefile after the EXTRA_CFLAGS option (too far above that will cause it not to work apparently):
CFLAGS = -arch ppc -arch ppc64 -arch i386 -arch x86_64

The above will make a universal binary for all Mac OS X.

If at some point, you get this message either in the error_log or by running 'apachectl configtest'

httpd: Syntax error on line 117 of /private/etc/apache2/httpd.conf: Cannot load
/usr/libexec/apache2/mod_fcgid.so into server:
dlopen(/usr/libexec/apache2/mod_fcgid.so, 10): no suitable image found.
Did find: /usr/libexec/apache2/mod_fcgid.so: mach-o, but wrong architecture

Then your mod_fcgid.so module probably was not built with x86_64 architecture... To check, type

file /usr/libexec/apache2/mod_fcgid.so

And it should list the architectures that the mod_fcgid.so was built for.

Enable mod_fcgid

Linux

At least with Debian/Ubuntu you can enable fcgid by typing (as root):

a2enmod fcgid

Mac OS X Leopard (10.5)

Add the following to /etc/apache2/httpd.conf:


AddHandler fcgid-script .fcgi
SocketPath /tmp/fcgid_sock
SharememPath /tmp/fcgid_shm
IPCConnectTimeout 20

mod_rewrite

We need to tell apache for what urls it should pass off the request to the fcgi script. This is accomplished with mod_rewrite which is normally turned on by default w/ apache 2 (at least the installations I have seen).

We'll start with the full setup required for gaworkflow.frontend, but test it with a "Hello World" style dispatch.fcgi script.

gaworkflow.frontend apache2 mod_rewrite setup

Linux

Add the following to your apache2 /etc/apache2/sites-available/default or other vhost specific file:

# Enable access to the django admin media
Alias /media /usr/lib/python2.5/site-packages/django/contrib/admin/media

# Turn on the rewrite engine
RewriteEngine on

# Enable http:///admin/ and related urls
RewriteRule ^/admin(.*)$ /usr/lib/cgi-bin/dispatch.fcgi/admin$1 [QSA,L]
RewriteRule ^/logout(.*)$ /usr/lib/cgi-bin/dispatch.fcgi/logout$1 [QSA,L]
RewriteRule ^/login(.*)$ /usr/lib/cgi-bin/dispatch.fcgi/login$1 [QSA,L]

Mac OS X Leopard

Add the following to your apache2 /etc/httpd.conf:

# Enable access to the django admin media
# NOTE: the following did not work for me... I had to copy the media directory to
# /Library/WebServer/Documents/media
#Alias /media /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/contrib/admin/media/

# Turn on the rewrite engine
RewriteEngine on

# Enable http:///admin urls
RewriteRule ^/eland_config(.*)$ /Library/WebServer/CGI-Executables/dispatch.fcgi/eland_config$1 [QSA,L]
RewriteRule ^/admin(.*)$ /Library/WebServer/CGI-Executables/dispatch.fcgi/admin$1 [QSA,L]
RewriteRule ^/logout(.*)$ /Library/WebServer/CGI-Executables/dispatch.fcgi/logout$1 [QSA,L]
RewriteRule ^/login(.*)$ /Library/WebServer/CGI-Executables/dispatch.fcgi/login$1 [QSA,L]

Restart Apache 2

Linux (Debian/Ubuntu)

sudo /etc/init.d/apache2 restart

Mac OS X Leopard

sudo apachectl restart

dispatch.fcgi

Hello World Test

Now to test that we have the mod_fcgid setup properly, create a file called dispatch.fcgi:

#!/usr/bin/python

def myapp(environ, start_response):
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['Hello World!\n']

if __name__ == '__main__':
from flup.server.fcgi import WSGIServer
WSGIServer(myapp).run()

And move it to your cgi-bin directory:

Linux cgi-bin

sudo mv dispatch.fcgi /usr/lib/cgi-bin/

Mac OS X Leopard cgi-bin

sudo mv dispatch.fcgi /Library/WebServer/CGI-Executables/

Linux/Mac OS X File Permissions

sudo chmod a+x /dispatch.fcgi

Mac OS X 10.5 users will need to update the Options for to include +ExecCGI... by default it is set to "Options None". The updated entry should look like:


AllowOverride None
Options +ExecCGI
Order allow,deny
Allow from all

WARNING: You will get a message saying "Forbidden" in the web browser if you do not update the directive above.

Web Browser Test

Point your web browser to http://localhost/admin/ and you should see if it prints "Hello World!". If you see "Hello World!" continue to the next section.

Installing the Django App dispatch.fcgi script

copy of mydjangoapp code

Make a copy of the mydjangoapp code and database and put it in a location that will be the "live" version of the code/database. On Mac OS X, I choose /Library/WebServer/mydjangoapp. Initialize the database like your normally would, but you will need to update the settings.py module so that DATABASE_NAME is an absolute path, otherwise you will get errors in the apache 2 error_log saying that the python code could not connect to the database. Assuming you also choose /Library/WebServer/mydjangoapp, your DATABASE_NAME variable in settings.py should be:

DATABASE_NAME = '/Library/WebServer/mydjangoapp/mydjangoapp.db'

You will also need to update the settings.py TEMPLATE_DIRS to be an absolute path (relative paths don't work for some reason) of "/Library/WebServer/gaworkflow/templates"... so the update version should look like:

TEMPLATE_DIRS = (
"/Library/WebServer/mydjangoapp/templates",
)

Also, you will need to make a link to the admin templates in our /Library/WebServer/mydjangoapp/templates directory by typing:

Mac OS X 10.5:

ln -s /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/contrib/admin/templates/admin /Library/WebServer/mydjangoapp/templates/admin
ln -s /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/contrib/admin/templates/admin_doc /Library/WebServer/mydjangoapp/templates/admin_doc
ln -s /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/contrib/admin/templates/widget /Library/WebServer/mydjangoapp/templates/widget
ln -s /System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/django/contrib/admin/templates/registration /Library/WebServer/mydjangoapp/templates/registration

Warning: The following directions will tell you how to get past the db read/access errors but may not be the best choice for security... you consider the possible security issues before following the following instructions.

To get the dispatch.fcgi to work properly, I needed to change ownership of /Library/WebServer/mydjangoapp to be owned by the apache2 user, which is _www on Mac OS X 10.5 and www-data on Debian/Ubuntu systems. The the apache user also needs access to the mydjangoapp.db as well. I ran the following commands on Mac OS X 10.5 when placing the files in /Library/WebServer/gaworkflow:

sudo chown _www:_www /Library/WebServer/mydjangoapp
sudo chown _www:_www /Library/WebServer/mydjangoapp/mydjangoapp.db
sudo chmod o-rwx /Library/WebServer/mydjangoapp/mydjangoapp.db
sudo chmod ug+rwx /Library/WebServer/mydjangoapp/mydjangoapp.db

new dispatch.fcgi ΒΆ

Replace the hello world dispatch.fcgi with the following script:

#!/usr/bin/python                                                            
import sys
sys.path += ['/Library/WebServer/mydjangoapp']
from flup.server.fcgi import WSGIServer
from django.core.handlers.wsgi import WSGIHandler
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'mydjangoapp.settings'
WSGIServer(WSGIHandler()).run()

You will need to update these two rows:

  • sys.path += ['/Library/WebServer/mydjangoapp']
  • os.environ['DJANGO_SETTINGS_MODULE'] = 'mydjangoapp.settings'

Where '/Library/WebServer/gaworkflow should be replaced by PYTHONPATH that would allow your Django package to be imported... one directory level below the directory containing init.py.

NOTE: Make sure you dispatch.fcgi is executable:

sudo chmod a+x dispatch.fcgi

If everything worked out properly, you should have a working installation of gaworkflow.frontend using mod_fcgid.

Trouble Shooting

  • Check apache error logs.
  • run: sudo apachectl configtest
  • Check that dispatch.fcgi has #!/usr/bin/python and not #!/usr/bin/env python as that will screw up the environment variables at least w/ Apache 2 that ships on Mac OS 10.5.
  • run 'python dispatch.fcgi' to see if you get any standard Python errors.

Apache2 - error_log -- [warn] mod_fcgid: stderr: OperationalError: unable to open database file

This means the database is not accessible by the user that is running apache. On Mac OS X 10.5, the user is _www. On Debian/Ubuntu Linux the user is www-data. See the "copy of gaworkflow code" section above for directions on changing file permissions for the database.

Labels: , , , , , , , , ,

Tuesday, September 18, 2007

bio.scipy.org

As I said I would do, I am now in the progress of setting up and seeding the biology in python community site, which can be found at bio.scipy.org. Part of the goal, as set at the SciPy 2007 Biology Birds of a Feather (BoF) meeting (organized by Titus Brown), is to establish a community site where anyone using Python in the field of Biology can come together, communicate, share ideas, and share code.

As part of the programming guide, I posted my "Preparing to learn python guide", in which, I mention the things about Python I wish I had known when I started learning Python. I am guessing it will be most useful to those who have programmed before in another language and are trying to learn Python.

Also, I posted a link to Software Carpentry in the programming guide. Thanks Greg for starting that project!

Labels: , , ,

Tuesday, August 28, 2007

Evil Spammers

Those evil spammers figured out I had misconfigured my postfix smtp configuration before I did and they started using my server to forward spam! The configuration has now been fixed, but I wanted to point out a useful link on setting up postfix smtp authentication using sasl:

How to force SMTP authentication in postfix by Kevin Bailey

Labels: , , ,

Friday, August 17, 2007

SciPy 2007 - Biology Birds of a Feather meeting

During the Biology birds of a feather meeting at SciPy 2007, which was organized by Titus Brown, we meet to discuss were those of us using Python for biology/bioinformatics should work together as a community. Two interesting thoughts came up:
  1. Need to establish python/biology community, via website, biology-in-python mailing list, rss, blogs, etc.
  2. Having a core set of "interfaces" for handling basic bioinformatics objects would allow independent projects to share these basic objects. I am sure others will describe this better and in more detail in the near future.
I personally believe that even if we only accomplish the first item, it would significantly help the bioinformatics community. I do hope the 2nd is accomplished as well.

What I see is that most people who are new to the python bioinformatics community end up finding BioPython. The only problem is that many people doing bioinformatics with Python who choose not to use BioPython end up doing there own thing an not communicating with the community. The idea here is to make a new python/biology community site were we can communicate, share code, share ideas, etc. In other words support the community as a whole rather than just having communities for individual projects.

I agreed to setup this initial site and seed it information, bioinformatics python packages by category, a place holder for cookbooks, advice for testing and software engineering (referencing Greg Wilsion's Software Carpentry material, etc.). I would like to thank Enthought as they have agreed to host the website. I will add the more information on the community site as it is being setup. We'll see how it goes.

Also, check out Titus Brown's Birds of a Feather post.

Labels: , , ,

Wednesday, June 13, 2007

Dell Inspiron 9400 Ubuntu 7.04 (Fiesty) X won't start & bcm43xx_microcode5.fw missing!

If you have Dell Inspiron 9400 and are having trouble with Ubuntu or Kubuntu 7.04 (Fiesty) and you are getting the following errors or behavior, then you have come to the right place (Note: End of post has details of the hardware in this Dell Inspiron 9400):

bcm43xx: Error: Microcode "bcm43xx_microcode5.fw" not available or load failed

and/or

Failed to start the X server:
Fatal server error: no screens found


Failed to start the X server problem:
Even if you manage to get the Ubuntu installed without getting X to start from the Ubuntu live cd, you will still need to fix the problem once it is installed. Here is how to you can get the Ubuntu live cd to work (assuming you have the ATI X1400 graphics card):
  1. Boot from the Ubuntu 7.04 live CD and wait for the message saying X could not start. Get through the error messages until it drops you into a bash shell.
  2. Use your favorite text editor (or nano if you can't find yours) to uncomment the universe line from /etc/apt/sources.list: sudo nano /etc/apt/sources.list
  3. sudo apt-get update
  4. sudo apt-get install xorg-driver-fglrx #Install the ati 3d accelerated driver
  5. sudo aticonfig --initial #creates a new xorg.conf setup for ati cards
  6. sudo modprobe fglrx #loads the ati fglrx driver
  7. sudo /etc/init.d/gdm restart #(for ubuntu) or sudo /etc/init.d/kdm restart #(for kubuntu)
  8. X should now start. Once you install you can repeat the same steps to get it working for the installed copy.
bcm43xx: Error: Microcode "bcm43xx_microcode5.fw" not available or load failed problem:
This is the error with the wireless card driver which requires firmware that is not installed by default. There is an easy way to install the firmware. Just install the bcm43xx-fwcutter package. It will ask you if you want it to automatically download install the firmware you need. Just say yes and it will do all the work.
  1. Follow step 2 from the previous section.
  2. sudo apt-get install bcm43xx-fwcutter #Make sure to tell it to download and install the firmware.

This Dell Inspiron 9400 Specs:
  • Intel Core 2 Duo T7200
  • 17 in UltraSharp TrueLife Wide-screen WUXGA (1920x1200 max resolution)
  • 2GB DRR2 533MHz RAM
  • 256MB Ati Mobility Radeon X1400 graphics card
  • 120 GB 5400RPM SATA HD
  • Integrated 10/100 Network Card (Broadcom BCM4401-B0 100Base-TX (from lspci))
  • Dell Wireless 1390 802.11g Mini-card (Broadcom Dell Wireless 1390 WLAN (from lspci))
  • 8X DVD+/-RW Drive
  • Integrated High Def. Audio (Intel 82801G High Def. Audio Controller (from lspci); works great by the way.


Labels: , , , , ,

Friday, May 25, 2007

** (beryl-manager:6606): CRITICAL **: can't execute beryl-xgl: Success

If you get the following error when trying to launch beryl-manager with KDE on Kbuntu 7.04 (Feisty) (likely with an ATI card from what I am reading):
** (beryl-manager:6606): CRITICAL **: can't execute beryl-xgl: Success
The problem most likely has to do with this bug:
https://bugs.launchpad.net/ubuntu/+source/beryl-core/+bug/95394
Which basically states that beryl-xgl was removed from beryl-core package due to how it is built.

Labels: