More on mod_wsgi issues

When you add wsgi to Apache it is so you can run Python on Apache. Thus you need, at a minimum, Python and Apache.

Python

If you use a utility such as virtualenv to compartmentalize several domain development environments on one server account, installing mod_wsgi may not and probably will not be associated with the correct Python library. Will not, that is, unless you explicitly point it out in the installation process. Be sure to plan in advance which Python install you are developing for, and be sure to flag it when installing mod_wsgi:

./configure --with-python=/home/django/domains/mydomain.com/bin/python2.5* ***

*** UPDATE (1/31/10): Please read the helpful comment and link from Graham Dumpleton, author of mod_wsgi, below. Per his direction, it is best to install mod_wsgi against the complete base Python installation and add references to the virtual environments from your Apache config or from the WSGI script file. Thanks for taking the time to clarify, and for making mod_wsgi available for general use.

That said, there are many ways to install mod_wsgi, from manual to managed by aptitude, get-apt or the like. The downside to using a managed install is the dependency on the vintage of their managed version. At last check, the aptitude repository of mod_wsgi is still at 1.3 while the current stable version on googlecode is 2.8. Thus, you may want to look into another method, perhaps a manual installation.

Apache

When manually installing mod_wsgi, be sure you have the appropriate development library for apache, in the case of ubuntu hardy heron, apache2_threaded_dev will allow you to install mod_wsgi. Furthermore, if you are using aptitude to install apache2_threaded_dev, go ahead and update the distro as a whole first by running sudo aptitude update. If you do not, you may find the install doesn't work, and there will be no indication as to what went wrong.

Finally, with these measures in place, try ./configure with the appropriate flags and see if you can get the correct library in place. Good luck.

*your location will be different from this example

MAMP with mod_wsgi

Once you get a database set up, sqlite or otherwise, you can use it with Django applications. You can serve Django directly from a command line or through Apache. In either case you will need Django to be installed. From a command line, simply run the django server and browse to it:

django-admin.py runserver

If your production server is going to be Apache, your best bet is to run Django under Apache locally. To do this you need a helper library for Apache to use. The de facto solution is mod_wsgi.

mod_wsgi

To get started with mod_wsgi, typically you download mod_wsgi from Graham Dumpleton's Google Code repository, you go to your command line, you untar it, you configure it, you make it and you make install it. This makes the code library available to wsgi-enabled services, in effect allowing you to run Django on Apache.

When you make install mod_wsgi, it places the libraries in your /lib directory. Then it links to your Apache directory where OSX keeps it, /usr/libexec/apache2. Installation includes automatic detection, or you can use some optional flags to specify resource paths, detailed in the wsgi installation guidelines.

Now your OSX Apache has modwsgi installed, but the Apache version in MAMP does not. To verify this, try making a Django app and serving it via MAMP. First, make a new django app in an appropriate directory:

django-admin.py startproject myproject

Next, open and modify the config for your MAMP Apache app, located in /Applications/Mamp/conf/apache/httpd.conf. For that matter, have a look at what your OSX Apache instance has in its config file. It is private/etc/apache2/httpd.conf. Searching for wsgi in this file yields:

LoadModule wsgi_module libexec/apache2/mod_wsgi.so

Looks good even if the relative directory location may be an issue. Insert this info into the MAMP httpd.conf and test it. Actually, there is no need because it will not work. This is due to the structure of MAMP and its proprietary use of Apache. Unlike a full install of Apache, the folders in MAMP have a different structure and are optimized to run modularly. Consequently, you cannot append a flag to your configure command in this manner:

./configure --with-apxs=/usr/sbin/apxs2-worker

because the structure available to Apache is not in MAMP, so detection will fail.

This leaves a couple of options. Either reconstruct some of the MAMP Apache folders so detection will succeed or look for another solution. Considering the evident lack of general success from many Google searches, I am letting this fish off the hook. Feel free to leave a comment if you know a way to make it work.

As a final consideration, mod_python appears to be a lot quicker to set up with MAMP. That could be a candidate if you are in a pinch. Otherwise, if serving Python via Apache is your goal, I recommend using the resources included with OS X and skipping MAMP altogether.

Magento and PHP5 Install Memory Error

If, when installing Magento Ecommerce, you experience some variant of this message:

Fatal error: Allowed memory size of 16777216 bytes exhausted (tried to allocate 24 bytes) in /home/userdir/sites/mywebsite.com/public_html/magento/lib/Zend/Db/Statement/Pdo.php on line 232

The solution is simple. Just go to your

/etc/php5/apache2/php.ini*

and set memory higher than the default of 16M. The error trace indicates a minimum requirement of 24M for successful installation.‡

But you knew that already, right?

^{* php.ini may be in a different directory
‡ full disclosure: I allotted 64MB since the slice I'm using has 512MB available.}

Apache DocumentRoot in Python and Django

Be very deliberate in your overall architecture when setting up new django projects. The directory dependencies resulting from an apache-nginx-python-django server configuration can make your path dependencies very complicated.

Generally put, being indiscriminate in your setup now will result in extreme challenges to resource deployment later on.

Here are some working methods that can greatly complicate your setup:

• Hosting several modules on one URL
• Having only one virtual host for many projects
  
• Hosting several modules on one server account
• Setting Python gateway modules off the root of your URL
• Using Python server.py-type gateways in addition to ordinary http gateways

Here are some ways to simplify your paths:

• Set up several virtual hosts
  
• Host only one Django project per URL (or as few as you can manage)
  
• Clearly structure your resources to discourage any overlap on your VPS or dedicated server
• Consider the trade-offs between a complex rig of sites and resources and several simple unrelated server environments. There's really no need to attempt the most complex possible solution unless that's your bag.

The nice thing about Python and Django is that they can handle either direction. There's a way to work everything out however you prefer, and the choice is yours to make.

That said, remember also that it is nice when projects just work without a lot of complex troubleshooting or traffic control.

Here is an example of a moderately complex configuration with a mixed bag of pros and cons.

Simple Endpoint Test for Asynchronous Unit and Integration Processes

In the creation of open-source RIAs, it can be very difficult to ensure connectivity of endpoints. There are potential snags and edge cases surrounding the following critical points:

• Security clearance of port
• Availability of port resource
• Conflicts between multiple running servers
  
• Properly-declared URI endpoints

As in point three, matters can be complicated when proxy servers, such as lighttpd or nginx are working in conjunction with apache. Most likely, both are open on different IP registers on port 80; apache proxy on localhost, and the lightweight server on the public IP.

An RIA will asynchronously connect via a RemoteObject call on another port. It requires a valid connection. Utilities like netcat can save you from a quagmire of ineffectual random testing.

First, open the port in apache, as it will be the ORM-enabled server. Typically, the port is 8000 or 8080 for arbitrary reasons of convention and familiarity. Make sure your 8000 or 8080 is not tied up with memchache or some other port-requisite ancillary utility. Open up ports.conf and add the port in the configuration script.

TEST THE PORT

Restart the apache server. Using a port utility such as netcat, test port connectivity. One quick way is to use netcat from a command line. Netcat will attempt to connect to any port(s) you specify and report the protocol it encountered:

nc -v -w 2 -z target 20-30

FIREWALLS

Don't forget to modify your security settings to allow the port! Specifically, if you have set up a firewall, iptables requires an entry allowing public connections over the port in question or connections will fail. If they have been changed, be sure to restart the apache server and that the rules have been saved either via command-line or the initializing script, usually /etc/network/interfaces.

Be sure your apache server's NameVirtualHost settings allow the wildcard * and not only localhost connections. This way, requests from your http-alt port will bypass your proxy server altogether. Apache might warn you thus, but it can be ignored:

[Thu Sep 03 16:20:42 2009] [warn] NameVirtualHost *:80 has no VirtualHosts

With these potential obstructions addressed, you should be able to tunnel your way to fame and fortune.

NEXT UP

How to put that port to use in an RIA RemoteObject call.

Setting up Post-Hooks for Subversion

The publicly available subversion book at svnbook.red-bean.com is both concise and comprehensive. It can help you deftly steer a subversion server in any way your inner administrator hankers to go.

I found subversion helpful in tracking versions. I found it less useful in quick commit-to-staging server process. As some bountiful moments in freenet #svn IRC led me to discover, using post-hook scripts is the way to go.

Check this info box on tigris.subversion.org, leading to this part of ch 5 of the svn book on using hooks to automate and extend commands. There are also some helpful examples of scripts that can be brought to bear on a project.

The code is built thus and left in the hook directory sans .tmpl extension:

REPOS="$1"
REV="$2"

# commit-email.pl "$REPOS" "$REV" commit-watchers@example.org
# log-commit.py --repository "$REPOS" --revision "$REV"

svn update /home/[username]/django/projects/mysite.com/projectdir >> /var/log/vhosts/svn-django-errorlog

Be sure to set your post-commit file to executable

chmod +x post-commit

Django AMF Service

Now that the basic Django framework is in place (admin, views, urls), the focus shifts to data and services. In the past, WebORB has been adept at providing RPC between ASP and RIA, so it's tempting to use it again in a Linux environment.

Since the current project is Apache, one option is to utilize Apache Tomcat. Tomcat is readily available in MediaTemple's (dv) configuration to provide a pure Java HTTP server environment. This would pair well with WebORB for Java, a robust middleware solution for RIA data synchronization.

Good enough but the tiers don't fully align with the planets yet. My goal is to utilize remote procedure calls to mySQL and take advantage of the schema I've built in Python-based Django. That will not be possible until my Python objects can be called from Java. So, is it possible to port Python code to WebORB via a Python -> Java compiler?

In a word, Jython.

Jython runs in Tomcat as an implementation of the Python programming language. By compiling the Django code, Jython could route between Flex on the front end and the mySQL admin system within the framework of Django.

Incidentally, I'm not going to implement this cookbook recipe just yet. I've worked with WebORB and am sold on it's strengths, but in this instance I'm looking for a lighter and more agnostic solution for the frontend than a uniform rich presentation server.

Namely, I'm going to move forward with Django AMF PyAMF. This allows some of Django's front-end capabilities to shine, and also enables javascript and ajax elements to be plugged into the site design.

More posts to come on this direction.

Out of curiosity, has anyone else tried the setup I outlined above, i.e. Django <-> Java <-> RPC tier <-> RIA? Furthermore, would you go a completely different route or change any of the tiers from the architecture I outlined? It is one of countless possibilities, and on that note, in my experience, one of the most direct and stable available.

Last (?) post about MediaTemple (dv) and Django

This is a refinement of a previous post on the topic of Django integration into a MediaTemple (dv) account. Following a talk with MT and reading some knowledgebase info, here is the quick route to Django configuration happiness - now more better.

This is how we do it:

Add to base config aka /etc/httpd/conf/httpd.conf the following in the appropriate section:

LoadModule wsgi_module modules/mod_wsgi.so

And in the appropriate section, add this too:

NameVirtualHost *:80

#Add a VirtualHost Section for each Name-based virtual server

<VirtualHost>
ServerAdmin email@email.com
ServerName www.URL.com
ServerAlias URL.com *.URL.com

DocumentRoot /var/www/vhosts/URL/httpdocs

<Directory /var/www/vhosts/URL/httpdocs>
Order allow,deny
Allow from all
</Directory>

</VirtualHost>

then save the file and exit the editor.

In the directory of path '/var/www/vhosts/(DOMAIN NAME)/conf' there is a file called 'httpd.include.' It holds unique config info for each virtual domain. It is rebuilt automatically from time to time based on Plesk. Instead of editing that file, make a new one called 'vhost.conf' (if it doesn't already exist) and add the following info to this file:

WSGIDaemonProcess URL.com processes=1 threads=1 display-name=%{GROUP}
WSGIProcessGroup URL.com
WSGIScriptAlias /myapp /home/username/django/scripts/URL.com.wsgi

<Directory /home/username/django/scripts>
Order allow,deny
Allow from all
</Directory>

ErrorLog /var/log/URL.com-error_log
CustomLog /var/log/URL.com-access_log common

Notes on the above, in brief

1. LoadModule is in httpd.conf
2. NameVirtualHost declaration is in httpd.conf
3. virtualhosts make use of /virtualhostname/conf/vhost.conf
4. WSGIScriptAlias declarations are in respective vhost.conf, not in httpd.conf*

* This is because WSGIScriptAlias in httpd.conf didn't work

Once that is configured as you like, simply restart the server in this manner:

/usr/local/psa/admin/sbin/websrvmng --reconfigure-vhost --vhost-name=mt-example.com 

service httpd graceful

Now test your django. With any luck, you will see a screen confirming a correct installation.

Serving Django on Apache

As an update on the previous post, looking into current Django docs, the Apache production server method of using mod_python to serve Django has been superseded by mod_wsgi. I had partially implemented the former method, so now I'm bringing it up to the latest spec.

According to google.code on Python WSGI, Apache 1.3, 2.0 or 2.2 are compatible. Apache on a MediaTemple dv account shows

httpd -v

Server version: Apache/2.2.3
Server built: Nov 12 2008 10:41:27

Python needs to be 2.3 through 2.5 and the vers on (mt) shows

Python 2.4.3 (#1, May 24 2008, 13:47:28)

So far, so what. Now to install wsgi...

wget http://modwsgi.googlecode.com/files/mod_wsgi-2.5.tar.gz
tar xvfz mod_wsgi-2.5.tar.gz
configure
make
make install

finally edit

httpd.conf

located in

/etc/httpd/conf.

Adding the lines:

LoadModule wsgi_module modules/mod_wsgi.so

and


<virtualhost *>
ServerAdmin jasonthewolf@gmail.com
ServerName www.unplugtheinternet.com
DocumentRoot /var/www/vhosts/unplugtheinternet.com
WSGIScriptAlias / /django/test01/apache/django.wsgi
</virtualhost>

In the above, WSGIScriptAlias deserves some explanation:
Description : WSGI maps a URL to a filesystem location and designates the target as a WSGI script.
Syntax : WSGIScriptAlias URL-path file-path|directory-path
Context : server config, virtual host
Module : mod_wsgi.c

A request for http://www.example.com/wsgi-scripts/name in this case would cause the server to run the WSGI application defined in /web/wsgi-scripts/name. This configuration is essentially equivalent to:

and restart apache with

apachectl restart

All reports are good from Apache so far. Now that the installation of wsgi is complete, back to Django implementation.

I once again edited httpd.conf, this time adding
WSGIScriptAlias / /path/to/mysite/apache/django.wsgi

which for my site and requirements was effectively:

WSGIScriptAlias /django/test01 /var/www/vhosts/unplugtheinternet.com/apache/django.wsgi

This is the folder I have chosen for my python tests. If they work out, I'll move them to the root of some url or another. Following the convention established in the example, I created directory apache and the used vi to create the textfile django.wsgi as:

import os
import sys

os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()