Pointers in Python resemble variable assignments in PHP5

I encountered the PHP5 clone operator '=&' and a very good illustration of its use. Incidentally, PHP5 variable assignment resembles the use of pointers in Python.

In short, a variable assignment in PHP is a reference to the assignment object. If an object ( $pointer = object ) is reassigned ( object = 'foo'; ), the pointer will reference the OLD object ( $pointer = object ). If however you clone the object ( $clone =& object), when the object is reassigned, ( object = 'foo' ), clone will change as well ( $clone = 'foo' ).

Cloning, then, is making the variable operand synonymous with its counterpart, rather than a reference to the counterpart. There are some particularly enlightening examples of 'clone' aka '=&' on php.net.

Eclipse updates pt 1

This day has allowed me to explore the capabilities of Eclipse as a standards-based SDK. Perhaps not news to those who already use it for such, but since I use Flash Builder almost exclusively for Flash-based development, this is fresh territory. This further exploration came about as the result of a recent back-end dev project where the IDE options played out as either:

• TextMate
• vi on the Terminal CLI
• Coda

I haven't delved into Coda, though I have a good amount of exposure on the other two. Let me state for the record that I endorse the viability and value of TextMate as an IDE. Now, clarifications duly stated, I would like an IDE that provides:

• managing repositories such as svn and git
• code lookup and completion for JavaScript, including 3rd party libraries
• code lookup and completion for PHP
• bracket pairing assistance that is broadside-of-a-barn obvious
• local file, folder and directory management
• quick access to function and property definitions (cmd-click seeking)
• optionally, FTP management

I'd be happy to address which of these are addressed to my satisfaction in the above-listed IDEs, but for now I will summarily say none of them hit all the targets. That could be due to my own habits - perhaps blinders on my part - I would love nothing more than to be further educated on this. Beyond that, some of the newer PHP frameworks at large (CodeIgniter, CakePHP, Symfony) boast unit testing, and I would prefer an IDE with debugging breakpoints and object inspection so I can see what fury my code hath wrought in the runtime.

Thus far, I have done the following to spec up Eclipse:

1. Update to Eclipse Helios 3.6
2. Install PHP and JavaScript language packs for Eclipse
3. Install eGit for Helios (tested and working)
4. Install Aptana (see mild apprehension*)
5. Install Eclipse Web Tools

*Aptana bills itself as:

a set of application development tools for Web 2.0 and ajax for use with programming languages such as JavaScript, Ruby, PHP and Python.

They had a big buzz out of the gate as a premium solution vendor to the open-source community. By late 2009 their revenue stream was lacking to the point of significant downscaling and they have consequently reduced their support and development staff. Their current 2.x release is bemoaned as unpredictable, buggy and meretricious, resulting in retrograde support for the deprecated 1.5 version package. Furthermore, Aptana was aquired less than a month ago by an outfit called Appcelerator. In short, Aptana has a legacy of step-child treatment.

Thus my concern with installing and testing out Aptana. Since it is one of only two commonly-employed means of third-party javascript code completion, I went ahead with it. Installing Aptana is the first step. Following success with that, its proprietary set of plug-ins allows jQuery to work with it. I succeeded with the first part, but unfortunately their plug-in service has proven to be currently offline. On to the next option: Eclipse Web Tools.

More on mod_wsgi issues

When you add wsgi to Apache it is so you can run Python on Apache. Thus you need, at a minimum, Python and Apache.

Python

If you use a utility such as virtualenv to compartmentalize several domain development environments on one server account, installing mod_wsgi may not and probably will not be associated with the correct Python library. Will not, that is, unless you explicitly point it out in the installation process. Be sure to plan in advance which Python install you are developing for, and be sure to flag it when installing mod_wsgi:

./configure --with-python=/home/django/domains/mydomain.com/bin/python2.5* ***

*** UPDATE (1/31/10): Please read the helpful comment and link from Graham Dumpleton, author of mod_wsgi, below. Per his direction, it is best to install mod_wsgi against the complete base Python installation and add references to the virtual environments from your Apache config or from the WSGI script file. Thanks for taking the time to clarify, and for making mod_wsgi available for general use.

That said, there are many ways to install mod_wsgi, from manual to managed by aptitude, get-apt or the like. The downside to using a managed install is the dependency on the vintage of their managed version. At last check, the aptitude repository of mod_wsgi is still at 1.3 while the current stable version on googlecode is 2.8. Thus, you may want to look into another method, perhaps a manual installation.

Apache

When manually installing mod_wsgi, be sure you have the appropriate development library for apache, in the case of ubuntu hardy heron, apache2_threaded_dev will allow you to install mod_wsgi. Furthermore, if you are using aptitude to install apache2_threaded_dev, go ahead and update the distro as a whole first by running sudo aptitude update. If you do not, you may find the install doesn't work, and there will be no indication as to what went wrong.

Finally, with these measures in place, try ./configure with the appropriate flags and see if you can get the correct library in place. Good luck.

*your location will be different from this example

MAMP with mod_wsgi

Once you get a database set up, sqlite or otherwise, you can use it with Django applications. You can serve Django directly from a command line or through Apache. In either case you will need Django to be installed. From a command line, simply run the django server and browse to it:

django-admin.py runserver

If your production server is going to be Apache, your best bet is to run Django under Apache locally. To do this you need a helper library for Apache to use. The de facto solution is mod_wsgi.

mod_wsgi

To get started with mod_wsgi, typically you download mod_wsgi from Graham Dumpleton's Google Code repository, you go to your command line, you untar it, you configure it, you make it and you make install it. This makes the code library available to wsgi-enabled services, in effect allowing you to run Django on Apache.

When you make install mod_wsgi, it places the libraries in your /lib directory. Then it links to your Apache directory where OSX keeps it, /usr/libexec/apache2. Installation includes automatic detection, or you can use some optional flags to specify resource paths, detailed in the wsgi installation guidelines.

Now your OSX Apache has modwsgi installed, but the Apache version in MAMP does not. To verify this, try making a Django app and serving it via MAMP. First, make a new django app in an appropriate directory:

django-admin.py startproject myproject

Next, open and modify the config for your MAMP Apache app, located in /Applications/Mamp/conf/apache/httpd.conf. For that matter, have a look at what your OSX Apache instance has in its config file. It is private/etc/apache2/httpd.conf. Searching for wsgi in this file yields:

LoadModule wsgi_module libexec/apache2/mod_wsgi.so

Looks good even if the relative directory location may be an issue. Insert this info into the MAMP httpd.conf and test it. Actually, there is no need because it will not work. This is due to the structure of MAMP and its proprietary use of Apache. Unlike a full install of Apache, the folders in MAMP have a different structure and are optimized to run modularly. Consequently, you cannot append a flag to your configure command in this manner:

./configure --with-apxs=/usr/sbin/apxs2-worker

because the structure available to Apache is not in MAMP, so detection will fail.

This leaves a couple of options. Either reconstruct some of the MAMP Apache folders so detection will succeed or look for another solution. Considering the evident lack of general success from many Google searches, I am letting this fish off the hook. Feel free to leave a comment if you know a way to make it work.

As a final consideration, mod_python appears to be a lot quicker to set up with MAMP. That could be a candidate if you are in a pinch. Otherwise, if serving Python via Apache is your goal, I recommend using the resources included with OS X and skipping MAMP altogether.

RemoteObject Endpoints in Django

A few quick assumptions about your project:

• It's built with a lightweight proxy server such as nginx or lighttpd
  
• It utilizes the Django web framework
• It remotes data using some form of ORM
  
• It is presented via Flash/Flex

If this is the case, the simplest and most straightforward way to configure your gateway is by setting it at the root URL:

#my /etc/nginx/sites-enabled/mysite.conf
server {
         listen                    50.5.5.50:80;
         server_name      myurl.com www.myurl.com;
         if ($host = 'www.myurl.com' )    {
               rewrite ^/(.*)$ http://myurl.com/$1 permanent;
         }

         location / {
               proxy_pass    http://127.0.0.1:80/;
               include            /etc/nginx/proxy.conf;
         }

         location /media/  {
                root                /home/django/domains/myurl.com/public_html/;
         }
}

Thus all web traffic that doesn't match '/media/' is passed to the urls.py for further processing. This is however at odds with your remoting requirements for Flash/Flex. To wit, you need a static file called crossdomain.xml to be available at your site's web root in order to meet the Flash client's security needs.

I looked for a good answer to this and found the Django-Flashpolicies by James Bennett effective and QUICK for this very purpose.

All you will need to do, essentially, is this:

easy_install django-flashpolicies

...and this...

url(r^'crossdomain.xml$',
  'flashpolicies.views.simple',
  {'domains': ['media.example.com', 'api.example.com']}),

...and you're good to go. Works better than a savage karate chop to the nose, most of the time anyway.

Hope that allows you to move forward quickly.

Apache DocumentRoot in Python and Django

Be very deliberate in your overall architecture when setting up new django projects. The directory dependencies resulting from an apache-nginx-python-django server configuration can make your path dependencies very complicated.

Generally put, being indiscriminate in your setup now will result in extreme challenges to resource deployment later on.

Here are some working methods that can greatly complicate your setup:

• Hosting several modules on one URL
• Having only one virtual host for many projects
  
• Hosting several modules on one server account
• Setting Python gateway modules off the root of your URL
• Using Python server.py-type gateways in addition to ordinary http gateways

Here are some ways to simplify your paths:

• Set up several virtual hosts
  
• Host only one Django project per URL (or as few as you can manage)
  
• Clearly structure your resources to discourage any overlap on your VPS or dedicated server
• Consider the trade-offs between a complex rig of sites and resources and several simple unrelated server environments. There's really no need to attempt the most complex possible solution unless that's your bag.

The nice thing about Python and Django is that they can handle either direction. There's a way to work everything out however you prefer, and the choice is yours to make.

That said, remember also that it is nice when projects just work without a lot of complex troubleshooting or traffic control.

Here is an example of a moderately complex configuration with a mixed bag of pros and cons.

Setting up PyAMF for Django on Slicehost VPS

I'm continuing to elaborate on a server environment for multiple django projects. In this case, they are serving flex applications on the front end and providing AMF data through PyAMF.

The structure allows domains to be added as needed in a /domains/ subdirectory. Each directory has a virtualenv set to encapsulate the packages installed and minimize complications.

Currently, I am installing PyAMF onto my current project dir using easy_install, from /domains/site.com/site.com/bin

./easy_install pyamf

it is important to run the command from within your virtual environment, otherwise it will default to your system install at /usr/lib/python2.5/site-packages, which isn't mapped to a server and won't be using pyamf at all.

Install PyAMF 0.42 successful. "So far so what," you say pithily. Well, it could certainly be worse than that. Remember that easy_install allows indiscriminate updating of installs via the command

./easy_install -U pyamf

Next some info about eclipse and subversive.

Eclipse Galileo with PyDev and Subclipse

Just installing a bunch of stuff so I can begin developing some django projects locally for a remote deployment. Consisting of:

• Eclipse 3.5 Galileo
• Subclipse 1.6.x
• Python 2.6
• ActiveState ActivePython (currently 2.6)
  

Getting it up and running.

Update: A WinXP install of Eclipse 3.5 will not be able to svn over ssh (see below instead) until you point out a valid ssh app in its config. A good method for this is on the breakitdownblog.

Get the latest PyDev for Eclipse Galileo at http://pydev.sourceforge.net/updates.

Don't forget, you'll have to manually install the connectors. Galileo will need SVNKit and/or JavaHL to reach your subversion server.

Enabling Bombproof Django Logging For Beginning a Project

"Hey, I installed Django and it doesn't work."
"Don't panic. Check the error logs."
"Logs? Django doesn't do logs."
"Python does. You have to add them into your project."
"Yes, but Django doesn't work. Now I am panicking."

And...cut. Beautifully acted all around. The point of the above is, there is a potential chicken-and-egg scenario brewing when you:

1. Have a nonfunctioning Django app.
2. Install logging to debug it.
3. Can't run the application to test the logging...
4. to run the application.

I poked around a bit in search of cut-n-paste logging scripts that get a project back into the light of day. I found a few, so thanks to lawfulsamurai and caktus for hearty illumination.

For brevity and to add a couple of caveats, I have combined what was published elsewhere into this quick how-to. Simply put, it will get you up and logging straight away. Just add it to your settings.py:

import logging
import logging.handlers

logger = logging.getLogger('project_logger')
logger.setLevel(logging.INFO)

# IMPORTANT, make sure python can write to the below log-file, or you will get an error
LOG_FILENAME = '/root/to/your/logging/dir/and/log-file'

LOG_MSG_FORMAT = '%(asctime)s %(levelname)s %(message)s'

handler = logging.FileHandler(LOG_FILENAME)

formatter = logging.Formatter(LOG_MSG_FORMAT)

handler.setFormatter(formatter)
logger.addHandler(handler)
logger.info('testing logging')

There are some fine debugging options out there e.g. djangologging and the django debug toolbar. The advantage of the above is, it is simply a python module so you can't get more native than that.

Feel free to try whichever you find is best and to comment on this option.

Basic Google App Engine command line

Some basic Python SDK commands in Google App Engine

• dev_appserver.py - the development web server
  
• appcfg.py - for uploading your app to App Engine

To run a local instance of a site named mysite within the Google App Engine localserver, invoke command line:

dev_appserver.py mysite (as opposed to python manage.py runserver)

When it comes to starting a project you can opt for either the built-in 0.96 of Django that is in the SDK or alternately bootstrap a 1.x version of Django. If the built-in is satisfactory, the first step in creating the new app engine site is identical to starting a self-standing Django project:

django-admin.py startproject mysite

If you choose the latter method, you will have more flexibility with the latest features of Django. This requires a different startup. You can simply checkout the latest helper app from google code. This serves as a platform for the applications you plan to create. Here's the command line:

svn checkout http://google-app-engine-django.googlecode.com/svn/trunk/ mysite

So now you're all set with a base. Don't forget that with 1.x of Django, you have to put in the Django library locally, probably best using the zipimport method, the details of which are in an earlier post on this blog. Regardless of which Django version you opt for, creating an application is done thus:

python manage.py startapp myapp

Next, begin connecting the apps as you would in Django standalone, via urls.py. There are also special requirements using the App Engine datastore instead of an relational database. More info on the differences can be found here.

PIL for Google App Engine's Python SDK

If you plan to use images in your Django-framwork Google App, you need to add the Python Image Library. If you are using os x 10.5, this bullet list can help set it up. This assumes you are working with the native Python 2.5 installation rather than a new install. Why install it again? You don't need to.

• Google's PIL documentation is fine but fragmentary.
• The update provided by Matt Kangas is really all you need.
• If you have not modified Terminal to run shell scripts, macfidelity provides details to get the job done.

That should be all that is required.

Django MVC Basics Pt 1: URLconf

The URLconf - mapping between URLs and view functions that should be called for those URLs.

from django.conf.urls.defaults import *

# Uncomment the next two lines to enable the admin:
# from django.contrib import admin
# admin.autodiscover()

urlpatterns = patterns('',
# Example:
# (r'^mysite/', include('mysite.foo.urls')),

# Uncomment the admin/doc line below and add 'django.contrib.admindocs'
# to INSTALLED_APPS to enable admin documentation:
# (r'^admin/doc/', include('django.contrib.admindocs.urls')),

# Uncomment the next line to enable the admin:
# (r'^admin/', include(admin.site.urls)),
)

Important to note are:
r - Python indication that the string is a "raw string", i.e. do not interpret characters.
' ' - string enclosed in single quotes
^ - caret, a regular expression meaning "require that the pattern matches the start of the string"
$ - dollar sign, regex meaning "require the pattern match the end of the string".

e.g. ensures this works: www.url.com/hello/
and this does not: www.url.com/hello/there (thanks to $)
and this does not:www.url.com/there/hello (thanks to ^)

See table of basic Regular Expressions - from the Django Book.

The second parameter in the 2-tuple sometimes begins with include(... and sometimes does not. It is the function to be associated with the specified URL. The function's first parameter is always the HttpRequest object. Always. Here's documentation on the HttpRequest object's API.

Python Image Library (PIL) on MediaTemple (dv)

When you're on MediaTemple (dv) with root access, first get the TAR from here. As of this post, the latest version is 1.1.6.

wget http://effbot.org/downloads/Imaging-1.1.6.tar.gz

Untar in the usual way.

tar zxvf Imaging-1.1.6.tar.gz

Then, go into the created folder and install the package using:

python setup.py install

If all goes well your ssh will echo many installation processes, following which you will have an amazingly easy time working with images via http.

Subversion on MediaTemple (dv)

Just completing some upgrades to my service to incorporate subversion. I don't know how I got by without it for so long. My first exposure to version control software was Visual SourceSafe back in the dark ages, and since then I've either used that or similar routines involving backups and multiple versions in similarly-named folders. Subversion is a lean, command-line, open-source version of the same. Here's what I did.

In the case of MediaTemple, Subversion is already installed. All you have to do to get started is invoke a new instance on the command line.

svnadmin create /name/of/project/folder

Then build a project to import into svn, such as

tmp/project/trunk
tmp/project/branches
tmp/project/tags

Jump into the trunk and add some files to it

file1
file2
file3

When you import the project, subversion vacuums it all up.

svn import /tmp/project file:///name/of/project/folder -m "initial import"

Note the -m, ensures you won't get an error for not setting the default text editor for subversion. In case you would like to set it, just use the command:

export SVN_EDITOR=vi

Now you can checkout files using the checkout command

svn checkout file:///name/of/project/folder/trunk project

And you can post your revisions using commit

svn commit

Notice the difference in directory structure between creating, importing and checking out. Creating and importing require only the folder, but checking out requires the additional trunk folder and project name specified.

Here are some additional resources of note for subversion installations, with an emphasis on MediaTemple (dv) installations.

Ayman Hourieh Subversion Tutorial - quick and to the point.

Tony Spencer's Tutorial - includes basic WebDAV integration, also quick and to the point.

Python and Django in MediaTemple (dv) server

Getting Django to run on a dv server is not as easy as their preconfigured - and cheaper - grid containers. While the shared grid containers are ready to go out of the box, the dv has some server admin changes to get through before you see that welcome

"Hello World".

Specifically, the httpd.conf needs additional lines of code to connect the web root to the Django application. To add a little spin to it, there's also the config file generated by Plesk. That needs some edits as well.

The order of statements in the zz010_psa_httpd.conf file can be a particular obstacle. Seems you have to have the WSGIScriptAlias declarations high up in the page or they aren't acted on properly as the procedure continues. Changing LogLevels to info gives the most detailed view from your logfiles but may not ultimately identify the problems.

In any case, here are the steps in a nutshell:

(added to zz010_psa_httpd.conf - plesk)

Include /var/www/vhosts/urlone.com/conf/httpd.include
Include /var/www/vhosts/urltwo.com/conf/httpd.include
WSGIScriptAlias /myapp /var/www/vhosts/urlone.com/wsgi-scripts/myapp.wsgi
WSGIScriptAlias /django/test01 /var/www/vhosts/urltwo.com/wsgi-scripts/myapp.wsgi

(added both httpd.includes to file structure)

Seems simple enough once it works. I hope this gets you out of a pinch.

Serving Django on Apache

As an update on the previous post, looking into current Django docs, the Apache production server method of using mod_python to serve Django has been superseded by mod_wsgi. I had partially implemented the former method, so now I'm bringing it up to the latest spec.

According to google.code on Python WSGI, Apache 1.3, 2.0 or 2.2 are compatible. Apache on a MediaTemple dv account shows

httpd -v

Server version: Apache/2.2.3
Server built: Nov 12 2008 10:41:27

Python needs to be 2.3 through 2.5 and the vers on (mt) shows

Python 2.4.3 (#1, May 24 2008, 13:47:28)

So far, so what. Now to install wsgi...

wget http://modwsgi.googlecode.com/files/mod_wsgi-2.5.tar.gz
tar xvfz mod_wsgi-2.5.tar.gz
configure
make
make install

finally edit

httpd.conf

located in

/etc/httpd/conf.

Adding the lines:

LoadModule wsgi_module modules/mod_wsgi.so

and


<virtualhost *>
ServerAdmin jasonthewolf@gmail.com
ServerName www.unplugtheinternet.com
DocumentRoot /var/www/vhosts/unplugtheinternet.com
WSGIScriptAlias / /django/test01/apache/django.wsgi
</virtualhost>

In the above, WSGIScriptAlias deserves some explanation:
Description : WSGI maps a URL to a filesystem location and designates the target as a WSGI script.
Syntax : WSGIScriptAlias URL-path file-path|directory-path
Context : server config, virtual host
Module : mod_wsgi.c

A request for http://www.example.com/wsgi-scripts/name in this case would cause the server to run the WSGI application defined in /web/wsgi-scripts/name. This configuration is essentially equivalent to:

and restart apache with

apachectl restart

All reports are good from Apache so far. Now that the installation of wsgi is complete, back to Django implementation.

I once again edited httpd.conf, this time adding
WSGIScriptAlias / /path/to/mysite/apache/django.wsgi

which for my site and requirements was effectively:

WSGIScriptAlias /django/test01 /var/www/vhosts/unplugtheinternet.com/apache/django.wsgi

This is the folder I have chosen for my python tests. If they work out, I'll move them to the root of some url or another. Following the convention established in the example, I created directory apache and the used vi to create the textfile django.wsgi as:

import os
import sys

os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Python and Django wrapup

Almost a wrapup.

Got django installed in /usr/lib/python2.4/site-packages/

Got django deployed in /var/www/vhosts/unplugtheinternet.com/django/test01/ with subfolder cms autogenerated

Got MySQLdb module added to python so django can use a MySQL database, and django has been successfully configured to build the schemas for the modules it deploys.

Created a vhost.conf to take care of webserver requests that python should handle (courtesy of Nick Sergeant's blog entry).

Still it isn't working properly. But since I'm leaving for a camping trip in ~1 hour, it's time to switch it off and go join my friends at Assateague Island.

Arrividerci pals.

MediaTemple, Python and Django

Just installing Django on my Mediatemple dv account.

First, log in to SSH as root user. Then use subversion, provided by mediatemple in

WebControl > Root Access & Developer Tools > Install Developer Tools,

which must at some point be enabled to be accessible. Following this,

Subversion grabs the latest version to /var/trunk/django.

In the case of dv accounts, python packages are located in /usr/lib/python2.4/site-packages

Create a symbolic link between these two locations using

ln -s /var/trunk/django /usr/lib/python2.4/site-packages

then add another symbolic link to bring the underlying python script to the root of the django trunk:

ln -s /var/trunk/django/bin/django-admin.py /usr/local/bin

Basically, what is happening here is:

Django Source /var/trunk : Subversion project root

\ --> accessible to /usr/lib : Libraries for programming and packages
\ --> accessible to /usr/local : Local hierarchy

More on this. Now:

import django

and


django.version

voila!

Next, you must add the MySQLdb interface for python if you are planning to use mySQL as your cms database. Here's a good reference for once you have the tarball in your directory. However, I'd rather wget it directly from the sourceforge site. Here is the command to use from root SSH:

wget http://downloads.sourceforge.net/mysql-python/MySQL-python-1.2.2.tar.gz

Works fine. Now install the module and it should run without an error.