Flexcairngorm extensions considered

Tracing the evolution of the Cairngorm framework reveals a great effort to isolate business logic. A coup of the framework has been to shift event flow from exclusively the display list towards a simple, efficient non-UIComponent propagation within an application architecture.

Specifically, the native event flow in Flex works hierarchically within the display list. Since display objects are by definition hierarchical, events flow through them from impetus to the bottom node and, if so configured, back again. This ensures full propagation of the event. A reasonable supposition on behalf of Cairngorm engineers is the difficulty of performing business logic apropos of a display hierarchy. Business logic requires a unique non-display set of delegates, services and model managers. The Cairngorm framework places events at the behest of these classes.

How then to bridge the new events and the old? This is where flexcairngorm takes the reins.

There is a very handy explanation of this, supplemental graphics and all, by Thomas Burleson on Gridlinked.info. He describes the process by which notification returns to the view following the propagation to business logic and the response back. It involves IResponders and a callback parameter. Browse gridlinked.info to learn all about it.

Once you've read it, the next step is coordinating flexcairngorm with Mate dependency injectors, using Mate EventMaps.

Flex MVC options

The set of instructions in rich applications tends to get quite complex. Thus, RIA code needs to be divided up into smaller components to manage complexity. Design patterns such as MVC are loose methodologies for doing this, with limitless implementations in practice. In Flex, MVC are often referred to as microarchitectures. They manage business logic for a client application, allowing decoupling between logic and UI.

The prominent first generation of mature MVC framework for Flash and Flex is called Cairngorm. It is now up to 2.2 with version 3 hovering indefinitely in beta. Cairngorm introduces a new system of event management called the CairngormEvent. It is more analog than extension of a Flex Event, since it has unique dispatch and propagation attributes.

The pros and cons of isolated event management, MVC and FlexEvent, have been largely considered in the developer community, resulting in some rethinking of Cairngorm event methodology. A prominent advance is UniversalMind's UM Cairngorm Extensions. This represents first and foremost an enrichment of the Cairngorm framework. If you plan on going down this path, UM Extensions may allow you to do more with it.

Otherwise, Flex is reaching the second iteration of third party frameworks, as Cairngorm gets more competitors. Mate and Swiz lead the pack, with retooled and reconsidered architectures for events and business logic structures. In some cases, microarchitectures are fully MXML in the second iteration.

By no means is the solution one-sided, as many groups have invested heavily in their extant codebase. In any case, all the options deserve a thorough look see. Some orientation will allow you to consider the best for your project.

Multiple AMF Django apps share a common gateway

A while back I wrote about a library called django-flashpolicies for providing a flexible remoting environment. Essentially, it frees up the root of your django project for whatever content you like while providing a crossdomain policy file there for remoting applications.

That is still one way to go, but as an advance on that, have a look at the Django-amf-gateway library.

The PyAMF team offers info on it as a way to structure a django project such that you can heap on all the apps you want and allow them to share a common http gateway. By simplifying gateway addresses on the server side django-amf-gateway simplifies resource management on the client side.

Works great in applicable cases.

More on mod_wsgi issues

When you add wsgi to Apache it is so you can run Python on Apache. Thus you need, at a minimum, Python and Apache.

Python

If you use a utility such as virtualenv to compartmentalize several domain development environments on one server account, installing mod_wsgi may not and probably will not be associated with the correct Python library. Will not, that is, unless you explicitly point it out in the installation process. Be sure to plan in advance which Python install you are developing for, and be sure to flag it when installing mod_wsgi:

./configure --with-python=/home/django/domains/mydomain.com/bin/python2.5* ***

*** UPDATE (1/31/10): Please read the helpful comment and link from Graham Dumpleton, author of mod_wsgi, below. Per his direction, it is best to install mod_wsgi against the complete base Python installation and add references to the virtual environments from your Apache config or from the WSGI script file. Thanks for taking the time to clarify, and for making mod_wsgi available for general use.

That said, there are many ways to install mod_wsgi, from manual to managed by aptitude, get-apt or the like. The downside to using a managed install is the dependency on the vintage of their managed version. At last check, the aptitude repository of mod_wsgi is still at 1.3 while the current stable version on googlecode is 2.8. Thus, you may want to look into another method, perhaps a manual installation.

Apache

When manually installing mod_wsgi, be sure you have the appropriate development library for apache, in the case of ubuntu hardy heron, apache2_threaded_dev will allow you to install mod_wsgi. Furthermore, if you are using aptitude to install apache2_threaded_dev, go ahead and update the distro as a whole first by running sudo aptitude update. If you do not, you may find the install doesn't work, and there will be no indication as to what went wrong.

Finally, with these measures in place, try ./configure with the appropriate flags and see if you can get the correct library in place. Good luck.

*your location will be different from this example

ORM options for Python server to Flex client

Let's say Python is your solution of choice for backend development and Flash technology is your preferred client. If your project uses a database, you will need a good ORM solution at some point. The options are many and here are a few of them:

• WebORB by MidnightCoders
• Django built-in ORM
• SQLAlchemy ORM

There are surely other options (roll your own, anybody?) but these come to mind readily. Now, how to choose which one? Let's look at the capabilities of each:

WebORB

Pros: MegaOptions, from RTMP Server to Secure Transactions and lots more, turnkey CRUD, GUI admin, GUI performance, plug and play, one-click ORM generator, ORM testing suite. The list goes on...

Cons: Costs a pretty penny last I checked. Could be considered bloaty. I would have to hear some performance testers weigh in on this.

Verdict: Great at limiting server-side development to a minimum. Great full-service solution. Would be a strong contender in a rich media studio or design shop.

====================================

Django built-in ORM

I'll use it when the time comes. Methods to replace Django's ORM outright with SQLAlchemy have been detailed on some intrepid developer blogs. I would rather not customize the framework too much, as it introduces potential troubleshooting down the line. You can use Django ORM and an addition ORM library like SQLAlchemy interchangeably, so that would be my first choice if going this route.

=====================================

SQLAlchemy

A very robust open source option with a surfeit of detailed documentation, this one regularly gets my vote. It is sufficiently explicit to require familiarity to use, but not too steep a curve to get started quickly. Extensions like Elixir simplify the code at the expense of greater abstraction. This is a case of using the right tool for the right project, so feel free to learn more about both.

MAMP with mod_wsgi

Once you get a database set up, sqlite or otherwise, you can use it with Django applications. You can serve Django directly from a command line or through Apache. In either case you will need Django to be installed. From a command line, simply run the django server and browse to it:

django-admin.py runserver

If your production server is going to be Apache, your best bet is to run Django under Apache locally. To do this you need a helper library for Apache to use. The de facto solution is mod_wsgi.

mod_wsgi

To get started with mod_wsgi, typically you download mod_wsgi from Graham Dumpleton's Google Code repository, you go to your command line, you untar it, you configure it, you make it and you make install it. This makes the code library available to wsgi-enabled services, in effect allowing you to run Django on Apache.

When you make install mod_wsgi, it places the libraries in your /lib directory. Then it links to your Apache directory where OSX keeps it, /usr/libexec/apache2. Installation includes automatic detection, or you can use some optional flags to specify resource paths, detailed in the wsgi installation guidelines.

Now your OSX Apache has modwsgi installed, but the Apache version in MAMP does not. To verify this, try making a Django app and serving it via MAMP. First, make a new django app in an appropriate directory:

django-admin.py startproject myproject

Next, open and modify the config for your MAMP Apache app, located in /Applications/Mamp/conf/apache/httpd.conf. For that matter, have a look at what your OSX Apache instance has in its config file. It is private/etc/apache2/httpd.conf. Searching for wsgi in this file yields:

LoadModule wsgi_module libexec/apache2/mod_wsgi.so

Looks good even if the relative directory location may be an issue. Insert this info into the MAMP httpd.conf and test it. Actually, there is no need because it will not work. This is due to the structure of MAMP and its proprietary use of Apache. Unlike a full install of Apache, the folders in MAMP have a different structure and are optimized to run modularly. Consequently, you cannot append a flag to your configure command in this manner:

./configure --with-apxs=/usr/sbin/apxs2-worker

because the structure available to Apache is not in MAMP, so detection will fail.

This leaves a couple of options. Either reconstruct some of the MAMP Apache folders so detection will succeed or look for another solution. Considering the evident lack of general success from many Google searches, I am letting this fish off the hook. Feel free to leave a comment if you know a way to make it work.

As a final consideration, mod_python appears to be a lot quicker to set up with MAMP. That could be a candidate if you are in a pinch. Otherwise, if serving Python via Apache is your goal, I recommend using the resources included with OS X and skipping MAMP altogether.

MAMP with sqlite

If you want a local enviro to test in, MAMP for Mac is a good way to go. Right out of the box, you get Apache, MySQL, SQLite and PHP support. Also in the mix is python 2.5, which ship with OS X.

With several apps at the ready, how do you get your head around the new environment? Piece by piece. First let's look at SQLite.

SQLite

Have a good read at some typical command line options to get you up and running. Go ahead and try it out. See if you can create a db, add some values, retrieve the values.

When you store a database, it is stored in the current directory. You can see what that is by using the command pwd, aka print working directory. Further documentation is available on the SQLite site. If you decide command line is not your style, one option is the SQLite Manager for Firefox. It's not for remote db admin but it'll do for local scenarios.

In summary, getting up and running with sqlite is simple. MAMP comes with MySQL, so that is another available option.

Send Failed Flex Error in WebORB

Is Flex via WebORB middleware sending a message similar to this?

Error: Server reported an error - Send failed

The resolution will most likely consist of looking over your xml declarations, both for the declared endpoint URI and the service it requests.

If you face such a challenge, read the very useful orientation by WebORB creator and principal Mark Pillar. Check this post to confront the matter head on and move forward.

Good luck.

Magento and PHP5 Install Memory Error

If, when installing Magento Ecommerce, you experience some variant of this message:

Fatal error: Allowed memory size of 16777216 bytes exhausted (tried to allocate 24 bytes) in /home/userdir/sites/mywebsite.com/public_html/magento/lib/Zend/Db/Statement/Pdo.php on line 232

The solution is simple. Just go to your

/etc/php5/apache2/php.ini*

and set memory higher than the default of 16M. The error trace indicates a minimum requirement of 24M for successful installation.‡

But you knew that already, right?

^{* php.ini may be in a different directory
‡ full disclosure: I allotted 64MB since the slice I'm using has 512MB available.}

Proxy Debuggers and Flex NetConnection Debugger

With AS2, setting up a NetConnection between a Flash Player and a server running Flash Remoting could be streamlined using the Flex NetConnection Debugger. It was included with Flex 1.0 ca. 2005 and only required two short steps to enable. Many developers included it in their remoting utilities shortlist.

Now that we're living in a CS4 age, the playing field has changed. All the functionality of the NetConnection Debugger has been integrated into the Flash library. This means that instead of launching separate apps for debugging, you can simply mock up a routine using calls to available library components.

Good enough, but if you prefer a dedicated debugging app, NetConnection Debugger is out, and some other options are in its place.

Midnight Coders App Puncher

Much has been said about Midnight Coders' contributions to RIA development. Their middleware tier is arguably the best muscle around and can be a real time-saver. I'm speaking of WebORB, but particular to debugging, have a look at another of their products, the RIA App Puncher.

Charles Web Debugging Proxy

Link to Charles.

ServiceCapture

Link to ServiceCapture.

Virtualenv vs Virtual-Python

If you harbor multiple versions of Python, as is usually a good idea, you can go at least two routes:

1. Virtualenv
2. Virtual-Python

Both organize your all-important PYTHONPATH environment variable and keep your preferred python executable at the ready. This in turn makes module add-ons, test projects and installations easy.

On that note, Easy_install recommends virtual-python.py as a quick way to keep organized. It is indeed easy to set up and concise as well.

Another option, the one I use incidentally, is virtualenv. Once installed, all you need to do to switch active versions is activate it via the command line.

In a newly created virtualenv there will be a bin/activate shell script, or a Scripts/activate.bat batch file on Windows.

On Posix systems you can do:

$ source bin/activate

Even better than that is Doug Hellmann's virtualenvwrapper which, as he describes, allows you to set a working python environment with a single command. It's easy to set up as well, just a couple of lines into your bash script and bombs away, your virtualenv is wrapped.

With these options, you can initiate new projects and test new python libraries without any conflicts or misgivings. Try it out sometime. What have you got to lose?

Mimic AMF Header to get an HTTP response

Following on the last post, you can use netcat to get an error from a python gateway. But how is it possible to get a successful response from a python gateway? As previously seen, you can use a client in a python command line. But what about using the netcat 'swiss-army knife' to elicit a successful response from an AMF gateway?

Read this page from OS Flash to learn the composition of an AMF header.

Putting available ports to use with AMF

In a previous post, see a detailed a procedure to choose and make available a tcp/alt port on a web server to avail the use of asynchronous data services. That post leads to this one, in which AMF calls are made over the available port.

To start, here is a method to test the port for an http response using netcat, via stearns.org:

echo -e "GET http://yoursite.com HTTP/1.0\n\n" | nc yoursite.com 80 | less

...or using the alternate available port...

echo -e "GET http://yoursite.com HTTP/1.0\n\n" | nc yoursite.com 8080 | less

This is a good example of using netcat as a client to get a response from an http service.

Analogous to this is the stated goal of this post, establishing connectivity between a lightweight client and an AMF service. Does that service by definition reside on an HTTP gateway? What type of gateway is the service? That remains to be answered.

For now, note that there is a starting example available from pyamf.com that allows you to test a connection to a third-party HTTP gateway. Titled PyAMF AMF Client, it works in the following manner to establish a simple methodology.

1. In SSH, launch a python command line
2. Load the modules in pyamf to initiate a test
3. Request the remote service and look for a successful response

Here, copied from the pyamf.com example, is the applicable sequence

from pyamf.remoting.client import RemotingService

client = RemotingService('http://demo.pyamf.org/gateway/recordset')
service = client.getService('service')

print service.getLanguages()

...which should directly result in the following response from pyamf.com

<pyamf.amf0.recordset><pyamf.amf0.recordset></pyamf.amf0.recordset></pyamf.amf0.recordset>

Why does this work as it should? Some of the reasons why it works

pyamf.remoting.client.RemotingService (see API)

We instantiated the class RemotingService, the instance call of which includes a parametric reference to an available gateway. Once you provide a valid gateway address, you have established a client for AMF calls.

getService method

The method parameter is the name of a service, in this case the service defined at http://demo.pyamf.org/gateway/recordset/. Try browsing to this address and you will see

400 Bad Request

To access this PyAMF gateway you must use POST requests (GET received)

The gateway provides information that it is a PyAMF gateway and only accepts POST requests. If you use netcat to contact the gateway...

echo -e "GET http://demo.pyamf.org/gateway/recordset HTTP/1.0\n\n" | nc demo.pyamf.org 80 | less

...you will get a similar response. If you netcat a POST to the gateway, you will get this...

HTTP/1.1 500 Internal Server Error
Date: Wed, 09 Sep 2009 15:21:18 GMT
Server: Apache/2.0.55 (Ubuntu) DAV/2 SVN/1.5.6 mod_python/3.3.1 Python/2.5.4 mod_wsgi/2.4 PHP/5.1.2 mod_ssl/2.0.55 OpenSSL/0.9.8a mod_perl/2.0.2 Perl/v5.8.7
Content-Length: 534
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator,
info@pyamf.org and inform them of the time the error occurred,
and anything you might have done that may have
caused the error.</p>
<p>More information about this error may be available
in the server error log.</p>
</body></html>

This is a longer response, but not a successful response. The question remains, how can a request to the server result in a service initiation? Let's look at some other possibilities.

RemoteObject Endpoints in Django

A few quick assumptions about your project:

• It's built with a lightweight proxy server such as nginx or lighttpd
  
• It utilizes the Django web framework
• It remotes data using some form of ORM
  
• It is presented via Flash/Flex

If this is the case, the simplest and most straightforward way to configure your gateway is by setting it at the root URL:

#my /etc/nginx/sites-enabled/mysite.conf
server {
         listen                    50.5.5.50:80;
         server_name      myurl.com www.myurl.com;
         if ($host = 'www.myurl.com' )    {
               rewrite ^/(.*)$ http://myurl.com/$1 permanent;
         }

         location / {
               proxy_pass    http://127.0.0.1:80/;
               include            /etc/nginx/proxy.conf;
         }

         location /media/  {
                root                /home/django/domains/myurl.com/public_html/;
         }
}

Thus all web traffic that doesn't match '/media/' is passed to the urls.py for further processing. This is however at odds with your remoting requirements for Flash/Flex. To wit, you need a static file called crossdomain.xml to be available at your site's web root in order to meet the Flash client's security needs.

I looked for a good answer to this and found the Django-Flashpolicies by James Bennett effective and QUICK for this very purpose.

All you will need to do, essentially, is this:

easy_install django-flashpolicies

...and this...

url(r^'crossdomain.xml$',
  'flashpolicies.views.simple',
  {'domains': ['media.example.com', 'api.example.com']}),

...and you're good to go. Works better than a savage karate chop to the nose, most of the time anyway.

Hope that allows you to move forward quickly.

Apache DocumentRoot in Python and Django

Be very deliberate in your overall architecture when setting up new django projects. The directory dependencies resulting from an apache-nginx-python-django server configuration can make your path dependencies very complicated.

Generally put, being indiscriminate in your setup now will result in extreme challenges to resource deployment later on.

Here are some working methods that can greatly complicate your setup:

• Hosting several modules on one URL
• Having only one virtual host for many projects
  
• Hosting several modules on one server account
• Setting Python gateway modules off the root of your URL
• Using Python server.py-type gateways in addition to ordinary http gateways

Here are some ways to simplify your paths:

• Set up several virtual hosts
  
• Host only one Django project per URL (or as few as you can manage)
  
• Clearly structure your resources to discourage any overlap on your VPS or dedicated server
• Consider the trade-offs between a complex rig of sites and resources and several simple unrelated server environments. There's really no need to attempt the most complex possible solution unless that's your bag.

The nice thing about Python and Django is that they can handle either direction. There's a way to work everything out however you prefer, and the choice is yours to make.

That said, remember also that it is nice when projects just work without a lot of complex troubleshooting or traffic control.

Here is an example of a moderately complex configuration with a mixed bag of pros and cons.

Simple Endpoint Test for Asynchronous Unit and Integration Processes

In the creation of open-source RIAs, it can be very difficult to ensure connectivity of endpoints. There are potential snags and edge cases surrounding the following critical points:

• Security clearance of port
• Availability of port resource
• Conflicts between multiple running servers
  
• Properly-declared URI endpoints

As in point three, matters can be complicated when proxy servers, such as lighttpd or nginx are working in conjunction with apache. Most likely, both are open on different IP registers on port 80; apache proxy on localhost, and the lightweight server on the public IP.

An RIA will asynchronously connect via a RemoteObject call on another port. It requires a valid connection. Utilities like netcat can save you from a quagmire of ineffectual random testing.

First, open the port in apache, as it will be the ORM-enabled server. Typically, the port is 8000 or 8080 for arbitrary reasons of convention and familiarity. Make sure your 8000 or 8080 is not tied up with memchache or some other port-requisite ancillary utility. Open up ports.conf and add the port in the configuration script.

TEST THE PORT

Restart the apache server. Using a port utility such as netcat, test port connectivity. One quick way is to use netcat from a command line. Netcat will attempt to connect to any port(s) you specify and report the protocol it encountered:

nc -v -w 2 -z target 20-30

FIREWALLS

Don't forget to modify your security settings to allow the port! Specifically, if you have set up a firewall, iptables requires an entry allowing public connections over the port in question or connections will fail. If they have been changed, be sure to restart the apache server and that the rules have been saved either via command-line or the initializing script, usually /etc/network/interfaces.

Be sure your apache server's NameVirtualHost settings allow the wildcard * and not only localhost connections. This way, requests from your http-alt port will bypass your proxy server altogether. Apache might warn you thus, but it can be ignored:

[Thu Sep 03 16:20:42 2009] [warn] NameVirtualHost *:80 has no VirtualHosts

With these potential obstructions addressed, you should be able to tunnel your way to fame and fortune.

NEXT UP

How to put that port to use in an RIA RemoteObject call.

Subversive SVN for Eclipse is Troubled

Kind of, yes. What is so terrible, in fact, is the great number of pitfalls waiting to be uncovered.

You can hope for no whammies but you will most likely trip the eclipse-subversive trap along the way to configuring a working setup. Problem is, JavaHL and SVNKit don't tend to work out of the box. You have to hunt them down from at least 2 locations, and doing so often results in installation conflicts.

If you are using SVNKit, it is because your svn over ssh is password-protected. If you go this route and don't mind the confusion of double passwords with very poor contextual info, you may be almost home free. The subversive interface interchanges the exact same control panel for both ssh and svn credentials, mediated by what is a very unreliable drop down for context. The combination is tricky at best.

If you choose JavaHL, it is most likely in order to configure an ssh key to engage your repository. SSH keys require JavaHL be enabled rather than SVNKit. You will be dismayed to discover how difficult it is to properly resolve a file key through the subversive interface. In all likelihood, it will fail silently.

To minimize the difficuly, the following is highly recommended:
Use SVNKit and password authentication for SSH
When the duplicate credentials dialogs mock you, just keep trying combinations based on your notes.

Good luck.

Update:
Subversive is easier to manage if your ssh and svn un/pw are the same.
It would be great if subversive could remember it if the SSH port is not 22. If it is not, you will have to input it every freaking time. Or!:
Visit Eclipse > Preferences > Team > SVN > Properties Configuration > Custom properties and add

SVN_SSH = ssh -p [portnum]

If you fail to synchronize, do over and over again. My subversive setup sometimes fails to synch 3-4 times before finally consenting.

Setting up PyAMF for Django on Slicehost VPS

I'm continuing to elaborate on a server environment for multiple django projects. In this case, they are serving flex applications on the front end and providing AMF data through PyAMF.

The structure allows domains to be added as needed in a /domains/ subdirectory. Each directory has a virtualenv set to encapsulate the packages installed and minimize complications.

Currently, I am installing PyAMF onto my current project dir using easy_install, from /domains/site.com/site.com/bin

./easy_install pyamf

it is important to run the command from within your virtual environment, otherwise it will default to your system install at /usr/lib/python2.5/site-packages, which isn't mapped to a server and won't be using pyamf at all.

Install PyAMF 0.42 successful. "So far so what," you say pithily. Well, it could certainly be worse than that. Remember that easy_install allows indiscriminate updating of installs via the command

./easy_install -U pyamf

Next some info about eclipse and subversive.

Setting up Post-Hooks for Subversion

The publicly available subversion book at svnbook.red-bean.com is both concise and comprehensive. It can help you deftly steer a subversion server in any way your inner administrator hankers to go.

I found subversion helpful in tracking versions. I found it less useful in quick commit-to-staging server process. As some bountiful moments in freenet #svn IRC led me to discover, using post-hook scripts is the way to go.

Check this info box on tigris.subversion.org, leading to this part of ch 5 of the svn book on using hooks to automate and extend commands. There are also some helpful examples of scripts that can be brought to bear on a project.

The code is built thus and left in the hook directory sans .tmpl extension:

REPOS="$1"
REV="$2"

# commit-email.pl "$REPOS" "$REV" commit-watchers@example.org
# log-commit.py --repository "$REPOS" --revision "$REV"

svn update /home/[username]/django/projects/mysite.com/projectdir >> /var/log/vhosts/svn-django-errorlog

Be sure to set your post-commit file to executable

chmod +x post-commit

Eclipse Galileo with PyDev and Subclipse

Just installing a bunch of stuff so I can begin developing some django projects locally for a remote deployment. Consisting of:

• Eclipse 3.5 Galileo
• Subclipse 1.6.x
• Python 2.6
• ActiveState ActivePython (currently 2.6)
  

Getting it up and running.

Update: A WinXP install of Eclipse 3.5 will not be able to svn over ssh (see below instead) until you point out a valid ssh app in its config. A good method for this is on the breakitdownblog.

Get the latest PyDev for Eclipse Galileo at http://pydev.sourceforge.net/updates.

Don't forget, you'll have to manually install the connectors. Galileo will need SVNKit and/or JavaHL to reach your subversion server.

Enabling Bombproof Django Logging For Beginning a Project

"Hey, I installed Django and it doesn't work."
"Don't panic. Check the error logs."
"Logs? Django doesn't do logs."
"Python does. You have to add them into your project."
"Yes, but Django doesn't work. Now I am panicking."

And...cut. Beautifully acted all around. The point of the above is, there is a potential chicken-and-egg scenario brewing when you:

1. Have a nonfunctioning Django app.
2. Install logging to debug it.
3. Can't run the application to test the logging...
4. to run the application.

I poked around a bit in search of cut-n-paste logging scripts that get a project back into the light of day. I found a few, so thanks to lawfulsamurai and caktus for hearty illumination.

For brevity and to add a couple of caveats, I have combined what was published elsewhere into this quick how-to. Simply put, it will get you up and logging straight away. Just add it to your settings.py:

import logging
import logging.handlers

logger = logging.getLogger('project_logger')
logger.setLevel(logging.INFO)

# IMPORTANT, make sure python can write to the below log-file, or you will get an error
LOG_FILENAME = '/root/to/your/logging/dir/and/log-file'

LOG_MSG_FORMAT = '%(asctime)s %(levelname)s %(message)s'

handler = logging.FileHandler(LOG_FILENAME)

formatter = logging.Formatter(LOG_MSG_FORMAT)

handler.setFormatter(formatter)
logger.addHandler(handler)
logger.info('testing logging')

There are some fine debugging options out there e.g. djangologging and the django debug toolbar. The advantage of the above is, it is simply a python module so you can't get more native than that.

Feel free to try whichever you find is best and to comment on this option.

Working with Alfresco in Java EE

I've dedicated some serious hours to installing and deploying Alfresco on a MediaTemple (dv) VPS. The result has been a revelation of sorts. It's the big time for open source enterprise content management (ECM) and there's no going back.

Alfresco is like having your own personal cloud-based Google Docs to customize and host as many secure accounts as you like. While set up and configuration are not a cakewalk, it can be done. I took some decent notes on the process, so I will check back in to detail the process of configuring MediaTemple's vps environment.

In particular, I will show a method to set up a new JDK, fine tune Plesk and the Apache environment, and finally modify some scripts in Alfresco with the result of a successful java build.

That aside, the Java EE with Alfresco is a beast on memory. If there is any hidden cost on running the community edition of Alfresco, that may be it.

Basic Google App Engine command line

Some basic Python SDK commands in Google App Engine

• dev_appserver.py - the development web server
  
• appcfg.py - for uploading your app to App Engine

To run a local instance of a site named mysite within the Google App Engine localserver, invoke command line:

dev_appserver.py mysite (as opposed to python manage.py runserver)

When it comes to starting a project you can opt for either the built-in 0.96 of Django that is in the SDK or alternately bootstrap a 1.x version of Django. If the built-in is satisfactory, the first step in creating the new app engine site is identical to starting a self-standing Django project:

django-admin.py startproject mysite

If you choose the latter method, you will have more flexibility with the latest features of Django. This requires a different startup. You can simply checkout the latest helper app from google code. This serves as a platform for the applications you plan to create. Here's the command line:

svn checkout http://google-app-engine-django.googlecode.com/svn/trunk/ mysite

So now you're all set with a base. Don't forget that with 1.x of Django, you have to put in the Django library locally, probably best using the zipimport method, the details of which are in an earlier post on this blog. Regardless of which Django version you opt for, creating an application is done thus:

python manage.py startapp myapp

Next, begin connecting the apps as you would in Django standalone, via urls.py. There are also special requirements using the App Engine datastore instead of an relational database. More info on the differences can be found here.

PIL for Google App Engine's Python SDK

If you plan to use images in your Django-framwork Google App, you need to add the Python Image Library. If you are using os x 10.5, this bullet list can help set it up. This assumes you are working with the native Python 2.5 installation rather than a new install. Why install it again? You don't need to.

• Google's PIL documentation is fine but fragmentary.
• The update provided by Matt Kangas is really all you need.
• If you have not modified Terminal to run shell scripts, macfidelity provides details to get the job done.

That should be all that is required.

Google App Engine with Django 1.0

Google App Engine has offered Django 0.96 out of the box since it launched in mid 2008. That has changed as of this month.

Applications can now choose either Django 0.96 or what is currently the latest stable release, 1.0.2. This means there is no need to post a zipfile to your appspot in order to utilize the latest Django. But is it that simple? Not quite.

While the cloud has the latest version, your local server environ does not. In other words, the SDK and appspot server are a bit different. This means you still need the Django Helper to run 1.0.2 locally, where you will probably be developing your site.

I went through this process and found many resources to shed light on the procedure. Some are more suited than others, due to good but out-of-date info on the woodpile. Here's what did work:

• Google Code documentation on the Python SDK - what I like to think of as the GAE personal web server, though it's actually that and a bit more.
• Google documentation on the Django Helper - required reading but terse. Provides and describes a turnkey local project running Django 1.0.2.
• Finally, from Bubble Foundry in Nederland, the paydirt, aka how to get it done, aka as direct as a battleaxe to the head. Thanks, I needed that!

Good luck setting up your dev environment. Google labs makes it not so bad after all.

Up and running with Eclipse 3.5

In a previous post, I mentioned the recent release of version 3.5 aka Galileo of the Eclipse Builder. Since that time, I have found several Galileo-compatible plug-ins that make it worthwhile to make the leap to 3.5 now rather than later.

I installed Eclipse 3.5 with the latest Graphical Editing Framework tools, 3.5.0, and the latest Aptana Studio, 1.5.1. All together this makes for an invaluable core development platform. Installing it took some trial and error, so I will share with you the successful install process I used. It took several error-prone attempts to get a clean build. Now it works just super. Here's how:

• Download Galileo 3.5 to your downloads folder and extract the zip
• Download GEF 3.5.0 from the downloads page.
• Extract the GEF 3.5.0 zip locally and add the folder resources from your unzipped GEF 3.5.0 into the corresponding Galileo 3.5 folders. Do not overwrite the folders, as sometimes happens when you unpack an archive into the destination folder. In other words, mixing these together is best done by hand.
• Run Galileo, which should fire up error free. So far so what. Now plug in some Aptana goodness. This is best done using Help>Install New Software... In the field marked Work with:, don't bother with 'Add...', just plug http://update.aptana.com/install/studio directly into the field and look for the Aptana Studio checkbox to appear below.

If this all works, you are good to go. The time saver of this how-to may not be apparent, so let me add for those who try different methods that automating the installation of GEF in one of the many apparently valid ways of doing this...will not succeed. Or at minimum, they will not succeed under OS X and Galileo Classic in Cocoa 32 bit, which is the version I opted for.

Hang loose, amigos.

Flash Actionscript vs Open Source Javascript Animation

The eye does not deceive: javascripted animation still has significant limitations. While it's perfectly suitable for image fades and graduated motion via jQuery or similar packages, more liberal applications tend to cause the jitters.

At the heart of this is a lack of graphics operation handlers - helper functions to assist processor-intensive math and graphics-array operations before they reach a bottleneck. Such co-processing operations are a cornerstone of rich media plug-in technologies like Flash and Silverlight.

In spite of this, it's interesting what can be accomplished with development kits like Processing. An open source animation suite and "electronic sketchbook for developing ideas," Processing has gained many followers in the realm of academia, while the down and dirty commercial market continues to patronize plug-in solutions.

With browser capabilities on the rise, there is a huge upside for better native animation in future browser versions. Consequently, the open source community is poised to gain more animation and graphics programming devotees.

Update: Google's Chrome is on the move with native graphics processing in javaScript. Have a look at this example via drawlogic that uses the HaXe (pron: 'hex') version of the Sandy library. You will need to launch Chrome to see it. These are some bold smoke signals for Adobe to heed.

Version Control with Subversion

Setting up version control for, among other things, Flex code repositories, is not too hard to set up with Mediatemple (dv) accounts. SVN is already ready to run from SSH.

I followed these steps to get it up and going. What's more, it works.

• Svn on mediatemple knowledgebase article is here
• Great overview on bit-101.com

Good luck with your own implementations.

Update: I set up a SVN repository server on my account and it is simple to use in Eclipse projects. Once your repository is configured, whenever you start a new project, simply right click it in Eclipse, then choose Team > Share Project... All you have to do is enter your username and password, and the IDE will take care of starting a new svn repos. The dark master will be pleased...

Django AMF Service

Now that the basic Django framework is in place (admin, views, urls), the focus shifts to data and services. In the past, WebORB has been adept at providing RPC between ASP and RIA, so it's tempting to use it again in a Linux environment.

Since the current project is Apache, one option is to utilize Apache Tomcat. Tomcat is readily available in MediaTemple's (dv) configuration to provide a pure Java HTTP server environment. This would pair well with WebORB for Java, a robust middleware solution for RIA data synchronization.

Good enough but the tiers don't fully align with the planets yet. My goal is to utilize remote procedure calls to mySQL and take advantage of the schema I've built in Python-based Django. That will not be possible until my Python objects can be called from Java. So, is it possible to port Python code to WebORB via a Python -> Java compiler?

In a word, Jython.

Jython runs in Tomcat as an implementation of the Python programming language. By compiling the Django code, Jython could route between Flex on the front end and the mySQL admin system within the framework of Django.

Incidentally, I'm not going to implement this cookbook recipe just yet. I've worked with WebORB and am sold on it's strengths, but in this instance I'm looking for a lighter and more agnostic solution for the frontend than a uniform rich presentation server.

Namely, I'm going to move forward with Django AMF PyAMF. This allows some of Django's front-end capabilities to shine, and also enables javascript and ajax elements to be plugged into the site design.

More posts to come on this direction.

Out of curiosity, has anyone else tried the setup I outlined above, i.e. Django <-> Java <-> RPC tier <-> RIA? Furthermore, would you go a completely different route or change any of the tiers from the architecture I outlined? It is one of countless possibilities, and on that note, in my experience, one of the most direct and stable available.

Django MVC Basics Pt 1: URLconf

The URLconf - mapping between URLs and view functions that should be called for those URLs.

from django.conf.urls.defaults import *

# Uncomment the next two lines to enable the admin:
# from django.contrib import admin
# admin.autodiscover()

urlpatterns = patterns('',
# Example:
# (r'^mysite/', include('mysite.foo.urls')),

# Uncomment the admin/doc line below and add 'django.contrib.admindocs'
# to INSTALLED_APPS to enable admin documentation:
# (r'^admin/doc/', include('django.contrib.admindocs.urls')),

# Uncomment the next line to enable the admin:
# (r'^admin/', include(admin.site.urls)),
)

Important to note are:
r - Python indication that the string is a "raw string", i.e. do not interpret characters.
' ' - string enclosed in single quotes
^ - caret, a regular expression meaning "require that the pattern matches the start of the string"
$ - dollar sign, regex meaning "require the pattern match the end of the string".

e.g. ensures this works: www.url.com/hello/
and this does not: www.url.com/hello/there (thanks to $)
and this does not:www.url.com/there/hello (thanks to ^)

See table of basic Regular Expressions - from the Django Book.

The second parameter in the 2-tuple sometimes begins with include(... and sometimes does not. It is the function to be associated with the specified URL. The function's first parameter is always the HttpRequest object. Always. Here's documentation on the HttpRequest object's API.

Last (?) post about MediaTemple (dv) and Django

This is a refinement of a previous post on the topic of Django integration into a MediaTemple (dv) account. Following a talk with MT and reading some knowledgebase info, here is the quick route to Django configuration happiness - now more better.

This is how we do it:

Add to base config aka /etc/httpd/conf/httpd.conf the following in the appropriate section:

LoadModule wsgi_module modules/mod_wsgi.so

And in the appropriate section, add this too:

NameVirtualHost *:80

#Add a VirtualHost Section for each Name-based virtual server

<VirtualHost>
ServerAdmin email@email.com
ServerName www.URL.com
ServerAlias URL.com *.URL.com

DocumentRoot /var/www/vhosts/URL/httpdocs

<Directory /var/www/vhosts/URL/httpdocs>
Order allow,deny
Allow from all
</Directory>

</VirtualHost>

then save the file and exit the editor.

In the directory of path '/var/www/vhosts/(DOMAIN NAME)/conf' there is a file called 'httpd.include.' It holds unique config info for each virtual domain. It is rebuilt automatically from time to time based on Plesk. Instead of editing that file, make a new one called 'vhost.conf' (if it doesn't already exist) and add the following info to this file:

WSGIDaemonProcess URL.com processes=1 threads=1 display-name=%{GROUP}
WSGIProcessGroup URL.com
WSGIScriptAlias /myapp /home/username/django/scripts/URL.com.wsgi

<Directory /home/username/django/scripts>
Order allow,deny
Allow from all
</Directory>

ErrorLog /var/log/URL.com-error_log
CustomLog /var/log/URL.com-access_log common

Notes on the above, in brief

1. LoadModule is in httpd.conf
2. NameVirtualHost declaration is in httpd.conf
3. virtualhosts make use of /virtualhostname/conf/vhost.conf
4. WSGIScriptAlias declarations are in respective vhost.conf, not in httpd.conf*

* This is because WSGIScriptAlias in httpd.conf didn't work

Once that is configured as you like, simply restart the server in this manner:

/usr/local/psa/admin/sbin/websrvmng --reconfigure-vhost --vhost-name=mt-example.com 

service httpd graceful

Now test your django. With any luck, you will see a screen confirming a correct installation.

Eclipse Galileo Drops!

Version 3.5 of the Eclipse Integrated Development Environment application has officially been released.

What is Eclipse? A free open-source software development platform you can install on mac, win or linux.

What does it run? Eclipse was originally written to support Java, but now it has language packs for over a dozen languages.

What is Galileo? For several years now the Eclipse Foundation, an organization of open source Eclipse developers, have released a new iterative version of Eclipse late June of each year. Galileo, code name for the latest Eclipse version, officially dropped a couple of days ago. It's version 3.5 of the venerable platform.

What do you use Eclipse for? Currently, I code Actionscript in Eclipse using the Flex SDK plug in. It streamlines all the complexities of developing RIAs, and it also incorporates Subversion, an open-source version control app.

What's next? Now that Galileo is officially released, is it worth upgrading to? Turns out, lots of applications are not ready to support it yet. I've found several blogs documenting attempted upgrades in Linux, Windows and OSX that ended in grief, and not just regarding the Flex Builder plug-in. Be sure to research compatibility before getting too deep into a new version of Eclipse.

Python Image Library (PIL) on MediaTemple (dv)

When you're on MediaTemple (dv) with root access, first get the TAR from here. As of this post, the latest version is 1.1.6.

wget http://effbot.org/downloads/Imaging-1.1.6.tar.gz

Untar in the usual way.

tar zxvf Imaging-1.1.6.tar.gz

Then, go into the created folder and install the package using:

python setup.py install

If all goes well your ssh will echo many installation processes, following which you will have an amazingly easy time working with images via http.

Subversion on MediaTemple (dv)

Just completing some upgrades to my service to incorporate subversion. I don't know how I got by without it for so long. My first exposure to version control software was Visual SourceSafe back in the dark ages, and since then I've either used that or similar routines involving backups and multiple versions in similarly-named folders. Subversion is a lean, command-line, open-source version of the same. Here's what I did.

In the case of MediaTemple, Subversion is already installed. All you have to do to get started is invoke a new instance on the command line.

svnadmin create /name/of/project/folder

Then build a project to import into svn, such as

tmp/project/trunk
tmp/project/branches
tmp/project/tags

Jump into the trunk and add some files to it

file1
file2
file3

When you import the project, subversion vacuums it all up.

svn import /tmp/project file:///name/of/project/folder -m "initial import"

Note the -m, ensures you won't get an error for not setting the default text editor for subversion. In case you would like to set it, just use the command:

export SVN_EDITOR=vi

Now you can checkout files using the checkout command

svn checkout file:///name/of/project/folder/trunk project

And you can post your revisions using commit

svn commit

Notice the difference in directory structure between creating, importing and checking out. Creating and importing require only the folder, but checking out requires the additional trunk folder and project name specified.

Here are some additional resources of note for subversion installations, with an emphasis on MediaTemple (dv) installations.

Ayman Hourieh Subversion Tutorial - quick and to the point.

Tony Spencer's Tutorial - includes basic WebDAV integration, also quick and to the point.

Python and Django in MediaTemple (dv) server

Getting Django to run on a dv server is not as easy as their preconfigured - and cheaper - grid containers. While the shared grid containers are ready to go out of the box, the dv has some server admin changes to get through before you see that welcome

"Hello World".

Specifically, the httpd.conf needs additional lines of code to connect the web root to the Django application. To add a little spin to it, there's also the config file generated by Plesk. That needs some edits as well.

The order of statements in the zz010_psa_httpd.conf file can be a particular obstacle. Seems you have to have the WSGIScriptAlias declarations high up in the page or they aren't acted on properly as the procedure continues. Changing LogLevels to info gives the most detailed view from your logfiles but may not ultimately identify the problems.

In any case, here are the steps in a nutshell:

(added to zz010_psa_httpd.conf - plesk)

Include /var/www/vhosts/urlone.com/conf/httpd.include
Include /var/www/vhosts/urltwo.com/conf/httpd.include
WSGIScriptAlias /myapp /var/www/vhosts/urlone.com/wsgi-scripts/myapp.wsgi
WSGIScriptAlias /django/test01 /var/www/vhosts/urltwo.com/wsgi-scripts/myapp.wsgi

(added both httpd.includes to file structure)

Seems simple enough once it works. I hope this gets you out of a pinch.

Serving Django on Apache

As an update on the previous post, looking into current Django docs, the Apache production server method of using mod_python to serve Django has been superseded by mod_wsgi. I had partially implemented the former method, so now I'm bringing it up to the latest spec.

According to google.code on Python WSGI, Apache 1.3, 2.0 or 2.2 are compatible. Apache on a MediaTemple dv account shows

httpd -v

Server version: Apache/2.2.3
Server built: Nov 12 2008 10:41:27

Python needs to be 2.3 through 2.5 and the vers on (mt) shows

Python 2.4.3 (#1, May 24 2008, 13:47:28)

So far, so what. Now to install wsgi...

wget http://modwsgi.googlecode.com/files/mod_wsgi-2.5.tar.gz
tar xvfz mod_wsgi-2.5.tar.gz
configure
make
make install

finally edit

httpd.conf

located in

/etc/httpd/conf.

Adding the lines:

LoadModule wsgi_module modules/mod_wsgi.so

and


<virtualhost *>
ServerAdmin jasonthewolf@gmail.com
ServerName www.unplugtheinternet.com
DocumentRoot /var/www/vhosts/unplugtheinternet.com
WSGIScriptAlias / /django/test01/apache/django.wsgi
</virtualhost>

In the above, WSGIScriptAlias deserves some explanation:
Description : WSGI maps a URL to a filesystem location and designates the target as a WSGI script.
Syntax : WSGIScriptAlias URL-path file-path|directory-path
Context : server config, virtual host
Module : mod_wsgi.c

A request for http://www.example.com/wsgi-scripts/name in this case would cause the server to run the WSGI application defined in /web/wsgi-scripts/name. This configuration is essentially equivalent to:

and restart apache with

apachectl restart

All reports are good from Apache so far. Now that the installation of wsgi is complete, back to Django implementation.

I once again edited httpd.conf, this time adding
WSGIScriptAlias / /path/to/mysite/apache/django.wsgi

which for my site and requirements was effectively:

WSGIScriptAlias /django/test01 /var/www/vhosts/unplugtheinternet.com/apache/django.wsgi

This is the folder I have chosen for my python tests. If they work out, I'll move them to the root of some url or another. Following the convention established in the example, I created directory apache and the used vi to create the textfile django.wsgi as:

import os
import sys

os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Python and Django wrapup

Almost a wrapup.

Got django installed in /usr/lib/python2.4/site-packages/

Got django deployed in /var/www/vhosts/unplugtheinternet.com/django/test01/ with subfolder cms autogenerated

Got MySQLdb module added to python so django can use a MySQL database, and django has been successfully configured to build the schemas for the modules it deploys.

Created a vhost.conf to take care of webserver requests that python should handle (courtesy of Nick Sergeant's blog entry).

Still it isn't working properly. But since I'm leaving for a camping trip in ~1 hour, it's time to switch it off and go join my friends at Assateague Island.

Arrividerci pals.

MediaTemple, Python and Django

Just installing Django on my Mediatemple dv account.

First, log in to SSH as root user. Then use subversion, provided by mediatemple in

WebControl > Root Access & Developer Tools > Install Developer Tools,

which must at some point be enabled to be accessible. Following this,

Subversion grabs the latest version to /var/trunk/django.

In the case of dv accounts, python packages are located in /usr/lib/python2.4/site-packages

Create a symbolic link between these two locations using

ln -s /var/trunk/django /usr/lib/python2.4/site-packages

then add another symbolic link to bring the underlying python script to the root of the django trunk:

ln -s /var/trunk/django/bin/django-admin.py /usr/local/bin

Basically, what is happening here is:

Django Source /var/trunk : Subversion project root

\ --> accessible to /usr/lib : Libraries for programming and packages
\ --> accessible to /usr/local : Local hierarchy

More on this. Now:

import django

and


django.version

voila!

Next, you must add the MySQLdb interface for python if you are planning to use mySQL as your cms database. Here's a good reference for once you have the tarball in your directory. However, I'd rather wget it directly from the sourceforge site. Here is the command to use from root SSH:

wget http://downloads.sourceforge.net/mysql-python/MySQL-python-1.2.2.tar.gz

Works fine. Now install the module and it should run without an error.

A Good ImageCache for Flex Lists

A while back I brought to life a dream custom component I had wanted for a long time. It's basically a horizontal scrolling thumbnail image holder, the ubiquitous kind that's in everything from YouTube to Flickr. Mine is a little different in that it combines the following:

• Hover side-scrolling
• Accurate shuttle indicator below images
• Accu-sizes to any display object container width
• and finally...it caches the images for no lag

Pretty much does the job for me. I'll post it soon, happy to share and hear feedback on it. And on that note, I ran across a legacy of cache issue explorations and solutions by other flex scripters. The earlier one, which I noted but haven't tried out, is Ely's SuperImage from a couple of years back. It touches on the all-important performance issue of non-caching assets in Flex components.

The follow-up, which I'm stoked to note is lean, lithe and easy to plug in, is thanksMister's ImageCache. There's even a sample app using the Flickr API. Tested and hereby endorsed. Hope you get some use out of it.

Flex talking to Flash

Here's another good refresher on getting a Flex app synched up with Flash swf resources, this time at giantflyingsaucer.com. I'll post a link to a tutorial on Flex with Flash-compiled swcs as soon as I come across one. Any favorites out there?

Accessing swf vars from Flex

Ever need your memory jogged on the topic of putting swf vars into your Flex app? Here's just the thing for that:

Flash cs3 as3 file "click.swf"

ActionScript Code:

click_mc.addEventListener(MouseEvent.CLICK, reportClick);
var output:String = new String();

function reportClick(evt:MouseEvent):void
  {
      for(var i:int = 0; i<5; i++) {
          output += "I have been clicked. i = " + i + "\n";
      }
  }

MXML Flex 2.0

ActionScript Code:

<?xml version="1.0" encoding="utf-8"?>

<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute">

<mx:Script>
  <![CDATA[
      private function updateTextField():void
      {
          output_txt.text += swf.content["output"];
      }
  ]]>
</mx:Script>

  <mx:SWFLoader x="10" y="10" source="click.swf" scaleContent="false" id="swf" click="updateTextField()"/>

  <mx:TextArea x="10" y="518" width="550" height="211" id="output_txt"/>

</mx:Application>

Made for Flex 2 but perfectly applicable to 3, 4 and perhaps beyond... Thanks to the forums at actionscript.org for this one.

Poster REST App for Firefox

Pretty useful plug in here to allow what otherwise requires modules, DLLs or IDEs. Don't know how I got by without it for so long.

Get Poster

On the PS side, why do low-level service applications typically lack quick client integration options? My web services visualization tool is the browser, not a command line or SDK. So let's put that app into the browser.

Poster does that. Thanks to creator Alex Milowski. Read his blog.

Beginnings of an RIA ECommerce solution

Another round of tests and some interesting results. I have successfully stubbed the mySQL database into Flex via .Net procedures created by WebORB. Also, I have added fields to the mySQL tables of interest without any unique key violations in the Miva CMS. I had to add primary keys to several tables in the database, but as stated, no adverse affects, meaning the db queries in miva's cms are fairly fast and loose. In this case that is a good thing.

Still working on the details, but so far it seems possible to do the following:

• Use Miva Merchant 5.5 as a backend solution for a rich ecommerce (web) application
• Use Flex 3 as a client frontend with RPC enabled products arriving from Miva
• Add custom fields as needed to elaborate on the data the Miva CMS typically allows

That's a good start. The test verifies an architecture consisting of Miva -> MySQL -> .NET -> Flex on the incoming side.

Now I'm looking into solutions for the Flex -> Google Checkout -> Miva on the shopping cart transaction side. For now, going to checkout will out of necessity pop out of the RIA window and open another. It's a small price to pay, especially considering there are virtually no single-state checkouts anywhere on the web. It hasn't hit the production line yet.

Since Miva is stable with additional fields in its DB but cannot administer the extra fields, my wishlist of useful add-ons now includes a secure Flex-based CRUD-enabled CMS to manage the custom fields added to Miva DB tables. It would be great to be able to post images and data through a form in Flex, and very possible time-permitting. For now, the method is direct DB data injection and FTP posting. Not good for a client admin, but fine for a developer at this point.

And the point is, it works!

More Miva Merchant

There have been some interesting revelations here. What I've found can be summarized thus:

• Miva Merchant up to vers 4 uses a proprietary form of SQL called MIVA SQL. While it uses Structured Query Language, it does not share compatibility with MS SQL. Thus it is locked in its own fishbowl.
• Miva Merchant 5 and on to current 5.5 gives resellers the option of configuring their product to reposit with MySQL instead of MIVA SQL, which provides a substantial performance boost. My webhost of choice for this project opted for the MySQL configuration. This is good.
  
• Miva Merchant offers a range of export options for financials, including .dat files and the like. They do not currently offer any RESTful services of any kind. Thus the question of whether to REST or RPC for Miva is conclusively RPC or bust.
  

Contacting both HostMySite and Miva Tech Support in CA has confirmed the above. They recommend using Miva forums, but there is but scant info on this topic that I have found. The webhost tech department has been very cool and receptive to my questions and has custom configured my Miva-driven MySQL DB to allow me to connect and read data directly from it. This means there is finally a way to export data from the CMS portion of Miva and feed it to my custom Flex front end.

Now then.

The question remains, how will WebORB interpret the schema of this Miva-built DB? I hope well. Hand-coding RPC is out of the budget and timeline for this project. I am relying on the savvy of WebORB to translate the schema into useful server and client side code libraries such that, if all goes well, I can pull the inventory on the fly from Miva to my Flex application.

Between this and a lack of experience working with a MySQL .NET hybrid, I'm hoping it will go smoothly. I'm now creating the DSN to allow ODBC between the weborbconsole and MySQL, so I'll let you know what happens.

MIVA MERCHANT

Working on a Flex ecommerce solution. Need to provide the client with a full suite of merchant tools including:

• Pre-provided tax rate data on orders within the US
• Price and quantity inventory info
• Fulfillment and basic CRM features
• Analytics
• Easy-to-use content management
• API to Google Checkout
  
• Portable to external spreadsheets and financial software
• Secure credit card transactions (natch)

Plenty of solutions offer the above. The catch is, in order to provide my own client side in Flex I require a direct db connection, preferably to SQL, to supply a stream of remote data to my shopping UI. I have created my own CMS that will do the job, but there are ornery caveats associated with an architecture including 2 repositories of data, my CMS and the ecommerce CMS, namely:

• consistency of state across tiers
• concurrency
• locking
• transactions
• rollback

Dual databases for one application is seldom if ever ideal, and certainly not in this case. Trouble is, unless there is some way to communicate between Flex and the Ecommerce solution, 2 databases are the only way to roll it out. But what form would this communication take? In the case of Flex, there are three services available for accessing data:

• HTTPService: Passes textual data over HTTP. This is often used to fetch RSS feeds, read XML generated by a server, or simply pass text-based data from server to client.
• WebService: Allows a Flex application to invoke server-side web services using SOAP and handle the results of web service calls. These web services may reside on the server that has served the rich Internet application or may call third-party web services through a proxy on the server.
• RemoteObject: Allows a Flex application to directly invoke methods on Java classes residing on the application server that has served the RIA. Data is transferred in binary format (Adobe Action Message Format or AMF) over HTTP or HTTPS, with the server translating Java and ActionScript objects as they pass over the wire.

That's a pretty rich options list. I can use RSS feeds, XML, Web services or even direct DB access via RemoteObject. Lots of options, yes, options galore. As for the custom CMS I mentioned, it is .NET and uses RemoteObject to pull from the SQL database. Now I seek a similar means of pulling data from the ecommerce system. Any will do. But as it appears, there are few options.

I have thorougly scoured practical ecommerce, a website devoted to ecommerce application vendors. Practical ecommerce website has a Cart of the Week feature in which they interview principals of a web ecommerce provider each week, weighing the pros and cons of their service. There are open source solutions, Microsoft solutions, Adobe solutions, many variants. They can be hosted or licensed options. I've read through many of them.

In addition, I have signed up for Miva Merchant 5.5 with the reseller that hosts my client site. They are one of the 800 pound gorillas of ecommerce, Google Checkout being the reigning heavyweight, but not a full service solution as Miva offers.

Miva Merchant offers database integration now, as of 5.5, via the services of MIVA SQL and optional MySQL support, but here's the rub. Try a google search for "miva sql" and venture a guess at the richness of answers google will provide. At my last search...462 pages. A pathetic level of information out there to say the least, and equally untenable are the docs offered by Miva. Here are a few plaintive queries that died on the vine on my hosting account forums:

• Where is the database?
• More database questions
• Database q's answered by tech support

I've put in a support request ticket and will continue to pursue the issue. For my stake in this, at the root of the matter lies a question of Flex and Ecommerce integration. I ask the ecommerce vendor community collectively, is there a solution that offers a full set of features as well as database access? Has any of the companies seen the value in offering a solution that can serve as a data feed and cms, or conversely are they all template-and-markup-based solutions? I'll post more once I have some concrete answers to this.

More Bugs In App Development

Encountered a scenario recently that underscores the difficulty of working in a multi-tier web app architecture. Recent project was SQL to .NET CMS and a Flex client side, spanned by Weborb.

Problem came about when I updated the .NET CMS, suddenly the front end flex app started throwing the application error, ____, stating a method in my weborb-generated code was coming up null. This was strangely the case not only for my code, but for the autogenerated testdrive as well.

I chalked it up to a misnamed package on either the server or client side, later to find that troubleshooting it was not helpful. I replaced the service locator xml files from my base install, which also did nothing to help.

Finally I built a test case and slowly bridged it from working to nonworking case, using varying test cases similar to the nonworking build. After lots of wasted time, I found what is either a memory leak, or an Eclipse bug.

Either way, I had to work around it and ultimately create a new build that is identical - as far as I can tell - to the nonworking version. Only it works.

The tier problem I speak of is the many levels of unknown in the overall architecture. It is difficult and time consuming to isolate layers one at a time to find the culprit. It would suggest that rich media apps require a team of specialized workers. Takes too long.

Bug in Weborb 3.6 for .NET prevents WDMF from generating server code

Here's an issue that could cost you lots of troubleshooting time to fix. While weborb is a stellar product IMO it is lacking in thorough support docs. Probably that is fine since a commercial client can purchase phone support, but I digress. The problem aka bug aka omission I refer to in weborb 3.6.0.3 results in:

Code Generation Failed

This is the result of an attempt to generate the all-important client-server serialization classes that weborb builds. The client classes are downloaded from the weborbconsole while the server classes are written to the bin directory assuming your permissions are properly set to allow this.

Permissions are easily set via Windows IIS or your webhost control panel. Permissions are also easily reviewed from your website of interest thanks to the diagnostics.aspx page provided. Potential issues are even coded red. Easy enough.

Problem is essentially the result of a wishlist item added to Weborb 3.6 and released 12/02/08, Visual Studio templates. I didn't even expect to find them, but did so and began enjoying the ease of deployment templates offer. Only problem is, they won't work unless you add the XSLT files in the ODBC directory from the original full vers of weborb30.

I tried other stuff for a while to fix the prob, then finally read the error text logs on my deployment and discovered the omission. As stated before, Weborb is the Barry Bonds of remoting, a real slugger for joining up RIAs for web and I can't say this enough. The fact that I could rely on the framework tools to hunt down the error and fix it is one reason why. Now, has anyone else had this problem? If so, I hope this helped.

Fig 1. Diagnostics aspx page for weborb









Fig 2. Creating a weborb-enabled website in Visual Studio 2008 using the weborb-provided template











Fig 3. The missing odbc folder, where it should be









Fig 4. What you should see if all goes well in the weborbconsole WDMF

SQL & LINQ for n-tier applications

As I've blogged a lot lately, WebORB bridges back end and presentation servers for AMF and DB CRUD functionality. Furthermore, once WebORB is configured properly, it deploys server and client code in an automated fashion, allowing this to happen. If you manage to rig it properly, it binds your data via method calls to the DB.

In developing an n-tier application, it's crucial to plan in advance how the tiers will provide for one another in the overall application chain. Justin J. Moses' blog of March 19th outlines the strength of utilizing SQL with LINQ in such a way. I especially enjoyed his definitions of generalized data transfer strategies as being either safety, hungry, greedy, or thrifty. Well done!

Since his description is more than adequate I won't go into it here, but seeing as it's a likely fit for my current project, I may post some notes later on data across the pipe with RIA.

ASP.NET MVC is the last gasp

I started web development with open source as long as 10 years ago. Then it was PHP on an Apache server and I've continued through PHP 5 creating everything from mySQL databases to custom frameworks and CMS. Not too long ago, a new client project required ASP.NET technology, so I learned how to use .NET too. Here are some pros and cons.

Pros: Lots of documentation. Large developer community. Many adopters in the corporate realm.

Cons: Several. One particular blog I found nailed the issue. It's a matter that parallels the PC vs Mac debate. In more abstract terms, it's a question of

Goal-Oriented vs Process-Oriented

Stripping away the layers of good arguments on all sides, I have found Microsoft solutions tend to favor process over goal orientation. That's not good for independence and it's not good for speed.

A specific example.
You're a developer intent on deploying a web application quickly using whatever tools are best.
With Apache and PHP, the process is contained discretely within the server environment.
With Windows and .NET, there is a ripple effect requiring the expertise of the IT department. You need to know all about system permissions, a great deal about SQL management. Really, lots of peripheral management issues come to drag your process to a crawl.

That aside, there are a lot of smart people using .NET, and I'm interested in learning more about quick ramp-ups for .NET solutions. I'm a fan of Visual Studio's GUI and the xml DataSet system in .NET. I only wish they were quicker to adopt the DRY principles of RoR in their framework.

Since most of my clients fall into the small to medium category, unless they are running 100% enterprise solutions, I recommend a PHP or RoR orientation.

I will still be looking at .NET MVC resources to see what they've done to address quicker CRUD deployment. It begs the question, what open-source solution will Microsoft roll out years after the competition next? That is why edgy developers tend toward Apple and Open Source.

SQL Schema Backups

I've got plenty of ebooks, some of which I've looked over, some not so much.

In general, due to over-verbosity, ebooks are a next-to-last ditch effort. I consult the ebook when the following has failed (usually in this order):

• Relying on past experience
• Googling for answers
• Regoogling with site-specific filters and phrases
• Trying bundled help files of application in question
• Forums
• Possibly contacting a help desk via online chat window

Finally...the ebook. If an issue gets to this point, I'm up the creek.

Problem with forums is, they're often vague and fragmentary. I'm not knocking - just looking for a little more meat on the bone.

The best answers are brief and demonstrative with plenty of screenshots. Bonus points if the writer is being engulfed by a green jellyfish.

WebORB : words from the author

Hear WebORB principal and lead architect Mark Piller talk about his product on theFlexShow.

What I found interesting:

• Detailed description of the AMF (Action Message Format) binary data protocol
  
• WebORB Business model is set up for small to medium businesses*

Good question by host "Why have AMF when there are other options?"

All options based on the HTTP(rotocol). The three basic options are HTML, Web services (SOAP, JSON, etc) and Remoting. One differentiator with Remoting and consequently AMF is its performance advantage. Another is the rapid data transmission it offers. Finally, security is a big part of it. Methods in Html and Web Services require security metadata, such as lock and key pairs, ciphers and such, so that if packets are intercepted, they cannot be compromised. Remoting on the other hand has easier security management. The container is running on the server side, so it is responsible for serializing the data before it is released into any security-sensitive zone. Interesting considerations.

Podcast has more.

WebORB Synopsis Pt. 1

WebORB by themidnightcoders, aka Dallas-based Mark Pillar and company, is a software application designed to facilitate and ease RIA remoting. Over its livespan, WebORB has grown to its current ambitious version (3.6) to become a sophisticated multi-tool of sorts, providing a broad range of services for a pastiche of server types.

How does a product such as WebORB come to be? Weborb is in fact one among a stable of solutions (amfphp, swx, BlazeDS, RubyAMF, LiveCylceDS et al) initiated to bridge the expanse between server side and client side in rich applications. The options in this lineup range from Open source and free to commercial and costly, from single protocol to advanced suite. WebORB among them could be said to toss the biggest lasso.

WebORB marries server technologies .NET, PHP, Rails and Java to presentation server clients Flex, Flash, AJAX and Silverlight. It allows the server side to do what it does best - provide a data bank and administration, while allowing the client to benefit from a live data source. There's a lot more on offer than simple connectivity, however.

WebORB can be deployed in modular fashion to an existing web application. Once integrated, it includes a proprietary console GUI to demo its many capabilities. As is explained in the console, WebORB offers three broad categories of connection services:

1. Remoting (AMF3)
2. Data Management (WDMF)
3. Real-time Messaging

Among these, it is fitting to begin with remoting, as remoting was the initial goal of applications like WebORB. In its maturity as a service, invocation of remoting is bidirectional, fully interoperable, reflexive and essentially transparent. In addition to basic transfer of assets, WebORB remoting includes class mapping, secure channels of invocation, tree control with remote data, server-side exception handling, logins and security, and supports ARP, an open source framework for Flash and Flex initially authored by Aral Balkan.

WebORB effectively encapsulates the full capabilities of Flex Remoting, and in so doing, it lays bare the limitations of Remoting. Namely, that shuttling large serialized datatypes between environments is crude. It results in a loss of organizational nuance, such as hierarchies, relationships, dependencies. Put another way, the schema is not transferred with the data, and thus the sorting abilities that are the inherent value of a data system are lost. Data Management can be viewed as a means of addressing this limitation.

Data Management, aka WebORB Data Management for Flex or WDMF, is a means of preserving the relationships surrounding remote data between server and client sides. In a nutshell, WebORB imports table relationships directly from the server database and utilizes its own innovative analytical methods to create all the necessary interoperability through scripts it generates for both sides. In this way, WebORB enables explicit relationships to exist through a generative act of surrogation. In short, it analyzes, it marries, it step back into the shadows. Not bad, Midnightcoders, not bad. An invaluable utility and a tremendous time saver, Data Management also includes full CRUD, enabling the rich client to act as CMS if desired.

Real-time Messaging is another facet of WebORB to be explored later. Feel free to visit themidnightcoders site to learn more.

Adios for now.