Mapping xml to Python objects

I need to process XML in my python code and prefer to deal with the data in the pythonic way. From a wide options available, I choose to play with 3 of them.

* generateDS
* gnosis.xml.objectify
* amara

Because I am a fan of XmlBeans, it seems that generateDS should be my favorite choice. However, when I tried to run the code-generating script on my schema file, it encountered some recursive problem and complained:

.... parent.collectElementNames(elementNames) File "C:\Python\Lib\site-packages\generateDS.py", line 478, in collectElementNames parent.collectElementNames(elementNames) File "C:\Python\Lib\site-packages\generateDS.py", line 475, in collectElementNames base = self.getBase() RuntimeError: maximum recursion depth exceeded

The same schema file works fine with XMLBeans however.

gnosis.xml.objectify and amara are similar in that they don't require a schema file and will just turn any well-formed xml file into python objects. I had no problem parse the xml files with both modules. However, the gnosis.xml.objectify module seems not aware of the namespace prefix. Given following xml element:

<ns:book xmlns:ns="http://mycompany/bookproject">..

It generates an object called ns_book. It does not recognize : as a namespace prefix and just blindly translates it to _. I tried to search for a solution, but could not find anything.

The amara module, however, correct translates the object name to book and also keeps the namespace information. It seems that I will use this module at this time.

2 tips about JBoss

* to retrieve the server name, such as "minimal", "default", "max", in your code

String serverName = System.getProperty( org.jboss.system.server.ServerConfig.SERVER_NAME); 

* to set an unique Xid for each JBoss instance

if you are running multiple JBoss instances on the same machine, you probably want to give each instance a unique Xid base name. To do it, you need to add
the following line in the jboss-service.xml:

<mbean code="org.jboss.tm.XidFactory" name="jboss:service=XidFactory">
    <attribute name="BaseGlobalId">UniqueInstanceName</attribute>
    <!--attribute name="Pad">true</attribute-->
</mbean>

Successfully installed Twisted

I need to install the latest Twisted with Python 2.4. Unfortunately I don't have Visual Studio on my machine. Is there a way to do it. I find this excellent instruction on how to install Python extensions without VS. It works as advertised. I am happily running Twisted now!

Upgrade to CruiseControl 2.4.0

Today I upgraded my CruiseControl to 2.4.0. The main reason is my project's SCM has been moved from Visual Source Safe to Perforce and I cannot get the CruiseControl 2.2.1 to work with Perforce. After the upgrade, everything comes back to normal.

Python web service

In my previous post, I have discussed using Perl (SoapLite) as client to consume a web service implemented using Java(Apache Axis). I am curious to find how it will work out for other lanuages such as Python and Ruby. Here is my try for Python.

The tool I use is ZSI which stands for Zolera SOAP Infrastructure. It provides a tool wsdl2py that can generate Python stub code from a wsdl file. Run the tool on the wsdl file described in my previous post, I get two Python modules: MessageRouter_services_types.py and MessageRouter_services.py. With the help of these two modules, consuming a web service is straight forward. Here is the code:

def sendTo(destination, payload):

 loc = MessageRoutingServiceLocator()
 kw = { 'tracefile' : sys.stdout }
 portType = loc.getMessageRouterPortType(**kw)

 dest = ns1.DestinationType_Def()
 dest._protocol = "http"
 dest._category = "post"
 dest._server = "127.0.0.1"
 dest._port = 80
 dest._target = "/router"

 msg = ns1.MessageBodyType_Def()
 msg._content = "payload"

 req = RouteRequestWrapper()
 req._destination = dest
 req._request = msg

 try:
  response = portType.RouteMessage(req)._response._content
  print "Response message is:\n %s" % response
 except Exception, e:
  response = repr(e)
  print "Failed: ", e
  raise e

Programming languages I use now

Programming languages I use

Java - the language I use to earn my bread-and-butter and the one I will choose to do serious work

Ruby - it's style is cute I think, and it is fun to work with,

Python - as powerful as it's name suggests, I have been bitten by the white spaces, but now recovered

Perl - there is more than one way to do it, but many times I prefer there is only one way

Complex test data setup in Fitnesse

I am creating functional specifications using Fitnesse for one of my systems. The system is a typical messaging system following the request/response pattern. Writing Fitnesse for this kind of system is straight forward and falls nicely into the BuildOperateCheck pattern. The only problem is that some types of requests are really really complex which could contain up to a few hundred different data fields. It is obviously not practical to setup all these fields in one table row using the ColumnFixture. I am thinking about how to deal with it.

One approach is to identify a small set of data fields that we will focus on testing on each page and create a fixture table that deals with only that small set of data. For the rest of the fields, we just assign some default values.

It is a reasonable assumption. Because on each Fitnesse page we usually test a specific requirement that most likely only involves several variables. For example, if I am designing a online book purchase system which is very complicated. But one of the requirement is to identify the tax rate. The tax rate are identified by the customer's postal code and county. Thus I only need to include "postal code" and "county" in my fixture table and I will be able to test that requirement.

In some rare cases, even that small set of data does not fit well in one table. In that situation, I managed to use multiple tables to collect the data for building a single request. It takes a little extra work but it makes the page more readable.

Fitnesse and coding convention

I have recently started to use Fitnesse to document the requirements of two of my projects. It is a wonderful tool that not only makes writing requirement more fun, but also makes the process works. Many of us are used to hundreds of pages of requirement document that only satisfies the "process police" but does little help to developers. The requirements written in Fitnesse is live and up to date. It just works!

But there is one thing I am scratching my head now which is the coding convention used by Fitnesse. For many fixtures, such as RowFixture, the convention is to match the column name to a public member variable in the fixture class. The traditional getter and setter approach is not supported here.

Many people oppose the idea of getter and setter and consider them evil. I am ok to either approaches. But the problem is the coding convention adopted by my organization is to have each member variable prefixed by the "m" and all member variables should be accessed through getters and setters. I cannot find a way to reconcile this problem. I won't be a problem to make a special case for this, but I really hope Fitnesse can support both approaches.