Successfully installed Twisted

I need to install the latest Twisted with Python 2.4. Unfortunately I don't have Visual Studio on my machine. Is there a way to do it. I find this excellent instruction on how to install Python extensions without VS. It works as advertised. I am happily running Twisted now!

Upgrade to CruiseControl 2.4.0

Today I upgraded my CruiseControl to 2.4.0. The main reason is my project's SCM has been moved from Visual Source Safe to Perforce and I cannot get the CruiseControl 2.2.1 to work with Perforce. After the upgrade, everything comes back to normal.

Python web service

In my previous post, I have discussed using Perl (SoapLite) as client to consume a web service implemented using Java(Apache Axis). I am curious to find how it will work out for other lanuages such as Python and Ruby. Here is my try for Python.

The tool I use is ZSI which stands for Zolera SOAP Infrastructure. It provides a tool wsdl2py that can generate Python stub code from a wsdl file. Run the tool on the wsdl file described in my previous post, I get two Python modules: MessageRouter_services_types.py and MessageRouter_services.py. With the help of these two modules, consuming a web service is straight forward. Here is the code:

def sendTo(destination, payload):

 loc = MessageRoutingServiceLocator()
 kw = { 'tracefile' : sys.stdout }
 portType = loc.getMessageRouterPortType(**kw)

 dest = ns1.DestinationType_Def()
 dest._protocol = "http"
 dest._category = "post"
 dest._server = "127.0.0.1"
 dest._port = 80
 dest._target = "/router"

 msg = ns1.MessageBodyType_Def()
 msg._content = "payload"

 req = RouteRequestWrapper()
 req._destination = dest
 req._request = msg

 try:
  response = portType.RouteMessage(req)._response._content
  print "Response message is:\n %s" % response
 except Exception, e:
  response = repr(e)
  print "Failed: ", e
  raise e

Programming languages I use now

Programming languages I use

Java - the language I use to earn my bread-and-butter and the one I will choose to do serious work

Ruby - it's style is cute I think, and it is fun to work with,

Python - as powerful as it's name suggests, I have been bitten by the white spaces, but now recovered

Perl - there is more than one way to do it, but many times I prefer there is only one way

Complex test data setup in Fitnesse

I am creating functional specifications using Fitnesse for one of my systems. The system is a typical messaging system following the request/response pattern. Writing Fitnesse for this kind of system is straight forward and falls nicely into the BuildOperateCheck pattern. The only problem is that some types of requests are really really complex which could contain up to a few hundred different data fields. It is obviously not practical to setup all these fields in one table row using the ColumnFixture. I am thinking about how to deal with it.

One approach is to identify a small set of data fields that we will focus on testing on each page and create a fixture table that deals with only that small set of data. For the rest of the fields, we just assign some default values.

It is a reasonable assumption. Because on each Fitnesse page we usually test a specific requirement that most likely only involves several variables. For example, if I am designing a online book purchase system which is very complicated. But one of the requirement is to identify the tax rate. The tax rate are identified by the customer's postal code and county. Thus I only need to include "postal code" and "county" in my fixture table and I will be able to test that requirement.

In some rare cases, even that small set of data does not fit well in one table. In that situation, I managed to use multiple tables to collect the data for building a single request. It takes a little extra work but it makes the page more readable.

Fitnesse and coding convention

I have recently started to use Fitnesse to document the requirements of two of my projects. It is a wonderful tool that not only makes writing requirement more fun, but also makes the process works. Many of us are used to hundreds of pages of requirement document that only satisfies the "process police" but does little help to developers. The requirements written in Fitnesse is live and up to date. It just works!

But there is one thing I am scratching my head now which is the coding convention used by Fitnesse. For many fixtures, such as RowFixture, the convention is to match the column name to a public member variable in the fixture class. The traditional getter and setter approach is not supported here.

Many people oppose the idea of getter and setter and consider them evil. I am ok to either approaches. But the problem is the coding convention adopted by my organization is to have each member variable prefixed by the "m" and all member variables should be accessed through getters and setters. I cannot find a way to reconcile this problem. I won't be a problem to make a special case for this, but I really hope Fitnesse can support both approaches.

Stomp

I have previous written about using web service as a mediator to let languages other than Java to communicate with JMS server. The approach is only for simple usage such as a testing tool. For more advanced usages, we can consider Stomp whcih is an open messaging protocol. There are various clients exist for Perl, Python and Ruby. On the server side, however, there is only ActiveMQ support.

Make Perl speak JMS (3)

Finally I will implement a Perl web service client. The tool I pick is SOAP::Lite (v0.55).

SOAP::Lite comes with a handy tool similar to wsdl2java called stubmaker. Run our WSDL through the tool and it generates a Perl module MessageRoutingService.pm. This module wrapps the call to the single RouteMessage API defined in the WSDL. Following code shows how to use this module:

my @dest = ( 
 SOAP::Data->name(protocol=>'jms'),
 SOAP::Data->name(category=>'queue'),
 SOAP::Data->name(server=>'my_jms_server'),
 SOAP::Data->name(port=>'7222'),
 SOAP::Data->name(target=>'my_queue'),
 SOAP::Data->name(user=>'user_a'),
 SOAP::Data->name(password=>'password_a') );

my @content = (SOAP::Data->name(content=>'Hello World'));
my @route_request = (SOAP::Data->name(destination=>\SOAP::Data->value(@dest)),
 SOAP::Data->name(request=>\SOAP::Data->value(@content)));

my $result = MessageRoutingService->RouteMessage(@route_request);
print $result->{content};

Sounds easy? There is a small catch here. The generated stub module needs a little tweak. Somehow it misses the URI namespace for the top method wrapper element .

my %methods = (
 RouteMessage => {
  endpoint => 'http://localhost:8080/common_router',
  soapaction => 'http://www.simple.code/router/action/RouteMessage',
  uri => '',
  parameters => [
   SOAP::Data->new(name => 'document', type => '', attr => {}),
  ],
 },
);

Without the URI, the method in the request looks like

 <RouteMessage xmlns="">

which is obviously wrong. After I manually add the URI in the stub to make it look like

my %methods = (
 RouteMessage => {
  endpoint => 'http://localhost:8080/common_router',
   soapaction => 'http://www.simple.code/router/action/RouteMessage',
   uri => 'http://www.simple.code/router/action/RouteMessage',
   parameters => [
    SOAP::Data->new(name => 'document', type => '', attr => {}),
   ],
  },
);

Now the request becomes

<RouteMessage xmlns="http://www.simple.code/router/action/RouteMessage">

and it is happily accpted by the Java Axis server.

Make Perl speak JMS (2)

Now the WSDL is ready, I can start to implement the WS server in Java. The tool of choice is Apache Axis 1.2.1.

With the wsdl2java tool, it is straight forward to generate the codes from the WSDL file. Because we will only use the server code, so we only need to change one class HttpBindingImpl.java.

private MessageRouter mRouter = new MessageRouter();

public code.simple.www.router._2005._11._7.RouteResponseType routeMessage
 (code.simple.www.router._2005._11._7.RouteRequestType document)
 throws java.rmi.RemoteException
{
 try
 {
  return mRouter.routeMessage(document);
 }
 catch(Exception ee)
 {
  throw new RemoteException("Failed to route the message", ee);
 }
}

Here we just forward the request to the MessageRouter class and let it handle the details.

Axis provides several different ways to deploy the web services. The most convenient way is to use JWS file but it has certain limitations. Here I choose to use the WSDD file and embed the Axis engine in the Tomcat servlet container. As of how to integrate Axis with a servlet container, I could not find a good documentation. Eventually I figured it out and here is the code

protected void doPost(HttpServletRequest req, HttpServletResponse resp) 
 throws ServletException, IOException
{
 PrintWriter pw = null;
 try
 {
  pw = resp.getWriter();
  resp.setContentType(CONTENT_TYPE);

  String reqMsg = readContent(req.getReader());
  mLogger.info("Got WS request:\n" + reqMsg);
  MessageContext msgContext = new MessageContext(mAxisServer);
  msgContext.setRequestMessage(new Message(reqMsg));
  msgContext.setTargetService("MessageRouterPort");

  String respMsgStr = null;
  try
  {
   mAxisServer.invoke(msgContext);
   respMsgStr = msgContext.getResponseMessage().getSOAPPartAsString();
  }
  catch (AxisFault fault)
  {
   Message respMsg = msgContext.getResponseMessage();
   if (respMsg == null)
   {
    respMsg = new Message(fault);
    ((org.apache.axis.SOAPPart)respMsg.getSOAPPart()).
     getMessage().setMessageContext(msgContext);
   }
   respMsgStr = respMsg.getSOAPPartAsString();
  }

  mLogger.info("Got WS response:\n" + respMsgStr);
  pw.write(respMsgStr);
 }
 catch (Exception me)
 {
  mLogger.error(me);
  throw new ServletException(me);
 }
 finally
 {
  if (pw != null)
  {
   pw.close();
  }
 }
}

public void init(ServletConfig cfg) throws ServletException
{
 super.init(cfg);
 EngineConfigurationFactory factory = 
  EngineConfigurationFactoryFinder.newFactory(cfg);
 EngineConfiguration config = factory.getServerEngineConfig();
 mAxisServer = new AxisServer(config);
}

The deployment descriptor for the servlet is:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
 "http://java.sun.com/dtd/web-app_2_3.dtd">
<web-app >
 <servlet>
  <servlet-name>MessageRoutingServlet</servlet-name>
  <servlet-class>code.simple.MessageRoutingServlet</servlet-class>
  <init-param>
   <param-name>servletId</param-name>
   <param-value>defaultServlet</param-value>
  </init-param>
 </servlet>

 <servlet-mapping>
  <servlet-name>MessageRoutingServlet</servlet-name>
  <url-pattern>/*</url-pattern>
 </servlet-mapping>
</web-app>

Make Perl speak JMS (1)

To add JMS capability to Plum, I need to find a way to let Perl talk "JMS". Unfortunately there is no standard wire protocol for JMS. The JMS spec. defines the Java API but not the wire protocol. This is understandable as JMS is a Java centric system. Some JMS implementations, such as JBoss 3.x and 4.x, just use the Java serialization as their wire protocol. Others may use their propriate formats. This is an effort to define a generic XML protocol that can be utilized by different JMS implementations, but it is not there yet.

For now, a relatively simple way is to create a web service in Java which will route any requests to JMS server. Then one can make any lanaguage to speak JMS as long as they can consume the web service. Of course the limitation is that only text message will be supported.

Here is my try. I created a simple web service which is defined by following WSDL:

<?xml version="1.0" encoding="UTF-8"?>
<definitions name="MessageRouter" 
 targetNamespace="http://www.simple.code/router.wsdl/2005/11/7"
 xmlns:wsdlns="http://www.simple.code/router.wsdl/2005/11/7"
 xmlns:router="http://www.simple.code/router/2005/11/7"
 xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 xmlns:stk="http://schemas.microsoft.com/soap-toolkit/wsdl-extension"
 xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
 xmlns="http://schemas.xmlsoap.org/wsdl/">

 <types>
  <xsd:schema targetNamespace="http://www.simple.code/router/2005/11/7" 
   elementFormDefault="qualified">

  <xsd:complexType name="DestinationType">
   <xsd:sequence>
    <xsd:element name="protocol" type="xsd:string"/>
    <xsd:element name="category" type="xsd:string"/>
    <xsd:element name="server" type="xsd:string"/>
    <xsd:element name="port" type="xsd:long"/>
    <xsd:element name="target" type="xsd:string"/>
    <xsd:element name="user" type="xsd:string" minOccurs="0"/>
    <xsd:element name="password" type="xsd:string" minOccurs="0"/>
   </xsd:sequence>
  </xsd:complexType>

  <xsd:complexType name="MessageHeadType">
   <xsd:sequence>
    <xsd:element name="header" type="xsd:string" maxOccurs="unbound"/>
   </xsd:sequence>
  </xsd:complexType>

  <xsd:complexType name="MessageBodyType">
   <xsd:sequence>
    <xsd:element name="content" type="xsd:string"/>
   </xsd:sequence>
  </xsd:complexType>

  <xsd:complexType name="RouteRequestType">
   <xsd:sequence>
    <xsd:element name="destination" type="router:DestinationType"/>
    <xsd:element name="head" type="router:MessageHeadType" minOccurs="0"/>
    <xsd:element name="request" type="router:MessageBodyType"/>
   </xsd:sequence>
  </xsd:complexType>

  <xsd:complexType name="RouteResponseType">
   <xsd:sequence>
    <xsd:element name="response" type="router:MessageBodyType"/>
   </xsd:sequence>
  </xsd:complexType>

  <xsd:element name="RouteRequest" type="router:RouteRequestType"/>
  <xsd:element name="RouteResponse" type="router:RouteResponseType"/>

  </xsd:schema>
 </types>

 <message name="RouteRequest">
  <part name="document" element="router:RouteRequest"/>
 </message>

 <message name="RouteResponse">
  <part name="document" element="router:RouteResponse"/>
 </message>

 <portType name="MessageRouterPortType">
  <operation name="RouteMessage">
   <input message="wsdlns:RouteRequest" />
   <output message="wsdlns:RouteResponse" />
  </operation>
 </portType>

 <binding name="HttpBinding" type="wsdlns:MessageRouterPortType">
  <stk:binding preferredEncoding="UTF-8" />
  <soap:binding style="document" 
   transport="http://schemas.xmlsoap.org/soap/http" />
  <operation name="RouteMessage">
   <soap:operation 
    soapAction="http://www.simple.code/router/action/RouteMessage" 
    style="document"/>
   <input>
    <soap:body use="literal"/>
   </input>
   <output>
    <soap:body use="literal"/>
   </output>
  </operation>
 </binding>

 <service name="MessageRoutingService">
  <port name="MessageRouterPort" binding="wsdlns:HttpBinding">
   <soap:address
    location="http://localhost:8080/common_router" />
  </port>
 </service>

</definitions>