JPz'log Coin Coin and Plop da Plop

8Jun/100

nose: testing in Python made easy

I have recently been doing some quick experiments, and chose Python to do that, mostly due to a need for efficient system-level programming APIs. Python just excels at that.

One thing that always annoyed me in Python is the relative poor state of testing frameworks. See, Ruby has RSpec, Groovy/Java have EasyB, Scala has ScalaTest, and much more.

By contrast, Python comes bundled with the unittest module which is basically a straight port of JUnit to Python. This is not the most Python-esque API around, and writing fixture classes can be overkill at times, especially when you are used to the elegance of a BDD testing framework like EasyB. What really surprised me is that there are actually few testing frameworks for Python.

I initially wrote a few tests using unittest, but got quickly bored by the API and the headaches for having the testing code being in another tree of the project, while still being able to import the modules that I wrote.

Fortunately some clever people wrote nose, a testing framework that just works awesomely well. It is actually a tests launcher rather than a testing framework.

It scans the arborescence from where you invoke it, looking for any directory and python file have "test" in its filename. When it encounters such Python file, it will look inside for unittest.UnitTest classes, docstrings,... and any function that also has "test" as part of its name. This gives you a great deal of flexibility, as it will also look for "setup_*" and "teardown_*" functions.

The other selling point in nose is that no matter how deep your test Python scripts / classes are located within your source tree, packages / modules resolution is made from the invocation directory. This means that your main source code folder can have the modules directories as well as a "tests" folder, which is not a Python module, that you can structure cleanly (think separating unit_tests, functionnal_tests, ...) and that will have no issues resolving the modules. Neat.

Finally, nose is flexible through its plugins: you can get HTML output, JUnit-style XML tests reports for continuous integration servers integration, code coverage reports and much more.

If you code in Python, I highly recommend that you give it a try!

Filed under: Uncategorized No Comments
18Jan/104

I like Spring WS (or SOAP made right)

I believe that the main backslash behind SOAP is that most implementations are deeply flawed. By that I mean that most SOAP implementors got it fundamentally wrong by mapping a RPC mindset over it, resulting in a flurry of frameworks and tools where:

  1. developers write their services as a class that exposes methods,
  2. the framework automagically expose it and generates a WSDL descriptor,
  3. developers write client code by generating a huge pile of verbose code from the WSDL descriptor,
  4. developers experience frequent failures (e.g., the WSDL changes because the service interface has slightly changed, or just because it has been put in production on another server, etc).

Some other errors include sporadic incompatibilities between frameworks that exchange SOAP messages (indeed, not everyone has the same definition of a method call and data types...). Fortunately, we have seen great efforts like the Web Services Interoperability Technologies to solve those incompatibilities issues. Yet, I find it quite ironic that people had to make a specification to solve incompatibilities for a technology that is expected to be interoperable and platform neutral...

Clearly, using SOAP for RPC with RPC-minded tool sets was a huge industrial mistake, as SOAP is fundamentally a messaging framework. Think of it as a platform-neutral and standards-based JMS over Internet protocols rather than RMI mapped over the Internet.

Over the years, SOAP frameworks have gradually added support for message-oriented APIs. As an example, JAX-WS supports both the "automagical" / flawed RPC mode, and a document-oriented mode where you can directly access the SOAP message XML structure. This has several advantages, including the fact that interoperability is simpler (every language / framework agrees on what an XML document is and how to deal with it).

In that respect, I found out that Spring WS is the best SOAP framework I came across in ages. Indeed, Spring WS enforces that you follow a "contract-first" and message-oriented approach. You can either write the WSDL yourself, or just the XML Schema definitions. Similarly, you can use any XML manipulation solution (plain DOM, SAX, Pull, JDOM, JAXB, ... whatever you like).

I did a demo in a recent lecture, so here are a few code excerpts to show you how clean it is if you don't mind the extra Spring XML configuration (this is where Java EE shines a lot BTW) :-)

The scenario was a simplistic contacts manager where you could add contacts and list them all. Nothing fancy here, but it nevertheless shows the strength of Spring WS.

The service implementation is a classic Spring bean split into an interface and an implementation:

package fr.insalyon.ws.contacts;
 
import java.util.List;
 
public interface AddressBook {
 
    public void add(Person person);
 
    public List<Person> all();
 
}
package fr.insalyon.ws.endpoint;
 
import fr.insalyon.ws.contacts.AddressBook;
import fr.insalyon.ws.contacts.Person;
 
import java.util.ArrayList;
import java.util.List;
 
import static java.util.Collections.unmodifiableList;
 
public class InMemoryAddressBook implements AddressBook {
 
    private List<Person> contacts = new ArrayList<Person>();
 
    @Override
    public void add(Person person) {
        contacts.add(person);
    }
 
    @Override
    public List<Person> all() {
        return unmodifiableList(contacts);
    }
 
}

Note: I am aware that the service implementation is not threadsafe and transacted. This needs to be set up manually in Spring, but I did not. Again, Java EE shines here (e.g., an EJB is transacted by default). Anyway this sample was just for a lecture demo, right :-)

My XML Schema definitions could not be any simpler as I defined data types and messages:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema attributeFormDefault="unqualified"
           elementFormDefault="qualified"
           targetNamespace="http://telecom.insa-lyon.fr/ns/schemas"
           xmlns:xs="http://www.w3.org/2001/XMLSchema">
 
    <xs:element name="contacts">
        <xs:complexType>
            <xs:sequence>
                <xs:element ref="sch:person" maxOccurs="unbounded" minOccurs="0"
                            xmlns:sch="http://telecom.insa-lyon.fr/ns/schemas"/>
            </xs:sequence>
        </xs:complexType>
    </xs:element>
 
    <xs:element name="person">
        <xs:complexType>
            <xs:sequence>
                <xs:element name="name" type="xs:string"/>
                <xs:element name="email" type="xs:string"/>
                <xs:element name="url" type="xs:string"/>
            </xs:sequence>
        </xs:complexType>
    </xs:element>
 
    <xs:element name="listRequest"/>
 
    <xs:element name="addRequest">
        <xs:complexType>
            <xs:sequence>
                <xs:element ref="sch:person" maxOccurs="1" minOccurs="1"
                            xmlns:sch="http://telecom.insa-lyon.fr/ns/schemas"/>
            </xs:sequence>
        </xs:complexType>
    </xs:element>
 
</xs:schema>

The endpoint implementation was quite straightforward (here with JDOM):

package fr.insalyon.ws.endpoint;
 
import fr.insalyon.ws.contacts.AddressBook;
import fr.insalyon.ws.contacts.Person;
import org.jdom.Element;
import org.jdom.JDOMException;
import org.jdom.Namespace;
import org.jdom.xpath.XPath;
import org.springframework.ws.server.endpoint.AbstractJDomPayloadEndpoint;
 
import java.util.List;
 
import static org.jdom.Namespace.getNamespace;
 
public class ContactsEndpoint extends AbstractJDomPayloadEndpoint {
 
    private final AddressBook addressBook;
 
    private final XPath nameXPath;
 
    private final XPath emailXPath;
 
    private final XPath urlXPath;
 
    private final Namespace ns;
 
    public ContactsEndpoint(AddressBook addressBook) throws JDOMException {
        this.addressBook = addressBook;
 
        ns = getNamespace("tc", "http://telecom.insa-lyon.fr/ns/schemas");
 
        nameXPath = XPath.newInstance("//tc:name");
        nameXPath.addNamespace(ns);
 
        emailXPath = XPath.newInstance("//tc:email");
        emailXPath.addNamespace(ns);
 
        urlXPath = XPath.newInstance("//tc:url");
        urlXPath.addNamespace(ns);
    }
 
    @Override
    protected Element invokeInternal(Element payload) throws Exception {
        if (payload.getName().equals("addRequest")) {
            return handleAddRequest(payload);
        } else if (payload.getName().equals("listRequest")) {
            return handleListRequest(payload); 
        }
        throw new Exception("Invalid SOAP payload: " + payload);
    }
 
    private Element handleAddRequest(Element payload) throws JDOMException {
        String name = nameXPath.valueOf(payload);
        String email = emailXPath.valueOf(payload);
        String url = urlXPath.valueOf(payload);
 
        addressBook.add(new Person(name, email, url));
        return null;
    }
 
    private Element handleListRequest(Element payload) {
        final List<Person> contacts = addressBook.all();
        Element root = new Element("contacts", ns);
 
        for (Person contact : contacts) {
            Element child = new Element("person", ns);
 
            child.addContent(new Element("name", ns).setText(contact.getName()));
            child.addContent(new Element("email", ns).setText(contact.getEmail()));
            child.addContent(new Element("url", ns).setText(contact.getUrl()));
            root.addContent(child);
        }
 
        return root;
    }
 
}

The remainder is a matter of Spring configuration:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
 
    <bean id="addressBook" class="fr.insalyon.ws.endpoint.InMemoryAddressBook"/>
 
    <bean id="contactsEndpoint" class="fr.insalyon.ws.endpoint.ContactsEndpoint">
        <constructor-arg ref="addressBook"/>
    </bean>
 
    <bean class="org.springframework.ws.server.endpoint.mapping.PayloadRootQNameEndpointMapping">
        <property name="mappings">
            <props>
                <prop key="{http://telecom.insa-lyon.fr/ns/schemas}addRequest">contactsEndpoint</prop>
                <prop key="{http://telecom.insa-lyon.fr/ns/schemas}listRequest">contactsEndpoint</prop>
            </props>
        </property>
        <property name="interceptors">
            <bean class="org.springframework.ws.server.endpoint.interceptor.PayloadLoggingInterceptor"/>
        </property>
    </bean>
 
    <bean id="contacts" class="org.springframework.ws.wsdl.wsdl11.DefaultWsdl11Definition">
        <property name="schema" ref="schema"/>
        <property name="portTypeName" value="Contacts"/>
        <property name="locationUri" value="/contactsService/"/>
        <property name="targetNamespace" value="http://telecom.insa-lyon.fr/ns/schemas"/>
    </bean>
 
    <bean id="schema" class="org.springframework.xml.xsd.SimpleXsdSchema">
        <property name="xsd" value="/WEB-INF/schema.xsd"/>
    </bean>
 
</beans>

... and web container configuration:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"
         version="2.4">
 
    <display-name>Contacts Web Service</display-name>
 
    <servlet>
        <servlet-name>spring-ws</servlet-name>
        <servlet-class>org.springframework.ws.transport.http.MessageDispatcherServlet</servlet-class>
    </servlet>
 
    <servlet-mapping>
        <servlet-name>spring-ws</servlet-name>
        <url-pattern>/*</url-pattern>
    </servlet-mapping>
 
</web-app>

Finally, testing the service is very simple with... my friend cURL to the rescue! ;-)

As an example, write those SOAP messages:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
                  xmlns:sch="http://telecom.insa-lyon.fr/ns/schemas">
    <soapenv:Header/>
    <soapenv:Body>
        <sch:addRequest>
            <sch:person>
                <sch:name>Julien</sch:name>
                <sch:email>julien.ponge@insa-lyon.fr</sch:email>
                <sch:url>http://julien.ponge.info/</sch:url>
            </sch:person>
        </sch:addRequest>
    </soapenv:Body>
</soapenv:Envelope>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:sch="http://telecom.insa-lyon.fr/ns/schemas">
   <soapenv:Header/>
   <soapenv:Body>
      <sch:listRequest/>
   </soapenv:Body>
</soapenv:Envelope>

Then you can make those add / list requests:

infinity:contactsws jponge$ curl -v --upload-file addRequest.xml -X POST -H "Content-Type: text/xml; charset=utf-8"  http://localhost:8080/contactsws/contactsService/
* About to connect() to localhost port 8080 (#0)
*   Trying ::1... connected
* Connected to localhost (::1) port 8080 (#0)
> POST /contactsws/contactsService/addRequest%2Exml HTTP/1.1
> User-Agent: curl/7.19.7 (i386-apple-darwin10.2.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: localhost:8080
> Accept: */*
> Content-Type: text/xml; charset=utf-8
> Content-Length: 503
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 202 Accepted
< Content-Length: 0
< Server: Jetty(6.1.22)
< 
* Connection #0 to host localhost left intact
* Closing connection #0
 
infinity:contactsws jponge$ curl -v --upload-file listRequest.xml -X POST -H "Content-Type: text/xml; charset=utf-8"  http://localhost:8080/contactsws/contactsService/
* About to connect() to localhost port 8080 (#0)
*   Trying ::1... connected
* Connected to localhost (::1) port 8080 (#0)
> POST /contactsws/contactsService/listRequest%2Exml HTTP/1.1
> User-Agent: curl/7.19.7 (i386-apple-darwin10.2.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: localhost:8080
> Accept: */*
> Content-Type: text/xml; charset=utf-8
> Content-Length: 230
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Accept: text/xml, text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
< SOAPAction: ""
< Content-Type: text/xml; charset=utf-8
< Content-Length: 365
< Server: Jetty(6.1.22)
< 
* Connection #0 to host localhost left intact
* Closing connection #0
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header/><SOAP-ENV:Body><tc:contacts xmlns:tc="http://telecom.insa-lyon.fr/ns/schemas"><tc:person><tc:name>Julien</tc:name><tc:email>julien.ponge@insa-lyon.fr</tc:email><tc:url>http://julien.ponge.info/</tc:url></tc:person></tc:contacts></SOAP-ENV:Body></SOAP-ENV:Envelope>
infinity:contactsws jponge$

By the way the complete code is Maven-ized and runs from a simple mvn jetty:run invocation. I can post that to GitHub if you guys ask for it.

Filed under: Uncategorized 4 Comments
18Jul/073

Off to Sydney

Hi everyone,

I'm leaving France tomorrow to stay a bit more than 3 weeks in Sydney, Australia, so don't be too surprised if the activity here is slow...

Have fun :-)

Filed under: English, IzPack, Uncategorized 3 Comments
6Jul/070

Safari for Windows actually works

While the previous versions of the Safari for Windows beta used to have a few rendering problems, it looks like the developers fixed the issues!

Filed under: English, Uncategorized No Comments
   Older Entries »

Info

View Julien Ponge's profile on LinkedIn
Fork me on GitHub

Twitter Updates

Archives

JPz'log is Digg proof thanks to caching by WP Super Cache