Monday, August 20, 2007
Easy sanity testing of a webapp with Python

Needed to whip up some automated sanity testing of a webapp, and Cactus seems to have a somewhat steep learning curve. Eventually, yes, this might be the way to go, because we should be able to integrate it nicely with Maven and CruiseControl, but I needed something quick.

In particular, I needed to check whether a certain number of images were showing up in a portion of a JSP-generated page.

Step 1. Prep the JSP to make it testable. Primarily, we just want to generate a unique HTML id attribute for the <img> tags in question:

<c:forEach items="${items}" var="item" varStatus="itemStatus">
  <c:set var="imageId">
    specialImage<c:out value="${itemStatus.index}"/>
  <img id="${imageId}" .../>

This numbers the images with ids like specialImage0, specialImage1, etc.

Step 2. Fetch and parse the page with Python's HTMLParser library. Basically, we write a parser that only pays attention to the tags we've marked in the JSP above:

import HTMLParser
import re

class TestSpecialImageCount(HTMLParser.HTMLParser):

    def reset(self):
        self.imageIds = []

    def handle_startendtag(self, tag, attrs):
        if tag == "img":
            for key, val in attrs:
                if key == "id":
                    if re.match("specialImage[0-9]+",val):
                        if val not in self.imageIds:

    def testPassed(self):
        return (len(self.imageIds) == 6)

Then running the test is as easy as:

import urllib
import TestSpecialImageCount

test = TestSpecialImageCount.TestSpecialImageCount()
page = urllib.urlopen("http://localhost:8080/webapp/home.htm").read()
if test.testPassed():
    print "OK"
    print "FAILED"

You could pretty easily batch this up and instantiate several parsers for different tests so you have one uber Python script to run all the tests (this is what we did). But the point here is that you don't have to write very much code at all to get the sanity test written, because you can write the HTML parsers so compactly in Python.

Thursday, August 9, 2007
Auto-deploying to Tomcat from CruiseControl with Maven

We have a web project that gets built using Maven; it was relatively simple to get this running under CruiseControl since CC has pretty good Maven2 support. However, we also wanted to have a successful build actually deployed to a Tomcat somewhere at a known location so that people could always check out the latest version.

I played around with various maven plugins for working with Tomcat, but had a really hard time getting them to work properly. Then I realized, even if I got the maven plugins working, I would need to do something else, because we run into PermGen errors on our Tomcats when we undeploy/redeploy, so we'd actually need to do an undeploy/restart-tomcat/deploy loop instead. Of course, none of the tomcat plugins can do the restart-tomcat bit (they can shut it down, but because they interact with the tomcat management interface, they can't actually start it up again!).

So, I went back down the script route. I wrote this script which can do the undeploy/restart/redeploy loop:



export CATALINA_HOME=/usr/local/lib/apache-tomcat-5.5.17
export JAVA_HOME=/usr/java/jdk1.5.0_09


# undeploy old WAR, if any
( echo "--silent"
  echo "--url http://localhost:$TOMCAT_PORT/manager/html/undeploy?path=/" ) | \
curl --config - > /dev/null

# stop the tomcat
(cd $CATALINA_HOME; bin/

# pause to wait for shutdown
sleep 5

# restart the tomcat
(cd $CATALINA_HOME; bin/

# drop the WAR in place
cp $CC_BUILD/webapp/target/ROOT.war $CATALINA_HOME/webapps

This assumes that you have the tomcat installed in the given CATALINA_HOME with the manager app enabled and set up with the appropriate admin credentials, and that the build directory for the CC project is in the "trunk/webapp" directory. Note also that we're deploying to the root context, so you will want to modify your undeploy URL if that's different.

The last step is to add the following CC project; assuming we have an existing "webapp" CC project in existence:

  <project name="webapp-tomcat" buildafterfailed="false">
      <buildstatus logdir="logs/webapp"/>
    <schedule interval="60">
      <exec command="/home/cruise/bin/redeploy"

This triggers off a successful CC build of "webapp" and just calls the script we wrote above. Nice and easy.