Butter Mountain

Introduction

After the disk in my laptop recently died I decided that it would be a good time to try and install the latest version of Open Solaris onto my laptop. Unfortunately the only option I have for installing new operating systems onto this laptop is via PXE boot, this has not been an issue in the past as I’ve been installing Debian / Ubuntu.

Solaris is fully capable of being network installed, in fact there are a multitude of tools out there that assist you in doing so. However, this is not true for the latest release of Open Solaris, support is planned but its currently not available. As a learning exercise I decided that I’d attempt to get this working on my laptop, turns out its entirely possible with a little bit of fiddling.

Note: This does not create a hands-off installer, it merely boots the livecd from the network. You can then install the system as you would normally from the gui installer.

Note 2: Part of this guide contains an ugly hack, its awful and may not work for you at all - you have been warned.

Requirements

You will need to the following to PXE / Net boot Open Solaris

• Time
• One copy of the Open Solaris CD Image.
• One system already running Open Solaris (to manipulate the boot archive)
• One PXE Boot Server (one part DHCP server, one part TFTP server and one part NFS server)
• One PXE Boot Client (with sufficient hardware to run Open Solaris)
• One network which both the client and server are connected to.

Setting up PXE Boot.

I’m going to assume that you know how to setup a PXE boot server, there are many documents regarding this for other operating systems which can be used to fill in the blanks here. Personally I used ubuntu Gutsy Gibbon as my PXE boot server.

The following steps explain which files need to be taken from the cdrom image and placed on your PXE boot server.

• Mount the cdrom image somewhere on your system (I chose /mnt/cdromimage/)
• From the cdrom image copy out the following files into your tftp directory (/tftpboot/ on my system)

# cp /mnt/cdromimage/boot/grub/pxegrub /tftpboot/boot/grub/
# cp /mnt/cdromimage/boot/x86.microroot /tftpboot/solaris/
# cp -r /mnt/cdromimage/platform/* /tftpboot/solaris/platform/

• Note: you do not need all of the contents of of the /platform/ directory, however, its only 6mb in total its easier to include it all than remove what we don’t need.

• Create the file /tftpboot/boot/grub/menu.lst with the following contents:

default=0
timeout=60
timeout=3
title Open Solaris PXE Install
kernel$ /solaris/platform/i86pc/kernel/$ISADIR/unix -m verbose -B install_server=[ip of your nfs server]:/export/opensolaris/
module /solaris/x86.microroot

• When setting up your dhcpd boot entry for the Open Solaris client machine ensure that you include the following dhcp options (this is for the ubuntu dhcpd server, this is different under Solaris)

option grubmenu code 150 = text;

host laptop {
hardware ethernet 00:XX:00:XX:00:XX;
option grubmenu “/boot/grub/menu.lst”;
fixed-address 192.168.1.34;
option subnet-mask 255.255.255.0;
option routers 192.168.1.10;
next-server 192.168.1.20;
filename “/boot/grub/pxegrub”;
}

With your dhcpd and tftpd services up and running you should now be in a position to boot the client, this will get you to a point where the system has loaded the kernel + microroot. The boot will fail when we attempt to mount the cd rom.

Editing the miniroot

To complete the boot process we need to update the microroot and tell it to mount the cdrom from an nfs server instead of from a local cdrom drive. Now is the time to load up the Open Solaris machine you installed earlier, mount the open solaris iso and copy the x86.miniroot file somewhere handy (ready for editing).
The miniroot is a gziped ufs file system image, its relatively easy for us to open it up and manipulate as we please.

# mv x86.microroot x86.microroot.gz;gunzip x86.microroot.gz
# lofiadm -a /path/to/x86.microroot /dev/lofi/1
# mount /dev/lofi/1 /mnt/microroot

So now we have the microroot mounted under /mnt/microroot, lets make some changes. First of all there are three utilities I’ve added to the microroot to aid in net booting. These can be added in the following way:

# cp /usr/sbin/prtconf /mnt/microroot/usr/sbin/
# cp /usr/sbin/i86/prtconf /mnt/microroot/usr/sbin/i86/
# cp /usr/bin/cut /mnt/microroot/usr/bin/
# cp /usr/bin/sed /mnt/microroot/usr/bin

We now have all the utilities required on the microroot to mount the cd image over nfs, the next step is to update the script that mounts the cdrom image.

Note: Here is where the nasty hack is, as I couldn’t convince nwamd (network automagic daemon) or the default network service to actually bring up the network correctly I run perform a dhcp request in the script that mounts the cdrom image. Ideally we should bring networking online via the networking physical service (or nwamd if thats what its actually designed to be used for) and have the live-fs-root service depend on the networking. Anyway, as there is an official net boot utility in the pipeline I hope this will suffice until thats available.

On with the hacking, edit the /mnt/microroot/lib/svc/method/live-fs-root startup script. Find the following line:

echo “\rPreparing live image for use” >/dev/msglog

Just below this line add in our few lines to grab the install_server value, send out a dhcp request followed by mounting the network file system.

# grab the install_server setting from boot
install_server=`/usr/sbin/prtconf -v /devices|/usr/bin/sed -n ‘/install_server/{;n;p;}’|/usr/bin/cut -f 2 -d\’`
# Awful hack - configure dhcp for all the plumbed interfaces
/sbin/ifconfig -a dhcp
# now mount the install server onto /.cdrom
/sbin/mount $install_server /.cdrom

The above few lines do not perform any sanity checks, if any of the commands fail you’ll just get dumped to a single user login prompt.

Now that we are finished editing the micro root its time to unmount and re-compress it before dumping it back into the tftp directory on your PXE boot server.

# umount /mnt/microroot
# lofiadm -d /dev/lofi/1
# gzip x86.microroot;mv x86.microroot.gz x86.microroot

Finally..

The last thing to do is to export the contents of the cd image over nfs, I chose to export it from /export/opensolaris/. It may help you to know that Solaris has issues mounting nfs v4 from Linux servers (certainly from ubuntu gutsy release), I had to disable nfs v4 services under linux to mount the nfs file system from my Solaris system.

If everything has gone smoothly you should now be in a position to boot your client machine, which once the pxe boot has completed should run just as it would had it been booted from cdrom.

If you get dumped to a single user login prompt its safe to assume something has gone wrong, I’d recommend logging in (root pw: opensolaris) and verifying that each of the steps we take in the mount cdrom script works correctly.

Since late 2007 I’ve been working on a project involving Django, initially we started developing the software using 0.96 stable release, however recently made the decision to move to the SVN release. One of the main reasons behind this was the availability of new features and of course plenty of bug fixes.

During some initial testing of our Django app I noticed that I could break referential integrity right down at the database level with the sql tables that were created. There are two reasons for this:

1. MySQL by default uses the MyISAM storage engine for its tables, this lacks support for transactions, row-level locking and foreign keys.
2. Django bug 6374 - Foreign key constraints not added across apps when creating tables.

Fixing point number one was fairly straight forward, you can simply reconfigure MySQL to use InnoDB as the default table type which has support for foreign keys among other things. Realising that we were hitting a bug in Django took a little bit longer, so how do you know if you’re hitting the bug (it’s not explained very well in the ticket).

Imagine the following simple model (please excuse the formatting).

from django.contrib.auth.models import User

Class Item(models.Model):
id = models.AutoField(primary_key=True)
title = models.CharField()
desc = models.TextField()
owner = modesl.ForeignKey(User)

When the SQL is generated for this model we would expect to see a foreign key constraint created, unfortunately Django was not creating one thus allow us to assign owners to items who do not exist in the owners table. The foreign key constraint should look something like this:

ALTER TABLE `item_item` ADD CONSTRAINT user_refs_id_4ce20cfc FOREIGN KEY (`owner`) REFERENCES `auth_user` (`id`);

If you’re running 0.96 then you could be running into this bug, its certainly something to look out for. If you are running the SVN release you’re in luck, as I’m pleased to say this bug was fixed in Changeset 7215 (Committed to the trunk 10th March 2008).

Speaking of changesets, 7477 is a big step forward, this merges the Query Set Refactor Branch with the SVN Trunk, if you’re still running 0.96 there are literally hundreds of reasons to move on up to SVN - though watch out for those backward incompatible changes.

After re-organising a large number of photos using eog I started to notice spikes of cpu usage while moving from image to image. At first this was just inconvenient but as time went on (and as more photos were viewed) the cpu usage spikes became longer and longer. Is my Ubuntu install getting slower? it feels like I’m back in Windows, time to investigate.

Quick Investigation.

Under Linux we can use strace to follow all the system calls a particular process makes, its very high level as we won’t be able to trace any library code. However, it may give us some a clue as to where we should start looking.
Using the command ’strace -o eog.strace eog *.jpg’ we can run eog for a little while and then look through the output for something interesting. After browsing through the output I found the following was appearing after we move from one image to the next but before the next image was displayed.

close(15) = 0
munmap(0xb634f000, 4096) = 0
time(NULL) = 1206991162
time(NULL) = 1206991162
time(NULL) = 1206991162

[ several hundred calls to the time() system call]

time(NULL) = 1206991162
time(NULL) = 1206991163
time(NULL) = 1206991163
gettimeofday({1206991163, 403748}, NULL) = 0
gettimeofday({1206991163, 404023}, NULL) = 0
gettimeofday({1206991163, 404288}, NULL) = 0

We call time() several hundred times in what appears to be a tight loop? something certainly seems broken or at the very least not optimal. Looking through the strace output which traced loading eog, displaying one image, moving to a new image and then exiting we called time() 25267 times.

Preparing To Go Deeper

Usually I debug apps under Solaris which has the awesome power of dtrace along with bundled binaries which are not stripped, this makes it really easy to trace library calls and user land calls. By default Ubuntu packages are shipped with all the binaries stripped which has upset some people I know. The Ubuntu devs are clearly not stupid and provide some debugging aids, not just debug builds (which aren’t really *that* useful in a production environment) but by providing separate packages which include the debugging symbols.

One can add the debug symbol packages to the Gutsy release of Ubuntu by adding the following to their /etc/apt/sources.list file:

deb http://ddebs.ubuntu.com/ gutsy main universe
deb http://ddebs.ubuntu.com/ gutsy-updates main universe

Once that has been done the debugging symbols packages will show up with ‘-dbgsym’ appended to them in apt, for now I am just going to install the eog-dbgsym package. I’m also making sure that I have gdb installed, as that is what I plan to dig deeper with.

Digging Deeper.

Firstly, lets start up eog as we did before but this time attach gdb and set a break point for the time() call.

# eog *.jpg
!! note I know eog’s pid is 23700 !!
# gdb -p 23700
!! loading gdb output….
0xffffe410 in __kernel_vsyscall ()
(gdb) b time
Breakpoint 1 at 0xb73fde25
(gdb) c
Continuing.

!! move to a new image with eog !!

[Switching to Thread -1228670288 (LWP 23700)]

Breakpoint 1, 0xb73fde25 in time () from /lib/tls/i686/cmov/libc.so.6
(gdb) bt
#0 0xb73fde25 in time () from /lib/tls/i686/cmov/libc.so.6
#1 0xb74fae7b in g_bookmark_file_set_mime_type () from /usr/lib/libglib-2.0.so.0
#2 0xb7a34499 in gtk_recent_manager_add_full () from /usr/lib/libgtk-x11-2.0.so.0
#3 0×08067c42 in eog_window_display_image (window=0×8124118, image=0×827ece8) at eog-window.c:870
#4 0×080683df in eog_job_load_cb (job=0×82c6400, data=0×8124118) at eog-window.c:1286
#5 0xb75b8c09 in g_cclosure_marshal_VOID__VOID () from /usr/lib/libgobject-2.0.so.0
#6 0xb75ab772 in g_closure_invoke () from /usr/lib/libgobject-2.0.so.0
#7 0xb75bc323 in ?? () from /usr/lib/libgobject-2.0.so.0
#8 0×082d5450 in ?? ()
#9 0×00000000 in ?? ()
(gdb)

I repeated this a few times to see if the back trace changed, it did not (much). We’re calling some code in libgtk, specifically gtk_recent_manager_add_full(). What is gtk recent manager? what are we bookmarking with ‘g_bookmark_file_set_mime_type()’, all very relevant questions I feel. To dig deeper lets install the gtk and glib debugging symbols packages and look at an improved stack trace (note: here we have passed the first few ‘time()’ break points and are now stuck in the loop which calls time() an excessive number of times.

Breakpoint 1, 0xb74b6e25 in time () from /lib/tls/i686/cmov/libc.so.6
(gdb) bt
#0 0xb74b6e25 in time () from /lib/tls/i686/cmov/libc.so.6
#1 0xb7aecacf in build_recent_info (bookmarks=0×8128a88, info=0×82c1728) at /build/buildd/gtk+2.0-2.12.0/gtk/gtkrecentmanager.c:1677
#2 0xb7aece1c in IA__gtk_recent_manager_get_items (manager=0×81189b0) at /build/buildd/gtk+2.0-2.12.0/gtk/gtkrecentmanager.c:1311
#3 0×08063f2a in eog_window_update_recent_files_menu (window=0×8124118) at eog-window.c:3468
#4 0xb7671c09 in IA__g_cclosure_marshal_VOID__VOID (closure=0×8279148, return_value=0×0, n_param_values=1, param_values=0xbfb05fac, invocation_hint=0xbfb05ebc,
marshal_data=0×80641c0) at /build/buildd/glib2.0-2.14.1/gobject/gmarshal.c:77
#5 0xb7664772 in IA__g_closure_invoke (closure=0×8279148, return_value=0×0, n_param_values=1, param_values=0xbfb05fac, invocation_hint=0xbfb05ebc)
at /build/buildd/glib2.0-2.14.1/gobject/gclosure.c:490
#6 0xb7675323 in signal_emit_unlocked_R (node=0×81255d8, detail=0, instance=0×81189b0, emission_return=0×0, instance_and_params=0xbfb05fac)
at /build/buildd/glib2.0-2.14.1/gobject/gsignal.c:2440
#7 0xb7676847 in IA__g_signal_emit_valist (instance=0×81189b0, signal_id=110, detail=0, var_args=0xbfb061ec “��˷�Ǯ���˷(b��!ծ�\210\212\022\b�9+\b”)
at /build/buildd/glib2.0-2.14.1/gobject/gsignal.c:2199
#8 0xb7676a09 in IA__g_signal_emit (instance=0×81189b0, signal_id=110, detail=0) at /build/buildd/glib2.0-2.14.1/gobject/gsignal.c:2243
#9 0xb7aec7cc in gtk_recent_manager_changed (recent_manager=0×8298778) at /build/buildd/gtk+2.0-2.12.0/gtk/gtkrecentmanager.c:1380
#10 0xb7aed521 in IA__gtk_recent_manager_add_full (manager=0×81189b0, uri=0×82b39d8 “file:///home/iain/Photo-0178.jpg”, data=0×82a2ba0)
at /build/buildd/gtk+2.0-2.12.0/gtk/gtkrecentmanager.c:1001
#11 0×08067c42 in eog_window_display_image (window=0×8124118, image=0×827ed70) at eog-window.c:870
#12 0×080683df in eog_job_load_cb (job=0×84d9dc0, data=0×8124118) at eog-window.c:1286
#13 0xb7671c09 in IA__g_cclosure_marshal_VOID__VOID (closure=0×82b8430, return_value=0×0, n_param_values=1, param_values=0xbfb0656c, invocation_hint=0xbfb0647c,
marshal_data=0×80680c0) at /build/buildd/glib2.0-2.14.1/gobject/gmarshal.c:77
#14 0xb7664772 in IA__g_closure_invoke (closure=0×82b8430, return_value=0×0, n_param_values=1, param_values=0xbfb0656c, invocation_hint=0xbfb0647c)
at /build/buildd/glib2.0-2.14.1/gobject/gclosure.c:490
#15 0xb7675323 in signal_emit_unlocked_R (node=0×83234e0, detail=0, instance=0×84d9dc0, emission_return=0×0, instance_and_params=0xbfb0656c)
at /build/buildd/glib2.0-2.14.1/gobject/gsignal.c:2440
#16 0xb7676847 in IA__g_signal_emit_valist (instance=0×84d9dc0, signal_id=207, detail=0,
var_args=0xbfb067ac “\b7\v\b\b7\v\b
#17 0xb7676a09 in IA__g_signal_emit (instance=0×84d9dc0, signal_id=207, detail=0) at /build/buildd/glib2.0-2.14.1/gobject/gsignal.c:2243
#18 0×08084c97 in eog_job_finished (job=0×84d9dc0) at eog-jobs.c:120
#19 0×0808454e in notify_finished (job=0×84d9dc0) at eog-job-queue.c:65
#20 0xb75ca551 in g_idle_dispatch (source=0xb4726718, callback=0×1a, user_data=0×84d9dc0) at /build/buildd/glib2.0-2.14.1/glib/gmain.c:4132
#21 0xb75cc11c in IA__g_main_context_dispatch (context=0×80e0010) at /build/buildd/glib2.0-2.14.1/glib/gmain.c:2061
#22 0xb75cf55f in g_main_context_iterate (context=0×80e0010, block=1, dispatch=1, self=0×80b75f8) at /build/buildd/glib2.0-2.14.1/glib/gmain.c:2694
#23 0xb75cf909 in IA__g_main_loop_run (loop=0×832feb0) at /build/buildd/glib2.0-2.14.1/glib/gmain.c:2898
#24 0xb7a919e4 in IA__gtk_main () at /build/buildd/gtk+2.0-2.12.0/gtk/gtkmain.c:1144
#25 0×08060c31 in main (argc=5, argv=Cannot access memory at address 0×4
) at main.c:221

Still not much closer but now we have some source files, functions and line numbers to go and look at, perhaps the problem isn’t just with eog but with gtk instead?

Read The Source

From the above stack trace and my tracing of eog I can see that we’re spending a lot of time in build_recent_info() gtkrecentmanager.c:1090, which takes two arguments, a GBookmarkFile and GtkRecentInfo. After some hunting through the source I found that the GBookmarkFile eventually references back to the real file “~/.recently-used.xbel” this file appears to be an xml formatted file for managing “recently” used documents.

Part of the gtk_recent_manager_add_full() function is to add the recently viewed file/image/movie/song to this history file, in this case its an image. Once we’ve added it (which includes adding a time stamp) we then update the last viewed images which are displayed in the file menu of eog. Also (though I found this out some what later) we send a signal to gnome-panel to update the ‘places -> recent documents’ menu.

Once eog (or gnome-panel) receives this ‘update’ signal they run through every item stored in the .recently-used.xbel file, calling time() for each entry allowing us to find the last five items viewed in eog (or for gnome-panel its the last ten). On the surface this seems like a fine idea, though somewhat inefficient (why not just call time() the once for each run through the file, not per entry) making the current implementation slightly flawed.

The Fatal Problem

There is no default limit to the amount of entries we store in the .recently-used.xbel file the more images you view in eog (or as I later found out any gtk app which has been coded to use gtk-recent-manager) the more entries there will be, however, eog will only ever display the last five. This coupled with when running through the list to display the last five entries, time() is called against each entry - this is what leads to the gradual slowdown I described at the beginning of this post.

A single entry in the .recently-used.xbel file is 14 lines long, based on my file I currently have 1734 entries (and growing each day), thus when I move from one image to the next in eog time() is called at least 1734 times in eog followed by gnome-panel performing a second 1734 calls. Clearly this is not acceptable, something must be done - on a slow 1.3ghz laptop this leads to about 5 - 10 seconds of 100% cpu time purely to display the last five images view in eog.

Possible Solutions

• Add a cron entry to delete the .recently-used.xbel file once a week

This is clearly a hack / workaround. It does not address the root cause, it also does not address calling time() multiple times without any need.

• Set a default limit of 25 entries per application to be stored and only call time() once per run through of the .recently-used.

A much better solution, this would limit the size of the .recently-used.xbel file and make sorting it more efficient. gnome-panel only displays the last ten viewed documents, eog only displays the last five. Therefore holding the last 25 entries is still overkill but its at least manageable.

Conclusion

I’d love to fix this my self but I just don’t have the time to dedicate to it, hopefully this blog entry will aid people who have found the same issue and possibly help the gtk developers to improve their software.

GTK Devs please fix this!

After reading about Sun hiring some Jython developers I decided to install it myself under Debian testing to see what all the fuss is about. As a big Python fan a world without the GIL (Gloabl Interpreter Lock) of CPython sounds just lovely.

After installing it I had a little poke about, along the way I hit upon this lovely gem.

iain@hostname ~>file `which jython`
/usr/bin/jython: perl script text executable

Shocking results! luckily its just a Debian wrapper script but theres a certain amount of irony involved thats for sure.

Introduction

A common use case for an xml-rpc based application server would be one which involves the use of one or more persistent objects. Without any kind of persistence a server handling xml-rpc requests will either be ineffective or inefficient.

Accessing Persistent objects in an exposed method/user handler with Apache xml-rpc 3.0 is a process involving several layers of redirection. This can be difficult for anyone who is not familiar with the ws-xmlrpc 3.0 code base. The purpose of this document is to provide a working example using the standalone servlet server which can assist people new to ws-xmlrpc 3.0.

Before We Begin

For those of you who are just interested in the source code to this example, that can be found here: ws-xmlrpc-3.0-buttermountain-example.zip

This document and the code has been created with the help of the ws-xmlrpc FAQ examples and the help of the Apache ws-xmlrpc mailing list.

It is assume that you have read the examples on the Apache ws-xmlrpc website and are familiar with the Java programming language.

Finally, I recommend that you open up the source code and read through the sample code with this document.

In The Beginning There Was Nothing

Before you even begin starting to write your handler methods (the methods which will be exposed via xml-rpc), you must first collect the required libraries and setup the correct directory structure. The directory structure I have opted for looks something like this:

org/buttermountain/xmlrpc/server
org/buttermountain/xmlrpc/client
org/buttermountain/pobject
javax
lib

The javax directory contains all of the Sun servlet classes and the lib directory contains all of the Apache ws-xmlrpc 3.0 jar’s and the Apache ws jar’s that they depend upon.

As for the other directories, xmlrpc/server contains all of our server files, xmlrpc/client contains a sample java ws-xmlrpc client. Finally, the pobject directory contains our very simple persistent object class.

User Handler Methods

When creating an xml-rpc server you will need to create a class which has its methods exposed via xml-rpc. With Apache ws-xmlrpc 3.0 this is done by creating two classes, one which is your interface (ExposedMethodsInterface.java) and one which is the implementation of this interface (ExposedMethodsImplementation.java).

The interface will merely define the methods, their inputs and return data types. For a method to be exposed via xml-rpc it must be public and have a return type that is not void, here is the example interface code

public interface ExposedMethodsInterface {
public void init(Pobject pobject);
public boolean setString(String string);
public String getString();
}

Three methods have been defined, firstly the init() method is a non exposed method which we will call from one of our extended classes to pass our persistent object in (much more on this later). The ‘get and set’ methods are public and will be exposed via xml-rpc, hopefully you can imagine what is going on in the underlying implementation, if not take a look at the ‘ExposedMethodsImplementation.java’
file provided in the example code.

A Sample Persistent Object

The persistent object for this example is incredibly simple, it exists solely to keep this example simple. The code is so short that I will include it here:

public class Pobject {
private String storedString = “”;
public synchronized void setString(String string){
storedString = string;
}
public synchronized String getString(){
return storedString;
}
}

As you can see, all it does is store a string and then allows the programmer the ability to ‘get and set’ the string.

Here lies the problem

Someone who is not familiar with the ws-xmlrpc 3.0 tool-kit may be thinking:

“Why do I even need an object like this? surely something as simple as a string could be stored in my handler class?”

This is where the problem begins, the user handler classes are completely stateless, up-to the point that a new instance of the handler class is created to handle each individual xml-rpc request. After the request is completed the handler instance is disposed of - never to be used again.

A solution to the problem

To get around this problem, one can call methods of the handler class after it has been initialised and just before it handles an xml-rpc request. Once one has the required knowledge this is a relatively simple task. However, judging by my experience and messages on the Apache xml-rpc mailing list, there is some confusion regarding how one can do this.

To access methods of the handler class in this way, three of the ws-xmlrpc classes must be extended and the correct methods within them must be overridden / extended. In the sample code there is another class which has been extended (AuthenticationHandler by BmAuthHandler.java) This includes another example from the ws-xmlrpc FAQ which explains how to enable basic HTTP authentication. I have left this in to provide a more complete example.

The classes call each other in the following Hierarchy:

ServletServer

|____> XmlRpcServlet
|____> RequestSpecificProcessorFactoryFactory
|____> Can make calls to user handler methods.\

We will be extending the XmlRpcServlet class and the RequestSpecificProcessorFactoryFactory class. It is within the XmlRpcServlet that your persistent objects will be initialised, it is within the RequestSpecificProcessorFactoryFactory class that you will make calls to your handler methods.

Your very own ‘ServletServer’

Making use of the ServletWebServer class is very simple and is the starting point of your standalone Apache ws-xmlrpc 3.0 server. The ServeletServer class merely initialises your servlet (in this case one that we have extended ourselves) and then starts up the servlet web server on the specified port. See the file:
‘org/buttermountain/xmlrpc/server/ServletServer.java’ for details.

Extending the XmlRpcServlet

The XmlRpcServlet is persistent, it is only initialised once (when using ServletServer, I have not tested this with anything else (such as TomCat)). This class is extended within ‘org/buttermountain/xmlrpc/server/BmXmlRpcServlet.java’. It is here that the example persistent
object (Pobject) class is initialised:

private Pobject pobject = new Pobject();

Within BmXmlRpcServlet The methods ‘isAuthenticated’ and ‘XmlRpcHandlerMapping’ are used to handle basic HTML authentication, you may ignore those for the purposes of this example. The method that we are interested in is the newPropertyHandlerMapping method. Is is
within this method that we initialise our next (and final) extended class the BmProcessorFactory which extends the RequestSpecificProcessorFactoryFactory class.

Extending the RequestSpecificProcessorFactoryFactory

The RequestSpecificProcessorFactoryFactory is an incredibly long variable name, my extended version of this class uses a much shortened name ‘BmProcessorFactory’ it can be found here ‘org/buttermountain/xmlrpc/server/BmProcessorFactory.java’. It is within this class
that you can actually call methods on your handler class.

I have implemented the ProcessorFactory in the following way, the initialisation method for this class takes in our persistent object from the XmlRpcServlet. The getRequestProcessor() method is then overridden, it is in this method that we make calls to our handler methods.

If you remember the ExposedMethodsInterface class had a method called ‘init()’ which took an object of type Pobject. The implementation for this method looks like this:

public void init(Pobject pobject){
this.pobject = pobject;
}

The variable ‘pobject’ is declared in the main body of the class allowing it to be accessible from all of the methods. This method is called from the BmProcessorFactory in the following way:
ExposedMethodsInterface proc = (ExposedMethodsInterface)
super.getRequestProcessor(pClass, pRequest);
proc.init(pobject);
return proc;

Finally in that one line ‘proc.init(pobject)’ you can pass the persistent object to your handler class. It would be here that you could call any other methods you may want to on your handler method, one thing you must make sure is that all of your handler classes implement an init() method in the same way. Any handler classes which do not will throw an XmlRpcException and refuse to work.

Pulling it all together

Before the xml-rpc server can be run you must create a mapping file, this file creates a mapping between exposed method names and method names local to java. The included file to handle this is called ‘XmlRpcServlet.properties’ and looks like this:

ExposedMethodsInterface=org.buttermountain.xmlrpc.server.ExposedMethodsImplementation
org.buttermountain.xmlrpc.server.ExposedMethodsInterface=org.buttermountain.xmlrpc.server.ExposedMethodsImplementation

The default location of this file is ‘org/apache/xmlrpc/webserver/XmlRpcServlet.properties’, However, I have overridden this location in the newXmlRpcHandlerMapping method of our BmXmlRpcServlet class.

Starting the xmlrpc server

The source I’ve included has a file listing the relative classpath (./classpath) and another file listing all of the source files (./sourcefiles).

Under Linux the following steps can be taken to compile and execute the code (assuming that java/javac are in your $PATH)

user@host $ ~/xmlrpc-example> export CLASSPATH=`cat classpath`
user@host $ ~/xmlrpc-example> javac @sourcefiles
user@host $ ~/xmlrpc-example> java org/buttermountain/xmlrpc/server/ServletServer
04-Nov-2006 21:16:06 org.apache.xmlrpc.webserver.XmlRpcServlet log
INFO: init

If you’ve not changed anything the Apache ws-xmlrpc server should now be running on localhost port 5555

Testing the example server

To make things even easier I’ve included two clients, one written in Java (ProxyClient.java) which is code taken from the Apache ws-xmlrpc client FAQ and one written in Python. Both clients perform the same operations but set a different string. These two clients should prove to you that the xml-rpc server is working as intended.

Under Linux (again) make sure you have started the server (as mentioned above), now in another terminal:

user@host $ ~/xmlrpc-example> export CLASSPATH=`cat classpath`
user@host $ ~/xmlrpc-example> python python_client/client.py
The oldstring was: <<< note: the default string is ""
setting string to: python xmlrpc client
The string is now: python xmlrpc client

user@host $ ~/xmlrpc-example> java org/buttermountain/xmlrpc/client/ProxyClient
The oldstring was: python xmlrpc client
setting string to: java xmlrpc client
The string is now: java xmlrpc client

Take a look at the source code of both those examples if you wish to see how they work in more detail, the output should look something like the above.

Note: The client and server code should work under any operating system where Java & Apache ws-xmlrpc 3.0 are supported. The Python client should work under any operating system which supports Python.
i.e the code should run on systems other than Linux.

Conclusion

I hope that this example will be useful to people attempting to use the Apache ws-xmlrpc 3.0 tools. If you have any questions / comments etc, please drop me an email.

Thank you to the kind people of the Apache ws-xmlrpc mailing list who helped me get to a working solution and to the Apache ws-xmlrpc developers who have created a fantastic (although sometimes complicated) tool kit.