This post is about something I did on ICTC, we are developing some applications that communicate with each others on over the network and we thought on compressed XMLRPC. I have been looking about this on Google, but didn't really found something. So, I try by myself and this is the result.
First, allow me to explain a bit more about xmlrpc. XMLRPC is nothing more than a remote call using the HTTP protocol the POST method and XML as content, then, the server do its work and return something in the same way. THen the client parses this xml and create the result in a native way. Just like if that method where in your program. This offer a language independece (the server may be written in some language and several clients in many other languages).
Then, implementing compressed xml should not be that difficult, I mean.. HTTP support the "Accept-Encoding" header and that should do the job. But hey... xmlrpclib does not include gzip support, so, we are going to do it.
The server part should subclass SimpleXMLRPCServer and in the constructor we are going to create a inner class that will be a request Handler, this handler is based on SimpleXMLRCPRequestHandler and implements the do_POST method, this method is called everytime something is sent to the server (on every call).
In do_POST method we are responsible for fetching the data, decompress it, parse it, handle the response and then send the reply to the client. This isn't so hard (check the code). The important part here is that we check for the "accept-encoding" header and that the vale of that header (if exists) is "gzip", if this is true then use zlib to decompress the data and when we have the result of the call, compress it using zlib and send it to the client.
def do_POST(self):
if self.headers.has_key('accept-encoding') and \
self.headers['accept-encoding'] == 'gzip':
# check if the path ir right
if not self.is_rpc_path_valid():
self.report_404()
return
try: #Try to get the data
max_chunk_size = 10*1024*1024
size_remaining = int(self.headers["content-length"])
L = []
while size_remaining>0:
chunk_size = min(size_remaining, max_chunk_size)
L.append(self.rfile.read(chunk_size))
size_remaining -= len(L[-1])
data = ''.join(L)
data = zlib.decompress(data) #Decompress
response = self.server._marshaled_dispatch(
data, getattr(self, '_dispatch', None)
)
except:
#Should be good to registre the exception somewhere
pass
else:
#We have an answer...
#Let's compress it
response = zlib.compress(response)
self.send_response(200)
self.send_header("Content-type", "text/xml")
self.send_header("Content-length", str(len(response)))
#As the call was compressed, then we compress the
#Answer too. Be sure to set the compressed header.
self.send_header("Accept-Encoding","gzip")
self.end_headers()
self.wfile.write(response)
#Close the connection
self.wfile.flush()
self.connection.shutdown(1)
else:
return SimpleXMLRPCRequestHandler.do_POST(self)
This take us the the other part of the call, the client. The client must implement gzip compression too. The client part is a bit simplier, you just have to subclass xmlrpclib's Transport and and in the "request" method compress the request_body before sending it. Obviously you have to append the "accept-encoding" header to the http connection. Once you have the result you have to check for the "accept-encoding" header too, if it is there, decompress using zlib's methods.
def request(self, host, handler, request_body, verbose=0):
'''
Encargado de manejar los request, envia los datos y recibe
las repuestas, por lo que aqui se hace la compresion de datos
y posteriormente se envian los datos recibidos a
la funcion self._parse_response o self._parse_gzipped_response
de acuerdo al caso.
Mas informacion:
http://docs.python.org/library/xmlrpclib.html
'''
h = self.make_connection(host)
if verbose:
h.set_debuglevel(1)
self.send_request(h, handler, request_body)
self.send_host(h, host)
self.send_user_agent(h)
#Compress if GZIP_ENCODED is true
if GZIP_ENCODED:
#Don't forget to set the header!
h.putheader('Accept-Encoding','gzip')
request_body = zlib.compress(request_body)
self.send_content(h, request_body)
errcode, errmsg, headers = h.getreply()
if errcode != 200:
raise ProtocolError(
host + handler,
errcode, errmsg,
headers
)
self.verbose = verbose
try:
sock = h._conn.sock
except AttributeError:
sock = None
# Revisar si la respuesta incluye los encabezados de
# compresion y si este es gzip
if headers.has_key('accept-encoding') and \
headers['accept-encoding'] == 'gzip':
return self._parse_gzipped_response(h.getfile(), sock)
return self._parse_response(h.getfile(), sock)
The whole code and a small example is here:
ictcxmlrpc.tar.bz2
