WSRF::Lite
==========
WSRF::Lite version 0.8.2.2 by Mark Mc Keown, mark.mckeown@man.ac.uk
with contributions from Andrew Porter, Sven van den Berghe, Stephen Pickles,
Mark Riding, Jonathan Chin, Mary Thompson, Bruno Harbulot, Mary Thompson,
Alex Peeters, Bjoern A. Zeeb, Glen Fu, Doug Claar, John Newman and
Jamie Vicary.
Stefan Zasada (sjzasada@lycos.co.uk) did most of the work implementing
WS-Security - a big thanks goes to Savas Parastatidis
(http://savas.parastatidis.name/) for helping us get it working with
.NET.
If you have any problems, bugs, feature requests or questions on
WSRF::Lite you can e-mail me at mark.mckeown@manchester.ac.uk.
There is now a bugzilla for WSRF::Lite at
http://kato.mvc.mcc.ac.uk/bugzilla/ and Wiki pages at
http://kato.mvc.mcc.ac.uk/rss-wiki/WSRFLite.
Contents
========
* Introduction.
* Requirements.
* Changes from v0.3
* Changes from v0.4
* Changes from v0.5
* Changes from v0.6
* Changes from v0.7
* Changes from version 0.8 - adding AJAX
* WSRF.
* Running the Container.
* Running the Secure Container.
* Process based Resources.
* MultiSession Resources.
* File based Resources.
* Logging.
* WSDL.
* WS-Security
* docs
* Further Reading.
Introduction
============
This is a Container for running WSRF compliant Services
written in Perl, the Container itself is written in Perl.
The Container can be run with or without SSL for security.
To create a WSRF Service it is simply a matter of writing
a Perl module, making sure the module inherits from a base
class that provides the standard WSRF functionality and
placing the module in the correct directory. You may also be
required to provide a Factory module, however most of the
default functionality can be inherited from a base Factory
class. All the work is based on the excellent SOAP::Lite
module that can be used to build Web Services in Perl -
(http://www.soaplite.com/). A very good site for SOAP::Lite
information is:
http://www.majordojo.com/archives/cat_soaplite_solutions.html
Further pointers to interesting papers and
Web sites are in the Further Reading section.
Requirements
============
This version of WSRF::Lite requires SOAP::Lite version 0.67
(or later) which can be downloaded from http;//www.soaplite.com/
or retrieved from CPAN.
The Container and associated modules are written in Perl and
have been tested and run on Perl 5.6 and later.
To use the Container you need to install the SOAP::Lite, LWP,
XML::DOM, Sys::Hostname::Long, DateTime, DateTime::Format::Epoch
DateTime::Format::W3CDTF and Storable modules.
These can all be installed using CPAN.
To use WS-Security for signing or verifying messages you will
also need: Crypt::OpenSSL::RSA, MIME::Base64, Digest::SHA1,
and Crypt::OpenSSL::X509 and XML::CanonicalizeXML - these can all
be retrieved from CPAN.
If you cannot use CPAN, because you do not have root privileges
for example you, you can install required modules by downloading
them and building them by hand. Another possibility is to
configure CPAN to run as a normal user, the file CPAN_help gives
pointers on how to do this.
If you run into problems with Module::Build, which is used by
DateTime see
http://kato.mvc.mcc.ac.uk/rss-wiki/SoftwareInstallation/ModuleBuild
You should update your CPAN shell issuing a "install Bundle::CPAN"
before doing anything else. People have reported error (unresolved
version conflicts) while installing required packages
(DateTime::Format::Epoch) with prior versions of CPAN (1.7x)
Changes from v0.3
=================
v0.3 uses the latest version of the WS-Addressing
specification, the major change is that ResourceProperties
have been dropped from WS-Addressing. WSRF::Lite used
ResourceProperties to route messages to particular
backend WS-Resources. The old WS-Addressing Endpoint
References (EPRs) for WSRF::Lite looked like:
http://vermont.mvc.mcc.ac.uk/WSRF/Counter
346435645764576456
The new version of WS-Addressing EPRs is:
http://vermont.mvc.mcc.ac.uk/WSRF/3464645764576456
So each WS-Resource will have a different value for the Address element.
This change may cause some older WSRF::Lite clients/services to break.
Also WS-Addressing may yet still change - it has not completed the
standardisation process. (If you have already developed services using
a previous version of WSRF::Lite it may be wise to wait for the version
of WSRF::Lite that implements the specifications when they become
standards)
This version of WSRF::Lite also provides better support for WS-Addressing
on the client side, the client will now automatically create the proper
SOAP headers if you pass them a WS_Address object:
$ans= WSRF::Lite
-> uri($uri) #set the namespace
-> wsaddress(WSRF::WS_Address->new()->Address($target)) #location of service
-> $func(@ARGV); #function + args to invoke
Note the use of WSRF::Lite instead of SOAP::Lite. Instead of using
->proxy($target) we now use ->wsaddress passing it a WS_Address
object created using the endpoint of the service. This client will
automatically create the correct WS-Addressing SOAP Headers, there
is no longer any need to create a SOAP::Header object with the
WS-Addressing information.
The WS_Address funtion 'from_envelope' takes either an WS-Addressing
EPR encoded as XML or as a SOM object and returns a WS_Address object:
eg.
$som = WSRF::Lite
->uri("http://localhost:50000/CounterFactory")
->wsaddress(WSRF::WS_Address->new()
->Address('http://localhost:50000/CounterFactory/'))
->createCounterResource();
$wsa = WSRF::WS_Address->from_envelope($som);
$ans = WSRF::Lite
->uri("http://localhost:50000/Counter")
->wsaddress($wsa)
->add('1');
This version of WSRF::Lite provides experimental support for signing
SOAP messages using x509 digital certificates according to the WS-Security
standard and WS-I Basic Security Profile. You should find two sample
client scripts ./client-scripts/sample_WS-Security_client.pl and
./test/signature_checker.pl. The WS-Security signing has been tested
against a .NET service and a service implemented with the Sun Java Web
Service Developer Pack (JWSDP) however more testing needs to be done (see
TODO).
There has been some changes to the Container scripts to support command
line arguments to set the port and hostname to use.
The WSRF::Container module has been re-written to make it eashier to
add HTTP handlers - it should be easy to add a handler to support
uploading files for example.
WS-Resources hosted in the Container now support HTTP GET, if you
use GET (from the LWP Perl package) on a WS-Resource endpoint you
get a copy of the WS-Resource's ResourceProperty document in a
SOAP message. You can also use a HTTP DELETE to destroy a WS-Resource
eg.
GET -m DELETE http://localhost:50000/WSRF/Counter/Counter/354634563456
(curl can also be used, there are problems with wget, see TODO)
WSRF::Lite v0.4 implements the latest version of the WSRF specifications,
these add the following operations: GetResourcePropertyDocument (gets
the ResoureProperty document, effectively all the Resource Properties,
of the WS-Resource, UpdateResourceProperty (updates a WS-Resource),
InsertResourceProperty (insert a property into the WS-Resource) and
DeleteResourceProeprty (delete a property from a WS-Resource). The last
three operations are effectively a breaking down of the functionality of
the SetResourceProperty operation - which is of course still available.
The WS specifications versions implemented are:
WS-Addressing -
http://www.w3.org/TR/2005/WD-ws-addr-core-20050331/
WSRF -
http://docs.oasis-open.org/wsrf/wsrf-ws_resource-1.2-spec-cd-01.pdf
http://docs.oasis-open.org/wsrf/wsrf-ws_resource_properties-1.2-spec-cd-01.pdf
http://docs.oasis-open.org/wsrf/wsrf-ws_resource_lifetime-1.2-spec-cd-01.pdf
http://docs.oasis-open.org/wsrf/wsrf-ws_service_group-1.2-spec-cd-01.pdf
http://docs.oasis-open.org/wsrf/wsrf-ws_base_faults-1.2-spec-cd-01.pdf
WS-Security -
http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0.pdf
http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0.pdf
http://www.ws-i.org/Profiles/BasicSecurityProfile-1.0.html
Changes from version 0.4
========================
The major changes is adding support for Makefile.PL etc, this
is the precursor for putting the module into CPAN.
The SContainer script now uses HTTP::Daemon::SSL to provide
support for SSL connections. Depending on the version of the
Linux kernel there were problems with the SSL connections
related to blocking in the OpenSSL libraries with WSRF::Lite
0.4. These have been handled by moving to using the
HTTP::Daemon::SSL module - thanks to Ross Nicoll for pointing
out the bug.
The module used to canonicalize the XML for signing SOAP messages
has been improved - in the original version there was a memory
leak. The module is now available on CPAN.
The client scripts now check the SOAP response from a service to
see if it has been digitally signed - it will issue a warning if
the message has not been signed.
In the sample WS-Resource modules you should see a number
of environmental variables which have been commented out:
# If these $ENV are set the SOAP message will be signed
# Points to the public key of the X509 certificate
#$ENV{HTTPS_CERT_FILE} = $ENV{HOME}."/.globus/hostcert.pem";
# Points to the private key of the cert - must be unencrypted
#$ENV{HTTPS_KEY_FILE} = $ENV{HOME}."/.globus/hostkey.pem";
# Tells WSRF::Lite to sign the message with the above cert
#$ENV{WSS_SIGN} = 'true';
If these are uncommented the WS-Resource will attempt to sign
the SOAP responses it sends back to the client. The sample
client scripts in the sample-clients directory have code that
shows you how to check for a digital signature.
A number of smaller bugs have also been fixed since WSRF::Lite
version 0.4. The WSRF specification has not yet finished going
through the standardization process so the namespaces are still
not fixed - so expect the namespaces to change at some time in
the future.
Changes from version 0.5
========================
Switched back to using WSRF::SSLDeamon for the SContainer,
the HTTP::Daemon::SSL module blocks on later versions of
Linux so I switched back to WSRF::SSLDeamon which was modified
to avoid the blocking behaviour.
Added some POD.
Changed the WSDL for the Counter WS-Resources - thanks to
Bruno Harbulot.
Changes from version 0.6
========================
Added the WSRF::WSRP::NotInsert{count} = 1 to counter examples,
this means a new value cannot be inserted using the SetResourceProperty
operation.
Fixed a memory leak in the client code, thanks to glukasz AT yahoo.com.
Added support for using HASHs as ResourceProperties, eg
$WSRF::WSRP::ResourceProperties{bar} = bless { foo => "mmmh" }, "BAR";
$WSRF::WSRP::PropertyNamespaceMap->{bar}{prefix} = "mmk";
$WSRF::WSRP::PropertyNamespaceMap->{bar}{namespace} =
"http://www.sve.man.ac.uk/Counter";
bar is a blessed HASH - the HASH will be serialized automatically for us
When using getResourceProperties the XML will look like:
mmmh
Changes From version 0.7
========================
This release contains mostly bug fixes from the previous
version. For WS-Resources that use Files to store state the
module Storable is used to serialize the state rather than storing
the state as XML in the file. This fixed some problems with
serializing the XML and also reduced the size of the files.
Support has been added to support "private" data for WS-Resources
that store state in files. This is data that has not been declared
as a ResourceProperty but which needs to stored between calls to
the WS-Resource. The data is stored in a hash called
%WSRF::WSRP::Private and is stored along with the ResourceProperties
in the file. An example of how to use the %WSRF::WSRP::Private is
given in ./modules/Session/Counter/Counter.pm
Changes From version 0.8 - adding AJAX
======================================
The major change to this version of WSRF::Lite is the addition of the
support for AJAX
(http://adaptivepath.com/publications/essays/archives/000385.php)
access to WS-Resources. Previous versions allowed you to do a
HTTP GET on the wsa:Address element from the EPR of a WS-Resource
and receive the ResourceProperties of the WS-Resource as XML.
For example entering the wsa:Address into the input field
of a Web browser caused it to display the ResourceProperties.
Now support has been added for XSLT, if an XSLT file is included
in WSRF::Lite for a WS-Resource then the Web browser will use
the XSLT to transform the XML of the ResourceProperties into another
format, XHTML for example. This allows the WS-Resource developer to
control how the ResourceProperties will be displayed in a Web
browser. The XHTML can also include a link to Javascript which the
Web browser will execute. The WS-Resource developer can create
Javascript so that it will automatically update the Web page
every few seconds. The WS-Resource developer can also use Javascript
to allow the Web browser user to update the ResourceProperties.
In WSRF::Lite HTTP GET is mapped to WSRF GetResourcePropertyDocument,
HTTP PUT to WSRF PutResourcePropertyDocument and HTTP DELETE to WSRF
Destroy. The Javascript can use HTTP PUT, GET and DELETE to manipulate
the WS-Resource. For HTTP PUT the Javascript client sends a version
of the WS-Resource's ResourceProperty document to replace the
existing one. Within WSRF::Lite this is done by invoking the
PutResourcePropertyDocument operation. Unfortunately WSRF::Lite does
not provide very much support for PutResourcePropertyDocument as its
functionality can vary so much for different types of WS-Resources.
For HTTP PUT only the XML of the ResourceProperty document needs to be
sent, it does not need to be encoded in SOAP.
XSLT, CSS and Javascript files have been provided for the Counter
example WS-Resource, you will find them in the same directory as the
Counter module file. The files must end in .xsl, .css and .js
respectively.
Arjun Sen did a Masters on using AJAX and WSRF::Lite to create clients,
his thesis is available at
http://www.sve.man.ac.uk/Research/AtoZ/ILCT/arjunsthesis.
Also in the docs directory is a paper on how to use AJAX with WSRF::Lite.
Web Service Resource Framework, WSRF
====================================
The following is a rough description of WSRF, it
is not perfect since the definition is changing
as WSRF goes through standardisation.
WSRF is the replacement for OGSI, OGSI was initially
concerned about Grid Services which were "stateful
Web Services" that provided standard mechanisms for
accessing and managing this state. OGSI was pretty
radical and WSRF is closer to main stream Web Service
thinking. The basic problem is that you want to allow
a client to manage some state/Resource through a standard
set of interfaces, WSRF provides a way of doing this (
there are others...).
Effectively you have a Web Service that acts as a front
end to a number of stateful resources, these resources
can be created and destroyed and may naturally expire.
The WSRF uses WS-Address to identify a WS-Resource,
in the WS-Address there is a Resource identifier that the
client MUST send to the Web Service in the SOAP Header
if it wants to use that Resource. For full details on
WSRF see http://www.globus.org/wsrf, in particular the
"Modeling Stateful Resources with Web Services" is a good
introduction.
WSRF::Lite provides a Container for hosting WSRF
Web Services, it also provides a number of base classes
for building WSRF Web Services that provide the
required operations: GetResourceProperty, Destroy,
GetMultipleResourceProperties, SetTerminationTime,
and SetResourceProperties.
WS-Resources can be implemented in various ways in
WSRF::Lite: a process can be used to hold the state of
a single resource (Process based WS-Resources), a process
can manage the state of a number of Resources (MultiSession
WS-Resources) and finally the state can be stored in a file
between calls to the service (File based WS-Resources).
Storing the state in a file between calls to the WS-Resource
means the WS-Resource can survive a Container failure.
Potentially the state could be stored in a Database.
See the section on Further Reading for pointers to
more detailed descriptions of WSRF etc...
Running the Container
=====================
The Container uses the directory /tmp/wsrf and /tmp/wsrf/data
- so you need to create these directories before starting the
Container.pl. (These paths can be chnaged by setting
$WSRF::Constants::SOCKETS_DIRECTORY and $WSRF::Constants::Data
in the Container script.)
The Container script is based on sample code from the
Perl CookBook, it is quite short and easy to modify.
To start the Container run ./Container.pl The Container
uses port 50000 by default - this can be changed in the
Container.pl script or using the command line argument -p.
(You can use any port you want, 50000 is just an arbitrary
number)
When the Container starts it should advertise which URL
it is listening on. Sometimes depending on how your
machine is configure it will use "localhost" as the
hostname, you can override this behaviour in the Container
script or using the -h option.
The Environmental Variable WSRF_MODULES should point to the
directory which holds the WSRF modules. The provided Modules
directory is ./modules and has a number of simple Counter
examples.
The Container and SContainer script now take command line arguments
for the hostname and port:
./Container -h vermont.mvc.mcc.ac.uk -p 40000
Running the Secure Container
============================
SContainer.pl is the secure version of the Container - you
will need to change the script to point to your host certificates,
the container needs to use a server certificate. The private key of
the host certificate should be unencrypted to allow the Container to
read it, the key should be read only by the user running the Container.
The same client scripts will work with the SContainer as the standard
Container but they will need to be modified to point to the user's
x509 digital certificate, also when they are invoked they will promote
for the password for the certificate private key unless it is unencrypted.
Process Based WS-Resources
==========================
In this case the container forks of a process to handle the
WS-Resource, it then passes SOAP to and from the service using
a UNIX domain socket to the outside world. There is an example
of the Counter Resource included in WSRF::Lite that uses process,
the Counter is just a simple counter that you can add and subtract
values from - the state of the counter is maintained between calls.
There are two modules involved in the process based Counter example:
./modules/Session/CounterFactory/Counterfactory.pm, the factory
that creates a counter resource for us and the actual counter
module, ./modules/WSRF/Counter/Counter.pm.
To create a counter resource use the client scripts provided in
./client-scripts eg.
wsrf_createCounterResource.pl http://localhost:50000/Session/CounterFactory/CounterFactory http://www.sve.man.ac.uk/CounterFactory
The README in the the ./client-scripts describes how to use the
other scripts to interact with the new Counter resource. The client
scripts check if the SOAP response from any service has been signed,
if it has not been you should see this is the output from the client:
SOAP Message not signed
WSRF::WSS::verify Fault - No Security Token in SOAP Header
By default the the sample WS-Resources do not sign their SOAP responses
so you can simply ignore this message.
Process based resource module files should be stored under
./modules/WSRF, the factory modules can be stored in ./modules/Session
since they do not have to maintain state between calls.
MultiSession Based WS-Resources
===============================
This is similar to the process based resource example, except
a single process manages the state of a number of resources.
The example Counter module is ./modules/MultiSession/Counter/Counter.pm,
there s no need for an extra factory module since the resource acts
as a factory itself.
To create a MultiSession based counter using the client scripts
run:
wsrf_createCounterResource.pl http://localhost:50000/MultiSession/Counter/Counter http://www.sve.man.ac.uk/Counter
MultiSession based resource module files should be stored in directories
under ./modules/MultiSession/
File Based WS-Resources
=======================
In this example the state of the service is stored in a file between
calls ie there are no extra processes forked of. The File based Counter
module file can be found in ./modules/Session/Counter/Counter.pm.
To start a file based counter resource use:
wsrf_createCounterResource.pl http://localhost:50000/Session/Counter/Counter http://www.sve.man.ac.uk/Counter
File based modules should be stored under the Session directory.
Logging
=======
Anything sent to STDOUT or STDERR by a Resource will end up in a file in
./modules/logs - the name of the file will be the same as the ID of the
Resource.
WSDL
====
In response to a HTTP GET the Container will return the WSDL for
a service eg:
curl http://localhost:50000/WSRF/Counter/Counter/45674576?WSDL
will return the WSDL for the service that uses the endpoint
http://localhost:50000/WSRF/Counter/Counter/45674576. To deploy a
WSDL file for a service create a WSDL file with the same name as
the module and with a ".WSDL" extension and put it in the same
directory as the module. The Container will dynamically set the
endpoint in the WSDL file for you when someone attempts to GET it.
For an example of a WSDL file see ./module/WSRF/Counter/Counter.WSDL
Unfortunately you have to create the WSDL file yourself, the
WSDL::Generator module might help you.
Configuring WS-Security
=======================
By default WSRF::Lite will sign all the WS-Addressing SOAP Headers,
a timestamp on the message, the actual X509 certificate used to
sign the message and the SOAP Body of the message. (The X509
certificate is included in the parts of the message that are signed
to prevent it being replaced by another certificate) This means
the messages signed by WSRF::Lite are very secure, however it may not
be necessary to sign all this for a particular message.
The things that should be signed are controlled by the
WSRF::WSS::ID_Xpath hash. You stop WSRF::Lite
from signing an element by setting to undef in the hash,
the following stops all the WS-Addressing elements being signed
(you could always just empty the hash)
$WSRF::WSS::ID_Xpath{To} = undef;
$WSRF::WSS::ID_Xpath{MessageID} = undef;
$WSRF::WSS::ID_Xpath{ReplyTo} = undef;
$WSRF::WSS::ID_Xpath{Action} = undef;
$WSRF::WSS::ID_Xpath{Timestamp} = undef;
$WSRF::WSS::ID_Xpath{BinarySecurityToken} = undef;
$WSRF::WSS::ID_Xpath{From} = undef;
$WSRF::WSS::ID_Xpath{RelatesTo} = undef;
It is not pretty using a global hash but it is necessary to fit in
with the way SOAP::Lite works. You should notice that the X509
certificate and timestamp is not included in the above hash, this is
because the WS-Security specification strongly recommends that they
should be signed. However, it is possible to override this using the
following:
$WSRF::WSS::sec_xpath= undef;
$WSRF::WSS::timestamp_xpath= undef;
The WSRF::WSS::ID_Xpath holds a set of XPaths to the elements in the
message that should be signed. WSRF::Lite iterates through this hash
and uses the associated XPath to sign that element. An example of an
Xpath expression for the WS-Addressing To header is:
$WSRF::WSS::ID_Xpath{To}=
'
(//. | //@* | //namespace::*)[ancestor-or-self::wsa:To]
';
To add another element to be signed simply add another entry into
the hash with the relevant XPath. If you are switching signing policies
in a particular program it might be wise to store a copy of
WSRF::WSS::ID_Xpath to fall back on.
docs
====
This directory contains a number of papers and tutorials on WSRF::Lite.
The README in the docs directory provides a brief description of each.
Further Reading
===============
Background to WSRF
------------------
To understand how things evolved to arrive at WSRF you
should read the following papers in order:
"The Anatomy of the Grid: Enabling Scalable Virtual Organizations"
http://www.globus.org/research/papers/anatomy.pdf
Introduction to Grids.
"The Physiology of the Grid: An Open Grid Services Architecture
for Distributed Systems Integration"
http://www.globus.org/research/papers/ogsa.pdf
"Grid Service" first introduced.
"The OGSI Primer"
https://forge.gridforum.org/projects/ogsi-wg/document/draft-ggf-ogsi-gridserviceprimer/en/1
A description of the Open Grid Service Infrastructure,
a formalization of the concepts introduced in the
Physiology paper. (NB the actual OGSI specification is
difficult to read)
"A Grid Application Framework based on Web Services Specifications
and Practices"
http://www.neresc.ac.uk/ws-gaf/documents.html
Provides a detailed critique of OGSI.
"Modeling Stateful Resources with Web Services"
"The WS-Resource Framework"
"From Open Grid Services Infrastructure to WS-Resource
Framework: Refactoring and Extension"
http://www.globus.org/wsrf
These provide an overview of WSRF, the successor
to OGSI.
WS-Security
-----------
The OASIS Technical Committee for WS-Security has useful documents
on WS-Security:
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss
The WS-I document "Security Challenges, Threats and Countermeasures"
provides good background on the issues addressed by WS-Security, it
is well worth a read - you can download it from here:
http://www.ws-i.org/Profiles/BasicSecurity/SecurityChallenges-1.0.pdf
Also worth reading is Stefan Zasada's MSc dissertation, hs did most
of the work implementing SOAP signing in WSRF::Lite as part of his
Masters project:
http://vermont.mvc.mcc.ac.uk/msc_dissertation.pdf
Other Web Services stuff of interest
------------------------------------
WS-Transfer
http://msdn.microsoft.com/ws/2004/09/ws-transfer/
From Microsoft, BEA et al, a competing specification to
WSRF
REST (Representational State Transfer)
http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
This is Roy Fieldings thesis - it describes how to
build a distributed system using HTTP, URIs etc.
Chapter 5 is well worth a read even if you are not
going to use REST.
"Data on the Outside versus Data on the Inside"
http://www-db.cs.wisc.edu/cidr/papers/P12.pdf
A very good paper by Pat Helland discussing
Web Services.
SOAP::Lite stuff
----------------
http://www.soaplite.com/
The Cookbook and Users Guide are worth reading.
http://www.majordojo.com/archives/cat_web_services.php
Provides advanced tips on using SOAP::Lite.
"Programming Web Service with Perl"
http://www.oreilly.com/catalog/pwebserperl/
by Randy J. Ray and Pavel Kulchenko
Discusses building Web Services with Perl,
lots of SOAP::Lite examples and a section
on REST.