• Each Web Service makes a description of its service available as a WSDL document. The WSDL describes technical details on how to access the service. Authorized remote clients can download the WSDL file and generate a stub that matches the SOAP service interface. Any RPC method in a Java stub can be called from clientside Java applications as if it were a local method. All leading Web Services toolkits provide WSDL-to-stub code generators.

• Web Services register themselves with central registry databases such as the UDDI registry. The client searches the UDDI, finds out the service it needs, fetches the WSDL file, generates the stub, and starts calling remote methods.

16.2 Introducing kSOAP

To build Web Services clients on J2ME devices, we first need a J2ME-compatible SOAP parser. Most standard Java SOAP libraries (such as the Apache Axis and Java Web SDK) are too heavy for small devices. The open source kSOAP project runs on all J2SE and J2ME platforms, including the MIDP. Built on top of the kXML parser, the entire kSOAP library is only 42 KB. However, as a trade-off for the lightness, kSOAP does not support the entire SOAP specification. It supports the most commonly used SOAP features and is sufficient for most Web Services that are currently available. Currently, almost all major kSOAP applications and tools are based on kSOAP release v1.2, which supports a core subset of SOAP 1.2 features. Although kSOAP v2.0 was released in November 2002, new features are still being developed and debugged. I recommend you use kSOAP v1.2 for important real-world applications but keep an eye on kSOAP v2.0. For developers, kSOAP v2.0 is easy once you understand the key concepts and programming models in v1.2. For the above reasons, we focus on kSOAP v1.2 in this chapter and give a programming tutorial for kSOAP v2.0 in Section 16.6.

16.2.1 What Is SOAP Parsing?

Every generic XML parser with namespace support understands SOAP messages and can extract information from them. In theory, we can always extract text information from a SOAP document using a generic XML parser and then convert those text strings to Java data objects when we need to use them. For example, the statement

int i = Integer.parseInt("123");

converts a text string "123" to an integer value 123. But such manual conversion burdens application programmers. Extracting Java data objects directly from a SOAP document provides a better approach. Enter the SOAP parser.

A SOAP parser is built on a generic XML parser with special type-mapping and text data-marshaling mechanisms. A SOAP parser understands the data-type information in SOAP messages and automatically converts the SOAP elements to Java data objects. The parser's real value is that it makes SOAP serialization and deserialization—and the entire wire protocol-transparent to object-oriented developers. The programmer just feeds Java objects into a SOAP writer, sends the message, waits for the server response, and then reads Java objects directly from the SOAP parser.

16.2.2 Simple Parsing Example Using kSOAP

Using kSOAP to make simple RPC calls is very easy. Listing 16.3 demonstrates how to make a SOAP call to Google's Web Services interface to obtain the spell suggestion of a phrase. The basic steps are the following:

Listing 16.3. Spell check using Google

// The URL of the SOAP interface

String endPointURL = "http://api.google.com/search/beta2"

String licenseKey = "Register with Google to get it";

public String spellCheck (String query) throws Exception {

  // Prepare request SOAP message in a memory object

  SoapObject method = new SoapObject("urn:GoogleSearch",

                                    "doSpellingSuggestion");

  method.addProperty("key", licenseKey);

  method.addProperty("phrase", query);



  // Prepare SOAP RPC call object.

  HttpTransport rpc = new HttpTransport(endPointURL, "\"\"");

  // Google uses 1999 SOAP standards.

  ClassMap cm = new ClassMap(Soap.VER10);

  rpc.setClassMap (cm);

  // Conduct RPC call through HTTP and

  // directly get results

  String spellSugg = (String) rpc.call (method);

  return spellSugg;

}

16.2.3 How the call() Method Works

The HttpTransport.call() method is local to us. But behind the scenes, it does a lot of remote work.

To fully understand how the call() method works, we need to look at its source code.

16.2.4 kSOAP Message Transport

Listing 16.4 shows the source code of the HttpTransport class. The HttpTransport class is currently the only message transport class in kSOAP. It is based on the very limited MIDP HTTP implementation. If you need HTTPS, cookie-aware HTTP, authentication-aware HTTP, or support for other network protocols, you have to plug in your own implementation.

Listing 16.4. The HttpTransport class

package org.ksoap.transport;



import java.io.*;

import javax.microedition.io.*;

import org.kxml.*;

import org.kxml.io.*;

import org.kxml.parser.*;

import org.ksoap.*;



public class HttpTransport {



  String url;



String soapAction = "\"\"";



SoapEnvelope requestEnvelope = new SoapEnvelope ();

SoapEnvelope responseEnvelope = new SoapEnvelope ();



HttpConnection connection;

OutputStream os;

InputStream is;

InputStreamReader reader;



private boolean connected = false;



public HttpTransport () { }



public HttpTransport (String url, String soapAction) {

  this.url = url;

  this.soapAction = soapAction;

}



public void setUrl (String url) {

  this.url = url;

}



public void setSoapAction (String soapAction) {

  this.soapAction = soapAction;

}



public void setClassMap (ClassMap classMap) {

  requestEnvelope.setClassMap (classMap);

  responseEnvelope.setClassMap (classMap);

}



public void call () throws I0Exception {



  ByteArrayOutputStream bos = new ByteArrayOutputStream ();

  XmlWriter xw =

    new XmlWriter (new OutputStreamWriter (bos));

  requestEnvelope.write (xw);

  xw.flush ();

  bos.write ('\r'); bos.write ('\n');

  byte [] requestData = bos.toByteArray ();

  bos = null; xw = null;



  try {

    connected = true;

    connection = (HttpConnection) Connector.open (url,

                                 Connector.READ_WRITE, true);

    connection.setRequestProperty ("SOAPAction", soapAction);

    connection.setRequestProperty ("Content-Type", "text/xml");

    connection.setRequestProperty (

         "Content-Length", ""+requestData.length);

    connection.setRequestProperty ("User-Agent", "kSOAP/1.0");

    connection.setRequestMethod(HttpConnection.POST);



    os = connection.openOutputStream ();

    os.write (requestData, 0, requestData.length);

    os.close ();



    requestData = null;

    is = connection.openInputStream ();

    reader = new InputStreamReader (is);

    XmlParser xp = new XmlParser (reader);

    responseEnvelope.parse (xp);

  } finally {

    if (!connected) throw

      new InterruptedIOException ();

    reset ();

  }

}



public Object call (SoapObject method) throws IOException {

  requestEnvelope.setBody (method);

  call ();



  if (responseEnvelope.getBody () instanceof SoapFault)

    throw((SoapFault)responseEnvelope.getBody ());



  return responseEnvelope.getResult ();

}



public void call (XmlIO request, XmlIO result)

                           throws IOException {

  requestEnvelope.setBody (request);

  responseEnvelope.setBody (result);



  if (responseEnvelope.getBody () instanceof SoapFault)

    throw((SoapFault)responseEnvelope.getBody ());

}



public void reset () {

    connected = false;

    if (reader != null) {

      try { reader.close (); }

      catch (Throwable e) {}

      reader = null;

    }

    if (is != null) {

      try { is.close (); }

      catch (Throwable e) {}

      is = null;

    }

    if (connection != null) {

      try { connection.close (); }

      catch (Throwable e) { }

      connection = null;

    }

  }

}

16.2.5 kSOAP Stub Generators

The Google spell check example is very easy to code by hand. But things get complex quickly when the remote service returns complex data objects. It is a pain to sort through a long WSDL document and figure out the exact SOAP interface. For complex services, automatically generated client stubs from WSDL files prove useful. For example, both Apache Axis and Sun Java Web Services Developer Pack offer WSDL2Java tools.

However, kSOAP is a small footprint library, not a complete Web Services toolkit. It lacks the tools to automatically generate client stubs. Fortunately, several J2ME IDEs provide such tools. kSOAP is adopted by IBM WebSphere Studio Device Developer (WSDD, a J2ME IDE based on Eclipse), SunONE Studio and CodeWarrior Wireless Studio as their default mobile Web Services client library. All of them offer GUI wizards that ask you the URL to the WSDL file and automatically generate the stubs into your current project source directory.

16.3 kSOAP Explained

In this section, we look under the hood to understand exactly how kSOAP v1.2 works. We also learn how to customize kSOAP for our special needs. For the impatient readers, a summary of the kSOAP API is available in Table 16.2 in Section 16.3.5. Now, let's start by examining how kSOAP maps SOAP data structures to Java objects.

16.3.1 The Default Mapping

By default, kSOAP understands the mappings described in Table 16.1. Notice that xsd:float is not supported due to the lack of Float support in CLDC v1.0.

When a simple kSOAP parser encounters a SOAP element, the parser reads the XML element into a Java object according to the following rules:

• If the SOAP element is one of the default primitive types in the table above, it converts to a Java object of a matching type.

• If the SOAP element has no children (a primitive element) but is not a default type, it converts to a SoapPrimitive object. We can retrieve the element's original SOAP type information from the SoapPrimitive.getNamespace() and SoapPrimitive.getName() methods. We can access the element's string value from the SoapPrimitive.toString() method.

• If the SOAP element has children (a complex element), it converts to a KvmSerializable object. KvmSerializable is an interface; the kSOAP package provides the interface's convenience implementation: SoapObject. Similar to SoapPrimitive objects, you can retrieve the element's original SOAP type information from the SoapObject.getNamespace() and SoapObject.getName() methods.

• A complex element's children convert to properties inside the parent SoapObject according to the above three rules in this list. Each property also has an associated PropertyInfo object containing information such as the SOAP element name and the property's Java object type. The PropertyInfo class supports element namespace in kSOAP v2.0 beta.

16.3.2 Object Structure

Since a SoapObject can take other SoapObjects as properties, we can use SoapObject to express complex hierarchical structures. An analogy to the XML DOM can be drawn here: The SoapObject resembles the parent node, and the property and PropertyInfo pairs resemble child nodes.

In simple cases, like the Google spell check example where the document contains only one primitive SOAP element, we can directly retrieve the mapped Java object or SoapPrimitive object through the SoapEnvelope.getBody() or SoapEnvelope.getResult() methods. If the SOAP message contains more information, the above two get methods return a root SoapObject, which contains the entire hierarchy of Java objects mapped from the SOAP elements according to the above rules.

Figure 16.2 shows such a structure. Each node on the tree represents a SOAP element in the parsed document. The original SOAP element name is stored in the PropertyInfo object, and the SOAP type attribute name/namespace is stored in the property object. Arrows represent property relationships. You can find the Java data objects' original SOAP type information by searching the mappings in the ClassMap object, which contains the information in the above table (more on ClassMap later). As we can see, the root SoapObject lacks a pairing PropertyInfo. Therefore, we lose the root element's name after parsing. However, that is not an issue when we read SOAP documents, since element names normally serve only as indexes for accessing Java data objects, and we do not need an index to access the root element.
 

16.3.3 Custom Mapping Through Data Marshal

To further automate the marshaling process to handle custom-defined types, we must prepare the parser for two tasks:

• The parser must know the mapping relationship between custom SOAP types to custom Java types. We complete custom mapping by adding matching type pairs to the current parser's ClassMap object.

Since all SOAP data are presented in plain text strings, the parser must know how to convert the string to a desired Java object. The parser converts the string through a Marshal object, which is registered with the parser's corresponding custom SOAP and Java type pair in the ClassMap object.

 

kSOAP provides a standard way to add custom type mapping and marshaling capabilities through the implementation of interface Marshal. The source code for the Marshal interface is shown in Listing 16.5. It declares a method to register itself with ClassMap and two call back methods.

Listing 16.5. The Marshal interface

public interface Marshal {

  public Object readInstance (SoapParser parser,

                  String namespace, String name,

          ElementType expected) throws IOException;



  public void writeInstance (SoapWriter writer,

               Object instance) throws IOException;



  public void register (ClassMap cm);

}

The kSOAP download package provides three Marshal implementations as both convenience tools and programming examples. Let's look at the MarshalDate class, which marshals xsd:dateTime type elements, for an example. Method Marshal.readInstance() actually reads the text string from the SOAP element and then converts it to a Java object. In the case of the MarshalDate,readInstance()'s source code is shown in Listing 16.6

Listing 16.6. The readInstance() method in the MarshalDate class

public Object readInstance (SoapParser parser,

                String namespace, String name,

        ElementType expected) throws IOException {



  parser.parser.read (); // Start tag

  Object result =

    IsoDate.stringToDate (parser.parser.readText (), IsoDate.DATE_TIME);

  parser.parser.read (); // End tag

  return result;

}

Similarly, method Marshal.writeInstance() describes how to serialize the Java object to a SOAP text string. In kSOAP writers, the method offers a more elegant alternative to using SoapPrimitive objects to handle unknown types. Interested readers can find the source code for that function from kSOAP source distribution. Method Marshal.register() (Listing 16.7) adds the matching custom SOAP type and Java type pair, as well as their processing Marshal object, to a ClassMap object.

Listing 16.7. The register() method for MarshalDate

public void register (ClassMap cm) {

  cm.addMapping (cm.xsd, "dateTime", MarshalDate.DATE_CLASS, this);

}

Now, let's look at a more complex example that demonstrates the use of SoapObject and Marshal.

16.3.4 A More Complex Example

The example scenario is a response from a stock trade SOAP service. The message is shown in Listing 16.8.

Listing 16.8. Demo stock trade response message

Michael Yuan XYZ 1000 true 123.45 2003-03-05T23:20:50.52Z

Listing 16.9 illustrates how to parse and retrieve information from the complex structure. Notice that the Price element is wrapped in a SoapPrimitive because xsd:float is not supported; the ExecTime element is correctly marshaled into a Java Date object because of the use of the MarshalDate.

Listing 16.9. Parsing the stock trade response

// Feed the response message into a

// reader object

XmlParser xp = new XmlParser (reader);



// Register Marshal for "xsd:dateTime" type

ClassMap cm = new ClassMap (Soap.VER12);

Marshal md = new MarshalDate ();

md.register (cm);

SoapEnvelope envelope = new SoapEnvelope (cm);

envelope.parse (xp);



// Get the parsed structure

SoapObject orderStatus = (SoapObject) envelope.getResult();



String customerName = (String)orderStatus.getProperty("CustomerName");



// Second layer of SoapObject

SoapObject transaction =

  (SoapObject)orderStatus.getProperty("Transaction");

String symbol = (String) transaction.getProperty ("Symbol");

Integer share = (Integer) transaction.getProperty ("Share");

Boolean buy = (Boolean) transaction.getProperty ("Buy");

SoapPrimitive price = (SoapPrimitive)transaction.getProperty("Price");

Date execTime = (Date) orderStatus.getProperty ("ExecTime");

In addition to the MarshalDate class, kSOAP provides two other Marshal implementations: MarshalBase64 and MarshalHashtable. Base64 is a method for encoding a binary stream into an ASCII string so that it can transport through email or XML/SOAP. MarshalBase64 marshals an xsd:based64Binary element into a Java byte array. kSOAP does not support SOAP attachments, but MarshalBase64 should allow users to send/receive binary data chunks. MarshalHashtable marshals a SOAP Map element into a Java hashtable.

16.3.5 Recap: The kSOAP API

We have covered almost all public classes and interfaces in the org.ksoap and org.kobjects.serialization packages. In recap, the classes and their descriptions are shown in Table 16.2.

16.4 Advanced kSOAP

Now we have seen the basic features of kSOAP. In addition, kSOAP offers limited support for SOAP arrays and document validation. We discuss these two features in this section.

16.4.1 Arrays

One of the important data types in any programming language is the array. kSOAP reads a SOAP array into a Java java.util.Vector object. Method Vector.elementAt(i) extracts the array's nth object. Depending on the arrayType, this object could be a SoapObject, a SoapPrimitive, a default Java type, or a marshaled Java object.

16.4.2 Validate Documents Using SoapTemplate

In the above examples, we take in only SOAP documents and parse them into SoapObjects as they are. However, in many cases, we require the response message to follow certain formats and wish the parser to validate it during the parsing. For example, we might require that the n:transaction-type SOAP elements in the stock trade response (Listing 16.8) contain an xsd:string value Symbol, an xsd:int value Share, an xsd:boolean value Buy, and an xsd:float value Price.

Our old friend, the ClassMap class, can validate the message. We must add into the ClassMap object a SoapObject template associated with the current parser. The SoapObject template is an empty SoapObject with information about the parent SOAP type, children (properties) element names, and their Java types. Again, the children can be templates themselves, which allows us to construct arbitrarily complex templates.

We add the SoapObject template by calling the ClassMap.addTemplate() method (Listing 16.10).

Listing 16.10. Validate and parse the stock trade response

XmlParser xp = new XmlParser (reader);



ClassMap cm = new ClassMap (Soap.VER12);



// Register Marshal for "xsd:dateTime" type

Marshal md = new MarshalDate ();

md.register (cm);



// Register the template for validation

SoapObject so = new SoapObject ("http://myns", "transaction");

so.addProperty ("Symbol", new String (""));

so.addProperty ("Share", new Integer (0));

so.addProperty ("Buy", new Boolean (true));

so.addProperty ("Price", new SoapPrimitive ("xsd", "float", ""));

cm.addTemplate (so);



SoapEnvelope envelope = new SoapEnvelope (cm);

envelope.parse (xp);



SoapObject orderStatus = (SoapObject) envelope.getResult();

// ... ...

If the parsing succeeds, we can proceed to extract data from orderStatus, as illustrated in Listing 16.9. If the SOAP message's n:transaction element fails to conform to the corresponding SoapObject template, the parser throws an exception and stops.

16.5 More kSOAP Examples

In this section, let's check out two kSOAP applications for real-world services.

16.5.1 The Google Web Services API Demo

In the previous sections, we demonstrated how to invoke a simple Google spell check Web Service. In fact, Google offers much more than the simple service. With a proper Google ID (free registration), we can use the Google Web Services interface to search the Web and get any Web site's Google cache. I developed a complete Google API toolkit based on kSOAP. The toolkit and its MIDP UI driver are available for download from this book's Web site (see "Resources"). The Web search service has the most complex interface. The response SOAP document has three arrays of equal length: They contain the search result sites' URLs, titles, and cache sizes respectively. We can use a Java data object SearchResults to encapsulate the Google search results (see Listing 16.11).

Listing 16.11. Class SearchResults encapsulates the Google search results

public class SearchResults {



  public Vector urls, titles, sizes;

  public int estimatedTotalResultsCount;

  public String searchTime;



  public SearchResults () {

    urls = new Vector ();

    titles = new Vector ();

    sizes = new Vector ();

    estimatedTotalResultsCount = 0;

    searchTime = "0.0";

  }



  public int getSize () {

    return urls.size();

  }

}

Listing 16.12 demonstrates how to send out a search query and marshal the response to a SearchResults object.

Listing 16.12. Google search

// The remote search method cannot return

// all the search results -- there could be

// tens of thousands of web sites.

//

// The "start" parameter specifies the index of

// the first entry to return.



public SearchResults search (String query,

                 int start) throws Exception {

  // Prepare request SOAP message in a memory object

  SoapObject method = new SoapObject("urn:GoogleSearch",

                                     "doGoogleSearch");

  // The free license key

  method.addProperty("key", licenseKey);

  // The query string to search

  method.addProperty("q", query);

  // The start index to return

  method.addProperty("start", new Integer(start));

  // Number of results to return

  // from the start index

  method.addProperty("maxResults", new Integer(10));

  method.addProperty("filter", new Boolean(true));

  method.addProperty("restrict", "");

  method.addProperty("safeSearch", new Boolean(false));

  method.addProperty("lr", "");

  method.addProperty("ie", "latin1");

  method.addProperty("oe", "latin1");



  // Prepare SOAP RPC call object.

  HttpTransport rpc = new HttpTransport(endPointURL, "\"\"");

  // Google uses 1999 SOAP standards.

  ClassMap cm = new ClassMap(Soap.VER10);

  rpc.setClassMap (cm);

  // Conduct RPC call through HTTP and get results

  SoapObject so = (SoapObject) rpc.call (method);

  SearchResults sr = new SearchResults ();

  sr.estimatedTotalResultsCount = ((Integer) so.getProperty(

         "estimatedTotalResultsCount")).intValue ();

  sr.searchTime = ((SoapPrimitive) so.getProperty(

                          "searchTime")).toString ();



  Vector items =

    (Vector) so.getProperty("resultElements");

  for (int i = 0; i < items.size(); i++) {

    SoapObject item = (SoapObject) items.elementAt (i);

    sr.sizes.addElement (item.getProperty ("cachedSize"));

    sr.urls.addElement (item.getProperty ("URL"));

    sr.titles.addElement (item.getProperty ("title"));

  }

  return sr;

}

16.5.2 SmartPhrases

SmartPhrases is a mobile dictionary and thesaurus, smart spelling checker, and real-world usage checker. It is an award-winning application developed by Ju Long and Michael Yuan. Its source code is freely available from this book's Web site.

To be able to access dictionaries from devices, SmartPhrases includes a SOAP dictionary gateway. The gateway connects to free Internet dictionary servers using the TCP/IP-based DICT protocol. It then exposes dictionary query services through a simple SOAP interface using Apache Axis. Any mobile or desktop SOAP client can access the backend dictionary from the gateway service.

SmartPhrases utilizes Google Web Services to go beyond simple dictionaries: Google allows the user to check the usage of a phrase in real-world scenarios; it has a modern vocabulary list and makes spell suggestions for the entire phrase. The architecture of the application is shown in Figure 16.3.

 

public String spellCheck (String query) throws Exception {

  // Prepare request SOAP message in a memory object

  SoapObject method = new SoapObject("urn:GoogleSearch",

                                "doSpellingSuggestion");

  method.addProperty("key", licenseKey);

  method.addProperty("phrase", query);



  // Google uses 1999 SOAP standards.

  SoapEnvelope envelope =

     new SoapSerializationEnvelope(SoapEnvelope.VER10);

  // Set the request object

  envelope.bodyOut = method;

  // Prepare SOAP RPC call object.

  HttpTransport rpc = new HttpTransport(endPointURL);

  // Make the method call (SoapAction is not needed)

  rpc.call(null, envelope);

  // Retrieve marshaled return value

  SoapObject ret = (SoapObject) envelope.bodyIn;

  String spellSugg = (String) ret.getProperty("return");

  return spellSugg;

}

import org.ksoap2.*;

import org.ksoap2.serialization.*;

import org.ksoap2.transport.*;



public class GoogleSearch {



  // other methods ...



  public byte [] getCache (String url) throws Exception {

    // Prepare request SOAP message in a memory object

    SoapObject method = new SoapObject("urn:GoogleSearch",

                                      "doGetCachedPage");

    method.addProperty("key", licenseKey);

    method.addProperty("url", url);



    // Google uses 1999 SOAP standards.

    SoapEnvelope envelope =

      new SoapSerializationEnvelope(SoapEnvelope.VER10);

    // Set the request object

    envelope.bodyOut = method;

    // Add Base64 marshal.

    // It is for both request and response data.

    Marshal mb = new MarshalBase64 ();

    mb.register((SoapSerializationEnvelope)envelope);

    // Prepare SOAP RPC call object.

    HttpTransport rpc = new HttpTransport(endPointURL);

    // Make the method call (SoapAction is not needed)

    rpc.call(null, envelope);

    // Retrieve marshaled return value

    SoapObject ret = (SoapObject) envelope.bodyIn;

    byte [] cache = (byte []) ret.getProperty("return");

    return cache;

  }

}

public class MarshalBase64 implements Marshal {



  static byte [] BA_WORKAROUND = new byte [0];

  public static Class BYTE_ARRAY_CLASS =

                    BA_WORKAROUND.getClass ();



  public Object readInstance (XmlPullParser parser,

                     String namespace, String name,

                             PropertyInfo expected)

         throws IOException, XmlPullParserException {



    Object result = Base64.decode(parser.nextText ());

    return result;

  }



  public void writeInstance (XmlSerializer writer,

                     Object obj) throws IOException {



    writer.text (Base64.encode ((byte[]) obj));

  }



  public void register (SoapSerializationEnvelope cm) {

    cm.addMapping (cm.xsd, "base64Binary",

              MarshalBase64.BYTE_ARRAY_CLASS, this);



    cm.addMapping (SoapEnvelope.ENC, "base64",

              MarshalBase64.BYTE_ARRAY_CLASS, this);

  }

}