Messaging is an important component in an increasingly service-oriented enterprise environment. In the past, providing the same service to multiple applications involved either rebuilding it each time or developing custom integration software. Messaging makes it easier to build or purchase specialized tools for particular applications, since a variety of applications can share a single expensive resource without each development team having to spend a lot of time developing interfaces.

Messaging provides great new opportunities to enterprise designers. Of course, there is a drawback: messages are by nature asynchronous. Messages sent by one application to another don't immediately arrive at the recipient, and the sender doesn't have any automatic confirmation of delivery. Of course, this situation can work to your advantage: you don't need 100% uptime for every system and network in your organization. And it's surprising how many applications don't require an immediate response, as long as the message is delivered.

On the integration front, centralized Extract, Transform, and Load (ETL)^[1] applications have begun to incorporate messaging support to support multiple different applications sharing a single ETL server. The systems accept commands and input data from messages. In custom development, applications that require expensive or difficult-to-maintain software can be centralized as well: if 10 web applications need to send faxes, a centralized fax server can provide that service to all of them, rather than requiring a fax modem on each web server with all the attendant hardware and administration. The web applications, whether running on Solaris, Linux, or something else entirely, can communicate with the fax service via messaging.

 ETL applications provide data translation and conversion for integration activities: synchronizing data between different schemas, translating from COBOL output files to relational databases, and so forth.

The patterns in this chapter focus on two aspects of messaging. We start by discussing message distribution strategies, including different ways to use a message within an application. The second half of the chapter presents patterns for baking messaging into your applications; they cover retrieving messages from a message service and developing strategies for processing messages in the most flexible way possible.

11.1 Messaging in J2EE

Most discussions of messaging in the J2EE world focus on JMS, the Java Message Service API, which is a core component of the J2EE standard. JMS provides access to heavy-duty messaging software. A JMS-enabled application can take advantage of a range of services, such as delivery guarantees, quality of service contracts, and an easy-to-use Java interface.

The patterns in this chapter apply to JMS applications, but many of them can be used alongside another message transport mechanism: trusty old Internet email. Email has gotten relatively short shrift in most texts on enterprise software development, and we're hard pressed to see why; it's easy to use, and the infrastructure is nearly ubiquitous. There's even a handy J2EE API, JavaMail, designed for dealing with it. We look at most of the patterns in this chapter through the lenses of both JMS and JavaMail. Of course, JavaMail and JMS are hardly equivalent, and we'll point out some of the differences in the next few pages.

11.1.1 Asynchronous Messaging Versus Web Services

This chapter doesn't cover message-oriented web services, partially because messaging is such a broad subject. If you think about it, most of what an application does can be cast in terms of messaging: method calls, for instance, are messages from one class to another, with immediate receipt and confirmation of delivery.

For our purposes, there's one major difference between asynchronous messaging and web services: web services are synchronous. When you're remotely accessing an EJB session bean or other Remote Procedure Call interface, it's expected that the service you're looking for will be available and online, and a web service is the same way. Web services use XML messages that conform to the SOAP specification, but they deliver the message immediately or not at all. The SOAP protocol doesn't mandate a delivery mechanism, but web services do. (HTTP—that's where the "web" part comes from.) These messages, like method calls, are synchronous: they happen immediately, and both sides of the exchange must coordinate with each other.

In this chapter, we're focusing on asynchronous messaging: communication between multiple systems in something other than real time. In an asynchronous exchange, the sender dispatches the message and goes on with other work. The recipient may receive the message milliseconds, minutes, hours, or days later, depending on the transport mechanism, system design, network conditions, and so forth (sunspots have been known to play a role here).

Even though the core web services concept is synchronous, that doesn't mean web services can't be deployed in support of an asynchronous application: you just need to write a little more code. Once you've done that, the underlying principles and patterns start to look a lot like the ones in this chapter.

11.2 Messaging and Integration

Back around the dawn of computing, sending a message between systems was easy: you wrote the data onto a magnetic tape, took the tape out of the computer, and carried it over to the other computer. Assuming the two computers shared a common data format, you were all set. When networks were invented, programmers devised integration techniques based on direct network connections. An application opened a connection to another application, and if the network was running and the target application was responding, the two systems could interact with each other, as long as they shared a common format for data. The problems with this approach should be familiar: the system wouldn't work if one application was down, or if there were intermittent network failures, or if the systems had different data formats. Building direct linkages and custom protocols were also a time-intensive and nonstandard approach, which had to be revisited with each new application or new version. And did we mention the data formats?

There are other approaches; many systems communicate with each other by reading and writing shared files, or by accessing shared tables within a database. The former method works well when two applications are running on the same physical machine or connected by a reliable network that supports file sharing. The latter works well for multiple applications and applications spread across a variety of systems. Both approaches have the benefit of being asynchronous: you don't need to have all the applications running in order for any single system to read or write data. But programmers still need to agree on formats and create the infrastructure to share the data resources. Programmers also have to be particularly careful of concurrency issues (see Chapter 10 for more on concurrency).

The original Unix mail system used the shared-files approach, allowing users on the same system to run the mail program at different times to leave messages for others and to read messages left for them. But with the rise of the Internet, and particularly with the development of always-on TCP/IP networks, users started to send messages to people using different computers, and even at different locations. The result was first UUCP and then SMTP, both of which provide a mechanism for routing messages between systems without having to actually create a direct connection between the sender and the recipient.

SMTP, in particular, did two things. First, it provided a standard, allowing different vendors and different platforms to send email to each other. MS Exchange, Lotus Notes, and Unix mail all use vastly different internal formats, but the common SMTP message and address formats made it largely a non-issue. Second, SMTP allowed administrators to set up an SMTP server for their domains. SMTP servers received messages from users and routed them to other systems for distribution. If the destination system was unavailable, the SMTP server could hold on to the outgoing message for a few hours and try again, rather than ignominiously failing.

SMTP servers were the first widely deployed messaging middleware. They introduced most users to the idea of asynchronous messaging, in which a message is sent and received at different times. In the last few years, asynchronous messaging has become a big deal in the enterprise development area because it provides the same advantages SMTP provides for email: it develops a standard method of packaging information and an infrastructure that allows messages to be sent reliably.

Figure 11-1 shows a simple use case diagram, where every single interaction could be implemented via a messaging approach. The customer emails a form to the software implementing the Purchase Product use case, which in turn sends a message to the software implementing the order processing use case, which, in turn, interacts with the Purchasing Server system actor via another set of messages.
 

 

11.2.1 Types of Messaging

Deciding to use asynchronous messaging in your application is a big step, but it's by no means the last. There are a number of ways to implement asynchronous messaging, each involving a different Quality of Service guarantee. The U.S. Postal Service, for instance, promises to deliver a first class letter within four days, and makes their best effort to deliver the letter intact. Most letters duly arrive in the expected timeframe and in reasonable condition. However, some letters get lost, and others are damaged. Because this isn't good enough for every snail-mail enabled application, the USPS provides a set of additional delivery services: return receipts, certified mail, Priority Mail, and Express Mail. An Express Mail letter is all but certain to arrive at its destination the following day, in good condition.

Messaging middleware is the same. Depending on the sophistication and configuration of your messaging platform, you can achieve a variety of different service levels. The higher levels involve more maintenance and are generally more expensive to configure. In this chapter, we're going to talk about two different kinds of messaging. The first is guaranteed messaging, which is implemented via middleware and provides high reliability and control. We use JMS to handle guaranteed messaging in Java. The second is nonguaranteed messaging, which lacks the reliability and security of guaranteed messaging, but can be implemented with much less programmer time and less cost for external middleware.

11.2.1.1 Guaranteed messaging

Guaranteed messaging uses middleware to ensure that a message is properly delivered to its destination, both ungarbled and in a reasonably timely fashion. The middleware implementing the messaging framework guarantees a message sent to a valid destination eventually arrives, even if the destination is temporarily unreachable.

Guaranteed delivery doesn't imply complete certainty. Specifying a nonexistent destination won't get your message delivered, regardless of how expensive your middleware is. And if a destination system is unavailable for a long period of time, the middleware will probably return the message to its sender rather than wait indefinitely. What is guaranteed, however, is the contract between the messaging client and the server governing how a message will be handled. Undeliverable messages won't vanish into the night but will be handled according to a specific and well-known set of rules.

Message Oriented Middleware systems implement guaranteed messaging. Senders and recipients work through a MOM server, which is responsible for tracking all elements of the message lifecycle. MOM servers often provide a range of options for the message content itself: XML, Java objects, raw binary data, and so on. J2EE clients interact with MOM servers using JMS, the Java Message Service. JMS, which is part of J2EE 1.3, provides a standard programmatic interface to the different MOM implementations, much as JDBC does for database servers. Sonic Software's SonicMQ and IBM's MQSeries products are two of the leading MOM products.^[2]

 For more on JMS, see Java Message Service by Richard Monson-Haefel and David Chappell (O'Reilly).

11.2.1.2 Nonguaranteed messaging

Nonguaranteed messaging is like standard postal service. Your message will probably get there, but you don't know exactly when or in what condition. You also don't know if something goes wrong: there's no notification when your letter slips out of the bag and gets wedged in a corner of the Duluth post office.

Internet email, as defined by RFC 822, is the archetypical nonguaranteed messaging system. In fact, it's very consistent: messages sent to a valid address almost never get lost, and are delivered in a timely manner. In a controlled environment, receipt rates in the five-nines (99.999%) range aren't out of the question. Of course, delivery guarantees are only part of the issue. RFC 822 messaging provides no guarantee that a message will only be delivered and processed once, or that the message won't be changed in transport (corporate firewalls and virus detection packages frequently modify the format of a message). The application is responsible for verifying integrity and making sure that the order of processing is handled correctly. There are also no security guarantees; with SMTP, for instance, it is trivially easy to create a "spoof" message that appears to be from a particular source: receipt of a message provides no assurance that the message is genuine.

In return for a little extra work, you get the basic global infrastructure for free. Nonguaranteed messaging is often the only option when dealing with widely distributed sites, getting integrations up and running quickly with external partners, or when one actor in the messaging interaction is a human.

In J2EE, email integration is handled via the JavaMail API, which, out of the box, supports sending messages via SMTP and receiving them via POP3 or IMAP.^[3] JavaMail is actually a generic messaging API, and additional plug-ins are available to support other messaging schemes, such as NNTP, the protocol behind Usenet.

11.2.1.3 Persistent and nonpersistent messages in JMS

There's another wrinkle worth mentioning before we get to the patterns. While most MOM systems support guaranteed messaging, you can use MOM in guaranteed or nonguaranteed mode. The tradeoff, as usual, is performance: the server can handle more messages in nonguaranteed mode than in guaranteed mode, but there is a chance that a message won't be delivered.

JMS allows outgoing messages to be tagged as either persistent or nonpersistent. A nonpersistent message will be delivered to recipients at most once, but possibly never. Marking a message as persistent tells the middleware to use a store-and-forward approach to message delivery, ensuring that a message is delivered once and only once. The two delivery mechanisms have different implications depending on the type of message distribution pattern you're using, so we'll touch on this subject a bit more in the next section.

11.3 Message Distribution Patterns

Message distribution is built around the concept of a message channel, which happens to be the fundamental idiom behind the JMS API. A message channel provides a named destination for messages. Senders can access the channel to send messages, and receivers can read messages off the channel. It's that simple. In the email world, a message channel consists of the combination of SMTP (for sending and transport) and a mailbox (for receipt). In JMS, a channel is represented as either a Queue or a Topic object, for point-to-point or publish-subscribe messaging, respectively. In a MOM environment, channels can be created and destroyed on the fly, or you can preconfigure them within the MOM system.

11.3.1 Point-to-Point Distribution Pattern

The simplest form of messaging is from one actor to one other actor. This situation applies when using messaging to perform remote procedure calls or to deliver document messages for processing by a particular server. In these cases, it is important that a request be sent once and only once, and that the message be received and processed once and only once. The Point-to-Point Distribution pattern defines this behavior for our applications.

In the ubiquitous purchasing system example, a purchase order should only be sent once, and should only be fulfilled once. This requirement is particularly important when messaging is used to tie together parts of a process: a message sent by an order handler requesting credit card processing should only bill the card once. There are a number of ways to enforce concurrency in distributed applications (see Chapter 10, if you haven't already), but ensuring that the issues don't arise in the first place is always an excellent start.

Figure 11-2 shows a basic point-to-point messaging system. Application 1 sends a message to the message server, which delivers it (some time later) to Application 2, which acknowledges receipt, preventing the messaging server from trying to deliver again later. After some additional processing, Application 2 sends a reply message, which is delivered back to Application 1 by the server.
 

 

 

JMS implements point-to-point messaging via the Queue interface. A client places a message on a queue, and another client later removes that message from the queue. Point-to-point distribution can be used to build more complex messaging systems, allowing us to use technology that is intrinsically point-to-point (such as SMTP email) to build more complex systems, as we'll see next.

11.3.2 Publish-Subscribe Pattern

Many enterprise activities require sending the same message, or substantially similar messages, to a large number of recipients. This can certainly be done via point-to-point messaging—as long as the application has a list of recipients for the message, it can loop through and send one to each. But this approach brings its own problems: not only must the application do the work of dispatching a separate message to each individual recipient, it must keep track of the identity of all the recipients. Every application that might send messages to multiple recipients needs a mechanism for tracking the recipient list, and coaxing multiple applications into sending messages to the same list of people becomes a coordination challenge (at best) or nightmare (at worst).

The Publish-Subscribe pattern addresses the maintenance and scalability issues of large numbers of recipients. In this scenario, a source application "publishes" a message to a channel. Recipients then register with the channel to receive messages. Recipients can be added or deleted at the message server level rather than the application level. In JMS, a publish-subscribe channel is called a topic and is implemented via the Topic interface. Figure 11-3 shows a diagram of publish-subscribe messaging.
 

In a guaranteed messaging scenario, publish-subscribe can be used to keep a large number of systems synchronized by having each system send a message announcing each change in state. We'll talk more about this with the Control Bus pattern at the end of this chapter. Publish-subscribe can also be used as part of an enterprise management approach: applications publish messages announcing their health (or lack thereof) and respond to command and control messages from a central location. (See Section 7-4 later in this chapter.)

JMS behaviors in a publish-subscribe situation can get fairly subtle. The simple case, where all of the subscribers are connected to the topic at the time a message is sent, is straightforward enough: the subscribers receive the message. But what happens when a subscriber logs on later?

JMS topics can be either persistent or ad hoc; persistent topics are configured at the server level rather than within the Java client itself. Subscriptions to topics can also be configured as either temporary or "durable." Creating a durable subscription allows the client to disconnect and reconnect later without missing the messages sent in between. Normal, nondurable subscriptions only allow a client to receive messages sent while the client is online.

Here's a simple example of connecting to a JMS topic and sending a persistent message to a topic named "greetings".

TopicConnectionFactory factory = // retrieve from JNDI

TopicConnection connection = 
        factory.createTopicConnection(  );
connection.start(  );

TopicSession session = connection.createTopicSession(
        false, Session.CLIENT_ACKNOWLEDGE);

Topic topic = session.createTopic("greetings");

if (topic == null) {
        System.err.println("Unable to load topic.");
        return;
}

TopicPublisher publisher = session.createPublisher(topic);
TextMessage message = session.createTextMessage("Hello!");
publisher.publish(topic, message, DeliveryMode.PERSISTENT, 2, 0);

publisher.close(  );
session.close(  );
connection.close(  );

You will generally use nonguaranteed messaging to implement publish-subscribe for human subscribers. Usenet newsgroups, which are enabled via the NNTP protocol, are a common example. Each newsgroup acts like a JMS topic: a message is sent to the newsgroup itself, and subscribers then connect to the newsgroup and read all of the messages that have been posted. Email-based list servers are an even simpler example: a message is sent to a single address and redistributed to a variety of other addresses.

A private NNTP news server can make a nice, inexpensive publish-subscribe mechanism for interapplication communication, at least where absolute auditability isn't required. News servers have been around for a long time, after all, which means that the underlying software has gotten quite stable.

11.3.3 Malformed Message Channel Pattern

MOM systems usually route undeliverable messages to a dead letter queue that can be monitored by the system administrator. When you're writing code that processes incoming messages, the dead letters aren't really your problem: they are undeliverable, so you never even see them (of course, if your application sends a message that turns out to be undeliverable, you should probably have some way of dealing with the consequences). Malformed messages, however, are your problem. A malformed message is deliverable but nonsensical: the application can't successfully process it. This is a bad situation, since a properly reliable application must provide some way to handle messaging related error conditions.

The Malformed Message Channel pattern provides a solution by specifying a particular destination to which an application can route all badly formatted messages (Figure 11-4). This step allows you to examine the messages manually and determine the source of the problem. Hopefully, malformed messages will be a relatively rare occurrence, so monitoring the channel will not be too onerous.
 

To implement a malformed message channel in JMS, simply add a Queue or a Topic to your messaging server, and have your application reroute all unreadable messages to the new channel. Most messaging servers provide a mechanism for browsing the contents of a channel, which might be all you need to review the malformed messages. In more complex circumstances, though, you can build an application that alerts the appropriate humans to take corrective action.

The incidence of malformed messages will probably be higher when SMTP is used for messaging. Beyond the ever-present danger of spam (which will manage to find your mailbox if it's anywhere near the Internet), firewalls, strange mail clients, and ill-behaved servers can subtly transform messages. Monitoring these problems becomes correspondingly more important.

11.4 Message Types

Within a particular messaging approach, several different types of messages can be sent. Each of these message type patterns can be built on top of the messaging systems discussed in the previous section. They can be implemented using guaranteed or nonguaranteed messaging on a variety of different protocols and with either point-to-point or publish-subscribe delivery.

All message types share a few common elements. A basic message has a sender attribute, a recipient attribute, a set of headers and a set of user-specified attributes. The use of the sender and recipient should be obvious. Headers and attributes serve to further define the message. An email message, for instance, will generally contain headers specifying the subject, the date the message was sent, the email client that was used, the encoding type used for the text, and the various servers that relayed the message.

The JMS APIs break a message up into three major parts: a standard header section, a user-defined header section, and a message body. Only the first of these sections is required, although a message without any user-specified content cannot deliver a whole lot of data. The following message type patterns help determine which data to store in the different parts of a message.

11.4.1 Event Message

Event handling in Java should be familiar to you by now; but on the off chance that it isn't, here's the primer: an object can register itself with another object as an event listener. Once the listener has been registered, the target has a handle to the listener and can use callback methods to notify the listening object that a particular event has occurred. This process allows programmers to easily modify runtime behavior by plugging new listeners in at appropriate places. Swing is full of listeners, as is the Servlet API (ServletContextListener, etc.). The Gang of Four identified this behavior as Observer, in which the actor is the object or set of objects performing the main actions, and the observer is the listener.

An event message extends the Observer model to a set of distributed applications. The basic informational content of an event message is implicit in the fact that the message has been sent. Event messages can be sent from one system to another to provide notification of lifecycle events within an application, or to announce the status of particular activities. Applications for this pattern include enterprise monitoring (see the discussion of the control bus later in this chapter) and centralized logging.

An important characteristic of event messages is that they do not require a reply. In fact, one could go so far as to say there is no requirement that event messages be received, either. In some cases this is true, and in others it isn't: whether events must be received should influence your choice of message transport.

In JMS, an event message can be implemented with the user-defined headers of a message. The benefit of this approach is simplicity: applications only need to agree on header names, so there is no need to define a format for the message data. Performance improves, too, since JMS and most MOM packages provide built-in functionality to filter messages based on their headers.

11.4.2 Document Message

Many applications need to share large data files with each other. A Document Message allows them to accomplish this via messaging. A document message is a message that contains a substantial payload. Depending on the systems involved, the document might be an XML file, a serialized object, or an actual word processing document.

Designing a document message raises many of the same interoperability questions as any other integration activity. XML-formatted messages can be consumed by a variety of systems, but require more computing power to process than simply reading raw bytes out of a message. Serialized Java objects can only be consumed by other Java systems with the appropriate class libraries installed, but are otherwise very easy to use.

Document messages are generally implemented in JMS by storing the document in the message body. The message headers should be used to store simple information about the document, such as its format and versioning information, while the actual data is put into the body. With Internet email, document messages can be implemented as a file attachment or a series of file attachments. Message headers can still be used to indicate the type of message being sent.

When using Internet email to transmit data, it makes sense to apply some basic security measures. At the minimum, you should include a digest of the data being sent. This record allows you to verify that the message wasn't accidentally tampered with in transit, but will not provide much (if any) security against deliberate sabotage. Encrypting the digest with a public key system allows you to ensure that a message hasn't been tampered with. Encrypting the whole document, of course, prevents it from being tampered with or viewed by unauthorized personnel.^[4]

Since that's an extremely brief introduction to digital signatures, the user is encouraged to consult Web Security, Privacy & Commerce by Simson Garfinkel with Gene Spafford (O'Reilly) and Java Cryptography by Jonathan Knudsen (O'Reilly).

11.4.3 Command Message

Distributed applications can be a command and control nightmare. Multiple web applications expose different interfaces to the administrator, and actions that affect multiple systems need to be repeated. Tools like JMX (the Java Management Extensions) alleviate this problem to a degree, but only within controlled environments. It can be irritating, but the real challenge comes when disparate applications need to control each other. Since user interfaces are generally not designed to be used by other computers, some kind of remote command infrastructure is required.

A command message allows one system to control another application, or a series of other applications, by sending a specially formatted message to that system. A command message includes instructions to perform a specific action, either via headers and attributes, or as part of the message payload. The recipient performs the appropriate action when the message is received. Command messages are closely related to the Command pattern from GoF. They can be thought of, in fact, as a remote version of this pattern.

Security is a particular concern with command messages. MOM products provide a variety of mechanisms for securing a message channel. If these products are adequate for your needs (they often are), receipt of a command message can be treated much like an RPC call. If you're using a transport system that doesn't provide security guarantees, such as email, consider the security options we discussed for document messages.

A document message can also be a command message. Many commands require data on which to act; there's no point in sending two messages, what with all the overhead involved in correlation—particularly since neither message will be useful without the other.

11.5 Correlating Messages

Asynchronous messaging means that the sender doesn't receive an immediate reply or confirmation of receipt. After sending a message for processing by a remote service, most applications require at least confirmation that the message was received and acted upon, and often require the results of the remote process. Remember—in a normal application, even if a method doesn't return a value, the system at least knows that the method has had a chance to execute.

A reply message is a message received in response to another message. Reply messages are generally document or event messages, but are distinguished by being associated with a particular message. Typically, this association is accomplished by assigning each outgoing message a unique identifier (MOM systems do this automatically). If an incoming message is a reply, it includes a reference to the message the remote application is replying to.

JMS includes built-in functionality for associating a reply message with the original. Each JMS message, upon creation, is assigned a message ID, which is accessible via the JMSMessageID property of the Message class. Messages also have a JMSCorrelationID field, not set by default. When composing a reply message, it's standard practice to take the JMSMessageID and store it in the JMSCorrelationID field. Assuming that message and reply are valid JMS messages, and that reply is a response to message, this is how we'd set the correlation:

reply.setJMSCorrelationID(message.getJMSMessageID(  ));

This approach presupposes that the original sender kept track of the message ID. Obviously, reply messages can be linked to their original in whatever manner is appropriate for your application. An identifier with real world significance (purchase orders or database record keys) might make more sense than the ID assigned by the JMS provider.

11.5.1 Sequenced Message Pattern

Message transport systems have limits. These limits frequently have to do with size. Even if the messaging system you're using doesn't impose a strict upward boundary on the size of a message, practical considerations often impose one anyway. Network connections can be interrupted and servers can run out of memory.

As a result, it is sometimes necessary to divide a message into several parts, each of which can be easily digested by the messaging system and the recipient. The Sequenced Messages pattern allows you to spread one logical message over a series of actual messages (Figure 11-5). The client must wait until it has received all of the messages in the sequence before it can proceed.
 

To send sequenced messages in JMS, you need to include at least two additional properties in each message, and usually three. The first is a sequence ID, which allows the recipient to group messages from particular sequences. The second property is the order of the current message in the sequence. For example, your application might assign a particular message sequence the ID "MSEQ-001", and each of the five messages in the sequence would include a message sequence ID property containing "MSEQ-001" and a message number from 1 to 5. If your application isn't able to determine the length of a message sequence by examining the messages itself, a third property, providing the number of messages in the sequence, is also required.^[5]

 If the end of the sequence is self-evident from examining the last message, you don't need to specify the number of segments up front; the recipient can look at the message number of the final message in order to determine how many messages were supposed to have been received. This approach can be particularly helpful when it's unclear at the start how large a sequence might end up being.

Here's a basic example, using two properties. For simplicity, we assume the content is presented as a set of text strings (perhaps an XML file broken up into pieces) created by a getContent( ) method:

        String sSeqID = "MSEQ-001";
        int msgIndex = 1;
    String[] contentPieces = getContent(...)

    for (int index = 0; index < contentPieces; index++) {
       TextMessage message = 
          session.createTextMessage(contentPieces[i]);
        message.setStringProperty("MessageSequenceID", sSeqID);
        message.setIntProperty("MessageNumber", index);
        publisher.publish(topic, message, delivery_mode, priority, 0);
     }

Since none of the messaging systems we've discussed support sequenced messages natively, error handling in the case of an incomplete sequence is very much in the hands of the client application.

11.6 Message Client Patterns

Once a message has been created and dispatched to an appropriate delivery channel, the next step is for the client to retrieve and process it. Retrieval mechanisms for messages need to balance speed of delivery, reliability, scalability, and ease of use. In this section, we'll look at some patterns for handling messages in client applications. The Message Handle pattern shows how to decouple message processing code from message retrieval code. The Polling Consumer and Event-Driven Consumer patterns explain how to get messages into your application. Message Façade demonstrates using messaging to isolate your business logic. Competing Consumers shows how to speed up message processing in high volume applications, and Message Selector will help you build more efficient client-side message processors.

11.6.1 Message Handler Pattern

All message clients perform two functions: they retrieve and process a message. It's possible to weave these two activities tightly together, but in more complex systems it can result in some pretty complex code. We want to isolate the latter function in a way that lets us share code across different message types. The Message Handler pattern abstracts the code responsible for receiving the actual message away from the code responsible for processing the message content.

This pattern allows us to change the method by which we receive a message without changing the code for processing the message. Changing message handling itself becomes easier, since we can drop message handlers in and out without major changes to the rest of the application (we can even do it dynamically). There's also a testing benefit, since you can write standalone test cases for an isolated handler method more easily than for an integrated component.

Example 11-1 shows a simple mail handler interface for incoming JavaMail messages. The code responsible for retrieving the message instantiates the appropriate handler and calls the handleMessage( ) method, passing in the new message. We'll see the message client code itself when we talk about the Polling Consumer pattern.

Example 11-1. Mail handler interface

public interface BlockingMailHandler 
{
  /** Process a message, returning true on success, false on failure.
        * Does not return until message is processed. */ 
  public boolean handleMessage(javax.mail.Message message);
}

A more complex application than this simple blocking handler might define a variety of message handler types, such as a threaded handler and a single thread model handler. A JMS example would look similar, but it would accept a javax.jms.Message object instead.

Most applications can get away with a transport-specific handler, since the mechanics of retrieving content from JMS and JavaMail message objects differ widely; most applications only use one transport mechanism for any particular use case. But occasionally, applications need to accept the same kind of message from a variety of different sources. One example of this situation is when some of the applications you're integrated with are separated by a firewall: remote applications can send their messages via email (probably encrypting the content) and local applications can use the MOM backbone. In these cases, you'll either want to provide two handleMessage( ) methods, or descend two different handler classes (one per message type) from the same base class (which contains the logic required for processing the message once it has been extracted from its transport container).

11.6.2 Polling Consumer Pattern

An important reason to use a messaging system is in order to allow one component of an integrated system to go down without immediately affecting all the others. The Polling Consumer pattern allows a client to periodically check for messages, rather than maintaining a constant connection to the message server. If the server is unavailable, the client can simply try again later, rather than ceasing all operation. Conversely, if the client goes down, the server will maintain the messages until the client starts up again. In this kind of client crash we still lose time, but we don't lose information.

A polling consumer periodically checks a message channel for new messages, reads them, and processes them. Generally, the consumer is implemented as a thread, which sleeps for a set amount of time before polling for new messages. Polling consumers are the most reliable way to receive Internet mail and are useful for JMS applications where an EJB container isn't used.

Example 11-2 shows the MailMonitor component. MailMonitor is a Java class that can be dropped into a standalone application (by instantiating and running the thread in the main( ) method) or into a web application (started, perhaps, by a ServletContextListener). The mail monitor checks a POP3 mailbox, reads every message, and invokes a BlockingMailHandler to process them. If a message is processed successfully, the client deletes it from the server. Once all messages are processed, the client waits 10 seconds and starts again.

Example 11-2. Standalone MailMonitor

import javax.mail.*;
import javax.mail.internet.*;
import java.io.*;

public class MailMonitor extends Thread {
  boolean interupted = false;
  BlockingMailHandler handler = null;
  private static String username = "contentmail";
  private static String password = "poppasswd";
  private static String mailServer = "mail.company.com";
  
  public MailMonitor(BlockingMailHandler mh) {
    handler = mh;
  }

  public void stopRunning(  ) {
    interupted = true;
  }
  
  public void run(  ) {
    Session session = Session.getDefaultInstance(System.getProperties(  ));
    Store store = null;
    try         {
      store = session.getStore("pop3");
    } catch (NoSuchProviderException nspe) {
      nspe.printStackTrace(  );
      return;
    }

    while(!interupted) {
      System.out.println("Looping to collect mail");
      try {
        if (!store.isConnected(  )) // should always be true
          store.connect(mailServer, -1, username, password);
        System.out.println("Connected");  
        Folder folder = store.getDefaultFolder(  );
        folder = folder.getFolder("INBOX");
        if (folder == null) {
          System.out.println("Unable to open INBOX");
          return;
        }

      System.out.println("Opening folders");
      // Try to open read/write. Open read-only if that fails.
      try {
        folder.open(Folder.READ_WRITE);
      } catch (MessagingException ex) {
        folder.open(Folder.READ_ONLY);
      }

      int totalMessages = folder.getMessageCount(  );
      int newMessages = folder.getNewMessageCount(  );
      System.out.println("Total Messages: " + totalMessages);
      try {
        Message messages[] = null;
        messages = folder.getMessages(1, totalMessages);

        for (int i = 0, i < messages.length; i++) {
          boolean mbDelete = handler.handleMessage(messages[i]);                                                                        
          // Delete the message
          if (mbDelete) {
            messages[i].setFlag(Flags.Flag.DELETED, true);
          }
        } // end for
      } catch (MessagingException e) {
        System.out.println("Unable to get Messages");
      }

      // Close the folder and store
      folder.close(true);
      store.close(  );

      } catch (MessagingException e) {
        System.out.println(e.toString(  ));
        return;
      } 
      
      try {
        this.sleep(10000);
      } catch (InterruptedException e) {
        System.out.println("Exiting");
        return;
      }
    }
  }
}

It's worth noting that in this design the mail handler is responsible for processing a message regardless of its validity. The mail handler implementation is also responsible for error reporting. The only time we want the handler to return false is when a time delay would allow the message to be processed later. Messages that cannot be processed now might be processable later, once other systems come back online or other messages arrive. That said, messaging systems make lousy persistent storage: you really should get the message out as quickly as possible, and deal with queuing in some other context, particularly if you don't have personal control over the mail server in question.

Polling consumers is also useful when an application doesn't need to process a message immediately, or when other factors such as high connection costs create economies of scale for processing multiple messages at once.

11.6.3 Event-Driven Consumer Pattern

Polling for messages can be a lot of work: your application is responsible for waking up periodically, checking for messages, and handling them in a safe fashion. Most of us have been spoiled by event-driven models in various areas of Java development. Servlets, for instance, are event-driven: the service( ) method is called in response to a specific request. The Event-Driven Consumer pattern allows us to forget about most of that code we built up in the Polling Consumer pattern section, in favor of a simple method that is invoked by the environment when necessary. The result is decreased code complexity and increased reuse, since more of the messaging retrieval logic is moved to the server.

Most JMS messages are handed in an event-driven manner. EJB 2.0 supports message-driven beans, which include an onMessage( ) method that is invoked whenever the application server detects a new message on a topic or queue. The base JMS API supports event-driven messages through the MessageListener interface. MessageListener defines an onMessage( ) method, which is called by the JMS subsystem whenever a new message is received.

Here's just about the simplest MessageListener class possible:

class MyMessageHandler implements MessageListener {

        public void onMessage(Message message) {
          TextMessage msg = (TextMessage)message;
          try { 
            System.out.println("Message Received:" + msg.getText(  )); 
          } catch (JMSException e) {
            System.out.println(e.getMessage(  ));
          }
        }
}

And here's how we'd associate it with a Topic (assuming we already have a session named session and a topic named topic):

   TopicSubscriber subscriber = session.createSubscriber(topic);
   MyMessageHandler handler = new MyMessageHandler(  );
   subscriber.setMessageListener(handler);

Each new message on the topic now triggers a call to onMessage( ). This process continues until message delivery on the connection is halted, the session is closed, or the program exits.

The underlying implementation depends on the messaging provider and driver. The driver might very well end up implementing a polling consumer behind the scenes, resulting in some overhead you might not ordinarily expect. Some providers deliver messages over RMI; the driver creates an RMI endpoint, which then delegates to the MessageListener. Check your messaging server documentation for more details.

JavaMail also provides an event-driven consumer via the classes in javax.mail.event, which support listening for changes to folders and message stores. However, we don't advocate the use of these classes: the implementations of the various Internet mail protocols don't support this kind of activity all that well, which means you'll have to test your application even more extensively than usual, and with the same exact infrastructure that you'll be using in production. It's better, in our opinion, to implement a polling consumer: it might be a bit more work, but you'll have tighter control and better visibility when things go wrong.

11.6.4 Message Façade Pattern

The Message Façade pattern describes an object that receives and acts on messages from the presentation tier. The message façade receives asynchronous messages from an external source, translates those messages into calls to appropriate business logic methods, executes the calls, and, if appropriate, sends a reply message containing the results. The rationale for isolating business logic behind the message façade is the same as for isolating it behind a business delegate or a session façade: easier maintenance and improved reuse.

Generally, the tradeoff to consider when developing business logic interfaces is whether you want to expose particular functionality in a synchronous or asynchronous manner. Most high-volume applications perform a surprisingly large amount of processing asynchronously; in an application deployed to a large number of users who may or may not have any particular loyalty to your company or institution, any activity that will take more than a second is a good candidate for asynchronous processing.

Message façades, like façades in general, have advantages and drawbacks. Since an application often takes several paths to its business logic (servlets, JMS, web services, and other routes as yet unknown), isolating that business logic behind a façade provides encapsulation and supports reuse. The business logic and domain model components themselves don't need to be aware of the message infrastructure and can be maintained separately. As with any messaging pattern, the business logic can be used asynchronously by the presentation tier. The cost is additional code, more infrastructure, and an additional layer of software between the user and the actual business process.

The Message Façade pattern is the third (and final) pattern in our series of business logic distribution patterns. We used business delegates to provide access to business logic within a JVM, via direct method invocations. We used session façades to provide synchronous access to remote business resources. Now, we'll use message facades to provide asynchronous access to remote resources.

Of course, our simple summary could be a little misleading. There's no reason that different kinds of façades can't, in the course of fulfilling their responsibilities, make calls to each of the others. A business delegate might call out to a remote session façade and fire off a batch of messages to a message façade. In general, creating a business delegate in order to allocate responsibility to the remote façade is a good idea, regardless of the façade type.

Implementing a message façade is easy. All you really have to do is implement a consumer. In EJB 2.0, you can create a message-driven bean that includes a JMS onMessage( ) method. Within the onMessage( ) method, you can call your business objects and assemble a reply message. Using an MDB rather than a regular JMS listener gives you transactional awareness: the message-driven bean will participate in any currently running transaction; if a database action fails, the message receipt can be rolled back as well.

You probably won't need to implement message façades with JavaMail all that often, but the process is the same: implement a polling consumer (using the MailMonitor.java framework or any other mechanism you wish) and add the business logic functions. When using JavaMail, you'll have to take more care with security, since you won't be able to count on the message server to verify that only authorized parties are sending messages to your application.

11.6.5 Message Selector Pattern

The profusion of message queues and topics is a frequent problem in JMS applications. Designers often establish separate queues or topics for each type of message in the application. Supporting a high number of consumers can eat up a lot of resources, and managing a large number of separate messaging channels can burden IS departments, particularly when each new message type requires the reconfiguration of the message server. Email messaging presents a similar problem, but in the form of multiple target email addresses.

The Message Selector pattern allows us to handle multiple message types on a single queue or topic without excessively complex logic because it requires the client to differentiate between the different types of messages on the same channel.

JMS has a fairly powerful selection criteria API, based on a subset of the SQL selection syntax. The selection criteria works against the properties set on a particular message. A message selector that only retrieves messages in which the MessageType property equals the string "StockCheck" would look like "MessageType = 'StockCheck'".

With the JMS publish-subscribe API, we specify the selector when we create the subscriber—it can't be changed after the fact. The boolean parameter at the end of the createSubscriber( ) call indicates whether messages originating from the local session should be ignored:

   TopicSubscriber subscriber = session.createSubscriber(topic, 
            "MessageType = 'StockCheck'", true);

For point-to-point messaging, we specify the selector in the call to the createConsumer( ) method, which has the same syntax—except that there is no boolean for ignoring local messages, since that doesn't make sense on a queue.

Using a selector has different consequences depending on the type of messaging used. If a message selector is applied to a queue, the remaining messages stay on the queue and can be read by the same session later on, or by other sessions (see the Competing Consumers pattern, below). When a message selector is used with a topic, the messages that aren't selected are lost to that subscriber, regardless of the durability of the subscription (the alternative would be completely unworkable: thousands upon thousands of undelivered messages, clogging up the middleware for subscriptions that have no intention of ever retrieving them).

11.6.6 Competing Consumers Pattern

Connecting front-line systems with back-office systems via messaging allows the front-line systems to keep on interacting with users, even while performing complex activities. This ability to multitask pushes the delay away from the end user, but it doesn't actually eliminate any bottlenecks. The work is spread out, but messages must still be processed.

Thankfully, there's no rule that says everything in life has to be linear. You can set multiple consumers loose on a single message queue and have them compete for the right to process the messages. Hence, the Competing Consumers pattern. Each consumer reads messages off the queue as quickly as it can and processes them in whatever manner is appropriate to the application. If the consumers start to fall behind we can just add more.

Implementing multiple handlers without a MOM system is a bit more complicated. Since multiple applications cannot reliably access a single Internet email address concurrently, you need to divide the messages between handlers at a central point. Assigning an email address to each of the handlers can accomplish this division. Incoming messages can then be spread across the handlers, either by having one handler that receives all the messages and resends them (which can be much faster than the actual message processing, allowing one client to take care of it), or by configuring the mail server, where possible, to distribute messages in a round-robin fashion. The various handlers can implement polling consumers pointed at the individual addresses.

11.7 Messaging and Integration

For the final section of this chapter, we're going to look at patterns that focus on integrating messaging into an enterprise environment. Of course, all of the patterns in this chapter affect integration to some degree. The Message Façade pattern, for instance, is sometimes known as a messaging adapter, allowing nonmessaging-aware applications to participate in message-based data exchanges. The patterns in this section focus on how to route messages between systems, and on what might be done with the content once they've arrived.

11.7.1 Pipes and Filters Pattern

Many applications involve multiple processing. For example, placing an order might require reserving stock at the warehouse, processing payment, and shipping. In a fully integrated enterprise, you might be able to do all of this in one system and at one time, but most companies aren't entirely integrated. Reserving stock might require someone to physically visit the warehouse, and payment processing might involve purchase order approvals or lengthy credit checks. In these cases, it makes sense to decouple the activities into separate components and connect them with messaging.

The Pipes and Filters pattern allows us to link a set of message handlers (filters, in this parlance) via a series of pipes. The pattern itself is an old one,^[6] and has been applied in a variety of contexts. Anyone who has "piped" the content of one program to another (cat textfile.txt | more, for example) should be familiar with the general concept. After each processing stage completes, the message is sent along another channel to the next handler in the sequence (Figure 11-8).