State of the art clustering with JBoss Messaging
Last week the messaging team released JBoss Messaging 1.2 Alpha 1.
We're really excited about this. JBoss Messaging 1.0 has been around since March, but this is the first release to include clustering functionality
Like any alpha it's experimental and it's very much a work in progress but the idea is to give you the community, a taste of what's in store for the 1.2 GA (General Availability) release due in February next year.
We're confident we have an absolutely state of the art clustering implementation.
Actually I know of only one messaging system that has a comparable feature set, and it's proprietary. As far as I know (and I am happy to be corrected) there is nothing in the open source space that touches us when it comes to clustering.
Just to whet your appetite here are some of the clustering features that you're going to get in 1.2: (not all are available in Alpha 1)
-
Seamless client failover.
If you're in the middle of doing some JMS stuff and the server fails, it will automatically fail over onto another server where you carry on where you left off. You don't have to catch any exceptions and code the failover yourself.
-
No single point of failure (SPOF) or single point of bottleneck (SPOB).
There are no "master queues" on particular nodes. No "HA Singletons". You can even remove the shared database if you want (see later).
-
Fully distributed topics.
You deploy a JMS topic on the cluster. You can create subscriptions on any node. When you send messages on any node they will reach the subscriptions on all the nodes (assuming the message selectors match).
We leverage the tried and test power of JGroups for all our network group management and transmission requirements.
-
Fully distributed queues.
You deploy a JMS queue on the cluster. Again, you can create consumers on any node and send messages on any node. Each node maintains it's own "partial queue", so there is no singleton "master queue" that all the nodes have to access.
-
Shared durable subscriptions.
Some clustering implementations "pin" durable subscriptions to a particular node (naming no names). We don't. If you create a durable subscription, you can consume from it from different nodes. And this doesn't mean accessing a shared "master subscripton" either. Like queues, we create partial subscriptions distributed across the nodes.
-
Pluggable routing policy.
If you send a message to a queue while attached to a particular node, should the message be routed to the partial queue on that node or on another node? This is is determined by the routing policy. By default it will always choose the local partial queue. But if the local queue has no consumers then it will choose another queue. This is fully pluggable and replaceable.
-
Intelligent message redistribution.
In my view this is one of the coolest features.
You've routed messages to partial queues, but consumers get closed or slow down on a particular node causing messages to build up or get stranded. On another node the consumers are processing messages faster than they are arriving, wasting CPU cycles as they wait for the next message to arrive. JBoss Messaging clustering allows nodes with fast consumers to pull messages from slower nodes in order to balance the load. Again, the policy as to how and when this happens is fully pluggable.
-
In memory persistent message replication.
This is going to be another very cool feature.
The idea here is as follows: Most messaging systems have the concept of "once and only once delivery". This is normally achieved by persisting messages in "permanent" storage (e.g. a database or file store). The problem is that permanent storage usually means writing to disk and disk synching, which is slow.
The problem gets further compounded when the storage is shared - e.g. a shared Oracle database, since not only do you have to disc synch, but you also have a point of contention, a bottleneck in the system which hampers the ability of the system to scale.
Ok, solutions such as Oracle RAC go some way to address these problems but scaling databases horizontally is not an easy (or cheap) problem to solve. (JBoss Messaging does of course support shared databases too if you don't want to use in memory message replication)
But there's another way we can increase the "durability" of a message without storing it to disk at all, and that's by replicating it in memory between different nodes. If the cluster is well constructed with good hardware, UPS etc then we can reach sufficiently levels of durability for many applications.
If we consider that casting data across a fast network should be faster than writing and synching it to "permanent" storage nor do we have a single point of bottleneck since we don't have a shared database, then this should allow us to construct a highly scalable, highly performant messaging system, and still provide levels of reliability normally only achievable using shared databases or local file based storage.
So, what does this all boil down to?
We're enabling a highly scalable grid based processing model where message processing load is smoothly and automatically distributed across the cluster, with no single point of failure or single point of bottleneck whilst maintaining the "once and only once" delivery promise.
(To any technical people reading this I am truly sorry for the amount of buzzwords in the preceeding paragraph)
That's the vision anyway, and we're confident we're going to be able to fulfill it.
But JBoss Messaging 1.2 is not just about clustering, there will be some other great new features too, including:
-
Fully recoverable XA resource implementation.
This is a must for some of our enterprise customers. When used with a good transaction manager (like JBoss Transactions) we can really put the D back in ACID. ...
-
High performance SEDA style processing model.
I won't go into the details of what SEDA means here (google is your friend), but this combined with an asynchonous, non blocking approach to our client/server communication, should enable us to provide down to the wire performance and also enable us to cope with very large numbers of connections. Again, I think these features are essential for any messaging system with aspirations on the "big league"
Ok, I'll step off my soap box now and move to other matters.
I also wanted to mention that we're really pleased that Clebert Suconic, the brains behind JBoss Serialization, has decided to join the messaging team. He should be a great and sorely needed addition to the team. :)
However, we have much to do, and the team comprises only three people including Ovidiu the project lead, so we are still actively looking for more talented and motivated developers to help out on the project with a view to a permanent role in the team.
This could be your chance to say you have worked on the number one enterprise messaging product out there, so please step forward!
Posted on Fri, 6 Oct 2006 07:35 by Tim Fox ( day(s) old) Trackbacks [0]
