Blog/manik/

Article on JBoss Cache 1.4.0 on InfoQ

[ Manik Surtani ] [Permalink]

Floyd Marinescu has an article on the latest JBoss Cache 1.4.0 "Jalapeno" here, on InfoQ, talking about some of the new features and performance enhancements.

Be sure to check it out - a nice summary that sits in between my original blog announcement and the detailed designs on buddy replication and some of the other enhancements in this release.

--
Manik Surtani
Lead, JBoss Cache

Posted on Mon, 17 Jul 2006 23:10 by Manik Surtani ( day(s) old) Trackbacks [0]

JBoss Cache 1.4.0.GA and 1.3.0.SP3 released

[ Manik Surtani ] [Permalink]

I'm proud to announce that JBoss Cache 1.4.0 "Jalapeno" has just been released.

I'd like to thank all those who contributed to this very important release, the community which is the driving force behind this project, and everybody who've been very patient in waiting for this release despite unavoidable delays.

This release marks huge performance improvements in replication throughput, as well as overall PojoCache performance. In addition, big new features such as Buddy Replication and Data Gravitation are now in production, which open up interesting possibilities for the scalability of your clustered application.

Here are some previous articles and posts regarding JBoss Cache 1.4.0 "Jalapeno":

Download JBoss Cache 1.4.0 "Jalapeno" here. Read about JBoss Cache 1.4.0 "Jalapeno" here. Talk about JBoss Cache 1.4.0 "Jalapeno" here.

Also, another service pack on the JBoss Cache 1.3.0 "Wasabi" branch has just been released - 1.3.0.SP3.

This contains critical bug fixes and performance improvements backported from JBoss Cache 1.4.0 "Jalapeno".

Download the release on the JBoss Cache download page, or see what's new in SP3.

--
Manik Surtani
Lead, JBoss Cache

Posted on Fri, 14 Jul 2006 05:55 by Manik Surtani ( day(s) old) Trackbacks [0]

Jalapeno beta released

[ Manik Surtani ] [Permalink]

JBoss Cache 1.4.0 "Jalapeno" beta has been released.

This is the first beta of the 1.4.0 series, which includes some big improvements over previous versions of JBoss Cache, including buddy replication, and huge performance improvements in replication, increase in throughput and overall PojoCache (previously known as TreeCacheAop) performance thanks to specialised marshalling and use of JBoss Serialization.

The beta is available for download here. Please do give this a try and provide feedback on the JBoss Cache user forums.

--
Manik Surtani
Lead, JBoss Cache

Posted on Sat, 13 May 2006 09:59 by admin ( day(s) old) Trackbacks [0]

JBoss Cache on Steroids

[ Manik Surtani ] [Permalink]

We've been yelling about this from the rooftops and I'm sure everyone's as excited about this as I am - one the key features of JBoss Cache 1.4.0 "Jalapeno" is Buddy Replication and Data Gravitation. Exciting technologies that will help the JBoss clustering stack scale much, much better for large clusters.

But there are couple interesting features about Jalapeno, which should be allowed to share the limelight with Buddy Replication.

Replication performance

First of all, there are big changes in the way we marshal method calls for RPC and data replication. This has resulted in very significant performance boosts in replication, regardless of the size of your cluster. Clustering efficiency has just been turbocharged. :-)

Preliminary tests have come back with the following results:

  • 20 threads, 1000 loops, replicating 10k of data.
  • Running in synchronous replication mode with no transactions.
  • 3 server cluster.

Results using the original marshalling mechanism 

Total time spent (ms): 127599 for 1000 loops
Throughtput: 156 requests/sec
Results using the new, optimised marshalling mechanism 
Total time spent (ms): 97196 for 1000 loops
Throughtput: 205 requests/sec
Over 30% increase in throughput and almost 25% quicker in overall timings.

So what about CPU usage? We then attached a profiler to these tests to analyse how much time is spent in the replication code. The key thing to look for is time spent in RpcDispatcher.handle().

Again, both in absolute time (26 secs vs. 20 secs) and relative time (26.5% vs. 20.8%) we see over 20% reduction in CPU time, despite handling 30% greater throughput.

PojoCache performance

Next, on PojoCache (previously called JBossCacheAop) headed by Ben Wang, we also see significant performance enhancements by performing bulk cache updates and removes during pojo attachment and detachment. Pojo-level operations have a speed improvement of a factor of 2, significant stuff. In addition, there is support for pojo field annotations in JDK5.0 such as @Transient and @Serializable.

Expect big things from Jalapeno. With Buddy Replication and such replication performance boosts, this certainly is going to be a very exciting release.

--
Manik Surtani
Lead, JBoss Cache

Posted on Fri, 21 Apr 2006 07:21 by admin ( day(s) old) Trackbacks [0]

JBoss Cache 1.3.0.GA "Wasabi" released

[ Manik Surtani ] [Permalink]

Much anticipated, JBoss Cache 1.3.0.GA, codenamed "Wasabi" has finally been released. And this is a monster of a release - lots of long awaited features including optimistic locking, invalidation, the ability to configure more than one cache loader, a new programmatic Option API to allow for configuration overrides on a per invocation basis, a few new remote cache loaders that allow you to use the a dedicated remote host with a lot of physical RAM as a cache loader, and plenty of overall performance enhancements, particularly around cache loaders.

I'd like to thank all contributors on the JBoss Cache project for making this possible - JBoss employees, partners, external contributors, everyone. There has been a lot of participation in this release in particular. Everything from features to fixes, performance enhancements to helping track down obscure concurrency issues, open discussion of designs and ideas to actually implementing some of them, the community has been a huge part of this release. This is what open source and the JBoss user community is all about. Thank you all again, you know who you are.

So what next then? JBoss Cache 1.4.0 "Jalapeno" is the next major release, with an aggressive release schedule, due in a very short few weeks. This is a very focused release, on a few specific features (namely buddy replication and some dramatic performance enhancements in replication we have in mind). We expect Jalapeno to blow your mind as well, expect early alphas of Jalapeno in the near future.

  • Download JBoss Cache 1.3.0.GA "Wasabi" here.
  • Discuss JBoss Cache 1.3.0.GA "Wasabi" here.
  • Read JBoss Cache 1.3.0.GA "Wasabi" documentation here.

Enjoy your wasabi,
--
Manik Surtani
Lead, JBoss Cache

Posted on Mon, 3 Apr 2006 20:04 by admin ( day(s) old) Trackbacks [0]

JBoss Cache 1.3.0.CR1 released

[ Manik Surtani ] [Permalink]

JBoss Cache 1.3.0.CR1 has just been released. I'd like to urge you all to download and try out this latest and greatest release, adding a number of interoperability and backward compatibility fixes over the previous betas as well as some significant performance improvements with regards to cache loaders - in addition to solving an obscure deadlock in the cache loader handling code when simultaneously creating a new node.

Feedback, as usual, on the user forums please.

--
Manik Surtani
Lead, JBoss Cache

Posted on Wed, 22 Mar 2006 05:51 by admin ( day(s) old) Trackbacks [0]

Buddy Replication and Data Gravitation

[ Manik Surtani ] [Permalink]

Buddy Replication - the clustering concept where state is replicated in-memory to a specific other server in a cluster rather than the entire cluster of servers - has always been seen as important for any application server to have in its clustering arsenal. High scalability while maintaining transparent failover are the usual benefits, but the biggest drawback has always been complexity in load balancers.

When a server crashes, it is the load balancer's job to locate the failed server's buddy, and redirect requests meant for the original server, to its buddy. Several solutions have been employed by the industry, and have so far included techniques such as encoding the address of a buddy in each request and maintaining a static lookup table of servers and their buddies. These approaches have been considered unnecessarily complex or inflexible, but always seen as a necessary evil to enable buddy replication.

JBoss Cache 1.4.0 "Jalapeno", due for release in May 2006, will have an implementation of buddy replication which uses a concept of Data Gravitation to deal with the problem of recovery after failure. JBoss Cache is the technology behind JBoss Application Server's HTTP and EJB session replication.

With Data Gravitation, after the failure of a server in a cluster, the load balancer does not care where the buddy is, but simply uses its load balancing algorithm to direct the request to the cluster as though it were a brand new request. Consider the following setup:

Lets assume each node serves HTTP requests and needs to maintain HTTP session data. Node A's session data is backed up on Node B, Node B's session data is backed up on Node C, etc. Now lets say Node A crashes.

Node B takes over Node A's data and treats it as its own. And now when requests come in for sessions originally on Node A, the load balancer picks a node to redirect the request to - lets say this is Node D. Node D, realising that it does not own the session requested, attempts to find the new data owner - in this case Node B - and transfers the session data from Node B to Node D. Node D now becomes the new owner of this session and Node B removes it from its in-memory state. Node D, now the new owner of this session, will be able to deal with future requests involving this state. Data is also always backed up - i.e., when Node B takes ownership of failed Node A's sessions, it will start to back it up (onto Node C) along with its own sessions. Once Node D takes ownership of this session, again it will be treated the same as the rest of Node D's sessions and get backed up (onto Node E).

It must be noted that this gravitation would only occur if and when needed (i.e., lazy gravitation), and even then, will only occur once per session since the requesting node would take ownership of this session and its data.

This peer based mechanism allows load balancers to work with no extra configuration or setting up, as data will always gravitate to the server that needs it. The other benefit is that when a server crashes, its data and load can get evenly distributed across the remaining servers as the load balancer directs requests based on its configured load balancing policy for new requests. In the other setups mentioned above, the buddy server would suddenly have its load doubled when a server failure occurs.

The only requirement placed on such a setup is that sticky sessions are used, to prevent data from changing owners repeatedly if requests attached to a single session are directed to different servers in the cluster each time. Given that sticky sessions are very commonly used as an architectural pattern anyway and most application servers tend to use sticky sessions by default, this is a relatively painless requirement.

The designs of Buddy Replication in JBoss Cache 1.4.0 "Jalapeno" are available on this wiki page, and can be discussed on this forum thread.

--
Manik Surtani
Lead, JBoss Cache

Posted on Mon, 13 Mar 2006 18:26 by admin ( day(s) old) Trackbacks [0]

JBoss Cache 1.3.0 "Wasabi" Beta1 available

[ Manik Surtani ] [Permalink]

JBoss Cache 1.3.0 "Wasabi" is now in beta and available for download on SourceForge.

Please see my previous blog entry about the main features of 1.3.0 "Wasabi" - most of them have made it into the release with the exception of Buddy Replication which has been deferred to JBoss Cache 1.4.0 "Jalapeno".

Also, please refer to the JBoss Cache Roadmap for more information on future releases.

Enjoy your wasabi!

--
Manik Surtani
Lead, JBoss Cache

Posted on Fri, 3 Mar 2006 04:15 by admin ( day(s) old) Trackbacks [0]

JBoss Cache Buddy Replication initial designs up for public review

[ Manik Surtani ] [Permalink]

Much anticipated by the user community, Buddy Replication is a feature that allows JBoss Cache instances to replicate to nominated buddies in a cluster rather than the entire cluster. Benefits include a high degree of scalability, with savings in both memory usage as well as network traffic, both of which would remain static even when nodes are added to a cluster. This differs from total replication where both memory usage and network traffic can increase geometrically with each node added to a cluster.

This feature affects all aspects of JBoss clustering, including HTTP session replication in Tomcat.

Buddy Replication is the main feature of JBoss Cache 1.4.0 "Jalapeno", due for release by mid-May 2006. Visit the JBoss Cache roadmap for more information on future releases, or the Buddy Replication design page or design discussion forum to comment on this.

--
Manik Surtani
Lead, JBoss Cache

Posted on Wed, 1 Mar 2006 16:25 by admin ( day(s) old) Trackbacks [0]

Optimistic Locking Performance in JBoss Cache 1.3.0 "Wasabi"

[ Manik Surtani ] [Permalink]

After hours with a profiler, a bunch of performance tests and several doppio espressos, the JBossCache optimimistic locking implementation slated for release in the upcoming JBossCache 1.3.0 "Wasabi" is finally looking performant enough for widespread use. While I expect to optimise things even further in this regard, the 3 largest bottlenecks have been taken care of and the performance gains are significant. This is a huge leap ahead, particularly when compared to the performance of the optimistic locking implementation released in JBossCache 1.2.4.

For more information on what's been improved, see this wiki page.

I expect to release a beta for public trial in the near future, but for the impatient, I'd encourage getting the latest from CVS and giving it a whirl.

Cheers,
Manik Surtani

Posted on Tue, 24 Jan 2006 14:11 by admin ( day(s) old) Trackbacks [0]

Buddy replication in JBoss clustering and alternatives

[ Manik Surtani ] [Permalink]

When talking about application server clustering, I am often asked the question about whether JBoss AS supports buddy replication for HTTP or SFSB sessions. The simple answer is no, not yet. The more complex - and useful - answer though, is no, not yet, but there is a lot you can do with what we will be releasing in JBossCache 1.3 "Wasabi" (the underpinning technology used by JBoss Clustering).

Let's start with a simple question: what is buddy replication? Buddy replication is the ability to replicate session data to specific servers in a cluster and not to everyone. The goals of buddy replication are:

  1. to achieve failover by ensuring that collective state is maintained even if some servers in a cluster go down.
  2. to minimise network traffic that is a result of full replication of state to all servers in a cluster.
  3. to minimise memory usage which is a result of each server maintaining not only its own state but the state of every server in the cluster.
While buddy replication is on the JBossCache roadmap and is slated for release around summer 2006, there are other exciting, new features in JBossCache 1.3 "Wasabi", due for release in Feb 2006. These features can be combined to achieve a similar effect to buddy replication.

  1. To achieve durability of persistent data: a chaining cache loader should be configured with a ClusteredCacheLoader and a JDBCCacheLoader.
    • Use the ClusteredCacheLoader to quickly retrieve data from other caches in the cluster.
    • Use a shared JDBCCacheLoader as the 'true' representation of all state in the system.
  2. To minimise network traffic: JBossCache 1.3 "Wasabi" supports invalidation in addition to replication. If a cluster were configured with invalidation rather than replication, we have effectively minimised network load since invalidation does not replicate in-memory state but rather simply broadcasts very light invalidation messages.
  3. To minimise memory usage on cluster nodes: since data is not replicated to all servers in the cluster, all servers don't maintain all session data in the cluster. This results in each server maintaining only it's own state in memory rather than the state of the entire cluster.
So, if your cluster consists of a large number of servers, you can still achieve the goals of buddy replication using the above features.

For more information and details, see my wiki page on JBossCacheBuddyReplication

Posted on Thu, 15 Dec 2005 11:00 by admin ( day(s) old) Trackbacks [0]

JBossCache 1.3.0 "Wasabi"

[ Manik Surtani ] [Permalink]

JBossCache 1.3.0 “Wasabi” is the next major release of JBossCache, containing a whole host of new features, including:

  • Optimistic node locking (see wiki page)
  • Invalidation instead of replication
  • Improved CacheLoader performance
  • Ability to ‘chain’ more than one CacheLoader
  • A new ‘Options’ API, allowing configuration overriding on a per-invocation basis
  • A new ‘ClusterCacheLoader’ that treats a remotely running caches in the same cluster as a cache loader source
  • JDBCCacheLoader with improved compatibility with MySQL JDBC drivers
  • Replication performance enhancements with JGroups 2.2.9
  • ‘Buddy’ replication – where state can just be replicated to a single other node in the cluster rather than broadcast to every node.

Naturally, this is just a roadmap, and not all features may make it into “Wasabi”. “Wasabi” is planned for release by the end of February, 2006, with betas available by mid-January and alphas/developer releases by December.

Please visit the JBossCache design forums to participate in design discussions regarding this release.

“Wasabi”. It’s hot. You like it. You know you want some.

Posted on Thu, 10 Nov 2005 10:03 by admin ( day(s) old) Trackbacks [0]

Smells like Optimistic Locking

[ Manik Surtani ] [Permalink]

Imagine a world where you have near-serializable data integrity in an in-memory, replicated caching product. All while being able to scale, perform, manage a high degree of concurrency. Smells like optimistic locking in JBossCache? You smell well. Your nose knows. You sniff terrif-ically. Ok, forget the last one, that was bad.

Locking, whether for reading or writing, has been a necessary evil when dealing with concurrent access on datasets. Locks are currently implemented in a pessimistic fashion in JBossCache. Meaning, data is locked, whether for reading or writing, throughout the duration of a transaction involving access to the data. This can lead to scalability issues and deadlocks.

Optimistic locking assumes that locks aren't necessary for concurrent access, and deals with synchronisation and merging of concurrently accessed data at the time of committing a transaction. Some locking is still necessary, but this is only for a very short duration at the commit boundary of a transaction.

The long and the short of it, you can now have a high level of data integrity while still maintaining a high degree of concurrency, leading to a very scalable caching product.

Details on obtaining, configuration, design, etc. are on the JBoss Wiki - feedback is very much appreciated!

Cheers,
Manik

Posted on Fri, 2 Sep 2005 09:51 by admin ( day(s) old) Trackbacks [0]