As Dave says in his blog, the good news is that it can handle thousands of connections.  However, the bad news is that there are clearly some tradeoffs.  To understand these, it’s important to understand what ActiveMQ does under the hood. 

Here is an example to show how ActiveMQ works.  Say you are going to build out a system capable of handling 4000 concurrent connections.  For now, we won’t assume anything in relation to number of queues, transactions per second, etc.  (Don’t worry, we’ll get there).  In order to get the best performance from ActiveMQ, it’s best if the total processing threads for the JVM stays below 1500 threads.

The thread model for ActiveMQ can be tweaked in several ways, but I’m going to focus on the configuration in Dave’s blog.  The nio connector is extremely important.  The tcp and ssl connectors by default will create a single thread per connection.  The nio connector uses a considerably smaller thread pool, which reduces the thread counts roughly by a factor of 4-5. 

Tradeoff:  You have to put encryption on the producer consumer, or if you can, have your servers fronted by a load balancer, preferably with SSL acceleration.  (I haven’t found much information about ActiveMQ, working behind a load balancer, but in my own tests, I’ve never had a problem with it).

Back to our example.  We have 4000 connections, which means we should have approximately 800 – 1000 threads allocated to processing connection data.  This can go larger depending on load, but we’ll consider it there for the sake of argument. Considering we want to keep the thread count below 1500, we have roughly 500 left to work with.

The next thing to consider is how threads are used in ActiveMQ.  The details are here. Suffice to say, if you make the recommended configuration changes, you’ll reduce the number of threads required to process messages.  The most important change you can make is to set the org.apache.activemq.UseDedicatedTaskRunner variable to false.The ActiveMQ documentation says this:

ActiveMQ can optionally use internally a thread pool to control dispatching of messages – but as a lot of deployment operating systems are good at handling a large number of threads, this is off by default.

Although, there are varying thoughts on the subject, I still believe that ActiveMQ should ship with this configuration.  The dedicated task runner basically tells ActiveMQ not to use thread pooling, and to create a single thread for every queue. 

Tradeoff: You can’t use a large number of destinations.  In a connection heavy configuration, plan on having no more than 500 queues, preferably less than 200.

The reasoning behind this is straightforward.  The more queues, the more time ActiveMQ spends on context switching, and the less time it spends on message delivery.  Back to our example, figure that 500 queues is approximately 250 threads under load.  Now we are down to 850 threads available.

Tradeoff: Broker networks will degrade your message performance dramatically.  If you can use another high availability strategy, then do so. If you have to use a broker network, then plan on having a sharded environment and only having broker network pairs.

Broker networks do a number of interesting things.  First, broker networks use advisory topics to get notifications of when consumers are available so that they can determine which server to deliver messages to.  This means that for every queue created, an advisory topic is created and a consumer for that advisory topic.  Figure you just created another 250 threads for those messages if you are in a pair.  Another side effect of advisory topics is producer flow control. For standard queues and topics, producer flow control can be shut off, which is a very good thing, since it will create a flow control thread for every queue.  Unfortunately, producer flow control cannot be shut off on advisory topics.  That means while the delivery threads work in the same thread pool, flow control will automatically add another 500 threads to your thread count.  If you are keeping track that means we are already at 2000 threads.

This means you have to shard your environment and reduce connections.  If a network of brokers is your only option, then plan on sharding your environment into server pairs and plan on manually balancing connections on those pairs.  This makes for some interesting message delivery issues, but it will allow ActiveMQ to process your threads.  If you can keep a single machine down to 1000 connections, you will have sufficient headroom on your broker network pair to failover all 1000 connections to the second machine in case of a failure. 

Best of luck with your configuration.