DougHughes.net

We have created two other articles so far which relate to the High Availability (HA) in web application environments. The first was an overall theoretical view of different levels of HA, from very basic to a more Enterprise level environment. The second article explored terminologies relating to Clustering, Load Balancing and Fail Over.

In this article we will explore, in more detail, what is required to set up Clustering to serve Load Balancing needs. We will use a mid-level Enterprise Infrastructure example, from a previous article, as a focus for our discussion; here is that Infrastructure…

Just below the mid point in this diagram we see a Hardware Clustering device. Here we are showing two Web-CF Servers behind that Hardware Clustering device, these principles apply no matter how many Web-CF servers there are, in excess of the minimum of 2, needing to be Load Balanced.

As a note point, most of these algorithms are also applicable to Software Based Load Balancing, I prefer to use Hardware Based Load Balancing for two reasons. Firstly, I have found it better to have an independent dedicated device as the Clustering device, one that is not part of the Cluster. Secondly, Hardware Clustering devices are more robust and a single Hardware Clustering device can support more Web-CF Servers than a Software based one.

As an historical point, at Allaire-Macromedia, we felt that Software Clustering should not be used in clusters of more than 8 servers. That is full-on Software based Clustering with no Hardware device. However there is now one very compelling reason to include Software in the mix. With any J2EE Server based infrastructure, in-memory sessions can be shared by all members in a Cluster, Cluster members are known as “Buddies”. It is important to note that with ColdFusion this is only available in a J2EE Multiple Instance install which means it is only available to us in an Enterprise version.

We will go into greater detail as to how we enable this is a follow on article on Software Based Clustering of which Load Balancing will be part. I will share one overall concept here though. My opinion is if you intend to use Session replication it is better to have the Hardware Clustering device simply handle Fail-Over and let the Software (CF or JRun) handle the Load Balancing aspect of clustering. As I say more on this in a later article. O

ne other important point when considering Clustering is to make sure that all web site content on every Cluster member is identical. This can be done manually but I do not recommend that. The best systems I have seen usually employ software based replication of content on some sort of scheduled basis. Here are the main Load Balancing algorithms with the pros and cons of each.

Random Allocation

In a Random allocation, the Hardware Load Balancer decides which server gets that request in a random fashion. I have never seen this used and cannot think of any reason why this should be used. We note it here for inclusion purposes only. The main con here is that the load will most likely not be distributed evenly.

Round Robin

The most simple and in my experience, most effective type of Load Balancing comes from the use of the most simple algorithm available; “RoundRobin”. There are qualifications though, which we will explore later. RoundRobin Load Balancing takes each user request and distributes in a sequential pattern around the Web-CF servers; like this (we assume two Web-CF servers only here, adding more servers does not change this pattern)…

• User 1 – Request 1 – Web-CF Server 1
• User 1 – Request 2 – Web-CF Server 2
• User 1 – Request 3 – Web-CF Server 1
• User 2 – Request 1 – Web-CF Server 2
• User 2 – Request 2 – Web-CF Server 1
• User 2 – Request 3 – Web-CF Server 2

RoundRobin evenly distributes load across all available Web-CF servers, the only thing in this case that can make load slightly uneven is where one users request runs a heavier process than another request or other users request. My experience though has always been that RoundRobin is the most evenly distributed algorithm to use.

So then, why do we not all use RoundRobin and leave it at that? The main reason is in memory sessions-variables etc. If we simply bounce a user around a cluster of servers and they have in memory-stateful data it will be lost once they leave the first Web-CF server they are on. There is a way to mitigate this by setting what are called “Sticky-Sessions” on the Clustering device. Sticky-sessions will make sure that once a user hits Web-CF Server 1, they stay there till that session is completed. In this case we get this sort of pattern…

• User 1 – Request 1 – Web-CF Server 1
• User 1 – Request 2 – Web-CF Server 1
• User 1 – Request 3 – Web-CF Server 1
• User 2 – Request 1 – Web-CF Server 2
• User 2 – Request 2 – Web-CF Server 2
• User 2 – Request 3 – Web-CF Server 2

In a RoundRobin Sticky-Session based Load Balancing the load is not distributed as evenly as with the basic RoundRobin, however in my experience, Sticky Sessions are often widely used and are typically preserved-tracked by a cookie based method.

Weighted Roundrobin

In this algorithm a weighting can be assigned to any server in the Cluster to signify that a particular server will receive more load than others in the Cluster; for instance twice as many requests as others in the Cluster. This may be desired if one server is more powerful than others. Although, ideally, all servers in a Cluster should be as identical as possible. I have not found many occasions where Weighted Round-Robin is necessary especially as Sticky-Sessions are often employed.

Least Connections – (Least Load?)

In this algorithm the Clustering device monitors each Cluster member to see which member currently has the least number of connections. The next request is then sent to that Cluster member. In theory this would seem to be a logical way of ensuring that no one Cluster member gets overloaded. There are two downsides to this algorithm, firstly if any Cluster member has to be restarted the Clustering device will see it as having the least requests on restart and can bombard that server with all requests causing instability, I have seen that happen. Secondly, when Sticky Sessions are enabled there is rarely a need to allocate requests based on load as site visitors are not moved around mid-session.

IP-URL Based Load Balancing

In this case users are moved around a cluster based on their source IP or on the URL they are requesting. This form of Load Balancing is rare, as far as my experience goes and if it where employed I would assume it might be done for locale-langauge reasons or perhaps security reasons relating to content. In essence this could unbalance rather than balance load.

Conclusion

My experience is that the most effective Load Balancing in a Cluster is Round-Robin with Sticky-Sessions, if it is necessary to preserve state in a single user session. These algorithms here typically apply to Software Clustering/Load Balancing also and we will cover that in a later blog posting.

One last point, if you are using something to monitor the “health” of a Cluster member, it is important that the health check itself does not actually create problems by hitting a heavy part of an application. I have actually seen health-checks cause severe problems several times. I realize this is more applicable to Clustering Fail Over but it is important, so I mention it here also.