DougHughes.net

Knowledge Crisis Looming?

I was inspired to create this blog post by one I came across whilst researching SSL Acceleration in a Clustered environment. I will return to that subject later. That blog post postulated a coming Talent Crisis because it is no longer going to be adequate to simply know about the web server, application server, etc.

I agree with the thrust of that article but I actually see it more as a Knowledge Crisis. I think those who can adequately create well written and thought out code can also fairly easily understand routing tables, or efficient database indexing, or figure out TCP/IP packet operability and needs and so on.

As data passing through infrastructure devices (Clustering, Switch, Firewalls) is no longer treated like water in a plumbing system (as a dumb material) I believe we will all need to become more aware of everything affecting web applications, or risk losing our usefulness and marketability. We are seeing definite trends where infrastructure devices are becoming “Content Aware”. For instance Clustering Devices and WAN Accelerators can parse http. Security devices are not just monitoring access via access control lists or looking for text or string matches.

In short applications will probably need to be coded to be infrastructure-effects aware and adaptable, hence my reference to considering the possible use of Clustering devices when architecting and developing applications. As I mentioned and In my opinion, this is not a issue relating to Talent but one relating to Knowledge. Those individuals and organizations that can meet the impending needs effectively will, in my opinion, prosper very well in the evolving application-infrastructure melding that is very much around the corner.

Clustering-HA – Hardware Load Balancing

We have created two other articles so far which relate to the High Availability (HA) in web application environments. The first was an overall theoretical view of different levels of HA, from very basic to a more Enterprise level environment. The second article explored terminologies relating to Clustering, Load Balancing and Fail Over.

In this article we will explore, in more detail, what is required to set up Clustering to serve Load Balancing needs. We will use a mid-level Enterprise Infrastructure example, from a previous article, as a focus for our discussion; here is that Infrastructure…

Just below the mid point in this diagram we see a Hardware Clustering device. Here we are showing two Web-CF Servers behind that Hardware Clustering device, these principles apply no matter how many Web-CF servers there are, in excess of the minimum of 2, needing to be Load Balanced.

As a note point, most of these algorithms are also applicable to Software Based Load Balancing, I prefer to use Hardware Based Load Balancing for two reasons. Firstly, I have found it better to have an independent dedicated device as the Clustering device, one that is not part of the Cluster. Secondly, Hardware Clustering devices are more robust and a single Hardware Clustering device can support more Web-CF Servers than a Software based one.

As an historical point, at Allaire-Macromedia, we felt that Software Clustering should not be used in clusters of more than 8 servers. That is full-on Software based Clustering with no Hardware device. However there is now one very compelling reason to include Software in the mix. With any J2EE Server based infrastructure, in-memory sessions can be shared by all members in a Cluster, Cluster members are known as “Buddies”. It is important to note that with ColdFusion this is only available in a J2EE Multiple Instance install which means it is only available to us in an Enterprise version.

We will go into greater detail as to how we enable this is a follow on article on Software Based Clustering of which Load Balancing will be part. I will share one overall concept here though. My opinion is if you intend to use Session replication it is better to have the Hardware Clustering device simply handle Fail-Over and let the Software (CF or JRun) handle the Load Balancing aspect of clustering. As I say more on this in a later article. O

ne other important point when considering Clustering is to make sure that all web site content on every Cluster member is identical. This can be done manually but I do not recommend that. The best systems I have seen usually employ software based replication of content on some sort of scheduled basis. Here are the main Load Balancing algorithms with the pros and cons of each.

Random Allocation

In a Random allocation, the Hardware Load Balancer decides which server gets that request in a random fashion. I have never seen this used and cannot think of any reason why this should be used. We note it here for inclusion purposes only. The main con here is that the load will most likely not be distributed evenly.

Round Robin

The most simple and in my experience, most effective type of Load Balancing comes from the use of the most simple algorithm available; “RoundRobin”. There are qualifications though, which we will explore later. RoundRobin Load Balancing takes each user request and distributes in a sequential pattern around the Web-CF servers; like this (we assume two Web-CF servers only here, adding more servers does not change this pattern)…

• User 1 – Request 1 – Web-CF Server 1
• User 1 – Request 2 – Web-CF Server 2
• User 1 – Request 3 – Web-CF Server 1
• User 2 – Request 1 – Web-CF Server 2
• User 2 – Request 2 – Web-CF Server 1
• User 2 – Request 3 – Web-CF Server 2

RoundRobin evenly distributes load across all available Web-CF servers, the only thing in this case that can make load slightly uneven is where one users request runs a heavier process than another request or other users request. My experience though has always been that RoundRobin is the most evenly distributed algorithm to use.

So then, why do we not all use RoundRobin and leave it at that? The main reason is in memory sessions-variables etc. If we simply bounce a user around a cluster of servers and they have in memory-stateful data it will be lost once they leave the first Web-CF server they are on. There is a way to mitigate this by setting what are called “Sticky-Sessions” on the Clustering device. Sticky-sessions will make sure that once a user hits Web-CF Server 1, they stay there till that session is completed. In this case we get this sort of pattern…

• User 1 – Request 1 – Web-CF Server 1
• User 1 – Request 2 – Web-CF Server 1
• User 1 – Request 3 – Web-CF Server 1
• User 2 – Request 1 – Web-CF Server 2
• User 2 – Request 2 – Web-CF Server 2
• User 2 – Request 3 – Web-CF Server 2

In a RoundRobin Sticky-Session based Load Balancing the load is not distributed as evenly as with the basic RoundRobin, however in my experience, Sticky Sessions are often widely used and are typically preserved-tracked by a cookie based method.

Weighted Roundrobin

In this algorithm a weighting can be assigned to any server in the Cluster to signify that a particular server will receive more load than others in the Cluster; for instance twice as many requests as others in the Cluster. This may be desired if one server is more powerful than others. Although, ideally, all servers in a Cluster should be as identical as possible. I have not found many occasions where Weighted Round-Robin is necessary especially as Sticky-Sessions are often employed.

Least Connections – (Least Load?)

In this algorithm the Clustering device monitors each Cluster member to see which member currently has the least number of connections. The next request is then sent to that Cluster member. In theory this would seem to be a logical way of ensuring that no one Cluster member gets overloaded. There are two downsides to this algorithm, firstly if any Cluster member has to be restarted the Clustering device will see it as having the least requests on restart and can bombard that server with all requests causing instability, I have seen that happen. Secondly, when Sticky Sessions are enabled there is rarely a need to allocate requests based on load as site visitors are not moved around mid-session.

IP-URL Based Load Balancing

In this case users are moved around a cluster based on their source IP or on the URL they are requesting. This form of Load Balancing is rare, as far as my experience goes and if it where employed I would assume it might be done for locale-langauge reasons or perhaps security reasons relating to content. In essence this could unbalance rather than balance load.

Conclusion

My experience is that the most effective Load Balancing in a Cluster is Round-Robin with Sticky-Sessions, if it is necessary to preserve state in a single user session. These algorithms here typically apply to Software Clustering/Load Balancing also and we will cover that in a later blog posting.

One last point, if you are using something to monitor the “health” of a Cluster member, it is important that the health check itself does not actually create problems by hitting a heavy part of an application. I have actually seen health-checks cause severe problems several times. I realize this is more applicable to Clustering Fail Over but it is important, so I mention it here also.

Clustering – The Father To The Man

There is a term heavily used in enterprise level environments called “High Availability” or HA as it is also known. There is a previous article where we discussed some HA concepts.

As the term implies it refers to something being available most of the time, a “high” amount of the time. There are many different infrastructures and procedures which are required in a successful HA implementation. In the first post in this series we will look at one key component of HA; Clustering.

In my experience, Clustering is an overall term which also encompasses Load Balancing and Fail-Over. The concepts I am laying out here can apply to any hardware or software and are not limited to ColdFusion and/or JRun. We will get much more specific as we progress through this series and will show details of ColdFusion-JRun in clustered deployments.

Load Balancing

We often hear the term load balancing used when in my opinion the correct term is clustering. Here is a typical example that I have encountered many times. Clients will often refer to a Hardware clustering device such as a BigIP F5 or Cisco Content Service Switch; as a Load Balancer.

Here is a quick simple illustration of the relationship of Clustering to Load Balancing. In the drawing below we have stripped down illustration of Load Balancing. Requests come via the Public Internet and eventually to a Hardware Clustering device (not a Load Balancer because as we will see shortly, its job is to do far more than Load Balance).

The Clustering device directs the requests from the Internet to one of two Web Servers which sit behind the Clustering device. These requests are sent on an alternating basis to Webserver1> Webserver2> Webserver1> Webserver2> etc. This form of Load Balancing is known as “RoundRobin”. Load is distributed totally evenly among all servers in the Cluster. This is a simple Cluster and we are illustrating Load Balancing only within this Cluster of 2 servers.

Fail Over

In this second scenario, Webserver2 is no longer available to service requests so the Clustering device is sending requests only to Webserver1. Typically, the way the Clustering Device knows that Webserver2 is not available is by the use of a “Heartbeat” ping which could be as simple as a low level (ICMP) ping or possible a URL request to a page which could be a simple HTML page where the Clustering device is looking for a successful 200 response or possibly some particular text on that page.

Summary

As we have shown here, the hardware device is not simply balancing the load across multiple Web Servers but is also making sure both are available and failing over requests (Fail Over) in the event of a server becoming unavailable. The hardware device is a Clustering device any time it is connected to more than one other device and where it manages the connections of those devices and Load Balancing is only one of its jobs. We will continue this series with more detailed information on Clustering and HA.

Having issues with CF mappings and Flash Remoting? Try this.

This tidbit just in: if you’ve had an issue where you have things in mappings (like ColdSpring, for example) and you’re having issues with those mappings resolving correctly while hitting CFCs via the built-in Flash Remoting gateway that comes with ColdFusion, try this:

Open {cf install dir}/web-inf/flex/remoting-config.xml and change

<use-mappings>false</use-mappings>

to

<use-mappings>true</use-mappings>

(line 24 in my remoting-config.xml file).

Restart your ColdFusion instance.

Test your Flex app with your CFCs in a mapped location instead of in the webroot.

It works? Great!

Why? Well, lets assume you’re trying to find a file via expandPath(“/coldspring”)… as near as I can tell, with that setting set to true, the remoting gateway will simply slap /coldspring onto the end of the webroot path and look in that directory, say “c:inetpubwwwrootcoldspring” when /coldspring is mapped to “c:webdevexternalcoldspring”. The remoting gateway never checks to see if there is a mapping called coldspring, it just assumes that / means “webroot-relative”. With that setting on, however, the remoting gateway checks with CF to see if there’s a /coldspring mapping before it does anything with the webroot path.

There’s some question as to why it is an issue with instantiating a CFC vs once the CFC has been instantiated… apparently once you’ve instantiated the CFC it works fine. I’m assuming that it’s because the gateway servlet is involved at first, and then the CFC servlet. The CFC servlet works fine and once it’s in control of the CFC it resolves paths correctly.

Just a guess, but it’s the only reason I see that things would behave the way they do.

Best of luck with this…

Model-Glue Training: Plenty of Seats Available and Only Two Weeks Left for Reduced Pricing

Ive blogged about this class several times, but to reiterate, Alagad will be offering a four day training class on Model-Glue in Arlington, Virginia between February 4th and 7th 2008. The class will take students from the foundations of CFC based programming to the intricacies of Model-Glue, ColdSpring and other related ColdFusion frameworks.

If you havent already heard about the class you can learn more here.

If youre interested in the class you might want register before December 15th. The price is currently $1000 for all four days. This is a steal for the level of training youll be getting. After the 15th the price goes up to $1200. Register now to save $200.

Registration closes on January 5th, so youll want initiate the registration process pretty quickly if you have to go though several levels of corporate approvals.

We still have quite a few seats available! Dont miss out on this.

If you have any questions about the class youre always welcome to send us a message though our contact us form or give us a call at (888) Alagad4.

Register for the Feb 4 to 7, 2008 Enterprise ColdFusion with Model-Glue Training Session.