Every week I read the ColdFusion Open-Source Update put out by Brian Rinaldi. Now, I used to pride myself on seeing Reactor mentioned frequently on that list. Unfortunately, several weeks have gone by without any real news from the Reactor camp.
This is really my fault. I honestly just haven’t felt much like blogging lately. I’ve been recovering from a stressful period in my live involving a new baby (who I am simply smitten by, by the way), selling my townhouse, moving into a new house, lots of work, too much email and a few new projects. All in all, I just caved in for a while. I’m feeling better now and I’m trying to try to blog more.
Furthermore, as much as I respect Brian, and as much as I know he tries not to be too critical or too imbalanced, I feel as if Reactor is getting bad press on his update. I understand he had some negative experiences with Reactor, though I’m not sure what they were.
Anyhow, in the attempt to try to shed some more positive light on Reactor, here’s some Reactor news I haven’t covered on my blog in the last two months:
Beth Bowen Takes the Bull By the Horns
Beth Bowden, who initially contributed Oracle support has been fixing my bugs. Since mid August she’s made nearly 40 commits to the framework. The fixes have includes a wide range of issues both large and small.
You know what? This is nothing small. I’ve done next to nothing compared to her in terms of bug fixes in that time period. She’s simply the unsung hero of Reactor and deserves wide praise and appreciation from the Reactor community in general.
Beth, take a bow!
Huge Performance Gains
As much as it pains me to say this, historically Reactor’s been a bit of a dog (in terms of speed) and a bit of a hog (in terms of memory usage).
There! I said it out loud! I feel liberated! I’ve secretly been beating myself senseless about this.
Actually, it’s not so secret. I asked for help from people in tracking down these issues months ago. The guys from Webappr stepped up the plate and determined that, yes, Reactor used a lot of ram. Furthermore, for some reason Reactor was “causing” Java to do a lot of full garbage collections (Full GCs).
These full GCs didn’t seem to be able to completely collect old memory, for some reason. As more and more memory was used the full GCs (which, by the way are a stop-the-world event) would take more and more time. Eventually servers would hang and need to be restarted.
The odd thing? It didn’t seem to happen to everyone. It although the problem occurred production mode, it was a real problem (at least for me) in development mode I personally would be guaranteed at least one manual CF restart a day. What a mess.
However, after a while Reactor’s memory usage did seem to level out. And, thankfully the Webapper guys informed me that full GCs could be disabled in JRun. That worked absolute miracles for stability. Unfortunately, Reactor continued to be slower and more memory intensive than I wanted it to be.
So, I spent quite a while thinking about the problems. There was nothing obvious that was specifically slowing down the works or eating up ram. I tried a dozens of techniques to identify where the problems were and tweaked lots code.
Ultimately I speculated that the problem lies, at least in part, with the use of so many bloody CFCs in Reactor. OO queries used to use a LOT of CFCs to do their job. So, I essentially rewrote them and removed two entire classes (choosing to store data in structs instead). This helped a bit, but not as much as I wanted.
I suspect that the deep inheritance tree on Reactor objects doesn’t help. So, I removed the base abstract object (which really served no essential function). I might in the future make the generation and use of DBMS agnostic files optional. I think that not using these might help a bit too.
The thing that made a huge difference was something I only did within the past week or so. Logically, OO queries are very dynamic when they don’t need to be. That is to say, if you create a query, join to a table, add a few where statements and sort the results, each time that code runs it’s going to have to go through all of the validation (which is expensive) and the translation of the OO query to a real SQL statement.
In development mode this process could take a quarter second. In production it was more like 60 ms (of course this is dependant on hardware, etc). This isn’t bad, but it’s not good either.
So, I decided to make a facade of OO queries which implemented the same interface as the actual OO queries. However, when you call methods the facade simply stores the commands in a structure. Then, when you run the query, the gateway, which use to be at least partially in charge of transforming the query to SQL, simply asks the facade for a file to include. The facade does a quick hash of the data you provided it and then looks for a file on disk that matches that hash. If the file is found it’s returned otherwise the facade validates the query and generates a static SQL statement (complete with cfqueryparams) to disk and returns the path to that file.
The gateway then simply includes the SQL statement into its cfquery tag. The first run of a query is on par with previous speeds. However, subsequent requests are more like 14 ms. That’s a 300% improvement in speed!
Duckies -or- Anything Goes
Speaking of speed, as is well known, a few popular frameworks have gone “typeless”. That is to say that all (or most) of the core framework’s methods have had their cffunction and cfattribute tags updated so that their returntype and type attributes are set to any, instead of specifying a specific data type. This essentially disables ColdFusion’s runtime type checking.
Because ColdFusion no longer checks the types when methods are called Reactor should theoretically get a lot faster. In practice it seems that this helped some cases but not others. Jared Rypka-Hauer praised it as being much faster while I’m not sure I saw much of a difference.
Anyhow, to make sure that the actual types were recorded for documentation I added two new made up attributes to the cffunction and cfargument tags, _returntype and _type. These hold the “real” data type and show up in the CFC’s metadata, but not in the ColdFusion-generated documentation, unfortunately.
Joe Rinehart contributed a feature whereby ColdSpring can inject itself into the ReactorFactory when it’s instantiated. Furthermore, Reactor lets you access the BeanFactory to get any ColdSpring configured bean. Simply use the _getBean() method. (Note, this won’t show up in metadata due to the fact it’s mixed in.) It’s debatable as to if this is a good thing, but it’s there!
Reactor Continues to Evolve
Simply put, there have been a ton of little things quietly changing over the past few months. Far too many to blog about.
I know the docs aren’t done yet. That’s my fault. I’ll get there soon! And before long we’ll be at a 1.0! (And no, contrary to certain reports, Reactor is still not a 1.0.)
Comments on: "Long Overdue Reactor News" (9)
Thanks for the updates Doug, I for one very much appreciate the work you put into Reactor. It has monumentally sped up the development process for me.
Doug, are there any examples of the Bean Factory support in action? I must have missed that on the list, but would like to see how that all fits togeter if there is any public code available.
Doug thank you for your continued works and efforts. I was a pleasure helping you whilst at Webapper and if I can do anything to help further, please let me know.
Hey Doug, congrats to you and the wife on your new addition. Seems to be a lot of that going on right now;) Anyway, keep up the phenomenol work on Reactor, your efforts are appreciated.
Doug, Thanks for the link and the postive comments about the update. I do want to address your concerns though that Reactor isn’t getting a fair shake on the update.
First, I do obviously have my own opinion regarding certain projects (both positive and negative), but I really do try to make the update and the list not reflective of those. Those I save for seperate posts or blog comments.
Second, I have had issues with Reactor running the OS list, but I have pointed out that it has more to do with my using an old version of Reactor than with anything else. I believe I have also taken care to always mention that any issues I am having are not reflective of the current version as I have not tested it personally.
Third, I actually follow your mailing list, and have a number of times tried to include updates discussed on the list (as I do some others), but what is updated isn’t always clear (partly because it does have a lot of traffic). Also, when I go to the archives, I can never seem to easily find a link to the message that discusses it so that I can link that off the update (is it just me? could be).
To be quite honest, the update takes me a lot of time and I do my best to follow what happens in every open source project whether I use it or not. I am on the list of nearly every project that has a list and on several others in part to follow everything that goes on to the best of my ability.
Some people have been kind enough to just send me a quick email when there is something they would like to have me highlight, which is a big help to me and also ensures that important news project doesn’t get overlooked. Perhaps you would consider that if you think I am overlooking your project for personal reasons, which would be a nice gesture even if I assure you that I am not doing any such thing. In fact, your post was already added to my list for next week before I even read it.
Brian – Whoops. I categorically never intended to imply that you were being at all unfair to Reactor. If it came across that way then it was simply an error.
What I meant to say (and hopefully I say it clearly this time) is that I havent made enough public announcements of Reactor news. Without that, you couldnt possibly (or at least reasonably) have any way of knowing about them and adding them to the update. Thats why I wrote the entry. I needed, nay; I have the responsibility, to share news. Ive been remiss in doing that.
Furthermore, I honestly dont know what problems youve run into with Reactor. Id try to address them if I did. I respect that you have chosen not to use Reactor. Do you mind mailing me later or posting a blog entry (or something) about your experiences? I dont exactly enjoy criticism, but that which doesnt kill me makes me stronger!
Anyhow, implying that you have had issues with Reactor, which you say you have had, is not intended to say that youre being unfair to Reactor. Youre not. In fact, Ive always thought youve been nearly painful fair and balanced.
Doug, thanks for the comment. I know you didn’t mean any criticism, I guess I just wanted to clarify that I take pains to be impartial on the list and updates. I suppose, in hindsight, my discussing that I was redoing the list because of issues I had encountered with Reactor may have seemed negative – but it really was an old version, I honestly didn’t think it was a reflection on the current version and I hope that was clear…and I remain supportive of the project and its goals even if, at the moment, I am choosing not to use it and I will continue to follow it closely.
P.S. Congrats on the baby, just had one of my own (my 2nd) and I know how that can negatively affect your ability to find time to write.
Glad to hear you’ve emerged from all that stress.
I look forward to further Reactor developments. Please keep blogging!
fyi~ the current release has a misplaced file in the mysql folder, currently ita mssql file
thanks 4 the great work