We all know, or at least should know that server-side validation of form data is a must – whether or not you use some sort of client-side validation. I have worked on quite a few projects and it seems like every project handles this a bit differently. In my mind, form validation could very easily be standardized so that we would not have to reinvent the wheel with each application. There are very complete JavaScript libraries out there such as qForms, so why is there nothing in ColdFusion?
Today, we are going to fix this. If you were tasked to build a ColdFusion server-side form validation library, what would it look like? Suppose the following requirements were given and lets see what kind of creativity can be applied.
- The validation library should be able to be dropped into any application and validate any type of form. In other words, there needs to be a clean “language” for specifying things like required fields, field types (text, numeric, postal code, bank account, etc.), error messages, and the like.
- The rendering of the error messages needs to be flexible … whether they are displayed in a single list, next to the associated field, in a JavaScript pop-up, or whatever … that should be up to the end developer/designer.
- The validation library should be extensible. No matter what kind of functionality is implemented, there will always be a weird validation rule that needs to be created for some application.
- The validation library should be able to support validating an entire form at once vs. validating one field at a time. In other words, the developer should be able to validate the form on submit and redisplay the form with errors, or he/she could use some sort of AJAX call to validate a specific field or fields upon an event.
So, while these requirements are not trivial, this leaves things pretty wide open. If you were tasked with building something like this, how would you do it? I have some ideas, but rather than limit peoples creativity, I want to see what can be come up with first.
DougHughes.net 
Comments on: "Form Validation – How do you do it?" (36)
I would say that the only display related requirement should be what you listed in #4 conversely I would leave out number 2 entirely. I would leave rendering completely out of it because that will be handled by the application this would be dropped into, in theory.
LikeLike
Paul – quite true. I did not state that very well, but I would expect to see some sort of error collection coming back that a developer / designer could use however they wanted to for the UI display.
Thanks.
LikeLike
Not sure how to answer the “how would you do it?” question. I would probably start by banging my head against the wall. 🙂
Seriously though, at our company we are about to embark on the similar adventure, only a little bigger. We want to abstract the form building also because … that’s what we need … a lot of similar but different dynamic forms. So I would be very interested in this discussion.
One thing that made our validation very difficult to abstract is when it depends on a value from another field, so some kind of dependency/binding would need to be implemented for it to work in our case, if you want to add that to requirements.
LikeLike
Ilya – dynamic form building is next. 🙂 I am working with an application right now that requires the client to be able to change the questions/answers on a form and while I don’t really like the way it is put together, I have not come up with a better method yet.
But, before getting side tracked into that discussion, I agree … a binding method for validation is required. There are a lot of special case validation methods like that that go beyond simple required or not validation. The form I am working with now has a basic password / confirm password field setup, so not only do I need to check that both were filled in, but both fields must match and also fit complexity requirements.
This is where something trivial like drop in form validation becomes much more complex.
LikeLike
It’s funny how timely this post is… As I type I am staring at the UML diagram that has a BaseComponent, Form and Field components and that’s it… and I was just about to head for that wall.. 🙂
Glad I am not alone.
LikeLike
Jared (Rypka-Hauer) was working on a generic validation library last year (it had some cute name beginning with S, I believe) but no one seemed interested in helping him with either requirements or testing (or development!).
LikeLike
I must have missed that one. I don’t know if it is just ColdFusion, but most open source projects don’t seem to garner much attention or help. Either way, I was mainly looking for ideas here, not necessarily code help. (Now searching for Jared’s info ….)
LikeLike
Yeah, Sean’s right… it was called Sting, after the sword in LotR that lights up when the environment was polluted.
I still have the sources.
The community didn’t really seem terribly interested, but the idea was to use it with apps in Fusebox, MG, Mach-II, etc., wherein you’d pass a struct that was, basically, the attributes struct in FB and the contents of the Event in Mach-II and MG and it would return a struct of exceptions generated. It was XML-configurable per application, extensible, framework-agnostic, etc. I was psyched, but just sort of lost momentum because of a total lack of interest.
In fact, Doug, I believe you were one of the people I discussed it with and didn’t get a whole lot of response. 😛 I’ll dig it out and dust it off. It’s been long enough I’m sure there’s a lot I’d do differently now. I’ve written several things like it since (though not as generic) and I don’t think it would be too hard to make a go of it again.
LikeLike
> so why is there nothing in ColdFusion?
Not to nitpick, but there is something built into coldfusion, and has been since at least CF 3.0. http://livedocs.adobe.com/coldfusion/6/Developing_ColdFusion_MX_Applications_with_CFML/formatData8.htm
Though it has not been modernized since it first appeared, and doesn’t qualify for your 4 requisites above.
LikeLike
Correction: near-total lack of interest.
There have been exactly 3 people in the last 18 months that thought it was worth the bother… I just sort of figured if nobody was going to use it what was the point of finishing it? Especially when there’s so many other projects to get involved with.
Still… my idea stands and I think it would be a good thing to finish.
If there’s actual interest I’m game for putting a little time into it.
LikeLike
@Jared, note that it was Jeff who posted this entry, not Doug! 🙂
@Jeremy, saying form validation has “not been modernized” is not accurate. CFMX7 updated all the form validation, adding lots of new types of validation (including regex) as well as improving when / where validation occurs.
LikeLike
Fair enough, Sean… I had NOT noticed that.
Sorry… I forget that Doug’s blog isn’t just Doug’s blog anymore. 😉
Anyway, the rest of what I said stands. 😛
@sean: I DID bring up the idea to you, Joe Rinehart, Doug Hughes, Spike, Rob Rohan, and a raft of other community folks. You’re the only one who showed even a momentary interest in the idea. Pretty much everyone else had the universal response “Why?” I’m NOT complaining about it… I’m just sayin’: that’s what happened. I’ve toyed with it again since, a bit, but it’s been a bit busy lately and I’ve just not been able to get real enthused about it.
LikeLike
@sean,
You’re right, of course. My apologies. I guess I meant it still follows the same basic format of using hidden form fields, as opposed to utilizing some of the newer technology or taking advantage of some of the newer CF features such as CFCs or something. My point was more to point out that it does exist than to criticize it.
@all
One idea we’ve had is to place the validation rules in a publicly available XML file, so the same rules could be utilized by both client- and server-side validators.
LikeLike
In working on just such a project I am heavily using Spry for the client side. Very flexible and easily tied into my CF work.
LikeLike
I was looking at number 4 and thought that maybe you should be able to validate a field at a time if you wanted to. If one is using AJAX type calls, so that user could potentially be notified of error before they finished with the whole thing.
Not sure of practicality at this point but just wanted to through it out there for suggestion.
For number 2. The error message itself should be customizable, I guess that’s obvious, but since I’m on the roll.
Also what would be great is to also be able to generate JS validation from data that’s used for CF validation.
… the wheels are turning 🙂
LikeLike
I really like the idea of there being a widely used validation framework/library. I had been watching for one on RIA Forge, but had instead started looking into rolling my own. As far as what it would do, there were similar uses as what you stated. Also the idea of being able to use some sort of naming convention on form fields and have them validated with JS (either onchange,onkeyup,onsubmit) and validated on CF side of course. I also like the idea of being able to validate either a struct or a generic bean with getters/setters as well, so you could test if a transfer object meets requirements before submitting for instance. At one place we had datatypes, like ssn, email, account number ect, which could be tested against in validation rules. Seemed to work out, although it wasn’t a finished thought.
LikeLike
Ilya – That is what I was going towards with #4 … basically that you would not have to validate the entire form at once and you could validate a single field at a time based upon some event.
In terms of the error messages, yes, they should be configurable. For that matter, they might not even be the text that is displayed. The returned error message could be a resource bundle key.
LikeLike
A couple of people have mentioned some sort of xml configuration for this … so, what would that look like? The problem I have run into before with such a thing is that in one form, the first name field might be required, but in the same app on a different form it isn’t.
You really need to have a configuration per form in an application, but that could get unwieldy very fast.
Ideas?
LikeLike
@Jeff: My config file was based on a per-event config for FB, MG and M2… that is, for any event needing validation, you’d set up a config block in the validator.
That way you only validate the pages you want to validate and you have a very nice deterministic way to tell what your library is doing.
Crap, now I’m all interested in doing this again… time to dust off the old sources I guess. 😉
Laterz,
J
LikeLike
To me, this is solving the wrong problem. Not that it isn’t nice to be able to validate forms, but what you really need is a library to validate beans – use your controller to load the form scope into the bean. That way you end up with a tool that’ll handle back end validation of data whether you’re importing from a text file, receiving it by email, accepting input from a Flex app or validating the form – they’re all the same core problem.
Most efficient approach is to support both bean and property validations. Bean validations capture dependency validations (If country = US, state must not be blank) and property validations are of the type “first name required” or “UserEmail is email”. I find most efficient way is to implement custom data types and each data type is associated to 0..n base validation methods. You also allow property overloading of custom data types (including validations and validation parameters) so you don’t need to add a new custom data type to make a minor change to a validation requirement. I also throw in validation against SQL data requirements for free so I check integers are numbers only and varchar(20)’s don’t have more than 20 characters.
LikeLike
Peter,
The point of my idea, I can’t speak for the ideas of others, is to validate the data from the Event or the Arguments scopes before handling them at all based on the data that’s supposed to be in those scopes.
I don’t want to handle it, process it, deal with it, then validate it… I want to make sure that form.fiddleMePhone is actually a phone number BEFORE I do anything with it. I want to make sure that what I have in my event’s inputs is right before I go handing any of the data off to the application. To my way of thinking this is especially important with websites where the raw HTTP data from the post, URL and CGI scopes are what our applications receive, including escaped data, anything extraneous from the URL and form fields, and so on… the web is 100 times dirtier than the inside of your average client-side Java application.
So no, it’s not solving the wrong problem, it’s solving the exact problem I want to solve… the real question is “Is this solving a problem that needs to be solved?” and the answer to that is “I won’t know until I’ve written and tested it.”
J
LikeLike
I will have to agree with Jared here as well. I want my validation as quick, simple, and light weight as possible. I don’t want to have to build a series of beans, stuff the data in, then validate it when I could just validate it up front with a lot less overhead on the application.
Along the lines of the ‘requirements’, I still want to see custom error messages, preferably in the form of resource bundle strings and I don’t feel that my beans should know anything about that. I also want to be able to validate one field vs. the entire form and bean level validation won’t do that either.
I don’t see either way as right or wrong – bean level validation just does not fit what I am looking to do.
LikeLike
@Jared/Jeff, Fair ’nuff. As always, no right or wrong – just different use cases and different approaches. I’d just rather write my validations once and let the server do a little more lifting than have to write different validation routiens for different scopes.
After all, it is ot like these things are trailing engine oil all over the floor – any form scope is comprised of a collection of strings. Apart from some kind of buffer everflow or magic character hack or something, I don’t really see the danger of passing those strings to be handled within a single model validation engine rather than coding validation into your controllers (which is what you’re effectively doing if it is form scope aware).
Of course, another alternative is to have a model API that accepts structs rather than beans as your input and to work around that. In that way the amount of passing goes down.
I do get the value of a simple “drop it in” form validation library. I just prefer to have a single integrated framework for my projects so I can rely heavily on convention and create apps in less time.
LikeLike
@peter
I get what you’re saying… but, first off what I’ve been saying isn’t about being form-scope aware. I’ve been saying that my idea would validate the contents of the attributes or event, which are the form and URL collected into one scope. Also, the concept behind Sting is that you can define the elements that your event should include or not include, reject a request if it contains certain vars or doesn’t contain others, and would validate against regex, pattern or explicit values.
None of these things can be easily done by passing the contents of your inputs into the model object, especially the part about rejecting requests if extra vars or missing vars show up in the post or get contents. It would (OK, WILL) contain standard validators like phone #, SSN, credit card, etc. It’s extremely declarative and very deterministic. The idea is to keep your application from ever encountering dirty or unacceptable data, reject bad URLs and basically act as the IO gatekeeper for your application.
I guess I’m just gonna have to finish it now, eh?
😀
J
LikeLike
@Peter, I like the custom data type idea. Have to think about that some more. It seems like every blog post I have read lately you commented on, you’re like everywhere, you must have “blog post answer generator” 🙂
@Jared, we would need bad form submits too. So bad data would need to get to the model.
@Jeff, I am with you on XML being unwieldy. For our app it will have to be DB driven. Too many forms.
I am thinking big here… Forms have collection of Fields, Fields have collection of validations, everything has a UUID and I can build an admin to connect the forms to fields and fields to (maybe insert custom data type here) validations and then pass form_id (or list of fields and form_id) to my model and get back error object. Then since I am defining the fields anyway might as well build the form too. 🙂
this is just a sketch, just some thoughts. It looks like this particular issue of validating forms is hard to generalize apart from your app and your controller/framework.
LikeLike
@Jared, Sounds cool. If I might make a suggestion, make your library just take a struct and have a separate method to load form and URL scopes into the struct. In that way you COULD use it for other method calls as long as they could pass in a struct (I have a feeling this may be the way you’re doing this anyway, but I don’t know for sure).
Actually, all of the things you’re doing I support as part of the way I load structs into the IBO and in addition the custom data types allow for cool stuff like auto concatenating three different form fields comprising the parts of a phone number into a single string automagically, but my approach is tightly integrated with everything else I do so for a single Software Product Line to generate all the apps in the world, it is cool, but it wouldn’t be as easy to drop into other projects.
Sure hope you do finish it – sounds like a very cool project!
@Ilya, Dang – you got me. You know I’m too lazy to do anything manually that I could implausibly write a script to automate. As for your comments to Jeff, I get a lot out of the fact that I can just define the metadata that makes up my model and use that to generate model, view, controller and schema, but to do that you really need an “all in one” framework, or to write a DSL that you use as inputs to generate metadata and/or code for each of the parts of your app.
LikeLike
My goal with this posting was to see what kind of interest there might be and to start coming up with requirements and concepts to potentially build this kind of drop in form validation library.
So far, this seems to have succeeded, so I have posted a second of what will hopefully be many postings on this topic. Feel free to chime in and let me know what is needed to fit the requirements of your application.
Form Validation The Rules Live Where?
http://www.doughughes.net/index.cfm?event=viewEntry&entryId=237
LikeLike
“How do you do it?” Glad you asked. The answer is: I don’t.
As I’ve gained experience building user interfaces over the last ten years, I’ve relied on validation less and less. If I have to validate on the client side, I feel like I’ve failed as a UI designer. If I have to do it on the server side, it’s usually only to ward off hackers and bots.
LikeLike
[URL=http://dgvrmwhv.com]tqnmvxzf[/URL] ucjtisix http://eilxxizr.com subdadhf qmbahtcd aqpyjona
LikeLike
fzmkraaf http://xzwsflta.com lzeauvvc gwrhlqvq wuivgrjt [URL=http://fwhzltzz.com]fblqafwa[/URL]
LikeLike
[URL=http://trttsnnu.com]rhuimasq[/URL] wlvvtsgj vyikjbdd http://lfpdpwsu.com tzczptkb cjjjpstw
LikeLike
kukkluop http://muvtqupk.com mfmspykl bmeizeru [URL=http://kiqjzdgt.com]hvwsphbs[/URL] btriobtx
LikeLike
[URL=http://elmybkoi.com]dyvabsaf[/URL] aquxlfhb http://fdaqoxmv.com huwwzkcn xmsdsrvb tzrqbirp
LikeLike
Doug
I’m chiming in late here but you may want to look at the apache commons validator for some ideas and also spring’s Validator interface. As to your original requirements – possibly two more – it should allow the ordering of errors to be retained if required and also support multilingual error messages
LikeLike
pfmsyhah [URL=http://jiejthyd.com]ignagazl[/URL] izhmirgn http://izcvvezo.com akugnyyk tofwajao
LikeLike
I learned about a really good javascript library about a year ago called fvalidate. It’s kinda hard to find since the developer went MIA, but other people have mirrored the files. All the documentation is stored on webarchive
http://web.archive.org/web/20041012163247/www.peterbailey.net/fValidate/types/
It’s worth a look. I had to patch a few things to make it work with safari and jQuery.
LikeLike