We’ve been in the HCL/IBM/Lotus Notes & Domino application development world for quite a while. We launched our first product at Lotusphere 96, more than 23 years ago. In all that time, we’ve spoken to a lot of Notes developers, development managers, project managers, quality managers and people with many other job titles. Those conversations are usually the starting points for our products. “How can we do X?” “We just want to do Y, but it must also do Z.” Many times, a product idea will come out of those conversations.
In 23 years though, there is one question that we’ve never fully been able to provide an answer to (in terms of a new product idea, at least). It’s a question we’ve heard over and over from people all over the world. The question is: “How can we automate testing of our Notes applications?”
That question, of course, is a lot easier to ask than it is to answer. The word “testing” can cover a multitude of different areas. Installation or deployment testing, functional testing, regression testing, compatibility testing, performance testing, usability testing, security testing, localization testing, accessibility testing, acceptance testing… all of these areas may need to be considered.
Layered on top of that, of course, is the fact that the native Notes development environment is not your typical IDE. Modern, agile-centric concepts such as unit testing and integration testing don’t really fit very well with the Notes way of doing things. In any case, I’m going to go out on a limb and suggest that most traditional Notes development these days is concerned with enhancing and maintaining existing applications, as opposed to building new apps from scratch. And retrofitting a test-driven development approach to a 20 year-old application may not be the best use of anyone’s time.
Sometimes I get the feeling that many people just put Notes application testing into the category called “too hard to think about right now.” It may be hard, but that doesn’t mean that there is nothing you can do. And so, I think the best advice when it comes to testing your Notes applications is: do what you can. Any testing is better than no testing, and, if there are things that just can’t be tested (or it doesn’t make economic sense to test them), then don’t lose sleep over it. Just do what you can.
When it comes to tooling, none of the mainstream testing tools companies explicitly support Notes (not even Rational Functional Tester). Allegedly there was once a product that was a full featured testing solution for Notes from a company called Smart Toucan, but it seems to have disappeared and I don’t know what happened to it.
Here at Teamstudio we do have tools that will cover some important areas of testing, and if you’re making new releases of Notes apps, you should consider using them.
Teamstudio Validator, for example, runs a static analysis of your Notes application and looks for a number of potential errors, before a user ever gets their grubby paws on it. The errors that Validator will find include:
Could not validate document with form
Validator examines every field on every form and attempts to calculate the default value and input translation/validation formula against the associated documents. If an error is present, a user would not be able to open or edit and save that document.
Broken links
Validator evaluates and checks any doclinks or URLs it finds in the database to avoid a user encountering a broken link.
Entry not found in index
Surely the most common error message ever displayed to a Notes user! This is usually caused by a reference being made to a document that doesn’t exist any more. Validator will find the reference and flag it up before a user ever sees it.
Field inconsistencies
One of the great things about Notes is that it’s very easy to change things after an application has been deployed. But there are potential unwanted side effects. For example, changing a field type after a database has been in use can cause problems unless you run an agent to update the existing data in that field.
Keyword fields with incorrect values
Related to field inconsistencies, this is where the values available in a keyword list have changed, but the existing data could contain values that are not now in the keyword list.
Missing dependencies
Domino Designer does not check dependencies in the design. So, for example, you could delete a subform or shared field from the design that is used on a form without triggering an error. The error message would only show up to the user at runtime.
In addition to those lovely things, Validator will also find save and replication conflicts, orphaned agent data notes and other things that could cause problems for your users down the road.
Just to be clear, Validator is certainly not a full-featured testing tool, in the traditional sense. But it is something that could form part of a suite of quality checks that are run against an application design before it’s rolled out to users. There are other things you can do too. For example, little known fact: Teamstudio Analyzer comes with a pack of 32 best practice audit filters that spot common errors in your code. Check it out here.
So just because there’s no one simple, easy solution to testing your Notes applications, that doesn’t mean it’s OK to do nothing at all. The most important thing is that you have a plan. Automate the bits you can automate, and then add some good old fashioned human testing. Consider tooling to cover the areas that can be automated. And use checklists to make sure that you don’t forget to test anything. Like many things in app development, consistency in testing is key.
To learn more about Teamstudio Validator, click below.