Archiving Lotus Notes databases. It sounds simple enough, but what does the term “archiving” actually mean, what are your options when it comes to archiving, and what are the pros and cons of each approach?
There are a couple of different reasons why you might be thinking about archiving Lotus Notes databases. One is for general cleanup and data hygiene purposes, and the other is because you are retiring old databases that are no longer needed. If your situation falls into the first category, you are most likely committed to the Notes/Domino platform and looking for ways to reduce the size of individual databases.
Archiving for Hygiene
The most common application of this type of archiving is aimed at Lotus Notes mail files. By their nature, mail databases tend to grow over time, as new mail arrives and not every user is diligent about deleting old or unwanted emails. (Even I am guilty of the practice of using my inbox as a filing system - as I write this, I’m looking at what I consider to be a perfectly clean inbox that contains almost 40,000 messages.)
Although HCL Domino is now capable of handling large database sizes and high numbers of documents, performance will inevitably degrade as the number of documents increases, and so it’s a good idea to institute an archiving policy to keep the size of the mail files under control.
The Domino platform allows you to set up archive databases and associated policies so that documents are manually archived by users or automatically archived based on a set of defined criteria. For example, you can set criteria to define which documents will be archived, such as documents that have not been opened for a certain period of time, then you can choose what actions to take with the document that meet those criteria. You could delete the archived documents altogether, or you could truncate them by removing file attachments and reducing the size of the message body to a maximum of 40KB of data.
The details of setting up mail archiving policies are well documented in the HCL Domino admin documentation. As a Domino administrator, you have the options to set automatic archiving policies, leave it to users to set up their own policies, to archive on the server, on the client, and many other options.
Archiving for Migration or Retirement Reasons
The second type of archiving of Lotus Notes databases is in the situation where you may be actively migrating away from the Lotus Notes platform, or perhaps you just have some older Notes databases that have fallen into disuse and are no longer required.
If you are actively migrating away from Lotus Notes, it’s a good idea to keep an archive copy of your old Notes databases, even if you have migrated all the data to a new system. In practice, that seldom happens, however. Migrating an application from Notes to some other system usually offers a good opportunity to clean up old data. You may, for example, decide only to migrate the last N years of data, where N is a small number such as 1 or 2. Especially in that case, but usually in any case, it’s wise to keep a backup archive copy of the old Lotus Notes database as it was at the time of retirement.
Even if you are not migrating away from the Lotus Notes platform, if you’ve had a Notes application infrastructure for any length of time, there are most likely older databases that are no longer used and have reached retirement age. It’s a good idea to scoop up these old, disused databases and archive them somewhere they can easily be accessed in the future.
If you’re archiving databases for either of these reasons, but especially if you are aiming to be able to shut down your Lotus Notes environment in the future, you most likely want to find an archive format that is independent of the Lotus Notes/Domino platform. Ideally, you’re looking for an archive format that is independent of any vendor. By their nature, archives of this type need to be stored safely for long periods of time, but also need to be easily accessible by any user who wants to look up some historical piece of data.
When it comes to choosing a medium for your archives, there’s an almost infinite number of choices available to you. Here are some options to consider:
Retain Notes-format NSF files
Every self respecting list of options starts with the most obvious one, which is essentially “do nothing”. You always have the choice to just squirrel your Notes databases away in their native format. Pros: it’s the simplest option, to start with at least. Also, you won’t have any issues with data fidelity - assuming you are keeping at least one Notes client around to access the data. Cons: You will need to continue to pay a subscription for the Notes client, and potentially Domino servers, if you want to share the archives. You’ll also need to continue to support that infrastructure which, essentially nullifies the benefit of archiving in the first place.
Export to CSV files
You have the option to export data from Notes without using any other tooling, just by exporting to CSV from a view. Pros: this is surely the least expensive option; no additional tooling is needed, and once your data is exported there are no additional run time licenses needed. Cons: this option of course is extremely limited in the types of data that can be exported. Notes will only export the contents of a view, and any rich text or file attachments are lost. Unless you have the scantest of data, this option is seldom useful.
Convert to “Modern” Database Format
There are tools available that will convert your Lotus Notes database to a more “modern” database format. For example, there’s a service offered by LDC Via that will convert Lotus Notes databases to MongoDB format. Pros: because the data is held in a database, it’s possible to recreate the security model and potentially to build new applications that access the data in different ways. Cons: unless you’re archiving simple databases based on one of the standard Notes templates, some customization work will be needed to build an app to provide a user interface for your data. That, coupled with the fact that there will be hosting and admin fees to pay for as long as you want to keep your data accessible, probably makes this the most expensive solution.
Export to HTML/PDF Format
There are also tools that allow conversion of your Lotus Notes databases to some combination of HTML and/or PDF format files. (If you hadn’t guessed already, this is where Teamstudio Export fits in.) We’re biased, of course, but we think this is the best solution. Pros: with this approach, it’s possible to provide near perfect fidelity of data presentation, in other words the data looks almost exactly the way it would do in the Notes client, except it’s in a browser. A web-based app is created that provides read-only access to the archived Lotus Notes database, recreating views and providing a full text search capability. In addition, while there is a cost associated with the tool, once the archives have been created, there are no additional hosting, admin or license fees to be paid. This makes it an extremely cost effective solution. Cons: it’s practically impossible to recreate the Notes security model using this approach, especially if, like Export, there is no requirement for a server of any kind. Our philosophy on this is that you are creating an archive, and by its nature an archive needs to contain all the data that was in the original database.
Export gives you the option of saving the Notes documents in HTML or PDF format. If you choose PDF format, that does make it easy to upload the files to an alternative content management system such as Microsoft SharePoint. However, there are some constraints when converting Lotus Notes databases to PDF format. Essentially you are taking a form that was designed to fit on a computer screen (i.e. with an infinite page length) and shoehorning it into a set of A4 or Letter sized pages. That sometimes means that there are some formatting compromises that need to be made. If you’ve ever tried to print out a Notes document (or a web page, come to that) onto an actual piece of paper, you’ll know what I’m talking about.
Whichever approach you choose for archiving your Lotus Notes databases, inevitably there will be some decisions that you need to take and there may be some compromises. While it’s not perfect for everybody, we feel that the approach we have taken with Export provides the best all round mix of high fidelity archives that are easy to create, simple and inexpensive to distribute to users, and equally easy for users to navigate, search and find the data they are looking for.
If you’re interested in trying out Export, you can download a free trial version that will export the first 10 documents in each view in your database. You don't need a credit card, and you don’t even need to provide your name or email address. Just download the product, run the installer and you’ll be archiving Notes databases in no time. To download the free trial version of Export, click below.