Many of our customers are engaged in projects to archive old Notes and Domino databases. And many of those customers have many, many databases that they need to archive. In some cases, it runs into the tens of thousands. Also, some of those databases can be very large. Something that sounds quite easy to begin with (“create HTML archives of all our old Lotus Notes databases”) can turn into a massive project very quickly.
If you’re engaged in such a project, you may be familiar with Teamstudio Export. Export allows you to easily create HTML archives from your elderly Notes and Domino databases. The resulting archives include all the visible views from the original application, support full text searching, and even render forms in the browser as if they were being viewed in a Notes client. Once an archive has been created, no further licensing costs are required; a web browser is all you need to access your read-only applications in perpetuity.
However, if you do have a large number of databases, and especially if those databases contain large quantities of data, you might be interested in some tips for how to optimize the archiving process in Export itself. Which is the purpose of this post! Here are our top five tips for making sure your Export process runs as fast and smoothly as possible.
1. Proximity and Connectivity to Domino Servers
The Export process itself runs on a Notes client workstation and accesses databases from a Domino server. If the workstation you are running Export on is not connected to the same physical network as the Domino server, there can be a performance degradation based on the network connectivity to the Domino server. If at all possible, run Export on the same network.
Of course, in our current state of COVID-19 induced lockdown, that may not be possible for many people. If you are currently working from home, you might want to consider having your organization set up a dedicated Virtual Machine for you to connect to for creating the archives.
If your organization has set up a Domino Passthru server to connect to your other Domino servers, then you may want to adjust this so that you are connecting directly to the Domino servers that you are archiving from. You can do this in your local Notes Personal Address Book (names.nsf) by creating Connection documents for each of your Domino servers and indicating either the Internet Host name or the Domino server’s IP address as the "Destination Server Address".
2. Health of Domino Databases
If the Domino databases that you are archiving have not been used in a long time (45+ days) then the view indexes will have been discarded by the Domino server to save space. During the archiving process, Export accesses all the views in the Domino database, which of course will recreate any discarded view indexes. This can be a very lengthy process to complete, especially for large databases that have a lot of views, which adds significantly to the amount of time needed to archive the database.
By performing some maintenance tasks on the Domino databases that you intend to archive before the archiving process starts, you will see a significant reduction in the amount of time the processes take. We recommend the following Domino administration tasks be run on the Domino databases to be archived:
Fixup
Compact
Updall
3. Available Disk Space on Workstation
If the Notes client workstation hosting Teamstudio Export is running low on disk space, the archiving processes will run slower. Export creates temporary files while archiving a database that may be two or three times the size of the database being archived.
If there are multiple hard drives on the computer, there is an option in Teamstudio Export's configuration settings to use an alternate hard drive for creating the temporary files that are created during the XML Archive creation process. To access the configuration settings menu, choose Configuration from the File menu. Export uses the system temporary folder for these files by default, which is on the C: drive for most users. If you need to use a different location, uncheck Use system folder for temporary files and select the desired folder.
4. Location of Archive Creation
If you have configured Teamstudio Export to create the archive files directly onto a file server, then this has a similar performance impact as outlined in item #1 above, especially if the file server is in a remote location. If this is the case, then we recommend first creating the archives onto the local hard drive and then manually moving the files onto the file server after they have been created.
5. Computer Performance
If the computer where Teamstudio Export is being used is not dedicated to the archiving processes (i.e. if it is being used for other programs and tasks) then the amount of available memory and network bandwidth will be shared across all running programs and tasks. We recommend using a dedicated computer or virtual machine for the archiving process, which means that the operating system will allocate more resources to Teamstudio Export and the archive process will go much faster.
These are the main factors that will influence the speed of the archive process. Some additional aspects of the archive creation that can slow down the process are:
The number of file attachments contained within the Domino database. Export retains the full fidelity of all file attachments. It does not compress any attachments, which ensures that the attachments will be exactly the same as they were in the Domino database, whereas all other aspects of the archives are compressed to save space.
The complexity of the forms that were used to create the Notes documents can also affect performance. When Export creates the XML Archive, it executes all @Formula language that is contained in the form to ensure that the "HTML Preview" functionality will contain the best representation of the Notes document. If the form is very complex, it will take longer to execute the @Formula language calculations.
By way of an example, here is some data from our own testing. With these recommendations applied, we were able to archive an 11.7GB Domino database that contains just under 15,000 documents in just under 1 hour, and a 5.7GB Domino database with just over 92,000 documents in 23 minutes. The specifications of the workstation we were using for these tests is: Microsoft Windows 10 64bit, Core i7-7700 CPU (2 cores), 8GB RAM, 150GB SSD Hard Drive, 1GB network connection to Domino server.
If you have questions about any of this, or would just like to share your own experiences of performance of Teamstudio Export, click below to start a conversation. We love to chat!