MSM Resilient Features

As users become more and more dependent on computerized applications, the impact of system failures significantly increases. In the worst case, a failure can severely damage an organization's ability to operate for prolonged periods of time while databases are validated, repaired, or restored. To solve this problem, system administrators look for ways to construct systems that minimize these disruptions in service.

The goal of these systems is to be available continuously to the users (i.e., non-stop operation during required access periods) or for the system to be quickly recoverable in the event of a temporary or permanent failure. Systems designed and configured with these requirements are referred to as resilient systems. Within MSM, a series of features assists the system administrator in implementing resilient systems.

These features can be used within a single computer system to improve the availability of the system or across multiple computer systems connected by a local area network to provide continuous system availability.

Bullet-Proof Databases
A complex network of pointers is used to link blocks together; this allows rapid access to information and dynamic expansion and contraction of space. Performance is enhanced through use of an in-memory buffer cache. Consequently, if a system failure occurs, the possibility for corruption of the database pointer network is high. If corruption occurs, repair can be difficult and time consuming.

The bullet-proofing feature ensures database integrity in the event of a system failure. This feature, called before-image journaling, records images of database blocks as they exist at the instant before any change is made to the block (e.g., SET, KILL, or block split). These images, which are recorded in a separate file, are used by the system to rollback the database to a point in time where structural integrity of the database is known to exist. MSM automatically rolls back the database the first time a volume group is mounted following a system failure.

MSM also provides after-image journaling. When this feature is used, the system automatically rolls forward the database using the information stored in the after-image journals. By using this combination of before-image and after-image journaling after a system failure, database structural integrity is ensured and information is current up to the last entry in the after-image journal file.

Database Integrity
There are three levels of database integrity:

• Structural integrity exists when the database validates cleanly. All the internal links and blocks are intact and valid.
• Temporal integrity exists when the order of database updates is preserved. If, for example, ^A and then ^B are updated, and the change to ^B is present, so is the change to ^A.
• Transactional integrity exists when the database is updated only by a “committed” transaction. Partial or aborted transactions are never committed to the database.

Note: Transactional integrity requires temporal integrity, which in turn requires structural integrity.

Transaction Processing
The MSM transaction processing feature allows the application developer to group multiple database updates (both SETs and KILLs) into a logical group; this is referred to as a transaction. When transaction processing is used within an application, the system ensures that only complete transactions are applied to the database. Other applications never see a partial transaction in the database; only complete transactions exist in the database after a system failure.

At the present time, MSM supports a subset of the full transaction processing standard that has been approved by the M Development Committee. In particular, MSM recognizes the complete syntax of all commands and special variables in the standard, but does not yet support the ability to restart or abort an in-progress transaction. By supporting the syntax of the commands and special variables, application developers can begin to develop or modify software to function with the new transaction processing capabilities.

In its current form, bullet-proof database technology recognizes the concept of transactions. In the event of a system failure, the roll-forward stage of the database recovery only applies to complete database transactions. A transaction is considered to be complete when the SET and KILL operations are bracketed by a TSTART entry and a TCOMMIT entry. Transactions that have a TSTART entry but no corresponding TCOMMIT entry are considered incomplete and are discarded during the recovery. Thus, partial transactions do not appear in the database.

Cross-System Journaling
MSM's cross-system journaling feature allows the system administrator to create a “hot standby” system. Using a local area network and the Distributed Data Processing (DDP) feature of the MSM-NET product, database updates to the primary system are logically mirrored to the secondary system. By applying these updates continuously, the databases remain virtually identical. Then, if the primary system fails, operation can be switched to the standby system.

Cross-system journaling relies on the after-image journaling feature of MSM. With after-image journaling, a chronological record of all SET and KILL operations is maintained in a separate auxiliary file. This file then becomes input to a real-time dynamic dejournaling daemon that runs on the primary system. This daemon reads the after-image file and applies all of the SET and KILL operations in the proper order to the secondary system through MSM-NET. If MSM-NET is not an available in a particular system, then dejournaling can be performed using the Network File System (NFS) capabilities available on most platforms.

In addition to the obvious benefit of having a “hot standby” system, the secondary system can also be used to perform pseudo online backups. This type of backup is typically used in a PC environment because MS-DOS is not a multi-user operating system. By suspending cross-system journaling, the secondary system can be shutdown and the appropriate DOS backup utility can be invoked. Upon completion, cross-system journaling can be restarted; the secondary system is then automatically updated by the primary system to reflect the changes made to the database during the backup operation.

Online Backups
Using the online backup feature, system administrators can backup MSM databases while database updates are taking place. To accommodate the various backup requirements that may exist at different sites, MSM provides three forms of online backup:

• Full Backup - Used to backup the entire database to an external backup device.
• Serial Incremental Backup - Used to backup database blocks that have been modified since the last full or incremental backup.
• Cumulative Incremental Backup - Used to backup database blocks that have been modified since the last full backup.

An online backup copies the MSM database to external media using utilities supplied by the host operating system. Because database updates are permitted during backup operations, MSM writes each modified block to the database and to an auxiliary spill file. The last step of an online backup appends the spill file, which contains a copy of each block modified during the backup, to the end of the external backup media. Together with the database file, this provides a complete image of the database.

Summary of Features

Platform Portability
All resilient system features are available for the MSM-Server for DOS, MSM-Server for UNIX, and MSM-Server for Windows NT platforms.

Bullet-Proof Databases
Using a combination of before-image and after-image journals, the integrity of the database pointer structure is maintained even after a system failure has occurred.

Transaction Processing
Used in conjunction with bullet-proof databases, transaction processing allows the logical integrity of data within an application to be maintained even after a system failure has occurred.

Cross-System Journaling
Database updates can be logically mirrored to a secondary system, providing a “hot standby” in the event of a system failure. When used in conjunction with MSM-LAT, time for switching to the hot standby is substantially reduced.

Online Backups
Full or incremental volume group backups can be performed while users are still active on the system and database updates are taking place.

Non-Stop Operation
Systems can be configured for 24-hour, non-stop operation in mission-critical applications.

Low Overhead
The overhead associated with MSM’s resilient system features is extremely low. There is no perceptible impact on users when these features are used.

Summary
MSM provides system administrators with the ability to design and implement mission-critical systems that must run 24 hours a day. For example, with the MSM resilient system features, it is no longer necessary to shutdown the system to perform a database backup to external media. In addition, in the event of a system failure, the time required to recover from the outage is kept to an absolute minimum because of the bullet-proof database feature. With MSM’s automatic recovery, operator intervention during the system restart is also minimized.

In critical applications where non-stop operation is required, a “hot standby” system can be used to provide full hardware redundancy and continuous availability. Coupled with the transaction processing capabilities, application-level data is consistent and accurate up to the point of the failure. Resilient systems can be configured using MSM-Server for DOS, MSM-Server for UNIX, and MSM-Server for Windows NT systems, or a combination of these systems. Expense is also kept to a minimum by using low-cost, readily available, off-the-shelf components to configure these high-availability systems.

Return to Main MSM Index

Home | M Technologies | Support | Company | Contact

MOVE TO CACHÉ

© Copyright 1996-2000 InterSystems Corporation. All Rights Reserved.
email: wwwadmin@intersys.com