What Causes Data Corruption?

The JADE database engine depends on disk writes being atomic; that is, either all or none of a block is transferred to the disk media. This fundamental requirement can be jeopardized by hardware factors, particularly disk controller operational characteristics.

The disk controller may not complete in-progress writes when power loss occurs or the RST bus reset signal is raised. This may result in data corruption due to partially written and therefore incomplete, or torn, blocks.

Before JADE release 5.2, torn blocks were not detected during recovery and corrupted blocks remained undetected until used at some later time. As the JADE 5.2 release recovery algorithms introduced a high level of integrity checking, torn blocks affecting structural integrity can be detected and exhibit as bTree index block access and update errors. Torn blocks affecting object instances, however, can go undetected until used at some later time. In either case, the error handling and reporting mechanisms simply reflect the error state of the operation and the blocks that are parties to it, and can in no way determine that the root cause of the problem is a torn block on disk.

It has become evident that a significant number of JADE sites are not adhering to the hardware recommendations outlined in the Environmental Considerations for Deploying JADE white paper (available on the JADE Web site at https://www.jadeworld.com/jade-platform/developer-centre/documentation/white-papers). JADE systems deployed on servers with storage hardware that does not satisfy at least the minimum requirements documented under "Storage Requirements" in the white paper are susceptible to this form of disk corruption under system reset or power loss conditions.

Furthermore, we have determined that those requirements, while necessary, are insufficient to protect against incomplete disk write operations (that is, a "write through" caching policy by itself does not guarantee that write operations are atomic). Torn blocks are less likely to occur on disk drives attached to a controller that implements robust failure management mechanisms.

The controller must:

• Intercept and manage the RST bus reset signal and have on-board battery backup and mirrored or ECC memory

• Perform operational check-pointing so that when power is restored it can restart and complete the write operations that are in progress

Only high-end storage systems specifically designed to host mission-critical databases are likely to satisfy all of the requirements that guarantee write atomicity.

Despite the availability of robust storage subsystems, we have also observed that corruption can be injected by the I/O data path between main memory and the disk subsystem. When JADE issues a write I/O to disk, there are layers of software and hardware (for example, operating system, file system, I/O driver, data-bus, interface card, and cable) that handle the operation and data movement to the disk controller. Software bugs or faulty components in this I/O data path can introduce data corruption. Research has shown that this is an industry‑wide issue with commodity hardware platforms that affects all database management systems.

For these reasons, we have implemented a software‑level detection scheme that enables block corruption due to incomplete disk writes, bugs in data path software layers, or faulty data path components to be detected at the earliest opportunity.