Data Integrity

6.2 Data Verification Methods

Data integrity refers to the accuracy and consistency of data throughout its lifecycle. Maintaining data integrity is crucial for reliable information processing and decision-making. This section describes and explains methods used to verify data during data entry and data transfer.

Data Verification During Data Entry

Data entry is a common point where errors can occur. Implementing verification methods at this stage helps to minimize these errors.

• Data Type Validation: Ensuring that the data entered matches the expected data type (e.g., a number field only accepts numbers, a date field accepts a valid date format).
• Range Checks: Verifying that numerical data falls within a predefined acceptable range (e.g., age must be between 0 and 120).
• Format Checks: Confirming that data adheres to a specific format (e.g., email address must contain an '@' symbol and a valid domain).
• Consistency Checks: Ensuring that related data fields are consistent with each other (e.g., if a customer's country is 'USA', their postal code should follow the US format).
• Lookup Tables: Comparing entered data against a pre-existing list of valid values (e.g., a dropdown list for selecting product categories).
• Required Field Checks: Ensuring that mandatory fields are not left blank.
• Checksums/Hashes: Calculating a checksum or hash value of the entered data and comparing it to a stored value to detect accidental errors during entry.

Data Verification During Data Transfer

Data can be corrupted during transmission between different systems or locations. Verification methods are essential to detect and potentially correct these errors.

Method	Description	Advantages	Disadvantages
Checksums	A simple mathematical calculation performed on the data before transmission. A checksum value is sent along with the data. The receiver performs the same calculation and compares the calculated checksum with the received checksum.	Easy to implement, relatively fast.	Can detect many common errors, but may not detect all types of errors.
Cyclic Redundancy Checks (CRC)	A more sophisticated checksum algorithm that provides a higher level of error detection. It uses polynomial division to generate a checksum.	Excellent error detection capabilities, widely used in data transmission.	More complex to implement than simple checksums.
Parity Bits	An extra bit added to a data unit to indicate the parity (even or odd) of the number of 1s in the data.	Simple to implement, can detect single-bit errors.	Limited error detection capability.
Error Correction Codes (ECC)	More advanced techniques that can not only detect errors but also correct them. Examples include Hamming codes and Reed-Solomon codes.	Can correct errors, provides high reliability.	More complex to implement, can be computationally expensive.

The choice of data verification method depends on the criticality of the data, the transmission medium, and the available resources.

Error Handling

When errors are detected, appropriate error handling mechanisms should be in place. This may involve:

• Prompting the user to re-enter the data.
• Logging the error for later analysis.
• Rejecting the invalid data.
• Attempting to correct the error (if possible).