13.3 Floating-point Numbers: Representation and Rounding Errors

Introduction

Floating-point numbers are a standard way computers represent real numbers. They are particularly useful for representing numbers with a wide range of magnitudes, from very small to very large. However, the binary representation of real numbers can lead to rounding errors, which are a fundamental limitation of floating-point arithmetic.

Binary Representation of Real Numbers

In the decimal system, we use powers of 10 to represent numbers. For example, 123.45 can be written as $1 \times 10^2 + 2 \times 10^1 + 3 \times 10^0 + 4 \times 10^{-1} + 5 \times 10^{-2}$. The binary system uses powers of 2. A floating-point number is typically represented in a format similar to scientific notation: $sign \times mantissa \times base^exponent$.

IEEE 754 Standard

The most common standard for floating-point representation is IEEE 754. This standard defines how floating-point numbers are stored in memory. A typical IEEE 754 single-precision (32-bit) floating-point number is divided into three parts:

• Sign bit (1 bit): Indicates whether the number is positive or negative (0 for positive, 1 for negative).
• Exponent (8 bits): Represents the power of 2. It's stored with a bias to allow for both positive and negative exponents.
• Mantissa (23 bits): Represents the significant digits of the number. It's normalized, meaning it's in the form 1.xxxxx x $2^exponent$.

Table: IEEE 754 Single-Precision Format

Field	Bits	Description
Sign Bit	1	0 for positive, 1 for negative
Exponent	8	Bias = 127. Actual exponent = stored exponent - 127
Mantissa	23	Represents the significant digits (normalized to 1.xxxxx x $2^exponent$)

Rounding Errors

Because real numbers cannot be perfectly represented in a finite number of bits, rounding errors occur. This happens when a real number cannot be exactly represented in the mantissa. The computer must round the number to fit within the available bits.

Consider the decimal number 0.1. In binary, this is an infinitely repeating fraction: 0.1 = 0.00011001100110011... Since we have a limited number of bits in the mantissa, the computer must round this value. The result is not exactly 0.1, but an approximation.

Example of Rounding Error

Let's consider a simple example:

1. Start with the decimal number 0.1
2. Convert 0.1 to binary: $0.1_{10} = 0.00011001100110011..._{2}$
3. With only 23 bits for the mantissa, we'll need to truncate this binary representation.
4. The result will be an approximation of 0.1, not the exact value.

Consequences of Rounding Errors

Rounding errors can accumulate over multiple calculations, leading to significant inaccuracies. This is especially problematic in scientific computing and financial applications where precision is critical.

For example, if we add two numbers that are very close together, the rounding error can cause a significant loss of precision. This is because the difference between the two numbers might be smaller than the smallest representable difference by the floating-point format.

Mitigation Techniques

While rounding errors cannot be completely eliminated, several techniques can help mitigate their effects:

• Careful Algorithm Design: Choose algorithms that are less susceptible to rounding errors.
• Interval Arithmetic: Represent numbers as intervals, which provide a range of possible values.
• Error Analysis: Analyze the potential for rounding errors in a particular calculation.

Conclusion

Floating-point numbers provide a convenient way to represent real numbers in computers, but they are subject to rounding errors due to the finite precision of their binary representation. Understanding these errors and their potential consequences is crucial for writing accurate and reliable numerical software.