IEEE 754 floating point

Why it exists

Fixed-point numbers have a fixed resolution, so they cannot hold both a galaxy's mass and an electron's. Scientific notation fixes this by separating the digits (the significand) from the scale (the exponent): $-6.022 \times 10^{23}$. IEEE 754 is that idea in binary, standardised so that every processor stores, rounds and compares floating-point numbers in the same way. In GATE the format is tested in three ways: encode or decode a number by hand, reason about the largest, smallest and most precise values (including the special cases), and spot the surprises that follow from rounding.

Prerequisites

You need binary fractions (number systems) and the idea of a biased (excess) code from signed number representation.

Precise definitions

A binary floating-point word has three fields, from the most significant bit down: sign $s$ (1 bit), biased exponent $e$ ($E$ bits) and fraction $f$ ($M$ bits).

Let $e$ be the unsigned value of the exponent field, $f$ the unsigned value of the fraction field read as an integer, and write the fraction as the binary fraction $f / 2^{M}$.

Normal numbers ($0 < e < 2^E - 1$):

$(-1)^s \times \left(1 + \frac{f}{2^M}\right) \times 2^{\,e - \text{bias}}.$

The leading 1 is not stored. It is the hidden bit (implicit leading 1), so the significand has $M + 1$ bits of precision (24 for single, 53 for double). The exponent field $e$ is biased (excess code): the true exponent is $e - \text{bias}$.

Zero ($e = 0$, $f = 0$): $+0$ and $-0$, two patterns that compare equal.

Denormal (subnormal) numbers ($e = 0$, $f \ne 0$):

$(-1)^s \times \frac{f}{2^M} \times 2^{\,1 - \text{bias}}.$

There is no hidden bit and the exponent is fixed at $1 - \text{bias}$ ($-126$ for single, $-1022$ for double). They fill the gap between zero and the smallest normal number (gradual underflow).

Infinity ($e = 2^E - 1$, $f = 0$): $\pm\infty$.

NaN, not a number ($e = 2^E - 1$, $f \ne 0$). There are $2(2^M - 1)$ NaN patterns. A NaN is the result of an invalid operation such as $0/0$ or $\infty - \infty$.

Key values.

Spacing. Within $[2^k, 2^{k+1})$ the numbers are evenly spaced $2^{k - M}$ apart. So the absolute gap doubles every time the exponent grows by one, while the relative gap stays between $2^{-M-1}$ and $2^{-M}$. All integers up to $2^{24}$ (single) or $2^{53}$ (double) are exact.

Rounding. The default mode is round to nearest, ties to even: the result is the closest representable number, and when exactly halfway, the one whose last fraction bit is 0. Other modes are round toward zero, toward $+\infty$ and toward $-\infty$.

Comparison. For non-negative finite numbers, comparing the 32-bit patterns as unsigned integers gives the same order as comparing the numbers (this is the reason the exponent is biased and placed before the fraction). $+0 = -0$. Any comparison with a NaN is false except "not equal", so $x \ne x$ is true for a NaN.

Exceptions. Invalid operation (result NaN), division by zero (result $\pm\infty$), overflow (result $\pm\infty$ or the largest finite number, depending on rounding mode), underflow, and inexact (the rounded result differs from the exact one).

Encoding a number by hand.

1. Convert the absolute value to binary, including the fraction.
2. Normalise to $1.xxx \times 2^{k}$ by shifting the point; $k$ is the true exponent.
3. $e = k + \text{bias}$, written in $E$ bits. The sign bit is 1 for a negative number.
4. The fraction field is the bits after the leading 1, padded with zeros on the right to $M$ bits. If there are more than $M$ bits, round (ties to even).

Decoding. Split the pattern into fields, check $e$ for the special cases, then apply the formula.

How it works

Step through eight real patterns. For each, the sign, exponent and fraction fields are shown as bit cells, with the fields' integer values and the whole word. The words are packed by the platform's own IEEE 754 routines, not typed in.

1.0 = 1.0 x 2^0
stored exponent = 0 + 127 = 127 = 0111 1111
fraction = 0 (the leading 1 is implied)

To see how a pattern is built, step through the hand method for $-118.625$ and then the rounding of $0.1$. The last step of the first animation is compared with the platform's own float packing.

-118.625 is negative
sign bit = 1, then work with 118.625

Notice three things as you step: the denormal has an all-zero exponent field, while infinity and NaN have an all-ones one; the fraction of $0.1$ ends in 1101 because the infinite tail 0011... was rounded up; and $2^{24}$ has an all-zero fraction (it is a power of two), yet its neighbour above is $2^{24} + 2$, so $2^{24} + 1$ cannot be stored.

Worked examples

#include <stdio.h>
#include <stdint.h>
#include <string.h>
#include <float.h>
 
static uint32_t bits_of(float x) {
    uint32_t u;
    memcpy(&u, &x, sizeof u);
    return u;
}
 
static float float_of(uint32_t u) {
    float x;
    memcpy(&x, &u, sizeof x);
    return x;
}
 
static void fields(uint32_t u) {
    printf("%08X: sign %u, exponent %u, fraction %u (0x%06X)\n", u, u >> 31, (u >> 23) & 0xFF, u & 0x7FFFFF, u & 0x7FFFFF);
}
 
int main(void) {
    uint32_t pi = 0x40490FDB;
    fields(pi);
    printf("value %.9g\n", float_of(pi));
    printf("-118.625 is %08X\n", bits_of(-118.625f));
    printf("0.1f is %08X = %.27f\n", bits_of(0.1f), (double)0.1f);
    printf("smallest denormal %.9g, FLT_MIN %.9g, FLT_MAX %.9g\n", float_of(1), FLT_MIN, FLT_MAX);
    volatile float big = 16777216.0f;
    float next = big + 1.0f;
    printf("2^24 + 1 == 2^24 in float: %d\n", next == big);
    double s = 0.1 + 0.2;
    printf("0.1 + 0.2 == 0.3 in double: %d, sum %.17g\n", s == 0.3, s);
    return 0;
}

40490FDB: sign 0, exponent 128, fraction 4788187 (0x490FDB)
value 3.14159274
-118.625 is C2ED4000
0.1f is 3DCCCCCD = 0.100000001490116119384765625
smallest denormal 1.40129846e-45, FLT_MIN 1.17549435e-38, FLT_MAX 3.40282347e+38
2^24 + 1 == 2^24 in float: 1
0.1 + 0.2 == 0.3 in double: 0, sum 0.30000000000000004

Cheat box

Traps and GATE patterns

Connections

Floating-point addition, multiplication and the effect of rounding are in floating-point arithmetic. The exponent is an excess code as in signed number representation, and the contrast with a fixed position of the point is in fixed-point arithmetic. In C, float and double use these formats, and (int) conversions truncate toward zero as in C types and operators.

PYQs

Consider the real valued variables X, Y and Z
represented using the IEEE -754 single precision
floating-point format. The binary representations
of X and Y in hexadecimal notation are as
follows:

X: 35C00000
Y: 34A00000

Let Z = X + Y.

Which one of the following is the binary
representation of Z, in hexadecimal notation?

The 32-bit IEEE 754 single precision
representation of a number is 0XC2710000.
The number in decimal representation is
________ (rounded off to two decimal places).

Consider three floating point numbers A, B and C
stored in registers RA, RB and RC, respectively as
per IEEE-754 single precision floating point
format. The 32-bit content stored in these registers
(in hexadecimal form) are as follows.

RA = 0xC1400000     RB = 0x42100000

RC = 0x41400000

Which one of the following is FALSE?

Self-check

Revision card

References