1. Fixed-point or Floating-point

Fixed or Floating-point

Fixed-point number

A fixed point number has its decimal point between two specific bits and cannot be moved.

Link to original

Scientific notation

Scientific notation is used to represent large or small decimal numbers. These have two parts, the $\color{red}\text{significand}$ (mantissa) and the $\color{blue}\text{exponent}$.

• $1.45\times 10^5=145,000$
• $4.2159\times 10^5=0.000000042159$

Link to original

We can similarly represent binary numbers in this form:

• $1.0101\times 2^5=101010_2=42_{10}$

Link to original

2. IEEE-754 format

IEEE-754 format

IEEE-754 format

The IEEE-754 format is a standard specified by the **IEEE** for representing floating-point binary numbers. The two versions of the format that are most common are **single precision** (32-bits) and **double precision** (64-bits).

Link to original

IEEE-754 formats specify:

• Number of bits for the significand and exponent.
• A bias for the exponent.

  Exponent bias

  The bias is an offset which is always applied to the exponent of a floating point number, $127$ for single precision and $1023$ for double precision.

  We use a bias instead of two’s complement as it allows for the numeric ordering to be the same as the lexicographic ordering which allows fast size comparisons of numbers.

  Example: convert single-precision exponent $0111010_2$:

  $$0111010_2=122_{10}$$

  Subtract the bias to find $122_{10}-127_{10}=-5_{10}$.

  Link to original

• A normalised format.

  Normalisation

  When the significand of a floating point number is normalised, there is an implied leading $1$ before the stored significand.

  $$01011000[..]\to 1.01011000[..]$$

  By normalising floating point values, we: ?

  • Gain an extra bit of precision because of the implied leading $1$.
  • Ensure that there is a unique representation for values. In non-normalised form, $2.5$ can be represented as $10.1\times 2^0$, $1.01\times 2^1$ and so on.

  Link to original

Bit lengths for IEEE-754 format

Each IEEE-754 format uses a different number of bits for each part of the number:

Format	Sign	Exponent	Significand	Total
single	1	8	23	32
double	1	11	52	64

Link to original

The sign bit behaves the same way as in signed-magntiude. The exponent and significand are not just unsigned.

title: Example 1
Take $11000001001011000000000000000000$.
We first break it down to individual components: $\color{green}1 \color{blue}1000010\color{red}01011$.
- $\color{green}1$ is the $\color{green}\text{sign bit}$ which tells us it is negative.
- $\color{blue}1000010$ is the $\color{blue}\text{exponent}$.
  This value is $130_{10} - 127_{10} (\text{bias}) = 3_{10}$.
- $\color{red}01011$ is the $\color{red}\text{significand}$.
  Significand is $1.01011$ with the implied leading $1$.
  
So, our final value is $- 1.01011 \times 2^3 = - 1010.11_2 \rightarrow - 10.75_{10}$.

title: Example 2
Take $-133.25_{10}$.
Convert to binary and normalise:
 
$$
	\begin{aligned}
		133.25 &\rightarrow 1000101.01 \times 2^0 \\
		&= 1.000010101 \times 2^7 \\
		&= {\color{red}00001010100000000000000_2} \\
		\\
		2^7 &\rightarrow 7 + 127 \\
		&= 137 \\
		&= {\color{blue}10000110_2}
	\end{aligned}
$$
 
Sign is negative so $\color{green}1$.
Hence final value is ${\color{green}1}{\color{blue}10000110}{\color{red}00001010100000000000000}$.

Special values

Using normalised form, it is not possible to represent $0$, so there is a special value specified which we use for $0$.

Decimal	IEEE-754 single precision
$\pm 0$	$0/10000000000000000000000000000000$
$\pm \infty$	$0/11111111100000000000000000000000$
$\text{NaN}$	$0/111111111\text{(any non-zero significand)}$

Converter

Link to original

3. Arithmetic with IEEE-754 format

Arithmetic with IEEE-754 format

Addition

To add two floating-point numbers:

• Adjust position of binary points so significands have the same exponent.
• If numbers have the same sign, add significands. Otherwise subtract smaller from larger.
• If required, normalise significand at the end.
• Select correct sign depending on values.

Subtraction

Use the relationship $a-b=a+(-b)$.

Multiplication

To multiply two floating-point numbers:

• Add the exponents.
• Multiply the significands.
• If required, normalise significand at the end.
• Select correct sign depending on values.

Division

Same as mult, but in step 1 subtract exponents and in set 2 divide significands.

Common Errors

• Overflow: magnitude is too large to store the format
• Underflow: magnitude is too small to store in the format Can often be confused with overflow in negative direction.
• Catastrophic cancellation: lack of significant figures to represent the result of colliding large and small numbers
• Lack of exact representations for common Base-10 numbers: $0.1+0.2=0.300000000000004$

Link to original

4. Range and precision

Range

Range is the difference between the maximum and minimum value re-presentable in a given format. Although it does not mean all values are re-presentable within that range.

The range of double precision numbers is:

• $-1\times 10^{308}\text{ to }-1\times 10^{-308}$ and $1\times 10^{308}\text{ to }1\times 10^{-308}$

Precision

Precision refers to how much information a representation gives us of a value.

• $3.14$ gives us $2$ digits of precision right of the decimal point.
• $3.142$ gives us $3$ digits of precision right of the decimal point.

More precision doesn’t mean more accuracy, $3.14$ is a more accurate representation of $\pi$ than $3.149$ even though it is less precise.

Link to original

5. Accuracy

Accuracy

Accuracy refers to how close a representation of a value is to the true value.

Relative error is a measure of accuracy of a representation. Relative error = $\frac{|T-R|}{T}\times 100$, where $T$ is the true value and $R$ is the value that is represented.

For example, we know there is no exact representation of $0.3_{10}$ in binary:

• In 5-bit fixed-point binary, with point between 4-3, the most accurate is $0.0100_2=0.25_{10}$. The accuracy is $\frac{|0.3-0.25|}{0.3}\times 100=16.67\%$.

Link to original