Machine Epsilon in Python

In this case, you actually don't want np.finfo. What you want is np.spacing, which calculates the distance between the input and the next larger number that can be exactly represented.

Essentially, np.spacing calculates "eps" for any given number. It uses the number's datatype (native python floats are 64-bit floats), so a np.float32 or np.float16 will give a different answer than a 64-bit float.

For example:

import numpy as np

print 'Float64, 1.0 -->', np.spacing(1.0)
print 'Float64, 1e12 -->', np.spacing(1e12)
print 'Float64, 1e-12 -->', np.spacing(1e-12)
print ''
print 'Float32, 1.0 -->', np.spacing(np.float32(1.0))
print 'Float32, 1e12 -->', np.spacing(np.float32(1e12))
print 'Float32, 1e-12 -->', np.spacing(np.float32(1e-12))

Which yields:

Float64, 1.0 --> 2.22044604925e-16
Float64, 1e12 --> 0.0001220703125
Float64, 1e-12 --> 2.01948391737e-28

Float32, 1.0 --> 1.19209e-07
Float32, 1e12 --> 65536.0
Float32, 1e-12 --> 1.0842e-19

Floating point numbers have a certain precision, to a few decimal places in scientific notation. The larger the number, the larger the least significant digit in that representation, and thus the larger the "epsilon" that could contribute to that number.

Thus, the epsilon is relative to the number it is added to, which is in fact stated in the documentation you cited: "... such that 1.0 + eps != 1.0". If the "reference" number is smaller by, e.g. one order of magnitude, then eps is smaller, too.

If that was not the case, you could not calculate at all with numbers smaller than eps (2.2e-16 in my case).