Monday, December 11, 2023

Avoiding catastrophic cancellation

Catastrophic cancellation occurs when subtracting two nearly equal numbers, leading to a significant loss of precision due to limited precision of floating-point numbers in a computer. Below are some examples and their better versions (reference):
log(a)−log(b) = log(a/b)

Computing cos(x)−1 for small values of x --> cos(x) ≈ 1−(x^2)/2 

No comments:

Post a Comment