Totally weird behaviour with doubles
I've never seen anything like this. Compare 271.66 with a double that adds up to 271.66, and they resolve as not equal. Try it for yourself:
Dim dbl1 As Double = 271.66
Dim dbl2 As Double = 0
dbl2 += 200
dbl2 += 71.66
If dbl1 <> dbl2 Then
System.Windows.Forms.MessageBox.Show("Not equal: dbl1= " & dbl1 & ": dbl2 = " & dbl2)
Else
System.Windows.Forms.MessageBox.Show("They're equal")
End If
I've only seen this happen with 271.66. Anybody else come across this? You can resolve it using Math.Round(dbl2, 2), but it's strange that it doesn't require that with other numbers.
[670 byte] By [
johnbeat] at [2007-11-11 7:13:16]
# 1 Re: Totally weird behaviour with doubles
The problem is that numbers must be represented internally as binary, and not all values can be expressed precisely as binary. See http://support.microsoft.com/kb/q42980/ for more information.
# 2 Re: Totally weird behaviour with doubles
As Phil's pointed out, not all numbers can be represented correctly (basically, all decimals that are the sum of powers of two, like 1/3) And as you found out, the only solution is to round them up.
Marco
mstraf at 2007-11-11 21:50:22 >
# 3 Re: Totally weird behaviour with doubles
I spend a major part of my time dealing with floating point arithmatic, it can be a major pain! Here's a funtion that I use for Floating point equality:
'Epsilon-based comparison for two floating points.
Public Shared Function Equality(ByVal myFirst As Double, ByVal mySecond As Double, ByVal myPrecision As Double) As Boolean
If Abs(myFirst - mySecond) < myPrecision Then
Return True
Else
Return False
End If
End Function
I typically use 0.00001 for the precision
