Floating point follies followup

In a response to my previous entry, Cory asks:

…how would you solve the comparison of two values if one is still on the stack and the other has been [stored in a register]? In other words, what would be the proper way to handle this type of situation (work around, etc.)?

A good question, and one that I should have addressed. You can get the compiler to insert explicit CONV.R8 opcodes by using the CDbl() conversion operator. So the code example would look like:

Dim d1, d2 As Double

d1 = Atn(-1)
d2 = Atn(-1)

If CDbl(d1) = CDbl(d2) Then
Console.WriteLine("True")
Else
Console.WriteLine("False")
End If

If you look at the code this compiles to, you should see CONV.R8 opcodes after each local is loaded, and that should force the values to both be truncated.

Leave a Reply

Your email address will not be published. Required fields are marked *