PDA

View Full Version : [Article] Rounding errors


ZaiRoN
September 7th, 2006, 14:15
David Goldberg: What every computer scientist should know about floating-point arithmetic

Available here: http://docs.sun.com/source/806-3568/ncg_goldberg.html

Thx to Que for the link.

SiGiNT
September 8th, 2006, 09:22
I was shocked when I accidentally encountered this about 15 years ago, while writing an estimating program for our process in GW Basic it was necessary to use a for - to - step, operation - try this if you still have an old copy of line based basic -

10 for x = 1 to 20 step .1
20 if x = 10 then print "I'm Here!"; x
30 next x

You'll never get the "I'm here" message, somewhere shortly after about 2 or 3 you start accumulating decimal places and you never exactly hit 10.00000 - appalling, considering any TI99 or Commodore had absolutely no problem with the code.

SiGiNT

Silver
September 8th, 2006, 12:53
Ah, floating point accuracy, the bane of 3d coders everywhere...