Posts Tagged ‘flops’

flops

August 30, 2010

At least in Javascript, that’s how it works.  Apparently floating-point operations are not exactly accurate in a computer, the way integer arithmetic or arbitrary-precision arithmetic is.

Javascript

scheherazade:~$ js
Rhino 1.7 release 2 2010 01 20
js> .1+.2
0.30000000000000004

Berkeley Calculator

scheherazade:~$ bc -l
bc 1.06.95
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
.0003^2 + .0004^2 - .0005^2
0

(from the 3,4,5 right triangle used in the SAT)