piclist 2001\05\31\173739a >
Thread: Brain Burp Rounding??
face picon face BY : Bill Westfield email (remove spam text)

   I don't understand your logic?? There are 500 values,
   ie, 000 to 499, then there are 500 values in the top
   half, ie, 500 to 999. The dividing point is *immediately*
   before 500, ie, 499.999999999999999 (forever)

   So if it starts with a 5 it absolutely must be in the
   top half. There is no favoring, just accuracy.

No, because we're not talking about "number of values" when rounding, we're
talking about a "distance."  There are (always) an infinite number of values
between any two numbers, but .5 is exactly half-way between 0 and 1, so
going consistantly either up or down introduces a slight bias.

Or, looking at all the values of precision N+1 that round to a particular
value, you'll find that there are 5 values that round up to that value, and
only 4 values that round down to it (plus the value itself.)

A trivial pieces of software ought to be able to calculate the accumulated
error over a large number of samples, which IS the value you want to
minimize.  Right?


http://www.piclist.com hint: To leave the PICList


In reply to: Your message of Fri, 1 Jun 2001 02:31:07 +1000
See also: www.piclist.com/techref/index.htm?key=brain+burp+rounding
Reply You must be a member of the piclist mailing list (not only a www.piclist.com member) to post to the piclist. This form requires JavaScript and a browser/email client that can handle form mailto: posts.
Subject (change) Brain Burp Rounding??

month overview.

new search...