piclist 2000\11\09\014946a >
Thread: AN575, FP24.A16, INT24 ROUNDING PROBLEM
www.piclist.com/techref/microchip/devices.htm?key=pic
flavicon
face BY : Nikolai Golovchenko email (remove spam text)

part 1 3306 bytes content-type:text/plain; charset=Windows-1251 (decoded quoted-printable)

Hi James,

I've found also a few more problems with the INT24 routine:

1) when AEXP==0, which means zero floating number, but
AARGB0:1 is not zero, return will be incorrect. So AARGB0:1
have to be cleared in that case.

2) problem with overflow checking (for example -32769
(0x8e8001) will result in 0xC78e, instead of 0x8000)

There is really something wrong with rounding. They teach in
school that if a number is in the middle between two
integers, the highest by absolute value is taken. For
example, MATLAB has four rounding functions:

ROUND - round to closest integer (like in school)
FIX - round towards zero (ignore fractional part)
CEIL - round towards plus infinity
FLOOR - round towards minus infinity

let x = [-1:0.25:1]

x = [-1:0.25:1]
x =
   -1 -0.75  -0.5 -0.25     0  0.25   0.5  0.75     1
round(x)
ans =
   -1    -1    -1     0     0     0     1     1     1
fix(x)
ans =
   -1     0     0     0     0     0     0     0     1
ceil(x)
ans =
   -1     0     0     0     0     1     1     1     1
floor(x)
ans =
   -1    -1    -1    -1     0     0     0     0     1

I assume that the INT24 routine should implement ROUND
function. But why there is a check for AARGB1.0 (LSb of
integer result)?...

Check attachment for a fixed version (I hope) of INT24
routine (fpint.asm).

Hope it helps,
Nikolai

---- Original Message ----
From: James Hillman <.....james@spam@spamspamBeGoneIND-INTERFACE.CO.UK>
Sent: Tuesday, November 07, 2000 14:49:34
 To: spamBeGonePICLISTspam.....MITVMA.MIT.EDU
Subj: [PIC]: AN575, FP24.A16, INT24 ROUNDING PROBLEM

{Quote hidden}


part 2 6916 bytes content-type:application/octet-stream; name="fpint.asm"
(decode)

part 3 146 bytes
--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics




<1564305322.20001109084607@yahoo.com>

In reply to: <006001c048b9$2e21f1a0$0800a8c0@IIRACS>
See also: www.piclist.com/techref/microchip/devices.htm?key=pic
Reply You must be a member of the piclist mailing list (not only a www.piclist.com member) to post to the piclist. This form requires JavaScript and a browser/email client that can handle form mailto: posts.
Subject (change) AN575, FP24.A16, INT24 ROUNDING PROBLEM

month overview.

new search...