piclist 2000\11\09\014946a >
Thread: AN575, FP24.A16, INT24 ROUNDING PROBLEM
www.piclist.com/techref/microchip/devices.htm?key=pic
BY : Nikolai Golovchenko email (remove spam text)

part 1 3306 bytes content-type:text/plain; charset=Windows-1251 (decoded quoted-printable)

Hi James,

I've found also a few more problems with the INT24 routine:

1) when AEXP==0, which means zero floating number, but
AARGB0:1 is not zero, return will be incorrect. So AARGB0:1
have to be cleared in that case.

2) problem with overflow checking (for example -32769
(0x8e8001) will result in 0xC78e, instead of 0x8000)

There is really something wrong with rounding. They teach in
school that if a number is in the middle between two
integers, the highest by absolute value is taken. For
example, MATLAB has four rounding functions:

ROUND - round to closest integer (like in school)
FIX - round towards zero (ignore fractional part)
CEIL - round towards plus infinity
FLOOR - round towards minus infinity

let x = [-1:0.25:1]

» x = [-1:0.25:1]
x =
-1 -0.75  -0.5 -0.25     0  0.25   0.5  0.75     1
» round(x)
ans =
-1    -1    -1     0     0     0     1     1     1
» fix(x)
ans =
-1     0     0     0     0     0     0     0     1
» ceil(x)
ans =
-1     0     0     0     0     1     1     1     1
» floor(x)
ans =
-1    -1    -1    -1     0     0     0     0     1
»
I assume that the INT24 routine should implement ROUND
function. But why there is a check for AARGB1.0 (LSb of
integer result)?...

Check attachment for a fixed version (I hope) of INT24
routine (fpint.asm).

Hope it helps,
Nikolai

---- Original Message ----
From: James Hillman <jamesIND-INTERFACE.CO.UK>
Sent: Tuesday, November 07, 2000 14:49:34
To: PICLISTMITVMA.MIT.EDU
Subj: [PIC]: AN575, FP24.A16, INT24 ROUNDING PROBLEM

{Quote hidden}

part 2 6916 bytes content-type:application/octet-stream; name="fpint.asm"
(decode)

part 3 146 bytes
--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

<1564305322.20001109084607@yahoo.com>