piclist 2000\11\09\065531a >
Thread: AN575, FP24.A16, INT24 ROUNDING PROBLEM
www.piclist.com/techref/microchip/devices.htm?key=pic
flavicon
face BY : James Hillman email (remove spam text)



Just for the record. To convert 24bit float to 16bit integer without a
rounding error :

As a quick fix I added 0.5 to the 24bit float using FPA24, cleared
FPFLAGS,RND to turn rounding off and converted the number to a 16bit integer
using INT24. I found that this always rounded correctly, but incurred the
delay introduced by the call to FPA24.

James Hillman

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics




<002f01c04a37$9ce6c500$0800a8c0@IIRACS> 7bit

See also: www.piclist.com/techref/microchip/devices.htm?key=pic
Reply You must be a member of the piclist mailing list (not only a www.piclist.com member) to post to the piclist. This form requires JavaScript and a browser/email client that can handle form mailto: posts.
Subject (change) AN575, FP24.A16, INT24 ROUNDING PROBLEM

month overview.

new search...