Post by thread:[OT] Overflow in VB

>-----Original Message----- >From: spam_OUTpiclist-bouncesTakeThisOuTmit.edu On Behalf Of Dennis Crawley >Sent: Sunday, March 27, 2005 6:05 PM > >Hi, >I don't understand why VB rises an overflow error >when this is performed > >DELFN = 84 / (256 * (256 * (256 * (3 + beta) + 2) + 2) + 2) > >but nothing happens if is written like this > >DELFN = 84 / (256 * (Val(256) * (256 * (3 + beta) + 2) + 2) + 2) > ^^^ >VB mysteries? Is the variable beta defined as an integer? If yes then: In most, if not all, Basic dialects the variable beta will be used to hold the intermediate results during calculation. (256 * (3 + beta) + 2) fits into an integer but the next * 256 overflows an integer sized variable. I believe the Val(256) function call is returning a variant that is used for the remaining intermediate calculations avoiding the overflow condition. While this works I don't think VB guarantees this method to work so, I recommend you define beta as a long instead of an integer. Alternatively you can copy beta to a long and use the long in the calculation. Paul Hutch

{Quote hidden}>> >>-----Original Message----- >> >From: <.....piclist-bouncesKILLspam@spam@mit.edu>piclist-bouncesKILLspammit.edu On Behalf >> Of Dennis Crawley >> >Sent: Sunday, March 27, 2005 6:05 PM >> > >> >Hi, >> >I don't understand why VB rises an overflow error >> >when this is performed >> > >> >DELFN = 84 / (256 * (256 * (256 * (3 + beta) + 2) + 2) + 2) >> > >> >but nothing happens if is written like this >> > >> >DELFN = 84 / (256 * (Val(256) * (256 * (3 + beta) + 2) + 2) + 2) >> > ^^^ >> >VB mysteries? > > >Is the variable beta defined as an integer? If yes then: > >In most, if not all, Basic dialects the variable beta will be used to hold >the intermediate results during calculation. (256 * (3 + beta) + 2) fits >into an integer but the next * 256 overflows an integer sized variable. > >I believe the Val(256) function call is returning a variant that is used for >the remaining intermediate calculations avoiding the overflow condition. >While this works I don't think VB guarantees this method to work so, I >recommend you define beta as a long instead of an integer. Alternatively you >can copy beta to a long and use the long in the calculation. > >Paul Hutch I think that you can also define the value 256 as a long by appending the '&' character. Try DELFN = 84& / (256& * (256& * (256& * (3 + beta) + 2) + 2) + 2) This will force VB to use a LONG type to store the results. I am not expert on this but I think you will find it works. BTW. There certainly are better forums for getting info on VB problems than the piclist. You may get a dozen or so better answers on one of these boards. Regards, Jim Robertson NEWFOUND ELECTRONICS

Paul Hutchinson wrote: > Is the variable beta defined as an integer? If yes then: Yes. > In most, if not all, Basic dialects the variable beta will be used to > hold the intermediate results during calculation. (256 * (3 + beta) + > 2) fits into an integer but the next * 256 overflows an integer sized > variable. That happens. > I believe the Val(256) function call is returning a variant that is > used for the remaining intermediate calculations avoiding the > overflow condition. While this works I don't think VB guarantees this > method to work so, I recommend you define beta as a long instead of > an integer. Alternatively you can copy beta to a long and use the > long in the calculation. I have forced the value "256" to a long and works. Thank you, Paul. Dennis Crawley Argentina ;) PS: I'm aware there are more sites, but I like this. BTW I'm making a program which returns a precise delay loop code, based on Andy Warren basic algorithm. It will be shared freely here.