Searching \ for '[OT] Overflow in VB' in subject line. ()
Make payments with PayPal - it's fast, free and secure! Help us get a faster server
FAQ page: www.piclist.com/techref/index.htm?key=overflow
Search entire site for: 'Overflow in VB'.

Exact match. Not showing close matches.
PICList Thread
'[OT] Overflow in VB'
2005\03\27@180650 by Dennis Crawley

flavicon
face

Hi,
I don't understand why VB rises an overflow error
when this is performed

DELFN = 84 / (256 * (256 * (256 * (3 + beta) + 2) + 2) + 2)

but nothing happens if is written like this

DELFN = 84 / (256 * (Val(256) * (256 * (3 + beta) + 2) + 2) + 2)
                    ^^^

VB mysteries?


Dennis Crawley
Argentina


2005\03\27@205953 by Paul Hutchinson

picon face
>-----Original Message-----
>From: spam_OUTpiclist-bouncesTakeThisOuTspammit.edu On Behalf Of Dennis Crawley
>Sent: Sunday, March 27, 2005 6:05 PM
>
>Hi,
>I don't understand why VB rises an overflow error
>when this is performed
>
>DELFN = 84 / (256 * (256 * (256 * (3 + beta) + 2) + 2) + 2)
>
>but nothing happens if is written like this
>
>DELFN = 84 / (256 * (Val(256) * (256 * (3 + beta) + 2) + 2) + 2)
>                     ^^^
>VB mysteries?

Is the variable beta defined as an integer? If yes then:

In most, if not all, Basic dialects the variable beta will be used to hold
the intermediate results during calculation. (256 * (3 + beta) + 2) fits
into an integer but the next * 256 overflows an integer sized variable.

I believe the Val(256) function call is returning a variant that is used for
the remaining intermediate calculations avoiding the overflow condition.
While this works I don't think VB guarantees this method to work so, I
recommend you define beta as a long instead of an integer. Alternatively you
can copy beta to a long and use the long in the calculation.

Paul Hutch


2005\03\28@055422 by Jim Robertson

flavicon
face

{Quote hidden}

I think that you can also define the value 256 as a long by appending the
'&' character.

Try DELFN = 84& / (256& * (256& * (256& * (3 + beta) + 2) + 2) + 2)

This will force VB to use a LONG type to store the results.

I am not expert on this but I think you will find it works.
BTW. There certainly are better forums for getting info on VB
problems than the piclist.  You may get a dozen or so better answers
on one of these boards.

Regards,

Jim Robertson
NEWFOUND ELECTRONICS



2005\03\28@074952 by Dennis Crawley

flavicon
face
Paul Hutchinson wrote:
> Is the variable beta defined as an integer? If yes then:

Yes.

> In most, if not all, Basic dialects the variable beta will be used to
> hold the intermediate results during calculation. (256 * (3 + beta) +
> 2) fits into an integer but the next * 256 overflows an integer sized
> variable.

That happens.

> I believe the Val(256) function call is returning a variant that is
> used for the remaining intermediate calculations avoiding the
> overflow condition. While this works I don't think VB guarantees this
> method to work so, I recommend you define beta as a long instead of
> an integer. Alternatively you can copy beta to a long and use the
> long in the calculation.

I have forced the value "256" to a long and works. Thank you, Paul.

Dennis Crawley
Argentina

;)
PS: I'm aware there are more sites, but I like this. BTW I'm making a
program which returns a precise delay loop code, based on Andy Warren basic
algorithm. It will be shared freely here.



More... (looser matching)
- Last day of these posts
- In 2005 , 2006 only
- Today
- New search...