Rogue Wave banner
Previous fileTop of DocumentContentsIndexNext file

8.3 Inexact Conditions

Division will use as many digits as necessary for an exact result or all available digits if the exact result requires more digits than are available. For this reason, using the result of a division in a subsequent operation can easily result in a loss of precision exception. This can be avoided by rounding the result to a smaller number of digits using the round function.

Consider the example:

which produces the result

Using this value in calculations will likely result in a number with too many digits and a resulting inexact error. The round function can be used to reduce the number of digits after the decimal place:

The pow operation should be used with particular caution since the call,

where x is an n-digit number, will produce a result that is up to n* p digits long. If the result has too many digits it will cause an overflow or precision error.


Previous fileTop of DocumentContentsIndexNext file

©Copyright 1999, Rogue Wave Software, Inc.
Contact Rogue Wave about documentation or support issues.