234 views
in GUI Development by

Hello,

I want to divide by ten a float,the problem is when doing so the value has an error that i don't know how to remove only with float operand admitted by EW.

Solutions online tends to use a double to minimize di error.

here is an example. the value i pass is aNewValue, the RemainingTime is the value divided by 10

i'd like to get a value of 31.8; 

 

1 Answer

+1 vote
by
 
Best answer

Hello Riccardo,

the 'error' is expected and inevitable behavior. From technical point of view, the floating point can't represent the value 31.8. The next closest value for the result of 318.0 / 10.0 is 31.79999... It is the nature of floating point. Using double precision floating point does not solve the root problem. 

During calculation the imprecision of floating points is usually not problematic. Unless the value has to be displayed as string to the user. In such case, use the string() built-in constructor to create from a floating point number a string with desired number of digits after the period sign. This operation (when truncating the digits) automatically rounds the last remaining digits. For example:

I hope it helps you further.

Best regards.

Paul Banach

by
Thansk, Paul.

Ask Embedded Wizard

Welcome to the question and answer site for Embedded Wizard users and UI developers.

Ask your question and receive answers from the Embedded Wizard support team or from other members of the community!

Embedded Wizard Website | Privacy Policy | Imprint

...