Diese Seite mit anderen teilen ...

Informationen zum Thema:
WinDev Forum
Beiträge im Thema:
Erster Beitrag:
vor 9 Monaten, 3 Wochen
Letzter Beitrag:
vor 9 Monaten, 3 Wochen
Beteiligte Autoren:
Stefan Bentvelsen, Erik Schwarz, Allard

[WD19] Calculation with rounding on 6 decimals

Startbeitrag von Stefan Bentvelsen am 31.10.2016 16:10

Hi all,

I was under the impression that using currencies for calculations was the way to use (max.) six decimal places, but that does not seem to be necessarily so. I get a result with only four decimal places instead of the expected 6 decimals.
Now I have used numerics as intermediate fields and found it to be correct calculation. What causes the difference in the calculation of currencies?

lcyKgPrijs = TblSlachtrapport.DEF_PrijsNoteringWeekprijs // 0.595
lcyBtwPerc = TblBtw.BTWPercentage // 6
lcyKgPrijsExcl = round(lcyKgPrijs / (100.0 + lcyBtwPerc) * 100.0, 6)
// lcyKgPrijsExcl is : 0.5613 --> 4 decimals, but should be 6 decimals

lnKgPrijs is numeric(5,7) = TblSlachtrapport.DEF_PrijsNoteringWeekprijs // 0.595
lnBtwPerc is numeric(5,7) = TblBtw.BTWPercentage // 6
lcyKgPrijsExcl = Round(lnKgPrijs / (100.0 + lnBtwPerc) * 100.0, 6)
// lcyKgPrijsExcl is : 0.561321 is OK

Thanks for any hints.



Is ths a currency : "DEF_PrijsNoteringWeekprijs"

If so hwo is that field defined in the analysis?

Maybe a currency var takes on the state of this field. And A nemeric one doesnot.


von Allard - am 31.10.2016 21:36
Sorry Allard for being not clear,

DEF_PrijsNoteringWeekprijs is indeed defined as currency in the analysis. Also the other fields in the first example are defined as currency.

von Stefan Bentvelsen - am 01.11.2016 07:28
Hi Stefan,

no miracle!

currency has 6 Digits for the decimal part.

your calculation first does: 0.595/106 -> 0.005613208
truncated to 6 decimals -> 0.005613
then multiply by 100 -> 0.5613

Change your calculation like (first multiply by 100):

lcyKgPrijsExcl = round(100 * lcyKgPrijs / (100.0 + lcyBtwPerc), 6)

then you get in the first step : 100*0.595 = 59.5
second step: 59.5/105 = 0.561321

In theory you could leave away the round function, because of the max 6 Digits of a currency, but would surely be not a good idea.



von Erik Schwarz - am 01.11.2016 08:41

second step: 59.5/106 = 0.561321

von Erik Schwarz - am 01.11.2016 08:43
Hi Erik,

thanks you for your explanation!
It makes clear what I have to do to make it work.

von Stefan Bentvelsen - am 01.11.2016 09:42
Zur Information:
MySnip.de hat keinen Einfluss auf die Inhalte der Beiträge. Bitte kontaktieren Sie den Administrator des Forums bei Problemen oder Löschforderungen über die Kontaktseite.
Falls die Kontaktaufnahme mit dem Administrator des Forums fehlschlägt, kontaktieren Sie uns bitte über die in unserem Impressum angegebenen Daten.