calculation with numbers with comma

[GinoTheCop]

Hi everyone.. I have a form page with 16 different fields. When user inputs or selects from dropdown lists, the page has to do a calculation.

The calculations are like
Y= 0.120218 * 0.010208 * 0.036481 (...16 different numbers)
sY = Y X 100
sX = 1 - sY

I usually get this
Y = 0,000000000000000000000011 (22 zeros after comma)
sY = 0,000000000000000000001092

its fine upto this point

but i get
sX= 1,000000000000000000000000

seems after the 16th digit after comma my code is ignoring numbers
how can i fix this?
_________________
Cuneyt Yetgin

[administrator]

You can use type-conversions using CDbl VBScript function. For example:


sY = CDbl(0.000000000000000000001092)
sX = CDbl(1.0) - sY

this should force double precision calculation and return a sub-type double result.

Try it and let me know if it worked?
_________________
Peter
ASP & ASP.NET Articles and Tutorials

[GinoTheCop]

I have converted every variable that takes place in the calculation to CDbl type but no luck... still can not substract from 1 The result is still 1 after substraction...
_________________
Cuneyt Yetgin

[GinoTheCop]

Below is the sample code that i get the error with..


<%

a = CDbl(0.02744)
b = CDbl(0.02016)
c = CDbl(0.04466)
d = CDbl(0.013068)
e = CDbl(0.123084)
f = CDbl(0.028561)
g = CDbl(0.008134)
h = CDbl(0.023056)
i = CDbl(0.014884)
j = CDbl(0.009234)
k = CDbl(0.079422)
l = CDbl(0.104006)
m = CDbl(0.003468)
n = CDbl(0.075808)
o = CDbl(0.02262)

sY = a*b*c*d*e*f*g*h*i*j*k*l*m*n*o
sYY = CDbl(sY) * CDbl(100.0)
sX = CDbl(1.0) - CDbl(sYY)
sXX = sX * 100

sV = FormatNumber(sY, 24)
sVV = FormatNumber(sYY, 24)
sW = FormatNumber(sX, 24)
sWW = FormatNumber(sXX, 24)

Response.Write "Y=<b>" &sY& "</b><br>" & "Y * 100=<b>" &SYY& "</b><br>" & "1 - (Y * 100)=<b>" &sX& "</b><br>" & "<b>%" &sXX& "</b><hr>"
Response.Write "Y=<b>" &sV& "</b><br>" & "Y * 100=<b>" &SVV& "</b><br>" & "1 - (Y * 100)=<b>" &sW& "</b><br>" & "<b>%" &sWW& "</b><hr>"
%>
_________________
Cuneyt Yetgin

[jon]

I'll try your code later and I'll let you know if I get the same problem.

[GinoTheCop]

Ok.. Thanks a lot.. hope we can find a solution for this.. By the way the code is to calculate the frequency of a DNA sample
_________________
Cuneyt Yetgin

[GinoTheCop]

Any ideas? Istill couldnt find a solution for this.. I also tried the same calculation with MS Excel.. Guess what, Excel can not make the same calculation either...
_________________
Cuneyt Yetgin

[raven]

[GinoTheCop]

No I just wanted to make sure everything is double.. Does not matter whether you double it or not.. You can not get the correct result after calculation... Still couldnt solve this..
_________________
Cuneyt Yetgin

[administrator]

[GinoTheCop]

Yes I did notice that ofter 15th digit it is ignored. But when you do this calculation with the default calculator that comes with windows it is possible.. So i thought maybe there is some way to go around this...
_________________
Cuneyt Yetgin

[administrator]

VBScript is a scripting language, which has limited capabilities. You can move your calculations into a custom COM object (DLL) and use this object within your ASP pages. This is the only work around I see in your case.

Of course you can move directly to ASP.NET, which will save you a lot of time and effort.
_________________
Peter
ASP & ASP.NET Articles and Tutorials

[GinoTheCop]

No this is a project I ve been running for 4 years now.. Thousands of lines of coding is involved and I do not have that much time to re write (and learn asp.net) everything. I was asked if i could add a frequency calculation function the the code. So I assume as you suggested I will have to look into a custom dll option... Thanx for your time and effort to help..
_________________
Cuneyt Yetgin

[administrator]

No problem

Use VB6 for the DLL, it should be very easy to create it and it should handle the calculations nicely.

Good luck and keep us posted!
_________________
Peter
ASP & ASP.NET Articles and Tutorials