I encountered this problem recently when testing to be sure that students were correctly entering the difference between two columns of experimental data. My checking was taking place in JavaScript, not in Perl and I found this article to be very helpful for what I was doing. I'm sure this is not directly applicable to your situation, Gene, but nevertheless may be helpful? I can provide more info on what I did and how my testing works if anyone is interested.
https://www.jacklmoore.com/notes/rounding-in-javascript/ Doug DOUGLAS G MILLS Director of Instructional Technology Department of Chemistry University of Illinois at Urbana-Champaign Chemistry Annex Box A2 601 S. Mathews | M/C 712 Urbana, IL 61801 217.244.5739 | fax: 217.244.8029 | dmi...@illinois.edu<mailto:dmi...@illinois.edu> chemistry.illinois.edu<http://chemistry.illinois.edu> [/var/folders/kv/f3wpy1cs51521x5grm_qx4tw0000gn/T/com.microsoft.Outlook/WebArchiveCopyPasteTempFiles/P698ojxP4tc7j8gMqgPv+E4KQAAAABJRU5ErkJggg==]<http://illinois.edu/> Under the Illinois Freedom of Information Act any written communication to or from university employees regarding university business is a public record and may be subject to public disclosure. From: LON-CAPA-users <lon-capa-users-boun...@mail.lon-capa.org> on behalf of Peter Riegler <p.rieg...@ostfalia.de> Reply-To: Discussion list for LON-CAPA users <lon-capa-users@mail.lon-capa.org> Date: Sunday, September 22, 2019 at 7:35 AM To: "Harding, Gene L" <glhar...@purdue.edu>, Discussion list for LON-CAPA users <lon-capa-users@mail.lon-capa.org> Subject: Re: [LON-CAPA-users] FW: Rounding Issue Hi Gene, most likely the problem is one of number representation (which contributes to rounding errors). The binary representation of $Av+0.05 is (by which I mean "the result of binary representation of $Av plus binary respresentation of 0.05 as a binary") 10.101100110011001100110011001100110011001100110011001 whereas the binary representation of 2.7 is 10.101100110011001100110011001100110011001100110011010 Both binary numbers differ only in the last two digits and the second one is larger. That explains why 2.7 is not accepted. I have done the computations on 64-bit machine which likely matches your server's resolution. I hope this is of help. Peter Am 22.09.2019 um 02:51 schrieb Harding, Gene L: Hi everyone, I have a quiz problem in LC that seems to be doing some odd things with the rounding. I have successfully recreated the problem reported by one of my students, but cannot figure out why it happens. The answer to the problem (part c) is 2.65 on the dot. The problem requests an answer with one place after the decimal, and the tolerance is +/- 0.05. In this case, since the value computes to exactly 2.65, LC should take anything from 2.6 to 2.7. In fact, in author view LC indicates it will take anything from 2.6 to 2.7: [cid:part1.99A52A7F.672D8F68@ostfalia.de] The trouble is that 2.6 works, 2.65 works, but 2.7 does not. This is the Perl script calculation (Rf=33, Ri=20): $Av = 1+$Rf/$Ri; # Gain This is the response code: <numericalresponse unit="" format="1f" answer="$Av" id="3"> <responseparam name="tol" type="tolerance" default="0.05" description="Numerical Tolerance" /> <textline readonly="no" spellcheck="none" /> </numericalresponse> Any idea what the problem is? I am baffled. Best regards, Gene L. Harding, PE Associate Professor of ECET Purdue University 574-520-4190 https://polytechnic.purdue.edu/south-bend/<https://urldefense.proofpoint.com/v2/url?u=https-3A__polytechnic.purdue.edu_south-2Dbend_&d=DwMD-g&c=OCIEmEwdEq_aNlsP4fF3gFqSN-E3mlr2t9JcDdfOZag&r=d6Snpw24IWBdGQpe7yVTtlsEXqfulT1FGlRs-RT3y9s&m=lNqg2OTmaYK8aMffQM-PWQXcFIk8OeCaYOBvj1cHHZA&s=--f9TNkjaTX7CsLeCDgFX1DlFFmex8OEDMVnuTv1iVI&e=> _______________________________________________ LON-CAPA-users mailing list LON-CAPA-users@mail.lon-capa.org<mailto:LON-CAPA-users@mail.lon-capa.org> http://mail.lon-capa.org/mailman/listinfo/lon-capa-users<https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.lon-2Dcapa.org_mailman_listinfo_lon-2Dcapa-2Dusers&d=DwMD-g&c=OCIEmEwdEq_aNlsP4fF3gFqSN-E3mlr2t9JcDdfOZag&r=d6Snpw24IWBdGQpe7yVTtlsEXqfulT1FGlRs-RT3y9s&m=lNqg2OTmaYK8aMffQM-PWQXcFIk8OeCaYOBvj1cHHZA&s=fFuQTInd6fZaePeX7oUVGIfuQupRxyBgTktumFXVqkU&e=> -- Peter Riegler Fakultät Informatik Ostfalia Hochschule für angewandte Wissenschaften - Hochschule Braunschweig/Wolfenbüttel Salzdahlumer Str. 46/48 38302 Wolfenbüttel Tel 05331 939 31540 Fax 05331 939 31542 Web www.ostfalia.de/pws/riegler<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ostfalia.de_pws_riegler&d=DwMD-g&c=OCIEmEwdEq_aNlsP4fF3gFqSN-E3mlr2t9JcDdfOZag&r=d6Snpw24IWBdGQpe7yVTtlsEXqfulT1FGlRs-RT3y9s&m=lNqg2OTmaYK8aMffQM-PWQXcFIk8OeCaYOBvj1cHHZA&s=zEp_gv4MSfwthw4VS8lD3vtw2EEyoA4XJmiQK8H2rFQ&e=>
_______________________________________________ LON-CAPA-users mailing list LON-CAPA-users@mail.lon-capa.org http://mail.lon-capa.org/mailman/listinfo/lon-capa-users
