Bugs item #2757642, was opened at 2009-04-12 23:12 Message generated for change (Tracker Item Submitted) made by johnrylander You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2757642&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: SQL/Core Group: MonetDB5 "stable" Status: Open Resolution: None Priority: 5 Private: No Submitted By: John Rylander (johnrylander) Assigned to: Niels Nes (nielsnes) Summary: Innacurate AVG of SQRT with 51M records Initial Comment: I'm using the latest stable Windows Vista x64-compatible release 5.10.2 based on the 1.28.2 kernel. On an table "biglist" with 51751608 records and just one field, a numeric 14,2 field called "rt", I executed the following: UPDATE biglist SET rt=1.21; (Rows affected 51751608) Then I tried: SELECT AVG(rt) FROM biglist; And of course it reported 1.21--fine. :) But then I tried: SELECT AVG(SQRT(rt)) from biglist; and instead of the expected 1.1, I got 0.64837467465745224. A wide variety of other SQRT aggregations also were wildly off the mark. (And I tried this earlier with a 7,2 numeric field--no joy.) Any suggestions? My apologies if this has already been fixed, or if I'm doing something wrong--I'm entirely new to MonetDB, but think it's an amazing project--amazing! I hope this can be resolved soon, so I can rely on it. :) Thanks to all the developers and testers! I hope to do some tiny part here with this report. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2757642&group_id=56967 ------------------------------------------------------------------------------ This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
