Re: [sqlite] Rounding Error

2019-11-05 Thread Jose Isaias Cabrera
Adrian Sherwin, on Monday, November 4, 2019 03:41 AM, wrote... > > Hi, > > I would like to report the following as a bug in SQLITE: > > The SQLITE "round" function fails to round between 4.1 and 4.6% of numbers > correctly to x decimal places when held as x+1 decimal places. > > The simplest

Re: [sqlite] Rounding Error

2019-11-04 Thread Keith Medcalf
On Monday, 4 November, 2019 02:10, Jay Kreibich wrote: >OK, no, I’m wrong. Because 1.05 rounds to 1.1, even though the >representation is 1.049523162841796875. The representation of 1.05 is 1.0500444089209 and the next closest representable number is

Re: [sqlite] Rounding Error

2019-11-04 Thread Keith Medcalf
On Monday, 4 November, 2019 02:16, Graham Holden wrote: >This is almost certainly because (according to >https://www.h-schmidt.net/FloatConverter/IEEE754.html) the number >"1.15" cannot be expressed exactly in floating-point; the nearest >expressible number being 1.1497615814208984375, and

Re: [sqlite] Rounding Error

2019-11-04 Thread Keith Medcalf
On Monday, 4 November, 2019 02:10, Jay Kreibich wrote: >> On Nov 4, 2019, at 2:59 AM, Jay Kreibich wrote: >>> On Nov 4, 2019, at 2:41 AM, Adrian Sherwin wrote: >>> The simplest example I have found with x=1 is: >>> "select round(1.15,1)" >>> Result: "1.1" (should be 1.2) >> SQLite uses the

Re: [sqlite] Rounding Error

2019-11-04 Thread Graham Holden
Monday, November 04, 2019, 8:41:48 AM, Adrian Sherwin wrote: > Hi, > I would like to report the following as a bug in SQLITE: > The SQLITE "round" function fails to round between 4.1 and 4.6% of numbers > correctly to x decimal places when held as x+1 decimal places. > The simplest example I

Re: [sqlite] Rounding Error

2019-11-04 Thread Jay Kreibich
> On Nov 4, 2019, at 2:59 AM, Jay Kreibich wrote: > > >> On Nov 4, 2019, at 2:41 AM, Adrian Sherwin wrote: >> >> Hi, >> >> I would like to report the following as a bug in SQLITE: >> >> The SQLITE "round" function fails to round between 4.1 and 4.6% of numbers >> correctly to x decimal

Re: [sqlite] Rounding Error

2019-11-04 Thread Simon Slavin
On 4 Nov 2019, at 8:41am, Adrian Sherwin wrote: > The simplest example I have found with x=1 is: > "select round(1.15,1)" > Result: "1.1" (should be 1.2) SQL1992 4.4.1 Characteristics of numbers: " An approximation obtained by truncation or rounding of a numerical value N for an T is a value

Re: [sqlite] Rounding Error

2019-11-04 Thread Jay Kreibich
> On Nov 4, 2019, at 2:41 AM, Adrian Sherwin wrote: > > Hi, > > I would like to report the following as a bug in SQLITE: > > The SQLITE "round" function fails to round between 4.1 and 4.6% of numbers > correctly to x decimal places when held as x+1 decimal places. > > The simplest example I

[sqlite] Rounding Error

2019-11-04 Thread Adrian Sherwin
Hi, I would like to report the following as a bug in SQLITE: The SQLITE "round" function fails to round between 4.1 and 4.6% of numbers correctly to x decimal places when held as x+1 decimal places. The simplest example I have found with x=1 is: "select round(1.15,1)" Result: "1.1" (should be

[sqlite] Rounding Error ,,, Last Patch was incorrect (Sorry)

2012-06-16 Thread Keith Medcalf
That last patch was a bad workaround -- please ignore it. The et_getdigit function does count the number of significant digits returned. However, it has a wrong boundary. Limiting the number of significant digits is more correct than changing the output precision. So, the following