Re: [sqlite] sqlite3AtoF giving incorrect results

Basically a variant of the final idea you had is what we went with. Thanks for the suggestions! Jim Borden Privacy Policy Marketing Preferences ___

Re: [sqlite] sqlite3AtoF giving incorrect results

On 2/27/19, Jim Borden wrote: > For reference here > was the query that I was working with: > > SELECT fl_result(prediction('EchoModel', dict_of('max_double', > 1.7976931348623153e+308))) FROM kv_default AS _doc WHERE (_doc.flags & 1 = > 0); > As a work-around, you could code up a UDF

Re: [sqlite] sqlite3AtoF giving incorrect results

The platforms exhibiting the behavior have so far been MSVC Windows 10 64-bit (both 32 and 64 bit processes), and Android x86. iOS 64-bit was not affected and iOS 32-bit was not testable due to outside factors but my guess is that it is not affected. macOS 64-bit was not affected. I tried to

Re: [sqlite] sqlite3AtoF giving incorrect results

On 2/27/19, Jim Borden wrote: > > My theory as to why this varies by platform is because of the use of long > double during this calculation, which can vary in size. On the failing > platforms they are the same size as regular double. This results in a > rounding error of the "scale" portion of

Re: [sqlite] sqlite3AtoF giving incorrect results

On 2/27/19, Jim Borden wrote: > Hello, > > I’ve been tracking down an odd problem where some of our results from an > application defined SQLite function are coming back as infinity on some > platforms (notably 32-bit Android, and all Windows). I’ve narrowed it down > to the following: > > Given

[sqlite] sqlite3AtoF giving incorrect results

Hello, I’ve been tracking down an odd problem where some of our results from an application defined SQLite function are coming back as infinity on some platforms (notably 32-bit Android, and all Windows). I’ve narrowed it down to the following: Given the example input

Re: [sqlite] sqlite3AtoF Handling of high numbers

Hello, On 2018-05-17 15:07, Stiefsohn, Roman wrote: Hello, i found out that SQLite is having problems with converting high numbers from string to double, located in the function "sqlite3AtoF": Yes -- it is having problems, however the problems are located in a compiler which lacks long

[sqlite] sqlite3AtoF Handling of high numbers

Hello, i found out that SQLite is having problems with converting high numbers from string to double, located in the function "sqlite3AtoF": Below is a table of input strings, and the returned double values, tested on Windows 7 64 bit with SQLite version 3.23.1: Input String:

Re: [sqlite] sqlite3AtoF()

Hello, On 2017-12-29 02:25, Richard Hipp wrote: Please test the latest trunk version (or any version after check-in https://www.sqlite.org/src/timeline?c=fd2e0e7a) and confirm that the modifications work for you. It works as expected. Thank you. -- best regards Cezary H. Noweta

Re: [sqlite] sqlite3AtoF()

On 12/25/17, Cezary H. Noweta wrote: > > Could you consider an exponentiation by squaring (in the main release) > instead of current n-multiplication of exponents? Please test the latest trunk version (or any version after check-in

[sqlite] sqlite3AtoF()

Hello, Could you consider an exponentiation by squaring (in the main release) instead of current n-multiplication of exponents? I've observed that an average time spent in sqlite3AtoF() is greatly reduced by 12% to over 30%, depending on an exponent's magnitude and a compiler. It is nothing more