Re: [sqlite] SIMD based JSON parsing for speeding up JSON extension

2019-02-25 Thread Andreas Kupries
> It's only a fair comparison if the simdjson code runs on the same system. > It might reach 10GB/s or 200MB/s… > > Another possible concern is whether the SQLite JSON code is 100% compliant AFAIK no known json parser is 100% compliant. > (I don't know if this is the case). There are some hair

Re: [sqlite] SIMD based JSON parsing for speeding up JSON extension

2019-02-25 Thread Wout Mertens
It's only a fair comparison if the simdjson code runs on the same system. It might reach 10GB/s or 200MB/s… Another possible concern is whether the SQLite JSON code is 100% compliant (I don't know if this is the case). There are some hairy edge cases in JSON (Unicode handling) that might slow down

Re: [sqlite] SIMD based JSON parsing for speeding up JSON extension

2019-02-25 Thread Richard Hipp
On 2/25/19, Richard Hipp wrote: > performance of just over 3GB/sec, which is slightly > faster than reported simdjson performance of 2.9GB/sec. Further analysis shows that SQLite was caching its parse tree, which was distorting the measurement. The following script adds a different string of spa

Re: [sqlite] SIMD based JSON parsing for speeding up JSON extension

2019-02-25 Thread Richard Hipp
On 2/25/19, Robert M. Münch wrote: > Hi, see: https://github.com/lemire/simdjson > > Can parse GB/s of JSON input. This might be a good candidate to use in the > extension. Thanks for the link. I downloaded one of the sample input files "gsoc-2018.json" and then ran the following test case using

[sqlite] SIMD based JSON parsing for speeding up JSON extension

2019-02-25 Thread Robert M. Münch
Hi, see: https://github.com/lemire/simdjson Can parse GB/s of JSON input. This might be a good candidate to use in the extension. -- Robert M. Münch signature.asc Description: OpenPGP digital signature ___ sqlite-users mailing list sqlite-users@mai