Even if we went to C++ JSON parsing and used this library, we would still have the following to worry about the following: - How do we get this library to write out the data in columnar format? - How would we ship this, given that Drill might get installed on machines that are pre-AVX The ideal way for Drill to benefit for something like this would be for the openjdk community to use/adapt this library to intrinsify(automatically replace java source for a library function, at runtime, with hand-coded assembly that is appropriate for the target machine) javax.json functions in the JVM. The JVM already does such intrinsification to leverage vector instructions for JDK functions like System.arraycopy() and some crypto functions.
Karthik On Thu, Feb 21, 2019 at 9:04 AM Aman Sinha <[email protected]> wrote: > Almost 20x faster parsing speed than the next fastest parser. Would be > good to explore for Drill (needs the transition from Java to C++ parsing). > > On Wed, Feb 20, 2019 at 10:14 PM Ted Dunning <[email protected]> > wrote: > > > This is interesting. > > > > https://twitter.com/kellabyte/status/1098447972809900037 > > >
