I remember when I was doing some old benchmarks, I had to build some
custom logic to get writes happening at a decent speed (some basic
batching and avoiding deserializing things when I could). I'm sure there
is much more we can do here, especially around our HTTP transport.
My impression is that Arrow can support the same kind of protocol
evolution that protobuf also allows (that's the big reason we chose it
the first time).
On 8/23/18 3:45 PM, Julian Hyde wrote:
This is a paper in VLDB 2018, "Don’t Hold My Data Hostage – A Case For Client
Protocol Redesign” by Mark Rassveldt and Hannes Muhleisen[1]. It claims that
database client protocols (inside ODBC and JDBC drivers) are very inefficient, and
has a compelling example where commercial drivers are 10x to 68x slower than net-cat.
One of the goals of Avatica is to do better. How are we doing? Are there any
ideas in the paper we could adopt? Would a closer partnership with Apache Arrow
help us achieve those goals?
Julian
[1] https://hannes.muehleisen.org/p852-muehleisen.pdf
<https://hannes.muehleisen.org/p852-muehleisen.pdf>
