Hey Lee!
On 7/9/20 9:09 PM, leerho wrote:
This is super! We look forward to any PRs that you feel would make sense to
have in the library that would make this integration easier or more complete.
Sure, will do when I can - right now I'm a bit flooded...but that used to
change :)
You might also want to look at this suggestion that came to us recently:
Support for "advanced" SQL types (in HLL) (in Hive)
<https://lists.apache.org/thread.html/r8660d693f17752d3ccfa09dc6a76a862f179e1868a33f9918b16c00f%40%3Cusers.datasketches.apache.org%3E>.
Yeah...these kind of stuff will need some tweaking/etc for sure. I'll talk with
Csaba about this stuff.
Also, at some point it may make more sense to move the code we have in our datasketches-hive repository into Hive (we would then deprecate our Hive repo), where it can be
more easily kept up-to-date as Hive evolves. This is how it works with our Druid integration. Having the DataSketches library tightly integrated with Hive will provide
significantly improved performance and requires much more intimate knowledge of the internals of Hive than we have in our DataSketches team.
Actually; doing the integration this way have provided some interesting
insights about what's easy/hard to extend Hive - and I think that should also
be improved a bit.
I agree that at some point the UDF glue methods should probably be moved into Hive (it will probably enable the DataSketches community to focus more on the core library
parts - and less on the integrations) - but I think as long as we don't have an official Hive release which supports DataSketches out of the box - we should keep the
datasketches-hive project (afterall that lib can be loaded into almost any version of Hive) - so I would definetly not rush it; but I'll keep this in mind :)
> Please stay in touch with us!
Of course - I'll be here...reading the list/etc :D
cheers,
Zoltan
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
