Hi everyone,
I just wrote up my notes for the sync on Wednesday. Feel free to comment if
I’ve missed anything or didn’t remember clearly.
Here’s a quick summary of the main points:
- Highlights: Spark stored procedures are available, Glue and Nessie
catalogs are committed, and Hive now supports DDL and projection pushdown!
Thanks to Anton, Jack, Ryan M., Peter, Christine, Marton, Adrian, and
Adrien!
- We’re going to aim for an 0.11.0 release in the next couple weeks to
avoid waiting until January
- It would be nice to get the Hive config changes in if possible
- Will try to get MERGE INTO done in Spark (DELETE FROM was merged
today!)
- Will also aim to get in Flink CDC writes, as well as Flink
streaming reads
- Please speak up if you need something specific!
- Support for relative paths is a good idea, will probably add them to
the spec
The full notes are here:
https://docs.google.com/document/d/1YuGhUdukLP5gGiqCbk0A5_Wifqe2CZWgOd3TbhY3UQg/edit#heading=h.r951w2dduwmy
--
Ryan Blue
Software Engineer
Netflix
