Re: Hive Metastore integration future

Regarding the Hive versions... from what I've seen over on the Hive mailing lists it sounds like Spark 3.0 will be targeting Hive 2.3.x based on whether they can get the Hive community to do a new point release of Hive before they finalise Spark 3.0. So hopefully that gets sorted in time and then I

Re: Hive Metastore integration future

Sorry for the late reply, everyone. This slipped to the bottom of my inbox. As for the Hive version, we were using 1.2.1 for the Spark runtime because that was the default in Spark. I think that changes in Spark 3.0, so when we move master over to build for 3.0, we should update the default Hive v

Re: Hive Metastore integration future

" It would be simply to gain full functionality of Hive" . That should read Iceberg. On Wed, Jan 29, 2020 at 12:55 PM Kristopher Kane wrote: > Adrian, "I'd imagine that keeping binary compatibility across Hive, Spark > and Iceberg will be quite a challenge." Yeah, this is what I'm afraid of > o

Re: Hive Metastore integration future

Adrian, "I'd imagine that keeping binary compatibility across Hive, Spark and Iceberg will be quite a challenge." Yeah, this is what I'm afraid of over time. Iceberg's big draw for me is only maintaining a processing engine (Spark), Iceberg and cloud storage compatibility and any potential Iceber

Re: Hive Metastore integration future

On the topic of Hive versions - we've definitely experienced some issues trying to programmatically use the iceberg-spark-runtime artifact in unit tests (it uses Hive 1.2 as mentioned above). We then tried to also use some other common HIve testing libraries like HiveRunner

Re: Hive Metastore integration future

Hi Kris, We use version 1.2.1 because the part that we're using hasn't changed much and we want to ensure compatibility with old metastore versions. Iceberg should work with newer metastores, and feel free to open a bug if you find a problem with one. We'll make sure to fix it to be compatible wit