Hello Kudu developers, Recently I have started work on upgrading Kudu to use Apache Hive 3.x. Given this is a major upgrade it does come with some challenges. As of Kudu 1.10.0 we use Hive in the HMS synchronization feature. This feature includes a Kudu server side notification listener and HMS client. It also includes a Java side HMS plugin to enforce Kudu imperatives within the HMS. That feature is useful on its own in many ways, but is also required for fine grained authorization via Apache Sentry.
The primary challenge is that Apache Sentry currently does not support Hive 3 and it will likely take a large effort to enable support. It is also unclear if there is anyone in the Sentry community that want's to contribute and release such support. I have started preliminary efforts to support Hive 3 in Kudu and the HMS synchronization feature. This includes 3 patches. The first patch <https://gerrit.cloudera.org/#/c/14018/> is changes that work in both Hive 2 and Hive 3 that minimize the work needed when we upgrade in the future. This can be committed to master when reviewed and ready. The second patch <https://gerrit.cloudera.org/#/c/14006> disables the sentry integration so I can test the changes required to support HMS synchronization on its own. Those changes and testing are the third patch <https://gerrit.cloudera.org/#/c/13256/>. Given fine grained authorization is a critical feature for many users, we can't remove Sentry support without providing an alternative authorization implementation. At the same time we have started work on authorization via Apache Ranger. Once that implementation exists and has been contributed/released we can make a decision about how to move forward. Given what we know today and the current situation here is my suggested plan: 1. Commit the Hive 3 preparation patch to simplify upgrading in the future 2. Verify the feasibility of upgrading with the mentioned POC patches, but do not commit them. - This means we will remain on Hive 2 until step 4 or 5 below. 3. Start work on an Apache Ranger integration for Kudu. 4. If Hive 3 support is added in Sentry, consider upgrading to Hive 3 then. 5. When Ranger support is complete, consider removing Sentry support in favor of Ranger and upgrade to Hive 3. - This may require a migration path from Sentry to Ranger. Please let me know if you have any thoughts or feedback on the above plan. Thank you, Grant -- Grant Henke Software Engineer | Cloudera [email protected] | twitter.com/gchenke | linkedin.com/in/granthenke
