[
https://issues.apache.org/jira/browse/DRILL-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612284#comment-16612284
]
Vitalii Diravka commented on DRILL-6604:
----------------------------------------
The HMS issues which were resolved in hive-3.1 from which Drill Metastore can
get the benefit:
HIVE-19171: Persist runtime statistics in metastore, HIVE-17626 - parent
1) better statistics for table scan operator if dynamic partition pruning is
involved
2) Better estimates for bloom filter initialization (setting expected entries
during merge)
HIVE-15436: Enhancing metastore APIs to retrieve only materialized views
instead of returning all tables
HIVE-16452: Database UUID for metastore DB (thrift API call for
get_metastore_uuid)
HIVE-17159 Make metastore a separately releasable module (the metastore
package is more independent from other Hive's sources)
HIVE-17167 - Create metastore specific configuration tool (separate HMS
configs)
HIVE-17466 - Metastore API to list unique partition-key-value combinations
HIVE-16520 - Cache hive metadata in metastore ([Hive Metadata
Caching|https://cwiki.apache.org/confluence/display/Hive/Hive+Metadata+Caching+Proposa])
HIVE-12274 - Increase width of columns used for general configuration in the
metastore
HIVE-16771 - Schematool should use MetastoreSchemaInfo to get the metastore
schema version from database
HIVE-16914 - Change HiveMetaStoreClient to AutoCloseable
HIVE-16955 - General Improvements To
org.apache.hadoop.hive.metastore.MetaStoreUtils
HIVE-16970 - General Improvements To
org.apache.hadoop.hive.metastore.cache.CacheUtils
The full list is [https://github.com/apache/hive/blob/master/RELEASE_NOTES.txt]
> Upgrade Drill Hive client to Hive3.1 version
> --------------------------------------------
>
> Key: DRILL-6604
> URL: https://issues.apache.org/jira/browse/DRILL-6604
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.14.0
> Reporter: Vitalii Diravka
> Assignee: Vitalii Diravka
> Priority: Major
> Fix For: Future
>
>
> Recently the Hive3.1 version is released.
> 3.1 versions of {{hive-exec}} and {{hive-metastore}} (main Drill Hive client
> libraries) are the last available artifacts in maven repo for now:
> [https://mvnrepository.com/artifact/org.apache.hive/hive-exec/3.1.0]
> [https://mvnrepository.com/artifact/org.apache.hive/hive-metastore/3.1.0]
> It is necessary to investigate what impact of update onto the newer major
> Hive client version.
> The initial conflicts after changing the {{hive.version}} in Drill root pom
> file from 2.3.2 to 3.1.0 are:
> {code}
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR]
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveUtilities.java:[58,39]
> error: cannot find symbol
> [ERROR] symbol: class MetaStoreUtils
> location: package org.apache.hadoop.hive.metastore
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveMetadataProvider.java:[34,39]
> error: cannot find symbol
> [INFO] 2 errors
> {code}
> {code}
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR]
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveUtilities.java:[575,15]
> error: cannot find symbol
> [ERROR] symbol: method setTransactionalTableScan(JobConf,boolean)
> location: class AcidUtils
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveFieldConverter.java:[216,92]
> error: incompatible types: org.apache.hadoop.hive.common.type.Timestamp
> cannot be converted to java.sql.Timestamp
> [ERROR]
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveFieldConverter.java:[225,82]
> error: incompatible types: org.apache.hadoop.hive.common.type.Date cannot be
> converted to java.sql.Date
> [ERROR]
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/DrillHiveMetaStoreClient.java:[100,17]
> error: cannot find symbol
> [ERROR] symbol: method setTokenStr(UserGroupInformation,String,String)
> location: class Utils
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[29,16]
> error: Required is not abstract and does not override abstract method
> getPrimitiveJavaObject(Object) in TimestampObjectInspector
> [ERROR]
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[35,30]
> error: getPrimitiveJavaObject(Object) in Required cannot implement
> getPrimitiveJavaObject(Object) in TimestampObjectInspector
> [ERROR] return type java.sql.Timestamp is not compatible with
> org.apache.hadoop.hive.common.type.Timestamp
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[44,29]
> error: getPrimitiveWritableObject(Object) in Required cannot implement
> getPrimitiveWritableObject(Object) in TimestampObjectInspector
> [ERROR] return type TimestampWritable is not compatible with
> TimestampWritableV2
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[54,16]
> error: Optional is not abstract and does not override abstract method
> getPrimitiveJavaObject(Object) in TimestampObjectInspector
> [ERROR]
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[60,30]
> error: getPrimitiveJavaObject(Object) in Optional cannot implement
> getPrimitiveJavaObject(Object) in TimestampObjectInspector
> [ERROR] return type java.sql.Timestamp is not compatible with
> org.apache.hadoop.hive.common.type.Timestamp
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[72,29]
> error: getPrimitiveWritableObject(Object) in Optional cannot implement
> getPrimitiveWritableObject(Object) in TimestampObjectInspector
> [ERROR] return type TimestampWritable is not compatible with
> TimestampWritableV2
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[29,16]
> error: Required is not abstract and does not override abstract method
> getPrimitiveJavaObject(Object) in DateObjectInspector
> [ERROR]
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[35,25]
> error: getPrimitiveJavaObject(Object) in Required cannot implement
> getPrimitiveJavaObject(Object) in DateObjectInspector
> [ERROR] return type java.sql.Date is not compatible with
> org.apache.hadoop.hive.common.type.Date
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[44,24]
> error: getPrimitiveWritableObject(Object) in Required cannot implement
> getPrimitiveWritableObject(Object) in DateObjectInspector
> [ERROR] return type DateWritable is not compatible with DateWritableV2
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[54,16]
> error: Optional is not abstract and does not override abstract method
> getPrimitiveJavaObject(Object) in DateObjectInspector
> [ERROR]
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[60,25]
> error: getPrimitiveJavaObject(Object) in Optional cannot implement
> getPrimitiveJavaObject(Object) in DateObjectInspector
> [ERROR] return type java.sql.Date is not compatible with
> org.apache.hadoop.hive.common.type.Date
> /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[72,24]
> error: getPrimitiveWritableObject(Object) in Optional cannot implement
> getPrimitiveWritableObject(Object) in DateObjectInspector
> [INFO] 16 errors
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
