Luca Toscano created HIVE-25020:
-----------------------------------
Summary: Prepared statement related exceptions from Metastore when
using org.mariadb.jdbc.Driver
Key: HIVE-25020
URL: https://issues.apache.org/jira/browse/HIVE-25020
Project: Hive
Issue Type: Improvement
Affects Versions: 2.3.6
Reporter: Luca Toscano
Hi everybody,
I am running Apache Bigtop 1.5 on Debian, and I am currently in the process of
migrating everything from Debian 9 (stretch) to 10 (buster). Part of the
upgrade involves the libmysql-dev vs libmariadb-dev packages, since the former
is not provided on Buster (hence the only jdbc driver available is
org.mariadb.jdbc.Driver).
Bigtop 1.5 ships with hive 2.3.6, Buster ships with libmariadb-java 2.3.0.
When I upgraded the hive-site.xml file with
javax.jdo.option.ConnectionDriverName=org.mariadb.jdbc.Driver I noticed some
issues on the Metastore side, for example
```
2021-04-13T06:46:15,564 ERROR [pool-11-thread-144]
metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
10) with error: javax.jdo.JDODataStoreException: Cannot set Object parameter:
value = MANAGED_TABLE for column "" : Could not set parameter at position 1
(values was 'MANAGED_TABLE')
Query - conn:30368(M) - "SELECT DISTINCT
'org.apache.hadoop.hive.metastore.model.MTable' AS
NUCLEUS_TYPE,A0.CREATE_TIME,A0.LAST_ACCESS_TIME,A0.OWNER,A0.RETENTION,A0.IS_REWRITE_ENABLED,A0.TBL_NAME,A0.TBL_TYPE,A0.TBL_ID
FROM TBLS A0 LEFT OUTER JOIN DBS B0 ON A0.DB_ID = B0.DB_ID WHERE
LOWER(B0.`NAME`) LIKE 'default' ESCAPE '\\' AND ((((A0.TBL_TYPE = ?) OR
(A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?)) OR (A0.TBL_TYPE = ?))"
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:391)
at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:267)
at
org.apache.hadoop.hive.metastore.ObjectStore.getTableMeta(ObjectStore.java:1265)
at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
at com.sun.proxy.$Proxy25.getTableMeta(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_meta(HiveMetaStore.java:2006)
at sun.reflect.GeneratedMethodAccessor74.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
at com.sun.proxy.$Proxy26.get_table_meta(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11429)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_meta.getResult(ThriftHiveMetastore.java:11413)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:594)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:589)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:589)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
The above was originated from Hue while trying to fetch metadata about tables
into a database. I tried to deploy libmysql-dev (stretch package deployed on
buster), reverting javax.jdo.option.ConnectionDriverName to
com.mysql.jdbc.Driver and all the issues disappeared. This is a workaround that
can work as interim solution, but I'd like to know if what I encountered above
is expected or if there is a suggested way to fix it (that could also be
included in Hive 3.x, it will be packaged in the next Bigtop version so it
would be perfectly acceptable for me).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
