I'm also interested in this SPIP. There was someone else also working on this, if I remember correctly.
@Mich Talebzadeh <mich.talebza...@gmail.com> , if you need any help with that issue, let me know. El vie, 6 jun 2025, 1:07, Mich Talebzadeh <mich.talebza...@gmail.com> escribió: > i started working on this by upgrading my hadoop to > > Hadoop 3.4.1 > > My Hive is > > Driver: Hive JDBC (version 4.0.1) > Transaction isolation: TRANSACTION_REPEATABLE_READ > Running init script /home/hduser/dba/bin/add_jars.hql > 25/06/05 23:33:44 [main]: WARN util.NativeCodeLoader: Unable to load > native-hadoop library for your platform... using builtin-java classes where > applicable > 0: jdbc:hive2://rhes75:10099/default> *set > hive.support.concurrency=false;* > No rows affected (0.027 seconds) > 0: jdbc:hive2://rhes75:10099/default> > 0: jdbc:hive2://rhes75:10099/default> Beeline version 4.0.1 by Apache Hive > > Now we hive transactional with Hive 4. which we did not have with the > prior versions. Compounded my metastore is on Oracle 12. > messing around with > set hive.support.concurrency=false; > show databases > . . . . . . . . . . . . . . . . . . > Error: Error running query > (state=,code=0) > *set hive.support.concurrency=true* > . . . . . . . . . . . . . . . . . . > No rows affected (0.002 seconds) > 0: jdbc:hive2://rhes75:10099/default> show databases > . . . . . . . . . . . . . . . . . . > +----------------+ > | database_name | > +----------------+ > | access | > | accounts | > > So the last time I worked on it I was trying to sort out the concurrency > issues > > Running simple queries on Hive > > FAILED: Error in acquiring locks: Error communicating with the metastore > ERROR : FAILED: Error in acquiring locks: Error communicating with the > metastore > org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with > the metastore > at > org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:183) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:475) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocksWithHeartbeatDelay(DbTxnManager.java:509) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:296) > at > org.apache.hadoop.hive.ql.lockmgr.HiveTxnManagerImpl.acquireLocks(HiveTxnManagerImpl.java:81) > at > org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:101) > at > org.apache.hadoop.hive.ql.DriverTxnHandler.acquireLocksInternal(DriverTxnHandler.java:328) > at > org.apache.hadoop.hive.ql.DriverTxnHandler.acquireLocks(DriverTxnHandler.java:232) > at > org.apache.hadoop.hive.ql.DriverTxnHandler.acquireLocksIfNeeded(DriverTxnHandler.java:144) > at org.apache.hadoop.hive.ql.Driver.lockAndRespond(Driver.java:356) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:197) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:154) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:149) > at > org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:185) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236) > at > org.apache.hive.service.cli.operation.SQLOperation.access$500(SQLOperation.java:90) > at > org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:336) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1953) > at > org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:356) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.TApplicationException: Internal error > processing lock > > Unfortunately something is missing somewhere > > They have seen this error with postgres Hive metastore DB as well. I need > to work on it when I have a chance > > HTH > > Dr Mich Talebzadeh, > Architect | Data Science | Financial Crime | Forensic Analysis | GDPR > > view my Linkedin profile > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > > > > > On Thu, 5 Jun 2025 at 22:20, Rozov, Vlad <vro...@amazon.com.invalid> > wrote: > >> Hi All, >> >> I want to start a discussion thread on the SPIP titled "Upgrade Apache >> Hive to 4.x” [JIRA <https://issues.apache.org/jira/browse/SPARK-52408>][ >> Doc >> <https://docs.google.com/document/d/1ejaGpuBvwBz2cD3Xj-QysShauBrdgYSh5yTxfAGvS1c/edit?usp=sharing>] >> that I was researching for some time now. >> >> The SPIP proposes upgrading the Apache Hive version used in Apache Spark >> builds from 2.3.10 to version 4.x. It also proposes discontinuing support >> for Apache Hive 2.x and 3.x, as these versions are no longer maintained by >> the Apache Hive community and have reached end-of-life (EOL). >> >> The key objectives of this proposal are to: >> >> >> - Maintain all existing functionality currently supported in Apache >> Hive 2.x that is compatible with Apache Hive 4.x >> - Ensure no functional or performance regressions occur >> - Provide the best upgrade path for current Apache Spark users, >> minimizing prerequisites and manual steps for those using Hive 2.x or 3.x >> >> >> I'd greatly appreciate your feedback, thoughts, and suggestions on this >> proposal! >> >> Thank you, >> >> Vlad >> >
