Hi Ratnesh, For 1, I think you might need to upgrade Metastore database schema. Here are some results from my experiments. In my experiments, Hive 4 merges HIVE-26537 (Deprecate older APIs in the HMS thrift interface), and the results might be different without merging HIVE-26537.
Exp #1. Can Hive 3 access ACID tables of Hive 4? 1) Create an ACID table my_acid_table4 in Hive 4, e.g.: CREATE TABLE my_acid_table4 ( name STRING, age INT ) STORED AS ORC TBLPROPERTIES ( 'transactional' = 'true' ); 2) Start Hive 3 and connect to Hive 4 Metastore 3) Reading the table fails: 0: jdbc:hive2://orange1:9853/> select * from my_acid_table4; ... Error: java.io.IOException: java.lang.RuntimeException: ORC split generation failed with exception: For input string: "00000_0" (state=,code=0) Caused by: java.lang.NumberFormatException: For input string: "00000_0" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:67) ~[?:?] at java.lang.Integer.parseInt(Integer.java:668) ~[?:?] at java.lang.Integer.parseInt(Integer.java:786) ~[?:?] at org.apache.hadoop.hive.ql.io.AcidUtils.parseBucketId(AcidUtils.java:336) ~[hive-exec-3.1.3.jar:3.1.3] at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.determineSplitStrategy(OrcInputFormat.java:2408) ~[hive-exec-3.1.3.jar:3.1.3] Exp #2. Can Hive 4 access ACID tables of Hive 3? 1) Create an ACID table my_acid_table3 in Hive 3, e.g.: CREATE TABLE my_acid_table3 ( name STRING, age INT ) STORED AS ORC TBLPROPERTIES ( 'transactional' = 'true' ); 2) Start Hive 4 and connect to Hive 3 Metastore 3) Reading the table fails: 0: jdbc:hive2://orange1:9852/> use test313acidtable; Error: Error while compiling statement: FAILED: SemanticException org.apache.thrift.TApplicationException: Invalid method name: 'get_database_req'; Query ID: hive_20250520082921_ff4ff6f6-93bc-46b8-a4fd-99609e356a2d (state=42000,code=40000) The results are just as expected. In another experiment, I manually applied the upgrade scripts (from upgrade-3.1.0-to-3.2.0.mysql.sql to upgrade-4.0.0-beta-1-to-4.0.0.mysql.sql), and the database created by Hive 3 works okay on Hive 4. However, the ACID table was in a clean state and did not have insert/delete/compaction operations. --- Sungwoo On Mon, May 19, 2025 at 5:10 PM Ratnesh Mishra <ratnesr...@gmail.com> wrote: > Hello, > > I'm working on migrating my Hive Setup from Hive 3 to Hive 4 in which I'm > facing a few issues > especially with ACID tables like > 1. Being not able to read data inserted via Hive 3 from Hive4 and vice > versa for transactional tables. > 2. Unable to trigger major Compaction from Hive3(getting stuck) > > P:S - I have a setup where I have some of my clusters using Hive3 and some > of them being upgraded to Hive4 using a common metastore with Hive4 > schema.Also I have made some metastore schema changes already to support > Hive4 schema from Hive3. > And for the time being I want this setup to work seamlessly as migrating > all of the clusters to Hive4 would take some considerable time. > Therefore I wanted to check if this behaviour is expected (for ACID > tables) and if there is a blog/document like Migrating Hive 1-2 to Hive 3 > <https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/upgrade-cdh/topics/ug_cdh_hive_preupgrade_tasks.html> > which > suggests proper steps for upgrading Hive schema and migrating data from > Hive 3 to Hive4. > > Thanks for your help. > Regards, > Ratnesh > > >
