I have not checked the detail about HVIE-28366. Maybe Denys can give more explanations.
But for your case: For iceberg table, IOW in Hive and Insert in Trino at the same time. If you want to ensure data correctness, You should ensure that Hive/Hs2 and Trino use the same Iceberg catalog. However IMO, We Hive/Hs2 use HMS catalog as Iceberg catalog, but Trino use its customized Iceberg catalog. That is to say, your two operations(IOW&Insert) are using two different catalogs, which is likely to cause data correctness. BTW, you said 'To resolve this issue, it seems necessary to use 'DbTxnManager' '. I am not sure if 'DbTxnManager' can fix the data correctness in case of concurrent Insert and IOW on Iceberg tables. Theoretically, 'DbTxnManager' should not have any effect on iceberg tables. I would like other folks who is familiar with 'DbTxnManager' to explain its effect on Iceberg tables. On 2025/03/26 06:55:28 박근형 wrote: > Thank you for your response. Based on HIVE-28366, it appears that data > correctness can be ensured through HMS Lock when 'Insert' and 'IOW' > operations are executed concurrently. If both operations are performed by > Hive, my understanding is that the problem is resolved regardless of the > 'hive.txn.manager' setting. > > However, I encountered persistent issues in a case where an IOW operation > is executed by Hive, and the Insert operation is run by a different engine, > specifically, Trino. To resolve this issue, it seems necessary to use > 'DbTxnManager'. > > Could you kindly share your thoughts on this matter? Am I approaching this > problem correctly, or is there another solution you recommend? > > Thank you! > > 2025년 3월 26일 (수) 오후 2:42, Butao Zhang <zhangbu...@apache.org>님이 작성: > > > Hi, IMO, 'org.apache.hadoop.hive.ql.lockmgr.DbTxnManager' is used for hive > > native acid table, not for Iceberg table. > > The lock mechanism in Iceberg table is controlled by HMS Catalog or other > > Iceberg Catalog(JDBC catalog, etc). So > > 'org.apache.hadoop.hive.ql.lockmgr.DbTxnManager' should not be used in your > > case(insert overwrite iceberg table). > > > > Thanks, > > Butao Zhang > > > > On 2025/03/26 04:54:33 박근형 wrote: > > > I am experiencing a possible deadlock issue while working with Hive > > version > > > 4.1.0. The issue occurs when an 'insert overwrite' query is executed on > > an > > > Iceberg table. Specifically, I have found that if the 'hive.txn.manager' > > > setting is 'org.apache.hadoop.hive.ql.lockmgr.DbTxnManager', a deadlock > > > seems to happen. > > > > > > I would like to know whether this is a bug or a limitation. Is it that > > > 'org.apache.hadoop.hive.ql.lockmgr.DbTxnManager' should not be used in > > this > > > case? I greatly appreciate any guidance or information on this issue. > > > > > >
