Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Thank you for the PR, Owen! I've added a few comments, please review them when you have time. Regards, Denys

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Hi, I have submitted the bug-fix PR we discussed. It seems that the GitHub workflow needs approval. Is there anything else I need to do? Regards, Owen 2025년 3월 27일 (목) 오후 6:11, Denys Kuzmenko 님이 작성: > Great, thank you! >

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Great, thank you!

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Note: HiveIcebergMetaHook acquires an HMS lock regardless of `iceberg.engine.hive.lock-enabled` config - needs to be refactored.

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Yes, I'd like to submit a bug-fix PR. Thank you for your proposal. 2025년 3월 27일 목요일, Denys Kuzmenko 님이 작성: > Sure, go ahead. > > To confirm, you'd like to submit a bug-fix PR as well, right? I started > looking into it, and I think a quick fix might look like: > > private HiveLock lockObject(or

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Sure, go ahead. To confirm, you'd like to submit a bug-fix PR as well, right? I started looking into it, and I think a quick fix might look like: private HiveLock lockObject(org.apache.hadoop.hive.metastore.api.Table hmsTable) { if (hiveLockEnabled(hmsTable, conf)) { return new Meta

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Dear Denys, Thank you for your confirmation. Im considering reporting a bug on JIRA and creating a PR. Please advise. Regards, Owen 2025년 3월 27일 (목) 오후 4:52, Denys Kuzmenko 님이 작성: > Note: HiveIcebergMetaHook acquires an HMS lock regardless of > `iceberg.engine.hive.lock-enabled` config - needs

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Hi Owen, It seems like a bug, but I need to check it more closely. There is no need for a stats task to take an EXCL_WRITE lock in a HiveIcebergMetaHook. Thanks, Denys

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Thank you for your response. I understand that even if Hive and Trino use the same Hive Metastore as a Catalog, additional implementation in TrinoCatalog is necessary to ensure data correctness. To clarify, the deadlock issue I am referring to is not a situation where the IOW lock cannot be obtain

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Hi, What do you mean by "possible deadlock issue"? - IOW is stuck trying to acquire a write lock (could you check this in HS2 logs)? - What is the value of `hive.txn.ext.locking.enabled` config? If 'false', pessimistic locking won't be enabled on iceberg tables. - Are you using any Hive-mana

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

I have not checked the detail about HVIE-28366. Maybe Denys can give more explanations. But for your case: For iceberg table, IOW in Hive and Insert in Trino at the same time. If you want to ensure data correctness, You should ensure that Hive/Hs2 and Trino use the same Iceberg catalog. Howev

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Thank you for your response. Based on HIVE-28366, it appears that data correctness can be ensured through HMS Lock when 'Insert' and 'IOW' operations are executed concurrently. If both operations are performed by Hive, my understanding is that the problem is resolved regardless of the 'hive.txn.man

Re: Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

Hi, IMO, 'org.apache.hadoop.hive.ql.lockmgr.DbTxnManager' is used for hive native acid table, not for Iceberg table. The lock mechanism in Iceberg table is controlled by HMS Catalog or other Iceberg Catalog(JDBC catalog, etc). So 'org.apache.hadoop.hive.ql.lockmgr.DbTxnManager' should not be us

Inquiry about Possible Deadlock Problem in Hive 4.1.0 with Insert overwrite Query on Iceberg Table

I am experiencing a possible deadlock issue while working with Hive version 4.1.0. The issue occurs when an 'insert overwrite' query is executed on an Iceberg table. Specifically, I have found that if the 'hive.txn.manager' setting is 'org.apache.hadoop.hive.ql.lockmgr.DbTxnManager', a deadlock see

Re: Problem in Hive Right Outer Join

Could you upload DDLs for those tables? Thanks. 2013/8/8 Jérôme Verdier : > Hi, > > I encountered a problem with Right Outer Join in Hive. > > Here is where is the problem : > > FROM default.ca ca > JOIN default.kpi_magasin mtransf > ON mtransf.co_societe = (CASE WHEN

Problem in Hive Right Outer Join

Hi, I encountered a problem with Right Outer Join in Hive. Here is where is the problem : FROM default.ca ca JOIN default.kpi_magasin mtransf ON mtransf.co_societe = (CASE WHEN ca.co_societe = 1 THEN 1 ELSE 2 END) AND mtransf.id_magasin = ca.id_magasin

Re: problem in Hive performance

Yes, when you set a property via the 'set' command on the Hive CLI, they live for the life of that particular client session. There is not 'golden rule' that increases your performance; it all depends on your installation, data and query pattern. Based on these you can consider leveraging some joi

RE: problem in hive

[mailto:vikas.srivast...@one97.net] Sent: Tuesday, October 11, 2011 8:57 PM To: user@hive.apache.org Subject: RE: problem in hive Yup!!! With Regards Vikas Srivastava DWH & Analytics Team M: +91 9560885900 P: + 91 120 4770102 Email: vikas.srivast...@one97.net W: www.one97world.com O

RE: problem in hive

To: user@hive.apache.org Subject: Re: problem in hive Is it hive database created in MySQL ?? On Tue, Oct 11, 2011 at 5:55 PM, Vikas Srivastava wrote: that also i have dine.. i put msql connector in that lib On Tue, Oct 11, 2011 at 5:39 PM, Ankit Jain wrote: Hello Vikas, I think yo

Re: problem in hive

>>>> Mail: harold.c...@sunplus-technology.com >>>> MSN: chen0...@hotmail.com >>>> SKYPE: chen0727 >>>> Mobil: 886-937545215 >>>> Tel: 886-2-8798-2988 #222 >>>> Fax:886-2-8751-5499 >>>> >>>> -Origina

Re: problem in hive

t; >>> >>> >>> Best Regard >>> Mail: harold.c...@sunplus-technology.com >>> MSN: chen0...@hotmail.com >>> SKYPE: chen0727 >>> Mobil: 886-937545215 >>> Tel: 886-2-8798-2988 #222 >>> Fax:886-2-8751-5499 >>>

Re: problem in hive

t;> Tel: 886-2-8798-2988 #222 >> Fax:886-2-8751-5499 >> >> -Original Message- >> From: vikas srivastava [mailto:vikas.srivast...@one97.net] >> Sent: Tuesday, October 11, 2011 3:29 PM >> To: user@hive.apache.org >> Subject: problem in hive >>

Re: problem in hive

gt; SKYPE: chen0727 > Mobil: 886-937545215 > Tel: 886-2-8798-2988 #222 > Fax:886-2-8751-5499 > > -Original Message- > From: vikas srivastava [mailto:vikas.srivast...@one97.net] > Sent: Tuesday, October 11, 2011 3:29 PM > To: user@hive.apache.org > Subject: pr

RE: problem in hive

srivastava [mailto:vikas.srivast...@one97.net] Sent: Tuesday, October 11, 2011 3:29 PM To: user@hive.apache.org Subject: problem in hive Hi All, I facing a problem like I have installed hive on a new server and provide the connectivity with the namenode, Create a user in mysql and give the proper

problem in hive

Hi All, I facing a problem like I have installed hive on a new server and provide the connectivity with the namenode, Create a user in mysql and give the proper privileges to that user, Put that user in hive-site.xml and now when I fired a query like Show tables; Not showing any table when I

problem in hive to access metadata

Hey Folks, I m using hive 0.7 and hardtop 0.20.2 I just configured a new server and installed hardtop then hive and grant all permissions to this server, so that it can access metadata from Name node, but now watt happening is its not able to run any query, I can create table but cant read those

Fwd: Problem in hive

need help on this -- Forwarded message -- From: Vikas Srivastava Date: Thu, Aug 25, 2011 at 6:37 PM Subject: Re: Problem in hive To: user@hive.apache.org Hey Ashu!!1 i have given full permission to hadoop user on new server(A) with user name and password. it can only read

Re: Problem in hive

etails like which version u r using and what >>>> sequence of queries u have executed. >>>> >>>> When I checked the trunk code this exception will come when getCols() >>>> returns null. Check u r metadata is in good state or not. >>>> >>>

Re: Problem in hive

ost some more details like which version u r using and what >>> sequence of queries u have executed. >>> >>> When I checked the trunk code this exception will come when getCols() >>> returns null. Check u r metadata is in good state or not. >>&g

Re: Problem in hive

*** >> >> When I checked the trunk code this exception will come when getCols() >> returns null. Check u r metadata is in good state or not. >> >> Thanks >> >> Chinna Rao Lalam >> >> ---------- Forwarded message -- >> Fr

Re: Problem in hive

*Vikas Srivastava* > Date: Tue, Aug 23, 2011 at 7:26 PM > Subject: Problem in hive > To: user@hive.apache.org > > > HI team, > > > i m facing this problem. > > show tables is running fine but when i run below query. > > hive> se

RE: Problem in hive

message -- From: Vikas Srivastava Date: Tue, Aug 23, 2011 at 7:26 PM Subject: Problem in hive To: user@hive.apache.org HI team, i m facing this problem. show tables is running fine but when i run below query. hive> select * from aircel_obd; FAILED: Hive Internal Er

Fwd: Problem in hive

-- Forwarded message -- From: Vikas Srivastava Date: Tue, Aug 23, 2011 at 7:26 PM Subject: Problem in hive To: user@hive.apache.org HI team, i m facing this problem. show tables is running fine but when i run below query. hive> select * from aircel_obd; FAILED: Hive Inter

Problem in hive

HI team, i m facing this problem. show tables is running fine but when i run below query. hive> select * from aircel_obd; FAILED: Hive Internal Error: java.lang.NullPointerException(null) java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java