[
https://issues.apache.org/jira/browse/HIVE-28711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fatih updated HIVE-28711:
-------------------------
Description:
Hi,
I am working on a POC for setting up Trino/HMS/HDFS. I patched the Ranger Hive
plugin to make it work with the HMS.
When I try to execute CREATE TABLE on the Iceberg catalog, I encounter the
following error:
{code:java}
2025-01-15T10:16:35,289 DEBUG [Metastore-Handler-Pool: Thread-169]
ipc.ProtobufRpcEngine2: Call: getListing took 1ms
2025-01-15T10:16:35,289 INFO [Metastore-Handler-Pool: Thread-169]
utils.MetaStoreServerUtils: Updating table stats for test92
2025-01-15T10:16:35,289 INFO [Metastore-Handler-Pool: Thread-169]
utils.MetaStoreServerUtils: Updated size of table test92 to 6430
2025-01-15T10:16:35,290 DEBUG [Metastore-Handler-Pool: Thread-169]
metastore.ObjectStore: Open transaction: count = 1, isActive = true,
isolationLevel = read-committed
2025-01-15T10:16:35,290 DEBUG [Metastore-Handler-Pool: Thread-169]
metastore.ObjectStore: Open transaction: count = 2, isActive = true,
isolationLevel = read-committed
2025-01-15T10:16:35,291 DEBUG [Metastore-Handler-Pool: Thread-169]
metastore.ObjectStore: Commit transaction: count = 1, isactive true
2025-01-15T10:16:35,300 DEBUG [Metastore-Handler-Pool: Thread-169]
metastore.ObjectStore: Commit transaction: count = 0, isactive true
2025-01-15T10:16:35,310 DEBUG [Metastore-Handler-Pool: Thread-169]
pool.PoolBase: objectstore - Reset (autoCommit) on connection
org.postgresql.jdbc.PgConnection@4f5f7722
2025-01-15T10:16:35,312 DEBUG [Metastore-Handler-Pool: Thread-169]
pool.PoolBase: objectstore-secondary - Reset (autoCommit) on connection
org.postgresql.jdbc.PgConnection@7e95525
2025-01-15T10:16:35,313 DEBUG [Metastore-Handler-Pool: Thread-169]
pool.PoolBase: objectstore-secondary - Reset (autoCommit) on connection
org.postgresql.jdbc.PgConnection@7e95525
2025-01-15T10:16:35,314 ERROR [Metastore-Handler-Pool: Thread-169]
metastore.ObjectStore: Unbalanced calls to open/commit Transaction
java.lang.RuntimeException: commitTransaction was called but
openTransactionCalls = 0. This probably indicates that there are unbalanced
calls to openTransaction/commitTransaction
at
org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:664)
~[hive-exec-4.0.1.jar:4.0.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_342]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_342]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_342]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_342]
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
~[hive-exec-4.0.1.jar:4.0.1]
at com.sun.proxy.$Proxy27.commitTransaction(Unknown Source) ~[?:?]
at
org.apache.hadoop.hive.metastore.HMSHandler.create_table_core(HMSHandler.java:2516)
~[hive-exec-4.0.1.jar:4.0.1]
at
org.apache.hadoop.hive.metastore.HMSHandler.create_table_req(HMSHandler.java:2624)
~[hive-exec-4.0.1.jar:4.0.1]
{code}
All privileged operations except CREATE TABLE, CREATE TABLE AS, and CREATE VIEW
work as expected. Additionally, when I attempt to create a table in the Trino
Hive catalog, it is created without any errors.
I suspect the issue is related to HMS behavior during privileged Iceberg table
creation.
Since this might also be related to Trino, I have opened a Trino issue as well
(https://github.com/trinodb/trino/issues/24705 )
However, I am not entirely sure which component is causing the issue. Could you
please assist me with this?
Thank you!
was:
Hi,
I am working on a POC for setting up Trino/HMS/HDFS. I patched the Ranger Hive
plugin to make it work with the HMS.
When I try to execute CREATE TABLE on the Iceberg catalog, I encounter the
following error:
{code:java}
2025-01-15T10:16:35,289 DEBUG [Metastore-Handler-Pool: Thread-169]
ipc.ProtobufRpcEngine2: Call: getListing took 1ms
2025-01-15T10:16:35,289 INFO [Metastore-Handler-Pool: Thread-169]
utils.MetaStoreServerUtils: Updating table stats for test92
2025-01-15T10:16:35,289 INFO [Metastore-Handler-Pool: Thread-169]
utils.MetaStoreServerUtils: Updated size of table test92 to 6430
2025-01-15T10:16:35,290 DEBUG [Metastore-Handler-Pool: Thread-169]
metastore.ObjectStore: Open transaction: count = 1, isActive = true,
isolationLevel = read-committed
2025-01-15T10:16:35,290 DEBUG [Metastore-Handler-Pool: Thread-169]
metastore.ObjectStore: Open transaction: count = 2, isActive = true,
isolationLevel = read-committed
2025-01-15T10:16:35,291 DEBUG [Metastore-Handler-Pool: Thread-169]
metastore.ObjectStore: Commit transaction: count = 1, isactive true
2025-01-15T10:16:35,300 DEBUG [Metastore-Handler-Pool: Thread-169]
metastore.ObjectStore: Commit transaction: count = 0, isactive true
2025-01-15T10:16:35,310 DEBUG [Metastore-Handler-Pool: Thread-169]
pool.PoolBase: objectstore - Reset (autoCommit) on connection
org.postgresql.jdbc.PgConnection@4f5f7722
2025-01-15T10:16:35,312 DEBUG [Metastore-Handler-Pool: Thread-169]
pool.PoolBase: objectstore-secondary - Reset (autoCommit) on connection
org.postgresql.jdbc.PgConnection@7e95525
2025-01-15T10:16:35,313 DEBUG [Metastore-Handler-Pool: Thread-169]
pool.PoolBase: objectstore-secondary - Reset (autoCommit) on connection
org.postgresql.jdbc.PgConnection@7e95525
2025-01-15T10:16:35,314 ERROR [Metastore-Handler-Pool: Thread-169]
metastore.ObjectStore: Unbalanced calls to open/commit Transaction
java.lang.RuntimeException: commitTransaction was called but
openTransactionCalls = 0. This probably indicates that there are unbalanced
calls to openTransaction/commitTransaction
at
org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:664)
~[hive-exec-4.0.1.jar:4.0.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_342]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_342]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_342]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_342]
at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
~[hive-exec-4.0.1.jar:4.0.1]
at com.sun.proxy.$Proxy27.commitTransaction(Unknown Source) ~[?:?]
at
org.apache.hadoop.hive.metastore.HMSHandler.create_table_core(HMSHandler.java:2516)
~[hive-exec-4.0.1.jar:4.0.1]
at
org.apache.hadoop.hive.metastore.HMSHandler.create_table_req(HMSHandler.java:2624)
~[hive-exec-4.0.1.jar:4.0.1]
{code}
All privileged operations except CREATE TABLE, CREATE TABLE AS, and CREATE VIEW
work as expected. Additionally, when I attempt to create a table in the Trino
Hive catalog, it is created without any errors.
*I suspect the issue is related to HMS behavior during privileged Iceberg table
creation.*
Since this might also be related to Trino, I have opened a Trino issue as well
(https://github.com/trinodb/trino/issues/24705 )
However, I am not entirely sure which component is causing the issue. Could you
please assist me with this?
Thank you!
> unbalanced calls to openTransaction/commitTransaction when creating iceberg
> table
> ----------------------------------------------------------------------------------
>
> Key: HIVE-28711
> URL: https://issues.apache.org/jira/browse/HIVE-28711
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 4.0.1
> Environment: HMS: 4.0.1
> Hadoop: 4.3.1
> Trino: 468
> Ranger: 3.0.0
> HMS&HDFS Kerberos enabled
> Reporter: Fatih
> Priority: Minor
> Attachments: hms_error.log
>
>
> Hi,
> I am working on a POC for setting up Trino/HMS/HDFS. I patched the Ranger
> Hive plugin to make it work with the HMS.
> When I try to execute CREATE TABLE on the Iceberg catalog, I encounter the
> following error:
> {code:java}
> 2025-01-15T10:16:35,289 DEBUG [Metastore-Handler-Pool: Thread-169]
> ipc.ProtobufRpcEngine2: Call: getListing took 1ms
> 2025-01-15T10:16:35,289 INFO [Metastore-Handler-Pool: Thread-169]
> utils.MetaStoreServerUtils: Updating table stats for test92
> 2025-01-15T10:16:35,289 INFO [Metastore-Handler-Pool: Thread-169]
> utils.MetaStoreServerUtils: Updated size of table test92 to 6430
> 2025-01-15T10:16:35,290 DEBUG [Metastore-Handler-Pool: Thread-169]
> metastore.ObjectStore: Open transaction: count = 1, isActive = true,
> isolationLevel = read-committed
> 2025-01-15T10:16:35,290 DEBUG [Metastore-Handler-Pool: Thread-169]
> metastore.ObjectStore: Open transaction: count = 2, isActive = true,
> isolationLevel = read-committed
> 2025-01-15T10:16:35,291 DEBUG [Metastore-Handler-Pool: Thread-169]
> metastore.ObjectStore: Commit transaction: count = 1, isactive true
> 2025-01-15T10:16:35,300 DEBUG [Metastore-Handler-Pool: Thread-169]
> metastore.ObjectStore: Commit transaction: count = 0, isactive true
> 2025-01-15T10:16:35,310 DEBUG [Metastore-Handler-Pool: Thread-169]
> pool.PoolBase: objectstore - Reset (autoCommit) on connection
> org.postgresql.jdbc.PgConnection@4f5f7722
> 2025-01-15T10:16:35,312 DEBUG [Metastore-Handler-Pool: Thread-169]
> pool.PoolBase: objectstore-secondary - Reset (autoCommit) on connection
> org.postgresql.jdbc.PgConnection@7e95525
> 2025-01-15T10:16:35,313 DEBUG [Metastore-Handler-Pool: Thread-169]
> pool.PoolBase: objectstore-secondary - Reset (autoCommit) on connection
> org.postgresql.jdbc.PgConnection@7e95525
> 2025-01-15T10:16:35,314 ERROR [Metastore-Handler-Pool: Thread-169]
> metastore.ObjectStore: Unbalanced calls to open/commit Transaction
> java.lang.RuntimeException: commitTransaction was called but
> openTransactionCalls = 0. This probably indicates that there are unbalanced
> calls to openTransaction/commitTransaction
> at
> org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:664)
> ~[hive-exec-4.0.1.jar:4.0.1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_342]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_342]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_342]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_342]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> ~[hive-exec-4.0.1.jar:4.0.1]
> at com.sun.proxy.$Proxy27.commitTransaction(Unknown Source) ~[?:?]
> at
> org.apache.hadoop.hive.metastore.HMSHandler.create_table_core(HMSHandler.java:2516)
> ~[hive-exec-4.0.1.jar:4.0.1]
> at
> org.apache.hadoop.hive.metastore.HMSHandler.create_table_req(HMSHandler.java:2624)
> ~[hive-exec-4.0.1.jar:4.0.1]
> {code}
> All privileged operations except CREATE TABLE, CREATE TABLE AS, and CREATE
> VIEW work as expected. Additionally, when I attempt to create a table in the
> Trino Hive catalog, it is created without any errors.
> I suspect the issue is related to HMS behavior during privileged Iceberg
> table creation.
> Since this might also be related to Trino, I have opened a Trino issue as
> well (https://github.com/trinodb/trino/issues/24705 )
> However, I am not entirely sure which component is causing the issue. Could
> you please assist me with this?
> Thank you!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
