[
https://issues.apache.org/jira/browse/HIVE-23960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17168668#comment-17168668
]
Pravin Sinha commented on HIVE-23960:
-------------------------------------
Thanks [~pvary] ,
There are two issues here:
a) The column stat may or may not be available for a given partition at that
point of time and get_partition_by_names method at the point where the fix is,
can determine that it has the colStats for column or not. If there is no col
stat available, That's what is happening currently, which later is leading to
situation like it doesn't have enough data(colNames) to form a valid SQL.
b) Another issue here is that the exception in the JDO layer is ignored in this
case, however the objectStore.transactionState is set to ROLLBACK. Ideally,
even exception should have been propagated back to the caller.
As a result the caller currently continues with another nested transaction
based operation.
It fails finally at the outer commit layer(caller's own commitTxn flow) where
it hits unbalanced calls. But ideally the second nested transaction should not
have been attempted as the {color:#172b4d}transactionStatus{color} at that
point was {color:#172b4d}TXN_STATUS.ROLLBACK{color}.
I think case b) is close to what you are suggesting?
In this patch I have fixed the issue (a) and had a plan to file a Jira for
issue (b).
Any thoughts?
> Partition with no column statistics leads to unbalanced calls to
> openTransaction/commitTransaction error during get_partitions_by_names
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-23960
> URL: https://issues.apache.org/jira/browse/HIVE-23960
> Project: Hive
> Issue Type: Task
> Reporter: Pravin Sinha
> Assignee: Pravin Sinha
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23960.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {color:#172b4d}Creating a partition with data and adding another partition is
> leading to unbalanced calls to open/commit transaction during
> get_partitions_by_names call.{color}
> {color:#172b4d}Issue was discovered during REPL DUMP operation which uses
> this HMS call to get the metadata of partition. This error occurs when there
> is a partition with no column statistics.{color}
> {color:#172b4d}To reproduce:{color}
> {code:java}
> CREATE TABLE student_part_acid(name string, age int, gpa double) PARTITIONED
> BY (ds string) STORED AS orc;
> LOAD DATA INPATH ‘/user/hive/partDir/student_part_acid/ds=20110924’ INTO
> TABLE student_part_acid partition(ds=20110924);
> ALTER TABLE student_part_acid ADD PARTITION (ds=20110925);
> Now if we try to preform REPL DUMP it fails with this the error "Unbalanced
> calls to open/commit transaction" on the HS2 side.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
