[jira] [Created] (HIVE-22905) Transaction is not aborted when query cancelled, only when session is closed
Aron Hamvas created HIVE-22905: -- Summary: Transaction is not aborted when query cancelled, only when session is closed Key: HIVE-22905 URL: https://issues.apache.org/jira/browse/HIVE-22905 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 3.1.2 Reporter: Aron Hamvas Assignee: Aron Hamvas Reproduction: # Start HMS # Start HS2 # Start beeline # Execute a query and cancel query after transaction is started and before locks are acquired (yeah, that will take a debugger with breakpoints in DbLockManager) # Do not terminate Hive session DbLockManager will keep spinning in the checkLock loop as long as the Hive session is not closed since abortTxns is not invoked on HMS, and until the lock is acquired (even though the query could have been canceled long time ago). Driver only checks in close() method if there are locks acquired but it does not check if a txn is open. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22651) Fix TestCrudCompactorOnTez#testCompactionWithSchemaEvolutionNoBucketsMultipleReducers()
Aron Hamvas created HIVE-22651: -- Summary: Fix TestCrudCompactorOnTez#testCompactionWithSchemaEvolutionNoBucketsMultipleReducers() Key: HIVE-22651 URL: https://issues.apache.org/jira/browse/HIVE-22651 Project: Hive Issue Type: Sub-task Components: Test Affects Versions: 3.1.2, 3.0.0 Reporter: Aron Hamvas Assignee: Aron Hamvas Test should not assume the bucket id value with multiple reducers; we should only expect that it will not change after major compaction. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22639) Bucket file name does not match bucket id after query based major compaction
Aron Hamvas created HIVE-22639:
--
Summary: Bucket file name does not match bucket id after query
based major compaction
Key: HIVE-22639
URL: https://issues.apache.org/jira/browse/HIVE-22639
Project: Hive
Issue Type: Bug
Components: Hive
Affects Versions: 3.1.0, 3.0.0
Reporter: Aron Hamvas
Assignee: Aron Hamvas
While debugging
{{TestCrudCompactorOnTez#testCompactionWithSchemaEvolutionAndBuckets()}}, it
has come to my attention, that even though before compaction, the file name of
the single bucket in the delta directories is {{bucket_1}}, in the new
base, the name of the new single bucket file is {{bucket_0}}. At the same
time, the bucket value in the ROW__ID of the records remain the same and
suggest that the bucket id is 1.
So the bucket id and the file name do not match. This could lead to problems.
The test itself does not reveal this issue, although I think that the tests
should check this, too. At the same time, the tests assume the exact bucket id
value in cases where it cannot be predicted and fail, even though the bucket it
does not change after the compaction, so the check should really pass.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-22578) CBO genOPTree is not failsafe for CTAS and VIEW statements
Aron Hamvas created HIVE-22578: -- Summary: CBO genOPTree is not failsafe for CTAS and VIEW statements Key: HIVE-22578 URL: https://issues.apache.org/jira/browse/HIVE-22578 Project: Hive Issue Type: Bug Components: CBO Affects Versions: 3.1.2 Reporter: Aron Hamvas Assignee: Aron Hamvas If CBO fails during OT generation, it is supposed to skip CBO and fall back to SemanticAnalyzer generating the OT for the original AST. However, for CTAS and VIEW statements, the original AST is discarded and replaced with the new AST. If OT generation for the new AST fails with a SemanticException, the compilation is destined to fail completely since we cannot fall back to the original and possibly valid AST. Also, the SemanticException sometimes gets hidden and misjudged as a missing column statistics issue due to bad error handling. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22560) ACID MERGE fails if NOT NULL constraint is used for column and PDD is enabled
Aron Hamvas created HIVE-22560:
--
Summary: ACID MERGE fails if NOT NULL constraint is used for
column and PDD is enabled
Key: HIVE-22560
URL: https://issues.apache.org/jira/browse/HIVE-22560
Project: Hive
Issue Type: Bug
Components: Query Planning
Affects Versions: 3.1.2
Reporter: Aron Hamvas
If hive.optimize.ppd=true is set, ACID MERGE can fail if the column used in
matching has NOT NULL constraint.
To reproduce the issue:
{code:sql}
CREATE TABLE SIMPLECDCMASTER (CLI_NUM VARCHAR(13) NOT NULL , FIELD1
VARCHAR(100));
CREATE TABLE SIMPLECDCDELTA (CLI_NUM VARCHAR(13), FIELD1 VARCHAR(100));
insert into simplecdcdelta values (123, 'abc');
merge into simplecdcmaster as m using simplecdcdelta as d on m.cli_num =
d.cli_num
when matched then update set field1=d.field1
when not matched then insert values (d.cli_num, d.field1);
{code}
The NOT NULL constrained is then enforced in the Filter operator processing the
output of the right outer join, where, of course, the not matched records have
nulls in all columns.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-22420) Driver.releaseLocksAndCommitOrRollback is not thread safe
Aron Hamvas created HIVE-22420:
--
Summary: Driver.releaseLocksAndCommitOrRollback is not thread safe
Key: HIVE-22420
URL: https://issues.apache.org/jira/browse/HIVE-22420
Project: Hive
Issue Type: Bug
Affects Versions: 3.1.0
Reporter: Aron Hamvas
Assignee: Aron Hamvas
When a transactional query is being executed and interrupted via HS2 close
operation request, both the background pool thread executing the query and the
HttpHandler thread running the close operation logic will eventually call the
below method:
{noformat}
Driver.releaseLocksAndCommitOrRollback(commit boolean)
{noformat}
Since this method is invoked several times in both threads, it can happen that
the two threads invoke it at the same time, and due to a race condition, the
txnId field of the DbTxnManager used by both threads could be set to 0 without
actually successfully aborting the transaction.
E.g. if the two threads reach the stopHeartbeat() call at the same time, one
will set the heartbeat task to null, the other will run into a
NullPointerException and due to the unsuccessful call, set the value of txnId
to 0 before the other thread (which successfully ran the stopHeartbeat() call)
could invoke rollback on HMS with the proper txnId.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-21999) Add ABFS configuration properties to HiveConf hidden list
Aron Hamvas created HIVE-21999: -- Summary: Add ABFS configuration properties to HiveConf hidden list Key: HIVE-21999 URL: https://issues.apache.org/jira/browse/HIVE-21999 Project: Hive Issue Type: Bug Reporter: Aron Hamvas We need to make sure that sensitive information such as ABFS credentials are not logged. Properties handled in HADOOP-15745 should be added to the HiveConf hidden list. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HIVE-21777) Maven jar goal is producing warning due to missing dependency
Aron Hamvas created HIVE-21777: -- Summary: Maven jar goal is producing warning due to missing dependency Key: HIVE-21777 URL: https://issues.apache.org/jira/browse/HIVE-21777 Project: Hive Issue Type: Improvement Affects Versions: 3.1.1, 4.0.0 Reporter: Aron Hamvas Assignee: Aron Hamvas org.apache.directory.client.ldap:ldap-client-directory is a test scope dependecy. Hive is using version 0.1 but 0.1-SNAPSHOT is also there as transitive dependency (omitted for collision with 0.1 which is already there on top level) causing warning in the maven default lifecycle execution: [WARNING] The POM for org.apache.directory.client.ldap:ldap-client-api:jar:0.1-SNAPSHOT is missing, no dependency information available The warning appears in the jar goal logs and it can easily be removed by excluding this transitive dependency. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
