[jira] [Commented] (HIVE-10242) ACID: insert overwrite prevents create table command
[ https://issues.apache.org/jira/browse/HIVE-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15379099#comment-15379099 ] Qiuzhuang Lian commented on HIVE-10242: --- In our production env, we still got this issue with hive-1.2.1. The error stack thrown from HS2 is as follows, 2016-07-15 04:30:00,028 ERROR [HiveServer2-Background-Pool: Thread-2903721]: lockmgr.DbTxnManager (DbTxnManager.java:heartbeat(295)) - Unable to find lock 214155 2016-07-15 04:30:00,028 WARN [HiveServer2-Background-Pool: Thread-2903721]: exec.Heartbeater (Heartbeater.java:heartbeat(83)) - Failed trying to heartbeat No record of lock could be found, may have timed out 2016-07-15 04:30:00,031 ERROR [HiveServer2-Background-Pool: Thread-2903721]: exec.Task (SessionState.java:printError(960)) - Ended Job = job_140599841_119253 with exception 'java.io.IOException(org.apache.hadoop.hive.ql.lockmgr.LockException: No record of lock could be found, may have timed out)' java.io.IOException: org.apache.hadoop.hive.ql.lockmgr.LockException: No record of lock could be found, may have timed out at org.apache.hadoop.hive.ql.exec.Heartbeater.heartbeat(Heartbeater.java:84) at org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:242) at org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:549) at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:437) at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1653) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1412) > ACID: insert overwrite prevents create table command > > > Key: HIVE-10242 > URL: https://issues.apache.org/jira/browse/HIVE-10242 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman > Fix For: 1.2.0 > > Attachments: HIVE-10242.2.patch, HIVE-10242.3.patch, > HIVE-10242.4.patch, HIVE-10242.patch > > > 1. insert overwirte table DB.T1 select ... from T2: this takes X lock on > DB.T1 and S lock on T2. > X lock makes sense because we don't want anyone reading T1 while it's > overwritten. S lock on T2 prevents if from being dropped while the query is > in progress. > 2. create table DB.T3: takes S lock on DB. > This S lock gets blocked by X lock on T1. S lock prevents the DB from being > dropped while create table is executed. > If the insert statement is long running, this blocks DDL ops on the same > database. This is a usability issue. > There is no good reason why X lock on a table within a DB and S lock on DB > should be in conflict. > (this is different from a situation where X lock is on a partition and S lock > is on the table to which this partition belongs. Here it makes sense. > Basically there is no SQL way to address all tables in a DB but you can > easily refer to all partitions of a table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10242) ACID: insert overwrite prevents create table command
[ https://issues.apache.org/jira/browse/HIVE-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14498411#comment-14498411 ] Eugene Koifman commented on HIVE-10242: --- The fix also logging improvements and a fix to the following situation where a lock (by client3) could be acquired when it should not be. client1: update table T client2: select * from T client3: update table T with all 3 clients executing concurrently and starting in above order, client 3 should be blocked until client1 finishes. ACID: insert overwrite prevents create table command Key: HIVE-10242 URL: https://issues.apache.org/jira/browse/HIVE-10242 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 1.0.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 1.2.0 Attachments: HIVE-10242.2.patch, HIVE-10242.3.patch, HIVE-10242.4.patch, HIVE-10242.patch 1. insert overwirte table DB.T1 select ... from T2: this takes X lock on DB.T1 and S lock on T2. X lock makes sense because we don't want anyone reading T1 while it's overwritten. S lock on T2 prevents if from being dropped while the query is in progress. 2. create table DB.T3: takes S lock on DB. This S lock gets blocked by X lock on T1. S lock prevents the DB from being dropped while create table is executed. If the insert statement is long running, this blocks DDL ops on the same database. This is a usability issue. There is no good reason why X lock on a table within a DB and S lock on DB should be in conflict. (this is different from a situation where X lock is on a partition and S lock is on the table to which this partition belongs. Here it makes sense. Basically there is no SQL way to address all tables in a DB but you can easily refer to all partitions of a table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10242) ACID: insert overwrite prevents create table command
[ https://issues.apache.org/jira/browse/HIVE-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14495826#comment-14495826 ] Alan Gates commented on HIVE-10242: --- +1 ACID: insert overwrite prevents create table command Key: HIVE-10242 URL: https://issues.apache.org/jira/browse/HIVE-10242 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 1.0.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-10242.2.patch, HIVE-10242.3.patch, HIVE-10242.4.patch, HIVE-10242.patch 1. insert overwirte table DB.T1 select ... from T2: this takes X lock on DB.T1 and S lock on T2. X lock makes sense because we don't want anyone reading T1 while it's overwritten. S lock on T2 prevents if from being dropped while the query is in progress. 2. create table DB.T3: takes S lock on DB. This S lock gets blocked by X lock on T1. S lock prevents the DB from being dropped while create table is executed. If the insert statement is long running, this blocks DDL ops on the same database. This is a usability issue. There is no good reason why X lock on a table within a DB and S lock on DB should be in conflict. (this is different from a situation where X lock is on a partition and S lock is on the table to which this partition belongs. Here it makes sense. Basically there is no SQL way to address all tables in a DB but you can easily refer to all partitions of a table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10242) ACID: insert overwrite prevents create table command
[
https://issues.apache.org/jira/browse/HIVE-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14495639#comment-14495639
]
Hive QA commented on HIVE-10242:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12725430/HIVE-10242.4.patch
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 8695 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3435/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3435/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3435/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12725430 - PreCommit-HIVE-TRUNK-Build
ACID: insert overwrite prevents create table command
Key: HIVE-10242
URL: https://issues.apache.org/jira/browse/HIVE-10242
Project: Hive
Issue Type: Bug
Components: Transactions
Affects Versions: 1.0.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Attachments: HIVE-10242.2.patch, HIVE-10242.3.patch,
HIVE-10242.4.patch, HIVE-10242.patch
1. insert overwirte table DB.T1 select ... from T2: this takes X lock on
DB.T1 and S lock on T2.
X lock makes sense because we don't want anyone reading T1 while it's
overwritten. S lock on T2 prevents if from being dropped while the query is
in progress.
2. create table DB.T3: takes S lock on DB.
This S lock gets blocked by X lock on T1. S lock prevents the DB from being
dropped while create table is executed.
If the insert statement is long running, this blocks DDL ops on the same
database. This is a usability issue.
There is no good reason why X lock on a table within a DB and S lock on DB
should be in conflict.
(this is different from a situation where X lock is on a partition and S lock
is on the table to which this partition belongs. Here it makes sense.
Basically there is no SQL way to address all tables in a DB but you can
easily refer to all partitions of a table)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10242) ACID: insert overwrite prevents create table command
[ https://issues.apache.org/jira/browse/HIVE-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492810#comment-14492810 ] Eugene Koifman commented on HIVE-10242: --- 1. The WebHCat stuff is there because I used it for testing. It has 1 line script to start hive services, etc. So the additions there are to be able to run it with MySQL based metastore to test concurrency. So this is not a product change but it's relevant. I can split it into a separate patch if necessary. 2. The default in LockInfo constructor matches previous implementation. W/o the default, switch would just fall off the end and type and state will get default values, i.e. null. 3. I find immutable objects easier to understand. There is no specific functional reason in this case. 4. lockTypeComparator static final in LockInfoComparator - there is exactly the same number of instances of both, that is 1 per TxnHandler.checkLock(). Also, checkLock() issues SQL queries and mostly likely is being called over the network from remote client. It seems unlikely to make any noticeable difference (even if we knew for a fact that the compiler won't inline it). So I thought code readability was more important here. 5. The changes in DbLockManager modifies package level method to make testing easier. This allows test code to attempt to acquire locks but not block if they cannot be acquired. It's the same idea as TxnHandler.numLocksInLockTable(). ACID: insert overwrite prevents create table command Key: HIVE-10242 URL: https://issues.apache.org/jira/browse/HIVE-10242 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 1.0.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-10242.2.patch, HIVE-10242.3.patch, HIVE-10242.patch 1. insert overwirte table DB.T1 select ... from T2: this takes X lock on DB.T1 and S lock on T2. X lock makes sense because we don't want anyone reading T1 while it's overwritten. S lock on T2 prevents if from being dropped while the query is in progress. 2. create table DB.T3: takes S lock on DB. This S lock gets blocked by X lock on T1. S lock prevents the DB from being dropped while create table is executed. If the insert statement is long running, this blocks DDL ops on the same database. This is a usability issue. There is no good reason why X lock on a table within a DB and S lock on DB should be in conflict. (this is different from a situation where X lock is on a partition and S lock is on the table to which this partition belongs. Here it makes sense. Basically there is no SQL way to address all tables in a DB but you can easily refer to all partitions of a table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10242) ACID: insert overwrite prevents create table command
[ https://issues.apache.org/jira/browse/HIVE-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492301#comment-14492301 ] Alan Gates commented on HIVE-10242: --- Why is the templeton stuff in this patch? I suspect it doesn't belong here. TxnHandler, in the LockInfo constructor. If you're going to add defaults for those switch cases they should throw errors rather than return null. Why did you change the fields in LockInfo to final? It's fine, I'm just curious. Why make a new class LockTypeComparator? This is in the tight loop of determining locks, so we want it to go as fast as we can. Rather than requiring an object creation and function call every time through this it's better to inline that code. I don't understand the changes to DbLockManager. What is the purpose of the changes here. ACID: insert overwrite prevents create table command Key: HIVE-10242 URL: https://issues.apache.org/jira/browse/HIVE-10242 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 1.0.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-10242.2.patch, HIVE-10242.3.patch, HIVE-10242.patch 1. insert overwirte table DB.T1 select ... from T2: this takes X lock on DB.T1 and S lock on T2. X lock makes sense because we don't want anyone reading T1 while it's overwritten. S lock on T2 prevents if from being dropped while the query is in progress. 2. create table DB.T3: takes S lock on DB. This S lock gets blocked by X lock on T1. S lock prevents the DB from being dropped while create table is executed. If the insert statement is long running, this blocks DDL ops on the same database. This is a usability issue. There is no good reason why X lock on a table within a DB and S lock on DB should be in conflict. (this is different from a situation where X lock is on a partition and S lock is on the table to which this partition belongs. Here it makes sense. Basically there is no SQL way to address all tables in a DB but you can easily refer to all partitions of a table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10242) ACID: insert overwrite prevents create table command
[ https://issues.apache.org/jira/browse/HIVE-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491723#comment-14491723 ] Eugene Koifman commented on HIVE-10242: --- the set of failures is almost identical to HIVE-10148 but the patches are not related [~alangates], could you review please? ACID: insert overwrite prevents create table command Key: HIVE-10242 URL: https://issues.apache.org/jira/browse/HIVE-10242 Project: Hive Issue Type: Bug Components: Transactions Affects Versions: 1.0.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Attachments: HIVE-10242.2.patch, HIVE-10242.3.patch, HIVE-10242.patch 1. insert overwirte table DB.T1 select ... from T2: this takes X lock on DB.T1 and S lock on T2. X lock makes sense because we don't want anyone reading T1 while it's overwritten. S lock on T2 prevents if from being dropped while the query is in progress. 2. create table DB.T3: takes S lock on DB. This S lock gets blocked by X lock on T1. S lock prevents the DB from being dropped while create table is executed. If the insert statement is long running, this blocks DDL ops on the same database. This is a usability issue. There is no good reason why X lock on a table within a DB and S lock on DB should be in conflict. (this is different from a situation where X lock is on a partition and S lock is on the table to which this partition belongs. Here it makes sense. Basically there is no SQL way to address all tables in a DB but you can easily refer to all partitions of a table) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10242) ACID: insert overwrite prevents create table command
[
https://issues.apache.org/jira/browse/HIVE-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491670#comment-14491670
]
Hive QA commented on HIVE-10242:
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12724771/HIVE-10242.3.patch
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 8678 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more
- did not produce a TEST-*.xml file
TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a
TEST-*.xml file
TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did
not produce a TEST-*.xml file
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testNewConnectionConfiguration
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3398/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3398/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3398/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12724771 - PreCommit-HIVE-TRUNK-Build
ACID: insert overwrite prevents create table command
Key: HIVE-10242
URL: https://issues.apache.org/jira/browse/HIVE-10242
Project: Hive
Issue Type: Bug
Components: Transactions
Affects Versions: 1.0.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Attachments: HIVE-10242.2.patch, HIVE-10242.3.patch, HIVE-10242.patch
1. insert overwirte table DB.T1 select ... from T2: this takes X lock on
DB.T1 and S lock on T2.
X lock makes sense because we don't want anyone reading T1 while it's
overwritten. S lock on T2 prevents if from being dropped while the query is
in progress.
2. create table DB.T3: takes S lock on DB.
This S lock gets blocked by X lock on T1. S lock prevents the DB from being
dropped while create table is executed.
If the insert statement is long running, this blocks DDL ops on the same
database. This is a usability issue.
There is no good reason why X lock on a table within a DB and S lock on DB
should be in conflict.
(this is different from a situation where X lock is on a partition and S lock
is on the table to which this partition belongs. Here it makes sense.
Basically there is no SQL way to address all tables in a DB but you can
easily refer to all partitions of a table)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
