[jira] [Commented] (HIVE-12266) When client exists abnormally, it doesn't release ACID locks
[ https://issues.apache.org/jira/browse/HIVE-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15506736#comment-15506736 ] Chaoyu Tang commented on HIVE-12266: [~wzheng], To my understand, this JIRA could only address the issue for CLI, but not for beeline. Since the shutdown hook could only be invoked when JVM running the Driver is shutdown, in beeline case, the beeline is running in a different JVM from HS2. Is that correct? > When client exists abnormally, it doesn't release ACID locks > > > Key: HIVE-12266 > URL: https://issues.apache.org/jira/browse/HIVE-12266 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-12266.1.patch, HIVE-12266.2.patch, > HIVE-12266.3.patch, HIVE-12266.branch-1.patch > > > if you start Hive CLI (locking enabled) and run some command that acquires > locks and ^C the shell before command completes the locks for the command > remain until they timeout. > I believe Beeline has the same issue. > Need to add proper hooks to release locks when command dies. (As much as > possible) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12266) When client exists abnormally, it doesn't release ACID locks
[ https://issues.apache.org/jira/browse/HIVE-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15038558#comment-15038558 ] Thejas M Nair commented on HIVE-12266: -- Adding 2.0.0 as fix version as it was committed to master as well. > When client exists abnormally, it doesn't release ACID locks > > > Key: HIVE-12266 > URL: https://issues.apache.org/jira/browse/HIVE-12266 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Fix For: 1.3.0, 2.0.0 > > Attachments: HIVE-12266.1.patch, HIVE-12266.2.patch, > HIVE-12266.3.patch, HIVE-12266.branch-1.patch > > > if you start Hive CLI (locking enabled) and run some command that acquires > locks and ^C the shell before command completes the locks for the command > remain until they timeout. > I believe Beeline has the same issue. > Need to add proper hooks to release locks when command dies. (As much as > possible) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12266) When client exists abnormally, it doesn't release ACID locks
[ https://issues.apache.org/jira/browse/HIVE-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987884#comment-14987884 ] Eugene Koifman commented on HIVE-12266: --- committed to master: https://github.com/apache/hive/commit/595fa9988fcb3e67b60845b44e1df4cc49ce38b2 patch doesn't apply on branch-1 > When client exists abnormally, it doesn't release ACID locks > > > Key: HIVE-12266 > URL: https://issues.apache.org/jira/browse/HIVE-12266 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12266.1.patch, HIVE-12266.2.patch, > HIVE-12266.3.patch > > > if you start Hive CLI (locking enabled) and run some command that acquires > locks and ^C the shell before command completes the locks for the command > remain until they timeout. > I believe Beeline has the same issue. > Need to add proper hooks to release locks when command dies. (As much as > possible) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12266) When client exists abnormally, it doesn't release ACID locks
[ https://issues.apache.org/jira/browse/HIVE-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987975#comment-14987975 ] Eugene Koifman commented on HIVE-12266: --- committed to branch-1 https://github.com/apache/hive/commit/87e5b4ef2f3a05f1c902b85588d1d96f8fe560b9 > When client exists abnormally, it doesn't release ACID locks > > > Key: HIVE-12266 > URL: https://issues.apache.org/jira/browse/HIVE-12266 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Fix For: 1.3.0 > > Attachments: HIVE-12266.1.patch, HIVE-12266.2.patch, > HIVE-12266.3.patch, HIVE-12266.branch-1.patch > > > if you start Hive CLI (locking enabled) and run some command that acquires > locks and ^C the shell before command completes the locks for the command > remain until they timeout. > I believe Beeline has the same issue. > Need to add proper hooks to release locks when command dies. (As much as > possible) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12266) When client exists abnormally, it doesn't release ACID locks
[ https://issues.apache.org/jira/browse/HIVE-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14985937#comment-14985937 ] Eugene Koifman commented on HIVE-12266: --- +1 > When client exists abnormally, it doesn't release ACID locks > > > Key: HIVE-12266 > URL: https://issues.apache.org/jira/browse/HIVE-12266 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12266.1.patch, HIVE-12266.2.patch, > HIVE-12266.3.patch > > > if you start Hive CLI (locking enabled) and run some command that acquires > locks and ^C the shell before command completes the locks for the command > remain until they timeout. > I believe Beeline has the same issue. > Need to add proper hooks to release locks when command dies. (As much as > possible) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12266) When client exists abnormally, it doesn't release ACID locks
[ https://issues.apache.org/jira/browse/HIVE-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14985883#comment-14985883 ] Wei Zheng commented on HIVE-12266: -- Test failures irrelevant. [~ekoifman] Can you take another look? > When client exists abnormally, it doesn't release ACID locks > > > Key: HIVE-12266 > URL: https://issues.apache.org/jira/browse/HIVE-12266 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12266.1.patch, HIVE-12266.2.patch, > HIVE-12266.3.patch > > > if you start Hive CLI (locking enabled) and run some command that acquires > locks and ^C the shell before command completes the locks for the command > remain until they timeout. > I believe Beeline has the same issue. > Need to add proper hooks to release locks when command dies. (As much as > possible) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12266) When client exists abnormally, it doesn't release ACID locks
[
https://issues.apache.org/jira/browse/HIVE-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983939#comment-14983939
]
Hive QA commented on HIVE-12266:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12769906/HIVE-12266.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 9745 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5874/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5874/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5874/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12769906 - PreCommit-HIVE-TRUNK-Build
> When client exists abnormally, it doesn't release ACID locks
>
>
> Key: HIVE-12266
> URL: https://issues.apache.org/jira/browse/HIVE-12266
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Wei Zheng
> Attachments: HIVE-12266.1.patch, HIVE-12266.2.patch,
> HIVE-12266.3.patch
>
>
> if you start Hive CLI (locking enabled) and run some command that acquires
> locks and ^C the shell before command completes the locks for the command
> remain until they timeout.
> I believe Beeline has the same issue.
> Need to add proper hooks to release locks when command dies. (As much as
> possible)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-12266) When client exists abnormally, it doesn't release ACID locks
[ https://issues.apache.org/jira/browse/HIVE-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983684#comment-14983684 ] Wei Zheng commented on HIVE-12266: -- Makes sense. But based on my experiments, the transaction manager was there in Driver constructor. Maybe we get it from the previous query in the same session? Is transaction manager corresponding to a session or to a query? > When client exists abnormally, it doesn't release ACID locks > > > Key: HIVE-12266 > URL: https://issues.apache.org/jira/browse/HIVE-12266 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12266.1.patch, HIVE-12266.2.patch > > > if you start Hive CLI (locking enabled) and run some command that acquires > locks and ^C the shell before command completes the locks for the command > remain until they timeout. > I believe Beeline has the same issue. > Need to add proper hooks to release locks when command dies. (As much as > possible) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12266) When client exists abnormally, it doesn't release ACID locks
[ https://issues.apache.org/jira/browse/HIVE-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983653#comment-14983653 ] Eugene Koifman commented on HIVE-12266: --- [~wzheng] I'm not sure it's a good idea to initialize HiveTxnManager in the ShutdownHook from Driver(). It may not even be set there yet. It is first set in Driver.compile() with "SessionState.get().initTxnMgr(conf);" So I think the manager should be passed to the hook there (or even from SessionState.initTxnMgr()) > When client exists abnormally, it doesn't release ACID locks > > > Key: HIVE-12266 > URL: https://issues.apache.org/jira/browse/HIVE-12266 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12266.1.patch, HIVE-12266.2.patch > > > if you start Hive CLI (locking enabled) and run some command that acquires > locks and ^C the shell before command completes the locks for the command > remain until they timeout. > I believe Beeline has the same issue. > Need to add proper hooks to release locks when command dies. (As much as > possible) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12266) When client exists abnormally, it doesn't release ACID locks
[ https://issues.apache.org/jira/browse/HIVE-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983688#comment-14983688 ] Eugene Koifman commented on HIVE-12266: --- Session, currently, though that may change in the future. > When client exists abnormally, it doesn't release ACID locks > > > Key: HIVE-12266 > URL: https://issues.apache.org/jira/browse/HIVE-12266 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 1.0.0 >Reporter: Eugene Koifman >Assignee: Wei Zheng > Attachments: HIVE-12266.1.patch, HIVE-12266.2.patch > > > if you start Hive CLI (locking enabled) and run some command that acquires > locks and ^C the shell before command completes the locks for the command > remain until they timeout. > I believe Beeline has the same issue. > Need to add proper hooks to release locks when command dies. (As much as > possible) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-12266) When client exists abnormally, it doesn't release ACID locks
[
https://issues.apache.org/jira/browse/HIVE-12266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982937#comment-14982937
]
Hive QA commented on HIVE-12266:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12769808/HIVE-12266.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5865/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5865/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5865/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-5865/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
fbe31ed..d7d0f6e master -> origin/master
+ git reset --hard HEAD
HEAD is now at fbe31ed Merge branch 'master' of
https://git-wip-us.apache.org/repos/asf/hive
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
+ git reset --hard origin/master
HEAD is now at d7d0f6e HIVE-12277: Hive macro results on macro_duplicate.q
different after adding ORDER BY (Pengcheng Xiong, reviewed by Jason Dere)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12769808 - PreCommit-HIVE-TRUNK-Build
> When client exists abnormally, it doesn't release ACID locks
>
>
> Key: HIVE-12266
> URL: https://issues.apache.org/jira/browse/HIVE-12266
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Wei Zheng
> Attachments: HIVE-12266.1.patch
>
>
> if you start Hive CLI (locking enabled) and run some command that acquires
> locks and ^C the shell before command completes the locks for the command
> remain until they timeout.
> I believe Beeline has the same issue.
> Need to add proper hooks to release locks when command dies. (As much as
> possible)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
