[jira] [Commented] (HIVE-10521) TxnHandler.timeOutTxns only times out some of the expired transactions
[
https://issues.apache.org/jira/browse/HIVE-10521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14531481#comment-14531481
]
Sushanth Sowmyan commented on HIVE-10521:
-
Given that precommit tests have now run, [~ekoifman]/[~alangates], could you
please verify and commit this patch?
TxnHandler.timeOutTxns only times out some of the expired transactions
--
Key: HIVE-10521
URL: https://issues.apache.org/jira/browse/HIVE-10521
Project: Hive
Issue Type: Bug
Components: Transactions
Affects Versions: 0.14.0, 1.0.0, 1.1.0
Reporter: Alan Gates
Assignee: Alan Gates
Attachments: HIVE-10521.2.patch, HIVE-10521.3.patch,
HIVE-10521.4.patch, HIVE-10521.patch
{code}
for (int i = 0; i 20 rs.next(); i++) deadTxns.add(rs.getLong(1));
// We don't care whether all of the transactions get deleted or not,
// if some didn't it most likely means someone else deleted them in the
interum
if (deadTxns.size() 0) abortTxns(dbConn, deadTxns);
{code}
While it makes sense to limit the number of transactions aborted in one pass
(since this get's translated to an IN clause) we should still make sure all
are timed out. Also, 20 seems pretty small as a batch size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10521) TxnHandler.timeOutTxns only times out some of the expired transactions
[
https://issues.apache.org/jira/browse/HIVE-10521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14528353#comment-14528353
]
Hive QA commented on HIVE-10521:
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12729817/HIVE-10521.3.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3732/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3732/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3732/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-3732/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 3f72f81 HIVE-5545 : HCatRecord getInteger method returns String
when used on Partition columns of type INT (Sushanth Sowmyan, reviewed by Jason
Dere)
+ git clean -f -d
+ git checkout master
Already on 'master'
+ git reset --hard origin/master
HEAD is now at 3f72f81 HIVE-5545 : HCatRecord getInteger method returns String
when used on Partition columns of type INT (Sushanth Sowmyan, reviewed by Jason
Dere)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12729817 - PreCommit-HIVE-TRUNK-Build
TxnHandler.timeOutTxns only times out some of the expired transactions
--
Key: HIVE-10521
URL: https://issues.apache.org/jira/browse/HIVE-10521
Project: Hive
Issue Type: Bug
Components: Transactions
Affects Versions: 0.14.0, 1.0.0, 1.1.0
Reporter: Alan Gates
Assignee: Alan Gates
Attachments: HIVE-10521.2.patch, HIVE-10521.3.patch, HIVE-10521.patch
{code}
for (int i = 0; i 20 rs.next(); i++) deadTxns.add(rs.getLong(1));
// We don't care whether all of the transactions get deleted or not,
// if some didn't it most likely means someone else deleted them in the
interum
if (deadTxns.size() 0) abortTxns(dbConn, deadTxns);
{code}
While it makes sense to limit the number of transactions aborted in one pass
(since this get's translated to an IN clause) we should still make sure all
are timed out. Also, 20 seems pretty small as a batch size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10521) TxnHandler.timeOutTxns only times out some of the expired transactions
[
https://issues.apache.org/jira/browse/HIVE-10521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14525331#comment-14525331
]
Eugene Koifman commented on HIVE-10521:
---
+1
TxnHandler.timeOutTxns only times out some of the expired transactions
--
Key: HIVE-10521
URL: https://issues.apache.org/jira/browse/HIVE-10521
Project: Hive
Issue Type: Bug
Components: Transactions
Affects Versions: 0.14.0, 1.0.0, 1.1.0
Reporter: Alan Gates
Assignee: Alan Gates
Attachments: HIVE-10521.2.patch, HIVE-10521.3.patch, HIVE-10521.patch
{code}
for (int i = 0; i 20 rs.next(); i++) deadTxns.add(rs.getLong(1));
// We don't care whether all of the transactions get deleted or not,
// if some didn't it most likely means someone else deleted them in the
interum
if (deadTxns.size() 0) abortTxns(dbConn, deadTxns);
{code}
While it makes sense to limit the number of transactions aborted in one pass
(since this get's translated to an IN clause) we should still make sure all
are timed out. Also, 20 seems pretty small as a batch size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10521) TxnHandler.timeOutTxns only times out some of the expired transactions
[
https://issues.apache.org/jira/browse/HIVE-10521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14524355#comment-14524355
]
Eugene Koifman commented on HIVE-10521:
---
[~alangates], if commit is only called once in timeOutTxns(), why break up the
delete into multiple statement?
The patch works functionally as is, but but wouldn't it be better to read the
whole list of txn ids into memory and then commit each abort bach separately?
It would probably make deadlocks less likely if transactions are shorter.
TxnHandler.timeOutTxns only times out some of the expired transactions
--
Key: HIVE-10521
URL: https://issues.apache.org/jira/browse/HIVE-10521
Project: Hive
Issue Type: Bug
Components: Transactions
Affects Versions: 0.14.0, 1.0.0, 1.1.0
Reporter: Alan Gates
Assignee: Alan Gates
Attachments: HIVE-10521.2.patch, HIVE-10521.3.patch, HIVE-10521.patch
{code}
for (int i = 0; i 20 rs.next(); i++) deadTxns.add(rs.getLong(1));
// We don't care whether all of the transactions get deleted or not,
// if some didn't it most likely means someone else deleted them in the
interum
if (deadTxns.size() 0) abortTxns(dbConn, deadTxns);
{code}
While it makes sense to limit the number of transactions aborted in one pass
(since this get's translated to an IN clause) we should still make sure all
are timed out. Also, 20 seems pretty small as a batch size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-10521) TxnHandler.timeOutTxns only times out some of the expired transactions
[
https://issues.apache.org/jira/browse/HIVE-10521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14519873#comment-14519873
]
Eugene Koifman commented on HIVE-10521:
---
All connections in TxnHandler are autoCommit=false
timeOutTxns() opens a cursor and calls abortTxns() for each batch.
abortTxns() calls commit(). So if the cursor has more than
TIMED_OUT_TXN_ABORT_BATCH_SIZE rows, it will end up calling commit while the
cursor is open... I don't think you can do that (and continue to read it after
that)
TestTxnHandler has unused import import junit.framework.Assert;
TxnHandler.timeOutTxns only times out some of the expired transactions
--
Key: HIVE-10521
URL: https://issues.apache.org/jira/browse/HIVE-10521
Project: Hive
Issue Type: Bug
Components: Transactions
Affects Versions: 0.14.0, 1.0.0, 1.1.0
Reporter: Alan Gates
Assignee: Alan Gates
Attachments: HIVE-10521.patch
{code}
for (int i = 0; i 20 rs.next(); i++) deadTxns.add(rs.getLong(1));
// We don't care whether all of the transactions get deleted or not,
// if some didn't it most likely means someone else deleted them in the
interum
if (deadTxns.size() 0) abortTxns(dbConn, deadTxns);
{code}
While it makes sense to limit the number of transactions aborted in one pass
(since this get's translated to an IN clause) we should still make sure all
are timed out. Also, 20 seems pretty small as a batch size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
