[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022639#comment-17022639
]
Hive QA commented on HIVE-20801:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991665/HIVE-20801.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17946 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20308/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20308/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20308/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991665 - PreCommit-HIVE-Build
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal Vijayaraghavan
>Assignee: Gopal Vijayaraghavan
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch, HIVE-20801.3.patch, HIVE-20801.3.patch, HIVE-20801.4.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager consuming resources
> centrally.
> Disabling this lock acquisition would improve the performance of non-ACID
> tables co-existing with a globally configured DbTxnManager implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022616#comment-17022616
]
Hive QA commented on HIVE-20801:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
0s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
59s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 20 new + 298 unchanged - 0
fixed = 318 total (was 298) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20308/dev-support/hive-personality.sh
|
| git revision | master / 4193b54 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20308/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20308/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal Vijayaraghavan
>Assignee: Gopal Vijayaraghavan
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch, HIVE-20801.3.patch, HIVE-20801.3.patch, HIVE-20801.4.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager consuming resources
> centrally.
> Disabling this lock acquisition would improve the performa
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022130#comment-17022130 ] Denys Kuzmenko commented on HIVE-20801: --- rebased > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal Vijayaraghavan >Assignee: Gopal Vijayaraghavan >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch, HIVE-20801.3.patch, HIVE-20801.4.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021517#comment-17021517
]
Hive QA commented on HIVE-20801:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976705/HIVE-20801.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20280/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20280/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20280/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-01-22 21:28:52.830
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-20280/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-01-22 21:28:52.833
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 05cabc8 HIVE-22666: Introduce TopNKey operator for PTF Reduce
Sink (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 05cabc8 HIVE-22666: Introduce TopNKey operator for PTF Reduce
Sink (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-01-22 21:28:53.513
+ rm -rf ../yetus_PreCommit-HIVE-Build-20280
+ mkdir ../yetus_PreCommit-HIVE-Build-20280
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20280
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20280/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: patch failed:
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:2561
Falling back to three-way merge...
Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:2338
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java'
cleanly.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:11: trailing whitespace.
"unlocked partitition by partition, instead of being transactionally
scoped.\n" +
/data/hiveptest/working/scratch/build.patch:70: trailing whitespace.
// Test non-strict read-locking mode - the read locks are only obtained for
the ACID side
error: patch failed:
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:2561
Falling back to three-way merge...
Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:2338
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java'
cleanly.
U common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
warning: 2 lines add whitespace errors.
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-20280
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12976705 - PreCommit-HIVE-Build
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal Vijayaraghavan
>Assignee: Gopal Vijayaragh
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16987340#comment-16987340 ] Jason Dere commented on HIVE-20801: --- Is anything going on with this Jira? I am +1 on this one, especially considering that on Apache Hive there is still the ability to have non-transactional managed tables. > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal Vijayaraghavan >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch, HIVE-20801.3.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900338#comment-16900338
]
Hive QA commented on HIVE-20801:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976705/HIVE-20801.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16724 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18259/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18259/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18259/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12976705 - PreCommit-HIVE-Build
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch, HIVE-20801.3.patch, HIVE-20801.3.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager consuming resources
> centrally.
> Disabling this lock acquisition would improve the performance of non-ACID
> tables co-existing with a globally configured DbTxnManager implementation.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900301#comment-16900301
]
Hive QA commented on HIVE-20801:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 2250 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 6 new + 562 unchanged - 0
fixed = 568 total (was 562) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18259/dev-support/hive-personality.sh
|
| git revision | master / 212b428 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18259/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18259/yetus/whitespace-eol.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18259/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch, HIVE-20801.3.patch, HIVE-20801.3.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer str
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16855001#comment-16855001
]
Hive QA commented on HIVE-20801:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954529/HIVE-20801.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16077 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17399/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17399/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17399/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954529 - PreCommit-HIVE-Build
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch, HIVE-20801.3.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager consuming resources
> centrally.
> Disabling this lock acquisition would improve the performance of non-ACID
> tables co-existing with a globally configured DbTxnManager implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854973#comment-16854973
]
Hive QA commented on HIVE-20801:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
43s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 5m
5s{color} | {color:blue} ql in master has 2261 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
54s{color} | {color:red} ql: The patch generated 6 new + 533 unchanged - 0
fixed = 539 total (was 533) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17399/dev-support/hive-personality.sh
|
| git revision | master / f62379b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17399/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17399/yetus/whitespace-eol.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17399/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch, HIVE-20801.3.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency desp
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854934#comment-16854934 ] Laszlo Bodor commented on HIVE-20801: - [~ashutoshc]: could you please check what the step should be with this? > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16783794#comment-16783794 ] Ashutosh Chauhan commented on HIVE-20801: - Now we only have ACID tables or insert-only (MM) tables or external tables. If etl is via spark or sqoop, those tables should be external and then there will be no locking on them. > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779546#comment-16779546 ] Gopal V commented on HIVE-20801: That allows 2 writers concurrently for non-ACID tables. The other problem is that in the non-ACID case, the lock need to be explicitly unlocked by the HS2 (in ACID case, txn timeout unlocks everything, in non-acid, a ^C can leave orphaned locks for upto 20 mins). > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778986#comment-16778986 ] Ashutosh Chauhan commented on HIVE-20801: - hive.txn.strict.locking.mode=false should be sufficient for that. No? > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778985#comment-16778985 ] Gopal V commented on HIVE-20801: bq. Is that acquiring locks for is slow? No, "insert into" locks out reads for non-ACID tables for atomicity. > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778980#comment-16778980 ] Ashutosh Chauhan commented on HIVE-20801: - bq. Just that Hive is slowed down by a magnitude when ACID is enabled for even 1 tables. >From where this slowness coming from? Is that acquiring locks for is slow? > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16740806#comment-16740806 ] Gopal V commented on HIVE-20801: [~ekoifman]: The ETL is via Spark and Sqoop, so the reads aren't consistent right now. Just that Hive is slowed down by a magnitude when ACID is enabled for even 1 tables. There is no read consistency when files are being modified outside of hive on HDFS. > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16740788#comment-16740788 ] Eugene Koifman commented on HIVE-20801: --- [~gopalv] what do you mean locks are advisory? By default, (with Acid on), non-transactional tables use standard S/X locks so so assuming things are configured properly (and no partial failures), reads should be consistent. It seems that the description of the property is misleading. Also, if you are disabling all locks for readers, why acquire any locks for writers? > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16740109#comment-16740109
]
Hive QA commented on HIVE-20801:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954529/HIVE-20801.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15713 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15588/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15588/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15588/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954529 - PreCommit-HIVE-Build
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch, HIVE-20801.3.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager consuming resources
> centrally.
> Disabling this lock acquisition would improve the performance of non-ACID
> tables co-existing with a globally configured DbTxnManager implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16740085#comment-16740085
]
Hive QA commented on HIVE-20801:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 6 new + 542 unchanged - 0
fixed = 548 total (was 542) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15588/dev-support/hive-personality.sh
|
| git revision | master / f713140 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15588/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15588/yetus/whitespace-eol.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15588/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch, HIVE-20801.3.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency desp
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739937#comment-16739937
]
Hive QA commented on HIVE-20801:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954488/HIVE-20801.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15713 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_conf] (batchId=27)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=275)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15584/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15584/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15584/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954488 - PreCommit-HIVE-Build
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager consuming resources
> centrally.
> Disabling this lock acquisition would improve the performance of non-ACID
> tables co-existing with a globally configured DbTxnManager implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739917#comment-16739917
]
Hive QA commented on HIVE-20801:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 6 new + 542 unchanged - 0
fixed = 548 total (was 542) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15584/dev-support/hive-personality.sh
|
| git revision | master / f713140 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15584/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15584/yetus/whitespace-eol.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15584/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16681079#comment-16681079
]
Hive QA commented on HIVE-20801:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12947348/HIVE-20801.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 15528 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_conf] (batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=85)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=231)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14828/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14828/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14828/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12947348 - PreCommit-HIVE-Build
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager consuming resources
> centrally.
> Disabling this lock acquisition would improve the performance of non-ACID
> tables co-existing with a globally configured DbTxnManager implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16681021#comment-16681021
]
Hive QA commented on HIVE-20801:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 6 new + 537 unchanged - 0
fixed = 543 total (was 537) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14828/dev-support/hive-personality.sh
|
| git revision | master / 5aac805 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14828/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14828/yetus/whitespace-eol.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14828/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager consuming resources
>
