[jira] [Updated] (HIVE-22510) Support decimal64 operations for column operands with different scales
[ https://issues.apache.org/jira/browse/HIVE-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22510: Attachment: HIVE-22510.11.patch Status: Patch Available (was: Open) > Support decimal64 operations for column operands with different scales > -- > > Key: HIVE-22510 > URL: https://issues.apache.org/jira/browse/HIVE-22510 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22510.11.patch, HIVE-22510.2.patch, > HIVE-22510.3.patch, HIVE-22510.4.patch, HIVE-22510.5.patch, > HIVE-22510.7.patch, HIVE-22510.9.patch > > > Right now, if the operands on the decimal64 operations are columns with > different scales, then we do not use the decimal64 vectorized version and > fall back to HiveDecimal vectorized version of the operator. In this Jira, we > will check if we can use decimal64 vectorized version, even if the scales are > different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22510) Support decimal64 operations for column operands with different scales
[ https://issues.apache.org/jira/browse/HIVE-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22510: Status: Open (was: Patch Available) > Support decimal64 operations for column operands with different scales > -- > > Key: HIVE-22510 > URL: https://issues.apache.org/jira/browse/HIVE-22510 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22510.2.patch, HIVE-22510.3.patch, > HIVE-22510.4.patch, HIVE-22510.5.patch, HIVE-22510.7.patch, HIVE-22510.9.patch > > > Right now, if the operands on the decimal64 operations are columns with > different scales, then we do not use the decimal64 vectorized version and > fall back to HiveDecimal vectorized version of the operator. In this Jira, we > will check if we can use decimal64 vectorized version, even if the scales are > different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Status: Open (was: Patch Available) > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch, > HIVE-22508.1.patch, HIVE-22508.2.patch, HIVE-22508.2.patch, > HIVE-22508.2.patch, HIVE-22508.2.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Attachment: HIVE-22508.2.patch > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch, > HIVE-22508.1.patch, HIVE-22508.2.patch, HIVE-22508.2.patch, > HIVE-22508.2.patch, HIVE-22508.2.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Status: Patch Available (was: Open) > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch, > HIVE-22508.1.patch, HIVE-22508.2.patch, HIVE-22508.2.patch, > HIVE-22508.2.patch, HIVE-22508.2.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Attachment: HIVE-20150.25.patch > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.24.patch, HIVE-20150.25.patch, HIVE-20150.25.patch, > HIVE-20150.4.patch, HIVE-20150.5.patch, HIVE-20150.6.patch, > HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Open (was: Patch Available) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.24.patch, HIVE-20150.25.patch, HIVE-20150.25.patch, > HIVE-20150.4.patch, HIVE-20150.5.patch, HIVE-20150.6.patch, > HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Patch Available (was: Open) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch, > HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch, > HIVE-20150.21.patch, HIVE-20150.22.patch, HIVE-20150.23.patch, > HIVE-20150.24.patch, HIVE-20150.25.patch, HIVE-20150.25.patch, > HIVE-20150.4.patch, HIVE-20150.5.patch, HIVE-20150.6.patch, > HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22617) Re-Enable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
[ https://issues.apache.org/jira/browse/HIVE-22617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22617: Parent: HIVE-22619 Issue Type: Sub-task (was: Test) > Re-Enable PreCommit test > org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 > --- > > Key: HIVE-22617 > URL: https://issues.apache.org/jira/browse/HIVE-22617 > Project: Hive > Issue Type: Sub-task >Reporter: Oliver Draese >Priority: Major > > The test was disabled via HIVE-22616 because it was flaky. If the test is > considered valid, it needs to be fixed and re-enabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22593) Dynamically partitioned MM (insert-only ACID) tables don't compact automatically
[
https://issues.apache.org/jira/browse/HIVE-22593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-22593:
-
Attachment: HIVE-22593.01.patch
Status: Patch Available (was: Open)
> Dynamically partitioned MM (insert-only ACID) tables don't compact
> automatically
>
>
> Key: HIVE-22593
> URL: https://issues.apache.org/jira/browse/HIVE-22593
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-22593.01.patch, HIVE-22593.01.patch
>
>
> Dynamic partitions of MM tables aren't entered into the HMS table
> TXN_COMPONENTS. On inserting into such tables we see this line in the HMS log:
> {code:java}
> Expected to move at least one record from txn_components to
> completed_txn_components when committing txn!{code}
> (This is not the case for non-partitioned MM tables.)
> Since the partitions aren't entered into COMPLETED_TXN_COMPONENTS, they
> aren't considered for automatic compaction.
> Probably the culprit is
> org.apache.hadoop.hive.ql.metadata.Hive#loadDynamicPartitions which has an
> isAcid parameter that is always false regarding MM tables, and also because
> MM tables' "write type" is AcidUtils.Operation.NOT_ACID and not INSERT.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22593) Dynamically partitioned MM (insert-only ACID) tables don't compact automatically
[
https://issues.apache.org/jira/browse/HIVE-22593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-22593:
-
Status: Open (was: Patch Available)
> Dynamically partitioned MM (insert-only ACID) tables don't compact
> automatically
>
>
> Key: HIVE-22593
> URL: https://issues.apache.org/jira/browse/HIVE-22593
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-22593.01.patch, HIVE-22593.01.patch
>
>
> Dynamic partitions of MM tables aren't entered into the HMS table
> TXN_COMPONENTS. On inserting into such tables we see this line in the HMS log:
> {code:java}
> Expected to move at least one record from txn_components to
> completed_txn_components when committing txn!{code}
> (This is not the case for non-partitioned MM tables.)
> Since the partitions aren't entered into COMPLETED_TXN_COMPONENTS, they
> aren't considered for automatic compaction.
> Probably the culprit is
> org.apache.hadoop.hive.ql.metadata.Hive#loadDynamicPartitions which has an
> isAcid parameter that is always false regarding MM tables, and also because
> MM tables' "write type" is AcidUtils.Operation.NOT_ACID and not INSERT.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-20972) Enable TestMiniLlapLocalCliDriver cbo_limit and cbo_rp_limit
[ https://issues.apache.org/jira/browse/HIVE-20972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20972: Attachment: HIVE-20972.03.patch > Enable TestMiniLlapLocalCliDriver cbo_limit and cbo_rp_limit > > > Key: HIVE-20972 > URL: https://issues.apache.org/jira/browse/HIVE-20972 > Project: Hive > Issue Type: Test >Reporter: Vihang Karajgaonkar >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20972.01.patch, HIVE-20972.02.patch, > HIVE-20972.02.patch, HIVE-20972.03.patch, HIVE-20972.03.patch, > HIVE-20972.03.patch, HIVE-20972.03.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22584) Flakyness in TestTaskExecutorService.testSetCapacity
[
https://issues.apache.org/jira/browse/HIVE-22584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993262#comment-16993262
]
Zoltan Haindrich commented on HIVE-22584:
-
from the preceeding comments I would assume that 4 should be running already
https://github.com/apache/hive/blob/8b18c3c70d5ec2a4549d9ab7e610d92ee8506d55/llap-server/src/test/org/apache/hadoop/hive/llap/daemon/impl/TestTaskExecutorService.java#L500
if you agree that can be fixed before commit (if you don't its also ok "as is"
because these falky tests are really ruin everyones days)
+1
thank you for improving the reliability of this test!
> Flakyness in TestTaskExecutorService.testSetCapacity
>
>
> Key: HIVE-22584
> URL: https://issues.apache.org/jira/browse/HIVE-22584
> Project: Hive
> Issue Type: Test
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22584.patch
>
>
> Very rarely the test fails:
> {code}
> java.lang.AssertionError: expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testSetCapacity(TestTaskExecutorService.java:515)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {code}
> See:
> https://builds.apache.org/job/PreCommit-HIVE-Build/19739/testReport/org.apache.hadoop.hive.llap.daemon.impl/TestTaskExecutorService/testSetCapacity/
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22486) Send only accessed columns for masking policies request
[
https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993260#comment-16993260
]
Hive QA commented on HIVE-22486:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988485/HIVE-22486.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19868/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19868/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19868/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12988485/HIVE-22486.patch was
found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988485 - PreCommit-HIVE-Build
> Send only accessed columns for masking policies request
> ---
>
> Key: HIVE-22486
> URL: https://issues.apache.org/jira/browse/HIVE-22486
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch,
> HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch,
> HIVE-22486.patch, HIVE-22486.patchaddendum
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, we send all columns for masking request, even if they are not
> accessed by the given query. We could send only those columns for which the
> masking policy will be necessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22486) Send only accessed columns for masking policies request
[
https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993259#comment-16993259
]
Hive QA commented on HIVE-22486:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988485/HIVE-22486.patch
{color:green}SUCCESS:{color} +1 due to 22 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17757 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACIDNoAutogather - did not produce a TEST-*.xml
file (likely timed out) (batchId=257)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19867/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19867/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19867/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988485 - PreCommit-HIVE-Build
> Send only accessed columns for masking policies request
> ---
>
> Key: HIVE-22486
> URL: https://issues.apache.org/jira/browse/HIVE-22486
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch,
> HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch,
> HIVE-22486.patch, HIVE-22486.patchaddendum
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, we send all columns for masking request, even if they are not
> accessed by the given query. We could send only those columns for which the
> masking policy will be necessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22486) Send only accessed columns for masking policies request
[
https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993242#comment-16993242
]
Hive QA commented on HIVE-22486:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
37s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
17s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19867/dev-support/hive-personality.sh
|
| git revision | master / 8b18c3c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19867/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19867/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Send only accessed columns for masking policies request
> ---
>
> Key: HIVE-22486
> URL: https://issues.apache.org/jira/browse/HIVE-22486
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch,
> HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch,
> HIVE-22486.patch, HIVE-22486.patchaddendum
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, we send all columns for masking request, even if they are not
> accessed by the given query. We could send only those columns for which the
> masking policy will be necessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22618) Fix checkstyle violations for ParseUtils
[ https://issues.apache.org/jira/browse/HIVE-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] okumin reassigned HIVE-22618: - > Fix checkstyle violations for ParseUtils > > > Key: HIVE-22618 > URL: https://issues.apache.org/jira/browse/HIVE-22618 > Project: Hive > Issue Type: Task > Components: Query Processor >Reporter: okumin >Assignee: okumin >Priority: Minor > > ParseUtils.java has incorrect indents and it causes any changes to violate > checkstyle. > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/ParseUtils.java#L274] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993225#comment-16993225
]
Hive QA commented on HIVE-22601:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988480/HIVE-22601.1.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 17763 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_empty_result]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_truncate]
(batchId=179)
org.apache.hadoop.hive.metastore.TestMetaStoreEventListenerOnlyOnCommit.testEventStatus
(batchId=234)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19866/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19866/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19866/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988480 - PreCommit-HIVE-Build
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.1.patch, HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: HIVE-22486.patchaddendum > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch, > HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch, > HIVE-22486.patch, HIVE-22486.patchaddendum > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: (was: HIVE-22486-addendum.01.patch) > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch, > HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch, > HIVE-22486.patch, HIVE-22486.patchaddendum > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: HIVE-22486.patchaddedum > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch, > HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch, > HIVE-22486.patch, HIVE-22486.patchaddendum > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: (was: HIVE-22486.patchaddedum) > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch, > HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch, > HIVE-22486.patch, HIVE-22486.patchaddendum > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993196#comment-16993196
]
Hive QA commented on HIVE-22601:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
38s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
48s{color} | {color:red} ql: The patch generated 10 new + 278 unchanged - 2
fixed = 288 total (was 280) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
16s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19866/dev-support/hive-personality.sh
|
| git revision | master / 8b18c3c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19866/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19866/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19866/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.1.patch, HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[
https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993187#comment-16993187
]
Hive QA commented on HIVE-22572:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988475/HIVE-22572.6.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 17759 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_lifetime]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_transactional]
(batchId=166)
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=366)
org.apache.hive.jdbc.TestJdbcWithMiniHA.org.apache.hive.jdbc.TestJdbcWithMiniHA
(batchId=288)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19864/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19864/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19864/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988475 - PreCommit-HIVE-Build
> NullPointerException when using dynamic semijoin reduction
> --
>
> Key: HIVE-22572
> URL: https://issues.apache.org/jira/browse/HIVE-22572
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22572.1.patch, HIVE-22572.2.patch,
> HIVE-22572.3.patch, HIVE-22572.4.patch, HIVE-22572.5.patch,
> HIVE-22572.6.patch, HIVE-22572.patch
>
>
> From HS2 logs
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[
https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993167#comment-16993167
]
Hive QA commented on HIVE-22572:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
29s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
17s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19864/dev-support/hive-personality.sh
|
| git revision | master / 8b18c3c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19864/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19864/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> NullPointerException when using dynamic semijoin reduction
> --
>
> Key: HIVE-22572
> URL: https://issues.apache.org/jira/browse/HIVE-22572
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22572.1.patch, HIVE-22572.2.patch,
> HIVE-22572.3.patch, HIVE-22572.4.patch, HIVE-22572.5.patch,
> HIVE-22572.6.patch, HIVE-22572.patch
>
>
> From HS2 logs
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
>
[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-22601:
--
Attachment: HIVE-22601.1.patch
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.1.patch, HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-22601:
--
Status: Patch Available (was: Open)
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.1.patch, HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-22601:
--
Status: Open (was: Patch Available)
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22558) Metastore: Passwords jceks should be read lazily, in case of connection pools
[
https://issues.apache.org/jira/browse/HIVE-22558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993154#comment-16993154
]
Hive QA commented on HIVE-22558:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988472/HIVE-22558.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17762 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_2]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_empty_result]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_truncate]
(batchId=179)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19863/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19863/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19863/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988472 - PreCommit-HIVE-Build
> Metastore: Passwords jceks should be read lazily, in case of connection pools
> -
>
> Key: HIVE-22558
> URL: https://issues.apache.org/jira/browse/HIVE-22558
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Standalone Metastore
>Reporter: Gopal Vijayaraghavan
>Assignee: Slim Bouguerra
>Priority: Major
> Attachments: HIVE-22558.1.patch, getDatabase-password-md5-hotpath.png
>
>
> The jceks file is parsed for every instance of the metastore conf to populate
> the password in plain-text, which is irrelevant for the scenario where the DB
> connection pool is already active.
> !getDatabase-password-md5-hotpath.png|width=640!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-22601:
--
Status: Patch Available (was: Open)
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases
[
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
okumin updated HIVE-22601:
--
Status: Open (was: Patch Available)
> Some columns will be lost when a UDTF has multiple aliases in some cases
>
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.2
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Attachments: HIVE-22601.patch
>
>
> Only one column will be retained when putting UDTFs with multiple aliases and
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1,
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3 |
> +-+
> | c |
> | f |
> +-+
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22616) Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
[
https://issues.apache.org/jira/browse/HIVE-22616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22616:
---
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks [~odraese].
Pushed to master, not waiting for tests on this since it is disabling a flaky
test. Tracking re-enabling it in HIVE-22617.
> Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
> -
>
> Key: HIVE-22616
> URL: https://issues.apache.org/jira/browse/HIVE-22616
> Project: Hive
> Issue Type: Test
> Components: Hive
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22616.patch, TestFail.pdf
>
>
> The test is flaky and produces following errors:
> {{java.io.FileNotFoundException: Source
> '/home/hiveptest/34.69.225.53-hiveptest-2/apache-github-source-source/itests/hive-unit/target/junit-qfile-results/clientpositive/join2.q.out'
> does not exist}}
> {{ at
> org.apache.commons.io.FileUtils.checkFileRequirements(FileUtils.java:1383)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1060)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1028)}}
> {{ at
> org.apache.hadoop.hive.ql.QOutProcessor.maskPatterns(QOutProcessor.java:162)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestUtil.checkCliDriverResults(QTestUtil.java:932)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestRunnerUtils.queryListRunnerMultiThreaded(QTestRunnerUtils.java:152)}}
> {{ at
> org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1(TestMTQueries.java:55)}}
> {{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
> {{ at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{ at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.lang.reflect.Method.invoke(Method.java:498)}}
> {{ at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)}}
> {{ at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)}}
> {{ at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)}}
> {{ at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)}}
> {{ at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)}}
> {{ at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)}}
> {{ at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)}}
> {{ at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)}}
> {{ at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)}}
> {{ at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)}}
> {{ at org.junit.runners.ParentRunner.run(ParentRunner.java:309)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22616) Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
[
https://issues.apache.org/jira/browse/HIVE-22616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993137#comment-16993137
]
Jesus Camacho Rodriguez commented on HIVE-22616:
+1
> Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
> -
>
> Key: HIVE-22616
> URL: https://issues.apache.org/jira/browse/HIVE-22616
> Project: Hive
> Issue Type: Test
> Components: Hive
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-22616.patch, TestFail.pdf
>
>
> The test is flaky and produces following errors:
> {{java.io.FileNotFoundException: Source
> '/home/hiveptest/34.69.225.53-hiveptest-2/apache-github-source-source/itests/hive-unit/target/junit-qfile-results/clientpositive/join2.q.out'
> does not exist}}
> {{ at
> org.apache.commons.io.FileUtils.checkFileRequirements(FileUtils.java:1383)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1060)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1028)}}
> {{ at
> org.apache.hadoop.hive.ql.QOutProcessor.maskPatterns(QOutProcessor.java:162)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestUtil.checkCliDriverResults(QTestUtil.java:932)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestRunnerUtils.queryListRunnerMultiThreaded(QTestRunnerUtils.java:152)}}
> {{ at
> org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1(TestMTQueries.java:55)}}
> {{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
> {{ at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{ at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.lang.reflect.Method.invoke(Method.java:498)}}
> {{ at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)}}
> {{ at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)}}
> {{ at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)}}
> {{ at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)}}
> {{ at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)}}
> {{ at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)}}
> {{ at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)}}
> {{ at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)}}
> {{ at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)}}
> {{ at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)}}
> {{ at org.junit.runners.ParentRunner.run(ParentRunner.java:309)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22616) Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
[
https://issues.apache.org/jira/browse/HIVE-22616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-22616:
-
Attachment: HIVE-22616.patch
> Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
> -
>
> Key: HIVE-22616
> URL: https://issues.apache.org/jira/browse/HIVE-22616
> Project: Hive
> Issue Type: Test
> Components: Hive
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-22616.patch, TestFail.pdf
>
>
> The test is flaky and produces following errors:
> {{java.io.FileNotFoundException: Source
> '/home/hiveptest/34.69.225.53-hiveptest-2/apache-github-source-source/itests/hive-unit/target/junit-qfile-results/clientpositive/join2.q.out'
> does not exist}}
> {{ at
> org.apache.commons.io.FileUtils.checkFileRequirements(FileUtils.java:1383)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1060)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1028)}}
> {{ at
> org.apache.hadoop.hive.ql.QOutProcessor.maskPatterns(QOutProcessor.java:162)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestUtil.checkCliDriverResults(QTestUtil.java:932)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestRunnerUtils.queryListRunnerMultiThreaded(QTestRunnerUtils.java:152)}}
> {{ at
> org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1(TestMTQueries.java:55)}}
> {{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
> {{ at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{ at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.lang.reflect.Method.invoke(Method.java:498)}}
> {{ at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)}}
> {{ at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)}}
> {{ at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)}}
> {{ at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)}}
> {{ at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)}}
> {{ at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)}}
> {{ at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)}}
> {{ at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)}}
> {{ at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)}}
> {{ at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)}}
> {{ at org.junit.runners.ParentRunner.run(ParentRunner.java:309)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22616) Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
[
https://issues.apache.org/jira/browse/HIVE-22616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-22616:
-
Status: Patch Available (was: Open)
Disabled test
> Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
> -
>
> Key: HIVE-22616
> URL: https://issues.apache.org/jira/browse/HIVE-22616
> Project: Hive
> Issue Type: Test
> Components: Hive
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-22616.patch, TestFail.pdf
>
>
> The test is flaky and produces following errors:
> {{java.io.FileNotFoundException: Source
> '/home/hiveptest/34.69.225.53-hiveptest-2/apache-github-source-source/itests/hive-unit/target/junit-qfile-results/clientpositive/join2.q.out'
> does not exist}}
> {{ at
> org.apache.commons.io.FileUtils.checkFileRequirements(FileUtils.java:1383)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1060)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1028)}}
> {{ at
> org.apache.hadoop.hive.ql.QOutProcessor.maskPatterns(QOutProcessor.java:162)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestUtil.checkCliDriverResults(QTestUtil.java:932)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestRunnerUtils.queryListRunnerMultiThreaded(QTestRunnerUtils.java:152)}}
> {{ at
> org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1(TestMTQueries.java:55)}}
> {{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
> {{ at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{ at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.lang.reflect.Method.invoke(Method.java:498)}}
> {{ at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)}}
> {{ at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)}}
> {{ at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)}}
> {{ at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)}}
> {{ at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)}}
> {{ at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)}}
> {{ at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)}}
> {{ at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)}}
> {{ at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)}}
> {{ at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)}}
> {{ at org.junit.runners.ParentRunner.run(ParentRunner.java:309)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22616) Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
[
https://issues.apache.org/jira/browse/HIVE-22616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-22616:
-
Attachment: TestFail.pdf
> Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
> -
>
> Key: HIVE-22616
> URL: https://issues.apache.org/jira/browse/HIVE-22616
> Project: Hive
> Issue Type: Test
> Components: Hive
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: TestFail.pdf
>
>
> The test is flaky and produces following errors:
> {{java.io.FileNotFoundException: Source
> '/home/hiveptest/34.69.225.53-hiveptest-2/apache-github-source-source/itests/hive-unit/target/junit-qfile-results/clientpositive/join2.q.out'
> does not exist}}
> {{ at
> org.apache.commons.io.FileUtils.checkFileRequirements(FileUtils.java:1383)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1060)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1028)}}
> {{ at
> org.apache.hadoop.hive.ql.QOutProcessor.maskPatterns(QOutProcessor.java:162)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestUtil.checkCliDriverResults(QTestUtil.java:932)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestRunnerUtils.queryListRunnerMultiThreaded(QTestRunnerUtils.java:152)}}
> {{ at
> org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1(TestMTQueries.java:55)}}
> {{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
> {{ at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{ at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.lang.reflect.Method.invoke(Method.java:498)}}
> {{ at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)}}
> {{ at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)}}
> {{ at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)}}
> {{ at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)}}
> {{ at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)}}
> {{ at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)}}
> {{ at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)}}
> {{ at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)}}
> {{ at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)}}
> {{ at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)}}
> {{ at org.junit.runners.ParentRunner.run(ParentRunner.java:309)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22616) Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
[
https://issues.apache.org/jira/browse/HIVE-22616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993134#comment-16993134
]
Oliver Draese commented on HIVE-22616:
--
Full test output in attachment TestFail.pdf
> Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
> -
>
> Key: HIVE-22616
> URL: https://issues.apache.org/jira/browse/HIVE-22616
> Project: Hive
> Issue Type: Test
> Components: Hive
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: TestFail.pdf
>
>
> The test is flaky and produces following errors:
> {{java.io.FileNotFoundException: Source
> '/home/hiveptest/34.69.225.53-hiveptest-2/apache-github-source-source/itests/hive-unit/target/junit-qfile-results/clientpositive/join2.q.out'
> does not exist}}
> {{ at
> org.apache.commons.io.FileUtils.checkFileRequirements(FileUtils.java:1383)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1060)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1028)}}
> {{ at
> org.apache.hadoop.hive.ql.QOutProcessor.maskPatterns(QOutProcessor.java:162)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestUtil.checkCliDriverResults(QTestUtil.java:932)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestRunnerUtils.queryListRunnerMultiThreaded(QTestRunnerUtils.java:152)}}
> {{ at
> org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1(TestMTQueries.java:55)}}
> {{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
> {{ at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{ at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.lang.reflect.Method.invoke(Method.java:498)}}
> {{ at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)}}
> {{ at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)}}
> {{ at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)}}
> {{ at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)}}
> {{ at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)}}
> {{ at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)}}
> {{ at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)}}
> {{ at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)}}
> {{ at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)}}
> {{ at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)}}
> {{ at org.junit.runners.ParentRunner.run(ParentRunner.java:309)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22616) Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
[
https://issues.apache.org/jira/browse/HIVE-22616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese reassigned HIVE-22616:
> Disable PreCommit test org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1
> -
>
> Key: HIVE-22616
> URL: https://issues.apache.org/jira/browse/HIVE-22616
> Project: Hive
> Issue Type: Test
> Components: Hive
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
>
> The test is flaky and produces following errors:
> {{java.io.FileNotFoundException: Source
> '/home/hiveptest/34.69.225.53-hiveptest-2/apache-github-source-source/itests/hive-unit/target/junit-qfile-results/clientpositive/join2.q.out'
> does not exist}}
> {{ at
> org.apache.commons.io.FileUtils.checkFileRequirements(FileUtils.java:1383)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1060)}}
> {{ at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1028)}}
> {{ at
> org.apache.hadoop.hive.ql.QOutProcessor.maskPatterns(QOutProcessor.java:162)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestUtil.checkCliDriverResults(QTestUtil.java:932)}}
> {{ at
> org.apache.hadoop.hive.ql.QTestRunnerUtils.queryListRunnerMultiThreaded(QTestRunnerUtils.java:152)}}
> {{ at
> org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1(TestMTQueries.java:55)}}
> {{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
> {{ at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{ at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.lang.reflect.Method.invoke(Method.java:498)}}
> {{ at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)}}
> {{ at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)}}
> {{ at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)}}
> {{ at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)}}
> {{ at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)}}
> {{ at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)}}
> {{ at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)}}
> {{ at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)}}
> {{ at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)}}
> {{ at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)}}
> {{ at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)}}
> {{ at org.junit.runners.ParentRunner.run(ParentRunner.java:309)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)}}
> {{ at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)}}
> {{ at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22558) Metastore: Passwords jceks should be read lazily, in case of connection pools
[
https://issues.apache.org/jira/browse/HIVE-22558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993132#comment-16993132
]
Hive QA commented on HIVE-22558:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
20s{color} | {color:blue} standalone-metastore/metastore-server in master has
183 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19863/dev-support/hive-personality.sh
|
| git revision | master / 22e298c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19863/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19863/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Metastore: Passwords jceks should be read lazily, in case of connection pools
> -
>
> Key: HIVE-22558
> URL: https://issues.apache.org/jira/browse/HIVE-22558
> Project: Hive
> Issue Type: Bug
> Components: Metastore, Standalone Metastore
>Reporter: Gopal Vijayaraghavan
>Assignee: Slim Bouguerra
>Priority: Major
> Attachments: HIVE-22558.1.patch, getDatabase-password-md5-hotpath.png
>
>
> The jceks file is parsed for every instance of the metastore conf to populate
> the password in plain-text, which is irrelevant for the scenario where the DB
> connection pool is already active.
> !getDatabase-password-md5-hotpath.png|width=640!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[ https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Draese updated HIVE-22572: - Attachment: HIVE-22572.6.patch > NullPointerException when using dynamic semijoin reduction > -- > > Key: HIVE-22572 > URL: https://issues.apache.org/jira/browse/HIVE-22572 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Oliver Draese >Assignee: Oliver Draese >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22572.1.patch, HIVE-22572.2.patch, > HIVE-22572.3.patch, HIVE-22572.4.patch, HIVE-22572.5.patch, > HIVE-22572.6.patch, HIVE-22572.patch > > > From HS2 logs > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[
https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993124#comment-16993124
]
Hive QA commented on HIVE-22572:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988467/HIVE-22572.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17762 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_capacity]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_temptable]
(batchId=187)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=276)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19861/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19861/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19861/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988467 - PreCommit-HIVE-Build
> NullPointerException when using dynamic semijoin reduction
> --
>
> Key: HIVE-22572
> URL: https://issues.apache.org/jira/browse/HIVE-22572
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22572.1.patch, HIVE-22572.2.patch,
> HIVE-22572.3.patch, HIVE-22572.4.patch, HIVE-22572.5.patch, HIVE-22572.patch
>
>
> From HS2 logs
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22558) Metastore: Passwords jceks should be read lazily, in case of connection pools
[ https://issues.apache.org/jira/browse/HIVE-22558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Slim Bouguerra updated HIVE-22558: -- Attachment: HIVE-22558.1.patch > Metastore: Passwords jceks should be read lazily, in case of connection pools > - > > Key: HIVE-22558 > URL: https://issues.apache.org/jira/browse/HIVE-22558 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Reporter: Gopal Vijayaraghavan >Assignee: Slim Bouguerra >Priority: Major > Attachments: HIVE-22558.1.patch, getDatabase-password-md5-hotpath.png > > > The jceks file is parsed for every instance of the metastore conf to populate > the password in plain-text, which is irrelevant for the scenario where the DB > connection pool is already active. > !getDatabase-password-md5-hotpath.png|width=640! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22558) Metastore: Passwords jceks should be read lazily, in case of connection pools
[ https://issues.apache.org/jira/browse/HIVE-22558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Slim Bouguerra updated HIVE-22558: -- Status: Patch Available (was: Open) > Metastore: Passwords jceks should be read lazily, in case of connection pools > - > > Key: HIVE-22558 > URL: https://issues.apache.org/jira/browse/HIVE-22558 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Reporter: Gopal Vijayaraghavan >Assignee: Slim Bouguerra >Priority: Major > Attachments: getDatabase-password-md5-hotpath.png > > > The jceks file is parsed for every instance of the metastore conf to populate > the password in plain-text, which is irrelevant for the scenario where the DB > connection pool is already active. > !getDatabase-password-md5-hotpath.png|width=640! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22558) Metastore: Passwords jceks should be read lazily, in case of connection pools
[ https://issues.apache.org/jira/browse/HIVE-22558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Slim Bouguerra reassigned HIVE-22558: - Assignee: Slim Bouguerra > Metastore: Passwords jceks should be read lazily, in case of connection pools > - > > Key: HIVE-22558 > URL: https://issues.apache.org/jira/browse/HIVE-22558 > Project: Hive > Issue Type: Bug > Components: Metastore, Standalone Metastore >Reporter: Gopal Vijayaraghavan >Assignee: Slim Bouguerra >Priority: Major > Attachments: getDatabase-password-md5-hotpath.png > > > The jceks file is parsed for every instance of the metastore conf to populate > the password in plain-text, which is irrelevant for the scenario where the DB > connection pool is already active. > !getDatabase-password-md5-hotpath.png|width=640! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[
https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993081#comment-16993081
]
Hive QA commented on HIVE-22572:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
32s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
17s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19861/dev-support/hive-personality.sh
|
| git revision | master / 22e298c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19861/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19861/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> NullPointerException when using dynamic semijoin reduction
> --
>
> Key: HIVE-22572
> URL: https://issues.apache.org/jira/browse/HIVE-22572
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22572.1.patch, HIVE-22572.2.patch,
> HIVE-22572.3.patch, HIVE-22572.4.patch, HIVE-22572.5.patch, HIVE-22572.patch
>
>
> From HS2 logs
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
>
[jira] [Commented] (HIVE-22599) Query results cache: 733 permissions check is not necessary
[
https://issues.apache.org/jira/browse/HIVE-22599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993059#comment-16993059
]
Jason Dere commented on HIVE-22599:
---
[~gopalv], do you mind reviewing this small change?
> Query results cache: 733 permissions check is not necessary
> ---
>
> Key: HIVE-22599
> URL: https://issues.apache.org/jira/browse/HIVE-22599
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-22599.1.patch
>
>
> The query results cache initialization makes a call to
> Utilties.ensurePathIsWritable(), which checks the results cache directory for
> 733 permissions (default cache dir is
> {{/tmp/hive/_resultscache_).}}
> The 733 permissions (at least the 033 part) are not actually necessary - we
> actually don't really want the results cache directory to be world-writable,
> and the subdirectories we create within this one are actually done with 700
> perms. So I think the call to Utilties.ensurePathIsWritable() can be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22599) Query results cache: 733 permissions check is not necessary
[
https://issues.apache.org/jira/browse/HIVE-22599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993056#comment-16993056
]
Hive QA commented on HIVE-22599:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988465/HIVE-22599.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17762 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19860/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19860/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19860/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988465 - PreCommit-HIVE-Build
> Query results cache: 733 permissions check is not necessary
> ---
>
> Key: HIVE-22599
> URL: https://issues.apache.org/jira/browse/HIVE-22599
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-22599.1.patch
>
>
> The query results cache initialization makes a call to
> Utilties.ensurePathIsWritable(), which checks the results cache directory for
> 733 permissions (default cache dir is
> {{/tmp/hive/_resultscache_).}}
> The 733 permissions (at least the 033 part) are not actually necessary - we
> actually don't really want the results cache directory to be world-writable,
> and the subdirectories we create within this one are actually done with 700
> perms. So I think the call to Utilties.ensurePathIsWritable() can be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22566) Drop table involved in materialized view leaves the table in inconsistent state
[
https://issues.apache.org/jira/browse/HIVE-22566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pablo Junge updated HIVE-22566:
---
Status: Open (was: Patch Available)
> Drop table involved in materialized view leaves the table in inconsistent
> state
> ---
>
> Key: HIVE-22566
> URL: https://issues.apache.org/jira/browse/HIVE-22566
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Pablo Junge
>Assignee: Pablo Junge
>Priority: Minor
> Attachments: HIVE-22566.patch
>
>
> If you try dropping a table which is part of the definition of a created
> materialized view, the table is not dropped, which is the desired state as it
> is part of the materialized view.
> However, there was a "drop" call to the table, so it tried to drop it but did
> not succeed, leaving it in an inconsistent state.
>
> Repro:
> ---
> 1) Create tables:
>
> {code:java}
> CREATE TABLE emps ( empid INT, deptno INT, name VARCHAR(256), salary
> FLOAT, hire_date TIMESTAMP)STORED AS ORC TBLPROPERTIES
> ('transactional'='true');
> CREATE TABLE depts ( deptno INT, deptname VARCHAR(256), locationid
> INT)STORED AS ORC TBLPROPERTIES ('transactional'='true');
> {code}
>
> 2) Create the VM:
>
> {code:java}
> CREATE MATERIALIZED VIEW mv1 AS SELECT empid, deptname, hire_date FROM emps
> JOIN depts ON (emps.deptno = depts.deptno) WHERE hire_date >= '2016-01-01';
> {code}
>
> 3) Following is in backend database at this point:
>
> {code:java}
> mysql> select TBL_ID, DB_ID, SD_ID, TBL_NAME, TBL_TYPE from TBLS where
> DB_ID=16;
> ++---+---+--+---+
> | TBL_ID | DB_ID | SD_ID | TBL_NAME | TBL_TYPE |
> ++---+---+--+---+
> | 81 | 16 | 81 | emps | MANAGED_TABLE |
> | 83 | 16 | 83 | depts | MANAGED_TABLE |
> | 84 | 16 | 84 | mv1 | MATERIALIZED_VIEW |
> ++---+---+--+---+
> 3 rows in set (0.00 sec)
> {code}
>
> 4) Let's drop the 'emps' table:
>
> {code:java}
> 0: jdbc:hive2://c1122-node2.squadron.support.> drop table emps;
> INFO : Compiling
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b):
> drop table emps
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b);
> Time taken: 0.05 seconds
> INFO : Executing
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b):
> drop table emps
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b);
> Time taken: 10.281 seconds
> INFO : OK
> No rows affected (16.949 seconds)
> {code}
> No issue displayed
>
> 5) List tables:
>
> {code:java}
> 0: jdbc:hive2://c1122-node2.squadron.support.> show tables;
> INFO : Compiling
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413):
> show tables
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from
> deserializer)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413);
> Time taken: 0.041 seconds
> INFO : Executing
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413):
> show tables
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413);
> Time taken: 0.016 seconds
> INFO : OK
> +---+
> | tab_name |
> +---+
> | depts |
> | emps |
> +---+
> 2 rows selected (0.08 seconds)
> {code}
>
> 6) Now, from the backend-db point of view:
>
> {code:java}
> mysql> select TBL_ID, DB_ID, SD_ID, TBL_NAME, TBL_TYPE from TBLS where
> DB_ID=16;
> ++---+---+--+---+
> | TBL_ID | DB_ID | SD_ID | TBL_NAME | TBL_TYPE |
> ++---+---+--+---+
> | 81 | 16 | NULL | emps | MANAGED_TABLE |
> | 83 | 16 | 83 | depts | MANAGED_TABLE |
> | 84 | 16 | 84 | mv1 | MATERIALIZED_VIEW |
> ++---+---+--+---+
> 3 rows in set (0.00 sec)
> {code}
> The table is left with NULL in SD_ID, making it not available.
>
> 7) From Metastore.log
>
> {code:java}
> 2019-12-02T20:00:25,545 INFO [pool-6-thread-195]: metastore.HiveMetaStore
>
[jira] [Updated] (HIVE-22566) Drop table involved in materialized view leaves the table in inconsistent state
[
https://issues.apache.org/jira/browse/HIVE-22566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pablo Junge updated HIVE-22566:
---
Attachment: HIVE-22566.patch
Status: Patch Available (was: Open)
> Drop table involved in materialized view leaves the table in inconsistent
> state
> ---
>
> Key: HIVE-22566
> URL: https://issues.apache.org/jira/browse/HIVE-22566
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Pablo Junge
>Assignee: Pablo Junge
>Priority: Minor
> Attachments: HIVE-22566.patch
>
>
> If you try dropping a table which is part of the definition of a created
> materialized view, the table is not dropped, which is the desired state as it
> is part of the materialized view.
> However, there was a "drop" call to the table, so it tried to drop it but did
> not succeed, leaving it in an inconsistent state.
>
> Repro:
> ---
> 1) Create tables:
>
> {code:java}
> CREATE TABLE emps ( empid INT, deptno INT, name VARCHAR(256), salary
> FLOAT, hire_date TIMESTAMP)STORED AS ORC TBLPROPERTIES
> ('transactional'='true');
> CREATE TABLE depts ( deptno INT, deptname VARCHAR(256), locationid
> INT)STORED AS ORC TBLPROPERTIES ('transactional'='true');
> {code}
>
> 2) Create the VM:
>
> {code:java}
> CREATE MATERIALIZED VIEW mv1 AS SELECT empid, deptname, hire_date FROM emps
> JOIN depts ON (emps.deptno = depts.deptno) WHERE hire_date >= '2016-01-01';
> {code}
>
> 3) Following is in backend database at this point:
>
> {code:java}
> mysql> select TBL_ID, DB_ID, SD_ID, TBL_NAME, TBL_TYPE from TBLS where
> DB_ID=16;
> ++---+---+--+---+
> | TBL_ID | DB_ID | SD_ID | TBL_NAME | TBL_TYPE |
> ++---+---+--+---+
> | 81 | 16 | 81 | emps | MANAGED_TABLE |
> | 83 | 16 | 83 | depts | MANAGED_TABLE |
> | 84 | 16 | 84 | mv1 | MATERIALIZED_VIEW |
> ++---+---+--+---+
> 3 rows in set (0.00 sec)
> {code}
>
> 4) Let's drop the 'emps' table:
>
> {code:java}
> 0: jdbc:hive2://c1122-node2.squadron.support.> drop table emps;
> INFO : Compiling
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b):
> drop table emps
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b);
> Time taken: 0.05 seconds
> INFO : Executing
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b):
> drop table emps
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b);
> Time taken: 10.281 seconds
> INFO : OK
> No rows affected (16.949 seconds)
> {code}
> No issue displayed
>
> 5) List tables:
>
> {code:java}
> 0: jdbc:hive2://c1122-node2.squadron.support.> show tables;
> INFO : Compiling
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413):
> show tables
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from
> deserializer)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413);
> Time taken: 0.041 seconds
> INFO : Executing
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413):
> show tables
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413);
> Time taken: 0.016 seconds
> INFO : OK
> +---+
> | tab_name |
> +---+
> | depts |
> | emps |
> +---+
> 2 rows selected (0.08 seconds)
> {code}
>
> 6) Now, from the backend-db point of view:
>
> {code:java}
> mysql> select TBL_ID, DB_ID, SD_ID, TBL_NAME, TBL_TYPE from TBLS where
> DB_ID=16;
> ++---+---+--+---+
> | TBL_ID | DB_ID | SD_ID | TBL_NAME | TBL_TYPE |
> ++---+---+--+---+
> | 81 | 16 | NULL | emps | MANAGED_TABLE |
> | 83 | 16 | 83 | depts | MANAGED_TABLE |
> | 84 | 16 | 84 | mv1 | MATERIALIZED_VIEW |
> ++---+---+--+---+
> 3 rows in set (0.00 sec)
> {code}
> The table is left with NULL in SD_ID, making it not available.
>
> 7) From Metastore.log
>
> {code:java}
> 2019-12-02T20:00:25,545 INFO
[jira] [Commented] (HIVE-22599) Query results cache: 733 permissions check is not necessary
[
https://issues.apache.org/jira/browse/HIVE-22599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993036#comment-16993036
]
Hive QA commented on HIVE-22599:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
27s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19860/dev-support/hive-personality.sh
|
| git revision | master / 22e298c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19860/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19860/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Query results cache: 733 permissions check is not necessary
> ---
>
> Key: HIVE-22599
> URL: https://issues.apache.org/jira/browse/HIVE-22599
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-22599.1.patch
>
>
> The query results cache initialization makes a call to
> Utilties.ensurePathIsWritable(), which checks the results cache directory for
> 733 permissions (default cache dir is
> {{/tmp/hive/_resultscache_).}}
> The 733 permissions (at least the 033 part) are not actually necessary - we
> actually don't really want the results cache directory to be world-writable,
> and the subdirectories we create within this one are actually done with 700
> perms. So I think the call to Utilties.ensurePathIsWritable() can be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22566) Drop table involved in materialized view leaves the table in inconsistent state
[
https://issues.apache.org/jira/browse/HIVE-22566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pablo Junge reassigned HIVE-22566:
--
Assignee: Pablo Junge
> Drop table involved in materialized view leaves the table in inconsistent
> state
> ---
>
> Key: HIVE-22566
> URL: https://issues.apache.org/jira/browse/HIVE-22566
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.0
>Reporter: Pablo Junge
>Assignee: Pablo Junge
>Priority: Minor
>
> If you try dropping a table which is part of the definition of a created
> materialized view, the table is not dropped, which is the desired state as it
> is part of the materialized view.
> However, there was a "drop" call to the table, so it tried to drop it but did
> not succeed, leaving it in an inconsistent state.
>
> Repro:
> ---
> 1) Create tables:
>
> {code:java}
> CREATE TABLE emps ( empid INT, deptno INT, name VARCHAR(256), salary
> FLOAT, hire_date TIMESTAMP)STORED AS ORC TBLPROPERTIES
> ('transactional'='true');
> CREATE TABLE depts ( deptno INT, deptname VARCHAR(256), locationid
> INT)STORED AS ORC TBLPROPERTIES ('transactional'='true');
> {code}
>
> 2) Create the VM:
>
> {code:java}
> CREATE MATERIALIZED VIEW mv1 AS SELECT empid, deptname, hire_date FROM emps
> JOIN depts ON (emps.deptno = depts.deptno) WHERE hire_date >= '2016-01-01';
> {code}
>
> 3) Following is in backend database at this point:
>
> {code:java}
> mysql> select TBL_ID, DB_ID, SD_ID, TBL_NAME, TBL_TYPE from TBLS where
> DB_ID=16;
> ++---+---+--+---+
> | TBL_ID | DB_ID | SD_ID | TBL_NAME | TBL_TYPE |
> ++---+---+--+---+
> | 81 | 16 | 81 | emps | MANAGED_TABLE |
> | 83 | 16 | 83 | depts | MANAGED_TABLE |
> | 84 | 16 | 84 | mv1 | MATERIALIZED_VIEW |
> ++---+---+--+---+
> 3 rows in set (0.00 sec)
> {code}
>
> 4) Let's drop the 'emps' table:
>
> {code:java}
> 0: jdbc:hive2://c1122-node2.squadron.support.> drop table emps;
> INFO : Compiling
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b):
> drop table emps
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b);
> Time taken: 0.05 seconds
> INFO : Executing
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b):
> drop table emps
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing
> command(queryId=hive_20191202200025_c13079d0-8695-4485-8a18-14804b8b014b);
> Time taken: 10.281 seconds
> INFO : OK
> No rows affected (16.949 seconds)
> {code}
> No issue displayed
>
> 5) List tables:
>
> {code:java}
> 0: jdbc:hive2://c1122-node2.squadron.support.> show tables;
> INFO : Compiling
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413):
> show tables
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from
> deserializer)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413);
> Time taken: 0.041 seconds
> INFO : Executing
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413):
> show tables
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing
> command(queryId=hive_20191202200125_ca12565b-1d4d-4433-a602-ecf685863413);
> Time taken: 0.016 seconds
> INFO : OK
> +---+
> | tab_name |
> +---+
> | depts |
> | emps |
> +---+
> 2 rows selected (0.08 seconds)
> {code}
>
> 6) Now, from the backend-db point of view:
>
> {code:java}
> mysql> select TBL_ID, DB_ID, SD_ID, TBL_NAME, TBL_TYPE from TBLS where
> DB_ID=16;
> ++---+---+--+---+
> | TBL_ID | DB_ID | SD_ID | TBL_NAME | TBL_TYPE |
> ++---+---+--+---+
> | 81 | 16 | NULL | emps | MANAGED_TABLE |
> | 83 | 16 | 83 | depts | MANAGED_TABLE |
> | 84 | 16 | 84 | mv1 | MATERIALIZED_VIEW |
> ++---+---+--+---+
> 3 rows in set (0.00 sec)
> {code}
> The table is left with NULL in SD_ID, making it not available.
>
> 7) From Metastore.log
>
> {code:java}
> 2019-12-02T20:00:25,545 INFO [pool-6-thread-195]: metastore.HiveMetaStore
> (HiveMetaStore.java:logInfo(907)) - 196:
[jira] [Commented] (HIVE-22587) hive.stats.ndv.error parameter documentation issue in HiveConf.java
[ https://issues.apache.org/jira/browse/HIVE-22587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993032#comment-16993032 ] Pablo Junge commented on HIVE-22587: Thanks to you [~ashutoshc] ! > hive.stats.ndv.error parameter documentation issue in HiveConf.java > --- > > Key: HIVE-22587 > URL: https://issues.apache.org/jira/browse/HIVE-22587 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Pablo Junge >Assignee: Pablo Junge >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22587.patch > > > hive.stats.ndv.error parameter documentation should specify that it only > affects the FM-Sketch algorithm -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[ https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Draese updated HIVE-22572: - Attachment: HIVE-22572.5.patch > NullPointerException when using dynamic semijoin reduction > -- > > Key: HIVE-22572 > URL: https://issues.apache.org/jira/browse/HIVE-22572 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Oliver Draese >Assignee: Oliver Draese >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22572.1.patch, HIVE-22572.2.patch, > HIVE-22572.3.patch, HIVE-22572.4.patch, HIVE-22572.5.patch, HIVE-22572.patch > > > From HS2 logs > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[ https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993015#comment-16993015 ] Oliver Draese commented on HIVE-22572: -- Test failures are not related to patch. Re-attached patch to trigger new test run. > NullPointerException when using dynamic semijoin reduction > -- > > Key: HIVE-22572 > URL: https://issues.apache.org/jira/browse/HIVE-22572 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Oliver Draese >Assignee: Oliver Draese >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22572.1.patch, HIVE-22572.2.patch, > HIVE-22572.3.patch, HIVE-22572.4.patch, HIVE-22572.5.patch, HIVE-22572.patch > > > From HS2 logs > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22599) Query results cache: 733 permissions check is not necessary
[
https://issues.apache.org/jira/browse/HIVE-22599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-22599:
--
Attachment: HIVE-22599.1.patch
> Query results cache: 733 permissions check is not necessary
> ---
>
> Key: HIVE-22599
> URL: https://issues.apache.org/jira/browse/HIVE-22599
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-22599.1.patch
>
>
> The query results cache initialization makes a call to
> Utilties.ensurePathIsWritable(), which checks the results cache directory for
> 733 permissions (default cache dir is
> {{/tmp/hive/_resultscache_).}}
> The 733 permissions (at least the 033 part) are not actually necessary - we
> actually don't really want the results cache directory to be world-writable,
> and the subdirectories we create within this one are actually done with 700
> perms. So I think the call to Utilties.ensurePathIsWritable() can be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22599) Query results cache: 733 permissions check is not necessary
[
https://issues.apache.org/jira/browse/HIVE-22599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-22599:
--
Attachment: (was: HIVE-22599.1.patch)
> Query results cache: 733 permissions check is not necessary
> ---
>
> Key: HIVE-22599
> URL: https://issues.apache.org/jira/browse/HIVE-22599
> Project: Hive
> Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-22599.1.patch
>
>
> The query results cache initialization makes a call to
> Utilties.ensurePathIsWritable(), which checks the results cache directory for
> 733 permissions (default cache dir is
> {{/tmp/hive/_resultscache_).}}
> The 733 permissions (at least the 033 part) are not actually necessary - we
> actually don't really want the results cache directory to be world-writable,
> and the subdirectories we create within this one are actually done with 700
> perms. So I think the call to Utilties.ensurePathIsWritable() can be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22583) LLAP cache always misses with non-vectorized serde readers such as OpenCSV
[ https://issues.apache.org/jira/browse/HIVE-22583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993009#comment-16993009 ] Slim Bouguerra commented on HIVE-22583: --- The fix looks good to me. I try to avoid using TEZ counters as a part of the test validation since they can change quite drastically if the plan changes but i understand that it is the only way for now to test the data in the cache. One of the idea i want to implement is to have a query flag that tells LLAP to run a query against the cache content only and that can enable such tests and will avoid to use the the TEZ counters. > LLAP cache always misses with non-vectorized serde readers such as OpenCSV > -- > > Key: HIVE-22583 > URL: https://issues.apache.org/jira/browse/HIVE-22583 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-22583.0.patch, HIVE-22583.1.patch, > HIVE-22583.2.patch > > > Although after the first read LLAP cache stores data of tables that are not > using the LazySimple serde, the stored data is then never used in the future > subsequent queries, causing a full cache miss and re-read each time. > Problem is rooted in SerdeEncodedDataReader#cacheFileData is not taking care > of creating an entry for the root/struct column of the table. The only cases > this is taken care of are when a vectorized reader is used _(e.g. > LazySimpleSerde's LazySimpleDeserializeRead)_, where > SerdeEncodedDataReader#processAsyncCacheData takes care of this. > This can be reproduced by either using a custom serde, like OpenCSV or using > LazySimpleSerde, but turning off _hive.llap.io.encode.vector.serde.enabled_. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[
https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16993000#comment-16993000
]
Hive QA commented on HIVE-22572:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988460/HIVE-22572.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17762 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query23]
(batchId=301)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=276)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19859/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19859/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19859/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988460 - PreCommit-HIVE-Build
> NullPointerException when using dynamic semijoin reduction
> --
>
> Key: HIVE-22572
> URL: https://issues.apache.org/jira/browse/HIVE-22572
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22572.1.patch, HIVE-22572.2.patch,
> HIVE-22572.3.patch, HIVE-22572.4.patch, HIVE-22572.patch
>
>
> From HS2 logs
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22461) NPE Metastore Transformer
[
https://issues.apache.org/jira/browse/HIVE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992952#comment-16992952
]
Naveen Gangam commented on HIVE-22461:
--
Change looks good to me. +1 for me. Thank you [~ychena]
> NPE Metastore Transformer
> -
>
> Key: HIVE-22461
> URL: https://issues.apache.org/jira/browse/HIVE-22461
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-22461.1.patch, HIVE-22461.5.patch,
> HIVE-22461.6.patch
>
>
> The stack looks as following:
> {noformat}
> 2019-10-08 18:09:12,198 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-328]: Starting translation for processor
> Hiveserver2#3.1.2000.7.0.2.0...@vc0732.halxg.cloudera.com on list 1
> 2019-10-08 18:09:12,198 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-328]:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16617)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:636)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:631)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:631)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-10-08 18:09:12,199 ERROR org.apache.thrift.server.TThreadPoolServer:
> [pool-6-thread-328]: Error occurred during processing of message.
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) ~[?:?]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_141]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source) ~[?:?]
> at
>
[jira] [Commented] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[
https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992950#comment-16992950
]
Hive QA commented on HIVE-22572:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
34s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
16s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19859/dev-support/hive-personality.sh
|
| git revision | master / 22e298c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19859/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19859/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> NullPointerException when using dynamic semijoin reduction
> --
>
> Key: HIVE-22572
> URL: https://issues.apache.org/jira/browse/HIVE-22572
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22572.1.patch, HIVE-22572.2.patch,
> HIVE-22572.3.patch, HIVE-22572.4.patch, HIVE-22572.patch
>
>
> From HS2 logs
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
> ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1]
> at
>
[jira] [Commented] (HIVE-22594) Reorder metastore thrift structures
[
https://issues.apache.org/jira/browse/HIVE-22594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992926#comment-16992926
]
Hive QA commented on HIVE-22594:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988453/HIVE-22594.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17762 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=276)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19857/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19857/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19857/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988453 - PreCommit-HIVE-Build
> Reorder metastore thrift structures
> ---
>
> Key: HIVE-22594
> URL: https://issues.apache.org/jira/browse/HIVE-22594
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Thrift API
>Affects Versions: 4.0.0
>Reporter: John Sherman
>Assignee: John Sherman
>Priority: Major
> Attachments: HIVE-22594.1.patch, HIVE-22594.2.patch,
> HIVE-22594.3.patch
>
>
> Currently hive_metastore.thrift C++ output does not compile due to structures
> like ScheduledQueryKey and ColumnStatistics being referenced before being
> defined later in the file. We should reorder the structures in the file so
> the C++ output is usable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22579) ACID v1: covered delta-only splits (without base) should be marked as covered (branch-2)
[
https://issues.apache.org/jira/browse/HIVE-22579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992913#comment-16992913
]
László Bodor commented on HIVE-22579:
-
pushed to branch-2, thanks for the review [~pvary]
> ACID v1: covered delta-only splits (without base) should be marked as covered
> (branch-2)
>
>
> Key: HIVE-22579
> URL: https://issues.apache.org/jira/browse/HIVE-22579
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 2.3.7
>
> Attachments: HIVE-22579.01.branch-2.patch,
> HIVE-22579.01.branch-2.patch
>
>
> There is a scenario when different SplitGenerator instances try to cover the
> delta-only buckets (having no base file) more than once, so there could be
> multiple OrcSplit instances generated for the same delta file, causing more
> tasks to read the same delta file more than once, causing duplicate records
> in a simple select star query.
> File structure for a 256 bucket table
> {code}
> drwxrwxrwx - hive hadoop 0 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013
> -rw-r--r-- 3 hive hadoop353 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013/bucket_00012
> -rw-r--r-- 3 hive hadoop 1642 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_/bucket_00012
> -rwxrwxrwx 3 hive hadoop 1635 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_/bucket_00012
> -rwxrwxrwx 3 hive hadoop 1808 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00043
> -rwxrwxrwx 3 hive hadoop 1633 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00171
> {code}
> in this case, when bucket_00171 file has a record, and there is no base file
> for that, a select (*) with ETL split strategy can generate 2 splits for the
> same delta bucket...
> the scenario of the issue:
> 1. ETLSplitStrategy contains a [covered[]
> array|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L763]
> which is [shared between the SplitInfo
> instances|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L824]
> to be created
> 2. a SplitInfo instance is created for [every base file (2 in this
> case)|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L809]
> 3. for every SplitInfo, [a SplitGenerator is
> created|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L925-L926],
> and in the constructor, [parent's getSplit is
> called|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L1251],
> which tries to take care of the deltas
> I'm not sure at the moment what's the intention of this, but this way,
> duplicated delta split can be generated, which can cause duplicated read
> later (note that both tasks read the same delta file: bucket_00171)
> {code}
> 2019-12-01T16:24:53,669 INFO [TezTR-127843_16_30_0_171_0
> (1575040127843_0016_30_00_000171_0)] orc.ReaderImpl: Reading ORC rows from
> hdfs://c3351-node2.squadron.support.hortonworks.com:8020/apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00171
> with {include: [true, true, true, true, true, true, true, true, true, true,
> true, true], offset: 0, length: 9223372036854775807, schema:
> struct}
> 2019-12-01T16:24:53,672 INFO [TezTR-127843_16_30_0_171_0
> (1575040127843_0016_30_00_000171_0)] lib.MRReaderMapred: Processing split:
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit
>
[jira] [Updated] (HIVE-22579) ACID v1: covered delta-only splits (without base) should be marked as covered (branch-2)
[
https://issues.apache.org/jira/browse/HIVE-22579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-22579:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> ACID v1: covered delta-only splits (without base) should be marked as covered
> (branch-2)
>
>
> Key: HIVE-22579
> URL: https://issues.apache.org/jira/browse/HIVE-22579
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 2.3.7
>
> Attachments: HIVE-22579.01.branch-2.patch,
> HIVE-22579.01.branch-2.patch
>
>
> There is a scenario when different SplitGenerator instances try to cover the
> delta-only buckets (having no base file) more than once, so there could be
> multiple OrcSplit instances generated for the same delta file, causing more
> tasks to read the same delta file more than once, causing duplicate records
> in a simple select star query.
> File structure for a 256 bucket table
> {code}
> drwxrwxrwx - hive hadoop 0 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013
> -rw-r--r-- 3 hive hadoop353 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013/bucket_00012
> -rw-r--r-- 3 hive hadoop 1642 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_/bucket_00012
> -rwxrwxrwx 3 hive hadoop 1635 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_/bucket_00012
> -rwxrwxrwx 3 hive hadoop 1808 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00043
> -rwxrwxrwx 3 hive hadoop 1633 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00171
> {code}
> in this case, when bucket_00171 file has a record, and there is no base file
> for that, a select (*) with ETL split strategy can generate 2 splits for the
> same delta bucket...
> the scenario of the issue:
> 1. ETLSplitStrategy contains a [covered[]
> array|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L763]
> which is [shared between the SplitInfo
> instances|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L824]
> to be created
> 2. a SplitInfo instance is created for [every base file (2 in this
> case)|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L809]
> 3. for every SplitInfo, [a SplitGenerator is
> created|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L925-L926],
> and in the constructor, [parent's getSplit is
> called|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L1251],
> which tries to take care of the deltas
> I'm not sure at the moment what's the intention of this, but this way,
> duplicated delta split can be generated, which can cause duplicated read
> later (note that both tasks read the same delta file: bucket_00171)
> {code}
> 2019-12-01T16:24:53,669 INFO [TezTR-127843_16_30_0_171_0
> (1575040127843_0016_30_00_000171_0)] orc.ReaderImpl: Reading ORC rows from
> hdfs://c3351-node2.squadron.support.hortonworks.com:8020/apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00171
> with {include: [true, true, true, true, true, true, true, true, true, true,
> true, true], offset: 0, length: 9223372036854775807, schema:
> struct}
> 2019-12-01T16:24:53,672 INFO [TezTR-127843_16_30_0_171_0
> (1575040127843_0016_30_00_000171_0)] lib.MRReaderMapred: Processing split:
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit
>
[jira] [Updated] (HIVE-22579) ACID v1: covered delta-only splits (without base) should be marked as covered (branch-2)
[
https://issues.apache.org/jira/browse/HIVE-22579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-22579:
Fix Version/s: 2.3.7
> ACID v1: covered delta-only splits (without base) should be marked as covered
> (branch-2)
>
>
> Key: HIVE-22579
> URL: https://issues.apache.org/jira/browse/HIVE-22579
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 2.3.7
>
> Attachments: HIVE-22579.01.branch-2.patch,
> HIVE-22579.01.branch-2.patch
>
>
> There is a scenario when different SplitGenerator instances try to cover the
> delta-only buckets (having no base file) more than once, so there could be
> multiple OrcSplit instances generated for the same delta file, causing more
> tasks to read the same delta file more than once, causing duplicate records
> in a simple select star query.
> File structure for a 256 bucket table
> {code}
> drwxrwxrwx - hive hadoop 0 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013
> -rw-r--r-- 3 hive hadoop353 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013/bucket_00012
> -rw-r--r-- 3 hive hadoop 1642 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/base_013/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_/bucket_00012
> -rwxrwxrwx 3 hive hadoop 1635 2019-11-29 15:55
> /apps/hive/warehouse/naresh.db/test1/delta_014_014_/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_/bucket_00012
> -rwxrwxrwx 3 hive hadoop 1808 2019-11-29 16:04
> /apps/hive/warehouse/naresh.db/test1/delta_015_015_/bucket_00140
> drwxrwxrwx - hive hadoop 0 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_
> -rwxrwxrwx 3 hive hadoop348 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00043
> -rwxrwxrwx 3 hive hadoop 1633 2019-11-29 16:06
> /apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00171
> {code}
> in this case, when bucket_00171 file has a record, and there is no base file
> for that, a select (*) with ETL split strategy can generate 2 splits for the
> same delta bucket...
> the scenario of the issue:
> 1. ETLSplitStrategy contains a [covered[]
> array|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L763]
> which is [shared between the SplitInfo
> instances|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L824]
> to be created
> 2. a SplitInfo instance is created for [every base file (2 in this
> case)|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L809]
> 3. for every SplitInfo, [a SplitGenerator is
> created|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L925-L926],
> and in the constructor, [parent's getSplit is
> called|https://github.com/apache/hive/blob/298f749ec7be04abb797fb119f3f0b923c8a1b27/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L1251],
> which tries to take care of the deltas
> I'm not sure at the moment what's the intention of this, but this way,
> duplicated delta split can be generated, which can cause duplicated read
> later (note that both tasks read the same delta file: bucket_00171)
> {code}
> 2019-12-01T16:24:53,669 INFO [TezTR-127843_16_30_0_171_0
> (1575040127843_0016_30_00_000171_0)] orc.ReaderImpl: Reading ORC rows from
> hdfs://c3351-node2.squadron.support.hortonworks.com:8020/apps/hive/warehouse/naresh.db/test1/delta_016_016_/bucket_00171
> with {include: [true, true, true, true, true, true, true, true, true, true,
> true, true], offset: 0, length: 9223372036854775807, schema:
> struct}
> 2019-12-01T16:24:53,672 INFO [TezTR-127843_16_30_0_171_0
> (1575040127843_0016_30_00_000171_0)] lib.MRReaderMapred: Processing split:
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat:OrcSplit
>
[jira] [Commented] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[ https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992894#comment-16992894 ] Oliver Draese commented on HIVE-22572: -- Test failures are not related to patch. Re-attached patch to trigger new test run. > NullPointerException when using dynamic semijoin reduction > -- > > Key: HIVE-22572 > URL: https://issues.apache.org/jira/browse/HIVE-22572 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Oliver Draese >Assignee: Oliver Draese >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22572.1.patch, HIVE-22572.2.patch, > HIVE-22572.3.patch, HIVE-22572.4.patch, HIVE-22572.patch > > > From HS2 logs > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[ https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Draese updated HIVE-22572: - Attachment: HIVE-22572.4.patch > NullPointerException when using dynamic semijoin reduction > -- > > Key: HIVE-22572 > URL: https://issues.apache.org/jira/browse/HIVE-22572 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Oliver Draese >Assignee: Oliver Draese >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22572.1.patch, HIVE-22572.2.patch, > HIVE-22572.3.patch, HIVE-22572.4.patch, HIVE-22572.patch > > > From HS2 logs > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22594) Reorder metastore thrift structures
[
https://issues.apache.org/jira/browse/HIVE-22594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992880#comment-16992880
]
Hive QA commented on HIVE-22594:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19857/dev-support/hive-personality.sh
|
| git revision | master / 22e298c |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19857/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore/metastore-common U:
standalone-metastore/metastore-common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19857/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reorder metastore thrift structures
> ---
>
> Key: HIVE-22594
> URL: https://issues.apache.org/jira/browse/HIVE-22594
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Thrift API
>Affects Versions: 4.0.0
>Reporter: John Sherman
>Assignee: John Sherman
>Priority: Major
> Attachments: HIVE-22594.1.patch, HIVE-22594.2.patch,
> HIVE-22594.3.patch
>
>
> Currently hive_metastore.thrift C++ output does not compile due to structures
> like ScheduledQueryKey and ColumnStatistics being referenced before being
> defined later in the file. We should reorder the structures in the file so
> the C++ output is usable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: (was: HIVE-22486.addendum.01.patch) > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486-addendum.01.patch, HIVE-22486.01.patch, > HIVE-22486.02.patch, HIVE-22486.03.patch, HIVE-22486.05.patch, > HIVE-22486.06.patch, HIVE-22486.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: (was: HIVE-22486.addendum.01.patch) > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486-addendum.01.patch, HIVE-22486.01.patch, > HIVE-22486.02.patch, HIVE-22486.03.patch, HIVE-22486.05.patch, > HIVE-22486.06.patch, HIVE-22486.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: HIVE-22486-addendum.01.patch > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486-addendum.01.patch, HIVE-22486.01.patch, > HIVE-22486.02.patch, HIVE-22486.03.patch, HIVE-22486.05.patch, > HIVE-22486.06.patch, HIVE-22486.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: (was: HIVE-22486.addendum.01.patch) > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486-addendum.01.patch, HIVE-22486.01.patch, > HIVE-22486.02.patch, HIVE-22486.03.patch, HIVE-22486.05.patch, > HIVE-22486.06.patch, HIVE-22486.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22594) Reorder metastore thrift structures
[ https://issues.apache.org/jira/browse/HIVE-22594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-22594: Attachment: HIVE-22594.3.patch > Reorder metastore thrift structures > --- > > Key: HIVE-22594 > URL: https://issues.apache.org/jira/browse/HIVE-22594 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore, Thrift API >Affects Versions: 4.0.0 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Attachments: HIVE-22594.1.patch, HIVE-22594.2.patch, > HIVE-22594.3.patch > > > Currently hive_metastore.thrift C++ output does not compile due to structures > like ScheduledQueryKey and ColumnStatistics being referenced before being > defined later in the file. We should reorder the structures in the file so > the C++ output is usable. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: HIVE-22486.addendum.01.patch > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch, > HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch, > HIVE-22486.addendum.01.patch, HIVE-22486.addendum.01.patch, > HIVE-22486.addendum.01.patch, HIVE-22486.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22486) Send only accessed columns for masking policies request
[ https://issues.apache.org/jira/browse/HIVE-22486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22486: --- Attachment: HIVE-22486.addendum.01.patch > Send only accessed columns for masking policies request > --- > > Key: HIVE-22486 > URL: https://issues.apache.org/jira/browse/HIVE-22486 > Project: Hive > Issue Type: Improvement > Components: CBO >Affects Versions: 4.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22486.01.patch, HIVE-22486.02.patch, > HIVE-22486.03.patch, HIVE-22486.05.patch, HIVE-22486.06.patch, > HIVE-22486.addendum.01.patch, HIVE-22486.addendum.01.patch, HIVE-22486.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Currently, we send all columns for masking request, even if they are not > accessed by the given query. We could send only those columns for which the > masking policy will be necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22461) NPE Metastore Transformer
[
https://issues.apache.org/jira/browse/HIVE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992755#comment-16992755
]
Hive QA commented on HIVE-22461:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988364/HIVE-22461.6.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17762 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19853/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19853/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19853/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988364 - PreCommit-HIVE-Build
> NPE Metastore Transformer
> -
>
> Key: HIVE-22461
> URL: https://issues.apache.org/jira/browse/HIVE-22461
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-22461.1.patch, HIVE-22461.5.patch,
> HIVE-22461.6.patch
>
>
> The stack looks as following:
> {noformat}
> 2019-10-08 18:09:12,198 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-328]: Starting translation for processor
> Hiveserver2#3.1.2000.7.0.2.0...@vc0732.halxg.cloudera.com on list 1
> 2019-10-08 18:09:12,198 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-328]:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.get_table_req(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16633)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_table_req.getResult(ThriftHiveMetastore.java:16617)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:636)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:631)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:631)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-10-08 18:09:12,199 ERROR org.apache.thrift.server.TThreadPoolServer:
> [pool-6-thread-328]: Error occurred during processing of message.
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> ~[hive-exec-3.1.2000.7.0.2.0-59.jar:3.1.2000.7.0.2.0-59]
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
>
[jira] [Updated] (HIVE-22498) Schema tool enhancements to merge catalogs
[ https://issues.apache.org/jira/browse/HIVE-22498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22498: - Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Fix has been committed to master. Thank you for the review [~samuelan]. > Schema tool enhancements to merge catalogs > -- > > Key: HIVE-22498 > URL: https://issues.apache.org/jira/browse/HIVE-22498 > Project: Hive > Issue Type: Sub-task >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22498.2.patch, HIVE-22498.patch > > > Schema tool currently supports relocation of database from one catalog to > another, one at a time. While having to do this one at a time is painful, it > also lacks support for converting them to external tables during migration, > in lieu of the changes to the translation layer where a MANAGED table is > strictly ACID-only table. > Hence we also need to convert them to external tables during relocation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-16690) Configure Tez cartesian product edge based on LLAP cluster size
[ https://issues.apache.org/jira/browse/HIVE-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992714#comment-16992714 ] Ashutosh Chauhan commented on HIVE-16690: - +1 > Configure Tez cartesian product edge based on LLAP cluster size > --- > > Key: HIVE-16690 > URL: https://issues.apache.org/jira/browse/HIVE-16690 > Project: Hive > Issue Type: Bug >Reporter: Zhiyuan Yang >Assignee: László Bodor >Priority: Major > Attachments: HIVE-16690.03.patch, HIVE-16690.1.patch, > HIVE-16690.2.patch, HIVE-16690.2.patch, HIVE-16690.2.patch, > HIVE-16690.2.patch, HIVE-16690.2.patch, HIVE-16690.2.patch, > HIVE-16690.2.patch, HIVE-16690.2.patch, HIVE-16690.addendum.patch > > > In HIVE-14731 we are using default value for target parallelism of fair > cartesian product edge. Ideally this should be set according to cluster size. > In case of LLAP it's pretty easy to get cluster size, i.e., number of > executors. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22461) NPE Metastore Transformer
[
https://issues.apache.org/jira/browse/HIVE-22461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992709#comment-16992709
]
Hive QA commented on HIVE-22461:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
27s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 374 unchanged - 33 fixed = 375 total (was 407) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
29s{color} | {color:green} standalone-metastore/metastore-server generated 0
new + 178 unchanged - 1 fixed = 178 total (was 179) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19853/dev-support/hive-personality.sh
|
| git revision | master / 879d45c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19853/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19853/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> NPE Metastore Transformer
> -
>
> Key: HIVE-22461
> URL: https://issues.apache.org/jira/browse/HIVE-22461
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.2
>Reporter: Yongzhi Chen
>Assignee: Yongzhi Chen
>Priority: Major
> Attachments: HIVE-22461.1.patch, HIVE-22461.5.patch,
> HIVE-22461.6.patch
>
>
> The stack looks as following:
> {noformat}
> 2019-10-08 18:09:12,198 INFO
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer:
> [pool-6-thread-328]: Starting translation for processor
> Hiveserver2#3.1.2000.7.0.2.0...@vc0732.halxg.cloudera.com on list 1
> 2019-10-08 18:09:12,198 ERROR
> org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-6-thread-328]:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer.transform(MetastoreDefaultTransformer.java:99)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getTableInternal(HiveMetaStore.java:3391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_req(HiveMetaStore.java:3352)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
>
[jira] [Commented] (HIVE-22570) Review of ExprNodeDesc.java
[
https://issues.apache.org/jira/browse/HIVE-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992693#comment-16992693
]
David Mollitor commented on HIVE-22570:
---
So, pretty easy...
There are only 7 calls to {{getCols}} and most all of them are passed to
{{Utilities.mergeUniqElems}} which will handle the empty list just fine. The
other places check for 'null or empty' (I won't bother improving those now).
There are many more references to {{getChildren()}} however, the first few that
I sampled don't even both to check for a 'null' return value, so they are at
risk of a NPE, which is very common and why returning 'null' is generally a bad
idea. For example:
{code}
/**
* Find the variable in the expression.
* @param expr the expression to look in
* @return the index of the variable or -1 if there is not exactly one
* variable.
*/
private int findVariable(ExprNodeDesc expr) {
int result = -1;
List children = expr.getChildren();
for(int i = 0; i < children.size(); ++i) {
ExprNodeDesc child = children.get(i);
if (child instanceof ExprNodeColumnDesc) {
// if we already found a variable, this isn't a sarg
if (result != -1) {
return -1;
} else {
result = i;
}
}
}
return result;
}
{code}
I can do some follow-on work to tighten up the code (and to remove all of the
'null' checks that do exist) in another ticket.
> Review of ExprNodeDesc.java
> ---
>
> Key: HIVE-22570
> URL: https://issues.apache.org/jira/browse/HIVE-22570
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22570.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-16690) Configure Tez cartesian product edge based on LLAP cluster size
[ https://issues.apache.org/jira/browse/HIVE-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-16690: Attachment: HIVE-16690.03.patch > Configure Tez cartesian product edge based on LLAP cluster size > --- > > Key: HIVE-16690 > URL: https://issues.apache.org/jira/browse/HIVE-16690 > Project: Hive > Issue Type: Bug >Reporter: Zhiyuan Yang >Assignee: László Bodor >Priority: Major > Attachments: HIVE-16690.03.patch, HIVE-16690.1.patch, > HIVE-16690.2.patch, HIVE-16690.2.patch, HIVE-16690.2.patch, > HIVE-16690.2.patch, HIVE-16690.2.patch, HIVE-16690.2.patch, > HIVE-16690.2.patch, HIVE-16690.2.patch, HIVE-16690.addendum.patch > > > In HIVE-14731 we are using default value for target parallelism of fair > cartesian product edge. Ideally this should be set according to cluster size. > In case of LLAP it's pretty easy to get cluster size, i.e., number of > executors. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-16690) Configure Tez cartesian product edge based on LLAP cluster size
[ https://issues.apache.org/jira/browse/HIVE-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992684#comment-16992684 ] László Bodor commented on HIVE-16690: - vectorized_mapjoin3 seems related, changed ordering in llap, uploading new patch > Configure Tez cartesian product edge based on LLAP cluster size > --- > > Key: HIVE-16690 > URL: https://issues.apache.org/jira/browse/HIVE-16690 > Project: Hive > Issue Type: Bug >Reporter: Zhiyuan Yang >Assignee: László Bodor >Priority: Major > Attachments: HIVE-16690.1.patch, HIVE-16690.2.patch, > HIVE-16690.2.patch, HIVE-16690.2.patch, HIVE-16690.2.patch, > HIVE-16690.2.patch, HIVE-16690.2.patch, HIVE-16690.2.patch, > HIVE-16690.2.patch, HIVE-16690.addendum.patch > > > In HIVE-14731 we are using default value for target parallelism of fair > cartesian product edge. Ideally this should be set according to cluster size. > In case of LLAP it's pretty easy to get cluster size, i.e., number of > executors. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22594) Reorder metastore thrift structures
[
https://issues.apache.org/jira/browse/HIVE-22594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992685#comment-16992685
]
Hive QA commented on HIVE-22594:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12988357/HIVE-22594.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17762 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query17]
(batchId=301)
org.apache.hadoop.hive.metastore.TestPartitionManagement.testPartitionDiscoveryTransactionalTable
(batchId=225)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19852/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19852/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19852/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12988357 - PreCommit-HIVE-Build
> Reorder metastore thrift structures
> ---
>
> Key: HIVE-22594
> URL: https://issues.apache.org/jira/browse/HIVE-22594
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Thrift API
>Affects Versions: 4.0.0
>Reporter: John Sherman
>Assignee: John Sherman
>Priority: Major
> Attachments: HIVE-22594.1.patch, HIVE-22594.2.patch
>
>
> Currently hive_metastore.thrift C++ output does not compile due to structures
> like ScheduledQueryKey and ColumnStatistics being referenced before being
> defined later in the file. We should reorder the structures in the file so
> the C++ output is usable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-17440) TestSparkCliDriver.testCliDriver[vectorized_ptf]
[ https://issues.apache.org/jira/browse/HIVE-17440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992665#comment-16992665 ] Zoltan Haindrich commented on HIVE-17440: - +1 > TestSparkCliDriver.testCliDriver[vectorized_ptf] > > > Key: HIVE-17440 > URL: https://issues.apache.org/jira/browse/HIVE-17440 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17440.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-17441) TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs]
[ https://issues.apache.org/jira/browse/HIVE-17441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992663#comment-16992663 ] Zoltan Haindrich commented on HIVE-17441: - +1 > TestSparkCliDriver.testCliDriver[explaindenpendencydiffengs] > > > Key: HIVE-17441 > URL: https://issues.apache.org/jira/browse/HIVE-17441 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17441.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-17442) TestCliDriver.testCliDriver[explaindenpendencydiffengs]
[ https://issues.apache.org/jira/browse/HIVE-17442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992661#comment-16992661 ] Zoltan Haindrich commented on HIVE-17442: - +1 > TestCliDriver.testCliDriver[explaindenpendencydiffengs] > --- > > Key: HIVE-17442 > URL: https://issues.apache.org/jira/browse/HIVE-17442 > Project: Hive > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: László Bodor >Priority: Major > Attachments: HIVE-17442.01.branch-2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22355) Beeline should not prompt for hive user and password when authentication is NONE
[ https://issues.apache.org/jira/browse/HIVE-22355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992633#comment-16992633 ] Mate Juhasz commented on HIVE-22355: Uploaded a new patch putting the above thoughts into code > Beeline should not prompt for hive user and password when authentication is > NONE > > > Key: HIVE-22355 > URL: https://issues.apache.org/jira/browse/HIVE-22355 > Project: Hive > Issue Type: Bug > Components: Beeline >Reporter: Mate Juhasz >Assignee: Mate Juhasz >Priority: Major > Attachments: HIVE-22355.1.patch, HIVE-22355.2.patch, > HIVE-22355.3.patch, HIVE-22355.4.patch > > > Beeline - without adding the jdbc url - prompts for username and password in > case hive.server2.authentication=NONE, which is possibly pointless and can be > misleading for users as any input is accepted. > In addition, Sqoop has dropped hive cli recently in favor of beeline and if > there is no authentication set in Hive, Sqoop fails to connect as the process > stops waiting for the user/password input. > I think it would be nice to check the auth type "NONE" before reading unused > inputs from the console before this point: > https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/Commands.java#L1641 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22355) Beeline should not prompt for hive user and password when authentication is NONE
[ https://issues.apache.org/jira/browse/HIVE-22355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated HIVE-22355: --- Attachment: HIVE-22355.4.patch > Beeline should not prompt for hive user and password when authentication is > NONE > > > Key: HIVE-22355 > URL: https://issues.apache.org/jira/browse/HIVE-22355 > Project: Hive > Issue Type: Bug > Components: Beeline >Reporter: Mate Juhasz >Assignee: Mate Juhasz >Priority: Major > Attachments: HIVE-22355.1.patch, HIVE-22355.2.patch, > HIVE-22355.3.patch, HIVE-22355.4.patch > > > Beeline - without adding the jdbc url - prompts for username and password in > case hive.server2.authentication=NONE, which is possibly pointless and can be > misleading for users as any input is accepted. > In addition, Sqoop has dropped hive cli recently in favor of beeline and if > there is no authentication set in Hive, Sqoop fails to connect as the process > stops waiting for the user/password input. > I think it would be nice to check the auth type "NONE" before reading unused > inputs from the console before this point: > https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/Commands.java#L1641 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22615) Replace Base64 in hive-common Package
[ https://issues.apache.org/jira/browse/HIVE-22615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22615: -- Status: Patch Available (was: Open) > Replace Base64 in hive-common Package > - > > Key: HIVE-22615 > URL: https://issues.apache.org/jira/browse/HIVE-22615 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22615.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22615) Replace Base64 in hive-common Package
[ https://issues.apache.org/jira/browse/HIVE-22615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22615: -- Attachment: HIVE-22615.1.patch > Replace Base64 in hive-common Package > - > > Key: HIVE-22615 > URL: https://issues.apache.org/jira/browse/HIVE-22615 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22615.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22615) Replace Base64 in hive-common Package
[ https://issues.apache.org/jira/browse/HIVE-22615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-22615: - > Replace Base64 in hive-common Package > - > > Key: HIVE-22615 > URL: https://issues.apache.org/jira/browse/HIVE-22615 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22614) Replace Base64 in hive-jdbc Package
[ https://issues.apache.org/jira/browse/HIVE-22614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22614: -- Attachment: HIVE-22614.1.patch > Replace Base64 in hive-jdbc Package > --- > > Key: HIVE-22614 > URL: https://issues.apache.org/jira/browse/HIVE-22614 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22614.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22614) Replace Base64 in hive-jdbc Package
[ https://issues.apache.org/jira/browse/HIVE-22614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22614: -- Status: Patch Available (was: Open) > Replace Base64 in hive-jdbc Package > --- > > Key: HIVE-22614 > URL: https://issues.apache.org/jira/browse/HIVE-22614 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22614.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22614) Replace Base64 in hive-jdbc Package
[ https://issues.apache.org/jira/browse/HIVE-22614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-22614: - > Replace Base64 in hive-jdbc Package > --- > > Key: HIVE-22614 > URL: https://issues.apache.org/jira/browse/HIVE-22614 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22611) Use JDK Base64 Classes
[ https://issues.apache.org/jira/browse/HIVE-22611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22611: -- Description: Replace dependency on thirdparty libraries with native support for Base-64 encode/decode. https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html Might be able to remove some dependencies afterwards. was: Replace dependency on thirdparty libraries with native support for Base-64 encode/decode. https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html > Use JDK Base64 Classes > -- > > Key: HIVE-22611 > URL: https://issues.apache.org/jira/browse/HIVE-22611 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > > Replace dependency on thirdparty libraries with native support for Base-64 > encode/decode. > https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html > Might be able to remove some dependencies afterwards. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-22610) Minor compaction for MM (insert-only) tables
[ https://issues.apache.org/jira/browse/HIVE-22610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-22610 started by Karen Coppage. > Minor compaction for MM (insert-only) tables > > > Key: HIVE-22610 > URL: https://issues.apache.org/jira/browse/HIVE-22610 > Project: Hive > Issue Type: Sub-task >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22594) Reorder metastore thrift structures
[
https://issues.apache.org/jira/browse/HIVE-22594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992619#comment-16992619
]
Hive QA commented on HIVE-22594:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19852/dev-support/hive-personality.sh
|
| git revision | master / 879d45c |
| Default Java | 1.8.0_111 |
| modules | C: standalone-metastore/metastore-common U:
standalone-metastore/metastore-common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19852/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reorder metastore thrift structures
> ---
>
> Key: HIVE-22594
> URL: https://issues.apache.org/jira/browse/HIVE-22594
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Thrift API
>Affects Versions: 4.0.0
>Reporter: John Sherman
>Assignee: John Sherman
>Priority: Major
> Attachments: HIVE-22594.1.patch, HIVE-22594.2.patch
>
>
> Currently hive_metastore.thrift C++ output does not compile due to structures
> like ScheduledQueryKey and ColumnStatistics being referenced before being
> defined later in the file. We should reorder the structures in the file so
> the C++ output is usable.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22613) Replace Base64 in hive-hbase-handler Package
[ https://issues.apache.org/jira/browse/HIVE-22613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22613: -- Attachment: HIVE-22613.1.patch > Replace Base64 in hive-hbase-handler Package > > > Key: HIVE-22613 > URL: https://issues.apache.org/jira/browse/HIVE-22613 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22613.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22613) Replace Base64 in hive-hbase-handler Package
[ https://issues.apache.org/jira/browse/HIVE-22613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22613: -- Status: Patch Available (was: Open) > Replace Base64 in hive-hbase-handler Package > > > Key: HIVE-22613 > URL: https://issues.apache.org/jira/browse/HIVE-22613 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22613.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22613) Replace Base64 in hive-hbase-handler Package
[ https://issues.apache.org/jira/browse/HIVE-22613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22613: -- Summary: Replace Base64 in hive-hbase-handler Package (was: Replace Base64 in hive-hbase-handler) > Replace Base64 in hive-hbase-handler Package > > > Key: HIVE-22613 > URL: https://issues.apache.org/jira/browse/HIVE-22613 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22613) Replace Base64 in hive-hbase-handler
[ https://issues.apache.org/jira/browse/HIVE-22613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-22613: - > Replace Base64 in hive-hbase-handler > > > Key: HIVE-22613 > URL: https://issues.apache.org/jira/browse/HIVE-22613 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
