[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Mishra updated HIVE-22201:
---
Affects Version/s: 3.0.0
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22201.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Mishra updated HIVE-22201:
---
Fix Version/s: 4.0.0
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22201.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22199) Ugrade findbugs to 3.0.5
[
https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929687#comment-16929687
]
Hive QA commented on HIVE-22199:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980297/HIVE-22199.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16753 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18595/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18595/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18595/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980297 - PreCommit-HIVE-Build
> Ugrade findbugs to 3.0.5
>
>
> Key: HIVE-22199
> URL: https://issues.apache.org/jira/browse/HIVE-22199
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22199.01.patch, HIVE-22199.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22199) Ugrade findbugs to 3.0.5
[
https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929682#comment-16929682
]
Hive QA commented on HIVE-22199:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense findbugs xml javac javadoc compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18595/dev-support/hive-personality.sh
|
| git revision | master / 396c161 |
| Default Java | 1.8.0_111 |
| modules | C: . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18595/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Ugrade findbugs to 3.0.5
>
>
> Key: HIVE-22199
> URL: https://issues.apache.org/jira/browse/HIVE-22199
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22199.01.patch, HIVE-22199.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929669#comment-16929669
]
Hive QA commented on HIVE-22079:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980289/HIVE-22079.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 183 failed/errored test(s), 16753 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_6]
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join12] (batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join13] (batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join22] (batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join28] (batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join29] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join2] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join31] (batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join3] (batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats2]
(batchId=96)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark4]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_cross_product_check_2]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer10]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer11]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer13]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer14]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer15]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer5]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer7]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer8]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer9]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cross_product_check_2]
(batchId=98)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynamic_partition_insert]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[empty_join] (batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[extrapolate_part_stats_partial]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_dyn_part]
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join26] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join28] (batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join32] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join33] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_star] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin47] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_mapjoin]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_subquery2]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_subquery]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition2]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition3]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition4]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition5]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_buckets] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_join_union]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_int_type_promotion]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge10] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge2] (batchId=99)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat2]
(batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin4]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin8]
(batchId=42)
[jira] [Commented] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929658#comment-16929658
]
Hive QA commented on HIVE-22079:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
22s{color} | {color:blue} ql in master has 2225 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
12s{color} | {color:red} ql generated 1 new + 2225 unchanged - 0 fixed = 2226
total (was 2225) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
59s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Should org.apache.hadoop.hive.ql.lib.DefaultGraphWalker$NodeLabeled be a
_static_ inner class? At DefaultGraphWalker.java:inner class? At
DefaultGraphWalker.java:[lines 96-110] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18594/dev-support/hive-personality.sh
|
| git revision | master / 396c161 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18594/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18594/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18594/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch
>
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22205) Upgrade zookeeper and curator versions
[ https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22205: - Attachment: HIVE-22205.patch > Upgrade zookeeper and curator versions > -- > > Key: HIVE-22205 > URL: https://issues.apache.org/jira/browse/HIVE-22205 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-22205.patch > > > Other components like hadoop have switched to using new ZK versions. So these > jars end up in classpath for hive services and could cause issues due to > in-compatible curator versions that hive uses. > So it makes sense for hive to upgrade the ZK and curator versions to try to > keep up. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22205) Upgrade zookeeper and curator versions
[ https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22205: - Status: Patch Available (was: In Progress) > Upgrade zookeeper and curator versions > -- > > Key: HIVE-22205 > URL: https://issues.apache.org/jira/browse/HIVE-22205 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-22205.patch > > > Other components like hadoop have switched to using new ZK versions. So these > jars end up in classpath for hive services and could cause issues due to > in-compatible curator versions that hive uses. > So it makes sense for hive to upgrade the ZK and curator versions to try to > keep up. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work started] (HIVE-22205) Upgrade zookeeper and curator versions
[ https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-22205 started by Naveen Gangam. > Upgrade zookeeper and curator versions > -- > > Key: HIVE-22205 > URL: https://issues.apache.org/jira/browse/HIVE-22205 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > > Other components like hadoop have switched to using new ZK versions. So these > jars end up in classpath for hive services and could cause issues due to > in-compatible curator versions that hive uses. > So it makes sense for hive to upgrade the ZK and curator versions to try to > keep up. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22205) Upgrade zookeeper and curator versions
[ https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam reassigned HIVE-22205: > Upgrade zookeeper and curator versions > -- > > Key: HIVE-22205 > URL: https://issues.apache.org/jira/browse/HIVE-22205 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > > Other components like hadoop have switched to using new ZK versions. So these > jars end up in classpath for hive services and could cause issues due to > in-compatible curator versions that hive uses. > So it makes sense for hive to upgrade the ZK and curator versions to try to > keep up. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[
https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929654#comment-16929654
]
Hive QA commented on HIVE-20983:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980217/HIVE-20983.3.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16753 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18593/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18593/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18593/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980217 - PreCommit-HIVE-Build
> Vectorization: Scale up small hashtables, when collisions are detected
> --
>
> Key: HIVE-20983
> URL: https://issues.apache.org/jira/browse/HIVE-20983
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch,
> HIVE-20983.3.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive's hashtable estimates are getting better with HyperLogLog stats in
> place, but an accurate estimate does not always result in a low number of
> collisions.
> The hashtables which contain a very small number of items tend to lose their
> O(1) lookup performance where there are collisions. Since collisions are easy
> to detect within the fast hashtable implementation, a rehashing to a higher
> size will help these small hashtables avoid collisions and go back to O(1)
> perf.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=312479=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312479
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 14/Sep/19 02:42
Start Date: 14/Sep/19 02:42
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r324409418
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUnixTime.java
##
@@ -0,0 +1,173 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import java.text.SimpleDateFormat;
+import java.time.ZoneId;
+import java.util.Date;
+import java.util.TimeZone;
+import org.apache.commons.lang.StringUtils;
+import org.apache.hadoop.hive.common.type.TimestampTZUtil;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.MapredContext;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentLengthException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.ql.session.SessionState;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector.Category;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters;
+import
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.Converter;
+import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.IntObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.LongObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.Text;
+
+/**
+ * GenericUDFFromUnixTime.
+ *
+ */
+@Description(name = "from_unixtime",
+value = "_FUNC_(unix_time, format) - returns unix_time in the specified
format",
+extended = "Example:\n"
++ " > SELECT _FUNC_(0, '-MM-dd HH:mm:ss') FROM src LIMIT 1;\n"
++ " '1970-01-01 00:00:00'")
+public class GenericUDFFromUnixTime extends GenericUDF {
+
+ private transient IntObjectInspector inputIntOI;
+ private transient LongObjectInspector inputLongOI;
+ private transient Converter inputTextConverter;
+ private transient ZoneId timeZone;
+ private transient final Text result = new Text();
+
+ private transient SimpleDateFormat formatter = new
SimpleDateFormat("-MM-dd HH:mm:ss");
+ private transient String lastFormat = null;
+
+
+ @Override
+ public ObjectInspector initialize(ObjectInspector[] arguments) throws
UDFArgumentException {
+if (arguments.length < 1) {
+ throw new UDFArgumentLengthException("The function " +
getName().toUpperCase() +
+ "requires at least one argument");
+}
+if (arguments.length > 2) {
+ throw new UDFArgumentLengthException("Too many arguments for the
function " + getName().toUpperCase());
+}
+for (ObjectInspector argument : arguments) {
+ if (argument.getCategory() != Category.PRIMITIVE) {
+throw new UDFArgumentException(getName().toUpperCase() +
+" only takes primitive types, got " + argument.getTypeName());
+ }
+}
+
+PrimitiveObjectInspector arg0OI = (PrimitiveObjectInspector) arguments[0];
+switch (arg0OI.getPrimitiveCategory()) {
+ case INT:
+inputIntOI = (IntObjectInspector) arguments[0];
+break;
+ case LONG:
+inputLongOI = (LongObjectInspector) arguments[0];
+break;
+ default:
+throw new UDFArgumentException("The function " +
getName().toUpperCase()
++ " takes only int/long types for first argument. Got Type:" +
arg0OI.getPrimitiveCategory().name());
+}
+
+if (arguments.length == 2) {
+ PrimitiveObjectInspector arg1OI = (PrimitiveObjectInspector)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=312477=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312477
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 14/Sep/19 02:37
Start Date: 14/Sep/19 02:37
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r324409305
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUnixTimeStamp.java
##
@@ -117,6 +121,19 @@ protected void initializeInput(ObjectInspector[]
arguments) throws UDFArgumentEx
+ " takes only string/date/timestamp/timestampwltz types. Got
Type:" + arg1OI
.getPrimitiveCategory().name());
}
+
+if (timeZone == null) {
+ timeZone = SessionState.get().getConf().getLocalTimeZone();
+ formatter.setTimeZone(TimeZone.getTimeZone(timeZone));
+}
+ }
+
+ public void configure(MapredContext context) {
Review comment:
No, this is the comment of the super method that it overrides (I added the
override notation since it was missing):
```
/**
* Additionally setup GenericUDF with MapredContext before initializing.
* This is only called in runtime of MapRedTask.
*
* @param context context
*/
```
Basically this is needed because we need to set up the timezone from the
configuration when we are about to execute the udf in each of the nodes, but we
only have access to the configuration object through this context. This is
called before calling the initialize method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 312477)
Time Spent: 40m (was: 0.5h)
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch, HIVE-22170.04.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=312475=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312475
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 14/Sep/19 02:22
Start Date: 14/Sep/19 02:22
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r324408973
##
File path: ql/src/java/org/apache/hadoop/hive/ql/udf/UDFFromUnixTime.java
##
@@ -31,12 +30,14 @@
/**
* UDFFromUnixTime.
*
+ * @deprecated Replaced by {@link
org.apache.hadoop.hive.ql.udf.generic.GenericUDFFromUnixTime}.
Review comment:
Yes, that's a good point. I will remove it from the patch for master and I
will keep it for the backport to 3.x branch in case users are referencing the
UDF directly in their queries.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 312475)
Time Spent: 0.5h (was: 20m)
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch, HIVE-22170.04.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[
https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929643#comment-16929643
]
Hive QA commented on HIVE-20983:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
50s{color} | {color:blue} ql in master has 2225 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18593/dev-support/hive-personality.sh
|
| git revision | master / 396c161 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18593/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Scale up small hashtables, when collisions are detected
> --
>
> Key: HIVE-20983
> URL: https://issues.apache.org/jira/browse/HIVE-20983
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch,
> HIVE-20983.3.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive's hashtable estimates are getting better with HyperLogLog stats in
> place, but an accurate estimate does not always result in a low number of
> collisions.
> The hashtables which contain a very small number of items tend to lose their
> O(1) lookup performance where there are collisions. Since collisions are easy
> to detect within the fast hashtable implementation, a rehashing to a higher
> size will help these small hashtables avoid collisions and go back to O(1)
> perf.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22200) Hash collision may cause column resolution to fail
[
https://issues.apache.org/jira/browse/HIVE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22200:
---
Attachment: HIVE-22200.patch
> Hash collision may cause column resolution to fail
> --
>
> Key: HIVE-22200
> URL: https://issues.apache.org/jira/browse/HIVE-22200
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-22200.patch, HIVE-22200.patch, HIVE-22200.patch
>
>
> {{ExprNodeDescUtils.getExprNodeColumnDesc}} extracts the
> {{ExprNodeColumnDesc}} (column descriptors) from an expression. In fact, it
> creates a map from hash to the object itself. If same hash value is generated
> for two different objects, this will result in a clash in the map and some
> expressions not being part of its values.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22200) Hash collision may cause column resolution to fail
[
https://issues.apache.org/jira/browse/HIVE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929633#comment-16929633
]
Hive QA commented on HIVE-22200:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980286/HIVE-22200.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16721 tests
executed
*Failed tests:*
{noformat}
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=233)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18592/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18592/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18592/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980286 - PreCommit-HIVE-Build
> Hash collision may cause column resolution to fail
> --
>
> Key: HIVE-22200
> URL: https://issues.apache.org/jira/browse/HIVE-22200
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-22200.patch, HIVE-22200.patch
>
>
> {{ExprNodeDescUtils.getExprNodeColumnDesc}} extracts the
> {{ExprNodeColumnDesc}} (column descriptors) from an expression. In fact, it
> creates a map from hash to the object itself. If same hash value is generated
> for two different objects, this will result in a clash in the map and some
> expressions not being part of its values.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22200) Hash collision may cause column resolution to fail
[
https://issues.apache.org/jira/browse/HIVE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929627#comment-16929627
]
Hive QA commented on HIVE-22200:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
16s{color} | {color:blue} ql in master has 2225 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} ql: The patch generated 0 new + 648 unchanged - 29
fixed = 648 total (was 677) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s{color} | {color:green} ql generated 0 new + 2224 unchanged - 1 fixed = 2224
total (was 2225) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18592/dev-support/hive-personality.sh
|
| git revision | master / 396c161 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18592/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hash collision may cause column resolution to fail
> --
>
> Key: HIVE-22200
> URL: https://issues.apache.org/jira/browse/HIVE-22200
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-22200.patch, HIVE-22200.patch
>
>
> {{ExprNodeDescUtils.getExprNodeColumnDesc}} extracts the
> {{ExprNodeColumnDesc}} (column descriptors) from an expression. In fact, it
> creates a map from hash to the object itself. If same hash value is generated
> for two different objects, this will result in a clash in the map and some
> expressions not being part of its values.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929622#comment-16929622
]
Hive QA commented on HIVE-22195:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980282/HIVE-22195.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16756 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18591/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18591/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18591/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980282 - PreCommit-HIVE-Build
> Configure authentication type for Zookeeper when different from the default
> cluster wide
>
>
> Key: HIVE-22195
> URL: https://issues.apache.org/jira/browse/HIVE-22195
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch,
> HIVE-22195.4.patch, HIVE-22195.5.patch
>
>
> This could be useful in case when cluster is kerberized, but Zookeeper is not.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-19261) Avro SerDe's InstanceCache should not be synchronized on retrieve
[
https://issues.apache.org/jira/browse/HIVE-19261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929610#comment-16929610
]
Mithun Radhakrishnan commented on HIVE-19261:
-
Hello, [~shengzhixia], et al.
This looks like a good fix, generally. A couple of suggestions:
# Please consider the rephrasing the ending of {{InstanceCache::retrieve()}}
as follows:
## {code:java}
Instance newInstance = makeInstance(hv, seenSchemas);
instance = cache.putIfAbsent(hv, newInstance);
return instance == null? newInstance : instance;{code}
# Please remove the documentation regarding
{{ConcurrentHashMap::computeIfAbsent()}}; it won't work here.
{{makeInstance()}} lands up making reentrant calls to
{{InstanceCache::retrieve()}}. {{computeIfAbsent()}} requires that the
{{cache}} not be modified within a single call to the {{putIfAbsent()}} lambda
(i.e. {{hv -> makeInstance(hv, seenSchemas)}}. As such, this will cause
{{retrieve()}} to hang on the first reentrant call.
> Avro SerDe's InstanceCache should not be synchronized on retrieve
> -
>
> Key: HIVE-19261
> URL: https://issues.apache.org/jira/browse/HIVE-19261
> Project: Hive
> Issue Type: Improvement
>Reporter: Fangshi Li
>Assignee: Fangshi Li
>Priority: Major
> Attachments: HIVE-19261.1.patch
>
>
> In HIVE-16175, upstream made a patch to fix the thread safety issue in
> AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache
> synchronized. While it should make InstanceCache thread-safe, making retrieve
> synchronized for the cache can be expensive in highly concurrent environment
> like Spark, as multiple threads need to be synchronized on entering the
> entire retrieve method.
> We are proposing another way to fix this thread safety issue by making the
> underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use
> atomic computeIfAbsent in the retrieve method to avoid synchronizing the
> entire method.
> While computeIfAbsent is only available on java 8 and java 7 is still
> supported in Hive,
> we use a pattern to simulate the behavior of computeIfAbsent. In the future,
> we should move to computeIfAbsent when Hive requires java 8.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (HIVE-19261) Avro SerDe's InstanceCache should not be synchronized on retrieve
[
https://issues.apache.org/jira/browse/HIVE-19261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929610#comment-16929610
]
Mithun Radhakrishnan edited comment on HIVE-19261 at 9/13/19 11:38 PM:
---
Hello, [~shengzhixia], et al.
This looks like a good fix, generally. A couple of suggestions:
# Please consider the rephrasing the ending of {{InstanceCache::retrieve()}}
as follows, for readability:
##
{code:java}
Instance newInstance = makeInstance(hv, seenSchemas);
instance = cache.putIfAbsent(hv, newInstance);
return instance == null? newInstance : instance;{code}
# Please remove the documentation regarding
{{ConcurrentHashMap::computeIfAbsent()}}; it won't work here.
{{makeInstance()}} lands up making reentrant calls to
{{InstanceCache::retrieve()}}. {{computeIfAbsent()}} requires that the
{{cache}} not be modified within a single call to the {{putIfAbsent()}} lambda
(i.e. {{hv -> makeInstance(hv, seenSchemas)}}. As such, this will cause
{{retrieve()}} to hang on the first reentrant call.
was (Author: mithun):
Hello, [~shengzhixia], et al.
This looks like a good fix, generally. A couple of suggestions:
# Please consider the rephrasing the ending of {{InstanceCache::retrieve()}}
as follows:
## {code:java}
Instance newInstance = makeInstance(hv, seenSchemas);
instance = cache.putIfAbsent(hv, newInstance);
return instance == null? newInstance : instance;{code}
# Please remove the documentation regarding
{{ConcurrentHashMap::computeIfAbsent()}}; it won't work here.
{{makeInstance()}} lands up making reentrant calls to
{{InstanceCache::retrieve()}}. {{computeIfAbsent()}} requires that the
{{cache}} not be modified within a single call to the {{putIfAbsent()}} lambda
(i.e. {{hv -> makeInstance(hv, seenSchemas)}}. As such, this will cause
{{retrieve()}} to hang on the first reentrant call.
> Avro SerDe's InstanceCache should not be synchronized on retrieve
> -
>
> Key: HIVE-19261
> URL: https://issues.apache.org/jira/browse/HIVE-19261
> Project: Hive
> Issue Type: Improvement
>Reporter: Fangshi Li
>Assignee: Fangshi Li
>Priority: Major
> Attachments: HIVE-19261.1.patch
>
>
> In HIVE-16175, upstream made a patch to fix the thread safety issue in
> AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache
> synchronized. While it should make InstanceCache thread-safe, making retrieve
> synchronized for the cache can be expensive in highly concurrent environment
> like Spark, as multiple threads need to be synchronized on entering the
> entire retrieve method.
> We are proposing another way to fix this thread safety issue by making the
> underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use
> atomic computeIfAbsent in the retrieve method to avoid synchronizing the
> entire method.
> While computeIfAbsent is only available on java 8 and java 7 is still
> supported in Hive,
> we use a pattern to simulate the behavior of computeIfAbsent. In the future,
> we should move to computeIfAbsent when Hive requires java 8.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22204) Beeline option to show/not show execution report
[
https://issues.apache.org/jira/browse/HIVE-22204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929602#comment-16929602
]
Ashutosh Chauhan commented on HIVE-22204:
-
+1 pending tests
> Beeline option to show/not show execution report
>
>
> Key: HIVE-22204
> URL: https://issues.apache.org/jira/browse/HIVE-22204
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-22204.patch
>
>
> Currently, {{\-\-silent=true}} will also remove the short report about
> execution (includes number of rows returned by a query and execution time).
> It would be interesting to control whether we want to show that report even
> if {{\-\-silent=true}}, e.g., using an option {{\-\-report=true}}. Default
> (existing) behavior should not change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929601#comment-16929601
]
Hive QA commented on HIVE-22195:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
55s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 61 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} llap-client in master has 26 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
14s{color} | {color:blue} standalone-metastore/metastore-server in master has
178 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
21s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
24s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 24s{color}
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} llap-client: The patch generated 1 new + 49 unchanged
- 1 fixed = 50 total (was 50) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
24s{color} | {color:red} service in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18591/dev-support/hive-personality.sh
|
| git revision | master / 396c161 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18591/yetus/patch-mvninstall-service.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18591/yetus/patch-compile-service.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18591/yetus/patch-compile-service.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18591/yetus/diff-checkstyle-llap-client.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18591/yetus/patch-findbugs-service.txt
|
| modules | C: common llap-client standalone-metastore/metastore-server service
U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18591/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Configure authentication type for Zookeeper when different from the default
> cluster wide
>
>
> Key: HIVE-22195
>
[jira] [Updated] (HIVE-22204) Beeline option to show/not show execution report
[
https://issues.apache.org/jira/browse/HIVE-22204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22204:
---
Status: Patch Available (was: In Progress)
> Beeline option to show/not show execution report
>
>
> Key: HIVE-22204
> URL: https://issues.apache.org/jira/browse/HIVE-22204
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-22204.patch
>
>
> Currently, {{\-\-silent=true}} will also remove the short report about
> execution (includes number of rows returned by a query and execution time).
> It would be interesting to control whether we want to show that report even
> if {{\-\-silent=true}}, e.g., using an option {{\-\-report=true}}. Default
> (existing) behavior should not change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work started] (HIVE-22204) Beeline option to show/not show execution report
[
https://issues.apache.org/jira/browse/HIVE-22204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22204 started by Jesus Camacho Rodriguez.
--
> Beeline option to show/not show execution report
>
>
> Key: HIVE-22204
> URL: https://issues.apache.org/jira/browse/HIVE-22204
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-22204.patch
>
>
> Currently, {{\-\-silent=true}} will also remove the short report about
> execution (includes number of rows returned by a query and execution time).
> It would be interesting to control whether we want to show that report even
> if {{\-\-silent=true}}, e.g., using an option {{\-\-report=true}}. Default
> (existing) behavior should not change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22204) Beeline option to show/not show execution report
[
https://issues.apache.org/jira/browse/HIVE-22204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22204:
---
Attachment: HIVE-22204.patch
> Beeline option to show/not show execution report
>
>
> Key: HIVE-22204
> URL: https://issues.apache.org/jira/browse/HIVE-22204
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-22204.patch
>
>
> Currently, {{\-\-silent=true}} will also remove the short report about
> execution (includes number of rows returned by a query and execution time).
> It would be interesting to control whether we want to show that report even
> if {{\-\-silent=true}}, e.g., using an option {{\-\-report=true}}. Default
> (existing) behavior should not change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929574#comment-16929574
]
Hive QA commented on HIVE-22201:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980281/HIVE-22201.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16738 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=159)
[parquet_complex_types_vectorization.q,cte_4.q,explainuser_2.q,mm_all.q,rcfile_merge2.q,alter_table_location2.q,bucket5.q,mm_dp.q,orc_struct_type_vectorization.q,parquet_map_type_vectorization.q,reduce_deduplicate.q,load_fs2.q,rcfile_merge3.q,external_table_purge.q,intersect_distinct.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18590/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18590/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18590/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980281 - PreCommit-HIVE-Build
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22201.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22204) Beeline option to show/not show execution report
[
https://issues.apache.org/jira/browse/HIVE-22204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22204:
---
Description: Currently, {{\-\-silent=true}} will also remove the short
report about execution (includes number of rows returned by a query and
execution time). It would be interesting to control whether we want to show
that report even if {{\-\-silent=true}}, e.g., using an option
{{\-\-report=true}}. Default (existing) behavior should not change. (was:
Currently, {{--silent=true}} will also remove the short report about execution
(includes number of rows returned by a query and execution time). It would be
interesting to control whether we want to show that report even if
{{--silent=true}}, e.g., using an option {{--report=true}}. Default (existing)
behavior should not change.)
> Beeline option to show/not show execution report
>
>
> Key: HIVE-22204
> URL: https://issues.apache.org/jira/browse/HIVE-22204
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
>
> Currently, {{\-\-silent=true}} will also remove the short report about
> execution (includes number of rows returned by a query and execution time).
> It would be interesting to control whether we want to show that report even
> if {{\-\-silent=true}}, e.g., using an option {{\-\-report=true}}. Default
> (existing) behavior should not change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22203) Break up DDLSemanticAnalyzer - extract Process related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22203?focusedWorklogId=312401=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312401 ] ASF GitHub Bot logged work on HIVE-22203: - Author: ASF GitHub Bot Created on: 13/Sep/19 22:15 Start Date: 13/Sep/19 22:15 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #774: HIVE-22203 Break up DDLSemanticAnalyzer - extract Process related analyzers URL: https://github.com/apache/hive/pull/774 DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: - have a separate class for each analyzers - have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #4: extract all the process related analyzers from DDLSemanticAnalyzer, and move them under the new package. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 312401) Remaining Estimate: 0h Time Spent: 10m > Break up DDLSemanticAnalyzer - extract Process related analyzers > > > Key: HIVE-22203 > URL: https://issues.apache.org/jira/browse/HIVE-22203 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22203.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #4: extract all the process related analyzers from DDLSemanticAnalyzer, > and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22204) Beeline option to show/not show execution report
[
https://issues.apache.org/jira/browse/HIVE-22204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-22204:
--
> Beeline option to show/not show execution report
>
>
> Key: HIVE-22204
> URL: https://issues.apache.org/jira/browse/HIVE-22204
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
>
> Currently, {{--silent=true}} will also remove the short report about
> execution (includes number of rows returned by a query and execution time).
> It would be interesting to control whether we want to show that report even
> if {{--silent=true}}, e.g., using an option {{--report=true}}. Default
> (existing) behavior should not change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22203) Break up DDLSemanticAnalyzer - extract Process related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22203: -- Attachment: HIVE-22203.01.patch > Break up DDLSemanticAnalyzer - extract Process related analyzers > > > Key: HIVE-22203 > URL: https://issues.apache.org/jira/browse/HIVE-22203 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22203.01.patch > > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #4: extract all the process related analyzers from DDLSemanticAnalyzer, > and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22203) Break up DDLSemanticAnalyzer - extract Process related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22203: -- Status: Patch Available (was: Open) > Break up DDLSemanticAnalyzer - extract Process related analyzers > > > Key: HIVE-22203 > URL: https://issues.apache.org/jira/browse/HIVE-22203 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22203.01.patch > > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #4: extract all the process related analyzers from DDLSemanticAnalyzer, > and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22194: -- Description: DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each analyzers * have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #3: extract all the privilege related analyzers from DDLSemanticAnalyzer, and move them under the new package. was: DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each analyzers * have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #2: extract all the privilege related analyzers from DDLSemanticAnalyzer, and move them under the new package. > Break up DDLSemanticAnalyzer - extract Privilege related analyzers > -- > > Key: HIVE-22194 > URL: https://issues.apache.org/jira/browse/HIVE-22194 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch, > HIVE-22194.03.patch > > Time Spent: 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #3: extract all the privilege related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22203) Break up DDLSemanticAnalyzer - extract Process related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22203: -- Description: DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each analyzers * have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #2: extract all the process related analyzers from DDLSemanticAnalyzer, and move them under the new package. was: DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each analyzers * have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #2: extract all the privilege related analyzers from DDLSemanticAnalyzer, and move them under the new package. > Break up DDLSemanticAnalyzer - extract Process related analyzers > > > Key: HIVE-22203 > URL: https://issues.apache.org/jira/browse/HIVE-22203 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the process related analyzers from DDLSemanticAnalyzer, > and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22203) Break up DDLSemanticAnalyzer - extract Process related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22203: -- Description: DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each analyzers * have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #4: extract all the process related analyzers from DDLSemanticAnalyzer, and move them under the new package. was: DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: * have a separate class for each analyzers * have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #2: extract all the process related analyzers from DDLSemanticAnalyzer, and move them under the new package. > Break up DDLSemanticAnalyzer - extract Process related analyzers > > > Key: HIVE-22203 > URL: https://issues.apache.org/jira/browse/HIVE-22203 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #4: extract all the process related analyzers from DDLSemanticAnalyzer, > and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22203) Break up DDLSemanticAnalyzer - extract Process related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-22203: - > Break up DDLSemanticAnalyzer - extract Process related analyzers > > > Key: HIVE-22203 > URL: https://issues.apache.org/jira/browse/HIVE-22203 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the privilege related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22194: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Break up DDLSemanticAnalyzer - extract Privilege related analyzers > -- > > Key: HIVE-22194 > URL: https://issues.apache.org/jira/browse/HIVE-22194 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch, > HIVE-22194.03.patch > > Time Spent: 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the privilege related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929555#comment-16929555
]
Hive QA commented on HIVE-22201:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 2225 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18590/dev-support/hive-personality.sh
|
| git revision | master / 38190f3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18590/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22201.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929554#comment-16929554 ] Jesus Camacho Rodriguez commented on HIVE-22194: +1 > Break up DDLSemanticAnalyzer - extract Privilege related analyzers > -- > > Key: HIVE-22194 > URL: https://issues.apache.org/jira/browse/HIVE-22194 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch, > HIVE-22194.03.patch > > Time Spent: 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the privilege related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929540#comment-16929540
]
Hive QA commented on HIVE-22194:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980278/HIVE-22194.03.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16753 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18589/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18589/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18589/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980278 - PreCommit-HIVE-Build
> Break up DDLSemanticAnalyzer - extract Privilege related analyzers
> --
>
> Key: HIVE-22194
> URL: https://issues.apache.org/jira/browse/HIVE-22194
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch,
> HIVE-22194.03.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #2: extract all the privilege related analyzers from
> DDLSemanticAnalyzer, and move them under the new package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929516#comment-16929516
]
Hive QA commented on HIVE-22194:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
35s{color} | {color:blue} ql in master has 2225 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} ql: The patch generated 0 new + 247 unchanged - 4
fixed = 247 total (was 251) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18589/dev-support/hive-personality.sh
|
| git revision | master / 38190f3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18589/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up DDLSemanticAnalyzer - extract Privilege related analyzers
> --
>
> Key: HIVE-22194
> URL: https://issues.apache.org/jira/browse/HIVE-22194
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch,
> HIVE-22194.03.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #2: extract all the privilege related analyzers from
> DDLSemanticAnalyzer, and move them under the new package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22199) Ugrade findbugs to 3.0.5
[ https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22199: -- Attachment: HIVE-22199.02.patch > Ugrade findbugs to 3.0.5 > > > Key: HIVE-22199 > URL: https://issues.apache.org/jira/browse/HIVE-22199 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22199.01.patch, HIVE-22199.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22199) Ugrade findbugs to 3.0.5
[ https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22199: -- Attachment: (was: HIVE-22199.02.patch) > Ugrade findbugs to 3.0.5 > > > Key: HIVE-22199 > URL: https://issues.apache.org/jira/browse/HIVE-22199 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22199.01.patch, HIVE-22199.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22199) Ugrade findbugs to 3.0.5
[
https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929499#comment-16929499
]
Hive QA commented on HIVE-22199:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980276/HIVE-22199.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 16721 tests
executed
*Failed tests:*
{noformat}
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=233)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=171)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18588/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18588/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18588/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980276 - PreCommit-HIVE-Build
> Ugrade findbugs to 3.0.5
>
>
> Key: HIVE-22199
> URL: https://issues.apache.org/jira/browse/HIVE-22199
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22199.01.patch, HIVE-22199.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22189) HMS Translation: Enforce strict locations for managed vs external tables.
[ https://issues.apache.org/jira/browse/HIVE-22189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929469#comment-16929469 ] Thejas M Nair commented on HIVE-22189: -- +1 > HMS Translation: Enforce strict locations for managed vs external tables. > - > > Key: HIVE-22189 > URL: https://issues.apache.org/jira/browse/HIVE-22189 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-22189.patch > > > Currently, HMS allows flexibility with location of a table. External tables > can be located within Hive managed warehouse space and managed tables can be > located within the external warehouse directory if the user chooses to do so. > There are certain advantages to restrict such flexibility. We could have > different encryption policies for different warehouses, different replication > policies etc. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22199) Ugrade findbugs to 3.0.5
[
https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929470#comment-16929470
]
Hive QA commented on HIVE-22199:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
41s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense findbugs xml javac javadoc compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18588/dev-support/hive-personality.sh
|
| git revision | master / 38190f3 |
| Default Java | 1.8.0_111 |
| modules | C: . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18588/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Ugrade findbugs to 3.0.5
>
>
> Key: HIVE-22199
> URL: https://issues.apache.org/jira/browse/HIVE-22199
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22199.01.patch, HIVE-22199.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22158) HMS Translation layer - Disallow non-ACID MANAGED tables.
[ https://issues.apache.org/jira/browse/HIVE-22158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929468#comment-16929468 ] Thejas M Nair commented on HIVE-22158: -- +1 > HMS Translation layer - Disallow non-ACID MANAGED tables. > - > > Key: HIVE-22158 > URL: https://issues.apache.org/jira/browse/HIVE-22158 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-22158.1.patch, HIVE-22158.1.patch, > HIVE-22158.2.patch > > > In the recent commits, we have allowed non-ACID MANAGED tables to be created > by clients that have some form of ACID WRITE capabilities. > I think it would make sense to disallow this entirely. MANAGED tables should > be ACID tables only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929449#comment-16929449
]
Hive QA commented on HIVE-22191:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980274/HIVE-22191.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18587/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18587/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18587/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-13 18:43:18.896
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18587/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-13 18:43:18.899
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 38190f3 HIVE-20683: Add the Ability to push Dynamic Between and
Bloom filters to Druid (Nishant Bangarwa, reviewed by Slim Bouguerra)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 38190f3 HIVE-20683: Add the Ability to push Dynamic Between and
Bloom filters to Druid (Nishant Bangarwa, reviewed by Slim Bouguerra)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-13 18:43:19.538
+ rm -rf ../yetus_PreCommit-HIVE-Build-18587
+ mkdir ../yetus_PreCommit-HIVE-Build-18587
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18587
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18587/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:10576
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java' with
conflicts.
Going to apply patch with: git apply -p0
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:10576
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java' with
conflicts.
U ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-18587
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980274 - PreCommit-HIVE-Build
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929448#comment-16929448
]
Hive QA commented on HIVE-22163:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980269/HIVE-22163.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16754 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join4]
(batchId=196)
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testSetCapacity
(batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18586/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18586/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18586/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980269 - PreCommit-HIVE-Build
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929406#comment-16929406
]
Hive QA commented on HIVE-22163:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 2225 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18586/dev-support/hive-personality.sh
|
| git revision | master / 38190f3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18586/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929380#comment-16929380
]
Hive QA commented on HIVE-21449:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980265/HIVE-21449.4.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16757 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18585/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18585/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18585/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980265 - PreCommit-HIVE-Build
> implement 'WITHIN GROUP' clause
> ---
>
> Key: HIVE-21449
> URL: https://issues.apache.org/jira/browse/HIVE-21449
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch,
> HIVE-21449.3.patch, HIVE-21449.4.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22079:
---
Status: Patch Available (was: Open)
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch
>
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22079:
---
Status: Open (was: Patch Available)
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch
>
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22079:
---
Attachment: HIVE-22079.5.patch
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch
>
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22194?focusedWorklogId=312164=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312164 ] ASF GitHub Bot logged work on HIVE-22194: - Author: ASF GitHub Bot Created on: 13/Sep/19 16:33 Start Date: 13/Sep/19 16:33 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #769: HIVE-22194 Break up DDLSemanticAnalyzer - extract Privilege related analyzers URL: https://github.com/apache/hive/pull/769#discussion_r324272620 ## File path: ql/src/java/org/apache/hadoop/hive/ql/parse/authorization/HiveAuthorizationTaskFactoryImpl.java ## @@ -28,21 +28,21 @@ import org.apache.hadoop.hive.metastore.api.PrincipalType; import org.apache.hadoop.hive.ql.ErrorMsg; import org.apache.hadoop.hive.ql.ddl.DDLWork; -import org.apache.hadoop.hive.ql.ddl.privilege.CreateRoleDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.DropRoleDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.GrantDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.GrantRoleDesc; import org.apache.hadoop.hive.ql.ddl.privilege.PrincipalDesc; import org.apache.hadoop.hive.ql.ddl.privilege.PrivilegeDesc; import org.apache.hadoop.hive.ql.ddl.privilege.PrivilegeObjectDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.RevokeDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.RevokeRoleDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.SetRoleDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.ShowCurrentRoleDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.ShowGrantDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.ShowPrincipalsDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.ShowRoleGrantDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.ShowRolesDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.grant.GrantDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.revoke.RevokeDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.create.CreateRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.drop.DropRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.grant.GrantRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.revoke.RevokeRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.set.SetRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.show.ShowRolesDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.showcurrent.ShowCurrentRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.show.grant.ShowGrantDesc; Review comment: Not sure what are you suggesting to be merged under the show packege. The HiveAuthorizationTaskFactoryImpl? Or role.show -> show.role? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 312164) Time Spent: 50m (was: 40m) > Break up DDLSemanticAnalyzer - extract Privilege related analyzers > -- > > Key: HIVE-22194 > URL: https://issues.apache.org/jira/browse/HIVE-22194 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch, > HIVE-22194.03.patch > > Time Spent: 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the privilege related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929344#comment-16929344
]
Hive QA commented on HIVE-21449:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
25s{color} | {color:blue} ql in master has 2225 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} ql: The patch generated 1 new + 11 unchanged - 0 fixed
= 12 total (was 11) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
34s{color} | {color:red} ql generated 27 new + 2198 unchanged - 27 fixed = 2225
total (was 2225) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 52s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to KW_IN246 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceSimilarExpressionAtom(CommonTree)
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceSimilarExpressionAtom(CommonTree)
At HiveParser_IdentifiersParser.java:[line 9715] |
| | Dead store to LPAREN236 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At HiveParser_IdentifiersParser.java:[line 9217] |
| | Dead store to RPAREN238 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At HiveParser_IdentifiersParser.java:[line 9224] |
| | Redundant nullcheck of sql11ReservedKeywordsUsedAsFunctionName80, which
is known to be non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.functionName()
Redundant null check at HiveParser_IdentifiersParser.java:is known to be
non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.functionName()
Redundant null check at HiveParser_IdentifiersParser.java:[line 3962] |
| | Redundant nullcheck of nonReserved316, which is known to be non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier()
Redundant null check at HiveParser_IdentifiersParser.java:is known to be
non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier()
Redundant null check at HiveParser_IdentifiersParser.java:[line 12464] |
| | Should org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA22
be a _static_ inner class? At HiveParser_IdentifiersParser.java:inner class?
At HiveParser_IdentifiersParser.java:[lines 32342-32355] |
| | Dead store to LA32_100 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA32.specialStateTransition(int,
IntStream) At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA32.specialStateTransition(int,
IntStream) At
[jira] [Updated] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[ https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-20983: Status: Patch Available (was: In Progress) > Vectorization: Scale up small hashtables, when collisions are detected > -- > > Key: HIVE-20983 > URL: https://issues.apache.org/jira/browse/HIVE-20983 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch, > HIVE-20983.3.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's hashtable estimates are getting better with HyperLogLog stats in > place, but an accurate estimate does not always result in a low number of > collisions. > The hashtables which contain a very small number of items tend to lose their > O(1) lookup performance where there are collisions. Since collisions are easy > to detect within the fast hashtable implementation, a rehashing to a higher > size will help these small hashtables avoid collisions and go back to O(1) > perf. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22200) Hash collision may cause column resolution to fail
[
https://issues.apache.org/jira/browse/HIVE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22200:
---
Attachment: HIVE-22200.patch
> Hash collision may cause column resolution to fail
> --
>
> Key: HIVE-22200
> URL: https://issues.apache.org/jira/browse/HIVE-22200
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-22200.patch, HIVE-22200.patch
>
>
> {{ExprNodeDescUtils.getExprNodeColumnDesc}} extracts the
> {{ExprNodeColumnDesc}} (column descriptors) from an expression. In fact, it
> creates a map from hash to the object itself. If same hash value is generated
> for two different objects, this will result in a clash in the map and some
> expressions not being part of its values.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22199) Ugrade findbugs to 3.0.5
[ https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929326#comment-16929326 ] Jesus Camacho Rodriguez commented on HIVE-22199: +1 (pending tests) > Ugrade findbugs to 3.0.5 > > > Key: HIVE-22199 > URL: https://issues.apache.org/jira/browse/HIVE-22199 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22199.01.patch, HIVE-22199.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929325#comment-16929325
]
Ana Jalba commented on HIVE-21508:
--
[~pvary]: Is there anything I'm doing wrong with attaching the patch? The
latest patch attached is for branch-3.1, but it looks like the tests are trying
to apply it to master.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.4.branch-3.1.patch,
> HIVE-21508.5.branch-3.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929315#comment-16929315
]
Hive QA commented on HIVE-21508:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980254/HIVE-21508.5.branch-3.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 126 failed/errored test(s), 14412 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=232)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=232)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=273)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=226)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=232)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestCloseableThreadLocal - did not produce a TEST-*.xml file (likely timed out)
(batchId=332)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestDatabaseName - did not produce a TEST-*.xml file (likely timed out)
(batchId=195)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=273)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=229)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=237)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestGetPartitionsUsingProjectionAndFilterSpecs - did not produce a TEST-*.xml
file (likely timed out) (batchId=228)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=232)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=228)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=236)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=231)
TestHiveMetaToolCommandLine - did not produce a TEST-*.xml file (likely timed
out) (batchId=232)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHmsServerAuthorization - did not produce a TEST-*.xml file (likely timed
out) (batchId=232)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestJSONMessageDeserializer - did not
[jira] [Commented] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929300#comment-16929300
]
slim bouguerra commented on HIVE-22055:
---
thanks for this great finding! LGTM.
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-22055.2.patch
>
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1) from load0_mm
> PREHOOK: type: QUERY
> PREHOOK: Input: default@load0_mm
> A masked pattern was here
> POSTHOOK: query: select count(1) from load0_mm
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@load0_mm
> A masked pattern was here
> 1500{code}
> Got:
> [ERROR] TestMiniLlapLocalCliDriver.testCliDriver:59 Client Execution
> succeeded but contained differences (error code = 1) after executing
> mm_loaddata.q
> 63c63
> < 1480
> —
> > 1500
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929259#comment-16929259
]
Hive QA commented on HIVE-21508:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 11s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-18584/patches/PreCommit-HIVE-Build-18584.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18584/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.4.branch-3.1.patch,
> HIVE-21508.5.branch-3.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[ https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22195: -- Attachment: HIVE-22195.5.patch > Configure authentication type for Zookeeper when different from the default > cluster wide > > > Key: HIVE-22195 > URL: https://issues.apache.org/jira/browse/HIVE-22195 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch, > HIVE-22195.4.patch, HIVE-22195.5.patch > > > This could be useful in case when cluster is kerberized, but Zookeeper is not. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929255#comment-16929255
]
Hive QA commented on HIVE-22195:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980251/HIVE-22195.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16745 tests
executed
*Failed tests:*
{noformat}
TestJdbcWithMiniLlapArrow - did not produce a TEST-*.xml file (likely timed
out) (batchId=284)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18583/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18583/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18583/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980251 - PreCommit-HIVE-Build
> Configure authentication type for Zookeeper when different from the default
> cluster wide
>
>
> Key: HIVE-22195
> URL: https://issues.apache.org/jira/browse/HIVE-22195
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch,
> HIVE-22195.4.patch
>
>
> This could be useful in case when cluster is kerberized, but Zookeeper is not.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Mishra updated HIVE-22201:
---
Attachment: HIVE-22201.01.patch
Status: Patch Available (was: Open)
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22201.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Mishra updated HIVE-22201:
---
Status: Open (was: Patch Available)
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Mishra updated HIVE-22201:
---
Status: Patch Available (was: Open)
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929207#comment-16929207
]
Hive QA commented on HIVE-22195:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 61 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
20s{color} | {color:blue} llap-client in master has 26 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
19s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
18s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 18s{color}
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} llap-client: The patch generated 3 new + 49 unchanged -
1 fixed = 52 total (was 50) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
20s{color} | {color:red} service in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18583/dev-support/hive-personality.sh
|
| git revision | master / 38190f3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18583/yetus/patch-mvninstall-service.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18583/yetus/patch-compile-service.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18583/yetus/patch-compile-service.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18583/yetus/diff-checkstyle-llap-client.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18583/yetus/patch-findbugs-service.txt
|
| modules | C: common llap-client service U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18583/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Configure authentication type for Zookeeper when different from the default
> cluster wide
>
>
> Key: HIVE-22195
> URL: https://issues.apache.org/jira/browse/HIVE-22195
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
>
[jira] [Work started] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-22202 started by Hankó Gergely. > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * remove unnecessary method parameters > * remove never used return values > * simplify collection handling > * use foreach where possible > * use member sort() instead of Collections.sort() > * use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hankó Gergely reassigned HIVE-22202: > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * remove unnecessary method parameters > * remove never used return values > * simplify collection handling > * use foreach where possible > * use member sort() instead of Collections.sort() > * use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22194: -- Attachment: (was: HIVE-22194.03.patch) > Break up DDLSemanticAnalyzer - extract Privilege related analyzers > -- > > Key: HIVE-22194 > URL: https://issues.apache.org/jira/browse/HIVE-22194 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch, > HIVE-22194.03.patch > > Time Spent: 40m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the privilege related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22194: -- Attachment: HIVE-22194.03.patch > Break up DDLSemanticAnalyzer - extract Privilege related analyzers > -- > > Key: HIVE-22194 > URL: https://issues.apache.org/jira/browse/HIVE-22194 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch, > HIVE-22194.03.patch > > Time Spent: 40m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the privilege related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22199) Ugrade findbugs to 3.0.5
[ https://issues.apache.org/jira/browse/HIVE-22199?focusedWorklogId=312033=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312033 ] ASF GitHub Bot logged work on HIVE-22199: - Author: ASF GitHub Bot Created on: 13/Sep/19 13:21 Start Date: 13/Sep/19 13:21 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #773: HIVE-22199 Ugrade findbugs to 3.0.5 URL: https://github.com/apache/hive/pull/773 Also upgraded the excludes, now all the generated sources in ql are excluded. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 312033) Remaining Estimate: 0h Time Spent: 10m > Ugrade findbugs to 3.0.5 > > > Key: HIVE-22199 > URL: https://issues.apache.org/jira/browse/HIVE-22199 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22199.01.patch, HIVE-22199.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22199) Ugrade findbugs to 3.0.5
[ https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22199: -- Labels: pull-request-available (was: ) > Ugrade findbugs to 3.0.5 > > > Key: HIVE-22199 > URL: https://issues.apache.org/jira/browse/HIVE-22199 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22199.01.patch, HIVE-22199.02.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929188#comment-16929188
]
Hive QA commented on HIVE-22194:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980245/HIVE-22194.03.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16753 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestActivePassiveHA.testClientConnectionsOnFailover
(batchId=284)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18582/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18582/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18582/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980245 - PreCommit-HIVE-Build
> Break up DDLSemanticAnalyzer - extract Privilege related analyzers
> --
>
> Key: HIVE-22194
> URL: https://issues.apache.org/jira/browse/HIVE-22194
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch,
> HIVE-22194.03.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #2: extract all the privilege related analyzers from
> DDLSemanticAnalyzer, and move them under the new package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22199) Ugrade findbugs to 3.0.5
[ https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22199: -- Attachment: HIVE-22199.02.patch > Ugrade findbugs to 3.0.5 > > > Key: HIVE-22199 > URL: https://issues.apache.org/jira/browse/HIVE-22199 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22199.01.patch, HIVE-22199.02.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22191:
--
Status: Patch Available (was: In Progress)
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22191:
--
Attachment: HIVE-22191.patch
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22191:
--
Description:
Simplify {{SemanticAnalyzer}} by:
- Remove dead code
- Simplify returning statements
- Use interfaces types for parameters/fields/variables where it is
straightforward to migrate
- Make visibility stricter where it is possible
- Check logging to use parametrised logging
- Removing unnecessary keywords (e.g.: {{static}})
- Some code part could be simplified by using Java 8 features
I think this is crucial step as this class already contains 15000+ lines of
code which is screaming for splitting into more reasonable classes.
was:
Simplify {{SemanticAnalyzer}} by:
- Removing not used lines
- Removing unnecessary keywords (e.g.: {{static}})
- Some code part could be simplified by using Java 8 features
I think this is crucial step as this class already contains 15000+ lines of
code which is screaming for splitting into more reasonable classes.
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929157#comment-16929157
]
Hive QA commented on HIVE-22194:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
37s{color} | {color:blue} ql in master has 2225 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} ql: The patch generated 0 new + 247 unchanged - 4
fixed = 247 total (was 251) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18582/dev-support/hive-personality.sh
|
| git revision | master / 38190f3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18582/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up DDLSemanticAnalyzer - extract Privilege related analyzers
> --
>
> Key: HIVE-22194
> URL: https://issues.apache.org/jira/browse/HIVE-22194
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch,
> HIVE-22194.03.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #2: extract all the privilege related analyzers from
> DDLSemanticAnalyzer, and move them under the new package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Attachment: HIVE-22163.5.patch
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Open (was: Patch Available)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Patch Available (was: Open)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Attachment: HIVE-21449.4.patch > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Status: Open (was: Patch Available) > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Status: Patch Available (was: Open) > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929145#comment-16929145
]
Hive QA commented on HIVE-21449:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980243/HIVE-21449.3.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16757 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18581/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18581/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18581/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980243 - PreCommit-HIVE-Build
> implement 'WITHIN GROUP' clause
> ---
>
> Key: HIVE-21449
> URL: https://issues.apache.org/jira/browse/HIVE-21449
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch,
> HIVE-21449.3.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929115#comment-16929115
]
Hive QA commented on HIVE-21449:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2225 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
34s{color} | {color:red} ql: The patch generated 1 new + 11 unchanged - 0 fixed
= 12 total (was 11) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
41s{color} | {color:red} ql generated 27 new + 2198 unchanged - 27 fixed = 2225
total (was 2225) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to KW_IN246 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceSimilarExpressionAtom(CommonTree)
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceSimilarExpressionAtom(CommonTree)
At HiveParser_IdentifiersParser.java:[line 9715] |
| | Dead store to LPAREN236 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At HiveParser_IdentifiersParser.java:[line 9217] |
| | Dead store to RPAREN238 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At HiveParser_IdentifiersParser.java:[line 9224] |
| | Redundant nullcheck of sql11ReservedKeywordsUsedAsFunctionName80, which
is known to be non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.functionName()
Redundant null check at HiveParser_IdentifiersParser.java:is known to be
non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.functionName()
Redundant null check at HiveParser_IdentifiersParser.java:[line 3962] |
| | Redundant nullcheck of nonReserved316, which is known to be non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier()
Redundant null check at HiveParser_IdentifiersParser.java:is known to be
non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier()
Redundant null check at HiveParser_IdentifiersParser.java:[line 12464] |
| | Should org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA22
be a _static_ inner class? At HiveParser_IdentifiersParser.java:inner class?
At HiveParser_IdentifiersParser.java:[lines 32342-32355] |
| | Dead store to LA32_100 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA32.specialStateTransition(int,
IntStream) At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA32.specialStateTransition(int,
IntStream) At
[jira] [Work stopped] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22201 stopped by Himanshu Mishra.
--
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929097#comment-16929097
]
Hive QA commented on HIVE-22163:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980239/HIVE-22163.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 16754 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=284)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=284)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomNonExistent
(batchId=284)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesRead
(batchId=284)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes
(batchId=284)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryElapsedTime
(batchId=284)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime
(batchId=284)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18580/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18580/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18580/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980239 - PreCommit-HIVE-Build
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work started] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22201 started by Himanshu Mishra.
--
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Mishra reassigned HIVE-22201:
--
Assignee: Himanshu Mishra
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?focusedWorklogId=311962=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311962
]
ASF GitHub Bot logged work on HIVE-22201:
-
Author: ASF GitHub Bot
Created on: 13/Sep/19 10:31
Start Date: 13/Sep/19 10:31
Worklog Time Spent: 10m
Work Description: himanshu-mishra commented on pull request #772:
HIVE-22201: No MapJoin when there are no big table candidates
URL: https://github.com/apache/hive/pull/772
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311962)
Remaining Estimate: 0h
Time Spent: 10m
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Reporter: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-22201:
--
Labels: pull-request-available (was: )
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Reporter: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929080#comment-16929080
]
Ana Jalba commented on HIVE-21508:
--
[~pvary]: Thank you for pushing the patch for branch-2.
I added another patch for branch-3.1 to start tests. Hopefully this time there
won't be this many tests failing.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.4.branch-3.1.patch,
> HIVE-21508.5.branch-3.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: HIVE-21508.5.branch-3.1.patch
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.4.branch-3.1.patch,
> HIVE-21508.5.branch-3.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[ https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22195: -- Attachment: HIVE-22195.4.patch > Configure authentication type for Zookeeper when different from the default > cluster wide > > > Key: HIVE-22195 > URL: https://issues.apache.org/jira/browse/HIVE-22195 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch, > HIVE-22195.4.patch > > > This could be useful in case when cluster is kerberized, but Zookeeper is not. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929074#comment-16929074
]
Hive QA commented on HIVE-22163:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2225 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18580/dev-support/hive-personality.sh
|
| git revision | master / 38190f3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18580/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
>
[jira] [Commented] (HIVE-22200) Hash collision may cause column resolution to fail
[
https://issues.apache.org/jira/browse/HIVE-22200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929059#comment-16929059
]
Hive QA commented on HIVE-22200:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980241/HIVE-22200.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16753 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18579/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18579/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18579/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980241 - PreCommit-HIVE-Build
> Hash collision may cause column resolution to fail
> --
>
> Key: HIVE-22200
> URL: https://issues.apache.org/jira/browse/HIVE-22200
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-22200.patch
>
>
> {{ExprNodeDescUtils.getExprNodeColumnDesc}} extracts the
> {{ExprNodeColumnDesc}} (column descriptors) from an expression. In fact, it
> creates a map from hash to the object itself. If same hash value is generated
> for two different objects, this will result in a clash in the map and some
> expressions not being part of its values.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
