[jira] [Commented] (HIVE-22239) Scale data size using column value ranges
[
https://issues.apache.org/jira/browse/HIVE-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937413#comment-16937413
]
Hive QA commented on HIVE-22239:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981248/HIVE-22239.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 206 failed/errored test(s), 17001 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status]
(batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status_disable_bitvector]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_filter]
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join_pkfk]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[confirm_initial_tbl_stats]
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer13]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer9]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[folder_predicate]
(batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[foldts] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fp_literal_arithmetic]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_orig_table]
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_arithmetic]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_merging]
(batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_12] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_13] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_1] (batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_1_newdb]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_5] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_8] (batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_disablecbo_1]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_mv] (batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullability_transitive_inference]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge11] (batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat1]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat2]
(batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_10]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_11]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_12]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_13]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_14]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_15]
(batchId=96)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_16]
(batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_17]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_2]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_3]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_5]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_7]
(batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_8]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_9]
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_div0]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_limit]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_offset_limit]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcr] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[remove_exprs_stats]
(batchId=97)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample7] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp_ints_casts]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[typechangetest]
(batchId=12)
[jira] [Commented] (HIVE-22221) Llap external client - Need to reduce LlapBaseInputFormat#getSplits() footprint
[ https://issues.apache.org/jira/browse/HIVE-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937410#comment-16937410 ] Jason Dere commented on HIVE-1: --- +1 > Llap external client - Need to reduce LlapBaseInputFormat#getSplits() > footprint > - > > Key: HIVE-1 > URL: https://issues.apache.org/jira/browse/HIVE-1 > Project: Hive > Issue Type: Bug > Components: llap, UDF >Reporter: Shubham Chaurasia >Assignee: Shubham Chaurasia >Priority: Major > Labels: pull-request-available > Attachments: HIVE-1.1.patch, HIVE-1.2.patch, > HIVE-1.3.patch, HIVE-1.4.patch, HIVE-1.5.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > While querying through llap external client, LlapBaseInputFormat#getSplits() > invokes get_splits() (GenericUDTFGetSplits) udtf under the hoods. > GenericUDTFGetSplits returns LlapInputSplit in which planBytes[] occupies > around 90% of the split size. > Depending on data size/partitions and plan, LlapInputSplit can grow upto 1mb > with planBytes[] being common to all the splits and occupying more than 850 > kb. Also, it sometimes causes OOM on HS2 depending on HS2 heap size. > This can be resolved by separating out common parts from actual splits and > reassembling them at client side. > We can also provide an option where client can say it does not want to > reassemble them and can take the control of reassembling in it's hands. > Splits can be broken like: > 1) schema split > 2) plan split > 3) actual split 1 > 4) actual split 2and so on. > This greatly reduces the memory(in my case from 5GB(~5000 splits) to around > 15MB) on server side and hence the data transfer. And this eliminates OOM on > HS2 side. > cc [~jdere] [~sankarh] [~thejas] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22239) Scale data size using column value ranges
[
https://issues.apache.org/jira/browse/HIVE-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937402#comment-16937402
]
Hive QA commented on HIVE-22239:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
53s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} standalone-metastore/metastore-common in master has
32 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
44s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 5m
41s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
21s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 25 new + 109 unchanged - 11 fixed = 134 total (was 120) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 1 new + 123 unchanged - 0
fixed = 124 total (was 123) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
12s{color} | {color:green} metastore-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
14s{color} | {color:red} standalone-metastore/metastore-server generated 2 new
+ 170 unchanged - 0 fixed = 172 total (was 170) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
4s{color} | {color:green} ql generated 0 new + 1566 unchanged - 1 fixed = 1566
total (was 1567) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 16s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | Integral division result cast to double or float in
org.apache.hadoop.hive.metastore.columnstats.aggr.TimestampColumnStatsAggregator.aggregate(List,
List, boolean) At TimestampColumnStatsAggregator.java:double or float in
org.apache.hadoop.hive.metastore.columnstats.aggr.TimestampColumnStatsAggregator.aggregate(List,
List, boolean) At TimestampColumnStatsAggregator.java:[line 102] |
| |
org.apache.hadoop.hive.metastore.columnstats.cache.TimestampColumnStatsDataInspector
doesn't override
org.apache.hadoop.hive.metastore.api.TimestampColumnStatsData.equals(Object)
At TimestampColumnStatsDataInspector.java:At
TimestampColumnStatsDataInspector.java:[line 1] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool |
[jira] [Commented] (HIVE-22205) Upgrade zookeeper and curator versions
[
https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937397#comment-16937397
]
Hive QA commented on HIVE-22205:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
22s{color} | {color:blue} standalone-metastore/metastore-common in master has
32 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} llap-client in master has 26 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
6s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
54s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} llap-server in master has 90 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue}
standalone-metastore/metastore-tools/metastore-benchmarks in master has 3
extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
20s{color} | {color:blue} upgrade-acid/pre-upgrade in master has 1 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
46s{color} | {color:blue} itests/util in master has 51 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 12m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} The patch storage-api passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} The patch standalone-metastore passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} The patch llap-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} The patch metastore-server passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} ql: The patch generated 0 new + 6 unchanged - 2
fixed = 6 total (was 8) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} The patch llap-server passed checkstyle {color} |
| {color:green}+1{color} |
[jira] [Updated] (HIVE-22241) Implement UDF to convert a date/timestamp from Gregorian-Julian hybrid calendar to proleptic Gregorian calendar
[ https://issues.apache.org/jira/browse/HIVE-22241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22241: --- Attachment: HIVE-22241.patch > Implement UDF to convert a date/timestamp from Gregorian-Julian hybrid > calendar to proleptic Gregorian calendar > --- > > Key: HIVE-22241 > URL: https://issues.apache.org/jira/browse/HIVE-22241 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22241.patch > > > UDF that converts a date/timestamp from *Gregorian-Julian hybrid* calendar, > i.e., calendar that supports both the Julian and Gregorian calendar systems > with the support of a single discontinuity, which corresponds by default to > the Gregorian date when the Gregorian calendar was instituted, to *proleptic > Gregorian calendar* (ISO 8601 standard), which is produced by extending the > Gregorian calendar backward to dates preceding its official introduction in > 1582. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-22241) Implement UDF to convert a date/timestamp from Gregorian-Julian hybrid calendar to proleptic Gregorian calendar
[ https://issues.apache.org/jira/browse/HIVE-22241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-22241 started by Jesus Camacho Rodriguez. -- > Implement UDF to convert a date/timestamp from Gregorian-Julian hybrid > calendar to proleptic Gregorian calendar > --- > > Key: HIVE-22241 > URL: https://issues.apache.org/jira/browse/HIVE-22241 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > UDF that converts a date/timestamp from *Gregorian-Julian hybrid* calendar, > i.e., calendar that supports both the Julian and Gregorian calendar systems > with the support of a single discontinuity, which corresponds by default to > the Gregorian date when the Gregorian calendar was instituted, to *proleptic > Gregorian calendar* (ISO 8601 standard), which is produced by extending the > Gregorian calendar backward to dates preceding its official introduction in > 1582. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22241) Implement UDF to convert a date/timestamp from Gregorian-Julian hybrid calendar to proleptic Gregorian calendar
[ https://issues.apache.org/jira/browse/HIVE-22241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22241: --- Status: Patch Available (was: In Progress) > Implement UDF to convert a date/timestamp from Gregorian-Julian hybrid > calendar to proleptic Gregorian calendar > --- > > Key: HIVE-22241 > URL: https://issues.apache.org/jira/browse/HIVE-22241 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > UDF that converts a date/timestamp from *Gregorian-Julian hybrid* calendar, > i.e., calendar that supports both the Julian and Gregorian calendar systems > with the support of a single discontinuity, which corresponds by default to > the Gregorian date when the Gregorian calendar was instituted, to *proleptic > Gregorian calendar* (ISO 8601 standard), which is produced by extending the > Gregorian calendar backward to dates preceding its official introduction in > 1582. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22241) Implement UDF to convert a date/timestamp from Gregorian-Julian hybrid calendar to proleptic Gregorian calendar
[ https://issues.apache.org/jira/browse/HIVE-22241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-22241: -- > Implement UDF to convert a date/timestamp from Gregorian-Julian hybrid > calendar to proleptic Gregorian calendar > --- > > Key: HIVE-22241 > URL: https://issues.apache.org/jira/browse/HIVE-22241 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > UDF that converts a date/timestamp from *Gregorian-Julian hybrid* calendar, > i.e., calendar that supports both the Julian and Gregorian calendar systems > with the support of a single discontinuity, which corresponds by default to > the Gregorian date when the Gregorian calendar was instituted, to *proleptic > Gregorian calendar* (ISO 8601 standard), which is produced by extending the > Gregorian calendar backward to dates preceding its official introduction in > 1582. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-22240) Function percentile_cont fails when array parameter passed
[
https://issues.apache.org/jira/browse/HIVE-22240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22240 started by Krisztian Kasa.
-
> Function percentile_cont fails when array parameter passed
> --
>
> Key: HIVE-22240
> URL: https://issues.apache.org/jira/browse/HIVE-22240
> Project: Hive
> Issue Type: Bug
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
>
> {code}
> SELECT
> percentile_cont(array(0.2, 0.5, 0.9)) WITHIN GROUP (ORDER BY value)
> FROM t_test;
> {code}
> hive.log:
> {code}
> 2019-09-24T21:00:43,203 ERROR [LocalJobRunner Map Task Executor #0]
> mr.ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:148)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapRunner.run(ExecMapRunner.java:37)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:465)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException: java.util.ArrayList cannot be cast to
> org.apache.hadoop.hive.serde2.io.HiveDecimalWritable
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:793)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> ... 11 more
> Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast
> to org.apache.hadoop.hive.serde2.io.HiveDecimalWritable
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileCont$PercentileContEvaluator.iterate(GenericUDAFPercentileCont.java:259)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:214)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:639)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:814)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:720)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:788)
> ... 17 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22240) Function percentile_cont fails when array parameter passed
[
https://issues.apache.org/jira/browse/HIVE-22240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa reassigned HIVE-22240:
-
> Function percentile_cont fails when array parameter passed
> --
>
> Key: HIVE-22240
> URL: https://issues.apache.org/jira/browse/HIVE-22240
> Project: Hive
> Issue Type: Bug
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
>
> {code}
> SELECT
> percentile_cont(array(0.2, 0.5, 0.9)) WITHIN GROUP (ORDER BY value)
> FROM t_test;
> {code}
> hive.log:
> {code}
> 2019-09-24T21:00:43,203 ERROR [LocalJobRunner Map Task Executor #0]
> mr.ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:148)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapRunner.run(ExecMapRunner.java:37)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:465)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:349)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException: java.util.ArrayList cannot be cast to
> org.apache.hadoop.hive.serde2.io.HiveDecimalWritable
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:793)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> ... 11 more
> Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast
> to org.apache.hadoop.hive.serde2.io.HiveDecimalWritable
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileCont$PercentileContEvaluator.iterate(GenericUDAFPercentileCont.java:259)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:214)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:639)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:814)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:720)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:788)
> ... 17 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22205) Upgrade zookeeper and curator versions
[
https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937381#comment-16937381
]
Hive QA commented on HIVE-22205:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981246/HIVE-22205.6.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16984 tests
executed
*Failed tests:*
{noformat}
TestJdbcWithMiniLlapArrow - did not produce a TEST-*.xml file (likely timed
out) (batchId=284)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.org.apache.hadoop.hive.cli.TestAccumuloCliDriver
(batchId=289)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18712/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18712/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18712/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981246 - PreCommit-HIVE-Build
> Upgrade zookeeper and curator versions
> --
>
> Key: HIVE-22205
> URL: https://issues.apache.org/jira/browse/HIVE-22205
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
> Attachments: HIVE-22205.2.patch, HIVE-22205.3.patch,
> HIVE-22205.4.patch, HIVE-22205.6.patch, HIVE-22205.patch
>
>
> Other components like hadoop have switched to using new ZK versions. So these
> jars end up in classpath for hive services and could cause issues due to
> in-compatible curator versions that hive uses.
> So it makes sense for hive to upgrade the ZK and curator versions to try to
> keep up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Status: Open (was: Patch Available) > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Bodor >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch, HIVE-21449.5.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Attachment: HIVE-21449.5.patch > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Bodor >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch, HIVE-21449.5.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Status: Patch Available (was: Open) > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Bodor >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch, HIVE-21449.5.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22221) Llap external client - Need to reduce LlapBaseInputFormat#getSplits() footprint
[
https://issues.apache.org/jira/browse/HIVE-1?focusedWorklogId=318029=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318029
]
ASF GitHub Bot logged work on HIVE-1:
-
Author: ASF GitHub Bot
Created on: 25/Sep/19 03:12
Start Date: 25/Sep/19 03:12
Worklog Time Spent: 10m
Work Description: ShubhamChaurasia commented on pull request #778:
HIVE-1: Llap external client - Need to reduce
LlapBaseInputFormat#getSplits() footprint
URL: https://github.com/apache/hive/pull/778#discussion_r327915751
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java
##
@@ -192,9 +199,31 @@ public PlanFragment(TezWork work, Schema schema, JobConf
jc) {
@Override
public void process(Object[] arguments) throws HiveException {
+initArgs(arguments);
+try {
+ SplitResult splitResult = getSplitResult(false);
+ InputSplit[] splits = schemaSplitOnly ? new
InputSplit[]{splitResult.schemaSplit} : splitResult.actualSplits;
+ for (InputSplit s : splits) {
+Object[] os = new Object[1];
+bos.reset();
+s.write(dos);
+byte[] frozen = bos.toByteArray();
+os[0] = frozen;
+forward(os);
+ }
+} catch (Exception e) {
+ throw new HiveException(e);
+}
+ }
-String query = stringOI.getPrimitiveJavaObject(arguments[0]);
-int num = intOI.get(arguments[1]);
+ protected void initArgs(Object[] arguments) {
+inputArgQuery = stringOI.getPrimitiveJavaObject(arguments[0]);
+inputArgNumSplits = intOI.get(arguments[1]);
+schemaSplitOnly = inputArgNumSplits == 0;
Review comment:
Sometimes at client side we only need schema and not the entire split
information.
In actual usage, when we define a dataframe in spark, it needs to know the
schema only.
This option was introduced in:
https://issues.apache.org/jira/browse/HIVE-19682
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 318029)
Time Spent: 0.5h (was: 20m)
> Llap external client - Need to reduce LlapBaseInputFormat#getSplits()
> footprint
> -
>
> Key: HIVE-1
> URL: https://issues.apache.org/jira/browse/HIVE-1
> Project: Hive
> Issue Type: Bug
> Components: llap, UDF
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-1.1.patch, HIVE-1.2.patch,
> HIVE-1.3.patch, HIVE-1.4.patch, HIVE-1.5.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> While querying through llap external client, LlapBaseInputFormat#getSplits()
> invokes get_splits() (GenericUDTFGetSplits) udtf under the hoods.
> GenericUDTFGetSplits returns LlapInputSplit in which planBytes[] occupies
> around 90% of the split size.
> Depending on data size/partitions and plan, LlapInputSplit can grow upto 1mb
> with planBytes[] being common to all the splits and occupying more than 850
> kb. Also, it sometimes causes OOM on HS2 depending on HS2 heap size.
> This can be resolved by separating out common parts from actual splits and
> reassembling them at client side.
> We can also provide an option where client can say it does not want to
> reassemble them and can take the control of reassembling in it's hands.
> Splits can be broken like:
> 1) schema split
> 2) plan split
> 3) actual split 1
> 4) actual split 2and so on.
> This greatly reduces the memory(in my case from 5GB(~5000 splits) to around
> 15MB) on server side and hence the data transfer. And this eliminates OOM on
> HS2 side.
> cc [~jdere] [~sankarh] [~thejas]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22225) add owner name for create database if missing
[
https://issues.apache.org/jira/browse/HIVE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937345#comment-16937345
]
Hive QA commented on HIVE-5:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981243/HIVE-5.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 17002 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=362)
org.apache.hadoop.hive.metastore.TestGetPartitionsUsingProjectionAndFilterSpecs.testGetPartitionsUsingValuesWithJDO
(batchId=225)
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=233)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMergeOnTezEdges
(batchId=333)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18711/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18711/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18711/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981243 - PreCommit-HIVE-Build
> add owner name for create database if missing
> -
>
> Key: HIVE-5
> URL: https://issues.apache.org/jira/browse/HIVE-5
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Sam An
>Assignee: Sam An
>Priority: Minor
> Attachments: HIVE-5.1.patch, HIVE-5.2.patch
>
>
> When Spark connects to HMS, the database owner name is not filled. This could
> happen to other clients as well. We shall fill this in HMS. Ownership info is
> useful for authorizer component in Ranger, etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937321#comment-16937321
]
Jesus Camacho Rodriguez commented on HIVE-22079:
+1
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch,
> HIVE-22079.6.patch, HIVE-22079.7.patch, HIVE-22079.8.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?focusedWorklogId=317972=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317972
]
ASF GitHub Bot logged work on HIVE-22079:
-
Author: ASF GitHub Bot
Created on: 25/Sep/19 01:38
Start Date: 25/Sep/19 01:38
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on issue #780: HIVE-22079: Post
order walker for iterating over expression tree
URL: https://github.com/apache/hive/pull/780#issuecomment-534813665
@jcamachor I have addressed the review comments.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317972)
Time Spent: 1h (was: 50m)
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch,
> HIVE-22079.6.patch, HIVE-22079.7.patch, HIVE-22079.8.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22079:
---
Status: Patch Available (was: Open)
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch,
> HIVE-22079.6.patch, HIVE-22079.7.patch, HIVE-22079.8.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22079:
---
Status: Open (was: Patch Available)
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch,
> HIVE-22079.6.patch, HIVE-22079.7.patch, HIVE-22079.8.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22079:
---
Attachment: HIVE-22079.8.patch
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch,
> HIVE-22079.6.patch, HIVE-22079.7.patch, HIVE-22079.8.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22225) add owner name for create database if missing
[
https://issues.apache.org/jira/browse/HIVE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937314#comment-16937314
]
Hive QA commented on HIVE-5:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 2 new + 426 unchanged - 0 fixed = 428 total (was 426) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18711/dev-support/hive-personality.sh
|
| git revision | master / 48ae7ef |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18711/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18711/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> add owner name for create database if missing
> -
>
> Key: HIVE-5
> URL: https://issues.apache.org/jira/browse/HIVE-5
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Sam An
>Assignee: Sam An
>Priority: Minor
> Attachments: HIVE-5.1.patch, HIVE-5.2.patch
>
>
> When Spark connects to HMS, the database owner name is not filled. This could
> happen to other clients as well. We shall fill this in HMS. Ownership info is
> useful for authorizer component in Ranger, etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22227) Tez bucket pruning produces wrong result with shared work optimization
[
https://issues.apache.org/jira/browse/HIVE-7?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-7:
---
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master
> Tez bucket pruning produces wrong result with shared work optimization
> --
>
> Key: HIVE-7
> URL: https://issues.apache.org/jira/browse/HIVE-7
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-7.1.patch
>
>
> *Reproducer*
> {code:sql}
> set hive.tez.bucket.pruning=true;
> set hive.optimize.shared.work=true;
> CREATE TABLE srcbucket_mapjoin_n16(key int, value string) partitioned by (ds
> string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE;
> CREATE TABLE tab_part_n10 (key int, value string) PARTITIONED BY(ds STRING)
> CLUSTERED BY (key) SORTED BY (key) INTO 4 BUCKETS STORED AS ORCFILE;
> CREATE TABLE srcbucket_mapjoin_part_n17 (key int, value string) partitioned
> by (ds string) CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE;
> load data local inpath '$HIVE_SRC/data/files/bmj/00_0' INTO TABLE
> srcbucket_mapjoin_n16 partition(ds='2008-04-08');
> load data local inpath '.$HIVE_SRC/data/files/bmj1/01_0' INTO TABLE
> srcbucket_mapjoin_n16 partition(ds='2008-04-08');
> load data local inpath '$HIVE_SRC/data/files/bmj/00_0' INTO TABLE
> srcbucket_mapjoin_part_n17 partition(ds='2008-04-08');
> load data local inpath '$HIVE_SRC/data/files/bmj/01_0' INTO TABLE
> srcbucket_mapjoin_part_n17 partition(ds='2008-04-08');
> load data local inpath '$HIVE_SRC/data/files/bmj/02_0' INTO TABLE
> srcbucket_mapjoin_part_n17 partition(ds='2008-04-08');
> set hive.optimize.bucketingsorting=false;
> insert overwrite table tab_part_n10 partition (ds='2008-04-08')
> select key,value from srcbucket_mapjoin_part_n17;
> CREATE TABLE tab_n9(key int, value string) PARTITIONED BY(ds STRING)
> CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS STORED AS ORCFILE;
> insert overwrite table tab_n9 partition (ds='2008-04-08')
> select key,value from srcbucket_mapjoin_n16;
> select * from
> (select * from tab_n9 where tab_n9.key = 0)a
> join
> (select * from tab_part_n10 where tab_part_n10.key = 98)b full outer join
> tab_part_n10 c on a.key = b.key and b.key = c.key
> order by 1,2,3,4,5,6,7,8,9;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937303#comment-16937303
]
Hive QA commented on HIVE-21449:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981227/HIVE-21449.5.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16988 tests
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=111)
[explainanalyze_3.q,orc_vectorization_ppd.q,topnkey.q,vector_non_string_partition.q,multi_count_distinct.q,explainanalyze_1.q,hybridgrace_hashjoin_1.q,tez_union_udtf.q,delete_orig_table.q,tez_complextype_with_null.q,hybridgrace_hashjoin_2.q,tez-tag.q,vector_topnkey.q,explainanalyze_5.q,vector_join_part_col_char.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18710/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18710/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18710/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981227 - PreCommit-HIVE-Build
> implement 'WITHIN GROUP' clause
> ---
>
> Key: HIVE-21449
> URL: https://issues.apache.org/jira/browse/HIVE-21449
> Project: Hive
> Issue Type: Improvement
>Reporter: Laszlo Bodor
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch,
> HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937278#comment-16937278
]
Hive QA commented on HIVE-21449:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
2s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 4 new + 512 unchanged - 0
fixed = 516 total (was 512) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18710/dev-support/hive-personality.sh
|
| git revision | master / 48ae7ef |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18710/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18710/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> implement 'WITHIN GROUP' clause
> ---
>
> Key: HIVE-21449
> URL: https://issues.apache.org/jira/browse/HIVE-21449
> Project: Hive
> Issue Type: Improvement
>Reporter: Laszlo Bodor
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch,
> HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22137) Implement alter/rename partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937263#comment-16937263
]
Hive QA commented on HIVE-22137:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981215/HIVE-22137.01.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17140 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18709/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18709/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18709/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981215 - PreCommit-HIVE-Build
> Implement alter/rename partition related methods on temporary tables
>
>
> Key: HIVE-22137
> URL: https://issues.apache.org/jira/browse/HIVE-22137
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22137.01.patch
>
>
> IMetaStoreClient exposes the following methods related to altering of
> partitions:
> {code:java}
> void alter_partition(String dbName, String tblName, Partition newPart);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart);
> void alter_partition(String dbName, String tblName, Partition newPart,
> EnvironmentContext environmentContext);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext, String writeIdList);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext,String writeIdList, long
> writeId);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts, EnvironmentContext environmentContext, String
> writeIdList, long writeId);
> void renamePartition(final String dbname, final String tableName, final
> List part_vals, final Partition newPart);
> void renamePartition(String catName, String dbname, String tableName,
> List part_vals, Partition newPart, String validWriteIds){code}
> These should be implemented, in order to completely support partition on
> temporary tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22221) Llap external client - Need to reduce LlapBaseInputFormat#getSplits() footprint
[
https://issues.apache.org/jira/browse/HIVE-1?focusedWorklogId=317897=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317897
]
ASF GitHub Bot logged work on HIVE-1:
-
Author: ASF GitHub Bot
Created on: 24/Sep/19 23:14
Start Date: 24/Sep/19 23:14
Worklog Time Spent: 10m
Work Description: jdere commented on pull request #778: HIVE-1: Llap
external client - Need to reduce LlapBaseInputFormat#getSplits() footprint
URL: https://github.com/apache/hive/pull/778#discussion_r327840568
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java
##
@@ -192,9 +199,31 @@ public PlanFragment(TezWork work, Schema schema, JobConf
jc) {
@Override
public void process(Object[] arguments) throws HiveException {
+initArgs(arguments);
+try {
+ SplitResult splitResult = getSplitResult(false);
+ InputSplit[] splits = schemaSplitOnly ? new
InputSplit[]{splitResult.schemaSplit} : splitResult.actualSplits;
+ for (InputSplit s : splits) {
+Object[] os = new Object[1];
+bos.reset();
+s.write(dos);
+byte[] frozen = bos.toByteArray();
+os[0] = frozen;
+forward(os);
+ }
+} catch (Exception e) {
+ throw new HiveException(e);
+}
+ }
-String query = stringOI.getPrimitiveJavaObject(arguments[0]);
-int num = intOI.get(arguments[1]);
+ protected void initArgs(Object[] arguments) {
+inputArgQuery = stringOI.getPrimitiveJavaObject(arguments[0]);
+inputArgNumSplits = intOI.get(arguments[1]);
+schemaSplitOnly = inputArgNumSplits == 0;
Review comment:
Under what circumstances does numSplits actually get set to 0, during actual
usage?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317897)
Time Spent: 20m (was: 10m)
> Llap external client - Need to reduce LlapBaseInputFormat#getSplits()
> footprint
> -
>
> Key: HIVE-1
> URL: https://issues.apache.org/jira/browse/HIVE-1
> Project: Hive
> Issue Type: Bug
> Components: llap, UDF
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-1.1.patch, HIVE-1.2.patch,
> HIVE-1.3.patch, HIVE-1.4.patch, HIVE-1.5.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> While querying through llap external client, LlapBaseInputFormat#getSplits()
> invokes get_splits() (GenericUDTFGetSplits) udtf under the hoods.
> GenericUDTFGetSplits returns LlapInputSplit in which planBytes[] occupies
> around 90% of the split size.
> Depending on data size/partitions and plan, LlapInputSplit can grow upto 1mb
> with planBytes[] being common to all the splits and occupying more than 850
> kb. Also, it sometimes causes OOM on HS2 depending on HS2 heap size.
> This can be resolved by separating out common parts from actual splits and
> reassembling them at client side.
> We can also provide an option where client can say it does not want to
> reassemble them and can take the control of reassembling in it's hands.
> Splits can be broken like:
> 1) schema split
> 2) plan split
> 3) actual split 1
> 4) actual split 2and so on.
> This greatly reduces the memory(in my case from 5GB(~5000 splits) to around
> 15MB) on server side and hence the data transfer. And this eliminates OOM on
> HS2 side.
> cc [~jdere] [~sankarh] [~thejas]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[ https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-22042: Status: Patch Available (was: In Progress) > Set hive.exec.dynamic.partition.mode=nonstrict by default > - > > Key: HIVE-22042 > URL: https://issues.apache.org/jira/browse/HIVE-22042 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: John Sherman >Priority: Minor > Attachments: HIVE-22042.1.patch, HIVE-22042.2.patch, > HIVE-22042.3.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[ https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-22042: Attachment: HIVE-22042.3.patch > Set hive.exec.dynamic.partition.mode=nonstrict by default > - > > Key: HIVE-22042 > URL: https://issues.apache.org/jira/browse/HIVE-22042 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: John Sherman >Priority: Minor > Attachments: HIVE-22042.1.patch, HIVE-22042.2.patch, > HIVE-22042.3.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[ https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-22042: Status: In Progress (was: Patch Available) > Set hive.exec.dynamic.partition.mode=nonstrict by default > - > > Key: HIVE-22042 > URL: https://issues.apache.org/jira/browse/HIVE-22042 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: John Sherman >Priority: Minor > Attachments: HIVE-22042.1.patch, HIVE-22042.2.patch, > HIVE-22042.3.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22137) Implement alter/rename partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937239#comment-16937239
]
Hive QA commented on HIVE-22137:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
7s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} standalone-metastore/metastore-server: The patch
generated 0 new + 4 unchanged - 2 fixed = 4 total (was 6) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 3 new + 17 unchanged - 0 fixed
= 20 total (was 17) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18709/dev-support/hive-personality.sh
|
| git revision | master / 48ae7ef |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18709/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18709/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18709/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Implement alter/rename partition related methods on temporary tables
>
>
> Key: HIVE-22137
> URL: https://issues.apache.org/jira/browse/HIVE-22137
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22137.01.patch
>
>
> IMetaStoreClient exposes the following methods related to altering of
> partitions:
> {code:java}
> void alter_partition(String dbName, String tblName, Partition newPart);
> void alter_partition(String catName, String dbName, String
[jira] [Commented] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[
https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937222#comment-16937222
]
Hive QA commented on HIVE-22042:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981210/HIVE-22042.2.patch
{color:green}SUCCESS:{color} +1 due to 292 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17000 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[infer_bucket_sort_num_buckets]
(batchId=194)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18708/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18708/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18708/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981210 - PreCommit-HIVE-Build
> Set hive.exec.dynamic.partition.mode=nonstrict by default
> -
>
> Key: HIVE-22042
> URL: https://issues.apache.org/jira/browse/HIVE-22042
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: John Sherman
>Priority: Minor
> Attachments: HIVE-22042.1.patch, HIVE-22042.2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22239) Scale data size using column value ranges
[ https://issues.apache.org/jira/browse/HIVE-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22239: -- Labels: pull-request-available (was: ) > Scale data size using column value ranges > - > > Key: HIVE-22239 > URL: https://issues.apache.org/jira/browse/HIVE-22239 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22239.patch > > > Currently, min/max values for columns are only used to determine whether a > certain range filter falls out of range and thus filters all rows or none at > all. If it does not, we just use a heuristic that the condition will filter > 1/3 of the input rows. Instead of using that heuristic, we can use another > one that assumes that data will be uniformly distributed across that range, > and calculate the selectivity for the condition accordingly. > This patch also includes the propagation of min/max column values from > statistics to the optimizer for timestamp type. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22239) Scale data size using column value ranges
[ https://issues.apache.org/jira/browse/HIVE-22239?focusedWorklogId=317871=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317871 ] ASF GitHub Bot logged work on HIVE-22239: - Author: ASF GitHub Bot Created on: 24/Sep/19 21:47 Start Date: 24/Sep/19 21:47 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #787: HIVE-22239 URL: https://github.com/apache/hive/pull/787 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 317871) Remaining Estimate: 0h Time Spent: 10m > Scale data size using column value ranges > - > > Key: HIVE-22239 > URL: https://issues.apache.org/jira/browse/HIVE-22239 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22239.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Currently, min/max values for columns are only used to determine whether a > certain range filter falls out of range and thus filters all rows or none at > all. If it does not, we just use a heuristic that the condition will filter > 1/3 of the input rows. Instead of using that heuristic, we can use another > one that assumes that data will be uniformly distributed across that range, > and calculate the selectivity for the condition accordingly. > This patch also includes the propagation of min/max column values from > statistics to the optimizer for timestamp type. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-22239) Scale data size using column value ranges
[ https://issues.apache.org/jira/browse/HIVE-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-22239 started by Jesus Camacho Rodriguez. -- > Scale data size using column value ranges > - > > Key: HIVE-22239 > URL: https://issues.apache.org/jira/browse/HIVE-22239 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22239.patch > > > Currently, min/max values for columns are only used to determine whether a > certain range filter falls out of range and thus filters all rows or none at > all. If it does not, we just use a heuristic that the condition will filter > 1/3 of the input rows. Instead of using that heuristic, we can use another > one that assumes that data will be uniformly distributed across that range, > and calculate the selectivity for the condition accordingly. > This patch also includes the propagation of min/max column values from > statistics to the optimizer for timestamp type. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22239) Scale data size using column value ranges
[ https://issues.apache.org/jira/browse/HIVE-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22239: --- Attachment: HIVE-22239.patch > Scale data size using column value ranges > - > > Key: HIVE-22239 > URL: https://issues.apache.org/jira/browse/HIVE-22239 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22239.patch > > > Currently, min/max values for columns are only used to determine whether a > certain range filter falls out of range and thus filters all rows or none at > all. If it does not, we just use a heuristic that the condition will filter > 1/3 of the input rows. Instead of using that heuristic, we can use another > one that assumes that data will be uniformly distributed across that range, > and calculate the selectivity for the condition accordingly. > This patch also includes the propagation of min/max column values from > statistics to the optimizer for timestamp type. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22239) Scale data size using column value ranges
[ https://issues.apache.org/jira/browse/HIVE-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22239: --- Status: Patch Available (was: In Progress) > Scale data size using column value ranges > - > > Key: HIVE-22239 > URL: https://issues.apache.org/jira/browse/HIVE-22239 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22239.patch > > > Currently, min/max values for columns are only used to determine whether a > certain range filter falls out of range and thus filters all rows or none at > all. If it does not, we just use a heuristic that the condition will filter > 1/3 of the input rows. Instead of using that heuristic, we can use another > one that assumes that data will be uniformly distributed across that range, > and calculate the selectivity for the condition accordingly. > This patch also includes the propagation of min/max column values from > statistics to the optimizer for timestamp type. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22239) Scale data size using column value ranges
[ https://issues.apache.org/jira/browse/HIVE-22239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-22239: -- > Scale data size using column value ranges > - > > Key: HIVE-22239 > URL: https://issues.apache.org/jira/browse/HIVE-22239 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > Currently, min/max values for columns are only used to determine whether a > certain range filter falls out of range and thus filters all rows or none at > all. If it does not, we just use a heuristic that the condition will filter > 1/3 of the input rows. Instead of using that heuristic, we can use another > one that assumes that data will be uniformly distributed across that range, > and calculate the selectivity for the condition accordingly. > This patch also includes the propagation of min/max column values from > statistics to the optimizer for timestamp type. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[
https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937210#comment-16937210
]
Hive QA commented on HIVE-22042:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
58s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
38s{color} | {color:red} branch/hcatalog no findbugs output file
(hcatalog/target/findbugsXml.xml) {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} The patch hcatalog passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch streaming passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hcatalog-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hive-blobstore passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 536
unchanged - 21 fixed = 536 total (was 557) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
38s{color} | {color:red} patch/hcatalog no findbugs output file
(hcatalog/target/findbugsXml.xml) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5
[jira] [Updated] (HIVE-22205) Upgrade zookeeper and curator versions
[ https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22205: - Attachment: HIVE-22205.6.patch > Upgrade zookeeper and curator versions > -- > > Key: HIVE-22205 > URL: https://issues.apache.org/jira/browse/HIVE-22205 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-22205.2.patch, HIVE-22205.3.patch, > HIVE-22205.4.patch, HIVE-22205.6.patch, HIVE-22205.patch > > > Other components like hadoop have switched to using new ZK versions. So these > jars end up in classpath for hive services and could cause issues due to > in-compatible curator versions that hive uses. > So it makes sense for hive to upgrade the ZK and curator versions to try to > keep up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22205) Upgrade zookeeper and curator versions
[ https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22205: - Status: Patch Available (was: Open) > Upgrade zookeeper and curator versions > -- > > Key: HIVE-22205 > URL: https://issues.apache.org/jira/browse/HIVE-22205 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-22205.2.patch, HIVE-22205.3.patch, > HIVE-22205.4.patch, HIVE-22205.6.patch, HIVE-22205.patch > > > Other components like hadoop have switched to using new ZK versions. So these > jars end up in classpath for hive services and could cause issues due to > in-compatible curator versions that hive uses. > So it makes sense for hive to upgrade the ZK and curator versions to try to > keep up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22205) Upgrade zookeeper and curator versions
[ https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22205: - Status: Open (was: Patch Available) > Upgrade zookeeper and curator versions > -- > > Key: HIVE-22205 > URL: https://issues.apache.org/jira/browse/HIVE-22205 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-22205.2.patch, HIVE-22205.3.patch, > HIVE-22205.4.patch, HIVE-22205.6.patch, HIVE-22205.patch > > > Other components like hadoop have switched to using new ZK versions. So these > jars end up in classpath for hive services and could cause issues due to > in-compatible curator versions that hive uses. > So it makes sense for hive to upgrade the ZK and curator versions to try to > keep up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[
https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937182#comment-16937182
]
Hive QA commented on HIVE-22038:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981209/HIVE-22038.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17000 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=273)
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testQueueingWithThreads
(batchId=319)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18707/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18707/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18707/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981209 - PreCommit-HIVE-Build
> Fix memory related sideeffects of opening/closing sessions
> --
>
> Key: HIVE-22038
> URL: https://issues.apache.org/jira/browse/HIVE-22038
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch,
> HIVE-22038.02.patch, HIVE-22038.02.patch, HIVE-22038.02.patch,
> HIVE-22038.02.patch, HIVE-22038.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22225) add owner name for create database if missing
[ https://issues.apache.org/jira/browse/HIVE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam An updated HIVE-5: -- Attachment: HIVE-5.2.patch > add owner name for create database if missing > - > > Key: HIVE-5 > URL: https://issues.apache.org/jira/browse/HIVE-5 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Sam An >Assignee: Sam An >Priority: Minor > Attachments: HIVE-5.1.patch, HIVE-5.2.patch > > > When Spark connects to HMS, the database owner name is not filled. This could > happen to other clients as well. We shall fill this in HMS. Ownership info is > useful for authorizer component in Ranger, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[
https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937145#comment-16937145
]
Hive QA commented on HIVE-22038:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
57s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18707/dev-support/hive-personality.sh
|
| git revision | master / 48ae7ef |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18707/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix memory related sideeffects of opening/closing sessions
> --
>
> Key: HIVE-22038
> URL: https://issues.apache.org/jira/browse/HIVE-22038
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch,
> HIVE-22038.02.patch, HIVE-22038.02.patch, HIVE-22038.02.patch,
> HIVE-22038.02.patch, HIVE-22038.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[
https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937115#comment-16937115
]
Hive QA commented on HIVE-22238:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981202/HIVE-22238.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 193 failed/errored test(s), 17000 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[updateBasicStats]
(batchId=69)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_3]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_4]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_5]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_rebuild_dummy]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_time_window]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar]
(batchId=172)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query23]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query10]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query11]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query12]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query13]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query15]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query16]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query17]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query18]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query19]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query1]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query1b]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query20]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query21]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query22]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query23]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query24]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query25]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query26]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query27]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query29]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query30]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query31]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query32]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query33]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query34]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query35]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query36]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query37]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query38]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query39]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query3]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query40]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query42]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query43]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query45]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query46]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query47]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query48]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query49]
(batchId=297)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query4]
(batchId=297)
[jira] [Commented] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[
https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937089#comment-16937089
]
Hive QA commented on HIVE-22238:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 5m
55s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18706/dev-support/hive-personality.sh
|
| git revision | master / 48ae7ef |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18706/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> PK/FK selectivity estimation underscales estimations
>
>
> Key: HIVE-22238
> URL: https://issues.apache.org/jira/browse/HIVE-22238
> Project: Hive
> Issue Type: Bug
> Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22238.01.patch
>
>
> at [this
> point|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2182]
> the parent operators rownum is scaled according to pkfkselectivity
> however [pkfkselectivity is
> computed|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2157]
> on a whole subtree.
> Scaling it by that amount will count in estimation already used when
> parentstats was calculated...so depending on the number of upstream joins -
> this may lead to severe underestimations
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937087#comment-16937087
]
Hive QA commented on HIVE-21884:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
25s{color} | {color:blue} standalone-metastore/metastore-common in master has
32 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
7s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
51s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} service in master has 49 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
47s{color} | {color:blue} itests/util in master has 51 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
45s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} standalone-metastore: The patch generated 19 new +
1679 unchanged - 15 fixed = 1698 total (was 1694) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 1 new + 493 unchanged - 0 fixed = 494 total (was 493) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} common: The patch generated 2 new + 367 unchanged - 0
fixed = 369 total (was 367) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch metastore passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 18 new + 1186 unchanged - 15 fixed = 1204 total (was 1201) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 59 new + 656 unchanged - 0
fixed = 715 total (was 656) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} service: The patch generated 0 new + 38 unchanged -
1 fixed = 38 total (was 39) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 3m
51s{color} | {color:red} root: The patch generated 98 new + 2823 unchanged - 16
fixed = 2921 total (was 2839) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} The patch hcatalog-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} The patch hive-minikdc passed checkstyle {color} |
|
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Fix Version/s: 4.0.0 > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Issue Type: Improvement (was: Bug) > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Bodor >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Status: Patch Available (was: Open) > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Attachment: HIVE-21449.5.patch > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Status: Open (was: Patch Available) > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22209) Creating a materialized view with no tables should be handled more gracefully
[
https://issues.apache.org/jira/browse/HIVE-22209?focusedWorklogId=317727=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317727
]
ASF GitHub Bot logged work on HIVE-22209:
-
Author: ASF GitHub Bot
Created on: 24/Sep/19 17:58
Start Date: 24/Sep/19 17:58
Worklog Time Spent: 10m
Work Description: scarlin-cloudera commented on pull request #782:
HIVE-22209: Graceful error msg when there is no table for a materiali…
URL: https://github.com/apache/hive/pull/782#discussion_r327755104
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
##
@@ -456,4 +456,22 @@ public boolean containsQueryWithoutSourceTable() {
}
return aliasToTabs.size()==0 && aliasToSubq.size()==0;
}
+
+ // returns false when the query block doesn't have
+ // a table defined, e.g. "select 5"
+ public boolean hasTableDefined() {
+// A table is not defined when there only exists a dummy
+// table in the alias, so if it is blank, we know a table
+// is defined.
+if (aliasToTabs.size() == 0) {
Review comment:
Actually, it's more than suspicious...I have a bug (was just thinking about
it)
Even if it is true that the size is 0 when there are no tables, there will
be a bug when there is one table that has one subquery that has no table.
So here's my proposal...put in an if check that returns false if there is
exactly one table defined, it is a DUMMY_TABLE, and it doesn't have any
subqueries (it looks like QB has that information). Does that sound good?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317727)
Time Spent: 1.5h (was: 1h 20m)
> Creating a materialized view with no tables should be handled more gracefully
> -
>
> Key: HIVE-22209
> URL: https://issues.apache.org/jira/browse/HIVE-22209
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Steve Carlin
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-22209.1.patch, HIVE-22209.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Currently, materialized views without a table reference are not supported.
> However, instead of printing a clear message about it, when a materialized
> view is created without a table reference, we fail with an unclear message.
> {code}
> > create materialized view mv_test1 as select 5;
> (...)
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Add request
> failed :
> INSERT INTO MV_TABLES_USED (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) )
> INFO : Completed executing
> command(queryId=hive_20190916203511_b609cccf-f5e3-45dd-abfd-6e869d94e39a);
> Time taken: 10.469 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaExcep
> tion(message:Add request failed : INSERT INTO MV_TABLES_USED
> (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) ) (state=08S01,code=1)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22209) Creating a materialized view with no tables should be handled more gracefully
[
https://issues.apache.org/jira/browse/HIVE-22209?focusedWorklogId=317722=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317722
]
ASF GitHub Bot logged work on HIVE-22209:
-
Author: ASF GitHub Bot
Created on: 24/Sep/19 17:52
Start Date: 24/Sep/19 17:52
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #782: HIVE-22209:
Graceful error msg when there is no table for a materiali…
URL: https://github.com/apache/hive/pull/782#discussion_r327752325
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
##
@@ -456,4 +456,22 @@ public boolean containsQueryWithoutSourceTable() {
}
return aliasToTabs.size()==0 && aliasToSubq.size()==0;
}
+
+ // returns false when the query block doesn't have
+ // a table defined, e.g. "select 5"
+ public boolean hasTableDefined() {
+// A table is not defined when there only exists a dummy
+// table in the alias, so if it is blank, we know a table
+// is defined.
+if (aliasToTabs.size() == 0) {
Review comment:
Can we check when you were getting `null`? It is suspicious... AFAIK, before
calling this method, we iterate through the same structure in L14102 in
SemanticAnalyzer and alias should not be null (btw, this code could be
integrated within this loop to avoid looping twice).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317722)
Time Spent: 1h 20m (was: 1h 10m)
> Creating a materialized view with no tables should be handled more gracefully
> -
>
> Key: HIVE-22209
> URL: https://issues.apache.org/jira/browse/HIVE-22209
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Steve Carlin
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-22209.1.patch, HIVE-22209.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently, materialized views without a table reference are not supported.
> However, instead of printing a clear message about it, when a materialized
> view is created without a table reference, we fail with an unclear message.
> {code}
> > create materialized view mv_test1 as select 5;
> (...)
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Add request
> failed :
> INSERT INTO MV_TABLES_USED (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) )
> INFO : Completed executing
> command(queryId=hive_20190916203511_b609cccf-f5e3-45dd-abfd-6e869d94e39a);
> Time taken: 10.469 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaExcep
> tion(message:Add request failed : INSERT INTO MV_TABLES_USED
> (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) ) (state=08S01,code=1)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937051#comment-16937051
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981192/HIVE-21844.15.patch
{color:green}SUCCESS:{color} +1 due to 15 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 17036 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb_schq]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_join_hash]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_reduce_side]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[update_all_partitioned]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_case_when_2]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_expressions]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_distinct_2]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_udf_string_to_boolean]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[windowing]
(batchId=176)
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=362)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18705/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18705/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18705/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981192 - PreCommit-HIVE-Build
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21884.01.patch,
> HIVE-21884.02.patch, HIVE-21884.03.patch, HIVE-21884.09.patch,
> HIVE-21884.10.patch, HIVE-21884.10.patch, HIVE-21884.11.patch,
> HIVE-21884.12.patch, HIVE-21884.13.patch, HIVE-21884.14.patch,
> HIVE-21884.14.patch, HIVE-21884.14.patch, Scheduled queries2.pdf
>
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22209) Creating a materialized view with no tables should be handled more gracefully
[
https://issues.apache.org/jira/browse/HIVE-22209?focusedWorklogId=317710=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317710
]
ASF GitHub Bot logged work on HIVE-22209:
-
Author: ASF GitHub Bot
Created on: 24/Sep/19 17:48
Start Date: 24/Sep/19 17:48
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #782: HIVE-22209:
Graceful error msg when there is no table for a materiali…
URL: https://github.com/apache/hive/pull/782#discussion_r327750279
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
##
@@ -456,4 +456,22 @@ public boolean containsQueryWithoutSourceTable() {
}
return aliasToTabs.size()==0 && aliasToSubq.size()==0;
}
+
+ // returns false when the query block doesn't have
+ // a table defined, e.g. "select 5"
+ public boolean hasTableDefined() {
+// A table is not defined when there only exists a dummy
+// table in the alias, so if it is blank, we know a table
+// is defined.
+if (aliasToTabs.size() == 0) {
+ return true;
+}
+for (String alias : aliasToTabs.keySet()) {
Review comment:
I was referring to `getTabAliases()`; I checked the code and realized that
this is basically the same. This should be fine.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317710)
Time Spent: 1h 10m (was: 1h)
> Creating a materialized view with no tables should be handled more gracefully
> -
>
> Key: HIVE-22209
> URL: https://issues.apache.org/jira/browse/HIVE-22209
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Steve Carlin
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-22209.1.patch, HIVE-22209.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Currently, materialized views without a table reference are not supported.
> However, instead of printing a clear message about it, when a materialized
> view is created without a table reference, we fail with an unclear message.
> {code}
> > create materialized view mv_test1 as select 5;
> (...)
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Add request
> failed :
> INSERT INTO MV_TABLES_USED (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) )
> INFO : Completed executing
> command(queryId=hive_20190916203511_b609cccf-f5e3-45dd-abfd-6e869d94e39a);
> Time taken: 10.469 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaExcep
> tion(message:Add request failed : INSERT INTO MV_TABLES_USED
> (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) ) (state=08S01,code=1)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22137) Implement alter/rename partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-22137:
-
Status: Patch Available (was: Open)
> Implement alter/rename partition related methods on temporary tables
>
>
> Key: HIVE-22137
> URL: https://issues.apache.org/jira/browse/HIVE-22137
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22137.01.patch
>
>
> IMetaStoreClient exposes the following methods related to altering of
> partitions:
> {code:java}
> void alter_partition(String dbName, String tblName, Partition newPart);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart);
> void alter_partition(String dbName, String tblName, Partition newPart,
> EnvironmentContext environmentContext);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext, String writeIdList);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext,String writeIdList, long
> writeId);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts, EnvironmentContext environmentContext, String
> writeIdList, long writeId);
> void renamePartition(final String dbname, final String tableName, final
> List part_vals, final Partition newPart);
> void renamePartition(String catName, String dbname, String tableName,
> List part_vals, Partition newPart, String validWriteIds){code}
> These should be implemented, in order to completely support partition on
> temporary tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22137) Implement alter/rename partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-22137:
-
Attachment: HIVE-22137.01.patch
> Implement alter/rename partition related methods on temporary tables
>
>
> Key: HIVE-22137
> URL: https://issues.apache.org/jira/browse/HIVE-22137
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22137.01.patch
>
>
> IMetaStoreClient exposes the following methods related to altering of
> partitions:
> {code:java}
> void alter_partition(String dbName, String tblName, Partition newPart);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart);
> void alter_partition(String dbName, String tblName, Partition newPart,
> EnvironmentContext environmentContext);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext, String writeIdList);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext,String writeIdList, long
> writeId);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts, EnvironmentContext environmentContext, String
> writeIdList, long writeId);
> void renamePartition(final String dbname, final String tableName, final
> List part_vals, final Partition newPart);
> void renamePartition(String catName, String dbname, String tableName,
> List part_vals, Partition newPart, String validWriteIds){code}
> These should be implemented, in order to completely support partition on
> temporary tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21975) Fix incremental compilation
[
https://issues.apache.org/jira/browse/HIVE-21975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-21975:
--
Labels: pull-request-available (was: )
> Fix incremental compilation
> ---
>
> Key: HIVE-21975
> URL: https://issues.apache.org/jira/browse/HIVE-21975
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Steve Carlin
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21975.1.patch, HIVE-21975.2.patch, HIVE-21975.patch
>
>
> we have an incremental compilation issue around SA ; mostly because of "?
> extends Serializable"
> it could be reproduced with:
> {code}
> git clean -dfx
> mvn install -pl ql -am -DskipTests
> touch `find . -name Sema*A*java` `find . -name Task*Factory.java`
> mvn install -pl ql -DskipTests
> {code}
> error is:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile
> (default-compile) on project hive-exec: Compilation failure: Compilation
> failure:
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[12573,60]
> incompatible types: java.util.List extends java.io.Serializable>> cannot be converted to
> java.util.List>
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[15187,49]
> incompatible types: java.util.List>
> cannot be converted to java.util.List extends java.io.Serializable>>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21975) Fix incremental compilation
[
https://issues.apache.org/jira/browse/HIVE-21975?focusedWorklogId=317642=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317642
]
ASF GitHub Bot logged work on HIVE-21975:
-
Author: ASF GitHub Bot
Created on: 24/Sep/19 16:42
Start Date: 24/Sep/19 16:42
Worklog Time Spent: 10m
Work Description: scarlin-cloudera commented on pull request #786:
HIVE-21975: Fixed incremental compilation.
URL: https://github.com/apache/hive/pull/786
When compiling files one by one, a compile error was produced
when assigning a Task to a Task.
This checkin fixes the compile error (related to HIVE-20008).
Sidenote: Even though most of the calls are more generalized,
the "extends Serialiable" clause is left in a few key places
to ensure that the Task object is always instantiated with a
serializable parameter.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317642)
Remaining Estimate: 0h
Time Spent: 10m
> Fix incremental compilation
> ---
>
> Key: HIVE-21975
> URL: https://issues.apache.org/jira/browse/HIVE-21975
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Steve Carlin
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21975.1.patch, HIVE-21975.2.patch, HIVE-21975.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> we have an incremental compilation issue around SA ; mostly because of "?
> extends Serializable"
> it could be reproduced with:
> {code}
> git clean -dfx
> mvn install -pl ql -am -DskipTests
> touch `find . -name Sema*A*java` `find . -name Task*Factory.java`
> mvn install -pl ql -DskipTests
> {code}
> error is:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile
> (default-compile) on project hive-exec: Compilation failure: Compilation
> failure:
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[12573,60]
> incompatible types: java.util.List extends java.io.Serializable>> cannot be converted to
> java.util.List>
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[15187,49]
> incompatible types: java.util.List>
> cannot be converted to java.util.List extends java.io.Serializable>>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22209) Creating a materialized view with no tables should be handled more gracefully
[
https://issues.apache.org/jira/browse/HIVE-22209?focusedWorklogId=317634=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317634
]
ASF GitHub Bot logged work on HIVE-22209:
-
Author: ASF GitHub Bot
Created on: 24/Sep/19 16:32
Start Date: 24/Sep/19 16:32
Worklog Time Spent: 10m
Work Description: scarlin-cloudera commented on pull request #782:
HIVE-22209: Graceful error msg when there is no table for a materiali…
URL: https://github.com/apache/hive/pull/782#discussion_r327715922
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
##
@@ -456,4 +456,22 @@ public boolean containsQueryWithoutSourceTable() {
}
return aliasToTabs.size()==0 && aliasToSubq.size()==0;
}
+
+ // returns false when the query block doesn't have
+ // a table defined, e.g. "select 5"
+ public boolean hasTableDefined() {
+// A table is not defined when there only exists a dummy
+// table in the alias, so if it is blank, we know a table
+// is defined.
+if (aliasToTabs.size() == 0) {
+ return true;
+}
+for (String alias : aliasToTabs.keySet()) {
Review comment:
I looked at the code again...do you men aliases? That would definitely be a
better struct to use than the hashmap of tableAliases. If you confirm this,
I'll make the change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317634)
Time Spent: 40m (was: 0.5h)
> Creating a materialized view with no tables should be handled more gracefully
> -
>
> Key: HIVE-22209
> URL: https://issues.apache.org/jira/browse/HIVE-22209
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Steve Carlin
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-22209.1.patch, HIVE-22209.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently, materialized views without a table reference are not supported.
> However, instead of printing a clear message about it, when a materialized
> view is created without a table reference, we fail with an unclear message.
> {code}
> > create materialized view mv_test1 as select 5;
> (...)
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Add request
> failed :
> INSERT INTO MV_TABLES_USED (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) )
> INFO : Completed executing
> command(queryId=hive_20190916203511_b609cccf-f5e3-45dd-abfd-6e869d94e39a);
> Time taken: 10.469 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaExcep
> tion(message:Add request failed : INSERT INTO MV_TABLES_USED
> (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) ) (state=08S01,code=1)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22209) Creating a materialized view with no tables should be handled more gracefully
[
https://issues.apache.org/jira/browse/HIVE-22209?focusedWorklogId=317635=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317635
]
ASF GitHub Bot logged work on HIVE-22209:
-
Author: ASF GitHub Bot
Created on: 24/Sep/19 16:32
Start Date: 24/Sep/19 16:32
Worklog Time Spent: 10m
Work Description: scarlin-cloudera commented on pull request #782:
HIVE-22209: Graceful error msg when there is no table for a materiali…
URL: https://github.com/apache/hive/pull/782#discussion_r327715946
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java
##
@@ -456,4 +456,22 @@ public boolean containsQueryWithoutSourceTable() {
}
return aliasToTabs.size()==0 && aliasToSubq.size()==0;
}
+
+ // returns false when the query block doesn't have
+ // a table defined, e.g. "select 5"
+ public boolean hasTableDefined() {
+// A table is not defined when there only exists a dummy
+// table in the alias, so if it is blank, we know a table
+// is defined.
+if (aliasToTabs.size() == 0) {
Review comment:
This seemed weird to me too, and perhaps I need a better solution. I may be
mistaken here, but I had tests fail when the table had no alias (e.g. 'select *
from tbl1' as opposed to 'select * from tbl1 alias1'). However, when we had no
table, there WAS an alias, and it was "DUMMY_TABLE";
I didn't see any structure within QB containing all table names. Is there
another place you think I can grab it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317635)
Time Spent: 50m (was: 40m)
> Creating a materialized view with no tables should be handled more gracefully
> -
>
> Key: HIVE-22209
> URL: https://issues.apache.org/jira/browse/HIVE-22209
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Steve Carlin
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-22209.1.patch, HIVE-22209.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently, materialized views without a table reference are not supported.
> However, instead of printing a clear message about it, when a materialized
> view is created without a table reference, we fail with an unclear message.
> {code}
> > create materialized view mv_test1 as select 5;
> (...)
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Add request
> failed :
> INSERT INTO MV_TABLES_USED (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) )
> INFO : Completed executing
> command(queryId=hive_20190916203511_b609cccf-f5e3-45dd-abfd-6e869d94e39a);
> Time taken: 10.469 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaExcep
> tion(message:Add request failed : INSERT INTO MV_TABLES_USED
> (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) ) (state=08S01,code=1)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22209) Creating a materialized view with no tables should be handled more gracefully
[
https://issues.apache.org/jira/browse/HIVE-22209?focusedWorklogId=317637=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317637
]
ASF GitHub Bot logged work on HIVE-22209:
-
Author: ASF GitHub Bot
Created on: 24/Sep/19 16:32
Start Date: 24/Sep/19 16:32
Worklog Time Spent: 10m
Work Description: scarlin-cloudera commented on pull request #782:
HIVE-22209: Graceful error msg when there is no table for a materiali…
URL: https://github.com/apache/hive/pull/782#discussion_r327716068
##
File path: ql/src/test/results/clientnegative/materialized_view_no_tbl.q.out
##
@@ -0,0 +1 @@
+FAILED: SemanticException Materialized view must have a table defined.
Review comment:
DU!!! Noob mistake. Not sure how I missed this! Will add on next
patch.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 317637)
Time Spent: 1h (was: 50m)
> Creating a materialized view with no tables should be handled more gracefully
> -
>
> Key: HIVE-22209
> URL: https://issues.apache.org/jira/browse/HIVE-22209
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Steve Carlin
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-22209.1.patch, HIVE-22209.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, materialized views without a table reference are not supported.
> However, instead of printing a clear message about it, when a materialized
> view is created without a table reference, we fail with an unclear message.
> {code}
> > create materialized view mv_test1 as select 5;
> (...)
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Add request
> failed :
> INSERT INTO MV_TABLES_USED (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) )
> INFO : Completed executing
> command(queryId=hive_20190916203511_b609cccf-f5e3-45dd-abfd-6e869d94e39a);
> Time taken: 10.469 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaExcep
> tion(message:Add request failed : INSERT INTO MV_TABLES_USED
> (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) ) (state=08S01,code=1)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936962#comment-16936962
]
Hive QA commented on HIVE-22191:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981187/HIVE-22191.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 16836 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[topnkey]
(batchId=111)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vector_join_part_col_char]
(batchId=111)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vector_topnkey]
(batchId=111)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18704/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18704/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18704/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981187 - PreCommit-HIVE-Build
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.2.patch, HIVE-22191.3.patch,
> HIVE-22191.4.patch, HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[ https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-22042: Status: In Progress (was: Patch Available) > Set hive.exec.dynamic.partition.mode=nonstrict by default > - > > Key: HIVE-22042 > URL: https://issues.apache.org/jira/browse/HIVE-22042 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: John Sherman >Priority: Minor > Attachments: HIVE-22042.1.patch, HIVE-22042.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[ https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-22042: Status: Patch Available (was: In Progress) > Set hive.exec.dynamic.partition.mode=nonstrict by default > - > > Key: HIVE-22042 > URL: https://issues.apache.org/jira/browse/HIVE-22042 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: John Sherman >Priority: Minor > Attachments: HIVE-22042.1.patch, HIVE-22042.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22084) Implement exchange partitions related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-22084:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master.
Thanks for the patch [~lpinter]!
> Implement exchange partitions related methods on temporary tables
> -
>
> Key: HIVE-22084
> URL: https://issues.apache.org/jira/browse/HIVE-22084
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22084.01.patch, HIVE-22084.02.patch,
> HIVE-22084.03.patch, HIVE-22084.04.patch, HIVE-22084.05.patch,
> HIVE-22084.06.patch, HIVE-22084.07.patch
>
>
> IMetaStoreClient exposes the following methods related to exchanging
> partitions:
> {code:java}
> Partition exchange_partition(Map partitionSpecs, String
> sourceDb, String sourceTable, String destdb, String destTableName);
> Partition exchange_partition(Map partitionSpecs, String
> sourceCat, String sourceDb, String sourceTable, String destCat, String
> destdb, String destTableName);
> List exchange_partitions(Map partitionSpecs,
> String sourceDb, String sourceTable, String destdb, String destTableName);
> List exchange_partitions(Map partitionSpecs,
> String sourceCat, String sourceDb, String sourceTable, String destCat, String
> destdb, String destTableName);{code}
> In order to support partitions on temporary tables, these methods must be
> implemented.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.9.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=317621=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317621 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 24/Sep/19 16:08 Start Date: 24/Sep/19 16:08 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #785: [WIP] HIVE-21737: Bump Apache Avro to 1.9.0 URL: https://github.com/apache/hive/pull/785 I took #674 and fixed the issues that broke, let's see if CI is happy this time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 317621) Time Spent: 20m (was: 10m) > Upgrade Avro to version 1.9.1 > - > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Minor > Labels: pull-request-available > Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner > version of Avro without Jackson in the public API. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[ https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-22042: Attachment: HIVE-22042.2.patch > Set hive.exec.dynamic.partition.mode=nonstrict by default > - > > Key: HIVE-22042 > URL: https://issues.apache.org/jira/browse/HIVE-22042 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: John Sherman >Priority: Minor > Attachments: HIVE-22042.1.patch, HIVE-22042.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[ https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22038: Attachment: HIVE-22038.02.patch > Fix memory related sideeffects of opening/closing sessions > -- > > Key: HIVE-22038 > URL: https://issues.apache.org/jira/browse/HIVE-22038 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch, > HIVE-22038.02.patch, HIVE-22038.02.patch, HIVE-22038.02.patch, > HIVE-22038.02.patch, HIVE-22038.02.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936926#comment-16936926
]
Hive QA commented on HIVE-22191:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 4 new + 526 unchanged - 38
fixed = 530 total (was 564) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
17s{color} | {color:green} ql generated 0 new + 1551 unchanged - 16 fixed =
1551 total (was 1567) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18704/dev-support/hive-personality.sh
|
| git revision | master / b9b1271 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18704/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18704/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.2.patch, HIVE-22191.3.patch,
> HIVE-22191.4.patch, HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[ https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22238: Component/s: Statistics > PK/FK selectivity estimation underscales estimations > > > Key: HIVE-22238 > URL: https://issues.apache.org/jira/browse/HIVE-22238 > Project: Hive > Issue Type: Bug > Components: Statistics >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: Statistics > Attachments: HIVE-22238.01.patch > > > at [this > point|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2182] > the parent operators rownum is scaled according to pkfkselectivity > however [pkfkselectivity is > computed|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2157] > on a whole subtree. > Scaling it by that amount will count in estimation already used when > parentstats was calculated...so depending on the number of upstream joins - > this may lead to severe underestimations -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[ https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22238: Attachment: HIVE-22238.01.patch > PK/FK selectivity estimation underscales estimations > > > Key: HIVE-22238 > URL: https://issues.apache.org/jira/browse/HIVE-22238 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22238.01.patch > > > at [this > point|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2182] > the parent operators rownum is scaled according to pkfkselectivity > however [pkfkselectivity is > computed|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2157] > on a whole subtree. > Scaling it by that amount will count in estimation already used when > parentstats was calculated...so depending on the number of upstream joins - > this may lead to severe underestimations -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[ https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22238: Labels: Statistics (was: ) > PK/FK selectivity estimation underscales estimations > > > Key: HIVE-22238 > URL: https://issues.apache.org/jira/browse/HIVE-22238 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: Statistics > Attachments: HIVE-22238.01.patch > > > at [this > point|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2182] > the parent operators rownum is scaled according to pkfkselectivity > however [pkfkselectivity is > computed|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2157] > on a whole subtree. > Scaling it by that amount will count in estimation already used when > parentstats was calculated...so depending on the number of upstream joins - > this may lead to severe underestimations -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936894#comment-16936894
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981186/HIVE-21884.14.patch
{color:green}SUCCESS:{color} +1 due to 15 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 16872 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_join_hash]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_reduce_side]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_case_when_2]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_expressions]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_distinct_2]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_udf_string_to_boolean]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_streaming]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[windowing]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_3]
(batchId=195)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18703/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18703/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18703/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981186 - PreCommit-HIVE-Build
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21884.01.patch,
> HIVE-21884.02.patch, HIVE-21884.03.patch, HIVE-21884.09.patch,
> HIVE-21884.10.patch, HIVE-21884.10.patch, HIVE-21884.11.patch,
> HIVE-21884.12.patch, HIVE-21884.13.patch, HIVE-21884.14.patch,
> HIVE-21884.14.patch, HIVE-21884.14.patch, Scheduled queries2.pdf
>
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[ https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22238: Labels: (was: Statistics) > PK/FK selectivity estimation underscales estimations > > > Key: HIVE-22238 > URL: https://issues.apache.org/jira/browse/HIVE-22238 > Project: Hive > Issue Type: Bug > Components: Statistics >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22238.01.patch > > > at [this > point|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2182] > the parent operators rownum is scaled according to pkfkselectivity > however [pkfkselectivity is > computed|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2157] > on a whole subtree. > Scaling it by that amount will count in estimation already used when > parentstats was calculated...so depending on the number of upstream joins - > this may lead to severe underestimations -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[ https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22238: Status: Patch Available (was: Open) > PK/FK selectivity estimation underscales estimations > > > Key: HIVE-22238 > URL: https://issues.apache.org/jira/browse/HIVE-22238 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22238.01.patch > > > at [this > point|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2182] > the parent operators rownum is scaled according to pkfkselectivity > however [pkfkselectivity is > computed|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2157] > on a whole subtree. > Scaling it by that amount will count in estimation already used when > parentstats was calculated...so depending on the number of upstream joins - > this may lead to severe underestimations -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21975) Fix incremental compilation
[
https://issues.apache.org/jira/browse/HIVE-21975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936869#comment-16936869
]
Zoltan Haindrich commented on HIVE-21975:
-
note: HIVE-8 have touched a lot of files; the patch doesn't apply anymore
> Fix incremental compilation
> ---
>
> Key: HIVE-21975
> URL: https://issues.apache.org/jira/browse/HIVE-21975
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Steve Carlin
>Priority: Major
> Attachments: HIVE-21975.1.patch, HIVE-21975.2.patch, HIVE-21975.patch
>
>
> we have an incremental compilation issue around SA ; mostly because of "?
> extends Serializable"
> it could be reproduced with:
> {code}
> git clean -dfx
> mvn install -pl ql -am -DskipTests
> touch `find . -name Sema*A*java` `find . -name Task*Factory.java`
> mvn install -pl ql -DskipTests
> {code}
> error is:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile
> (default-compile) on project hive-exec: Compilation failure: Compilation
> failure:
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[12573,60]
> incompatible types: java.util.List extends java.io.Serializable>> cannot be converted to
> java.util.List>
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[15187,49]
> incompatible types: java.util.List>
> cannot be converted to java.util.List extends java.io.Serializable>>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21844.15.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21884.01.patch, > HIVE-21884.02.patch, HIVE-21884.03.patch, HIVE-21884.09.patch, > HIVE-21884.10.patch, HIVE-21884.10.patch, HIVE-21884.11.patch, > HIVE-21884.12.patch, HIVE-21884.13.patch, HIVE-21884.14.patch, > HIVE-21884.14.patch, HIVE-21884.14.patch, Scheduled queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22238) PK/FK selectivity estimation underscales estimations
[ https://issues.apache.org/jira/browse/HIVE-22238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-22238: --- > PK/FK selectivity estimation underscales estimations > > > Key: HIVE-22238 > URL: https://issues.apache.org/jira/browse/HIVE-22238 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > > at [this > point|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2182] > the parent operators rownum is scaled according to pkfkselectivity > however [pkfkselectivity is > computed|https://github.com/apache/hive/blob/5098d155a1e6a164253f5fa98755273bc34085df/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L2157] > on a whole subtree. > Scaling it by that amount will count in estimation already used when > parentstats was calculated...so depending on the number of upstream joins - > this may lead to severe underestimations -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-22220) Upgrade Accumulo version to the latest 2.0 release
[ https://issues.apache.org/jira/browse/HIVE-0?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-0 started by Ivan Suller. -- > Upgrade Accumulo version to the latest 2.0 release > -- > > Key: HIVE-0 > URL: https://issues.apache.org/jira/browse/HIVE-0 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > > To make Thrift upgrade (HIVE-21000) possible we have to upgrade Accumulo to > at least 2.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22220) Upgrade Accumulo version to the latest 2.0 release
[ https://issues.apache.org/jira/browse/HIVE-0?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-0: -- Assignee: Ivan Suller > Upgrade Accumulo version to the latest 2.0 release > -- > > Key: HIVE-0 > URL: https://issues.apache.org/jira/browse/HIVE-0 > Project: Hive > Issue Type: Improvement >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > > To make Thrift upgrade (HIVE-21000) possible we have to upgrade Accumulo to > at least 2.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22191:
--
Attachment: HIVE-22191.4.patch
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.2.patch, HIVE-22191.3.patch,
> HIVE-22191.4.patch, HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22191:
--
Status: In Progress (was: Patch Available)
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.2.patch, HIVE-22191.3.patch,
> HIVE-22191.4.patch, HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22191:
--
Status: Patch Available (was: In Progress)
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.2.patch, HIVE-22191.3.patch,
> HIVE-22191.4.patch, HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936819#comment-16936819
]
Hive QA commented on HIVE-21884:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 16s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-18703/patches/PreCommit-HIVE-Build-18703.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18703/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21884.01.patch, HIVE-21884.02.patch,
> HIVE-21884.03.patch, HIVE-21884.09.patch, HIVE-21884.10.patch,
> HIVE-21884.10.patch, HIVE-21884.11.patch, HIVE-21884.12.patch,
> HIVE-21884.13.patch, HIVE-21884.14.patch, HIVE-21884.14.patch,
> HIVE-21884.14.patch, Scheduled queries2.pdf
>
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22214) Explain vectorization should disable user level explain
[ https://issues.apache.org/jira/browse/HIVE-22214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22214: Fix Version/s: 4.0.0 > Explain vectorization should disable user level explain > --- > > Key: HIVE-22214 > URL: https://issues.apache.org/jira/browse/HIVE-22214 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Trivial > Fix For: 4.0.0 > > Attachments: HIVE-22214.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22214) Explain vectorization should disable user level explain
[ https://issues.apache.org/jira/browse/HIVE-22214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22214: Resolution: Fixed Status: Resolved (was: Patch Available) > Explain vectorization should disable user level explain > --- > > Key: HIVE-22214 > URL: https://issues.apache.org/jira/browse/HIVE-22214 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Trivial > Fix For: 4.0.0 > > Attachments: HIVE-22214.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22214) Explain vectorization should disable user level explain
[ https://issues.apache.org/jira/browse/HIVE-22214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936813#comment-16936813 ] Rajesh Balamohan commented on HIVE-22214: - Thanks [~gopalv]. Committed to master. > Explain vectorization should disable user level explain > --- > > Key: HIVE-22214 > URL: https://issues.apache.org/jira/browse/HIVE-22214 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Trivial > Attachments: HIVE-22214.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22084) Implement exchange partitions related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936794#comment-16936794
]
Hive QA commented on HIVE-22084:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981153/HIVE-22084.07.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17000 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18702/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18702/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18702/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981153 - PreCommit-HIVE-Build
> Implement exchange partitions related methods on temporary tables
> -
>
> Key: HIVE-22084
> URL: https://issues.apache.org/jira/browse/HIVE-22084
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22084.01.patch, HIVE-22084.02.patch,
> HIVE-22084.03.patch, HIVE-22084.04.patch, HIVE-22084.05.patch,
> HIVE-22084.06.patch, HIVE-22084.07.patch
>
>
> IMetaStoreClient exposes the following methods related to exchanging
> partitions:
> {code:java}
> Partition exchange_partition(Map partitionSpecs, String
> sourceDb, String sourceTable, String destdb, String destTableName);
> Partition exchange_partition(Map partitionSpecs, String
> sourceCat, String sourceDb, String sourceTable, String destCat, String
> destdb, String destTableName);
> List exchange_partitions(Map partitionSpecs,
> String sourceDb, String sourceTable, String destdb, String destTableName);
> List exchange_partitions(Map partitionSpecs,
> String sourceCat, String sourceDb, String sourceTable, String destCat, String
> destdb, String destTableName);{code}
> In order to support partitions on temporary tables, these methods must be
> implemented.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21884.14.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21884.01.patch, HIVE-21884.02.patch, > HIVE-21884.03.patch, HIVE-21884.09.patch, HIVE-21884.10.patch, > HIVE-21884.10.patch, HIVE-21884.11.patch, HIVE-21884.12.patch, > HIVE-21884.13.patch, HIVE-21884.14.patch, HIVE-21884.14.patch, > HIVE-21884.14.patch, Scheduled queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22084) Implement exchange partitions related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936747#comment-16936747
]
Hive QA commented on HIVE-22084:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
53s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18702/dev-support/hive-personality.sh
|
| git revision | master / cf6cbda |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18702/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Implement exchange partitions related methods on temporary tables
> -
>
> Key: HIVE-22084
> URL: https://issues.apache.org/jira/browse/HIVE-22084
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22084.01.patch, HIVE-22084.02.patch,
> HIVE-22084.03.patch, HIVE-22084.04.patch, HIVE-22084.05.patch,
> HIVE-22084.06.patch, HIVE-22084.07.patch
>
>
> IMetaStoreClient exposes the following methods related to exchanging
> partitions:
> {code:java}
> Partition exchange_partition(Map partitionSpecs, String
> sourceDb, String sourceTable, String destdb, String destTableName);
> Partition exchange_partition(Map partitionSpecs, String
> sourceCat, String sourceDb, String sourceTable, String destCat, String
> destdb, String destTableName);
> List exchange_partitions(Map partitionSpecs,
> String sourceDb, String sourceTable, String destdb, String destTableName);
> List exchange_partitions(Map
[jira] [Updated] (HIVE-22235) CommandProcessorResponse should not be an exception
[ https://issues.apache.org/jira/browse/HIVE-22235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22235: -- Labels: pull-request-available (was: ) > CommandProcessorResponse should not be an exception > --- > > Key: HIVE-22235 > URL: https://issues.apache.org/jira/browse/HIVE-22235 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22235.01.patch > > > The CommandProcessorResponse class extends Exception. This may be convenient, > but it is wrong, as a response is not an exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22235) CommandProcessorResponse should not be an exception
[ https://issues.apache.org/jira/browse/HIVE-22235?focusedWorklogId=317447=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317447 ] ASF GitHub Bot logged work on HIVE-22235: - Author: ASF GitHub Bot Created on: 24/Sep/19 12:33 Start Date: 24/Sep/19 12:33 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #784: HIVE-22235 CommandProcessorResponse should not be an exception URL: https://github.com/apache/hive/pull/784 The CommandProcessorResponse class extends Exception. This may be convenient, but it is wrong, as a response is not an exception. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 317447) Remaining Estimate: 0h Time Spent: 10m > CommandProcessorResponse should not be an exception > --- > > Key: HIVE-22235 > URL: https://issues.apache.org/jira/browse/HIVE-22235 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22235.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The CommandProcessorResponse class extends Exception. This may be convenient, > but it is wrong, as a response is not an exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22235) CommandProcessorResponse should not be an exception
[
https://issues.apache.org/jira/browse/HIVE-22235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936720#comment-16936720
]
Hive QA commented on HIVE-22235:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981147/HIVE-22235.01.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16836 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18701/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18701/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18701/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981147 - PreCommit-HIVE-Build
> CommandProcessorResponse should not be an exception
> ---
>
> Key: HIVE-22235
> URL: https://issues.apache.org/jira/browse/HIVE-22235
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22235.01.patch
>
>
> The CommandProcessorResponse class extends Exception. This may be convenient,
> but it is wrong, as a response is not an exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22235) CommandProcessorResponse should not be an exception
[
https://issues.apache.org/jira/browse/HIVE-22235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936682#comment-16936682
]
Hive QA commented on HIVE-22235:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 4 new + 106 unchanged - 19
fixed = 110 total (was 125) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18701/dev-support/hive-personality.sh
|
| git revision | master / cf6cbda |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18701/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18701/yetus/whitespace-eol.txt
|
| modules | C: ql hcatalog/server-extensions U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18701/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CommandProcessorResponse should not be an exception
> ---
>
> Key: HIVE-22235
> URL: https://issues.apache.org/jira/browse/HIVE-22235
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22235.01.patch
>
>
> The CommandProcessorResponse class extends Exception. This may be convenient,
> but it is wrong, as a response is not an exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22231) Hive query with big size via knox fails with 'Broken pipe (Write failed)
[
https://issues.apache.org/jira/browse/HIVE-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936659#comment-16936659
]
Hive QA commented on HIVE-22231:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981146/HIVE-22231.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16836 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18700/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18700/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18700/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981146 - PreCommit-HIVE-Build
> Hive query with big size via knox fails with 'Broken pipe (Write failed)
>
>
> Key: HIVE-22231
> URL: https://issues.apache.org/jira/browse/HIVE-22231
> Project: Hive
> Issue Type: Bug
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22231.1.patch, HIVE-22231.2.patch
>
>
> Posting large data through knox is causing the Broken pipe (Write failed)
> java.net.SocketException. Issue here that HS2 is responding with 401 even
> before complete query is transferred.
> HS2 has to wait until all the data is received and then respond with 401.
> That way knox can re-open the connection and send the negotiate header with
> the data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22236) Fail to create View selecting View containing NOT IN subquery
[
https://issues.apache.org/jira/browse/HIVE-22236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936643#comment-16936643
]
Zoltan Matyus commented on HIVE-22236:
--
Attached minimum script to reproduce.
> Fail to create View selecting View containing NOT IN subquery
> -
>
> Key: HIVE-22236
> URL: https://issues.apache.org/jira/browse/HIVE-22236
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Minor
> Attachments: HIVE-22236.q
>
>
> * Given a complicated view with a select statement that has subquery
> containing "{{NOT IN}}"
> * Hive fails to create a simple view as {{SELECT * FROM complicated_view}}
> * (with CBO disabled).
> The unparse replacements of the complicated view will be applied to the text
> of the simple view, resulting in {{IllegalArgumentException: replace: range
> invalid}} exceptions from {{org.antlr.runtime.TokenRewriteStream.replace}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22236) Fail to create View selecting View containing NOT IN subquery
[
https://issues.apache.org/jira/browse/HIVE-22236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Matyus updated HIVE-22236:
-
Attachment: HIVE-22236.q
> Fail to create View selecting View containing NOT IN subquery
> -
>
> Key: HIVE-22236
> URL: https://issues.apache.org/jira/browse/HIVE-22236
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Minor
> Attachments: HIVE-22236.q
>
>
> * Given a complicated view with a select statement that has subquery
> containing "{{NOT IN}}"
> * Hive fails to create a simple view as {{SELECT * FROM complicated_view}}
> * (with CBO disabled).
> The unparse replacements of the complicated view will be applied to the text
> of the simple view, resulting in {{IllegalArgumentException: replace: range
> invalid}} exceptions from {{org.antlr.runtime.TokenRewriteStream.replace}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
