[jira] [Updated] (HIVE-20174) Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation Functions
[ https://issues.apache.org/jira/browse/HIVE-20174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20174: Attachment: HIVE-20174.02.patch > Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation > Functions > > > Key: HIVE-20174 > URL: https://issues.apache.org/jira/browse/HIVE-20174 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20174.01.patch, HIVE-20174.02.patch > > > Write new UT tests that use random data and intentional isRepeating batches > to checks for NULL and Wrong Results for vectorized aggregation functions. > > BUGs found: > 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count > = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there > was an overflow. > 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP, > INTERVAL_DAY_TIME, and String Family. Eliminated redundant code. > 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL > modes (HIVE-18758). > 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in > COMPLETE and FINAL modes. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20174) Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation Functions
[ https://issues.apache.org/jira/browse/HIVE-20174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20174: Status: In Progress (was: Patch Available) > Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation > Functions > > > Key: HIVE-20174 > URL: https://issues.apache.org/jira/browse/HIVE-20174 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20174.01.patch, HIVE-20174.02.patch > > > Write new UT tests that use random data and intentional isRepeating batches > to checks for NULL and Wrong Results for vectorized aggregation functions. > > BUGs found: > 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count > = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there > was an overflow. > 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP, > INTERVAL_DAY_TIME, and String Family. Eliminated redundant code. > 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL > modes (HIVE-18758). > 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in > COMPLETE and FINAL modes. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20174) Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation Functions
[ https://issues.apache.org/jira/browse/HIVE-20174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20174: Status: Patch Available (was: In Progress) > Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation > Functions > > > Key: HIVE-20174 > URL: https://issues.apache.org/jira/browse/HIVE-20174 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20174.01.patch, HIVE-20174.02.patch > > > Write new UT tests that use random data and intentional isRepeating batches > to checks for NULL and Wrong Results for vectorized aggregation functions. > > BUGs found: > 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count > = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there > was an overflow. > 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP, > INTERVAL_DAY_TIME, and String Family. Eliminated redundant code. > 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL > modes (HIVE-18758). > 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in > COMPLETE and FINAL modes. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20079) Populate more accurate rawDataSize for parquet format
[
https://issues.apache.org/jira/browse/HIVE-20079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544385#comment-16544385
]
Hive QA commented on HIVE-20079:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931668/HIVE-20079.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 14645 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=191)
[druidmini_dynamic_partition.q,druidmini_expressions.q,druidmini_test_alter.q,druidmini_test1.q,druidmini_test_insert.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_struct_type_vectorization]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_types_non_dictionary_encoding_vectorization]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_types_vectorization]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_10]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_11]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_12]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_1]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_parquet_projection]
(batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_parquet_types]
(batchId=70)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz]
(batchId=192)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_joins]
(batchId=192)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_masking]
(batchId=192)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12619/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12619/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12619/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931668 - PreCommit-HIVE-Build
> Populate more accurate rawDataSize for parquet format
> -
>
> Key: HIVE-20079
> URL: https://issues.apache.org/jira/browse/HIVE-20079
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-20079.1.patch, HIVE-20079.2.patch
>
>
> Run the following queries and you will see the raw data for the table is 4
> (that is the number of fields) incorrectly. We need to populate correct data
> size so data can be split properly.
> {noformat}
> SET hive.stats.autogather=true;
> CREATE TABLE parquet_stats (id int,str string) STORED AS PARQUET;
> INSERT INTO parquet_stats values(0, 'this is string 0'), (1, 'string 1');
> DESC FORMATTED parquet_stats;
> {noformat}
> {noformat}
> Table Parameters:
> COLUMN_STATS_ACCURATE true
> numFiles1
> numRows 2
> rawDataSize 4
> totalSize 373
> transient_lastDdlTime 1530660523
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20079) Populate more accurate rawDataSize for parquet format
[
https://issues.apache.org/jira/browse/HIVE-20079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544382#comment-16544382
]
Hive QA commented on HIVE-20079:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} ql: The patch generated 3 new + 13 unchanged - 6 fixed
= 16 total (was 19) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12619/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12619/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12619/yetus/whitespace-eol.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12619/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Populate more accurate rawDataSize for parquet format
> -
>
> Key: HIVE-20079
> URL: https://issues.apache.org/jira/browse/HIVE-20079
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-20079.1.patch, HIVE-20079.2.patch
>
>
> Run the following queries and you will see the raw data for the table is 4
> (that is the number of fields) incorrectly. We need to populate correct data
> size so data can be split properly.
> {noformat}
> SET hive.stats.autogather=true;
> CREATE TABLE parquet_stats (id int,str string) STORED AS PARQUET;
> INSERT INTO parquet_stats values(0, 'this is string 0'), (1, 'string 1');
> DESC FORMATTED parquet_stats;
> {noformat}
> {noformat}
> Table Parameters:
> COLUMN_STATS_ACCURATE true
> numFiles1
> numRows 2
> rawDataSize 4
> totalSize 373
> transient_lastDdlTime 1530660523
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20056) SparkPartitionPruner shouldn't be triggered by Spark tasks
[
https://issues.apache.org/jira/browse/HIVE-20056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544379#comment-16544379
]
Hive QA commented on HIVE-20056:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931667/HIVE-20056.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14649 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12618/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12618/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12618/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931667 - PreCommit-HIVE-Build
> SparkPartitionPruner shouldn't be triggered by Spark tasks
> --
>
> Key: HIVE-20056
> URL: https://issues.apache.org/jira/browse/HIVE-20056
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-20056.1.patch
>
>
> It looks like {{SparkDynamicPartitionPruner}} is being called by every Spark
> task because it gets created whenever {{getRecordReader}} is called on the
> associated {{InputFormat}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20056) SparkPartitionPruner shouldn't be triggered by Spark tasks
[
https://issues.apache.org/jira/browse/HIVE-20056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544375#comment-16544375
]
Hive QA commented on HIVE-20056:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 1 new + 30 unchanged - 0 fixed
= 31 total (was 30) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12618/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12618/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12618/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SparkPartitionPruner shouldn't be triggered by Spark tasks
> --
>
> Key: HIVE-20056
> URL: https://issues.apache.org/jira/browse/HIVE-20056
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-20056.1.patch
>
>
> It looks like {{SparkDynamicPartitionPruner}} is being called by every Spark
> task because it gets created whenever {{getRecordReader}} is called on the
> associated {{InputFormat}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17593) DataWritableWriter strip spaces for CHAR type before writing, but predicate generator doesn't do same thing.
[
https://issues.apache.org/jira/browse/HIVE-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544372#comment-16544372
]
Hive QA commented on HIVE-17593:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931657/HIVE-17593.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14649 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12617/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12617/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12617/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931657 - PreCommit-HIVE-Build
> DataWritableWriter strip spaces for CHAR type before writing, but predicate
> generator doesn't do same thing.
>
>
> Key: HIVE-17593
> URL: https://issues.apache.org/jira/browse/HIVE-17593
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.0, 3.0.0
>Reporter: Junjie Chen
>Assignee: Junjie Chen
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-17593.2.patch, HIVE-17593.3.patch,
> HIVE-17593.4.patch, HIVE-17593.5.patch, HIVE-17593.patch
>
>
> DataWritableWriter strip spaces for CHAR type before writing. While when
> generating predicate, it does NOT do same striping which should cause data
> missing!
> In current version, it doesn't cause data missing since predicate is not well
> push down to parquet due to HIVE-17261.
> Please see ConvertAstTosearchArg.java, getTypes treats CHAR and STRING as
> same which will build a predicate with tail spaces.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17593) DataWritableWriter strip spaces for CHAR type before writing, but predicate generator doesn't do same thing.
[
https://issues.apache.org/jira/browse/HIVE-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544358#comment-16544358
]
Hive QA commented on HIVE-17593:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 1 new + 58 unchanged - 1 fixed
= 59 total (was 59) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12617/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12617/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12617/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> DataWritableWriter strip spaces for CHAR type before writing, but predicate
> generator doesn't do same thing.
>
>
> Key: HIVE-17593
> URL: https://issues.apache.org/jira/browse/HIVE-17593
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.3.0, 3.0.0
>Reporter: Junjie Chen
>Assignee: Junjie Chen
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-17593.2.patch, HIVE-17593.3.patch,
> HIVE-17593.4.patch, HIVE-17593.5.patch, HIVE-17593.patch
>
>
> DataWritableWriter strip spaces for CHAR type before writing. While when
> generating predicate, it does NOT do same striping which should cause data
> missing!
> In current version, it doesn't cause data missing since predicate is not well
> push down to parquet due to HIVE-17261.
> Please see ConvertAstTosearchArg.java, getTypes treats CHAR and STRING as
> same which will build a predicate with tail spaces.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20174) Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation Functions
[ https://issues.apache.org/jira/browse/HIVE-20174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20174: Description: Write new UT tests that use random data and intentional isRepeating batches to checks for NULL and Wrong Results for vectorized aggregation functions. BUGs found: 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there was an overflow. 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP, INTERVAL_DAY_TIME, and String Family. Eliminated redundant code. 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL modes (HIVE-18758). 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in COMPLETE and FINAL modes. was: Write new UT tests that use random data and intentional isRepeating batches to checks for NULL and Wrong Results for vectorized aggregation functions. BUGs found: 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there was an overflow. 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP, INTERVAL_DAY_TIME, and String Family. Eliminated redundant code. 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL modes. 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in COMPLETE and FINAL modes. > Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation > Functions > > > Key: HIVE-20174 > URL: https://issues.apache.org/jira/browse/HIVE-20174 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20174.01.patch > > > Write new UT tests that use random data and intentional isRepeating batches > to checks for NULL and Wrong Results for vectorized aggregation functions. > > BUGs found: > 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count > = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there > was an overflow. > 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP, > INTERVAL_DAY_TIME, and String Family. Eliminated redundant code. > 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL > modes (HIVE-18758). > 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in > COMPLETE and FINAL modes. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20174) Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation Functions
[ https://issues.apache.org/jira/browse/HIVE-20174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20174: Description: Write new UT tests that use random data and intentional isRepeating batches to checks for NULL and Wrong Results for vectorized aggregation functions. BUGs found: 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there was an overflow. 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP, INTERVAL_DAY_TIME, and String Family. Eliminated redundant code. 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL modes. 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in COMPLETE and FINAL modes. was: Write new UT tests that use random data and intentional isRepeating batches to checks for NULL and Wrong Results for vectorized aggregation functions. BUGs found: 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there was an overflow. 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP and INTERVAL_DAY_TIME. Eliminated redundant code. 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL modes. 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in COMPLETE and FINAL modes. > Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation > Functions > > > Key: HIVE-20174 > URL: https://issues.apache.org/jira/browse/HIVE-20174 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20174.01.patch > > > Write new UT tests that use random data and intentional isRepeating batches > to checks for NULL and Wrong Results for vectorized aggregation functions. > > BUGs found: > 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count > = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there > was an overflow. > 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP, > INTERVAL_DAY_TIME, and String Family. Eliminated redundant code. > 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL > modes. > 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in > COMPLETE and FINAL modes. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20175) Missing ASF for some class with druid
[
https://issues.apache.org/jira/browse/HIVE-20175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544348#comment-16544348
]
Hive QA commented on HIVE-20175:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931648/HIVE-20175.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14649 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12616/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12616/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12616/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931648 - PreCommit-HIVE-Build
> Missing ASF for some class with druid
> --
>
> Key: HIVE-20175
> URL: https://issues.apache.org/jira/browse/HIVE-20175
> Project: Hive
> Issue Type: Bug
>Reporter: Saijin Huang
>Assignee: Saijin Huang
>Priority: Trivial
> Attachments: HIVE-20175.1.patch
>
>
> when testing druid for unit testing,some class miss the ASF header
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20174) Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation Functions
[ https://issues.apache.org/jira/browse/HIVE-20174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20174: Description: Write new UT tests that use random data and intentional isRepeating batches to checks for NULL and Wrong Results for vectorized aggregation functions. BUGs found: 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there was an overflow. 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP and INTERVAL_DAY_TIME. Eliminated redundant code. 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL modes. 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in COMPLETE and FINAL modes. was: Write new UT tests that use random data and intentional isRepeating batches to checks for NULL and Wrong Results for vectorized aggregation functions. BUGs found: 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there was an overflow. 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP and INTERVAL_DAY_TIME. 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL modes. 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in COMPLETE and FINAL modes. > Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation > Functions > > > Key: HIVE-20174 > URL: https://issues.apache.org/jira/browse/HIVE-20174 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20174.01.patch > > > Write new UT tests that use random data and intentional isRepeating batches > to checks for NULL and Wrong Results for vectorized aggregation functions. > > BUGs found: > 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count > = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there > was an overflow. > 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP > and INTERVAL_DAY_TIME. Eliminated redundant code. > 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL > modes. > 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in > COMPLETE and FINAL modes. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20175) Missing ASF for some class with druid
[
https://issues.apache.org/jira/browse/HIVE-20175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544334#comment-16544334
]
Hive QA commented on HIVE-20175:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
18s{color} | {color:blue} itests/qtest-druid in master has 6 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} itests/qtest-druid: The patch generated 0 new + 2
unchanged - 1 fixed = 2 total (was 3) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 6
unchanged - 1 fixed = 6 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12616/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests/qtest-druid itests/hive-unit U: itests |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12616/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Missing ASF for some class with druid
> --
>
> Key: HIVE-20175
> URL: https://issues.apache.org/jira/browse/HIVE-20175
> Project: Hive
> Issue Type: Bug
>Reporter: Saijin Huang
>Assignee: Saijin Huang
>Priority: Trivial
> Attachments: HIVE-20175.1.patch
>
>
> when testing druid for unit testing,some class miss the ASF header
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20079) Populate more accurate rawDataSize for parquet format
[
https://issues.apache.org/jira/browse/HIVE-20079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544333#comment-16544333
]
Sahil Takiar commented on HIVE-20079:
-
Some of the updates to the q.out files change {{rawDataSize}} to a value of 0,
which doesn't look right - e.g. parquet_vectorization_0.q.out
So does {{BlockMetaData#getTotalByteSize}} return the size of the block when
loaded into memory? It looks like we close the file first and then re-open it
to get this metadata, is there any way to collect it while the file is still
open?
> Populate more accurate rawDataSize for parquet format
> -
>
> Key: HIVE-20079
> URL: https://issues.apache.org/jira/browse/HIVE-20079
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-20079.1.patch, HIVE-20079.2.patch
>
>
> Run the following queries and you will see the raw data for the table is 4
> (that is the number of fields) incorrectly. We need to populate correct data
> size so data can be split properly.
> {noformat}
> SET hive.stats.autogather=true;
> CREATE TABLE parquet_stats (id int,str string) STORED AS PARQUET;
> INSERT INTO parquet_stats values(0, 'this is string 0'), (1, 'string 1');
> DESC FORMATTED parquet_stats;
> {noformat}
> {noformat}
> Table Parameters:
> COLUMN_STATS_ACCURATE true
> numFiles1
> numRows 2
> rawDataSize 4
> totalSize 373
> transient_lastDdlTime 1530660523
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19360) CBO: Add an "optimizedSQL" to QueryPlan object
[
https://issues.apache.org/jira/browse/HIVE-19360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544331#comment-16544331
]
Hive QA commented on HIVE-19360:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931647/HIVE-19360.7.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14649 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_5]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_ssb]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_ssb_2]
(batchId=170)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12615/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12615/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12615/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931647 - PreCommit-HIVE-Build
> CBO: Add an "optimizedSQL" to QueryPlan object
> ---
>
> Key: HIVE-19360
> URL: https://issues.apache.org/jira/browse/HIVE-19360
> Project: Hive
> Issue Type: Improvement
> Components: CBO, Diagnosability
>Affects Versions: 3.1.0
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19360.1.patch, HIVE-19360.2.patch,
> HIVE-19360.3.patch, HIVE-19360.4.patch, HIVE-19360.5.patch,
> HIVE-19360.6.patch, HIVE-19360.7.patch
>
>
> Calcite RelNodes can be converted back into SQL (as the new JDBC storage
> handler does), which allows Hive to print out the post CBO plan as a SQL
> query instead of having to guess the join orders from the subsequent Tez plan.
> The query generated might not be always valid SQL at this point, but is a
> world ahead of DAG plans in readability.
> Eg. tpc-ds Query4 CTEs gets expanded to
> {code}
> SELECT t16.$f3 customer_preferred_cust_flag
> FROM
> (SELECT t0.c_customer_id $f0,
>SUM((t2.ws_ext_list_price -
> t2.ws_ext_wholesale_cost - t2.ws_ext_discount_amt + t2.ws_ext_sales_price) /
> CAST(2 AS DECIMAL(10, 0))) $f8
>FROM
> (SELECT c_customer_sk,
> c_customer_id,
> c_first_name,
> c_last_name,
> c_preferred_cust_flag,
> c_birth_country,
> c_login,
> c_email_address
> FROM default.customer
> WHERE c_customer_sk IS NOT NULL
> AND c_customer_id IS NOT NULL) t0
>INNER JOIN (
> (SELECT ws_sold_date_sk,
> ws_bill_customer_sk,
> ws_ext_discount_amt,
> ws_ext_sales_price,
> ws_ext_wholesale_cost,
> ws_ext_list_price
> FROM default.web_sales
> WHERE ws_bill_customer_sk IS NOT NULL
> AND ws_sold_date_sk IS NOT NULL) t2
>INNER JOIN
> (SELECT d_date_sk,
> CAST(2002 AS INTEGER) d_year
> FROM default.date_dim
> WHERE d_year = 2002
> AND d_date_sk IS NOT NULL) t4 ON t2.ws_sold_date_sk =
> t4.d_date_sk) ON t0.c_customer_sk = t2.ws_bill_customer_sk
>GROUP BY t0.c_customer_id,
> t0.c_first_name,
> t0.c_last_name,
> t0.c_preferred_cust_flag,
> t0.c_birth_country,
> t0.c_login,
> t0.c_email_address) t7
> INNER JOIN (
> (SELECT t9.c_customer_id $f0,
>t9.c_preferred_cust_flag $f3,
>
> SUM((t11.ss_ext_list_price - t11.ss_ext_wholesale_cost -
> t11.ss_ext_discount_amt + t11.ss_ext_sales_price) / CAST(2 AS DECIMAL(10,
> 0))) $f8
>FROM
> (SELECT c_customer_sk,
> c_customer_id,
> c_first_name,
> c_last_name,
> c_preferred_cust_flag,
> c_birth_country,
> c_login,
> c_email_address
>
[jira] [Commented] (HIVE-19360) CBO: Add an "optimizedSQL" to QueryPlan object
[
https://issues.apache.org/jira/browse/HIVE-19360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544323#comment-16544323
]
Hive QA commented on HIVE-19360:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 9 new + 500 unchanged - 5
fixed = 509 total (was 505) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12615/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12615/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests/hive-blobstore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12615/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CBO: Add an "optimizedSQL" to QueryPlan object
> ---
>
> Key: HIVE-19360
> URL: https://issues.apache.org/jira/browse/HIVE-19360
> Project: Hive
> Issue Type: Improvement
> Components: CBO, Diagnosability
>Affects Versions: 3.1.0
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19360.1.patch, HIVE-19360.2.patch,
> HIVE-19360.3.patch, HIVE-19360.4.patch, HIVE-19360.5.patch,
> HIVE-19360.6.patch, HIVE-19360.7.patch
>
>
> Calcite RelNodes can be converted back into SQL (as the new JDBC storage
> handler does), which allows Hive to print out the post CBO plan as a SQL
> query instead of having to guess the join orders from the subsequent Tez plan.
> The query generated might not be always valid SQL at this point, but is a
> world ahead of DAG plans in readability.
> Eg. tpc-ds Query4 CTEs gets expanded to
> {code}
> SELECT t16.$f3 customer_preferred_cust_flag
> FROM
> (SELECT t0.c_customer_id $f0,
>SUM((t2.ws_ext_list_price -
> t2.ws_ext_wholesale_cost - t2.ws_ext_discount_amt + t2.ws_ext_sales_price) /
>
[jira] [Commented] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure
[
https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544313#comment-16544313
]
Hive QA commented on HIVE-19166:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931643/HIVE-19166.12.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14650 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[strict_managed_tables_sysdb]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=165)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12614/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12614/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12614/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931643 - PreCommit-HIVE-Build
> TestMiniLlapLocalCliDriver sysdb failure
>
>
> Key: HIVE-19166
> URL: https://issues.apache.org/jira/browse/HIVE-19166
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19166.04.patch, HIVE-19166.05.patch,
> HIVE-19166.06.patch, HIVE-19166.09.patch, HIVE-19166.1.patch,
> HIVE-19166.10.patch, HIVE-19166.11.patch, HIVE-19166.12.patch,
> HIVE-19166.2.patch, HIVE-19166.3.patch
>
>
> Broken by HIVE-18715
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20079) Populate more accurate rawDataSize for parquet format
[
https://issues.apache.org/jira/browse/HIVE-20079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544305#comment-16544305
]
Aihua Xu commented on HIVE-20079:
-
patch-2: fix affected unit tests.
> Populate more accurate rawDataSize for parquet format
> -
>
> Key: HIVE-20079
> URL: https://issues.apache.org/jira/browse/HIVE-20079
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-20079.1.patch, HIVE-20079.2.patch
>
>
> Run the following queries and you will see the raw data for the table is 4
> (that is the number of fields) incorrectly. We need to populate correct data
> size so data can be split properly.
> {noformat}
> SET hive.stats.autogather=true;
> CREATE TABLE parquet_stats (id int,str string) STORED AS PARQUET;
> INSERT INTO parquet_stats values(0, 'this is string 0'), (1, 'string 1');
> DESC FORMATTED parquet_stats;
> {noformat}
> {noformat}
> Table Parameters:
> COLUMN_STATS_ACCURATE true
> numFiles1
> numRows 2
> rawDataSize 4
> totalSize 373
> transient_lastDdlTime 1530660523
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20079) Populate more accurate rawDataSize for parquet format
[
https://issues.apache.org/jira/browse/HIVE-20079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-20079:
Attachment: HIVE-20079.2.patch
> Populate more accurate rawDataSize for parquet format
> -
>
> Key: HIVE-20079
> URL: https://issues.apache.org/jira/browse/HIVE-20079
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Affects Versions: 2.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-20079.1.patch, HIVE-20079.2.patch
>
>
> Run the following queries and you will see the raw data for the table is 4
> (that is the number of fields) incorrectly. We need to populate correct data
> size so data can be split properly.
> {noformat}
> SET hive.stats.autogather=true;
> CREATE TABLE parquet_stats (id int,str string) STORED AS PARQUET;
> INSERT INTO parquet_stats values(0, 'this is string 0'), (1, 'string 1');
> DESC FORMATTED parquet_stats;
> {noformat}
> {noformat}
> Table Parameters:
> COLUMN_STATS_ACCURATE true
> numFiles1
> numRows 2
> rawDataSize 4
> totalSize 373
> transient_lastDdlTime 1530660523
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure
[
https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544302#comment-16544302
]
Hive QA commented on HIVE-19166:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/util in master has 52 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 3
unchanged - 0 fixed = 4 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12614/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12614/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: metastore ql itests/hive-unit itests/util U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12614/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestMiniLlapLocalCliDriver sysdb failure
>
>
> Key: HIVE-19166
> URL: https://issues.apache.org/jira/browse/HIVE-19166
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19166.04.patch, HIVE-19166.05.patch,
> HIVE-19166.06.patch, HIVE-19166.09.patch, HIVE-19166.1.patch,
> HIVE-19166.10.patch, HIVE-19166.11.patch, HIVE-19166.12.patch,
> HIVE-19166.2.patch, HIVE-19166.3.patch
>
>
> Broken by HIVE-18715
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20056) SparkPartitionPruner shouldn't be triggered by Spark tasks
[
https://issues.apache.org/jira/browse/HIVE-20056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-20056:
Status: Patch Available (was: Open)
> SparkPartitionPruner shouldn't be triggered by Spark tasks
> --
>
> Key: HIVE-20056
> URL: https://issues.apache.org/jira/browse/HIVE-20056
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-20056.1.patch
>
>
> It looks like {{SparkDynamicPartitionPruner}} is being called by every Spark
> task because it gets created whenever {{getRecordReader}} is called on the
> associated {{InputFormat}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20056) SparkPartitionPruner shouldn't be triggered by Spark tasks
[
https://issues.apache.org/jira/browse/HIVE-20056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-20056:
Attachment: HIVE-20056.1.patch
> SparkPartitionPruner shouldn't be triggered by Spark tasks
> --
>
> Key: HIVE-20056
> URL: https://issues.apache.org/jira/browse/HIVE-20056
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-20056.1.patch
>
>
> It looks like {{SparkDynamicPartitionPruner}} is being called by every Spark
> task because it gets created whenever {{getRecordReader}} is called on the
> associated {{InputFormat}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19532) fix tests for master-txnstats branch
[
https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544285#comment-16544285
]
Hive QA commented on HIVE-19532:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931634/HIVE-19532.18.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12613/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12613/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12613/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-07-14 17:44:57.555
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-12613/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-07-14 17:44:57.560
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 1b5903b HIVE-20006: Make materializations invalidation cache
work with multiple active remote metastores (Jesus Camacho Rodriguez, reviewed
by Ashutosh Chauhan)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 1b5903b HIVE-20006: Make materializations invalidation cache
work with multiple active remote metastores (Jesus Camacho Rodriguez, reviewed
by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-07-14 17:44:58.819
+ rm -rf ../yetus_PreCommit-HIVE-Build-12613
+ mkdir ../yetus_PreCommit-HIVE-Build-12613
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-12613
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12613/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:80
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java'
with conflicts.
error: patch failed:
standalone-metastore/metastore-common/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:1
error:
standalone-metastore/metastore-common/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp:
patch does not apply
error: patch failed:
standalone-metastore/metastore-common/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:1
error:
standalone-metastore/metastore-common/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h:
patch does not apply
error: patch failed:
standalone-metastore/metastore-common/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp:1
error:
standalone-metastore/metastore-common/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp:
patch does not apply
error: patch failed:
standalone-metastore/metastore-common/src/gen/thrift/gen-cpp/hive_metastore_types.cpp:1
error:
standalone-metastore/metastore-common/src/gen/thrift/gen-cpp/hive_metastore_types.cpp:
patch does not apply
error: patch failed:
standalone-metastore/metastore-common/src/gen/thrift/gen-cpp/hive_metastore_types.h:1
error:
standalone-metastore/metastore-common/src/gen/thrift/gen-cpp/hive_metastore_types.h:
patch does not apply
error: patch failed:
standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FindSchemasByColsResp.java:350
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FindSchemasByColsResp.java'
with conflicts.
error: patch failed:
standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Materialization.java:589
Falling back to
[jira] [Commented] (HIVE-20164) Murmur Hash : Make sure CTAS and IAS use correct bucketing version
[
https://issues.apache.org/jira/browse/HIVE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544282#comment-16544282
]
Hive QA commented on HIVE-20164:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931630/HIVE-20164.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 14650 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas] (batchId=7)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_vectorization_original]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ctas]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainanalyze_2]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_is_not_distinct_from]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_nullsafe]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_analyze]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[quotedid_smb]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[smb_cache]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_bmj_schema_evolution]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[acid_vectorization_original_tez]
(batchId=107)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3]
(batchId=106)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ctas] (batchId=111)
org.apache.hadoop.hive.ql.TestAcidOnTez.testMapJoinOnTez (batchId=237)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12611/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12611/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12611/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931630 - PreCommit-HIVE-Build
> Murmur Hash : Make sure CTAS and IAS use correct bucketing version
> --
>
> Key: HIVE-20164
> URL: https://issues.apache.org/jira/browse/HIVE-20164
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-20164.1.patch, HIVE-20164.2.patch
>
>
> With the migration to Murmur hash, CTAS and IAS from old table version to new
> table version does not work as intended and data is hashed using old hash
> logic.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20164) Murmur Hash : Make sure CTAS and IAS use correct bucketing version
[
https://issues.apache.org/jira/browse/HIVE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544269#comment-16544269
]
Hive QA commented on HIVE-20164:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 4 new + 39 unchanged - 0 fixed
= 43 total (was 39) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12611/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12611/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12611/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Murmur Hash : Make sure CTAS and IAS use correct bucketing version
> --
>
> Key: HIVE-20164
> URL: https://issues.apache.org/jira/browse/HIVE-20164
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-20164.1.patch, HIVE-20164.2.patch
>
>
> With the migration to Murmur hash, CTAS and IAS from old table version to new
> table version does not work as intended and data is hashed using old hash
> logic.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19408) Improve show materialized views statement to show more information about invalidation
[
https://issues.apache.org/jira/browse/HIVE-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544251#comment-16544251
]
Hive QA commented on HIVE-19408:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931629/HIVE-19408.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 14644 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=191)
[druidmini_dynamic_partition.q,druidmini_expressions.q,druidmini_test_alter.q,druidmini_test1.q,druidmini_test_insert.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_4] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_query5]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_names]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_truncate]
(batchId=27)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cte_mat_4]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table]
(batchId=179)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[temp_table]
(batchId=148)
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters.testShowTablesAll
(batchId=243)
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters.testShowTablesSelected
(batchId=243)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12610/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12610/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12610/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931629 - PreCommit-HIVE-Build
> Improve show materialized views statement to show more information about
> invalidation
> -
>
> Key: HIVE-19408
> URL: https://issues.apache.org/jira/browse/HIVE-19408
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-19408.patch
>
>
> We should show more useful information in addition to materialized view name.
> For instance, information about whether the materialized view contents are
> up-to-date or not, and which table(s) have changed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19408) Improve show materialized views statement to show more information about invalidation
[
https://issues.apache.org/jira/browse/HIVE-19408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544240#comment-16544240
]
Hive QA commented on HIVE-19408:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 10 new + 206 unchanged - 5
fixed = 216 total (was 211) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12610/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12610/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12610/yetus/whitespace-eol.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12610/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve show materialized views statement to show more information about
> invalidation
> -
>
> Key: HIVE-19408
> URL: https://issues.apache.org/jira/browse/HIVE-19408
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-19408.patch
>
>
> We should show more useful information in addition to materialized view name.
> For instance, information about whether the materialized view contents are
> up-to-date or not, and which table(s) have changed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20174) Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation Functions
[
https://issues.apache.org/jira/browse/HIVE-20174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544235#comment-16544235
]
Hive QA commented on HIVE-20174:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931624/HIVE-20174.01.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 14659 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[multiple_agg]
(batchId=265)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_2] (batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_aggregate_9]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_parquet_types]
(batchId=70)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_aggregate_9]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction2]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet_types]
(batchId=173)
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgLongEmpty
(batchId=302)
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgLongNulls
(batchId=302)
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testAvgLongRepeatNulls
(batchId=302)
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testStdDevLongRepeatNulls
(batchId=302)
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testStdLongEmpty
(batchId=302)
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testVarPopLongRepeatNulls
(batchId=302)
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testVarSampLongEmpty
(batchId=302)
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testVarianceLongEmpty
(batchId=302)
org.apache.hadoop.hive.ql.exec.vector.TestVectorGroupByOperator.testVarianceLongNulls
(batchId=302)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12609/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12609/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12609/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931624 - PreCommit-HIVE-Build
> Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation
> Functions
>
>
> Key: HIVE-20174
> URL: https://issues.apache.org/jira/browse/HIVE-20174
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-20174.01.patch
>
>
> Write new UT tests that use random data and intentional isRepeating batches
> to checks for NULL and Wrong Results for vectorized aggregation functions.
>
> BUGs found:
> 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count
> = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there
> was an overflow.
> 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP
> and INTERVAL_DAY_TIME.
> 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL
> modes.
> 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in
> COMPLETE and FINAL modes.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20174) Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation Functions
[
https://issues.apache.org/jira/browse/HIVE-20174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544229#comment-16544229
]
Hive QA commented on HIVE-20174:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
37s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
50s{color} | {color:red} ql: The patch generated 249 new + 1355 unchanged - 9
fixed = 1604 total (was 1364) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
57s{color} | {color:red} ql generated 1 new + 2288 unchanged - 3 fixed = 2289
total (was 2291) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 40s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Redundant nullcheck of inputExpression, which is known to be non-null in
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.getVectorAggregationDesc(String,
List, GenericUDAFEvaluator, TypeInfo, GenericUDAFEvaluator$Mode,
VectorizationContext) Redundant null check at Vectorizer.java:is known to be
non-null in
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.getVectorAggregationDesc(String,
List, GenericUDAFEvaluator, TypeInfo, GenericUDAFEvaluator$Mode,
VectorizationContext) Redundant null check at Vectorizer.java:[line 4273] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12609/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12609/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12609/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12609/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12609/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation
> Functions
>
>
> Key: HIVE-20174
> URL: https://issues.apache.org/jira/browse/HIVE-20174
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-20174.01.patch
>
>
> Write new UT tests that use random data and intentional isRepeating batches
> to checks for NULL and Wrong
[jira] [Updated] (HIVE-20174) Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation Functions
[ https://issues.apache.org/jira/browse/HIVE-20174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20174: Description: Write new UT tests that use random data and intentional isRepeating batches to checks for NULL and Wrong Results for vectorized aggregation functions. BUGs found: 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there was an overflow. 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP and INTERVAL_DAY_TIME. 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL modes. 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in COMPLETE and FINAL modes. was:Write new UT tests that use random data and intentional isRepeating batches to checks for NULL and Wrong Results for vectorized aggregation functions. > Vectorization: Fix NULL / Wrong Results issues in GROUP BY Aggregation > Functions > > > Key: HIVE-20174 > URL: https://issues.apache.org/jira/browse/HIVE-20174 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20174.01.patch > > > Write new UT tests that use random data and intentional isRepeating batches > to checks for NULL and Wrong Results for vectorized aggregation functions. > > BUGs found: > 1) AVG/VARIANCE (family) in PARTIAL1 mode was returning NULL instead of count > = 0, sum = 0 (All data types). For AVG DECIMAL, only return NULL if there > was an overflow. > 2) AVG/MIN/MAX was not detecting repeated NULL correctly for the TIMESTAMP > and INTERVAL_DAY_TIME. > 3) Fix incorrect calculation for VARIANCE (family) in PARTIAL2 and FINAL > modes. > 4) Fix row-mode AVG DECIMAL to enforce output type precision and scale in > COMPLETE and FINAL modes. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20165) Enable ZLIB for streaming ingest
[
https://issues.apache.org/jira/browse/HIVE-20165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544219#comment-16544219
]
Hive QA commented on HIVE-20165:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931623/HIVE-20165.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 14602 tests
executed
*Failed tests:*
{noformat}
TestBeeLineHistory - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestBeelineArgParsing - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestHiveSchemaTool - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestIncrementalRows - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestShutdownHook - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
TestTableOutputFormat - did not produce a TEST-*.xml file (likely timed out)
(batchId=198)
org.apache.hadoop.hive.ql.exec.tez.TestDynamicPartitionPruner.testSingleSourceMultipleFiltersOrdering1
(batchId=304)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12608/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12608/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12608/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931623 - PreCommit-HIVE-Build
> Enable ZLIB for streaming ingest
>
>
> Key: HIVE-20165
> URL: https://issues.apache.org/jira/browse/HIVE-20165
> Project: Hive
> Issue Type: Bug
> Components: Streaming, Transactions
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20165.1.patch, HIVE-20165.2.patch
>
>
> Per [~gopalv]'s recommendation tried running streaming ingest with and
> without zlib. Following are the numbers
>
> *Compression: NONE*
> Total rows committed: 9380
> Throughput: *156* rows/second
> $ hdfs dfs -du -s -h /apps/hive/warehouse/prasanth.db/culvert
> *14.1 G* /apps/hive/warehouse/prasanth.db/culvert
>
> *Compression: ZLIB*
> Total rows committed: 9210
> Throughput: *1535000* rows/second
> $ hdfs dfs -du -s -h /apps/hive/warehouse/prasanth.db/culvert
> *7.4 G* /apps/hive/warehouse/prasanth.db/culvert
>
> ZLIB is getting us 2x compression and only 2% lesser throughput. We should
> enable ZLIB by default for streaming ingest.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20165) Enable ZLIB for streaming ingest
[
https://issues.apache.org/jira/browse/HIVE-20165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544216#comment-16544216
]
Hive QA commented on HIVE-20165:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12608/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql streaming U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12608/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Enable ZLIB for streaming ingest
>
>
> Key: HIVE-20165
> URL: https://issues.apache.org/jira/browse/HIVE-20165
> Project: Hive
> Issue Type: Bug
> Components: Streaming, Transactions
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20165.1.patch, HIVE-20165.2.patch
>
>
> Per [~gopalv]'s recommendation tried running streaming ingest with and
> without zlib. Following are the numbers
>
> *Compression: NONE*
> Total rows committed: 9380
> Throughput: *156* rows/second
> $ hdfs dfs -du -s -h /apps/hive/warehouse/prasanth.db/culvert
> *14.1 G* /apps/hive/warehouse/prasanth.db/culvert
>
> *Compression: ZLIB*
> Total rows committed: 9210
> Throughput: *1535000* rows/second
> $ hdfs dfs -du -s -h /apps/hive/warehouse/prasanth.db/culvert
> *7.4 G* /apps/hive/warehouse/prasanth.db/culvert
>
> ZLIB is getting us 2x compression and only 2% lesser throughput. We should
> enable ZLIB by default for streaming ingest.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19668) Over 30% of the heap wasted by duplicate org.antlr.runtime.CommonToken's and duplicate strings
[
https://issues.apache.org/jira/browse/HIVE-19668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544209#comment-16544209
]
Hive QA commented on HIVE-19668:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931615/HIVE-19668.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14649 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12606/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12606/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12606/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931615 - PreCommit-HIVE-Build
> Over 30% of the heap wasted by duplicate org.antlr.runtime.CommonToken's and
> duplicate strings
> --
>
> Key: HIVE-19668
> URL: https://issues.apache.org/jira/browse/HIVE-19668
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Misha Dmitriev
>Assignee: Misha Dmitriev
>Priority: Major
> Attachments: HIVE-19668.01.patch, HIVE-19668.02.patch,
> HIVE-19668.03.patch, HIVE-19668.04.patch, image-2018-05-22-17-41-39-572.png
>
>
> I've recently analyzed a HS2 heap dump, obtained when there was a huge memory
> spike during compilation of some big query. The analysis was done with jxray
> ([www.jxray.com).|http://www.jxray.com)./] It turns out that more than 90% of
> the 20G heap was used by data structures associated with query parsing
> ({{org.apache.hadoop.hive.ql.parse.QBExpr}}). There are probably multiple
> opportunities for optimizations here. One of them is to stop the code from
> creating duplicate instances of {{org.antlr.runtime.CommonToken}} class. See
> a sample of these objects in the attached image:
> !image-2018-05-22-17-41-39-572.png|width=879,height=399!
> Looks like these particular {{CommonToken}} objects are constants, that don't
> change once created. I see some code, e.g. in
> {{org.apache.hadoop.hive.ql.parse.CalcitePlanner}}, where such objects are
> apparently repeatedly created with e.g. {{new
> CommonToken(HiveParser.TOK_INSERT, "TOK_INSERT")}} If these 33 token kinds
> are instead created once and reused, we will save more than 1/10th of the
> heap in this scenario. Plus, since these objects are small but very numerous,
> getting rid of them will remove a gread deal of pressure from the GC.
> Another source of waste are duplicate strings, that collectively waste 26.1%
> of memory. Some of them come from CommonToken objects that have the same text
> (i.e. for multiple CommonToken objects the contents of their 'text' Strings
> are the same, but each has its own copy of that String). Other duplicate
> strings come from other sources, that are easy enough to fix by adding
> String.intern() calls.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19668) Over 30% of the heap wasted by duplicate org.antlr.runtime.CommonToken's and duplicate strings
[
https://issues.apache.org/jira/browse/HIVE-19668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544178#comment-16544178
]
Hive QA commented on HIVE-19668:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
52s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 4 new + 773 unchanged - 0
fixed = 777 total (was 773) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12606/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12606/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12606/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Over 30% of the heap wasted by duplicate org.antlr.runtime.CommonToken's and
> duplicate strings
> --
>
> Key: HIVE-19668
> URL: https://issues.apache.org/jira/browse/HIVE-19668
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Misha Dmitriev
>Assignee: Misha Dmitriev
>Priority: Major
> Attachments: HIVE-19668.01.patch, HIVE-19668.02.patch,
> HIVE-19668.03.patch, HIVE-19668.04.patch, image-2018-05-22-17-41-39-572.png
>
>
> I've recently analyzed a HS2 heap dump, obtained when there was a huge memory
> spike during compilation of some big query. The analysis was done with jxray
> ([www.jxray.com).|http://www.jxray.com)./] It turns out that more than 90% of
> the 20G heap was used by data structures associated with query parsing
> ({{org.apache.hadoop.hive.ql.parse.QBExpr}}). There are probably multiple
> opportunities for optimizations here. One of them is to stop the code from
> creating duplicate instances of {{org.antlr.runtime.CommonToken}} class. See
> a sample of these objects in the attached image:
> !image-2018-05-22-17-41-39-572.png|width=879,height=399!
> Looks like these particular {{CommonToken}} objects are constants, that don't
> change once created. I see some code, e.g. in
> {{org.apache.hadoop.hive.ql.parse.CalcitePlanner}}, where such objects are
> apparently
[jira] [Commented] (HIVE-20172) StatsUpdater failed with GSS Exception while trying to connect to remote metastore
[
https://issues.apache.org/jira/browse/HIVE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544157#comment-16544157
]
Hive QA commented on HIVE-20172:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931598/HIVE-20172.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14649 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12605/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12605/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12605/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931598 - PreCommit-HIVE-Build
> StatsUpdater failed with GSS Exception while trying to connect to remote
> metastore
> --
>
> Key: HIVE-20172
> URL: https://issues.apache.org/jira/browse/HIVE-20172
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 2.1.1
> Environment: Hive-1.2.1,Hive2.1,java8
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20172.patch
>
>
> StatsUpdater task failed with GSS Exception while trying to connect to remote
> Metastore.
> {code}
> org.apache.thrift.transport.TTransportException: GSS initiate failed
> at
> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
>
> at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
> at
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
>
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
>
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
>
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
>
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:487)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:282)
>
> at
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:76)
>
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564)
>
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:92)
>
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138)
>
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110)
>
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3526)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3558)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:533)
> at
> org.apache.hadoop.hive.ql.txn.compactor.Worker$StatsUpdater.gatherStats(Worker.java:300)
>
> at
> org.apache.hadoop.hive.ql.txn.compactor.CompactorMR.run(CompactorMR.java:265)
> at org.apache.hadoop.hive.ql.txn.compactor.Worker$1.run(Worker.java:177)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
>
> at org.apache.hadoop.hive.ql.txn.compactor.Worker.run(Worker.java:174)
> )
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:534)
>
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:282)
>
> at
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:76)
>
> {code}
> since metastore client
[jira] [Commented] (HIVE-20172) StatsUpdater failed with GSS Exception while trying to connect to remote metastore
[
https://issues.apache.org/jira/browse/HIVE-20172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544141#comment-16544141
]
Hive QA commented on HIVE-20172:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 1 new + 11 unchanged - 0 fixed
= 12 total (was 11) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12605/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12605/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12605/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> StatsUpdater failed with GSS Exception while trying to connect to remote
> metastore
> --
>
> Key: HIVE-20172
> URL: https://issues.apache.org/jira/browse/HIVE-20172
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 2.1.1
> Environment: Hive-1.2.1,Hive2.1,java8
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20172.patch
>
>
> StatsUpdater task failed with GSS Exception while trying to connect to remote
> Metastore.
> {code}
> org.apache.thrift.transport.TTransportException: GSS initiate failed
> at
> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
>
> at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
> at
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
>
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
>
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
>
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
>
> at
>
[jira] [Updated] (HIVE-17593) DataWritableWriter strip spaces for CHAR type before writing, but predicate generator doesn't do same thing.
[ https://issues.apache.org/jira/browse/HIVE-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junjie Chen updated HIVE-17593: --- Attachment: HIVE-17593.5.patch > DataWritableWriter strip spaces for CHAR type before writing, but predicate > generator doesn't do same thing. > > > Key: HIVE-17593 > URL: https://issues.apache.org/jira/browse/HIVE-17593 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.0, 3.0.0 >Reporter: Junjie Chen >Assignee: Junjie Chen >Priority: Major > Labels: pull-request-available > Attachments: HIVE-17593.2.patch, HIVE-17593.3.patch, > HIVE-17593.4.patch, HIVE-17593.5.patch, HIVE-17593.patch > > > DataWritableWriter strip spaces for CHAR type before writing. While when > generating predicate, it does NOT do same striping which should cause data > missing! > In current version, it doesn't cause data missing since predicate is not well > push down to parquet due to HIVE-17261. > Please see ConvertAstTosearchArg.java, getTypes treats CHAR and STRING as > same which will build a predicate with tail spaces. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19940) Push predicates with deterministic UDFs with RBO
[
https://issues.apache.org/jira/browse/HIVE-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544135#comment-16544135
]
Hive QA commented on HIVE-19940:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931585/HIVE-19940.5.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14650 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcDriver2.testSelectExecAsync2 (batchId=249)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12604/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12604/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12604/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931585 - PreCommit-HIVE-Build
> Push predicates with deterministic UDFs with RBO
>
>
> Key: HIVE-19940
> URL: https://issues.apache.org/jira/browse/HIVE-19940
> Project: Hive
> Issue Type: Improvement
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-19940.1.patch, HIVE-19940.2.patch,
> HIVE-19940.3.patch, HIVE-19940.4.patch, HIVE-19940.5.patch
>
>
> With RBO, predicates with any UDF doesn't get pushed down. It makes sense to
> not pushdown the predicates with non-deterministic function as the meaning of
> the query changes after the predicate is resolved to use the function. But
> pushing a deterministic function is beneficial.
> Test Case:
> {code}
> set hive.cbo.enable=false;
> CREATE TABLE `testb`(
>`cola` string COMMENT '',
>`colb` string COMMENT '',
>`colc` string COMMENT '')
> PARTITIONED BY (
>`part1` string,
>`part2` string,
>`part3` string)
> STORED AS AVRO;
> CREATE TABLE `testa`(
>`col1` string COMMENT '',
>`col2` string COMMENT '',
>`col3` string COMMENT '',
>`col4` string COMMENT '',
>`col5` string COMMENT '')
> PARTITIONED BY (
>`part1` string,
>`part2` string,
>`part3` string)
> STORED AS AVRO;
> insert into testA partition (part1='US', part2='ABC', part3='123')
> values ('12.34', '100', '200', '300', 'abc'),
> ('12.341', '1001', '2001', '3001', 'abcd');
> insert into testA partition (part1='UK', part2='DEF', part3='123')
> values ('12.34', '100', '200', '300', 'abc'),
> ('12.341', '1001', '2001', '3001', 'abcd');
> insert into testA partition (part1='US', part2='DEF', part3='200')
> values ('12.34', '100', '200', '300', 'abc'),
> ('12.341', '1001', '2001', '3001', 'abcd');
> insert into testA partition (part1='CA', part2='ABC', part3='300')
> values ('12.34', '100', '200', '300', 'abc'),
> ('12.341', '1001', '2001', '3001', 'abcd');
> insert into testB partition (part1='CA', part2='ABC', part3='300')
> values ('600', '700', 'abc'), ('601', '701', 'abcd');
> insert into testB partition (part1='CA', part2='ABC', part3='400')
> values ( '600', '700', 'abc'), ( '601', '701', 'abcd');
> insert into testB partition (part1='UK', part2='PQR', part3='500')
> values ('600', '700', 'abc'), ('601', '701', 'abcd');
> insert into testB partition (part1='US', part2='DEF', part3='200')
> values ( '600', '700', 'abc'), ('601', '701', 'abcd');
> insert into testB partition (part1='US', part2='PQR', part3='123')
> values ( '600', '700', 'abc'), ('601', '701', 'abcd');
> -- views with deterministic functions
> create view viewDeterministicUDFA partitioned on (vpart1, vpart2, vpart3) as
> select
> cast(col1 as decimal(38,18)) as vcol1,
> cast(col2 as decimal(38,18)) as vcol2,
> cast(col3 as decimal(38,18)) as vcol3,
> cast(col4 as decimal(38,18)) as vcol4,
> cast(col5 as char(10)) as vcol5,
> cast(part1 as char(2)) as vpart1,
> cast(part2 as char(3)) as vpart2,
> cast(part3 as char(3)) as vpart3
> from testa
> where part1 in ('US', 'CA');
> create view viewDeterministicUDFB partitioned on (vpart1, vpart2, vpart3) as
> select
> cast(cola as decimal(38,18)) as vcolA,
> cast(colb as decimal(38,18)) as vcolB,
> cast(colc as char(10)) as vcolC,
> cast(part1 as char(2)) as vpart1,
> cast(part2 as char(3)) as vpart2,
> cast(part3 as char(3)) as vpart3
> from testb
> where part1 in ('US', 'CA');
> explain
> select vcol1, vcol2, vcol3, vcola, vcolb
> from viewDeterministicUDFA a inner join viewDeterministicUDFB b
> on a.vpart1 = b.vpart1
> and a.vpart2 = b.vpart2
> and a.vpart3 = b.vpart3
> and
[jira] [Commented] (HIVE-19940) Push predicates with deterministic UDFs with RBO
[
https://issues.apache.org/jira/browse/HIVE-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544122#comment-16544122
]
Hive QA commented on HIVE-19940:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12604/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12604/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Push predicates with deterministic UDFs with RBO
>
>
> Key: HIVE-19940
> URL: https://issues.apache.org/jira/browse/HIVE-19940
> Project: Hive
> Issue Type: Improvement
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-19940.1.patch, HIVE-19940.2.patch,
> HIVE-19940.3.patch, HIVE-19940.4.patch, HIVE-19940.5.patch
>
>
> With RBO, predicates with any UDF doesn't get pushed down. It makes sense to
> not pushdown the predicates with non-deterministic function as the meaning of
> the query changes after the predicate is resolved to use the function. But
> pushing a deterministic function is beneficial.
> Test Case:
> {code}
> set hive.cbo.enable=false;
> CREATE TABLE `testb`(
>`cola` string COMMENT '',
>`colb` string COMMENT '',
>`colc` string COMMENT '')
> PARTITIONED BY (
>`part1` string,
>`part2` string,
>`part3` string)
> STORED AS AVRO;
> CREATE TABLE `testa`(
>`col1` string COMMENT '',
>`col2` string COMMENT '',
>`col3` string COMMENT '',
>`col4` string COMMENT '',
>`col5` string COMMENT '')
> PARTITIONED BY (
>`part1` string,
>`part2` string,
>`part3` string)
> STORED AS AVRO;
> insert into testA partition (part1='US', part2='ABC', part3='123')
> values ('12.34', '100', '200', '300', 'abc'),
> ('12.341', '1001', '2001', '3001', 'abcd');
> insert into testA partition (part1='UK', part2='DEF', part3='123')
> values ('12.34', '100', '200', '300', 'abc'),
> ('12.341', '1001', '2001', '3001', 'abcd');
> insert into testA partition (part1='US',
[jira] [Commented] (HIVE-20116) TezTask is using parent logger
[
https://issues.apache.org/jira/browse/HIVE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544108#comment-16544108
]
Hive QA commented on HIVE-20116:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931582/HIVE-20116.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14649 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12603/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12603/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12603/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931582 - PreCommit-HIVE-Build
> TezTask is using parent logger
> --
>
> Key: HIVE-20116
> URL: https://issues.apache.org/jira/browse/HIVE-20116
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20116.1.patch, HIVE-20116.2.patch,
> HIVE-20116.3.patch
>
>
> TezTask is using parent's logger (Task). It should instead use its own class
> name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20116) TezTask is using parent logger
[
https://issues.apache.org/jira/browse/HIVE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544106#comment-16544106
]
Hive QA commented on HIVE-20116:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
58s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 3 new + 30 unchanged - 0 fixed
= 33 total (was 30) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
58s{color} | {color:red} ql generated 2 new + 2291 unchanged - 0 fixed = 2293
total (was 2291) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Class org.apache.hadoop.hive.ql.exec.tez.TezTask defines non-transient
non-serializable instance field dagClient In TezTask.java:instance field
dagClient In TezTask.java |
| | org.apache.hadoop.hive.ql.exec.tez.TezTask is Serializable; consider
declaring a serialVersionUID At TezTask.java:a serialVersionUID At
TezTask.java:[lines 98-684] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12603/dev-support/hive-personality.sh
|
| git revision | master / 1b5903b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12603/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12603/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12603/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TezTask is using parent logger
> --
>
> Key: HIVE-20116
> URL: https://issues.apache.org/jira/browse/HIVE-20116
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20116.1.patch, HIVE-20116.2.patch,
> HIVE-20116.3.patch
>
>
> TezTask is using parent's logger (Task). It should instead use its own class
> name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20163) Simplify StringSubstrColStart Initialization
[
https://issues.apache.org/jira/browse/HIVE-20163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544093#comment-16544093
]
Hive QA commented on HIVE-20163:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12931580/HIVE-20163.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14651 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12602/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12602/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12602/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12931580 - PreCommit-HIVE-Build
> Simplify StringSubstrColStart Initialization
>
>
> Key: HIVE-20163
> URL: https://issues.apache.org/jira/browse/HIVE-20163
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20163.1.patch, HIVE-20163.2.patch
>
>
> * Remove code
> * Remove exception handling
> * Remove {{printStackTrace}} call
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20175) Missing ASF for some class with druid
[ https://issues.apache.org/jira/browse/HIVE-20175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saijin Huang updated HIVE-20175: Status: Patch Available (was: Open) > Missing ASF for some class with druid > -- > > Key: HIVE-20175 > URL: https://issues.apache.org/jira/browse/HIVE-20175 > Project: Hive > Issue Type: Bug >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Trivial > Attachments: HIVE-20175.1.patch > > > when testing druid for unit testing,some class miss the ASF header -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20175) Missing ASF for some class with druid
[ https://issues.apache.org/jira/browse/HIVE-20175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saijin Huang updated HIVE-20175: Attachment: HIVE-20175.1.patch > Missing ASF for some class with druid > -- > > Key: HIVE-20175 > URL: https://issues.apache.org/jira/browse/HIVE-20175 > Project: Hive > Issue Type: Bug >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Trivial > Attachments: HIVE-20175.1.patch > > > when testing druid for unit testing,some class miss the ASF header -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20175) Missing ASF for some class with druid
[ https://issues.apache.org/jira/browse/HIVE-20175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saijin Huang updated HIVE-20175: Description: when testing druid for unit testing,some class miss the ASF header > Missing ASF for some class with druid > -- > > Key: HIVE-20175 > URL: https://issues.apache.org/jira/browse/HIVE-20175 > Project: Hive > Issue Type: Bug >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Trivial > > when testing druid for unit testing,some class miss the ASF header -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20175) Missing ASF for some class with druid
[ https://issues.apache.org/jira/browse/HIVE-20175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saijin Huang reassigned HIVE-20175: --- > Missing ASF for some class with druid > -- > > Key: HIVE-20175 > URL: https://issues.apache.org/jira/browse/HIVE-20175 > Project: Hive > Issue Type: Improvement >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Trivial > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20175) Missing ASF for some class with druid
[ https://issues.apache.org/jira/browse/HIVE-20175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saijin Huang updated HIVE-20175: Issue Type: Bug (was: Improvement) > Missing ASF for some class with druid > -- > > Key: HIVE-20175 > URL: https://issues.apache.org/jira/browse/HIVE-20175 > Project: Hive > Issue Type: Bug >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Trivial > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20006) Make materializations invalidation cache work with multiple active remote metastores
[ https://issues.apache.org/jira/browse/HIVE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20006: --- Description: The main points: - Only MVs that use transactional tables can have a time window value of 0. Those are the only MVs that can be guaranteed to not be outdated when a query is executed. - For MVs that +cannot be outdated+, comparison is based on valid write id lists. - For MVs that +can be outdated+: ** The window for valid outdated MVs can be specified in intervals of 1 minute. ** A materialized view is outdated if it was built before that time window and any source table has been modified since. A time window of -1 means to always use the materialized view for rewriting without any checks concerning its validity. If a materialized view uses an external table, the only way to trigger the rewriting would be to set the property to -1, since currently we do not capture for validation purposes whether the external source tables have been modified since the MV was created or not. was: The main points: - Only MVs that use transactional tables and are stored in transactional tables can have a time window value of 0. Those are the only MVs that can be guaranteed to not be outdated when a query is executed. - For MVs that +cannot be outdated+, comparison is based on valid write id lists. - For MVs that +can be outdated+: ** The window for valid outdated MVs can be specified in intervals of 1 minute. ** A materialized view is outdated if it was built before that time window and any source table has been modified since. A time window of -1 means to always use the materialized view for rewriting without any checks concerning its validity. If a materialized view uses an external table, the only way to trigger the rewriting would be to set the property to -1, since currently we do not capture for validation purposes whether the external source tables have been modified since the MV was created or not. > Make materializations invalidation cache work with multiple active remote > metastores > > > Key: HIVE-20006 > URL: https://issues.apache.org/jira/browse/HIVE-20006 > Project: Hive > Issue Type: Improvement > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Critical > Fix For: 4.0.0 > > Attachments: HIVE-19027.01.patch, HIVE-19027.02.patch, > HIVE-19027.03.patch, HIVE-19027.04.patch, HIVE-20006.01.patch, > HIVE-20006.02.patch, HIVE-20006.03.patch, HIVE-20006.04.patch, > HIVE-20006.05.patch, HIVE-20006.06.patch, HIVE-20006.07.patch, > HIVE-20006.patch > > > The main points: > - Only MVs that use transactional tables can have a time window value of 0. > Those are the only MVs that can be guaranteed to not be outdated when a query > is executed. > - For MVs that +cannot be outdated+, comparison is based on valid write id > lists. > - For MVs that +can be outdated+: > ** The window for valid outdated MVs can be specified in intervals of 1 > minute. > ** A materialized view is outdated if it was built before that time window > and any source table has been modified since. > A time window of -1 means to always use the materialized view for rewriting > without any checks concerning its validity. If a materialized view uses an > external table, the only way to trigger the rewriting would be to set the > property to -1, since currently we do not capture for validation purposes > whether the external source tables have been modified since the MV was > created or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19027) Make materializations invalidation cache work with multiple active remote metastores
[ https://issues.apache.org/jira/browse/HIVE-19027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19027: --- Description: The main points: - Only MVs that use transactional tables can have a time window value of 0. Those are the only MVs that can be guaranteed to not be outdated when a query is executed. - For MVs that +cannot be outdated+, comparison is based on valid write id lists. - For MVs that +can be outdated+: ** The window for valid outdated MVs can be specified in intervals of 1 minute. ** A materialized view is outdated if it was built before that time window and any source table has been modified since. A time window of -1 means to always use the materialized view for rewriting without any checks concerning its validity. If a materialized view uses an external table, the only way to trigger the rewriting would be to set the property to -1, since currently we do not capture for validation purposes whether the external source tables have been modified since the MV was created or not. was: The main points: - Only MVs that use transactional tables and are stored in transactional tables can have a time window value of 0. Those are the only MVs that can be guaranteed to not be outdated when a query is executed. - For MVs that +cannot be outdated+, comparison is based on valid write id lists. - For MVs that +can be outdated+: ** The window for valid outdated MVs can be specified in intervals of 1 minute. ** A materialized view is outdated if it was built before that time window and any source table has been modified since. A time window of -1 means to always use the materialized view for rewriting without any checks concerning its validity. If a materialized view uses an external table, the only way to trigger the rewriting would be to set the property to -1, since currently we do not capture for validation purposes whether the external source tables have been modified since the MV was created or not. > Make materializations invalidation cache work with multiple active remote > metastores > > > Key: HIVE-19027 > URL: https://issues.apache.org/jira/browse/HIVE-19027 > Project: Hive > Issue Type: Improvement > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Critical > Fix For: 3.1.0 > > Attachments: HIVE-19027.01.patch, HIVE-19027.02.patch, > HIVE-19027.03.patch, HIVE-19027.04.patch > > > The main points: > - Only MVs that use transactional tables can have a time window value of 0. > Those are the only MVs that can be guaranteed to not be outdated when a query > is executed. > - For MVs that +cannot be outdated+, comparison is based on valid write id > lists. > - For MVs that +can be outdated+: > ** The window for valid outdated MVs can be specified in intervals of 1 > minute. > ** A materialized view is outdated if it was built before that time window > and any source table has been modified since. > A time window of -1 means to always use the materialized view for rewriting > without any checks concerning its validity. If a materialized view uses an > external table, the only way to trigger the rewriting would be to set the > property to -1, since currently we do not capture for validation purposes > whether the external source tables have been modified since the MV was > created or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19360) CBO: Add an "optimizedSQL" to QueryPlan object
[
https://issues.apache.org/jira/browse/HIVE-19360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-19360:
---
Attachment: HIVE-19360.7.patch
> CBO: Add an "optimizedSQL" to QueryPlan object
> ---
>
> Key: HIVE-19360
> URL: https://issues.apache.org/jira/browse/HIVE-19360
> Project: Hive
> Issue Type: Improvement
> Components: CBO, Diagnosability
>Affects Versions: 3.1.0
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19360.1.patch, HIVE-19360.2.patch,
> HIVE-19360.3.patch, HIVE-19360.4.patch, HIVE-19360.5.patch,
> HIVE-19360.6.patch, HIVE-19360.7.patch
>
>
> Calcite RelNodes can be converted back into SQL (as the new JDBC storage
> handler does), which allows Hive to print out the post CBO plan as a SQL
> query instead of having to guess the join orders from the subsequent Tez plan.
> The query generated might not be always valid SQL at this point, but is a
> world ahead of DAG plans in readability.
> Eg. tpc-ds Query4 CTEs gets expanded to
> {code}
> SELECT t16.$f3 customer_preferred_cust_flag
> FROM
> (SELECT t0.c_customer_id $f0,
>SUM((t2.ws_ext_list_price -
> t2.ws_ext_wholesale_cost - t2.ws_ext_discount_amt + t2.ws_ext_sales_price) /
> CAST(2 AS DECIMAL(10, 0))) $f8
>FROM
> (SELECT c_customer_sk,
> c_customer_id,
> c_first_name,
> c_last_name,
> c_preferred_cust_flag,
> c_birth_country,
> c_login,
> c_email_address
> FROM default.customer
> WHERE c_customer_sk IS NOT NULL
> AND c_customer_id IS NOT NULL) t0
>INNER JOIN (
> (SELECT ws_sold_date_sk,
> ws_bill_customer_sk,
> ws_ext_discount_amt,
> ws_ext_sales_price,
> ws_ext_wholesale_cost,
> ws_ext_list_price
> FROM default.web_sales
> WHERE ws_bill_customer_sk IS NOT NULL
> AND ws_sold_date_sk IS NOT NULL) t2
>INNER JOIN
> (SELECT d_date_sk,
> CAST(2002 AS INTEGER) d_year
> FROM default.date_dim
> WHERE d_year = 2002
> AND d_date_sk IS NOT NULL) t4 ON t2.ws_sold_date_sk =
> t4.d_date_sk) ON t0.c_customer_sk = t2.ws_bill_customer_sk
>GROUP BY t0.c_customer_id,
> t0.c_first_name,
> t0.c_last_name,
> t0.c_preferred_cust_flag,
> t0.c_birth_country,
> t0.c_login,
> t0.c_email_address) t7
> INNER JOIN (
> (SELECT t9.c_customer_id $f0,
>t9.c_preferred_cust_flag $f3,
>
> SUM((t11.ss_ext_list_price - t11.ss_ext_wholesale_cost -
> t11.ss_ext_discount_amt + t11.ss_ext_sales_price) / CAST(2 AS DECIMAL(10,
> 0))) $f8
>FROM
> (SELECT c_customer_sk,
> c_customer_id,
> c_first_name,
> c_last_name,
> c_preferred_cust_flag,
> c_birth_country,
> c_login,
> c_email_address
> FROM default.customer
> WHERE c_customer_sk IS NOT NULL
> AND c_customer_id IS NOT NULL) t9
>INNER JOIN (
> (SELECT ss_sold_date_sk,
> ss_customer_sk,
> ss_ext_discount_amt,
> ss_ext_sales_price,
> ss_ext_wholesale_cost,
> ss_ext_list_price
> FROM default.store_sales
> WHERE ss_customer_sk IS NOT NULL
> AND ss_sold_date_sk IS NOT NULL) t11
>INNER JOIN
> (SELECT d_date_sk,
> CAST(2002 AS INTEGER) d_year
> FROM default.date_dim
> WHERE d_year = 2002
> AND d_date_sk IS NOT NULL) t13 ON
> t11.ss_sold_date_sk = t13.d_date_sk) ON t9.c_customer_sk = t11.ss_customer_sk
>GROUP BY t9.c_customer_id,
> t9.c_first_name,
> t9.c_last_name,
> t9.c_preferred_cust_flag,
>
[jira] [Commented] (HIVE-20163) Simplify StringSubstrColStart Initialization
[
https://issues.apache.org/jira/browse/HIVE-20163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544078#comment-16544078
]
Hive QA commented on HIVE-20163:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
52s{color} | {color:blue} ql in master has 2291 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} ql: The patch generated 0 new + 0 unchanged - 4
fixed = 0 total (was 4) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12602/dev-support/hive-personality.sh
|
| git revision | master / ab9e954 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12602/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Simplify StringSubstrColStart Initialization
>
>
> Key: HIVE-20163
> URL: https://issues.apache.org/jira/browse/HIVE-20163
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20163.1.patch, HIVE-20163.2.patch
>
>
> * Remove code
> * Remove exception handling
> * Remove {{printStackTrace}} call
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20006) Make materializations invalidation cache work with multiple active remote metastores
[ https://issues.apache.org/jira/browse/HIVE-20006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20006: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Patch had already been reviewed in HIVE-19027. Pushed to master > Make materializations invalidation cache work with multiple active remote > metastores > > > Key: HIVE-20006 > URL: https://issues.apache.org/jira/browse/HIVE-20006 > Project: Hive > Issue Type: Improvement > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Critical > Fix For: 4.0.0 > > Attachments: HIVE-19027.01.patch, HIVE-19027.02.patch, > HIVE-19027.03.patch, HIVE-19027.04.patch, HIVE-20006.01.patch, > HIVE-20006.02.patch, HIVE-20006.03.patch, HIVE-20006.04.patch, > HIVE-20006.05.patch, HIVE-20006.06.patch, HIVE-20006.07.patch, > HIVE-20006.patch > > > The main points: > - Only MVs that use transactional tables and are stored in transactional > tables can have a time window value of 0. Those are the only MVs that can be > guaranteed to not be outdated when a query is executed. > - For MVs that +cannot be outdated+, comparison is based on valid write id > lists. > - For MVs that +can be outdated+: > ** The window for valid outdated MVs can be specified in intervals of 1 > minute. > ** A materialized view is outdated if it was built before that time window > and any source table has been modified since. > A time window of -1 means to always use the materialized view for rewriting > without any checks concerning its validity. If a materialized view uses an > external table, the only way to trigger the rewriting would be to set the > property to -1, since currently we do not capture for validation purposes > whether the external source tables have been modified since the MV was > created or not. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure
[ https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16544069#comment-16544069 ] Daniel Dai commented on HIVE-19166: --- There is real issue in information schema when there's no restriction (security off). Patch also include changes in Vineet's last patch (removing "select sequence_name from sequence_table order by sequence_name limit 5 "). > TestMiniLlapLocalCliDriver sysdb failure > > > Key: HIVE-19166 > URL: https://issues.apache.org/jira/browse/HIVE-19166 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19166.04.patch, HIVE-19166.05.patch, > HIVE-19166.06.patch, HIVE-19166.09.patch, HIVE-19166.1.patch, > HIVE-19166.10.patch, HIVE-19166.11.patch, HIVE-19166.12.patch, > HIVE-19166.2.patch, HIVE-19166.3.patch > > > Broken by HIVE-18715 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure
[ https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-19166: -- Attachment: HIVE-19166.12.patch > TestMiniLlapLocalCliDriver sysdb failure > > > Key: HIVE-19166 > URL: https://issues.apache.org/jira/browse/HIVE-19166 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19166.04.patch, HIVE-19166.05.patch, > HIVE-19166.06.patch, HIVE-19166.09.patch, HIVE-19166.1.patch, > HIVE-19166.10.patch, HIVE-19166.11.patch, HIVE-19166.12.patch, > HIVE-19166.2.patch, HIVE-19166.3.patch > > > Broken by HIVE-18715 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure
[ https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-19166: -- Attachment: (was: HIVE-19166.12.patch) > TestMiniLlapLocalCliDriver sysdb failure > > > Key: HIVE-19166 > URL: https://issues.apache.org/jira/browse/HIVE-19166 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-19166.04.patch, HIVE-19166.05.patch, > HIVE-19166.06.patch, HIVE-19166.09.patch, HIVE-19166.1.patch, > HIVE-19166.10.patch, HIVE-19166.11.patch, HIVE-19166.12.patch, > HIVE-19166.2.patch, HIVE-19166.3.patch > > > Broken by HIVE-18715 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
