[jira] [Commented] (HIVE-20292) Bad join ordering in tpcds query93 with primary constraint defined
[
https://issues.apache.org/jira/browse/HIVE-20292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569762#comment-16569762
]
Hive QA commented on HIVE-20292:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
53s{color} | {color:blue} ql in master has 2301 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 1 new + 176 unchanged - 0
fixed = 177 total (was 176) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13054/dev-support/hive-personality.sh
|
| git revision | master / 27bdbda |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13054/yetus/diff-checkstyle-ql.txt
|
| modules | C: itests ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13054/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Bad join ordering in tpcds query93 with primary constraint defined
> --
>
> Key: HIVE-20292
> URL: https://issues.apache.org/jira/browse/HIVE-20292
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20292.1.patch, HIVE-20292.2.patch,
> HIVE-20292.3.patch, HIVE-20292.4.patch, HIVE-20292.5.patch,
> HIVE-20292.6.patch, HIVE-20292.7.patch
>
>
> Query 93 has join (including outer) b/w store_sales, store_return and reason.
> Without constraints store_return is joined with reason and then with
> store_sales.
> But if a primary key is added on store_return (alter table store_returns add
> constraint tpcds_pk_sr primary key (sr_item_sk, sr_ticket_number) disable
> novalidate rely) join order becomes ((store_sales, store_return), reason)
> which is very inefficient.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19097) related equals and in operators may cause inaccurate stats estimations
[
https://issues.apache.org/jira/browse/HIVE-19097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-19097:
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Zoltan!
> related equals and in operators may cause inaccurate stats estimations
> --
>
> Key: HIVE-19097
> URL: https://issues.apache.org/jira/browse/HIVE-19097
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19097.01.patch, HIVE-19097.02.patch,
> HIVE-19097.03.patch, HIVE-19097.04.patch, HIVE-19097.05.patch,
> HIVE-19097.06.patch, HIVE-19097.06wip01.patch, HIVE-19097.06wip02.patch,
> HIVE-19097.07.patch, HIVE-19097.08.patch, HIVE-19097.08.patch,
> HIVE-19097.09.patch, HIVE-19097.10.patch, HIVE-19097.11.patch,
> HIVE-19097.12.patch, HIVE-19097.13.patch, HIVE-19097.partial.patch
>
>
> tpcds#74 is optimized in a way that for date_dim the condition contains IN
> and = for the same column
> {code:java}
> | Map Operator Tree: |
> | TableScan |
> | alias: date_dim |
> | filterExpr: (((d_year) IN (2001, 2002) and (d_year =
> 2002) and d_date_sk is not null) or ((d_year) IN (2001, 2002) and (d_year =
> 2001) and d_date_sk is not null)) (type: boolean) |
> | Statistics: Num rows: 73049 Data size: 876588 Basic
> stats: COMPLETE Column stats: COMPLETE |
> | Filter Operator |
> | predicate: ((d_year) IN (2001, 2002) and (d_year =
> 2002) and d_date_sk is not null) (type: boolean) |
> | Statistics: Num rows: 4 Data size: 48 Basic stats:
> COMPLETE Column stats: COMPLETE |
> {code}
> the "real" row count will be 365
> for separate {{IN}} and {{=}} the estimation is very good; but if both are
> present it becomes (very) underestimated.
> {code:java}
> set hive.query.results.cache.enabled=false;
> drop table if exists t1;
> drop table if exists t8;
> create table t1 (a integer,b integer);
> create table t8 like t1;
> insert into t1 values (1,1),(2,2),(3,3),(4,4),(5,5);
> insert into t8
> select * from t1 union all select * from t1 union all select * from t1 union
> all select * from t1 union all
> select * from t1 union all select * from t1 union all select * from t1 union
> all select * from t1
> ;
> analyze table t1 compute statistics for columns;
> analyze table t8 compute statistics for columns;
> explain analyze select sum(a) from t8 where b in (2,3) group by b;
> explain analyze select sum(a) from t8 where b=2 group by b;
> explain analyze select sum(a) from t1 where b in (2,3) and b=2 group by b;
> explain analyze select sum(a) from t8 where b in (2,3) and b=2 group by b;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-7248) UNION ALL in hive returns incorrect results on Hbase backed table
[
https://issues.apache.org/jira/browse/HIVE-7248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569758#comment-16569758
]
Naveen Gangam commented on HIVE-7248:
-
[~navis] This still seems to be an issue
(https://issues.apache.org/jira/browse/HIVE-20206) . Was there a reason this
was never committed to the source? if there isnt, should we revive this issue
and rebase the patch? Thanks
> UNION ALL in hive returns incorrect results on Hbase backed table
> -
>
> Key: HIVE-7248
> URL: https://issues.apache.org/jira/browse/HIVE-7248
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 0.12.0, 0.13.0, 0.13.1
>Reporter: Mala Chikka Kempanna
>Assignee: Navis
>Priority: Major
> Attachments: HIVE-7248.1.patch.txt, HIVE-7248.2.patch.txt,
> HIVE-7248.3.patch.txt
>
>
> The issue can be recreated with following steps
> 1) In hbase
> create 'TABLE_EMP','default'
> 2) On hive
> sudo -u hive hive
> CREATE EXTERNAL TABLE TABLE_EMP(FIRST_NAME string,LAST_NAME
> string,CDS_UPDATED_DATE string,CDS_PK string) STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH
> SERDEPROPERTIES("hbase.columns.mapping" =
> "default:FIRST_NAME,default:LAST_NAME,default:CDS_UPDATED_DATE,:key",
> "hbase.scan.cache" = "500", "hbase.scan.cacheblocks" = "false" )
> TBLPROPERTIES("hbase.table.name" =
> "TABLE_EMP",'serialization.null.format'='');
> 3) On hbase insert the following data
> put 'TABLE_EMP', '1', 'default:FIRST_NAME', 'Srini'
> put 'TABLE_EMP', '1', 'default:LAST_NAME', 'P'
> put 'TABLE_EMP', '1', 'default:CDS_UPDATED_DATE', '2014-06-16 00:00:00'
> put 'TABLE_EMP', '2', 'default:FIRST_NAME', 'Aravind'
> put 'TABLE_EMP', '2', 'default:LAST_NAME', 'K'
> put 'TABLE_EMP', '2', 'default:CDS_UPDATED_DATE', '2014-06-16 00:00:00'
> 4) On hive execute the following query
> hive
> SELECT *
> FROM (
> SELECT CDS_PK
> FROM TABLE_EMP
> WHERE
> CDS_PK >= '0'
> AND CDS_PK <= '9'
> AND CDS_UPDATED_DATE IS NOT NULL
> UNION ALL SELECT CDS_PK
> FROM TABLE_EMP
> WHERE
> CDS_PK >= 'a'
> AND CDS_PK <= 'z'
> AND CDS_UPDATED_DATE IS NOT NULL
> )t ;
> 5) Output of the query
> 1
> 1
> 2
> 2
> 6) Output of just
> SELECT CDS_PK
> FROM TABLE_EMP
> WHERE
> CDS_PK >= '0'
> AND CDS_PK <= '9'
> AND CDS_UPDATED_DATE IS NOT NULL
> is
> 1
> 2
> 7) Output of just
> SELECT CDS_PK
> FROM TABLE_EMP
> WHERE
> CDS_PK >= 'a'
> AND CDS_PK <= 'z'
> AND CDS_UPDATED_DATE IS NOT NULL
> Empty
> 8) UNION is used to combine the result from multiple SELECT statements into a
> single result set. Hive currently only supports UNION ALL (bag union), in
> which duplicates are not eliminated
> Accordingly above query should return output
> 1
> 2
> instead it is giving wrong output
> 1
> 1
> 2
> 2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20277) Vectorization: Case expressions that return BOOLEAN are not supported for FILTER
[
https://issues.apache.org/jira/browse/HIVE-20277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569753#comment-16569753
]
Matt McCline commented on HIVE-20277:
-
A successful test run.
> Vectorization: Case expressions that return BOOLEAN are not supported for
> FILTER
>
>
> Key: HIVE-20277
> URL: https://issues.apache.org/jira/browse/HIVE-20277
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Matt McCline
>Priority: Major
> Attachments: HIVE-20277.02.patch, HIVE-20277.03.patch,
> HIVE-20277.04.patch, HIVE-20277.WIP.01.patch
>
>
> In cases like Query89, the vertex with the filter is not vectorized.
> {code}
>Filter Operator
> predicate: CASE WHEN ((avg_window_0 <> 0.0D)) THEN
> (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (null) END
> (type: boolean)
> {code}
> {code}
> Reducer 3
> Execution mode: llap
> Reduce Vectorization:
> enabled: true
> enableConditionsMet: hive.vectorized.execution.reduce.enabled
> IS true, hive.execution.engine tez IN [tez, spark] IS true
> notVectorizedReason: FILTER operator: Unexpected hive type
> name void
> vectorized: false
> {code}
> The query specifically has
> {code}
> where case when (avg_monthly_sales <> 0) then (abs(sum_sales -
> avg_monthly_sales) / avg_monthly_sales) else null end > 0.1
> {code}
> while rewriting it to
> {code}
> where case when (avg_monthly_sales <> 0) then (abs(sum_sales -
> avg_monthly_sales) / avg_monthly_sales) > 0.1 else false end
> {code}
> does vectorize into
> {code}
> Filter Operator
> Filter Vectorization:
> className: VectorFilterOperator
> native: true
> predicateExpression: SelectColumnIsTrue(col
> 12:boolean)(children: VectorUDFAdaptor(CASE WHEN ((avg_window_0 <> 0.0D))
> THEN (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (false)
> END)(children: DoubleColNotEqualDoubleScalar(col 7:double, val 0.0) ->
> 8:boolean, DoubleColGreaterDoubleScalar(col 9:double, val 0.1)(children:
> DoubleColDivideDoubleColumn(col 10:double, col 7:double)(children:
> FuncAbsDoubleToDouble(col 9:double)(children:
> DoubleColSubtractDoubleColumn(col 6:double, col 7:double) -> 9:double) ->
> 10:double) -> 9:double) -> 11:boolean) -> 12:boolean)
> predicate: CASE WHEN ((avg_window_0 <> 0.0D)) THEN
> (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (false) END
> (type: boolean)
> Statistics: Num rows: 11 Data size: 5291 Basic stats:
> COMPLETE Column stats: COMPLETE
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20316) Skip external table file listing for create table event.
[ https://issues.apache.org/jira/browse/HIVE-20316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan reassigned HIVE-20316: --- > Skip external table file listing for create table event. > > > Key: HIVE-20316 > URL: https://issues.apache.org/jira/browse/HIVE-20316 > Project: Hive > Issue Type: Bug > Components: HCatalog, repl >Affects Versions: 3.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, replication > > We are currently skipping external table replication. We shall also skip > listing all the files in create table event generation for external tables > (when external table replication is disabled). > External tables might have very large number of files, so it would take long > time to list them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20277) Vectorization: Case expressions that return BOOLEAN are not supported for FILTER
[
https://issues.apache.org/jira/browse/HIVE-20277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569744#comment-16569744
]
Hive QA commented on HIVE-20277:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12934414/HIVE-20277.04.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14860 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13053/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13053/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13053/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12934414 - PreCommit-HIVE-Build
> Vectorization: Case expressions that return BOOLEAN are not supported for
> FILTER
>
>
> Key: HIVE-20277
> URL: https://issues.apache.org/jira/browse/HIVE-20277
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Matt McCline
>Priority: Major
> Attachments: HIVE-20277.02.patch, HIVE-20277.03.patch,
> HIVE-20277.04.patch, HIVE-20277.WIP.01.patch
>
>
> In cases like Query89, the vertex with the filter is not vectorized.
> {code}
>Filter Operator
> predicate: CASE WHEN ((avg_window_0 <> 0.0D)) THEN
> (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (null) END
> (type: boolean)
> {code}
> {code}
> Reducer 3
> Execution mode: llap
> Reduce Vectorization:
> enabled: true
> enableConditionsMet: hive.vectorized.execution.reduce.enabled
> IS true, hive.execution.engine tez IN [tez, spark] IS true
> notVectorizedReason: FILTER operator: Unexpected hive type
> name void
> vectorized: false
> {code}
> The query specifically has
> {code}
> where case when (avg_monthly_sales <> 0) then (abs(sum_sales -
> avg_monthly_sales) / avg_monthly_sales) else null end > 0.1
> {code}
> while rewriting it to
> {code}
> where case when (avg_monthly_sales <> 0) then (abs(sum_sales -
> avg_monthly_sales) / avg_monthly_sales) > 0.1 else false end
> {code}
> does vectorize into
> {code}
> Filter Operator
> Filter Vectorization:
> className: VectorFilterOperator
> native: true
> predicateExpression: SelectColumnIsTrue(col
> 12:boolean)(children: VectorUDFAdaptor(CASE WHEN ((avg_window_0 <> 0.0D))
> THEN (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (false)
> END)(children: DoubleColNotEqualDoubleScalar(col 7:double, val 0.0) ->
> 8:boolean, DoubleColGreaterDoubleScalar(col 9:double, val 0.1)(children:
> DoubleColDivideDoubleColumn(col 10:double, col 7:double)(children:
> FuncAbsDoubleToDouble(col 9:double)(children:
> DoubleColSubtractDoubleColumn(col 6:double, col 7:double) -> 9:double) ->
> 10:double) -> 9:double) -> 11:boolean) -> 12:boolean)
> predicate: CASE WHEN ((avg_window_0 <> 0.0D)) THEN
> (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (false) END
> (type: boolean)
> Statistics: Num rows: 11 Data size: 5291 Basic stats:
> COMPLETE Column stats: COMPLETE
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20277) Vectorization: Case expressions that return BOOLEAN are not supported for FILTER
[
https://issues.apache.org/jira/browse/HIVE-20277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569725#comment-16569725
]
Hive QA commented on HIVE-20277:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 2301 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 7 new + 382 unchanged - 3
fixed = 389 total (was 385) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13053/dev-support/hive-personality.sh
|
| git revision | master / 27bdbda |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13053/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13053/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Case expressions that return BOOLEAN are not supported for
> FILTER
>
>
> Key: HIVE-20277
> URL: https://issues.apache.org/jira/browse/HIVE-20277
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Matt McCline
>Priority: Major
> Attachments: HIVE-20277.02.patch, HIVE-20277.03.patch,
> HIVE-20277.04.patch, HIVE-20277.WIP.01.patch
>
>
> In cases like Query89, the vertex with the filter is not vectorized.
> {code}
>Filter Operator
> predicate: CASE WHEN ((avg_window_0 <> 0.0D)) THEN
> (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (null) END
> (type: boolean)
> {code}
> {code}
> Reducer 3
> Execution mode: llap
> Reduce Vectorization:
> enabled: true
> enableConditionsMet: hive.vectorized.execution.reduce.enabled
> IS true, hive.execution.engine tez IN [tez, spark] IS true
> notVectorizedReason: FILTER operator: Unexpected hive type
> name void
> vectorized: false
> {code}
> The query specifically has
> {code}
> where case when (avg_monthly_sales <> 0) then (abs(sum_sales -
> avg_monthly_sales) / avg_monthly_sales) else null end > 0.1
> {code}
> while rewriting it to
> {code}
> where case when (avg_monthly_sales <> 0)
[jira] [Commented] (HIVE-20300) VectorFileSinkArrowOperator
[
https://issues.apache.org/jira/browse/HIVE-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569711#comment-16569711
]
Hive QA commented on HIVE-20300:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12934413/HIVE-20300.3.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 14866 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz]
(batchId=193)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_joins]
(batchId=193)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_masking]
(batchId=193)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1]
(batchId=193)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13052/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13052/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13052/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12934413 - PreCommit-HIVE-Build
> VectorFileSinkArrowOperator
> ---
>
> Key: HIVE-20300
> URL: https://issues.apache.org/jira/browse/HIVE-20300
> Project: Hive
> Issue Type: Improvement
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-20300.1.patch, HIVE-20300.2.patch,
> HIVE-20300.3.patch
>
>
> Bypass the row-mode FileSinkOperator for pushing Arrow format to the
> LlapOutputFormatService.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20300) VectorFileSinkArrowOperator
[
https://issues.apache.org/jira/browse/HIVE-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569691#comment-16569691
]
Hive QA commented on HIVE-20300:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 2301 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} itests/hive-unit: The patch generated 4 new + 58
unchanged - 0 fixed = 62 total (was 58) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 156 new + 449 unchanged - 238
fixed = 605 total (was 687) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
6s{color} | {color:red} ql generated 1 new + 2301 unchanged - 0 fixed = 2302
total (was 2301) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 59s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Class
org.apache.hadoop.hive.ql.exec.vector.filesink.VectorFileSinkArrowOperator
defines non-transient non-serializable instance field vContext In
VectorFileSinkArrowOperator.java:instance field vContext In
VectorFileSinkArrowOperator.java |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13052/dev-support/hive-personality.sh
|
| git revision | master / 27bdbda |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13052/yetus/diff-checkstyle-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13052/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13052/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13052/yetus/new-findbugs-ql.html
|
| modules | C: common itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13052/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This
[jira] [Updated] (HIVE-20292) Bad join ordering in tpcds query93 with primary constraint defined
[ https://issues.apache.org/jira/browse/HIVE-20292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20292: --- Status: Patch Available (was: Open) > Bad join ordering in tpcds query93 with primary constraint defined > -- > > Key: HIVE-20292 > URL: https://issues.apache.org/jira/browse/HIVE-20292 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20292.1.patch, HIVE-20292.2.patch, > HIVE-20292.3.patch, HIVE-20292.4.patch, HIVE-20292.5.patch, > HIVE-20292.6.patch, HIVE-20292.7.patch > > > Query 93 has join (including outer) b/w store_sales, store_return and reason. > Without constraints store_return is joined with reason and then with > store_sales. > But if a primary key is added on store_return (alter table store_returns add > constraint tpcds_pk_sr primary key (sr_item_sk, sr_ticket_number) disable > novalidate rely) join order becomes ((store_sales, store_return), reason) > which is very inefficient. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20292) Bad join ordering in tpcds query93 with primary constraint defined
[ https://issues.apache.org/jira/browse/HIVE-20292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20292: --- Status: Open (was: Patch Available) > Bad join ordering in tpcds query93 with primary constraint defined > -- > > Key: HIVE-20292 > URL: https://issues.apache.org/jira/browse/HIVE-20292 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20292.1.patch, HIVE-20292.2.patch, > HIVE-20292.3.patch, HIVE-20292.4.patch, HIVE-20292.5.patch, > HIVE-20292.6.patch, HIVE-20292.7.patch > > > Query 93 has join (including outer) b/w store_sales, store_return and reason. > Without constraints store_return is joined with reason and then with > store_sales. > But if a primary key is added on store_return (alter table store_returns add > constraint tpcds_pk_sr primary key (sr_item_sk, sr_ticket_number) disable > novalidate rely) join order becomes ((store_sales, store_return), reason) > which is very inefficient. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20292) Bad join ordering in tpcds query93 with primary constraint defined
[ https://issues.apache.org/jira/browse/HIVE-20292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20292: --- Attachment: HIVE-20292.7.patch > Bad join ordering in tpcds query93 with primary constraint defined > -- > > Key: HIVE-20292 > URL: https://issues.apache.org/jira/browse/HIVE-20292 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20292.1.patch, HIVE-20292.2.patch, > HIVE-20292.3.patch, HIVE-20292.4.patch, HIVE-20292.5.patch, > HIVE-20292.6.patch, HIVE-20292.7.patch > > > Query 93 has join (including outer) b/w store_sales, store_return and reason. > Without constraints store_return is joined with reason and then with > store_sales. > But if a primary key is added on store_return (alter table store_returns add > constraint tpcds_pk_sr primary key (sr_item_sk, sr_ticket_number) disable > novalidate rely) join order becomes ((store_sales, store_return), reason) > which is very inefficient. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20292) Bad join ordering in tpcds query93 with primary constraint defined
[ https://issues.apache.org/jira/browse/HIVE-20292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20292: --- Attachment: (was: HIVE-20292.7.patch) > Bad join ordering in tpcds query93 with primary constraint defined > -- > > Key: HIVE-20292 > URL: https://issues.apache.org/jira/browse/HIVE-20292 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20292.1.patch, HIVE-20292.2.patch, > HIVE-20292.3.patch, HIVE-20292.4.patch, HIVE-20292.5.patch, HIVE-20292.6.patch > > > Query 93 has join (including outer) b/w store_sales, store_return and reason. > Without constraints store_return is joined with reason and then with > store_sales. > But if a primary key is added on store_return (alter table store_returns add > constraint tpcds_pk_sr primary key (sr_item_sk, sr_ticket_number) disable > novalidate rely) join order becomes ((store_sales, store_return), reason) > which is very inefficient. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20292) Bad join ordering in tpcds query93 with primary constraint defined
[ https://issues.apache.org/jira/browse/HIVE-20292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20292: --- Attachment: HIVE-20292.7.patch > Bad join ordering in tpcds query93 with primary constraint defined > -- > > Key: HIVE-20292 > URL: https://issues.apache.org/jira/browse/HIVE-20292 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20292.1.patch, HIVE-20292.2.patch, > HIVE-20292.3.patch, HIVE-20292.4.patch, HIVE-20292.5.patch, > HIVE-20292.6.patch, HIVE-20292.7.patch > > > Query 93 has join (including outer) b/w store_sales, store_return and reason. > Without constraints store_return is joined with reason and then with > store_sales. > But if a primary key is added on store_return (alter table store_returns add > constraint tpcds_pk_sr primary key (sr_item_sk, sr_ticket_number) disable > novalidate rely) join order becomes ((store_sales, store_return), reason) > which is very inefficient. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20312) Allow arrow clients to use their own BufferAllocator with LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-20312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569679#comment-16569679
]
Hive QA commented on HIVE-20312:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12934412/HIVE-20312.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 14860 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.addNoSuchTable[Remote]
(batchId=218)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.createGetDrop[Remote]
(batchId=218)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.createTableWithConstraintsPkInOtherCatalog[Remote]
(batchId=218)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.createTableWithConstraintsPk[Remote]
(batchId=218)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.doubleAddUniqueConstraint[Remote]
(batchId=218)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.getNoSuchCatalog[Remote]
(batchId=218)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.getNoSuchDb[Remote]
(batchId=218)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.getNoSuchTable[Remote]
(batchId=218)
org.apache.hadoop.hive.metastore.client.TestUniqueConstraint.inOtherCatalog[Remote]
(batchId=218)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13051/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13051/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13051/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12934412 - PreCommit-HIVE-Build
> Allow arrow clients to use their own BufferAllocator with
> LlapOutputFormatService
> -
>
> Key: HIVE-20312
> URL: https://issues.apache.org/jira/browse/HIVE-20312
> Project: Hive
> Issue Type: Improvement
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-20312.1.patch
>
>
> Clients should be able to provide their own BufferAllocator to
> LlapBaseInputFormat if allocator operations depend on client-side logic. For
> example, clients may want to manage the allocator hierarchy per client-side
> task, thread, etc..
> Currently the client is forced to use one global RootAllocator per process.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20162) Do Not Print StackTraces to STDERR in AbstractJoinTaskDispatcher
[
https://issues.apache.org/jira/browse/HIVE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam updated HIVE-20162:
-
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
> Do Not Print StackTraces to STDERR in AbstractJoinTaskDispatcher
>
>
> Key: HIVE-20162
> URL: https://issues.apache.org/jira/browse/HIVE-20162
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Anurag Mantripragada
>Priority: Minor
> Labels: newbie, noob
> Fix For: 4.0.0
>
> Attachments: HIVE-20162.1.patch
>
>
> https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/AbstractJoinTaskDispatcher.java
> {code}
> } catch (Exception e) {
> e.printStackTrace();
> throw new SemanticException("Generate Map Join Task Error: " +
> e.getMessage());
> }
> {code}
> Remove the call to {{printStackTrace}} and just throw the error. If the
> stack trace really is needed (doubtful), then pass it to the
> {{SemanticException}} constructor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20162) Do Not Print StackTraces to STDERR in AbstractJoinTaskDispatcher
[
https://issues.apache.org/jira/browse/HIVE-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569672#comment-16569672
]
Naveen Gangam commented on HIVE-20162:
--
Fix has been committed to master for 4.0.0 release. Thank you for your
contribution [~anuragmantri]
> Do Not Print StackTraces to STDERR in AbstractJoinTaskDispatcher
>
>
> Key: HIVE-20162
> URL: https://issues.apache.org/jira/browse/HIVE-20162
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Anurag Mantripragada
>Priority: Minor
> Labels: newbie, noob
> Fix For: 4.0.0
>
> Attachments: HIVE-20162.1.patch
>
>
> https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/AbstractJoinTaskDispatcher.java
> {code}
> } catch (Exception e) {
> e.printStackTrace();
> throw new SemanticException("Generate Map Join Task Error: " +
> e.getMessage());
> }
> {code}
> Remove the call to {{printStackTrace}} and just throw the error. If the
> stack trace really is needed (doubtful), then pass it to the
> {{SemanticException}} constructor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20277) Vectorization: Case expressions that return BOOLEAN are not supported for FILTER
[
https://issues.apache.org/jira/browse/HIVE-20277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-20277:
Status: Patch Available (was: In Progress)
Again.
> Vectorization: Case expressions that return BOOLEAN are not supported for
> FILTER
>
>
> Key: HIVE-20277
> URL: https://issues.apache.org/jira/browse/HIVE-20277
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Matt McCline
>Priority: Major
> Attachments: HIVE-20277.02.patch, HIVE-20277.03.patch,
> HIVE-20277.04.patch, HIVE-20277.WIP.01.patch
>
>
> In cases like Query89, the vertex with the filter is not vectorized.
> {code}
>Filter Operator
> predicate: CASE WHEN ((avg_window_0 <> 0.0D)) THEN
> (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (null) END
> (type: boolean)
> {code}
> {code}
> Reducer 3
> Execution mode: llap
> Reduce Vectorization:
> enabled: true
> enableConditionsMet: hive.vectorized.execution.reduce.enabled
> IS true, hive.execution.engine tez IN [tez, spark] IS true
> notVectorizedReason: FILTER operator: Unexpected hive type
> name void
> vectorized: false
> {code}
> The query specifically has
> {code}
> where case when (avg_monthly_sales <> 0) then (abs(sum_sales -
> avg_monthly_sales) / avg_monthly_sales) else null end > 0.1
> {code}
> while rewriting it to
> {code}
> where case when (avg_monthly_sales <> 0) then (abs(sum_sales -
> avg_monthly_sales) / avg_monthly_sales) > 0.1 else false end
> {code}
> does vectorize into
> {code}
> Filter Operator
> Filter Vectorization:
> className: VectorFilterOperator
> native: true
> predicateExpression: SelectColumnIsTrue(col
> 12:boolean)(children: VectorUDFAdaptor(CASE WHEN ((avg_window_0 <> 0.0D))
> THEN (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (false)
> END)(children: DoubleColNotEqualDoubleScalar(col 7:double, val 0.0) ->
> 8:boolean, DoubleColGreaterDoubleScalar(col 9:double, val 0.1)(children:
> DoubleColDivideDoubleColumn(col 10:double, col 7:double)(children:
> FuncAbsDoubleToDouble(col 9:double)(children:
> DoubleColSubtractDoubleColumn(col 6:double, col 7:double) -> 9:double) ->
> 10:double) -> 9:double) -> 11:boolean) -> 12:boolean)
> predicate: CASE WHEN ((avg_window_0 <> 0.0D)) THEN
> (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (false) END
> (type: boolean)
> Statistics: Num rows: 11 Data size: 5291 Basic stats:
> COMPLETE Column stats: COMPLETE
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20277) Vectorization: Case expressions that return BOOLEAN are not supported for FILTER
[
https://issues.apache.org/jira/browse/HIVE-20277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-20277:
Status: In Progress (was: Patch Available)
> Vectorization: Case expressions that return BOOLEAN are not supported for
> FILTER
>
>
> Key: HIVE-20277
> URL: https://issues.apache.org/jira/browse/HIVE-20277
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Matt McCline
>Priority: Major
> Attachments: HIVE-20277.02.patch, HIVE-20277.03.patch,
> HIVE-20277.04.patch, HIVE-20277.WIP.01.patch
>
>
> In cases like Query89, the vertex with the filter is not vectorized.
> {code}
>Filter Operator
> predicate: CASE WHEN ((avg_window_0 <> 0.0D)) THEN
> (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (null) END
> (type: boolean)
> {code}
> {code}
> Reducer 3
> Execution mode: llap
> Reduce Vectorization:
> enabled: true
> enableConditionsMet: hive.vectorized.execution.reduce.enabled
> IS true, hive.execution.engine tez IN [tez, spark] IS true
> notVectorizedReason: FILTER operator: Unexpected hive type
> name void
> vectorized: false
> {code}
> The query specifically has
> {code}
> where case when (avg_monthly_sales <> 0) then (abs(sum_sales -
> avg_monthly_sales) / avg_monthly_sales) else null end > 0.1
> {code}
> while rewriting it to
> {code}
> where case when (avg_monthly_sales <> 0) then (abs(sum_sales -
> avg_monthly_sales) / avg_monthly_sales) > 0.1 else false end
> {code}
> does vectorize into
> {code}
> Filter Operator
> Filter Vectorization:
> className: VectorFilterOperator
> native: true
> predicateExpression: SelectColumnIsTrue(col
> 12:boolean)(children: VectorUDFAdaptor(CASE WHEN ((avg_window_0 <> 0.0D))
> THEN (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (false)
> END)(children: DoubleColNotEqualDoubleScalar(col 7:double, val 0.0) ->
> 8:boolean, DoubleColGreaterDoubleScalar(col 9:double, val 0.1)(children:
> DoubleColDivideDoubleColumn(col 10:double, col 7:double)(children:
> FuncAbsDoubleToDouble(col 9:double)(children:
> DoubleColSubtractDoubleColumn(col 6:double, col 7:double) -> 9:double) ->
> 10:double) -> 9:double) -> 11:boolean) -> 12:boolean)
> predicate: CASE WHEN ((avg_window_0 <> 0.0D)) THEN
> (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (false) END
> (type: boolean)
> Statistics: Num rows: 11 Data size: 5291 Basic stats:
> COMPLETE Column stats: COMPLETE
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20277) Vectorization: Case expressions that return BOOLEAN are not supported for FILTER
[
https://issues.apache.org/jira/browse/HIVE-20277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-20277:
Attachment: HIVE-20277.04.patch
> Vectorization: Case expressions that return BOOLEAN are not supported for
> FILTER
>
>
> Key: HIVE-20277
> URL: https://issues.apache.org/jira/browse/HIVE-20277
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Matt McCline
>Priority: Major
> Attachments: HIVE-20277.02.patch, HIVE-20277.03.patch,
> HIVE-20277.04.patch, HIVE-20277.WIP.01.patch
>
>
> In cases like Query89, the vertex with the filter is not vectorized.
> {code}
>Filter Operator
> predicate: CASE WHEN ((avg_window_0 <> 0.0D)) THEN
> (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (null) END
> (type: boolean)
> {code}
> {code}
> Reducer 3
> Execution mode: llap
> Reduce Vectorization:
> enabled: true
> enableConditionsMet: hive.vectorized.execution.reduce.enabled
> IS true, hive.execution.engine tez IN [tez, spark] IS true
> notVectorizedReason: FILTER operator: Unexpected hive type
> name void
> vectorized: false
> {code}
> The query specifically has
> {code}
> where case when (avg_monthly_sales <> 0) then (abs(sum_sales -
> avg_monthly_sales) / avg_monthly_sales) else null end > 0.1
> {code}
> while rewriting it to
> {code}
> where case when (avg_monthly_sales <> 0) then (abs(sum_sales -
> avg_monthly_sales) / avg_monthly_sales) > 0.1 else false end
> {code}
> does vectorize into
> {code}
> Filter Operator
> Filter Vectorization:
> className: VectorFilterOperator
> native: true
> predicateExpression: SelectColumnIsTrue(col
> 12:boolean)(children: VectorUDFAdaptor(CASE WHEN ((avg_window_0 <> 0.0D))
> THEN (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (false)
> END)(children: DoubleColNotEqualDoubleScalar(col 7:double, val 0.0) ->
> 8:boolean, DoubleColGreaterDoubleScalar(col 9:double, val 0.1)(children:
> DoubleColDivideDoubleColumn(col 10:double, col 7:double)(children:
> FuncAbsDoubleToDouble(col 9:double)(children:
> DoubleColSubtractDoubleColumn(col 6:double, col 7:double) -> 9:double) ->
> 10:double) -> 9:double) -> 11:boolean) -> 12:boolean)
> predicate: CASE WHEN ((avg_window_0 <> 0.0D)) THEN
> (((abs((_col6 - avg_window_0)) / avg_window_0) > 0.1D)) ELSE (false) END
> (type: boolean)
> Statistics: Num rows: 11 Data size: 5291 Basic stats:
> COMPLETE Column stats: COMPLETE
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20312) Allow arrow clients to use their own BufferAllocator with LlapOutputFormatService
[
https://issues.apache.org/jira/browse/HIVE-20312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569648#comment-16569648
]
Hive QA commented on HIVE-20312:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} llap-ext-client: The patch generated 1 new + 42
unchanged - 0 fixed = 43 total (was 42) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13051/dev-support/hive-personality.sh
|
| git revision | master / c6275ee |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13051/yetus/diff-checkstyle-llap-ext-client.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13051/yetus/whitespace-eol.txt
|
| modules | C: llap-ext-client U: llap-ext-client |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13051/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Allow arrow clients to use their own BufferAllocator with
> LlapOutputFormatService
> -
>
> Key: HIVE-20312
> URL: https://issues.apache.org/jira/browse/HIVE-20312
> Project: Hive
> Issue Type: Improvement
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-20312.1.patch
>
>
> Clients should be able to provide their own BufferAllocator to
> LlapBaseInputFormat if allocator operations depend on client-side logic. For
> example, clients may want to manage the allocator hierarchy per client-side
> task, thread, etc..
> Currently the client is forced to use one global RootAllocator per process.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20314) Include partition pruning in materialized view rewriting
[ https://issues.apache.org/jira/browse/HIVE-20314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20314: --- Resolution: Fixed Fix Version/s: 3.2.0 4.0.0 Status: Resolved (was: Patch Available) Pushed to master, branch-3. Thanks for reviewing [~ashutoshc] > Include partition pruning in materialized view rewriting > > > Key: HIVE-20314 > URL: https://issues.apache.org/jira/browse/HIVE-20314 > Project: Hive > Issue Type: Improvement > Components: Materialized views >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20314.01.patch, HIVE-20314.patch > > > To be able to reduce the cost of the expression using the materialized view > when some of its partitions are pruned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[
https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569639#comment-16569639
]
Hive QA commented on HIVE-20315:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12934411/HIVE-20315.04.patch
{color:green}SUCCESS:{color} +1 due to 118 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14860 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13050/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13050/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13050/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12934411 - PreCommit-HIVE-Build
> Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary
> casts/conversions
> ---
>
> Key: HIVE-20315
> URL: https://issues.apache.org/jira/browse/HIVE-20315
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-20315.01.patch, HIVE-20315.02.patch,
> HIVE-20315.03.patch, HIVE-20315.04.patch
>
>
> Generate multi-byte Unicode characters in addition to regular single byte
> characters for random data.
> Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in
> vectorization without padding.
> Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL
> constants.
> Fix NULL / Wrong Results issues in VectorElt.
> Change performance Q files to generate non-user EXPLAIN with VECTORIZATION
> display so unnecesary CAST / DECIMAL_64 conversions are visible.
> Add test for vectorized STRUCT field reference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20300) VectorFileSinkArrowOperator
[ https://issues.apache.org/jira/browse/HIVE-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569631#comment-16569631 ] Eric Wohlstadter commented on HIVE-20300: - [~mmccline] [~teddy.choi] I'll make a followup ticket to use randomized schemas/data TestArrowColumnarBatchSerDe. I think that is orthogonal to this ticket, as these tests weren't randomized previously. > VectorFileSinkArrowOperator > --- > > Key: HIVE-20300 > URL: https://issues.apache.org/jira/browse/HIVE-20300 > Project: Hive > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-20300.1.patch, HIVE-20300.2.patch, > HIVE-20300.3.patch > > > Bypass the row-mode FileSinkOperator for pushing Arrow format to the > LlapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20314) Include partition pruning in materialized view rewriting
[ https://issues.apache.org/jira/browse/HIVE-20314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569629#comment-16569629 ] Ashutosh Chauhan commented on HIVE-20314: - +1 > Include partition pruning in materialized view rewriting > > > Key: HIVE-20314 > URL: https://issues.apache.org/jira/browse/HIVE-20314 > Project: Hive > Issue Type: Improvement > Components: Materialized views >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20314.01.patch, HIVE-20314.patch > > > To be able to reduce the cost of the expression using the materialized view > when some of its partitions are pruned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20300) VectorFileSinkArrowOperator
[ https://issues.apache.org/jira/browse/HIVE-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569628#comment-16569628 ] Eric Wohlstadter edited comment on HIVE-20300 at 8/6/18 1:14 AM: - [~jdere] [~mmccline] Updated patch based on comments: # Lazy initialize VectorFileSinkArrowOperator.recordWriter # Make VectorFileSinkArrowOperator fields transient unless they were created in the constructor was (Author: ewohlstadter): [~jdere] [~mmccline] Updated patch based on comments: # Lazy initialize VectorFileSinkArrowOperator.recordWriter # Make VectorFileSinkArrowOperator transient unless they were created in the constructor > VectorFileSinkArrowOperator > --- > > Key: HIVE-20300 > URL: https://issues.apache.org/jira/browse/HIVE-20300 > Project: Hive > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-20300.1.patch, HIVE-20300.2.patch, > HIVE-20300.3.patch > > > Bypass the row-mode FileSinkOperator for pushing Arrow format to the > LlapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20300) VectorFileSinkArrowOperator
[ https://issues.apache.org/jira/browse/HIVE-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569628#comment-16569628 ] Eric Wohlstadter commented on HIVE-20300: - [~jdere] [~mmccline] Updated patch based on comments: # Lazy initialize VectorFileSinkArrowOperator.recordWriter # Make VectorFileSinkArrowOperator transient unless they were created in the constructor > VectorFileSinkArrowOperator > --- > > Key: HIVE-20300 > URL: https://issues.apache.org/jira/browse/HIVE-20300 > Project: Hive > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-20300.1.patch, HIVE-20300.2.patch, > HIVE-20300.3.patch > > > Bypass the row-mode FileSinkOperator for pushing Arrow format to the > LlapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20300) VectorFileSinkArrowOperator
[ https://issues.apache.org/jira/browse/HIVE-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated HIVE-20300: Status: Patch Available (was: Open) > VectorFileSinkArrowOperator > --- > > Key: HIVE-20300 > URL: https://issues.apache.org/jira/browse/HIVE-20300 > Project: Hive > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-20300.1.patch, HIVE-20300.2.patch, > HIVE-20300.3.patch > > > Bypass the row-mode FileSinkOperator for pushing Arrow format to the > LlapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20300) VectorFileSinkArrowOperator
[ https://issues.apache.org/jira/browse/HIVE-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated HIVE-20300: Attachment: HIVE-20300.3.patch > VectorFileSinkArrowOperator > --- > > Key: HIVE-20300 > URL: https://issues.apache.org/jira/browse/HIVE-20300 > Project: Hive > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-20300.1.patch, HIVE-20300.2.patch, > HIVE-20300.3.patch > > > Bypass the row-mode FileSinkOperator for pushing Arrow format to the > LlapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20300) VectorFileSinkArrowOperator
[ https://issues.apache.org/jira/browse/HIVE-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated HIVE-20300: Status: Open (was: Patch Available) > VectorFileSinkArrowOperator > --- > > Key: HIVE-20300 > URL: https://issues.apache.org/jira/browse/HIVE-20300 > Project: Hive > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-20300.1.patch, HIVE-20300.2.patch, > HIVE-20300.3.patch > > > Bypass the row-mode FileSinkOperator for pushing Arrow format to the > LlapOutputFormatService. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[
https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569624#comment-16569624
]
Hive QA commented on HIVE-20315:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
43s{color} | {color:blue} serde in master has 195 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
24s{color} | {color:blue} ql in master has 2301 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} serde: The patch generated 2 new + 6 unchanged - 0
fixed = 8 total (was 6) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} root: The patch generated 2 new + 9 unchanged - 0
fixed = 11 total (was 9) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
3s{color} | {color:red} ql: The patch generated 74 new + 1611 unchanged - 79
fixed = 1685 total (was 1690) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
2s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13050/dev-support/hive-personality.sh
|
| git revision | master / df5caa0 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13050/yetus/diff-checkstyle-serde.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13050/yetus/diff-checkstyle-root.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13050/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13050/yetus/whitespace-eol.txt
|
| modules | C: serde . ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13050/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary
> casts/conversions
> ---
>
> Key: HIVE-20315
> URL: https://issues.apache.org/jira/browse/HIVE-20315
> Project: Hive
>
[jira] [Updated] (HIVE-20312) Allow arrow clients to use their own BufferAllocator with LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-20312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated HIVE-20312: Status: Patch Available (was: In Progress) > Allow arrow clients to use their own BufferAllocator with > LlapOutputFormatService > - > > Key: HIVE-20312 > URL: https://issues.apache.org/jira/browse/HIVE-20312 > Project: Hive > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-20312.1.patch > > > Clients should be able to provide their own BufferAllocator to > LlapBaseInputFormat if allocator operations depend on client-side logic. For > example, clients may want to manage the allocator hierarchy per client-side > task, thread, etc.. > Currently the client is forced to use one global RootAllocator per process. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20312) Allow arrow clients to use their own BufferAllocator with LlapOutputFormatService
[ https://issues.apache.org/jira/browse/HIVE-20312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated HIVE-20312: Attachment: HIVE-20312.1.patch > Allow arrow clients to use their own BufferAllocator with > LlapOutputFormatService > - > > Key: HIVE-20312 > URL: https://issues.apache.org/jira/browse/HIVE-20312 > Project: Hive > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: HIVE-20312.1.patch > > > Clients should be able to provide their own BufferAllocator to > LlapBaseInputFormat if allocator operations depend on client-side logic. For > example, clients may want to manage the allocator hierarchy per client-side > task, thread, etc.. > Currently the client is forced to use one global RootAllocator per process. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[ https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20315: Attachment: HIVE-20315.04.patch > Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary > casts/conversions > --- > > Key: HIVE-20315 > URL: https://issues.apache.org/jira/browse/HIVE-20315 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20315.01.patch, HIVE-20315.02.patch, > HIVE-20315.03.patch, HIVE-20315.04.patch > > > Generate multi-byte Unicode characters in addition to regular single byte > characters for random data. > Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in > vectorization without padding. > Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL > constants. > Fix NULL / Wrong Results issues in VectorElt. > Change performance Q files to generate non-user EXPLAIN with VECTORIZATION > display so unnecesary CAST / DECIMAL_64 conversions are visible. > Add test for vectorized STRUCT field reference. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[ https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20315: Status: Patch Available (was: In Progress) > Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary > casts/conversions > --- > > Key: HIVE-20315 > URL: https://issues.apache.org/jira/browse/HIVE-20315 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20315.01.patch, HIVE-20315.02.patch, > HIVE-20315.03.patch, HIVE-20315.04.patch > > > Generate multi-byte Unicode characters in addition to regular single byte > characters for random data. > Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in > vectorization without padding. > Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL > constants. > Fix NULL / Wrong Results issues in VectorElt. > Change performance Q files to generate non-user EXPLAIN with VECTORIZATION > display so unnecesary CAST / DECIMAL_64 conversions are visible. > Add test for vectorized STRUCT field reference. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[ https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20315: Description: Generate multi-byte Unicode characters in addition to regular single byte characters for random data. Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in vectorization without padding. Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL constants. Fix NULL / Wrong Results issues in VectorElt. Change performance Q files to generate non-user EXPLAIN with VECTORIZATION display so unnecesary CAST / DECIMAL_64 conversions are visible. Add test for vectorized STRUCT field reference. was: Generate multi-byte Unicode characters in addition to regular single byte characters for random data. Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in vectorization without padding. Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL constants. Fix NULL / Wrong Results issues in VectorElt. Change performance Q files to generate non-user EXPLAIN with VECTORIZATION display so unnecesary CAST / DECIMAL_64 conversions are visible. > Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary > casts/conversions > --- > > Key: HIVE-20315 > URL: https://issues.apache.org/jira/browse/HIVE-20315 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20315.01.patch, HIVE-20315.02.patch, > HIVE-20315.03.patch > > > Generate multi-byte Unicode characters in addition to regular single byte > characters for random data. > Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in > vectorization without padding. > Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL > constants. > Fix NULL / Wrong Results issues in VectorElt. > Change performance Q files to generate non-user EXPLAIN with VECTORIZATION > display so unnecesary CAST / DECIMAL_64 conversions are visible. > Add test for vectorized STRUCT field reference. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[ https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20315: Status: In Progress (was: Patch Available) > Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary > casts/conversions > --- > > Key: HIVE-20315 > URL: https://issues.apache.org/jira/browse/HIVE-20315 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20315.01.patch, HIVE-20315.02.patch, > HIVE-20315.03.patch > > > Generate multi-byte Unicode characters in addition to regular single byte > characters for random data. > Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in > vectorization without padding. > Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL > constants. > Fix NULL / Wrong Results issues in VectorElt. > Change performance Q files to generate non-user EXPLAIN with VECTORIZATION > display so unnecesary CAST / DECIMAL_64 conversions are visible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN
[
https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569531#comment-16569531
]
Hive QA commented on HIVE-14162:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12934407/HIVE-14162.7.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14862 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13049/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13049/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13049/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12934407 - PreCommit-HIVE-Build
> Allow disabling of long running job on Hive On Spark On YARN
>
>
> Key: HIVE-14162
> URL: https://issues.apache.org/jira/browse/HIVE-14162
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Thomas Scott
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-14162.1.patch, HIVE-14162.2.patch,
> HIVE-14162.3.patch, HIVE-14162.4.patch, HIVE-14162.5.patch,
> HIVE-14162.6.patch, HIVE-14162.7.patch
>
>
> Hive On Spark launches a long running process on the first query to handle
> all queries for that user session. In some use cases this is not desired, for
> instance when using Hue with large intervals between query executions.
> Could we have a property that would cause long running spark jobs to be
> terminated after each query execution and started again for the next one?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN
[
https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569529#comment-16569529
]
Hive QA commented on HIVE-14162:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
36s{color} | {color:blue} ql in master has 2301 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 0
unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 13 new + 156 unchanged - 0
fixed = 169 total (was 156) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
41s{color} | {color:red} ql generated 1 new + 2300 unchanged - 1 fixed = 2301
total (was 2301) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.close()
does not release lock on all paths At SparkSessionImpl.java:on all paths At
SparkSessionImpl.java:[line 190] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13049/dev-support/hive-personality.sh
|
| git revision | master / df5caa0 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13049/yetus/diff-checkstyle-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13049/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13049/yetus/new-findbugs-ql.html
|
| modules | C: common itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13049/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Allow disabling of long running job on Hive On Spark On YARN
>
>
> Key: HIVE-14162
> URL:
[jira] [Updated] (HIVE-14162) Allow disabling of long running job on Hive On Spark On YARN
[ https://issues.apache.org/jira/browse/HIVE-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-14162: Attachment: HIVE-14162.7.patch > Allow disabling of long running job on Hive On Spark On YARN > > > Key: HIVE-14162 > URL: https://issues.apache.org/jira/browse/HIVE-14162 > Project: Hive > Issue Type: Sub-task > Components: Spark >Reporter: Thomas Scott >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-14162.1.patch, HIVE-14162.2.patch, > HIVE-14162.3.patch, HIVE-14162.4.patch, HIVE-14162.5.patch, > HIVE-14162.6.patch, HIVE-14162.7.patch > > > Hive On Spark launches a long running process on the first query to handle > all queries for that user session. In some use cases this is not desired, for > instance when using Hue with large intervals between query executions. > Could we have a property that would cause long running spark jobs to be > terminated after each query execution and started again for the next one? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[
https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569513#comment-16569513
]
Hive QA commented on HIVE-20315:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} serde in master has 195 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
28s{color} | {color:blue} ql in master has 2301 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} serde: The patch generated 2 new + 6 unchanged - 0
fixed = 8 total (was 6) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} root: The patch generated 2 new + 9 unchanged - 0
fixed = 11 total (was 9) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
57s{color} | {color:red} ql: The patch generated 46 new + 1672 unchanged - 18
fixed = 1718 total (was 1690) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13048/dev-support/hive-personality.sh
|
| git revision | master / df5caa0 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13048/yetus/diff-checkstyle-serde.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13048/yetus/diff-checkstyle-root.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13048/yetus/diff-checkstyle-ql.txt
|
| modules | C: serde . ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13048/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary
> casts/conversions
> ---
>
> Key: HIVE-20315
> URL: https://issues.apache.org/jira/browse/HIVE-20315
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments:
[jira] [Commented] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[
https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569511#comment-16569511
]
Hive QA commented on HIVE-20315:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12934405/HIVE-20315.03.patch
{color:green}SUCCESS:{color} +1 due to 117 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14859 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13048/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13048/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13048/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12934405 - PreCommit-HIVE-Build
> Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary
> casts/conversions
> ---
>
> Key: HIVE-20315
> URL: https://issues.apache.org/jira/browse/HIVE-20315
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-20315.01.patch, HIVE-20315.02.patch,
> HIVE-20315.03.patch
>
>
> Generate multi-byte Unicode characters in addition to regular single byte
> characters for random data.
> Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in
> vectorization without padding.
> Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL
> constants.
> Fix NULL / Wrong Results issues in VectorElt.
> Change performance Q files to generate non-user EXPLAIN with VECTORIZATION
> display so unnecesary CAST / DECIMAL_64 conversions are visible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[ https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20315: Status: Patch Available (was: In Progress) > Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary > casts/conversions > --- > > Key: HIVE-20315 > URL: https://issues.apache.org/jira/browse/HIVE-20315 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20315.01.patch, HIVE-20315.02.patch, > HIVE-20315.03.patch > > > Generate multi-byte Unicode characters in addition to regular single byte > characters for random data. > Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in > vectorization without padding. > Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL > constants. > Fix NULL / Wrong Results issues in VectorElt. > Change performance Q files to generate non-user EXPLAIN with VECTORIZATION > display so unnecesary CAST / DECIMAL_64 conversions are visible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[ https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20315: Attachment: HIVE-20315.03.patch > Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary > casts/conversions > --- > > Key: HIVE-20315 > URL: https://issues.apache.org/jira/browse/HIVE-20315 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20315.01.patch, HIVE-20315.02.patch, > HIVE-20315.03.patch > > > Generate multi-byte Unicode characters in addition to regular single byte > characters for random data. > Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in > vectorization without padding. > Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL > constants. > Fix NULL / Wrong Results issues in VectorElt. > Change performance Q files to generate non-user EXPLAIN with VECTORIZATION > display so unnecesary CAST / DECIMAL_64 conversions are visible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[ https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20315: Status: In Progress (was: Patch Available) > Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary > casts/conversions > --- > > Key: HIVE-20315 > URL: https://issues.apache.org/jira/browse/HIVE-20315 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20315.01.patch, HIVE-20315.02.patch > > > Generate multi-byte Unicode characters in addition to regular single byte > characters for random data. > Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in > vectorization without padding. > Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL > constants. > Fix NULL / Wrong Results issues in VectorElt. > Change performance Q files to generate non-user EXPLAIN with VECTORIZATION > display so unnecesary CAST / DECIMAL_64 conversions are visible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[
https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569459#comment-16569459
]
Hive QA commented on HIVE-20315:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12934401/HIVE-20315.02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13047/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13047/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13047/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-08-05 13:06:26.426
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-13047/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-08-05 13:06:26.430
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at df5caa0 HIVE-20301: Enable vectorization for materialized view
rewriting tests (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at df5caa0 HIVE-20301: Enable vectorization for materialized view
rewriting tests (Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-08-05 13:06:27.970
+ rm -rf ../yetus_PreCommit-HIVE-Build-13047
+ mkdir ../yetus_PreCommit-HIVE-Build-13047
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-13047
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-13047/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:6436: trailing whitespace.
reduceColumnNullOrder:
/data/hiveptest/working/scratch/build.patch:6437: trailing whitespace.
reduceColumnSortOrder:
/data/hiveptest/working/scratch/build.patch:7331: trailing whitespace.
reduceColumnNullOrder:
/data/hiveptest/working/scratch/build.patch:7332: trailing whitespace.
reduceColumnSortOrder:
/data/hiveptest/working/scratch/build.patch:11129: trailing whitespace.
reduceColumnNullOrder:
warning: squelched 3536 whitespace errors
warning: 3541 lines add whitespace errors.
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc5946827154302052863.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc5946827154302052863.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
protoc-jar: executing: [/tmp/protoc3833379949968721191.exe, --version]
libprotoc 2.5.0
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
[jira] [Updated] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[ https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20315: Status: Patch Available (was: In Progress) > Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary > casts/conversions > --- > > Key: HIVE-20315 > URL: https://issues.apache.org/jira/browse/HIVE-20315 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20315.01.patch, HIVE-20315.02.patch > > > Generate multi-byte Unicode characters in addition to regular single byte > characters for random data. > Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in > vectorization without padding. > Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL > constants. > Fix NULL / Wrong Results issues in VectorElt. > Change performance Q files to generate non-user EXPLAIN with VECTORIZATION > display so unnecesary CAST / DECIMAL_64 conversions are visible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[ https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20315: Attachment: HIVE-20315.02.patch > Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary > casts/conversions > --- > > Key: HIVE-20315 > URL: https://issues.apache.org/jira/browse/HIVE-20315 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20315.01.patch, HIVE-20315.02.patch > > > Generate multi-byte Unicode characters in addition to regular single byte > characters for random data. > Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in > vectorization without padding. > Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL > constants. > Fix NULL / Wrong Results issues in VectorElt. > Change performance Q files to generate non-user EXPLAIN with VECTORIZATION > display so unnecesary CAST / DECIMAL_64 conversions are visible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20315) Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary casts/conversions
[ https://issues.apache.org/jira/browse/HIVE-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20315: Status: In Progress (was: Patch Available) > Vectorization: Fix more NULL / Wrong Results issues and avoid unnecessary > casts/conversions > --- > > Key: HIVE-20315 > URL: https://issues.apache.org/jira/browse/HIVE-20315 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20315.01.patch > > > Generate multi-byte Unicode characters in addition to regular single byte > characters for random data. > Don't CAST from STRING/VARCHAR/CHAR TO STRING since all are stored in > vectorization without padding. > Fix vectorized BETWEEN expression work to avoid unnecessary CAST of DECIMAL > constants. > Fix NULL / Wrong Results issues in VectorElt. > Change performance Q files to generate non-user EXPLAIN with VECTORIZATION > display so unnecesary CAST / DECIMAL_64 conversions are visible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19878) Hive On Spark support AM shut down when there is no job submit
[
https://issues.apache.org/jira/browse/HIVE-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569405#comment-16569405
]
Hive QA commented on HIVE-19878:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927758/HIVE-19878.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 59 failed/errored test(s), 14050 tests
executed
*Failed tests:*
{noformat}
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=186)
[infer_bucket_sort_reducers_power_two.q,list_bucket_dml_10.q,orc_merge9.q,leftsemijoin_mr.q,bucket6.q,bucketmapjoin7.q,uber_reduce.q,empty_dir_in_table.q,vector_outer_join2.q,spark_explain_groupbyshuffle.q,spark_dynamic_partition_pruning.q,spark_combine_equivalent_work.q,orc_merge1.q,spark_use_op_stats.q,orc_merge_diff_fs.q,quotedid_smb.q,truncate_column_buckets.q,spark_vectorized_dynamic_partition_pruning.q,spark_in_process_launcher.q,orc_merge3.q]
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=187)
[infer_bucket_sort_num_buckets.q,gen_udf_example_add10.q,spark_explainuser_1.q,spark_use_ts_stats_for_mapjoin.q,orc_merge6.q,orc_merge5.q,bucketmapjoin6.q,spark_opt_shuffle_serde.q,temp_table_external.q,spark_dynamic_partition_pruning_6.q,dynamic_rdd_cache.q,auto_sortmerge_join_16.q,vector_outer_join3.q,spark_dynamic_partition_pruning_7.q,schemeAuthority.q,parallel_orderby.q,vector_outer_join1.q,load_hdfs_file_with_space_in_the_name.q,spark_dynamic_partition_pruning_recursive_mapjoin.q,spark_dynamic_partition_pruning_mapjoin_only.q]
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=188)
[insert_overwrite_directory2.q,spark_dynamic_partition_pruning_4.q,import_exported_table.q,vector_outer_join0.q,bucket4.q,orc_merge4.q,infer_bucket_sort_merge.q,orc_merge_incompat1.q,root_dir_external_table.q,constprog_partitioner.q,constprog_semijoin.q,external_table_with_space_in_location_path.q,spark_constprog_dpp.q,spark_dynamic_partition_pruning_3.q,load_fs2.q,infer_bucket_sort_map_operators.q,spark_dynamic_partition_pruning_2.q,vector_inner_join.q,spark_multi_insert_parallel_orderby.q,remote_script.q]
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=189)
[scriptfile1.q,vector_outer_join5.q,file_with_header_footer.q,input16_cc.q,bucket5.q,orc_merge2.q,reduce_deduplicate.q,schemeAuthority2.q,spark_dynamic_partition_pruning_5.q,orc_merge8.q,orc_merge_incompat2.q,infer_bucket_sort_bucketed_table.q,vector_outer_join4.q,disable_merge_for_bucketing.q,orc_merge7.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=108)
[bucketmapjoin4.q,bucket_map_join_spark4.q,union21.q,groupby2_noskew.q,timestamp_2.q,date_join1.q,mergejoins.q,smb_mapjoin_11.q,auto_sortmerge_join_3.q,mapjoin_test_outer.q,vectorization_9.q,merge2.q,groupby6_noskew.q,auto_join_without_localtask.q,multi_join_union.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=109)
[join_cond_pushdown_unqual4.q,union_remove_7.q,join13.q,join_vc.q,groupby_cube1.q,parquet_vectorization_2.q,bucket_map_join_spark2.q,sample3.q,smb_mapjoin_19.q,union23.q,union.q,union31.q,cbo_udf_udaf.q,ptf_decimal.q,bucketmapjoin2.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=110)
[parallel_join1.q,union27.q,union12.q,groupby7_map_multi_single_reducer.q,varchar_join1.q,join7.q,join_reorder4.q,skewjoinopt2.q,bucketsortoptimize_insert_2.q,smb_mapjoin_17.q,script_env_var1.q,groupby7_map.q,bucketsortoptimize_insert_8.q,stats16.q,union20.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=111)
[ptf_general_queries.q,auto_join_reordering_values.q,sample2.q,join1.q,decimal_join.q,mapjoin_subquery2.q,join32_lessSize.q,mapjoin1.q,skewjoinopt18.q,union_remove_18.q,join25.q,groupby3.q,groupby9.q,bucketsortoptimize_insert_6.q,ctas.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=112)
[groupby_map_ppr.q,nullgroup4_multi_distinct.q,join_rc.q,union14.q,order2.q,smb_mapjoin_12.q,vector_cast_constant.q,union_remove_4.q,parquet_vectorization_1.q,auto_join11.q,udaf_collect_set.q,vectorization_12.q,groupby_sort_skew_1_23.q,smb_mapjoin_25.q,skewjoinopt12.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=113)
[skewjoinopt15.q,auto_join18.q,list_bucket_dml_2.q,input1_limit.q,load_dyn_part3.q,union_remove_14.q,auto_sortmerge_join_14.q,auto_sortmerge_join_15.q,union10.q,bucket_map_join_tez2.q,groupby5_map_skew.q,load_dyn_part7.q,join_reorder.q,bucketmapjoin8.q,union34.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
[jira] [Commented] (HIVE-19878) Hive On Spark support AM shut down when there is no job submit
[
https://issues.apache.org/jira/browse/HIVE-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569393#comment-16569393
]
Hive QA commented on HIVE-19878:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} spark-client in master has 10 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} common: The patch generated 3 new + 424 unchanged - 0
fixed = 427 total (was 424) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} spark-client: The patch generated 5 new + 34 unchanged
- 1 fixed = 39 total (was 35) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
37s{color} | {color:red} spark-client generated 2 new + 10 unchanged - 0 fixed
= 12 total (was 10) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:spark-client |
| | Boxing/unboxing to parse a primitive new
org.apache.hive.spark.client.RemoteDriver(String[]) At RemoteDriver.java:new
org.apache.hive.spark.client.RemoteDriver(String[]) At RemoteDriver.java:[line
174] |
| | Inconsistent synchronization of
org.apache.hive.spark.client.RemoteDriver$ClientListener.lastDAGCompletionTime;
locked 66% of time Unsynchronized access at RemoteDriver.java:66% of time
Unsynchronized access at RemoteDriver.java:[line 560] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-13046/dev-support/hive-personality.sh
|
| git revision | master / df5caa0 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13046/yetus/diff-checkstyle-common.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13046/yetus/diff-checkstyle-spark-client.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13046/yetus/new-findbugs-spark-client.html
|
| modules | C: common spark-client U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-13046/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive On Spark support AM shut down when there is no job submit
>
[jira] [Commented] (HIVE-20290) Lazy initialize ArrowColumnarBatchSerDe so it doesn't allocate buffers during GetSplits
[
https://issues.apache.org/jira/browse/HIVE-20290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569385#comment-16569385
]
Hive QA commented on HIVE-20290:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12934395/HIVE-20290.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14859 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/13045/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13045/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13045/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12934395 - PreCommit-HIVE-Build
> Lazy initialize ArrowColumnarBatchSerDe so it doesn't allocate buffers during
> GetSplits
> ---
>
> Key: HIVE-20290
> URL: https://issues.apache.org/jira/browse/HIVE-20290
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Blocker
> Attachments: HIVE-20290.1.patch, HIVE-20290.2.patch
>
>
> When using {{GenericUDTFGetSplits}} to create {{LlapInputSplit}} for
> submission to {{LlapOutputFormatService}}, the physical plan generation
> initializes whatever SerDe is being used.
> {{ArrowColumnarBatchSerDe}} initializes buffers for Arrow and
> {{VectorizedRowBatch}} at this point inside HS2 which are never used.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
