date:20200203

[jira] [Assigned] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Reinhard Eilmsteiner (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reinhard Eilmsteiner reassigned HIVE-22817:
---

Assignee: Reinhard Eilmsteiner

> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Assignee: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz, hive.log
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64

2020-02-03 Thread Zhenyu Zheng (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029618#comment-17029618
 ] 

Zhenyu Zheng commented on HIVE-21939:
-

Added the full build log for using protobuf version 3.7.1 as attachment

> protoc:2.5.0  dependence has broken building on aarch64
> ---
>
> Key: HIVE-21939
> URL: https://issues.apache.org/jira/browse/HIVE-21939
> Project: Hive
>  Issue Type: Bug
>Reporter: liusheng
>Assignee: Zhenyu Zheng
>Priority: Blocker
>  Labels: pull-request-available
> Attachments: hive_build_with_protobuf_371.log
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When I try to build master of Hive from source code on "aarch64" server, I 
> met following error:
> [ERROR] Failed to execute goal 
> com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project 
> hive-standalone-metastore-common: Error resolving artifact: 
> com.google.protobuf:protoc:2.5.0: Could not find artifact 
> com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central 
> ([https://repo.maven.apache.org/maven2)]
> that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required 
> artifact, which does not have released package for "aarch64" platform.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan

2020-02-03 Thread Krisztian Kasa (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22808:
--
Status: Patch Available  (was: Open)

> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch, 
> HIVE-22808.2.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27  int,
> int_col_39  int,
> boolean_col_18  boolean,
> varchar0045_col_23  varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10  int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8   boolean,
> boolean_col_16  boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26  decimal(38, 25),
> smallint_col_9  smallint,
> int_col_18  int
> );
> explain cbo 
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16, 
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4 
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in 
> projections like:
> {code:java}
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
>  The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor 
> RelFieldTrimmer which suppose to remove unused columns in the 
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree 
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
>   HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
>   HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
>   HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
>   HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8, 
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner], 
> algorithm=[none], cost=[not available])
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> HiveFilter(condition=[IS NOT NULL($1)])
>   HiveTableScan(table=[[default, table_7]], 
> table:alias=[a3])
>   HiveProject(boolean_col_16=[$0], 
> timestamp_col_5=[$1], timestamp_col_15=[$2], timestamp_col_30=[$3], 
> int_col_18=[$4], BLOCK__OFFSET__INSIDE__FILE=[$5], INPUT__FILE__NAME=[$6], 
> ROW__ID=[$7], CAST=[CAST($4):BIGINT])
> HiveFilter(condition=[IS NOT 
> NULL(CAST($4):BIGINT)])
>   HiveTableScan(table=[[default, table_10]], 
> table:alias=[a4])
>   HiveProject($f0=[$0], $f1=[$1])
>

[jira] [Updated] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan

2020-02-03 Thread Krisztian Kasa (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22808:
--
Attachment: HIVE-22808.2.patch

> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch, 
> HIVE-22808.2.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27  int,
> int_col_39  int,
> boolean_col_18  boolean,
> varchar0045_col_23  varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10  int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8   boolean,
> boolean_col_16  boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26  decimal(38, 25),
> smallint_col_9  smallint,
> int_col_18  int
> );
> explain cbo 
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16, 
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4 
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in 
> projections like:
> {code:java}
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
>  The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor 
> RelFieldTrimmer which suppose to remove unused columns in the 
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree 
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
>   HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
>   HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
>   HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
>   HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8, 
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner], 
> algorithm=[none], cost=[not available])
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> HiveFilter(condition=[IS NOT NULL($1)])
>   HiveTableScan(table=[[default, table_7]], 
> table:alias=[a3])
>   HiveProject(boolean_col_16=[$0], 
> timestamp_col_5=[$1], timestamp_col_15=[$2], timestamp_col_30=[$3], 
> int_col_18=[$4], BLOCK__OFFSET__INSIDE__FILE=[$5], INPUT__FILE__NAME=[$6], 
> ROW__ID=[$7], CAST=[CAST($4):BIGINT])
> HiveFilter(condition=[IS NOT 
> NULL(CAST($4):BIGINT)])
>   HiveTableScan(table=[[default, table_10]], 
> table:alias=[a4])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}],

[jira] [Updated] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan

2020-02-03 Thread Krisztian Kasa (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22808:
--
Status: Open  (was: Patch Available)

> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch, 
> HIVE-22808.2.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27  int,
> int_col_39  int,
> boolean_col_18  boolean,
> varchar0045_col_23  varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10  int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8   boolean,
> boolean_col_16  boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26  decimal(38, 25),
> smallint_col_9  smallint,
> int_col_18  int
> );
> explain cbo 
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16, 
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4 
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in 
> projections like:
> {code:java}
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
>  The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor 
> RelFieldTrimmer which suppose to remove unused columns in the 
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree 
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
>   HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
>   HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
>   HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
>   HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8, 
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner], 
> algorithm=[none], cost=[not available])
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> HiveFilter(condition=[IS NOT NULL($1)])
>   HiveTableScan(table=[[default, table_7]], 
> table:alias=[a3])
>   HiveProject(boolean_col_16=[$0], 
> timestamp_col_5=[$1], timestamp_col_15=[$2], timestamp_col_30=[$3], 
> int_col_18=[$4], BLOCK__OFFSET__INSIDE__FILE=[$5], INPUT__FILE__NAME=[$6], 
> ROW__ID=[$7], CAST=[CAST($4):BIGINT])
> HiveFilter(condition=[IS NOT 
> NULL(CAST($4):BIGINT)])
>   HiveTableScan(table=[[default, table_10]], 
> table:alias=[a4])
>   HiveProject($f0=[$0], $f1=[$1])
>

[jira] [Updated] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64

2020-02-03 Thread Zhenyu Zheng (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenyu Zheng updated HIVE-21939:

Attachment: hive_build_with_protobuf_371.log

> protoc:2.5.0  dependence has broken building on aarch64
> ---
>
> Key: HIVE-21939
> URL: https://issues.apache.org/jira/browse/HIVE-21939
> Project: Hive
>  Issue Type: Bug
>Reporter: liusheng
>Assignee: Zhenyu Zheng
>Priority: Blocker
>  Labels: pull-request-available
> Attachments: hive_build_with_protobuf_371.log
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When I try to build master of Hive from source code on "aarch64" server, I 
> met following error:
> [ERROR] Failed to execute goal 
> com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project 
> hive-standalone-metastore-common: Error resolving artifact: 
> com.google.protobuf:protoc:2.5.0: Could not find artifact 
> com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central 
> ([https://repo.maven.apache.org/maven2)]
> that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required 
> artifact, which does not have released package for "aarch64" platform.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22502) ConcurrentModificationException in TriggerValidatorRunnable stops trigger processing

2020-02-03 Thread Prasanth Jayachandran (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029603#comment-17029603
 ] 

Prasanth Jayachandran commented on HIVE-22502:
--

Thanks for the patch! +1

> ConcurrentModificationException in TriggerValidatorRunnable stops trigger 
> processing
> 
>
> Key: HIVE-22502
> URL: https://issues.apache.org/jira/browse/HIVE-22502
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-22502.1.patch
>
>
> An other thread is modifying the list the contains the sessions while 
> TriggerValidatorRunnable is traversing it. This causes the 
> TriggerValidatorRunnable thread to die and triggers are no longer properly 
> monitored.
>  
> {code:java}
> <12>1 2019-11-14T00:31:12.187Z 
> hiveserver2-0.hiveserver2-service.compute-1572769905-6965.svc.cluster.local 
> hiveserver2 1 fa2f30b6-ffb3-11e9-93ba-0a257c2413a2 [mdc@18060 
> class="tez.TriggerValidatorRunnable" level="WARN" thread="TriggerValidator"] 
> TriggerValidatorRunnable caught exception.<12>1 2019-11-14T00:31:12.187Z 
> hiveserver2-0.hiveserver2-service.compute-1572769905-6965.svc.cluster.local 
> hiveserver2 1 fa2f30b6-ffb3-11e9-93ba-0a257c2413a2 [mdc@18060 
> class="tez.TriggerValidatorRunnable" level="WARN" thread="TriggerValidator"] 
> TriggerValidatorRunnable caught 
> exception.java.util.ConcurrentModificationException at 
> java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966) at 
> java.util.LinkedList$ListItr.next(LinkedList.java:888) at 
> java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1044) at 
> org.apache.hadoop.hive.ql.exec.tez.TriggerValidatorRunnable.run(TriggerValidatorRunnable.java:49)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748)             {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22775) Use the qt:authorizer option in qtests

2020-02-03 Thread Zoltan Haindrich (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-22775:

Attachment: HIVE-22775.03.patch

> Use the qt:authorizer option in qtests
> --
>
> Key: HIVE-22775
> URL: https://issues.apache.org/jira/browse/HIVE-22775
> Project: Hive
>  Issue Type: Improvement
>  Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22775.01.patch, HIVE-22775.02.patch, 
> HIVE-22775.02.patch, HIVE-22775.03.patch, HIVE-22775.03.patch
>
>
> qt:authorizer sets up things for auth tests; [~mgergely] suggested to use it 
> in existing tests 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Reinhard Eilmsteiner (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029600#comment-17029600
 ] 

Reinhard Eilmsteiner commented on HIVE-22817:
-

Thx Javi,

Will play around with these... I wonder why this is not mentioned in the 
quick-start guide. Can _I_ add that?

R,R

> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz, hive.log
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22647) enable session pool by default

2020-02-03 Thread Ashutosh Chauhan (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-22647:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Pushed to master. Thanks, Attila!

> enable session pool by default
> --
>
> Key: HIVE-22647
> URL: https://issues.apache.org/jira/browse/HIVE-22647
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22647.1.patch, HIVE-22647.2.patch
>
>
> Non pooled session my leak when the client doesn't close the connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan

2020-02-03 Thread Krisztian Kasa (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22808:
--
Status: Patch Available  (was: Open)

> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27  int,
> int_col_39  int,
> boolean_col_18  boolean,
> varchar0045_col_23  varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10  int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8   boolean,
> boolean_col_16  boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26  decimal(38, 25),
> smallint_col_9  smallint,
> int_col_18  int
> );
> explain cbo 
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16, 
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4 
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in 
> projections like:
> {code:java}
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
>  The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor 
> RelFieldTrimmer which suppose to remove unused columns in the 
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree 
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
>   HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
>   HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
>   HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
>   HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8, 
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner], 
> algorithm=[none], cost=[not available])
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> HiveFilter(condition=[IS NOT NULL($1)])
>   HiveTableScan(table=[[default, table_7]], 
> table:alias=[a3])
>   HiveProject(boolean_col_16=[$0], 
> timestamp_col_5=[$1], timestamp_col_15=[$2], timestamp_col_30=[$3], 
> int_col_18=[$4], BLOCK__OFFSET__INSIDE__FILE=[$5], INPUT__FILE__NAME=[$6], 
> ROW__ID=[$7], CAST=[CAST($4):BIGINT])
> HiveFilter(condition=[IS NOT 
> NULL(CAST($4):BIGINT)])
>   HiveTableScan(table=[[default, table_10]], 
> table:alias=[a4])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
>

[jira] [Updated] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan

2020-02-03 Thread Krisztian Kasa (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22808:
--
Status: Open  (was: Patch Available)

> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27  int,
> int_col_39  int,
> boolean_col_18  boolean,
> varchar0045_col_23  varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10  int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8   boolean,
> boolean_col_16  boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26  decimal(38, 25),
> smallint_col_9  smallint,
> int_col_18  int
> );
> explain cbo 
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16, 
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4 
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in 
> projections like:
> {code:java}
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
>  The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor 
> RelFieldTrimmer which suppose to remove unused columns in the 
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree 
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
>   HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
>   HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
>   HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
>   HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8, 
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner], 
> algorithm=[none], cost=[not available])
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> HiveFilter(condition=[IS NOT NULL($1)])
>   HiveTableScan(table=[[default, table_7]], 
> table:alias=[a3])
>   HiveProject(boolean_col_16=[$0], 
> timestamp_col_5=[$1], timestamp_col_15=[$2], timestamp_col_30=[$3], 
> int_col_18=[$4], BLOCK__OFFSET__INSIDE__FILE=[$5], INPUT__FILE__NAME=[$6], 
> ROW__ID=[$7], CAST=[CAST($4):BIGINT])
> HiveFilter(condition=[IS NOT 
> NULL(CAST($4):BIGINT)])
>   HiveTableScan(table=[[default, table_10]], 
> table:alias=[a4])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
>

[jira] [Updated] (HIVE-22808) HiveRelFieldTrimmer does not handle HiveTableFunctionScan

2020-02-03 Thread Krisztian Kasa (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22808:
--
Attachment: HIVE-22808.2.patch

> HiveRelFieldTrimmer does not handle HiveTableFunctionScan
> -
>
> Key: HIVE-22808
> URL: https://issues.apache.org/jira/browse/HIVE-22808
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22808.1.patch, HIVE-22808.2.patch
>
>
> *Repro*
> {code:java}
> CREATE TABLE table_16 (
> timestamp_col_19timestamp,
> timestamp_col_29timestamp,
> int_col_27  int,
> int_col_39  int,
> boolean_col_18  boolean,
> varchar0045_col_23  varchar(45)
> );
> CREATE TABLE table_7 (
> int_col_10  int,
> bigint_col_3bigint
> );
> CREATE TABLE table_10 (
> boolean_col_8   boolean,
> boolean_col_16  boolean,
> timestamp_col_5 timestamp,
> timestamp_col_15timestamp,
> timestamp_col_30timestamp,
> decimal3825_col_26  decimal(38, 25),
> smallint_col_9  smallint,
> int_col_18  int
> );
> explain cbo 
> SELECT
> DISTINCT COALESCE(a4.timestamp_col_15, IF(a4.boolean_col_16, 
> a4.timestamp_col_30, a4.timestamp_col_5)) AS timestamp_col
> FROM table_7 a3
> RIGHT JOIN table_10 a4 
> WHERE (a3.bigint_col_3) >= (a4.int_col_18)
> INTERSECT ALL
> SELECT
> COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> ) AS timestamp_col
> FROM table_16 a1
> GROUP BY COALESCE(LEAST(
> COALESCE(a1.timestamp_col_19, CAST('2010-03-29 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2014-08-16 00:00:00' AS 
> TIMESTAMP))
> ),
> GREATEST(
> COALESCE(a1.timestamp_col_19, CAST('2013-07-01 00:00:00' AS 
> TIMESTAMP)),
> COALESCE(a1.timestamp_col_29, CAST('2028-06-18 00:00:00' AS 
> TIMESTAMP)))
> );
> {code}
> CBO Plan contains unnecessary columns or all columns from a table in 
> projections like:
> {code:java}
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> {code}
> *Cause*
>  The plan contains a HiveTableFunctionScan operator:
> {code:java}
> HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> {code}
> HiveTableFunctionScan is not handled by HiveRelFieldTrimmer nor 
> RelFieldTrimmer which suppose to remove unused columns in the 
> CalcitePlanner.applyPreJoinOrderingTransforms(...) phase. The whole subtree 
> rooted from HiveTableFunctionScan is ignored.
> Whole plan:
> {code:java}
> CBO PLAN:
> HiveProject($f0=[$1])
>   HiveTableFunctionScan(invocation=[replicate_rows($0, $1)], 
> rowType=[RecordType(BIGINT $f0, TIMESTAMP(9) $f1)])
> HiveProject($f0=[$2], $f1=[$0])
>   HiveFilter(condition=[=($1, 2)])
> HiveAggregate(group=[{0}], agg#0=[count($1)], agg#1=[min($1)])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveUnion(all=[true])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
>   HiveProject($f0=[$0])
> HiveAggregate(group=[{0}])
>   HiveProject($f0=[CASE(IS NOT NULL($7), $7, if($5, $8, 
> $6))])
> HiveJoin(condition=[>=($1, $13)], joinType=[inner], 
> algorithm=[none], cost=[not available])
>   HiveProject(int_col_10=[$0], bigint_col_3=[$1], 
> BLOCK__OFFSET__INSIDE__FILE=[$2], INPUT__FILE__NAME=[$3], 
> CAST=[CAST($4):RecordType(BIGINT writeid, INTEGER bucketid, BIGINT rowid)])
> HiveFilter(condition=[IS NOT NULL($1)])
>   HiveTableScan(table=[[default, table_7]], 
> table:alias=[a3])
>   HiveProject(boolean_col_16=[$0], 
> timestamp_col_5=[$1], timestamp_col_15=[$2], timestamp_col_30=[$3], 
> int_col_18=[$4], BLOCK__OFFSET__INSIDE__FILE=[$5], INPUT__FILE__NAME=[$6], 
> ROW__ID=[$7], CAST=[CAST($4):BIGINT])
> HiveFilter(condition=[IS NOT 
> NULL(CAST($4):BIGINT)])
>   HiveTableScan(table=[[default, table_10]], 
> table:alias=[a4])
>   HiveProject($f0=[$0], $f1=[$1])
> HiveAggregate(group=[{0}], agg#0=[count()])
>

[jira] [Commented] (HIVE-22824) JoinProjectTranspose rule should skip Projects containing windowing expression

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029545#comment-17029545
 ] 

Hive QA commented on HIVE-22824:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992527/HIVE-22824.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 17959 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20452/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20452/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20452/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992527 - PreCommit-HIVE-Build

> JoinProjectTranspose rule should skip Projects containing windowing expression
> --
>
> Key: HIVE-22824
> URL: https://issues.apache.org/jira/browse/HIVE-22824
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22824.1.patch
>
>
> Otherwise this rule could end up creating plan with windowing expression 
> within join condition which hive doesn't know how to process.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22824) JoinProjectTranspose rule should skip Projects containing windowing expression

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029507#comment-17029507
 ] 

Hive QA commented on HIVE-22824:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  9m 
 3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
38s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
39s{color} | {color:red} ql: The patch generated 3 new + 7 unchanged - 1 fixed 
= 10 total (was 8) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 33s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20452/dev-support/hive-personality.sh
 |
| git revision | master / 842fdd2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20452/yetus/diff-checkstyle-ql.txt
 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20452/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> JoinProjectTranspose rule should skip Projects containing windowing expression
> --
>
> Key: HIVE-22824
> URL: https://issues.apache.org/jira/browse/HIVE-22824
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22824.1.patch
>
>
> Otherwise this rule could end up creating plan with windowing expression 
> within join condition which hive doesn't know how to process.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22589) Add storage support for ProlepticCalendar in ORC, Parquet, and Avro

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029492#comment-17029492
 ] 

Hive QA commented on HIVE-22589:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  9m 
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  4m 
20s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
23s{color} | {color:blue} storage-api in master has 58 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  2m 
27s{color} | {color:blue} standalone-metastore/metastore-common in master has 
35 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
33s{color} | {color:blue} common in master has 63 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
40s{color} | {color:blue} serde in master has 197 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
47s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
48s{color} | {color:blue} llap-server in master has 90 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m  
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  9m 
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  9m 
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
11s{color} | {color:red} standalone-metastore/metastore-common: The patch 
generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
15s{color} | {color:red} common: The patch generated 4 new + 370 unchanged - 0 
fixed = 374 total (was 370) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
15s{color} | {color:red} serde: The patch generated 2 new + 105 unchanged - 3 
fixed = 107 total (was 108) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
52s{color} | {color:red} ql: The patch generated 21 new + 1268 unchanged - 11 
fixed = 1289 total (was 1279) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
14s{color} | {color:red} llap-server: The patch generated 3 new + 185 unchanged 
- 1 fixed = 188 total (was 186) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  2m 
17s{color} | {color:red} root: The patch generated 31 new + 1935 unchanged - 15 
fixed = 1966 total (was 1950) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  3m 
47s{color} | {color:red} ql generated 3 new + 1532 unchanged - 1 fixed = 1535 
total (was 1533) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 13m 
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
34s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 93m 22s{color} | 
{color:black} {color} |
\\
\\
||

[jira] [Commented] (HIVE-22589) Add storage support for ProlepticCalendar in ORC, Parquet, and Avro

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029484#comment-17029484
 ] 

Hive QA commented on HIVE-22589:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992528/HIVE-22589.06.patch

{color:green}SUCCESS:{color} +1 due to 26 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 101 failed/errored test(s), 17978 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
 (batchId=304)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_bloom_filter_orc_file_dump]
 (batchId=98)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_nullscan] 
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats2] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats] 
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4] 
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnStatsUpdateForStatsOptimizer_2]
 (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[deleteAnalyze] 
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[extrapolate_part_stats_full]
 (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[extrapolate_part_stats_partial]
 (batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_mv] (batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[materialized_view_create_acid]
 (batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge10] (batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge11] (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge12] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_schema_evolution_float]
 (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_stats] 
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_nonpart] 
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_part2] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_part] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_sizebug] 
(batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[type_change_test_fraction]
 (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[type_change_test_fraction_vectorized]
 (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[typechangetest] 
(batchId=12)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
 (batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynamic_semijoin_user_level]
 (batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_nullscan] 
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters1]
 (batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters]
 (batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge10] 
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge1] 
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge2] 
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge3] 
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge4] 
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] 
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
 (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_no_buckets]
 (batchId=185)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[alter_merge_orc]
 (batchId=184)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[alter_merge_stats_orc]
 (batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez2]
 (batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[column_table_stats_orc]
 (batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
 (batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[deleteAnalyze]
 (batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction]
 (batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_opt_vectorization]
 (batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization2]
 (batchId=165)

[jira] [Commented] (HIVE-22816) QueryCache: Queries using views can have them cached after CTE expansion

2020-02-03 Thread Jesus Camacho Rodriguez (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029427#comment-17029427
 ] 

Jesus Camacho Rodriguez commented on HIVE-22816:


+1

> QueryCache: Queries using views can have them cached after CTE expansion
> 
>
> Key: HIVE-22816
> URL: https://issues.apache.org/jira/browse/HIVE-22816
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Gopal Vijayaraghavan
>Assignee: Gopal Vijayaraghavan
>Priority: Major
> Attachments: HIVE-22816.1.patch
>
>
> {code}
> create view ss_null as select * from store_Sales where ss_Sold_date_sk is 
> null;
> select count(ss_ticket_number) from ss_null;
> with ss_null_cte as 
> (select * from store_Sales where ss_Sold_date_sk is null)
> select count(ss_ticket_number) from ss_null_cte;
> {code}
> Are treated differently by the query cache, however their execution is 
> identical.
> CBO rewrites the view query into AST form as follows
> {code}
> SELECT COUNT(`ss_ticket_number`) AS `$f0`
> FROM `tpcds_bin_partitioned_acid_orc_1`.`store_sales`
> WHERE `ss_sold_date_sk` IS NULL
> {code}
> But retains the write-entity for the VIRTUAL_VIEW for Ranger authorization 
> {code}
> 0: jdbc:hive2://localhost:10013> explain dependency select count(distinct 
> ss_ticket_number) from ss_null;
> ++
> |  Explain   |
> ++
> | 
> {"input_tables":[{"tablename":"tpcds_bin_partitioned_acid_orc_1@ss_null","tabletype":"VIRTUAL_VIEW"},{"tablename":"tpcds_bin_partitioned_acid_orc_1@store_sales","tabletype":"MANAGED_TABLE","tableParents":"[tpcds_bin_partitioned_acid_orc_1@ss_null]"}],"input_partitions":[{"partitionName":"tpcds_bin_partitioned_acid_orc_1@store_sales@ss_sold_date_sk=__HIVE_DEFAULT_PARTITION__"}]}
>  |
> ++
> {code}
> Causing Query cache to print out
> {code}
> parse.CalcitePlanner: Not eligible for results caching - query contains 
> non-transactional tables [ss_null]
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22589) Add storage support for ProlepticCalendar in ORC, Parquet, and Avro

2020-02-03 Thread Jesus Camacho Rodriguez (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-22589:
---
Attachment: HIVE-22589.06.patch

> Add storage support for ProlepticCalendar in ORC, Parquet, and Avro
> ---
>
> Key: HIVE-22589
> URL: https://issues.apache.org/jira/browse/HIVE-22589
> Project: Hive
>  Issue Type: Bug
>  Components: Avro, ORC, Parquet
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 4.0.0, 3.2.0, 3.1.3
>
> Attachments: HIVE-22589.01.patch, HIVE-22589.02.patch, 
> HIVE-22589.03.patch, HIVE-22589.04.patch, HIVE-22589.05.patch, 
> HIVE-22589.06.patch, HIVE-22589.patch, HIVE-22589.patch
>
>
> Hive recently moved its processing to the proleptic calendar, which has 
> created some issues for users who have dates before 1580 AD.
> HIVE-22405 extended the column vectors for times & dates to encode which 
> calendar they are using.
> This issue is to support proleptic calendar in ORC, Parquet, and Avro, when 
> files are written/read by Hive. To preserve compatibility with other engines 
> until they upgrade their readers, files will be written using hybrid calendar 
> by default. Default behavior when files do not contain calendar information 
> in their metadata is configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22824) JoinProjectTranspose rule should skip Projects containing windowing expression

2020-02-03 Thread Vineet Garg (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vineet Garg updated HIVE-22824:
---
Attachment: HIVE-22824.1.patch

> JoinProjectTranspose rule should skip Projects containing windowing expression
> --
>
> Key: HIVE-22824
> URL: https://issues.apache.org/jira/browse/HIVE-22824
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22824.1.patch
>
>
> Otherwise this rule could end up creating plan with windowing expression 
> within join condition which hive doesn't know how to process.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22824) JoinProjectTranspose rule should skip Projects containing windowing expression

2020-02-03 Thread Vineet Garg (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vineet Garg updated HIVE-22824:
---
Status: Patch Available  (was: Open)

> JoinProjectTranspose rule should skip Projects containing windowing expression
> --
>
> Key: HIVE-22824
> URL: https://issues.apache.org/jira/browse/HIVE-22824
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22824.1.patch
>
>
> Otherwise this rule could end up creating plan with windowing expression 
> within join condition which hive doesn't know how to process.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-22824) JoinProjectTranspose rule should skip Projects containing windowing expression

2020-02-03 Thread Vineet Garg (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vineet Garg reassigned HIVE-22824:
--


> JoinProjectTranspose rule should skip Projects containing windowing expression
> --
>
> Key: HIVE-22824
> URL: https://issues.apache.org/jira/browse/HIVE-22824
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> Otherwise this rule could end up creating plan with windowing expression 
> within join condition which hive doesn't know how to process.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22589) Add storage support for ProlepticCalendar in ORC, Parquet, and Avro

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029392#comment-17029392
 ] 

Hive QA commented on HIVE-22589:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992522/HIVE-22589.05.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20450/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20450/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20450/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-02-04 00:25:41.213
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-20450/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-02-04 00:25:41.216
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 842fdd2 HIVE-21341 : Sensible defaults : 
hive.server2.idle.operation.timeout and hive.server2.idle.session.timeout are 
too high (Ashutosh Chauhan via Thejas Nair)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 842fdd2 HIVE-21341 : Sensible defaults : 
hive.server2.idle.operation.timeout and hive.server2.idle.session.timeout are 
too high (Ashutosh Chauhan via Thejas Nair)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-02-04 00:25:41.955
+ rm -rf ../yetus_PreCommit-HIVE-Build-20450
+ mkdir ../yetus_PreCommit-HIVE-Build-20450
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20450
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20450/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
fatal: git diff header lacks filename information when removing 0 leading 
pathname components (line 234)
Trying to apply the patch with -p1
error: cannot apply binary patch to 'data/files/avro_legacy_mixed_dates.avro' 
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/avro_legacy_mixed_dates.avro' 
without full index line
error: data/files/avro_legacy_mixed_dates.avro: patch does not apply
error: cannot apply binary patch to 
'data/files/avro_legacy_mixed_timestamps.avro' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 
'data/files/avro_legacy_mixed_timestamps.avro' without full index line
error: data/files/avro_legacy_mixed_timestamps.avro: patch does not apply
error: cannot apply binary patch to 'data/files/orc_legacy_mixed_dates.orc' 
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/orc_legacy_mixed_dates.orc' 
without full index line
error: data/files/orc_legacy_mixed_dates.orc: patch does not apply
error: cannot apply binary patch to 
'data/files/orc_legacy_mixed_timestamps.orc' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 
'data/files/orc_legacy_mixed_timestamps.orc' without full index line
error: data/files/orc_legacy_mixed_timestamps.orc: patch does not apply
error: cannot apply binary patch to 
'data/files/parquet_legacy_mixed_dates.parq' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 
'data/files/parquet_legacy_mixed_dates.parq' without full index line
error: data/files/parquet_legacy_mixed_dates.parq: patch does not apply
error: cannot apply binary patch to

[jira] [Commented] (HIVE-22647) enable session pool by default

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029391#comment-17029391
 ] 

Hive QA commented on HIVE-22647:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992520/HIVE-22647.2.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 17959 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20449/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20449/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20449/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992520 - PreCommit-HIVE-Build

> enable session pool by default
> --
>
> Key: HIVE-22647
> URL: https://issues.apache.org/jira/browse/HIVE-22647
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22647.1.patch, HIVE-22647.2.patch
>
>
> Non pooled session my leak when the client doesn't close the connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22647) enable session pool by default

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029359#comment-17029359
 ] 

Hive QA commented on HIVE-22647:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
36s{color} | {color:blue} common in master has 63 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 27s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20449/dev-support/hive-personality.sh
 |
| git revision | master / 842fdd2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common U: common |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20449/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> enable session pool by default
> --
>
> Key: HIVE-22647
> URL: https://issues.apache.org/jira/browse/HIVE-22647
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22647.1.patch, HIVE-22647.2.patch
>
>
> Non pooled session my leak when the client doesn't close the connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-02-03 Thread Ramesh Kumar Thangarajan (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029345#comment-17029345
 ] 

Ramesh Kumar Thangarajan commented on HIVE-22805:
-

Your patch looks good to me. Thank you for fixing this the right way!!

Is it ok to call shallowCopyTo from copySelected? Do you think it makes sense 
to implement shallowCopyTo() for classes inherited from 
MultiValuedColumnVector? Although we do not call shallowCopyTo() directly for 
Map or List, if we have a case with nested list like:
{code:java}
select IF(1=1, ARRAY(ARRAY("a", "b"),ARRAY("c", "d")), NULL)
{code}
, then we may throw some similar error since 
MultiValuedColumnVector::shallowCopyTo is not implemented – since we have the 
new ListColumnVector::copySelected calling the shallowCopyTo for its child and 
similarly for MapColumnVector too.

> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
>   at 
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>  
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22821) Add necessary endpoints for proactive cache eviction

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029335#comment-17029335
 ] 

Hive QA commented on HIVE-22821:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m 
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  3m 
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
49s{color} | {color:blue} common in master has 63 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
32s{color} | {color:blue} llap-common in master has 90 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
24s{color} | {color:blue} llap-client in master has 27 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m 
16s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
50s{color} | {color:blue} llap-server in master has 90 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
47s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
41s{color} | {color:red} ql: The patch generated 9 new + 2 unchanged - 0 fixed 
= 11 total (was 2) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
14s{color} | {color:red} llap-server: The patch generated 2 new + 54 unchanged 
- 0 fixed = 56 total (was 54) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 17 line(s) that end in whitespace. Use 
git apply --whitespace=fix <>. Refer 
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 
41s{color} | {color:red} llap-common generated 8 new + 90 unchanged - 0 fixed = 
98 total (was 90) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  4m  
8s{color} | {color:red} ql generated 3 new + 1533 unchanged - 0 fixed = 1536 
total (was 1533) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 
55s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100 
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
16s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 32s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:llap-common |
|  |  
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$EvictEntityRequestProto.PARSER
 isn't final but should be  At LlapDaemonProtocolProtos.java:be  At 
LlapDaemonProtocolProtos.java:[line 22695] |
|  |  Class 
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$EvictEntityRequestProto
 defines non-transient non-serializable instance field unknownFields  In 
LlapDaemonProtocolProtos.java:instance field unknownFields  In 
LlapDaemonProtocolProtos.java |
|  |  
org.apache.hadoop.hive.llap.daemon.rpc.LlapDaemonProtocolProtos$EvictEntityResponseProto.PARSER
 isn't final but should be  At LlapDaemonProtocolProtos.java:be  At 
LlapDaemonProtocolProtos.java:[line 24450] |
|  |  Class

[jira] [Resolved] (HIVE-22667) ORC version upgrade including ORC-578

2020-02-03 Thread David Lavati (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Lavati resolved HIVE-22667.
-
Resolution: Duplicate

> ORC version upgrade including ORC-578
> -
>
> Key: HIVE-22667
> URL: https://issues.apache.org/jira/browse/HIVE-22667
> Project: Hive
>  Issue Type: Bug
>Reporter: David Lavati
>Priority: Major
>
> This is a preemptive ticket for bumping the ORC version to something that 
> will include ORC-578:
> While working with ORC-578 and hive, the new orc dependency of threeten-extra 
> caused NoClassDefFoundError while running tez queries containing timestamps, 
> as some required classes from this dependency didn't get bundled into the fat 
> jars that are copied to tez clients.
>  
> To resolve this, threeten-extra has to be included in the hive-exec fat jar 
> in ql/pom.xml.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (HIVE-22254) Mappings.NoElementException: no target in mapping, in `MaterializedViewAggregateRule

2020-02-03 Thread Jesus Camacho Rodriguez (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029317#comment-17029317
 ] 

Jesus Camacho Rodriguez edited comment on HIVE-22254 at 2/3/20 9:46 PM:


It could... Though there may be two issues here: one is that the MV is not hit 
because we do not push the Aggregate through the Join (similar to HIVE-22262) 
and the other problem is the CBO crash (which I believe you are not seeing 
anymore, please confirm that it is not somehow hidden in the logs). If CBO is 
not crashing and Aggregate through Join will be tackled in HIVE-22262, we can 
close this one as duplicate and add the test as part of HIVE-22262.


was (Author: jcamachorodriguez):
It could... Though there may be two issues here: one is that the MV is not hit 
because we do not push the Aggregate through the Join (similar to HIVE-22262) 
and the other problem is the CBO crash (which I believe you are not seeing 
anymore). If CBO is not crashing and Aggregate through Join will be tackled in 
HIVE-22262, we can close this one as duplicate and add the test as part of 
HIVE-22262.

> Mappings.NoElementException: no target in mapping, in 
> `MaterializedViewAggregateRule
> 
>
> Key: HIVE-22254
> URL: https://issues.apache.org/jira/browse/HIVE-22254
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO, Materialized views
>Affects Versions: 3.1.2
>Reporter: Steve Carlin
>Priority: Minor
> Attachments: ojoin_full.sql
>
>
> A Mappings.NoElementException happens on an edge condition for a query using 
> a materialized view.
> The query contains a "group by" clause which contains fields from both sides 
> of a join.  There is no real reason to group by this same field twice, but 
> there is also no reason that this shouldn't succeed.
> Attached is a script which causes this failure.  The query causing the 
> problem looks like this:
> explain extended select sum(1)
> from fact inner join dim1
> on fact.f1 = dim1.pk1
> group by f1, pk1;



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22254) Mappings.NoElementException: no target in mapping, in `MaterializedViewAggregateRule

2020-02-03 Thread Jesus Camacho Rodriguez (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029317#comment-17029317
 ] 

Jesus Camacho Rodriguez commented on HIVE-22254:


It could... Though there may be two issues here: one is that the MV is not hit 
because we do not push the Aggregate through the Join (similar to HIVE-22262) 
and the other problem is the CBO crash (which I believe you are not seeing 
anymore). If CBO is not crashing and Aggregate through Join will be tackled in 
HIVE-22262, we can close this one as duplicate and add the test as part of 
HIVE-22262.

> Mappings.NoElementException: no target in mapping, in 
> `MaterializedViewAggregateRule
> 
>
> Key: HIVE-22254
> URL: https://issues.apache.org/jira/browse/HIVE-22254
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO, Materialized views
>Affects Versions: 3.1.2
>Reporter: Steve Carlin
>Priority: Minor
> Attachments: ojoin_full.sql
>
>
> A Mappings.NoElementException happens on an edge condition for a query using 
> a materialized view.
> The query contains a "group by" clause which contains fields from both sides 
> of a join.  There is no real reason to group by this same field twice, but 
> there is also no reason that this shouldn't succeed.
> Attached is a script which causes this failure.  The query causing the 
> problem looks like this:
> explain extended select sum(1)
> from fact inner join dim1
> on fact.f1 = dim1.pk1
> group by f1, pk1;



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22254) Mappings.NoElementException: no target in mapping, in `MaterializedViewAggregateRule

2020-02-03 Thread Vineet Garg (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029315#comment-17029315
 ] 

Vineet Garg commented on HIVE-22254:


[~jcamachorodriguez] No mv is not being hit, but I am curious should it have?

> Mappings.NoElementException: no target in mapping, in 
> `MaterializedViewAggregateRule
> 
>
> Key: HIVE-22254
> URL: https://issues.apache.org/jira/browse/HIVE-22254
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO, Materialized views
>Affects Versions: 3.1.2
>Reporter: Steve Carlin
>Priority: Minor
> Attachments: ojoin_full.sql
>
>
> A Mappings.NoElementException happens on an edge condition for a query using 
> a materialized view.
> The query contains a "group by" clause which contains fields from both sides 
> of a join.  There is no real reason to group by this same field twice, but 
> there is also no reason that this shouldn't succeed.
> Attached is a script which causes this failure.  The query causing the 
> problem looks like this:
> explain extended select sum(1)
> from fact inner join dim1
> on fact.f1 = dim1.pk1
> group by f1, pk1;



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22589) Add storage support for ProlepticCalendar in ORC, Parquet, and Avro

2020-02-03 Thread Jesus Camacho Rodriguez (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-22589:
---
Status: Patch Available  (was: In Progress)

> Add storage support for ProlepticCalendar in ORC, Parquet, and Avro
> ---
>
> Key: HIVE-22589
> URL: https://issues.apache.org/jira/browse/HIVE-22589
> Project: Hive
>  Issue Type: Bug
>  Components: Avro, ORC, Parquet
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 4.0.0, 3.2.0, 3.1.3
>
> Attachments: HIVE-22589.01.patch, HIVE-22589.02.patch, 
> HIVE-22589.03.patch, HIVE-22589.04.patch, HIVE-22589.05.patch, 
> HIVE-22589.patch, HIVE-22589.patch
>
>
> Hive recently moved its processing to the proleptic calendar, which has 
> created some issues for users who have dates before 1580 AD.
> HIVE-22405 extended the column vectors for times & dates to encode which 
> calendar they are using.
> This issue is to support proleptic calendar in ORC, Parquet, and Avro, when 
> files are written/read by Hive. To preserve compatibility with other engines 
> until they upgrade their readers, files will be written using hybrid calendar 
> by default. Default behavior when files do not contain calendar information 
> in their metadata is configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22589) Add storage support for ProlepticCalendar in ORC, Parquet, and Avro

2020-02-03 Thread Jesus Camacho Rodriguez (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-22589:
---
Attachment: HIVE-22589.05.patch

> Add storage support for ProlepticCalendar in ORC, Parquet, and Avro
> ---
>
> Key: HIVE-22589
> URL: https://issues.apache.org/jira/browse/HIVE-22589
> Project: Hive
>  Issue Type: Bug
>  Components: Avro, ORC, Parquet
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 4.0.0, 3.2.0, 3.1.3
>
> Attachments: HIVE-22589.01.patch, HIVE-22589.02.patch, 
> HIVE-22589.03.patch, HIVE-22589.04.patch, HIVE-22589.05.patch, 
> HIVE-22589.patch, HIVE-22589.patch
>
>
> Hive recently moved its processing to the proleptic calendar, which has 
> created some issues for users who have dates before 1580 AD.
> HIVE-22405 extended the column vectors for times & dates to encode which 
> calendar they are using.
> This issue is to support proleptic calendar in ORC, Parquet, and Avro, when 
> files are written/read by Hive. To preserve compatibility with other engines 
> until they upgrade their readers, files will be written using hybrid calendar 
> by default. Default behavior when files do not contain calendar information 
> in their metadata is configurable.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22254) Mappings.NoElementException: no target in mapping, in `MaterializedViewAggregateRule

2020-02-03 Thread Jesus Camacho Rodriguez (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029304#comment-17029304
 ] 

Jesus Camacho Rodriguez commented on HIVE-22254:


[~vgarg], is the materialized view hit? If this is not reproducible anymore, 
can we add the test attached to the JIRA and close this issue? Thanks

> Mappings.NoElementException: no target in mapping, in 
> `MaterializedViewAggregateRule
> 
>
> Key: HIVE-22254
> URL: https://issues.apache.org/jira/browse/HIVE-22254
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO, Materialized views
>Affects Versions: 3.1.2
>Reporter: Steve Carlin
>Priority: Minor
> Attachments: ojoin_full.sql
>
>
> A Mappings.NoElementException happens on an edge condition for a query using 
> a materialized view.
> The query contains a "group by" clause which contains fields from both sides 
> of a join.  There is no real reason to group by this same field twice, but 
> there is also no reason that this shouldn't succeed.
> Attached is a script which causes this failure.  The query causing the 
> problem looks like this:
> explain extended select sum(1)
> from fact inner join dim1
> on fact.f1 = dim1.pk1
> group by f1, pk1;



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22819) Refactor Hive::listFilesCreatedByQuery to make it faster for object stores

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029297#comment-17029297
 ] 

Hive QA commented on HIVE-22819:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992513/HIVE-22819.1.patch

{color:green}SUCCESS:{color} +1 due to 19 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17959 tests 
executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.cli.TestPermsGrp.testCustomPerms (batchId=216)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20447/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20447/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20447/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992513 - PreCommit-HIVE-Build

> Refactor Hive::listFilesCreatedByQuery to make it faster for object stores
> --
>
> Key: HIVE-22819
> URL: https://issues.apache.org/jira/browse/HIVE-22819
> Project: Hive
>  Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Attachments: HIVE-22819.1.patch
>
>
> {color:#ff}Hive::listFilesCreatedByQuery{color} does an exists(), an 
> isDir() and then a listing call. This can be expensive in object stores. We 
> should instead directly list the files in the directory (we'd have to handle 
> an exception if the directory does not exists, but issuing a single call to 
> the object store would most likely still end up being more performant). 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-21341) Sensible defaults : hive.server2.idle.operation.timeout and hive.server2.idle.session.timeout are too high

2020-02-03 Thread Ashutosh Chauhan (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-21341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-21341:

Fix Version/s: 4.0.0
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

Pushed to master.

> Sensible defaults : hive.server2.idle.operation.timeout and 
> hive.server2.idle.session.timeout are too high
> --
>
> Key: HIVE-21341
> URL: https://issues.apache.org/jira/browse/HIVE-21341
> Project: Hive
>  Issue Type: Improvement
>  Components: Configuration, HiveServer2
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21341.1.patch, HIVE-21341.2.patch, HIVE-21341.patch
>
>
> Defaults are too high, which results in extra resources being held too long 
> in HS2 memory.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22647) enable session pool by default

2020-02-03 Thread Ashutosh Chauhan (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-22647:

Attachment: HIVE-22647.2.patch

> enable session pool by default
> --
>
> Key: HIVE-22647
> URL: https://issues.apache.org/jira/browse/HIVE-22647
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22647.1.patch, HIVE-22647.2.patch
>
>
> Non pooled session my leak when the client doesn't close the connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22647) enable session pool by default

2020-02-03 Thread Ashutosh Chauhan (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-22647:

Status: Patch Available  (was: Open)

> enable session pool by default
> --
>
> Key: HIVE-22647
> URL: https://issues.apache.org/jira/browse/HIVE-22647
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22647.1.patch, HIVE-22647.2.patch
>
>
> Non pooled session my leak when the client doesn't close the connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22647) enable session pool by default

2020-02-03 Thread Ashutosh Chauhan (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-22647:

Status: Open  (was: Patch Available)

> enable session pool by default
> --
>
> Key: HIVE-22647
> URL: https://issues.apache.org/jira/browse/HIVE-22647
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22647.1.patch, HIVE-22647.2.patch
>
>
> Non pooled session my leak when the client doesn't close the connection.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22819) Refactor Hive::listFilesCreatedByQuery to make it faster for object stores

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029251#comment-17029251
 ] 

Hive QA commented on HIVE-22819:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
38s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
30s{color} | {color:blue} hcatalog/streaming in master has 11 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
27s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m  
9s{color} | {color:red} ql: The patch generated 47 new + 2707 unchanged - 24 
fixed = 2754 total (was 2731) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
21s{color} | {color:red} itests/hive-unit: The patch generated 22 new + 177 
unchanged - 21 fixed = 199 total (was 198) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  4m  
6s{color} | {color:red} ql generated 1 new + 1532 unchanged - 1 fixed = 1533 
total (was 1533) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 33s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
|  |  Dead store to filename in 
org.apache.hadoop.hive.ql.io.AcidUtils.parseBucketId(Path)  At 
AcidUtils.java:org.apache.hadoop.hive.ql.io.AcidUtils.parseBucketId(Path)  At 
AcidUtils.java:[line 434] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20447/dev-support/hive-personality.sh
 |
| git revision | master / f83bfbe |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20447/yetus/diff-checkstyle-ql.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20447/yetus/diff-checkstyle-itests_hive-unit.txt
 |
| findbugs | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20447/yetus/new-findbugs-ql.html
 |
| modules | C: ql hcatalog/streaming streaming itests/hive-unit U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20447/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically

[jira] [Commented] (HIVE-22818) Preparation for jetty 9.4.26 upgrade

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029225#comment-17029225
 ] 

Hive QA commented on HIVE-22818:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992499/HIVE-22818.03.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 17959 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20446/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20446/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20446/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992499 - PreCommit-HIVE-Build

> Preparation for jetty 9.4.26 upgrade
> 
>
> Key: HIVE-22818
> URL: https://issues.apache.org/jira/browse/HIVE-22818
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22818.01.patch, HIVE-22818.02.patch, 
> HIVE-22818.03.patch
>
>
> Make some code adjustment, before upgrading jetty to 9.4.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22818) Preparation for jetty 9.4.26 upgrade

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029180#comment-17029180
 ] 

Hive QA commented on HIVE-22818:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  9m 
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
38s{color} | {color:blue} service in master has 51 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m  4s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20446/dev-support/hive-personality.sh
 |
| git revision | master / f83bfbe |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: service U: service |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20446/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Preparation for jetty 9.4.26 upgrade
> 
>
> Key: HIVE-22818
> URL: https://issues.apache.org/jira/browse/HIVE-22818
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22818.01.patch, HIVE-22818.02.patch, 
> HIVE-22818.03.patch
>
>
> Make some code adjustment, before upgrading jetty to 9.4.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22254) Mappings.NoElementException: no target in mapping, in `MaterializedViewAggregateRule

2020-02-03 Thread Vineet Garg (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029178#comment-17029178
 ] 

Vineet Garg commented on HIVE-22254:


[~scarlin] I am unable to reproduce this on upstream using the attached repro. 
Is this still an issue?

> Mappings.NoElementException: no target in mapping, in 
> `MaterializedViewAggregateRule
> 
>
> Key: HIVE-22254
> URL: https://issues.apache.org/jira/browse/HIVE-22254
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO, Materialized views
>Affects Versions: 3.1.2
>Reporter: Steve Carlin
>Priority: Minor
> Attachments: ojoin_full.sql
>
>
> A Mappings.NoElementException happens on an edge condition for a query using 
> a materialized view.
> The query contains a "group by" clause which contains fields from both sides 
> of a join.  There is no real reason to group by this same field twice, but 
> there is also no reason that this shouldn't succeed.
> Attached is a script which causes this failure.  The query causing the 
> problem looks like this:
> explain extended select sum(1)
> from fact inner join dim1
> on fact.f1 = dim1.pk1
> group by f1, pk1;



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22795) Create new parser and udf module from ql

2020-02-03 Thread Jesus Camacho Rodriguez (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-22795:
---
Fix Version/s: 4.0.0
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

Pushed to master, thanks [~vgarg], [~kgyrtkirk]

> Create new parser and udf module from ql
> 
>
> Key: HIVE-22795
> URL: https://issues.apache.org/jira/browse/HIVE-22795
> Project: Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22795.01.patch, HIVE-22795.02.patch, 
> HIVE-22795.03.patch, HIVE-22795.patch
>
>
> ql is a huge module. I propose to start splitting it by creating new module 
> `parser` and `udf` to encapsulate some classes related to SQL parsing and UDF 
> declaration, respectively.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22809) Support materialized view rebuild as a scheduled query

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029164#comment-17029164
 ] 

Hive QA commented on HIVE-22809:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992498/HIVE-22809.02.patch

{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17960 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.schq.TestScheduledQueryIntegration.testScheduledQueryExecutionImpersonation
 (batchId=282)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20445/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20445/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20445/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992498 - PreCommit-HIVE-Build

> Support materialized view rebuild as a scheduled query
> --
>
> Key: HIVE-22809
> URL: https://issues.apache.org/jira/browse/HIVE-22809
> Project: Hive
>  Issue Type: Sub-task
>  Components: Materialized views
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22809.01.patch, HIVE-22809.02.patch
>
>
> scheduled queries build heavily on unparsing the original query - to free it 
> from context free stuff; meanwhile the mv rebuild replaces the query with a 
> new command



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22795) Create new parser and udf module from ql

2020-02-03 Thread Vineet Garg (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029137#comment-17029137
 ] 

Vineet Garg commented on HIVE-22795:


I went over it and it looks good to me. +1

> Create new parser and udf module from ql
> 
>
> Key: HIVE-22795
> URL: https://issues.apache.org/jira/browse/HIVE-22795
> Project: Hive
>  Issue Type: Improvement
>  Components: Build Infrastructure
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22795.01.patch, HIVE-22795.02.patch, 
> HIVE-22795.03.patch, HIVE-22795.patch
>
>
> ql is a huge module. I propose to start splitting it by creating new module 
> `parser` and `udf` to encapsulate some classes related to SQL parsing and UDF 
> declaration, respectively.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22578) CBO genOPTree is not failsafe for CTAS and VIEW statements

2020-02-03 Thread Jesus Camacho Rodriguez (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029122#comment-17029122
 ] 

Jesus Camacho Rodriguez commented on HIVE-22578:


[~hamvas.aron], if there is a bug in CBO, we need to fix it. Copying the AST 
when we start optimization is just a workaround that harms compilation 
performance badly.

Which error were you observing? This patch does not contain a test to reproduce 
it.

> CBO genOPTree is not failsafe for CTAS and VIEW statements
> --
>
> Key: HIVE-22578
> URL: https://issues.apache.org/jira/browse/HIVE-22578
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Affects Versions: 3.0.0, 3.1.2
>Reporter: Aron Hamvas
>Assignee: Aron Hamvas
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22578.2.patch, HIVE-22578.patch
>
>
> If CBO fails during OT generation, it is supposed to skip CBO and fall back 
> to SemanticAnalyzer generating the OT for the original AST. However, for CTAS 
> and VIEW statements, the original AST is discarded and replaced with the new 
> AST. 
> If OT generation for the new AST fails with a SemanticException, the 
> compilation is destined to fail completely since we cannot fall back to the 
> original and possibly valid AST.
> Also, the SemanticException sometimes gets hidden and misjudged as a missing 
> column statistics issue due to bad error handling.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22809) Support materialized view rebuild as a scheduled query

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029117#comment-17029117
 ] 

Hive QA commented on HIVE-22809:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  2m  
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m 
11s{color} | {color:blue} ql in master has 1537 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
46s{color} | {color:blue} itests/util in master has 53 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
41s{color} | {color:red} ql: The patch generated 1 new + 100 unchanged - 1 
fixed = 101 total (was 101) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} itests/util: The patch generated 0 new + 18 
unchanged - 1 fixed = 18 total (was 19) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  5m  
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  1m  
1s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total 
(was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 21s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20445/dev-support/hive-personality.sh
 |
| git revision | master / 5acbffe |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20445/yetus/diff-checkstyle-ql.txt
 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20445/yetus/whitespace-eol.txt
 |
| javadoc | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20445/yetus/diff-javadoc-javadoc-ql.txt
 |
| modules | C: ql itests itests/util U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20445/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Support materialized view rebuild as a scheduled query
> --
>
> Key: HIVE-22809
> URL: https://issues.apache.org/jira/browse/HIVE-22809
> Project: Hive
>  Issue Type: Sub-task
>  Components: Materialized views
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22809.01.patch, HIVE-22809.02.patch
>
>
> scheduled queries build heavily on unparsing the

[jira] [Comment Edited] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Javi Roman (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029111#comment-17029111
 ] 

Javi Roman edited comment on HIVE-22817 at 2/3/20 5:28 PM:
---

[~eilmrein] Hive execute the queries with an "execution engine", this execution 
engine is a mapreduce on top of HDFS, you can select Apache Spark or Apache Tez 
as execution engine. If your are setting up from scratch Apache Hive (without 
using Cloudera or Hortonworks) you need to take a look to Apache Spark (using 
YARN or Mesos)  or Apache Tez (a YARN application on HDFS).


was (Author: javiroman):
[~eilmrein] Hive execute the queries with an "execution engine", this execution 
engine a mapreduce on top of HDFS, you can select Apache Spark or Apache Tez as 
execution engine. If your are setting up from scratch Apache Hive (without 
using Cloudera or Hortonworks) you need to take a look to Apache Spark (using 
YARN or Mesos)  or Apache Tez (a YARN application on HDFS).

> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz, hive.log
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Javi Roman (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029111#comment-17029111
 ] 

Javi Roman commented on HIVE-22817:
---

[~eilmrein] Hive execute the queries with an "execution engine", this execution 
engine a mapreduce on top of HDFS, you can select Apache Spark or Apache Tez as 
execution engine. If your are setting up from scratch Apache Hive (without 
using Cloudera or Hortonworks) you need to take a look to Apache Spark (using 
YARN or Mesos)  or Apache Tez (a YARN application on HDFS).

> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz, hive.log
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22775) Use the qt:authorizer option in qtests

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029089#comment-17029089
 ] 

Hive QA commented on HIVE-22775:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992497/HIVE-22775.03.patch

{color:green}SUCCESS:{color} +1 due to 85 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17959 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[url_hook] 
(batchId=298)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20444/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20444/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20444/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992497 - PreCommit-HIVE-Build

> Use the qt:authorizer option in qtests
> --
>
> Key: HIVE-22775
> URL: https://issues.apache.org/jira/browse/HIVE-22775
> Project: Hive
>  Issue Type: Improvement
>  Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22775.01.patch, HIVE-22775.02.patch, 
> HIVE-22775.02.patch, HIVE-22775.03.patch
>
>
> qt:authorizer sets up things for auth tests; [~mgergely] suggested to use it 
> in existing tests 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22819) Refactor Hive::listFilesCreatedByQuery to make it faster for object stores

2020-02-03 Thread Marton Bod (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marton Bod updated HIVE-22819:
--
Attachment: HIVE-22819.1.patch
Status: Patch Available  (was: Open)

> Refactor Hive::listFilesCreatedByQuery to make it faster for object stores
> --
>
> Key: HIVE-22819
> URL: https://issues.apache.org/jira/browse/HIVE-22819
> Project: Hive
>  Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Attachments: HIVE-22819.1.patch
>
>
> {color:#ff}Hive::listFilesCreatedByQuery{color} does an exists(), an 
> isDir() and then a listing call. This can be expensive in object stores. We 
> should instead directly list the files in the directory (we'd have to handle 
> an exception if the directory does not exists, but issuing a single call to 
> the object store would most likely still end up being more performant). 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22775) Use the qt:authorizer option in qtests

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029064#comment-17029064
 ] 

Hive QA commented on HIVE-22775:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
53s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m  
1s{color} | {color:blue} ql in master has 1537 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
50s{color} | {color:blue} itests/util in master has 53 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
14s{color} | {color:red} itests/util: The patch generated 1 new + 18 unchanged 
- 1 fixed = 19 total (was 19) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  5m  
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 21s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20444/dev-support/hive-personality.sh
 |
| git revision | master / 5acbffe |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20444/yetus/diff-checkstyle-itests_util.txt
 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20444/yetus/whitespace-eol.txt
 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20444/yetus/whitespace-tabs.txt
 |
| modules | C: ql itests/util U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20444/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Use the qt:authorizer option in qtests
> --
>
> Key: HIVE-22775
> URL: https://issues.apache.org/jira/browse/HIVE-22775
> Project: Hive
>  Issue Type: Improvement
>  Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22775.01.patch, HIVE-22775.02.patch, 
> HIVE-22775.02.patch, HIVE-22775.03.patch
>
>
> qt:authorizer sets up things for auth tests; [~mgergely] suggested to use it 
> in existing tests 



--
This message was sent by Atlassian Jira

[jira] [Commented] (HIVE-21737) Upgrade Avro to version 1.9.1

2020-02-03 Thread David Mollitor (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029052#comment-17029052
 ] 

David Mollitor commented on HIVE-21737:
---

Support for Hive-on-Spark is being dropped by Cloudera.  Perhaps the first 
order of business is simply to drop it from Hive.  That may pave the way for 
upgrading Avro.

> Upgrade Avro to version 1.9.1
> -
>
> Key: HIVE-21737
> URL: https://issues.apache.org/jira/browse/HIVE-21737
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Ismaël Mejía
>Assignee: Fokko Driesprong
>Priority: Major
>  Labels: pull-request-available
> Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner 
> version of Avro without Jackson in the public API. Worth the update.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22814) ArrayIndexOutOfBound in the vectorization getDataTypePhysicalVariation

2020-02-03 Thread Ashutosh Chauhan (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029049#comment-17029049
 ] 

Ashutosh Chauhan commented on HIVE-22814:
-

[~rameshkumar] can you please describe what the issue was and how does this fix 
it? 
Can you please also add a testcase for the bug?

> ArrayIndexOutOfBound in the vectorization getDataTypePhysicalVariation
> --
>
> Key: HIVE-22814
> URL: https://issues.apache.org/jira/browse/HIVE-22814
> Project: Hive
>  Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22814.1.patch
>
>
> ArrayIndexOutOfBound in the vectorization getDataTypePhysicalVariation



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22821) Add necessary endpoints for proactive cache eviction

2020-02-03 Thread Jira



 [ 
https://issues.apache.org/jira/browse/HIVE-22821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ádám Szita updated HIVE-22821:
--
Status: Patch Available  (was: Open)

> Add necessary endpoints for proactive cache eviction
> 
>
> Key: HIVE-22821
> URL: https://issues.apache.org/jira/browse/HIVE-22821
> Project: Hive
>  Issue Type: Sub-task
>  Components: llap
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22821.0.patch
>
>
> Implement the parts required for iHS2 -> LLAP daemons communication:
>  * protobuf message schema and endpoints
>  * Hive configuration
>  * for use cases:
>  ** dropping db
>  ** dropping table
>  ** dropping partition from a table



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22821) Add necessary endpoints for proactive cache eviction

2020-02-03 Thread Jira



 [ 
https://issues.apache.org/jira/browse/HIVE-22821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ádám Szita updated HIVE-22821:
--
Attachment: HIVE-22821.0.patch

> Add necessary endpoints for proactive cache eviction
> 
>
> Key: HIVE-22821
> URL: https://issues.apache.org/jira/browse/HIVE-22821
> Project: Hive
>  Issue Type: Sub-task
>  Components: llap
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22821.0.patch
>
>
> Implement the parts required for iHS2 -> LLAP daemons communication:
>  * protobuf message schema and endpoints
>  * Hive configuration
>  * for use cases:
>  ** dropping db
>  ** dropping table
>  ** dropping partition from a table



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-22821) Add necessary endpoints for proactive cache eviction

2020-02-03 Thread Jira



 [ 
https://issues.apache.org/jira/browse/HIVE-22821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ádám Szita reassigned HIVE-22821:
-


> Add necessary endpoints for proactive cache eviction
> 
>
> Key: HIVE-22821
> URL: https://issues.apache.org/jira/browse/HIVE-22821
> Project: Hive
>  Issue Type: Sub-task
>  Components: llap
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
>
> Implement the parts required for iHS2 -> LLAP daemons communication:
>  * protobuf message schema and endpoints
>  * Hive configuration
>  * for use cases:
>  ** dropping db
>  ** dropping table
>  ** dropping partition from a table



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-22820) Proactive LLAP cache eviction

2020-02-03 Thread Jira



 [ 
https://issues.apache.org/jira/browse/HIVE-22820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ádám Szita reassigned HIVE-22820:
-


> Proactive LLAP cache eviction
> -
>
> Key: HIVE-22820
> URL: https://issues.apache.org/jira/browse/HIVE-22820
> Project: Hive
>  Issue Type: Improvement
>  Components: llap
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
>
> Currently LLAP cache eviction is only based on an eviction policy, that is 
> reactive in a way that it only evicts buffers whenever there's space required 
> for new ones.
> It does not take for example dropped entities (partitions, tables or even 
> dbs) into consideration. We should also implement a proactive side of the 
> eviction that does this for us, and in turn could increase overall cache hit 
> ratios.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-21215) Read Parquet INT64 timestamp

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-21215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029033#comment-17029033
 ] 

Hive QA commented on HIVE-21215:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992488/HIVE-21215.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 17965 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20443/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20443/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20443/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992488 - PreCommit-HIVE-Build

> Read Parquet INT64 timestamp
> 
>
> Key: HIVE-21215
> URL: https://issues.apache.org/jira/browse/HIVE-21215
> Project: Hive
>  Issue Type: New Feature
>Reporter: Karen Coppage
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-21215.1.patch, HIVE-21215.2.patch
>
>
> This patch enables Hive to start reading timestamps from Parquet written with 
> the new semantics:
> With Parquet version 1.11, a new timestamp LogicalType with base INT64 and 
> the following metadata is introduced:
>  * boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC 
> (aka Instant semantics) or not (LocalDateTime semantics).
>  * enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp
> Upon reading, the semantics of these new timestamps will be determined by 
> their metadata, while the semantics of INT96 timestamps will continue to be 
> deduced from the writer metadata.
>  This feature will be behind a flag for now.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-21215) Read Parquet INT64 timestamp

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-21215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028992#comment-17028992
 ] 

Hive QA commented on HIVE-21215:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  2m  
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
35s{color} | {color:blue} common in master has 63 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m 
13s{color} | {color:blue} ql in master has 1537 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
41s{color} | {color:red} ql: The patch generated 5 new + 18 unchanged - 1 fixed 
= 23 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m  8s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20443/dev-support/hive-personality.sh
 |
| git revision | master / 5acbffe |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20443/yetus/diff-checkstyle-ql.txt
 |
| modules | C: common ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20443/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Read Parquet INT64 timestamp
> 
>
> Key: HIVE-21215
> URL: https://issues.apache.org/jira/browse/HIVE-21215
> Project: Hive
>  Issue Type: New Feature
>Reporter: Karen Coppage
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-21215.1.patch, HIVE-21215.2.patch
>
>
> This patch enables Hive to start reading timestamps from Parquet written with 
> the new semantics:
> With Parquet version 1.11, a new timestamp LogicalType with base INT64 and 
> the following metadata is introduced:
>  * boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC 
> (aka Instant semantics) or not (LocalDateTime semantics).
>  * enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp
> Upon reading, the semantics of these new timestamps will be determined by 
> their metadata, while the semantics of INT96 timestamps will continue to be 
> deduced from the writer metadata.
>  This feature will be behind a flag for now.



--
This message was sent by Atlassian

[jira] [Updated] (HIVE-22818) Preparation for jetty 9.4.26 upgrade

2020-02-03 Thread Laszlo Pinter (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laszlo Pinter updated HIVE-22818:
-
Attachment: HIVE-22818.03.patch

> Preparation for jetty 9.4.26 upgrade
> 
>
> Key: HIVE-22818
> URL: https://issues.apache.org/jira/browse/HIVE-22818
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22818.01.patch, HIVE-22818.02.patch, 
> HIVE-22818.03.patch
>
>
> Make some code adjustment, before upgrading jetty to 9.4.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22818) Preparation for jetty 9.4.26 upgrade

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028961#comment-17028961
 ] 

Hive QA commented on HIVE-22818:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992486/HIVE-22818.02.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17960 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2] 
(batchId=91)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.org.apache.hadoop.hive.ql.TestWarehouseExternalDir
 (batchId=279)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testExternalDefaultPaths 
(batchId=279)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20442/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20442/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20442/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992486 - PreCommit-HIVE-Build

> Preparation for jetty 9.4.26 upgrade
> 
>
> Key: HIVE-22818
> URL: https://issues.apache.org/jira/browse/HIVE-22818
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22818.01.patch, HIVE-22818.02.patch
>
>
> Make some code adjustment, before upgrading jetty to 9.4.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-22819) Refactor Hive::listFilesCreatedByQuery to make it faster for object stores

2020-02-03 Thread Marton Bod (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marton Bod reassigned HIVE-22819:
-


> Refactor Hive::listFilesCreatedByQuery to make it faster for object stores
> --
>
> Key: HIVE-22819
> URL: https://issues.apache.org/jira/browse/HIVE-22819
> Project: Hive
>  Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
>
> {color:#ff}Hive::listFilesCreatedByQuery{color} does an exists(), an 
> isDir() and then a listing call. This can be expensive in object stores. We 
> should instead directly list the files in the directory (we'd have to handle 
> an exception if the directory does not exists, but issuing a single call to 
> the object store would most likely still end up being more performant). 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22809) Support materialized view rebuild as a scheduled query

2020-02-03 Thread Zoltan Haindrich (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-22809:

Attachment: HIVE-22809.02.patch

> Support materialized view rebuild as a scheduled query
> --
>
> Key: HIVE-22809
> URL: https://issues.apache.org/jira/browse/HIVE-22809
> Project: Hive
>  Issue Type: Sub-task
>  Components: Materialized views
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22809.01.patch, HIVE-22809.02.patch
>
>
> scheduled queries build heavily on unparsing the original query - to free it 
> from context free stuff; meanwhile the mv rebuild replaces the query with a 
> new command



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22775) Use the qt:authorizer option in qtests

2020-02-03 Thread Zoltan Haindrich (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-22775:

Attachment: HIVE-22775.03.patch

> Use the qt:authorizer option in qtests
> --
>
> Key: HIVE-22775
> URL: https://issues.apache.org/jira/browse/HIVE-22775
> Project: Hive
>  Issue Type: Improvement
>  Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22775.01.patch, HIVE-22775.02.patch, 
> HIVE-22775.02.patch, HIVE-22775.03.patch
>
>
> qt:authorizer sets up things for auth tests; [~mgergely] suggested to use it 
> in existing tests 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64

2020-02-03 Thread Zhenyu Zheng (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028914#comment-17028914
 ] 

Zhenyu Zheng commented on HIVE-21939:
-

github PR added [https://github.com/apache/hive/pull/893]

> protoc:2.5.0  dependence has broken building on aarch64
> ---
>
> Key: HIVE-21939
> URL: https://issues.apache.org/jira/browse/HIVE-21939
> Project: Hive
>  Issue Type: Bug
>Reporter: liusheng
>Assignee: Zhenyu Zheng
>Priority: Blocker
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When I try to build master of Hive from source code on "aarch64" server, I 
> met following error:
> [ERROR] Failed to execute goal 
> com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project 
> hive-standalone-metastore-common: Error resolving artifact: 
> com.google.protobuf:protoc:2.5.0: Could not find artifact 
> com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central 
> ([https://repo.maven.apache.org/maven2)]
> that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required 
> artifact, which does not have released package for "aarch64" platform.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64

2020-02-03 Thread Zhenyu Zheng (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-21939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenyu Zheng reassigned HIVE-21939:
---

Assignee: Zhenyu Zheng

> protoc:2.5.0  dependence has broken building on aarch64
> ---
>
> Key: HIVE-21939
> URL: https://issues.apache.org/jira/browse/HIVE-21939
> Project: Hive
>  Issue Type: Bug
>Reporter: liusheng
>Assignee: Zhenyu Zheng
>Priority: Blocker
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When I try to build master of Hive from source code on "aarch64" server, I 
> met following error:
> [ERROR] Failed to execute goal 
> com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project 
> hive-standalone-metastore-common: Error resolving artifact: 
> com.google.protobuf:protoc:2.5.0: Could not find artifact 
> com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central 
> ([https://repo.maven.apache.org/maven2)]
> that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required 
> artifact, which does not have released package for "aarch64" platform.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (HIVE-21939) protoc:2.5.0 dependence has broken building on aarch64

2020-02-03 Thread ASF GitHub Bot (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-21939?focusedWorklogId=380924=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-380924
 ]

ASF GitHub Bot logged work on HIVE-21939:
-

Author: ASF GitHub Bot
Created on: 03/Feb/20 12:48
Start Date: 03/Feb/20 12:48
Worklog Time Spent: 10m 
  Work Description: ZhengZhenyu commented on pull request #893: HIVE-21939: 
Switch to use protocbuf version 3.7.1 instead of 2.5.0
URL: https://github.com/apache/hive/pull/893
 
 
   protobuf 2.5.0 does not support Aarch64 platform and is outdated in some 
distro, Hadoop has the same problem and had upgraded to protobuf to 3.7.1. Do 
the same for Hive.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 380924)
Time Spent: 0.5h  (was: 20m)

> protoc:2.5.0  dependence has broken building on aarch64
> ---
>
> Key: HIVE-21939
> URL: https://issues.apache.org/jira/browse/HIVE-21939
> Project: Hive
>  Issue Type: Bug
>Reporter: liusheng
>Priority: Blocker
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When I try to build master of Hive from source code on "aarch64" server, I 
> met following error:
> [ERROR] Failed to execute goal 
> com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project 
> hive-standalone-metastore-common: Error resolving artifact: 
> com.google.protobuf:protoc:2.5.0: Could not find artifact 
> com.google.protobuf:protoc:exe:linux-aarch_64:2.5.0 in central 
> ([https://repo.maven.apache.org/maven2)]
> that is because Hive using the "com.google.protobuf:protoc:2.5.0" as required 
> artifact, which does not have released package for "aarch64" platform.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22818) Preparation for jetty 9.4.26 upgrade

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028908#comment-17028908
 ] 

Hive QA commented on HIVE-22818:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  9m 
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
39s{color} | {color:blue} service in master has 51 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 58s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20442/dev-support/hive-personality.sh
 |
| git revision | master / 5acbffe |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: service U: service |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20442/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Preparation for jetty 9.4.26 upgrade
> 
>
> Key: HIVE-22818
> URL: https://issues.apache.org/jira/browse/HIVE-22818
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22818.01.patch, HIVE-22818.02.patch
>
>
> Make some code adjustment, before upgrading jetty to 9.4.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-21215) Read Parquet INT64 timestamp

2020-02-03 Thread Marta Kuczora (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-21215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marta Kuczora updated HIVE-21215:
-
Attachment: HIVE-21215.2.patch

> Read Parquet INT64 timestamp
> 
>
> Key: HIVE-21215
> URL: https://issues.apache.org/jira/browse/HIVE-21215
> Project: Hive
>  Issue Type: New Feature
>Reporter: Karen Coppage
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-21215.1.patch, HIVE-21215.2.patch
>
>
> This patch enables Hive to start reading timestamps from Parquet written with 
> the new semantics:
> With Parquet version 1.11, a new timestamp LogicalType with base INT64 and 
> the following metadata is introduced:
>  * boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC 
> (aka Instant semantics) or not (LocalDateTime semantics).
>  * enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp
> Upon reading, the semantics of these new timestamps will be determined by 
> their metadata, while the semantics of INT96 timestamps will continue to be 
> deduced from the writer metadata.
>  This feature will be behind a flag for now.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22818) Preparation for jetty 9.4.26 upgrade

2020-02-03 Thread Laszlo Pinter (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laszlo Pinter updated HIVE-22818:
-
Attachment: HIVE-22818.02.patch

> Preparation for jetty 9.4.26 upgrade
> 
>
> Key: HIVE-22818
> URL: https://issues.apache.org/jira/browse/HIVE-22818
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22818.01.patch, HIVE-22818.02.patch
>
>
> Make some code adjustment, before upgrading jetty to 9.4.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22818) Preparation for jetty 9.4.26 upgrade

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028884#comment-17028884
 ] 

Hive QA commented on HIVE-22818:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992472/HIVE-22818.01.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17959 tests 
executed
*Failed tests:*
{noformat}
org.apache.hive.service.TestDFSErrorHandling.testAccessDenied (batchId=286)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20441/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20441/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20441/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992472 - PreCommit-HIVE-Build

> Preparation for jetty 9.4.26 upgrade
> 
>
> Key: HIVE-22818
> URL: https://issues.apache.org/jira/browse/HIVE-22818
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22818.01.patch
>
>
> Make some code adjustment, before upgrading jetty to 9.4.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Reinhard Eilmsteiner (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028883#comment-17028883
 ] 

Reinhard Eilmsteiner commented on HIVE-22817:
-

Sorry for my ignorance, but what is tez? No mention in the getting started 
guide.

> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz, hive.log
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Zoltan Haindrich (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028876#comment-17028876
 ] 

Zoltan Haindrich edited comment on HIVE-22817 at 2/3/20 11:11 AM:
--

I think you should put the tez jars onto the classpath; especially tez-api.jar
but...without tez being functional; hs2 used to start up...



was (Author: kgyrtkirk):
I think you should put the tez jars onto the classpath; especially tez-api.jar


> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz, hive.log
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Zoltan Haindrich (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028876#comment-17028876
 ] 

Zoltan Haindrich commented on HIVE-22817:
-

I think you should put the tez jars onto the classpath; especially tez-api.jar


> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz, hive.log
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22818) Preparation for jetty 9.4.26 upgrade

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028849#comment-17028849
 ] 

Hive QA commented on HIVE-22818:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
36s{color} | {color:blue} service in master has 51 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 57s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20441/dev-support/hive-personality.sh
 |
| git revision | master / 5acbffe |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: service U: service |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20441/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Preparation for jetty 9.4.26 upgrade
> 
>
> Key: HIVE-22818
> URL: https://issues.apache.org/jira/browse/HIVE-22818
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22818.01.patch
>
>
> Make some code adjustment, before upgrading jetty to 9.4.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Reinhard Eilmsteiner (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028847#comment-17028847
 ] 

Reinhard Eilmsteiner edited comment on HIVE-22817 at 2/3/20 10:31 AM:
--

Thx Zoltan. Found the first error: problems with accessing hdfs:/tmp

Now i Get another error java.lang.NoClassDefFoundError: 
org/apache/tez/dag/api/TezConfiguration

Attaching the log (now that I know where t o find it)


was (Author: eilmrein):
Thx Zoltan. Found the first error: problems with accessing hdfs:/tmp

Now i Get another error java.lang.NoClassDefFoundError: 
org/apache/tez/dag/api/TezConfiguration

Attaching the loh

> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz, hive.log
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Reinhard Eilmsteiner (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reinhard Eilmsteiner updated HIVE-22817:

Attachment: hive.log

> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz, hive.log
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Reinhard Eilmsteiner (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028847#comment-17028847
 ] 

Reinhard Eilmsteiner commented on HIVE-22817:
-

Thx Zoltan. Found the first error: problems with accessing hdfs:/tmp

Now i Get another error java.lang.NoClassDefFoundError: 
org/apache/tez/dag/api/TezConfiguration

Attaching the loh

> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz, hive.log
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028838#comment-17028838
 ] 

Hive QA commented on HIVE-22601:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992470/HIVE-22601.3.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 17960 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20440/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20440/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20440/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992470 - PreCommit-HIVE-Build

> Some columns will be lost when a UDTF has multiple aliases in some cases
> 
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 2.1.1, 2.2.0, 3.1.2, 2.3.6
>Reporter: okumin
>Assignee: Owen O'Malley
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22601.1.patch, HIVE-22601.2.patch, 
> HIVE-22601.3.patch, HIVE-22601.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Only one column will be retained when putting UDTFs with multiple aliases and 
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1, 
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3  |
> +-+
> | c   |
> | f   |
> +-+
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Zoltan Haindrich (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028825#comment-17028825
 ] 

Zoltan Haindrich commented on HIVE-22817:
-

have you also checked wether there is a log file at 
/tmp/__user_running_hive__/hive.log ?
I'm not sure why are some template files in the archive

if you want a quick way to experiment with hive; you might want to give a try 
to:
https://github.com/kgyrtkirk/hive-dev-box

> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22729) Provide a failure reason for failed compactions

2020-02-03 Thread Laszlo Pinter (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028807#comment-17028807
 ] 

Laszlo Pinter commented on HIVE-22729:
--

Thank you [~pvary], [~klcopp] [~dkuzmenko]!

> Provide a failure reason for failed compactions
> ---
>
> Key: HIVE-22729
> URL: https://issues.apache.org/jira/browse/HIVE-22729
> Project: Hive
>  Issue Type: Improvement
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22729.01.patch, HIVE-22729.02.patch, 
> HIVE-22729.03.patch, HIVE-22729.04.patch, HIVE-22729.05.patch, 
> HIVE-22729.06.patch, HIVE-22729.07.patch, HIVE-22729.08.patch, 
> HIVE-22729.09.patch, HIVE-22729.10.patch, HIVE-22729.11.patch, 
> HIVE-22729.12.patch, HIVE-22729.13.patch
>
>
> We should provide a compaction failure reason as easily accessible as 
> possible. Like in the result of the {{SHOW COMPACTIONS}} command.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22729) Provide a failure reason for failed compactions

2020-02-03 Thread Peter Vary (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Vary updated HIVE-22729:
--
Fix Version/s: 4.0.0
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

Pushed to master.
Thanks for the patch [~lpinter], and [~klcopp], [~dkuzmenko] for the review!

CC: [~thejas], [~ngangam], as this is HMS related too

> Provide a failure reason for failed compactions
> ---
>
> Key: HIVE-22729
> URL: https://issues.apache.org/jira/browse/HIVE-22729
> Project: Hive
>  Issue Type: Improvement
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22729.01.patch, HIVE-22729.02.patch, 
> HIVE-22729.03.patch, HIVE-22729.04.patch, HIVE-22729.05.patch, 
> HIVE-22729.06.patch, HIVE-22729.07.patch, HIVE-22729.08.patch, 
> HIVE-22729.09.patch, HIVE-22729.10.patch, HIVE-22729.11.patch, 
> HIVE-22729.12.patch, HIVE-22729.13.patch
>
>
> We should provide a compaction failure reason as easily accessible as 
> possible. Like in the result of the {{SHOW COMPACTIONS}} command.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22818) Preparation for jetty 9.4.26 upgrade

2020-02-03 Thread Laszlo Pinter (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laszlo Pinter updated HIVE-22818:
-
Attachment: HIVE-22818.01.patch

> Preparation for jetty 9.4.26 upgrade
> 
>
> Key: HIVE-22818
> URL: https://issues.apache.org/jira/browse/HIVE-22818
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22818.01.patch
>
>
> Make some code adjustment, before upgrading jetty to 9.4.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22818) Preparation for jetty 9.4.26 upgrade

2020-02-03 Thread Laszlo Pinter (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laszlo Pinter updated HIVE-22818:
-
Status: Patch Available  (was: Open)

> Preparation for jetty 9.4.26 upgrade
> 
>
> Key: HIVE-22818
> URL: https://issues.apache.org/jira/browse/HIVE-22818
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22818.01.patch
>
>
> Make some code adjustment, before upgrading jetty to 9.4.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-22818) Preparation for jetty 9.4.26 upgrade

2020-02-03 Thread Laszlo Pinter (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laszlo Pinter reassigned HIVE-22818:



> Preparation for jetty 9.4.26 upgrade
> 
>
> Key: HIVE-22818
> URL: https://issues.apache.org/jira/browse/HIVE-22818
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
>
> Make some code adjustment, before upgrading jetty to 9.4.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases

2020-02-03 Thread Hive QA (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028795#comment-17028795
 ] 

Hive QA commented on HIVE-22601:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  9m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m  
2s{color} | {color:blue} ql in master has 1537 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
40s{color} | {color:red} ql: The patch generated 10 new + 282 unchanged - 2 
fixed = 292 total (was 284) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 59s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20440/dev-support/hive-personality.sh
 |
| git revision | master / 7792ded |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20440/yetus/diff-checkstyle-ql.txt
 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20440/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Some columns will be lost when a UDTF has multiple aliases in some cases
> 
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 2.1.1, 2.2.0, 3.1.2, 2.3.6
>Reporter: okumin
>Assignee: Owen O'Malley
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22601.1.patch, HIVE-22601.2.patch, 
> HIVE-22601.3.patch, HIVE-22601.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Only one column will be retained when putting UDTFs with multiple aliases and 
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1, 
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3  |
> +-+
> | c   |
> | f   |
> +-+
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-21737) Upgrade Avro to version 1.9.1

2020-02-03 Thread Jira



[ 
https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028777#comment-17028777
 ] 

Ismaël Mejía commented on HIVE-21737:
-

For the interested the ongoing email saga
[https://lists.apache.org/thread.html/rc6c672ad4a5e255957d54d80ff83bf48eacece2828a86bc6cedd9c4c%40%3Cdev.hive.apache.org%3E]

 

> Upgrade Avro to version 1.9.1
> -
>
> Key: HIVE-21737
> URL: https://issues.apache.org/jira/browse/HIVE-21737
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Ismaël Mejía
>Assignee: Fokko Driesprong
>Priority: Major
>  Labels: pull-request-available
> Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner 
> version of Avro without Jackson in the public API. Worth the update.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22601) Some columns will be lost when a UDTF has multiple aliases in some cases

2020-02-03 Thread okumin (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

okumin updated HIVE-22601:
--
Attachment: HIVE-22601.3.patch

> Some columns will be lost when a UDTF has multiple aliases in some cases
> 
>
> Key: HIVE-22601
> URL: https://issues.apache.org/jira/browse/HIVE-22601
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 2.1.1, 2.2.0, 3.1.2, 2.3.6
>Reporter: okumin
>Assignee: Owen O'Malley
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22601.1.patch, HIVE-22601.2.patch, 
> HIVE-22601.3.patch, HIVE-22601.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Only one column will be retained when putting UDTFs with multiple aliases and 
> a top-level UNION together.
> For example, the result of the following SQL should have three columns, c1, 
> c2 and c3.
> {code:java}
> SELECT stack(1, 'a', 'b', 'c') AS (c1, c2, c3)
> UNION ALL
> SELECT stack(1, 'd', 'e', 'f') AS (c1, c2, c3);
> {code}
> However, It's only the c3 column which I can get.
> {code:java}
> +-+
> | _u1.c3  |
> +-+
> | c   |
> | f   |
> +-+
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Reinhard Eilmsteiner (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028757#comment-17028757
 ] 

Reinhard Eilmsteiner commented on HIVE-22817:
-

Added config and two script files:
 # hive.log: Start of the server
 # root.log: Some stuff while the server was running

> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22817) hiveserver2 does not open port

2020-02-03 Thread Reinhard Eilmsteiner (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Reinhard Eilmsteiner updated HIVE-22817:

Attachment: doc.tar.gz

> hiveserver2 does not open port
> --
>
> Key: HIVE-22817
> URL: https://issues.apache.org/jira/browse/HIVE-22817
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.2
> Environment: hadoop 3.1.0
> jdk1.8.0_241
>Reporter: Reinhard Eilmsteiner
>Priority: Major
> Attachments: doc.tar.gz
>
>
> After having installed hive, starting the hiveserver2 does not open a network 
> port as documented in 
> [https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-Requirements]
> netstat does not show a java process, nor does telnet or beeline connect to 
> 127.0.0.1:1
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22578) CBO genOPTree is not failsafe for CTAS and VIEW statements

2020-02-03 Thread Aron Hamvas (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028754#comment-17028754
 ] 

Aron Hamvas commented on HIVE-22578:


[~jcamachorodriguez], the problem is that with CBO enabled, the original AST is 
lost during the execution of this method. If CBO fails due and the rewritten 
AST is not good, there is no way to fall back to the original one, unless we 
save it. 

> CBO genOPTree is not failsafe for CTAS and VIEW statements
> --
>
> Key: HIVE-22578
> URL: https://issues.apache.org/jira/browse/HIVE-22578
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Affects Versions: 3.0.0, 3.1.2
>Reporter: Aron Hamvas
>Assignee: Aron Hamvas
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22578.2.patch, HIVE-22578.patch
>
>
> If CBO fails during OT generation, it is supposed to skip CBO and fall back 
> to SemanticAnalyzer generating the OT for the original AST. However, for CTAS 
> and VIEW statements, the original AST is discarded and replaced with the new 
> AST. 
> If OT generation for the new AST fails with a SemanticException, the 
> compilation is destined to fail completely since we cannot fall back to the 
> original and possibly valid AST.
> Also, the SemanticException sometimes gets hidden and misjudged as a missing 
> column statistics issue due to bad error handling.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

93 matches

Mail list logo