[jira] [Commented] (HIVE-13837) current_timestamp() output format is different in some cases
[
https://issues.apache.org/jira/browse/HIVE-13837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307233#comment-15307233
]
Pengcheng Xiong commented on HIVE-13837:
checked every q tests, they are either not reproducible or not related. pushed
to master. Thanks [~jdere] for the review!
> current_timestamp() output format is different in some cases
>
>
> Key: HIVE-13837
> URL: https://issues.apache.org/jira/browse/HIVE-13837
> Project: Hive
> Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-13837.01.patch, HIVE-13837.02.patch
>
>
> As [~jdere] reports:
> {code}
> current_timestamp() udf returns result with different format in some cases.
> select current_timestamp() returns result with decimal precision:
> {noformat}
> hive> select current_timestamp();
> OK
> 2016-04-14 18:26:58.875
> Time taken: 0.077 seconds, Fetched: 1 row(s)
> {noformat}
> But output format is different for select current_timestamp() from all100k
> union select current_timestamp() from over100k limit 5;
> {noformat}
> hive> select current_timestamp() from all100k union select
> current_timestamp() from over100k limit 5;
> Query ID = hrt_qa_20160414182956_c4ed48f2-9913-4b3b-8f09-668ebf55b3e3
> Total jobs = 1
> Launching Job 1 out of 1
> Tez session was closed. Reopening...
> Session re-established.
> Status: Running (Executing on YARN cluster with App id
> application_1460611908643_0624)
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 1 100
> 0 0
> Map 4 .. llap SUCCEEDED 1 100
> 0 0
> Reducer 3 .. llap SUCCEEDED 1 100
> 0 0
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 0.92 s
>
> --
> OK
> 2016-04-14 18:29:56
> Time taken: 10.558 seconds, Fetched: 1 row(s)
> {noformat}
> explain plan for select current_timestamp();
> {noformat}
> hive> explain extended select current_timestamp();
> OK
> ABSTRACT SYNTAX TREE:
>
> TOK_QUERY
>TOK_INSERT
> TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
> TOK_SELECT
> TOK_SELEXPR
> TOK_FUNCTION
>current_timestamp
> STAGE DEPENDENCIES:
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: _dummy_table
> Row Limit Per Split: 1
> GatherStats: false
> Select Operator
> expressions: 2016-04-14 18:30:57.206 (type: timestamp)
> outputColumnNames: _col0
> ListSink
> Time taken: 0.062 seconds, Fetched: 30 row(s)
> {noformat}
> explain plan for select current_timestamp() from all100k union select
> current_timestamp() from over100k limit 5;
> {noformat}
> hive> explain extended select current_timestamp() from all100k union select
> current_timestamp() from over100k limit 5;
> OK
> ABSTRACT SYNTAX TREE:
>
> TOK_QUERY
>TOK_FROM
> TOK_SUBQUERY
> TOK_QUERY
> TOK_FROM
>TOK_SUBQUERY
> TOK_UNIONALL
> TOK_QUERY
> TOK_FROM
>TOK_TABREF
> TOK_TABNAME
> all100k
> TOK_INSERT
>TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
>TOK_SELECT
> TOK_SELEXPR
> TOK_FUNCTION
> current_timestamp
> TOK_QUERY
> TOK_FROM
>TOK_TABREF
> TOK_TABNAME
> over100k
> TOK_INSERT
>TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
>TOK_SELECT
> TOK_SELEXPR
> TOK_FUNCTION
> current_timestamp
>
[jira] [Updated] (HIVE-13837) current_timestamp() output format is different in some cases
[
https://issues.apache.org/jira/browse/HIVE-13837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-13837:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> current_timestamp() output format is different in some cases
>
>
> Key: HIVE-13837
> URL: https://issues.apache.org/jira/browse/HIVE-13837
> Project: Hive
> Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-13837.01.patch, HIVE-13837.02.patch
>
>
> As [~jdere] reports:
> {code}
> current_timestamp() udf returns result with different format in some cases.
> select current_timestamp() returns result with decimal precision:
> {noformat}
> hive> select current_timestamp();
> OK
> 2016-04-14 18:26:58.875
> Time taken: 0.077 seconds, Fetched: 1 row(s)
> {noformat}
> But output format is different for select current_timestamp() from all100k
> union select current_timestamp() from over100k limit 5;
> {noformat}
> hive> select current_timestamp() from all100k union select
> current_timestamp() from over100k limit 5;
> Query ID = hrt_qa_20160414182956_c4ed48f2-9913-4b3b-8f09-668ebf55b3e3
> Total jobs = 1
> Launching Job 1 out of 1
> Tez session was closed. Reopening...
> Session re-established.
> Status: Running (Executing on YARN cluster with App id
> application_1460611908643_0624)
> --
> VERTICES MODESTATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> --
> Map 1 .. llap SUCCEEDED 1 100
> 0 0
> Map 4 .. llap SUCCEEDED 1 100
> 0 0
> Reducer 3 .. llap SUCCEEDED 1 100
> 0 0
> --
> VERTICES: 03/03 [==>>] 100% ELAPSED TIME: 0.92 s
>
> --
> OK
> 2016-04-14 18:29:56
> Time taken: 10.558 seconds, Fetched: 1 row(s)
> {noformat}
> explain plan for select current_timestamp();
> {noformat}
> hive> explain extended select current_timestamp();
> OK
> ABSTRACT SYNTAX TREE:
>
> TOK_QUERY
>TOK_INSERT
> TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
> TOK_SELECT
> TOK_SELEXPR
> TOK_FUNCTION
>current_timestamp
> STAGE DEPENDENCIES:
> Stage-0 is a root stage
> STAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: _dummy_table
> Row Limit Per Split: 1
> GatherStats: false
> Select Operator
> expressions: 2016-04-14 18:30:57.206 (type: timestamp)
> outputColumnNames: _col0
> ListSink
> Time taken: 0.062 seconds, Fetched: 30 row(s)
> {noformat}
> explain plan for select current_timestamp() from all100k union select
> current_timestamp() from over100k limit 5;
> {noformat}
> hive> explain extended select current_timestamp() from all100k union select
> current_timestamp() from over100k limit 5;
> OK
> ABSTRACT SYNTAX TREE:
>
> TOK_QUERY
>TOK_FROM
> TOK_SUBQUERY
> TOK_QUERY
> TOK_FROM
>TOK_SUBQUERY
> TOK_UNIONALL
> TOK_QUERY
> TOK_FROM
>TOK_TABREF
> TOK_TABNAME
> all100k
> TOK_INSERT
>TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
>TOK_SELECT
> TOK_SELEXPR
> TOK_FUNCTION
> current_timestamp
> TOK_QUERY
> TOK_FROM
>TOK_TABREF
> TOK_TABNAME
> over100k
> TOK_INSERT
>TOK_DESTINATION
> TOK_DIR
> TOK_TMP_FILE
>TOK_SELECT
> TOK_SELEXPR
> TOK_FUNCTION
> current_timestamp
> _u1
> TOK_INSERT
>TOK_DESTINATION
>
[jira] [Updated] (HIVE-13648) ORC Schema Evolution doesn't support same type conversion for VARCHAR, CHAR, or DECIMAL when maxLength or precision/scale is different
[
https://issues.apache.org/jira/browse/HIVE-13648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-13648:
Description:
E.g. when a data file is copied in has a VARCHAR maxLength that doesn't match
the DDL's maxLength. This error is produced:
{code}
java.io.IOException: ORC does not support type conversion from file type
varchar(145) (36) to reader type varchar(114) (36)
{code}
was:
New ORC data type conversion has some edges cases like CHAR/VAR different max
lengths and DECIMAL different scale/precision that need attention.
Also, wondering if some of the orc_* Q test failures (HIVE-13178 patch 092)
were due to DECIMAL data type precision not being copied correctly?
> ORC Schema Evolution doesn't support same type conversion for VARCHAR, CHAR,
> or DECIMAL when maxLength or precision/scale is different
> --
>
> Key: HIVE-13648
> URL: https://issues.apache.org/jira/browse/HIVE-13648
> Project: Hive
> Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
>
> E.g. when a data file is copied in has a VARCHAR maxLength that doesn't match
> the DDL's maxLength. This error is produced:
> {code}
> java.io.IOException: ORC does not support type conversion from file type
> varchar(145) (36) to reader type varchar(114) (36)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13648) ORC Schema Evolution doesn't support same type conversion for VARCHAR, CHAR, or DECIMAL when maxLength or precision/scale is different
[ https://issues.apache.org/jira/browse/HIVE-13648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-13648: Summary: ORC Schema Evolution doesn't support same type conversion for VARCHAR, CHAR, or DECIMAL when maxLength or precision/scale is different (was: Look at ORC data type conversion edge cases (CHAR, VARCHAR, DECIMAL)) > ORC Schema Evolution doesn't support same type conversion for VARCHAR, CHAR, > or DECIMAL when maxLength or precision/scale is different > -- > > Key: HIVE-13648 > URL: https://issues.apache.org/jira/browse/HIVE-13648 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > > New ORC data type conversion has some edges cases like CHAR/VAR different max > lengths and DECIMAL different scale/precision that need attention. > Also, wondering if some of the orc_* Q test failures (HIVE-13178 patch 092) > were due to DECIMAL data type precision not being copied correctly? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-5999) Allow other characters for LINES TERMINATED BY
[ https://issues.apache.org/jira/browse/HIVE-5999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated HIVE-5999: Status: Patch Available (was: Open) > Allow other characters for LINES TERMINATED BY > --- > > Key: HIVE-5999 > URL: https://issues.apache.org/jira/browse/HIVE-5999 > Project: Hive > Issue Type: Improvement > Components: Beeline, Database/Schema, Hive >Affects Versions: 0.12.0 >Reporter: Mariano Dominguez >Assignee: Nemon Lou >Priority: Critical > Labels: Delimiter, Hive, Row, SerDe > Attachments: HIVE-5999.1.patch, HIVE-5999.patch > > > LINES TERMINATED BY only supports newline '\n' right now. > It would be nice to loosen this constraint and allow other characters. > This limitation seems to be hardcoded here: > https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java#L171 > The DDL Definition on the Hive Language manual shows this as a configurable > property whereas it is not. This may lead to mileading assement of being able > to choose a choice of field delimiter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-5999) Allow other characters for LINES TERMINATED BY
[ https://issues.apache.org/jira/browse/HIVE-5999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemon Lou updated HIVE-5999: Attachment: HIVE-5999.1.patch Fixing failure test. > Allow other characters for LINES TERMINATED BY > --- > > Key: HIVE-5999 > URL: https://issues.apache.org/jira/browse/HIVE-5999 > Project: Hive > Issue Type: Improvement > Components: Beeline, Database/Schema, Hive >Affects Versions: 0.12.0 >Reporter: Mariano Dominguez >Assignee: Nemon Lou >Priority: Critical > Labels: Delimiter, Hive, Row, SerDe > Attachments: HIVE-5999.1.patch, HIVE-5999.patch > > > LINES TERMINATED BY only supports newline '\n' right now. > It would be nice to loosen this constraint and allow other characters. > This limitation seems to be hardcoded here: > https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java#L171 > The DDL Definition on the Hive Language manual shows this as a configurable > property whereas it is not. This may lead to mileading assement of being able > to choose a choice of field delimiter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13713) We miss vectorization in a case of count(*) when aggregation mode is COMPLETE
[
https://issues.apache.org/jira/browse/HIVE-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-13713:
Fix Version/s: 2.2.0
2.1.0
> We miss vectorization in a case of count(*) when aggregation mode is COMPLETE
> -
>
> Key: HIVE-13713
> URL: https://issues.apache.org/jira/browse/HIVE-13713
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 2.1.0, 2.2.0
>
> Attachments: HIVE-13713.01.patch, HIVE-13713.02.patch,
> HIVE-13713.03.patch
>
>
> E.g. vectorization_limit.q doesn't vectorize Reducer 2 for the query:
> {code}
> "select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint
> order by ctinyint limit 20"
> {code}
> It was producing a stack trace with this error... when trying to vectorize
> the COMPLETE mode GROUP BY operator.
> {code}
> Vector aggregate not implemented: "count" for type: "NONE
> (reduce-merge-partial = true)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13713) We miss vectorization in a case of count(*) when aggregation mode is COMPLETE
[
https://issues.apache.org/jira/browse/HIVE-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307023#comment-15307023
]
Matt McCline commented on HIVE-13713:
-
Committed to master and branch-2.1
> We miss vectorization in a case of count(*) when aggregation mode is COMPLETE
> -
>
> Key: HIVE-13713
> URL: https://issues.apache.org/jira/browse/HIVE-13713
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 2.1.0, 2.2.0
>
> Attachments: HIVE-13713.01.patch, HIVE-13713.02.patch,
> HIVE-13713.03.patch
>
>
> E.g. vectorization_limit.q doesn't vectorize Reducer 2 for the query:
> {code}
> "select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint
> order by ctinyint limit 20"
> {code}
> It was producing a stack trace with this error... when trying to vectorize
> the COMPLETE mode GROUP BY operator.
> {code}
> Vector aggregate not implemented: "count" for type: "NONE
> (reduce-merge-partial = true)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13713) We miss vectorization in a case of count(*) when aggregation mode is COMPLETE
[
https://issues.apache.org/jira/browse/HIVE-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-13713:
Attachment: HIVE-13713.03.patch
Removed a few Q file updates (vector_udf1.q.out, vector_complex_all.q.out)
because they are unrelated.
> We miss vectorization in a case of count(*) when aggregation mode is COMPLETE
> -
>
> Key: HIVE-13713
> URL: https://issues.apache.org/jira/browse/HIVE-13713
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-13713.01.patch, HIVE-13713.02.patch,
> HIVE-13713.03.patch
>
>
> E.g. vectorization_limit.q doesn't vectorize Reducer 2 for the query:
> {code}
> "select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint
> order by ctinyint limit 20"
> {code}
> It was producing a stack trace with this error... when trying to vectorize
> the COMPLETE mode GROUP BY operator.
> {code}
> Vector aggregate not implemented: "count" for type: "NONE
> (reduce-merge-partial = true)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13818) Fast Vector MapJoin Long hashtable has to handle all integral types
[ https://issues.apache.org/jira/browse/HIVE-13818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307020#comment-15307020 ] Matt McCline commented on HIVE-13818: - Committed to master and branch-2.1 > Fast Vector MapJoin Long hashtable has to handle all integral types > --- > > Key: HIVE-13818 > URL: https://issues.apache.org/jira/browse/HIVE-13818 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Matt McCline >Assignee: Gopal V >Priority: Critical > Fix For: 2.1.0, 2.2.0 > > Attachments: HIVE-13818.01.patch, HIVE-13818.02.patch, > HIVE-13818.1.patch, vector_bug.q, vector_bug.q.out > > > Changes for HIVE-13682 did fix a bug in Fast Hash Tables, but evidently not > this issue according to Gopal/Rajesh/Nita. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13818) Fast Vector MapJoin Long hashtable has to handle all integral types
[ https://issues.apache.org/jira/browse/HIVE-13818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-13818: Resolution: Fixed Status: Resolved (was: Patch Available) > Fast Vector MapJoin Long hashtable has to handle all integral types > --- > > Key: HIVE-13818 > URL: https://issues.apache.org/jira/browse/HIVE-13818 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Matt McCline >Assignee: Gopal V >Priority: Critical > Fix For: 2.1.0, 2.2.0 > > Attachments: HIVE-13818.01.patch, HIVE-13818.02.patch, > HIVE-13818.1.patch, vector_bug.q, vector_bug.q.out > > > Changes for HIVE-13682 did fix a bug in Fast Hash Tables, but evidently not > this issue according to Gopal/Rajesh/Nita. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13818) Fast Vector MapJoin Long hashtable has to handle all integral types
[ https://issues.apache.org/jira/browse/HIVE-13818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-13818: Fix Version/s: 2.2.0 > Fast Vector MapJoin Long hashtable has to handle all integral types > --- > > Key: HIVE-13818 > URL: https://issues.apache.org/jira/browse/HIVE-13818 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Matt McCline >Assignee: Gopal V >Priority: Critical > Fix For: 2.1.0, 2.2.0 > > Attachments: HIVE-13818.01.patch, HIVE-13818.02.patch, > HIVE-13818.1.patch, vector_bug.q, vector_bug.q.out > > > Changes for HIVE-13682 did fix a bug in Fast Hash Tables, but evidently not > this issue according to Gopal/Rajesh/Nita. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13849) Wrong plan for hive.optimize.sort.dynamic.partition=true
[
https://issues.apache.org/jira/browse/HIVE-13849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306994#comment-15306994
]
Jesus Camacho Rodriguez commented on HIVE-13849:
{noformat}
Test Name DurationAge
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_optimization
2 min 4 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_opt_vectorization
1 min 13 sec1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join32 7.2 sec 1
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_select_read_only_encrypted_tbl
1 min 38 sec1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization
1 min 37 sec1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
2 min 11 sec1
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
1 min 11 sec1
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
1 min 11 sec1
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestShortestJobFirstComparator.testWaitQueueComparatorWithinDagPriority
5 sec 1
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testPreemptionQueueComparator
5 sec 4
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
3.1 sec 33
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_static
1 min 28 sec49
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic
1 min 19 sec49
org.apache.hive.minikdc.TestHiveAuthFactory.testStartTokenManagerForMemoryTokenStore
2.6 sec 49
org.apache.hive.minikdc.TestHiveAuthFactory.testStartTokenManagerForDBTokenStore
0.34 sec49
org.apache.hive.minikdc.TestMiniHiveKdc.testLogin 1 min 53 sec49
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
82 ms 69
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_selectindate13 sec
101
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avrocountemptytbl
9.9 sec 101
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_order_null 33 sec
101
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_join_with_different_encryption_keys
1 min 33 sec101
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
9.6 sec 101
org.apache.hadoop.hive.cli.TestMinimrCliDriver.org.apache.hadoop.hive.cli.TestMinimrCliDriver
1 min 8 sec 101
{noformat}
> Wrong plan for hive.optimize.sort.dynamic.partition=true
>
>
> Key: HIVE-13849
> URL: https://issues.apache.org/jira/browse/HIVE-13849
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-13849.patch
>
>
> To reproduce:
> {noformat}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.optimize.sort.dynamic.partition=true;
> CREATE TABLE non_acid(key string, value string) PARTITIONED BY(ds string, hr
> int) CLUSTERED BY(key) INTO 2 BUCKETS STORED AS ORC;
> explain insert into table non_acid partition(ds,hr) select * from srcpart
> sort by value;
> {noformat}
> CC'ed [~ashutoshc], [~ekoifman]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13807) Extend metadata provider to pull up predicates through Union
[
https://issues.apache.org/jira/browse/HIVE-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-13807:
---
Resolution: Fixed
Fix Version/s: 2.1.0
Status: Resolved (was: Patch Available)
Pushed to master, branch-2.1. Thanks for reviewing [~ashutoshc]!
> Extend metadata provider to pull up predicates through Union
>
>
> Key: HIVE-13807
> URL: https://issues.apache.org/jira/browse/HIVE-13807
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Fix For: 2.1.0
>
> Attachments: HIVE-13807.01.patch, HIVE-13807.patch
>
>
> Follow-up of HIVE-13068.
> Currently, when we pull up predicates through Union operation with Calcite
> metadata provider, we just create a single disjunction of pulled up
> predicates.
> E.g. Assume operators _I1, I2, I3_ with predicates {{P1, P2, P3}} that can be
> pulled up through them, respectively.
> For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
> that {{Pu = P1 OR P2 OR P3}}.
> While this is correct, we miss some chances for simplification e.g. if there
> are common factors in P1, P2, and P3. Further, this inference differs
> slightly from the way that the metadata provider pulls up predicates for
> other operators (a list of predicates), thus breaking some assumptions and
> missing some optimization opportunities.
> Ex. ql/src/test/results/clientpositive/input26.q.out
> {noformat}
> explain
> select * from (
> select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
> '11' order by a.key limit 5)pa
> union all
> select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
> '14' limit 5)pb
> )subq
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13807) Extend metadata provider to pull up predicates through Union
[
https://issues.apache.org/jira/browse/HIVE-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-13807:
---
Attachment: HIVE-13807.01.patch
> Extend metadata provider to pull up predicates through Union
>
>
> Key: HIVE-13807
> URL: https://issues.apache.org/jira/browse/HIVE-13807
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Fix For: 2.1.0
>
> Attachments: HIVE-13807.01.patch, HIVE-13807.patch
>
>
> Follow-up of HIVE-13068.
> Currently, when we pull up predicates through Union operation with Calcite
> metadata provider, we just create a single disjunction of pulled up
> predicates.
> E.g. Assume operators _I1, I2, I3_ with predicates {{P1, P2, P3}} that can be
> pulled up through them, respectively.
> For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
> that {{Pu = P1 OR P2 OR P3}}.
> While this is correct, we miss some chances for simplification e.g. if there
> are common factors in P1, P2, and P3. Further, this inference differs
> slightly from the way that the metadata provider pulls up predicates for
> other operators (a list of predicates), thus breaking some assumptions and
> missing some optimization opportunities.
> Ex. ql/src/test/results/clientpositive/input26.q.out
> {noformat}
> explain
> select * from (
> select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
> '11' order by a.key limit 5)pa
> union all
> select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
> '14' limit 5)pb
> )subq
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13859) mask() UDF not retaining day and month field values
[
https://issues.apache.org/jira/browse/HIVE-13859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306961#comment-15306961
]
Hive QA commented on HIVE-13859:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12806808/HIVE-13859.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 10178 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestJdbcWithMiniHA - did not produce a TEST-*.xml file
TestJdbcWithMiniMr - did not produce a TEST-*.xml file
TestOperationLoggingAPIWithTez - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLocksInSubquery
{noformat}
Test results:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/458/testReport
Console output:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/458/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-458/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12806808 - PreCommit-HIVE-MASTER-Build
> mask() UDF not retaining day and month field values
> ---
>
> Key: HIVE-13859
> URL: https://issues.apache.org/jira/browse/HIVE-13859
> Project: Hive
> Issue Type: Bug
> Components: UDF
>Affects Versions: 2.1.0
>Reporter: Madhan Neethiraj
> Fix For: 2.1.0
>
> Attachments: HIVE-13859.patch
>
>
> For date type parameters, mask() UDF replaces year/month/day field values
> with the values given in arguments to the UDF. Argument value -1 is treated
> as special, to specify that mask() should retain the value in the parameter.
> This allows to selectively mask only year/month/day fields.
> Specifying "-1" does not retain the values for day/month fields; however the
> year value is retained, as shown below.
> {code}
> 0: jdbc:hive2://localhost:1> select id, join_date from employee where id
> < 4;
> +-+-+--+
> | id | join_date |
> +-+-+--+
> | 1 | 2012-01-01 |
> | 2 | 2014-02-01 |
> | 3 | 2013-03-01 |
> +-+-+--+
> 3 rows selected (0.435 seconds)
> 0: jdbc:hive2://localhost:1> select id, mask(join_date, -1, -1, -1,
> -1,-1, -1,-1,-1) join_date from employee where id < 4;
> +-+-+--+
> | id | join_date |
> +-+-+--+
> | 1 | 2012-01-01 |
> | 2 | 2014-01-01 |
> | 3 | 2013-01-01 |
> +-+-+--+
> 3 rows selected (0.344 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13713) We miss vectorization in a case of count(*) when aggregation mode is COMPLETE
[
https://issues.apache.org/jira/browse/HIVE-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306890#comment-15306890
]
Hive QA commented on HIVE-13713:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12806889/HIVE-13713.02.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 10169 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestJdbcWithMiniHA - did not produce a TEST-*.xml file
TestJdbcWithMiniMr - did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_grouping_sets.q-mapjoin_mapjoin.q-cte_5.q-and-12-more
- did not produce a TEST-*.xml file
TestOperationLoggingAPIWithTez - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_udf1
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLocksInSubquery
{noformat}
Test results:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/457/testReport
Console output:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/457/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-457/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12806889 - PreCommit-HIVE-MASTER-Build
> We miss vectorization in a case of count(*) when aggregation mode is COMPLETE
> -
>
> Key: HIVE-13713
> URL: https://issues.apache.org/jira/browse/HIVE-13713
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-13713.01.patch, HIVE-13713.02.patch
>
>
> E.g. vectorization_limit.q doesn't vectorize Reducer 2 for the query:
> {code}
> "select ctinyint, count(distinct(cdouble)) from alltypesorc group by ctinyint
> order by ctinyint limit 20"
> {code}
> It was producing a stack trace with this error... when trying to vectorize
> the COMPLETE mode GROUP BY operator.
> {code}
> Vector aggregate not implemented: "count" for type: "NONE
> (reduce-merge-partial = true)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-5999) Allow other characters for LINES TERMINATED BY
[
https://issues.apache.org/jira/browse/HIVE-5999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306810#comment-15306810
]
Hive QA commented on HIVE-5999:
---
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12806799/HIVE-5999.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 10178 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestJdbcWithMiniHA - did not produce a TEST-*.xml file
TestJdbcWithMiniMr - did not produce a TEST-*.xml file
TestOperationLoggingAPIWithTez - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_dynamicserde
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input_lazyserde
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLocksInSubquery
{noformat}
Test results:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/456/testReport
Console output:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/456/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-456/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12806799 - PreCommit-HIVE-MASTER-Build
> Allow other characters for LINES TERMINATED BY
> ---
>
> Key: HIVE-5999
> URL: https://issues.apache.org/jira/browse/HIVE-5999
> Project: Hive
> Issue Type: Improvement
> Components: Beeline, Database/Schema, Hive
>Affects Versions: 0.12.0
>Reporter: Mariano Dominguez
>Assignee: Nemon Lou
>Priority: Critical
> Labels: Delimiter, Hive, Row, SerDe
> Attachments: HIVE-5999.patch
>
>
> LINES TERMINATED BY only supports newline '\n' right now.
> It would be nice to loosen this constraint and allow other characters.
> This limitation seems to be hardcoded here:
> https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java#L171
> The DDL Definition on the Hive Language manual shows this as a configurable
> property whereas it is not. This may lead to mileading assement of being able
> to choose a choice of field delimiter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13861) Fix up nullability issue that might be created by pull up constants rules
[
https://issues.apache.org/jira/browse/HIVE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-13861:
---
Resolution: Fixed
Fix Version/s: 2.1.0
Status: Resolved (was: Patch Available)
Fails are not related. Pushed to master, branch-2.1. Thanks for reviewing
[~ashutoshc]!
> Fix up nullability issue that might be created by pull up constants rules
> -
>
> Key: HIVE-13861
> URL: https://issues.apache.org/jira/browse/HIVE-13861
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Fix For: 2.1.0
>
> Attachments: HIVE-13861.01.patch
>
>
> When we pull up constants through Union or Sort operators, we might end up
> rewriting the original expression into an expression whose schema has
> different nullability properties for some of its columns.
> This results in AssertionError of the following kind:
> {noformat}
> ...
> org.apache.hive.service.cli.HiveSQLException: Error running query:
> java.lang.AssertionError: Internal error: Cannot add expression of different
> type to set:
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13861) Fix up nullability issue that might be created by pull up constants rules
[
https://issues.apache.org/jira/browse/HIVE-13861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306721#comment-15306721
]
Jesus Camacho Rodriguez commented on HIVE-13861:
{noformat}
Test Name DurationAge
org.apache.hive.hcatalog.listener.TestDbNotificationListener.cleanupNotifs
1 min 0 sec 1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_1
79 ms 1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dynpart_hashjoin_2
65 ms 1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_3
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_4
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_5
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_constprog_dpp
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mrr 1 ms
1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_main
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_smb_1
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_join_part_col_char
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_schema_evolution
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez1
2 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_bucket_map_join_tez2
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_mapjoin_decimal
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_tests
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_lvj_mapjoin
3 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_1
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_vector_dynpart_hashjoin_2
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_1 0 ms
1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_2 1 ms
1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_3 1 ms
1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_4 1 ms
1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_5 0 ms
1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_hash
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_insert_overwrite_local_directory_1
2 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_ppd_basic
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_bmj_schema_evolution
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llapdecider
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_nullscan
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join_result_complex
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_hybridgrace_hashjoin_2
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_fsstat
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_multiinsert
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_dml 1 ms
1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_join
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_dynamic_partition
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_llap_udf
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_multi_union
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union2
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_decimal
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union_group_by
1 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_joins_explain
0 ms1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.org.apache.hadoop.hive.cli.TestMiniLlapCliDriver
1 ms1
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestShortestJobFirstComparator.testWaitQueueComparatorWithinDagPriority
5 sec 1
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_1
5 ms2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_cte_mat_2
2 ms2
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_self_join
9 ms2
[jira] [Commented] (HIVE-13863) Improve AnnotateWithStatistics with support for cartesian product
[
https://issues.apache.org/jira/browse/HIVE-13863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306713#comment-15306713
]
Jesus Camacho Rodriguez commented on HIVE-13863:
{noformat}
Test Name DurationAge
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer5
12 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative2
4.5 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer9
8.6 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input23 4 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats11 11 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_map_join_1
2 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_6
9.6 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_no_hooks4.5 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join42 26 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_between_columns
11 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin2 5.9 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin14.6 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin10 7.5 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin12 8.9 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin3 5.9 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_2 5.5 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin_negative
4.6 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_unqualcolumnrefs
5.7 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_join116 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_views 51 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_outer_join5 4.4 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer15
7.7 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin1 min
19 sec1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer4
8.2 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer10
6.6 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_cross_product_check_2
8 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin1 1.6 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pointlookup219 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer11
13 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constprog2 3 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_11
14 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_3
4.7 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join26 1.1 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_transform_hint
0.73 sec1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parallel_join0 18 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_4
5.6 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_complex_alias 9.6 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_25 9.7 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_distinct
6.6 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer7
6.9 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
21 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constprog_partitioner
7.3 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join37 0.7 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sort_merge_join_desc_7
22 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pointlookup316 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer1
6.4 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketcontext_4 15 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer6
13 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join36 6.4 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucketmapjoin13 11 sec
1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cross_product_check_1
9.5 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_cond_pushdown_unqual3
5.6 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer2
11 sec 1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_smb_mapjoin_14 32 sec
1
[jira] [Commented] (HIVE-12983) Provide a builtin function to get Hive version
[
https://issues.apache.org/jira/browse/HIVE-12983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306708#comment-15306708
]
Hive QA commented on HIVE-12983:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12806797/HIVE-12983.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 10180 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestJdbcWithMiniHA - did not produce a TEST-*.xml file
TestJdbcWithMiniMr - did not produce a TEST-*.xml file
TestOperationLoggingAPIWithTez - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLocksInSubquery
{noformat}
Test results:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/455/testReport
Console output:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/455/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-455/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12806797 - PreCommit-HIVE-MASTER-Build
> Provide a builtin function to get Hive version
> --
>
> Key: HIVE-12983
> URL: https://issues.apache.org/jira/browse/HIVE-12983
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Affects Versions: 2.0.0
>Reporter: Lenni Kuff
>Assignee: Lenni Kuff
> Attachments: HIVE-12983.1.patch, HIVE-12983.2.patch
>
>
> It would be nice to have a builtin function that would return the Hive
> version. This would make it easier for a users and tests to programmatically
> check the Hive version in a SQL script. It's also useful so a client can
> check the Hive version on a remote cluster.
> For example:
> {code}
> beeline> SELECT version();
> 2.1.0-SNAPSHOT r208ab352311a6cbbcd1f7fcd40964da2dbc6703d
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13808) Use constant expressions to backtrack when we create ReduceSink
[
https://issues.apache.org/jira/browse/HIVE-13808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-13808:
---
Resolution: Fixed
Fix Version/s: 2.1.0
Status: Resolved (was: Patch Available)
Pushed to master, branch-2.1. Thanks for the review [~ashutoshc]!
> Use constant expressions to backtrack when we create ReduceSink
> ---
>
> Key: HIVE-13808
> URL: https://issues.apache.org/jira/browse/HIVE-13808
> Project: Hive
> Issue Type: Sub-task
> Components: Parser
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Fix For: 2.1.0
>
> Attachments: HIVE-13808.01.patch, HIVE-13808.patch
>
>
> Follow-up of HIVE-13068.
> When we create a RS with constant expressions as keys/values, and immediately
> after we create a SEL operator that backtracks the expressions from the RS.
> Currently, we automatically create references for all the keys/values.
> Before, we could rely on Hive ConstantPropagate to propagate the constants to
> the SEL. However, after HIVE-13068, Hive ConstantPropagate does not get
> exercised anymore. Thus, we can simply create constant expressions when we
> create the SEL operator instead of a reference.
> Ex. ql/src/test/results/clientpositive/vector_coalesce.q.out
> {noformat}
> EXPLAIN SELECT cdouble, cstring1, cint, cfloat, csmallint, coalesce(cdouble,
> cstring1, cint, cfloat, csmallint) as c
> FROM alltypesorc
> WHERE (cdouble IS NULL)
> ORDER BY cdouble, cstring1, cint, cfloat, csmallint, c
> LIMIT 10
> {noformat}
> Plan:
> {noformat}
> EXPLAIN SELECT cdouble, cstring1, cint, cfloat, csmallint, coalesce(cdouble,
> cstring1, cint, cfloat, csmallint) as c
> FROM alltypesorc
> WHERE (cdouble IS NULL)
> ORDER BY cdouble, cstring1, cint, cfloat, csmallint, c
> LIMIT 10
> POSTHOOK: type: QUERY
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: alltypesorc
> Statistics: Num rows: 12288 Data size: 2641964 Basic stats:
> COMPLETE Column stats: NONE
> Filter Operator
> predicate: cdouble is null (type: boolean)
> Statistics: Num rows: 6144 Data size: 1320982 Basic stats:
> COMPLETE Column stats: NONE
> Select Operator
> expressions: cstring1 (type: string), cint (type: int),
> cfloat (type: float), csmallint (type: smallint),
> COALESCE(null,cstring1,cint,cfloat,csmallint) (type: string)
> outputColumnNames: _col1, _col2, _col3, _col4, _col5
> Statistics: Num rows: 6144 Data size: 1320982 Basic stats:
> COMPLETE Column stats: NONE
> Reduce Output Operator
> key expressions: null (type: double), _col1 (type: string),
> _col2 (type: int), _col3 (type: float), _col4 (type: smallint), _col5 (type:
> string)
> sort order: ++
> Statistics: Num rows: 6144 Data size: 1320982 Basic stats:
> COMPLETE Column stats: NONE
> TopN Hash Memory Usage: 0.1
> Execution mode: vectorized
> Reduce Operator Tree:
> Select Operator
> expressions: KEY.reducesinkkey0 (type: double), KEY.reducesinkkey1
> (type: string), KEY.reducesinkkey2 (type: int), KEY.reducesinkkey3 (type:
> float), KEY.reducesinkkey4 (type: smallint), KEY.reducesinkkey5 (type: string)
> outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
> Statistics: Num rows: 6144 Data size: 1320982 Basic stats: COMPLETE
> Column stats: NONE
> Limit
> Number of rows: 10
> Statistics: Num rows: 10 Data size: 2150 Basic stats: COMPLETE
> Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 10 Data size: 2150 Basic stats: COMPLETE
> Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> Stage: Stage-0
> Fetch Operator
> limit: 10
> Processor Tree:
> ListSink
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-11233) Include Apache Phoenix support in HBaseStorageHandler
[
https://issues.apache.org/jira/browse/HIVE-11233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306632#comment-15306632
]
Svetozar Ivanov commented on HIVE-11233:
https://reviews.apache.org/r/43811/
> Include Apache Phoenix support in HBaseStorageHandler
> -
>
> Key: HIVE-11233
> URL: https://issues.apache.org/jira/browse/HIVE-11233
> Project: Hive
> Issue Type: New Feature
> Components: HBase Handler
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Svetozar Ivanov
>Assignee: Svetozar Ivanov
> Labels: Binary, Hbase, Numeric, Phoenix, Sortable
> Attachments: HIVE-11233-branch-1.2.patch,
> HIVE-11233-branch-2.0.patch, HIVE-11233.1.patch, HIVE-11233.2.patch,
> HIVE-11233.3.patch, HIVE-11233.4.patch, HIVE-11233.patch
>
>
> Currently HBaseStorageHandler doesn't provide mechanism for storage of binary
> sortable key and values. It is necessary when given HBase table is used for
> persistence by Apache Hive and Apache Phoenix. In that way all byte arrays
> read or written by Hive will be compatible with binary sortable format used
> in Phoenix.
> It turns out the major difference is in all numeric data types accordingly
> officially provided documentation -
> https://phoenix.apache.org/language/datatypes.html.
> That's how I'm using it in my code:
> {code}
> private static String buildWithSerDeProperties(TableDescriptor
> tableDescriptor) {
> Map serdePropertiesMap = new HashMap<>();
> serdePropertiesMap.put(HBaseSerDe.HBASE_TABLE_NAME,
> tableDescriptor.getTableName());
> serdePropertiesMap.put(HBaseSerDe.HBASE_TABLE_DEFAULT_STORAGE_TYPE,
> BINARY_STORAGE_TYPE);
> serdePropertiesMap.put(HBaseSerDe.HBASE_COLUMNS_MAPPING,
> buildHBaseColumnsDefinition(tableDescriptor));
> serdePropertiesMap.put(HBaseSerDe.HBASE_VALUE_FACTORY_CLASS,
> PhoenixValueFactory.class.getName());
> /* Use different key factory for simple and composite primary key */
> if (tableDescriptor.getPkDescriptors().size() == 1) {
> serdePropertiesMap.put(HBaseSerDe.HBASE_KEY_FACTORY_CLASS,
> PhoenixKeyFactory.class.getName());
> } else {
> serdePropertiesMap.put(HBaseSerDe.HBASE_COMPOSITE_KEY_FACTORY,
> PhoenixCompositeKeyFactory.class.getName());
> }
> String serDeProperties = serdePropertiesMap.entrySet().stream()
> .map(e -> quoteInSingleQuotes(e.getKey()) + " = " +
> quoteInSingleQuotes(e.getValue()))
> .collect(Collectors.joining(COLUMNS_SEPARATOR));
> logger.debug("SERDEPROPERTIES are [{}]", serDeProperties);
> return serDeProperties;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13895) HoS start-up overhead in yarn-client mode
[
https://issues.apache.org/jira/browse/HIVE-13895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306579#comment-15306579
]
Xuefu Zhang commented on HIVE-13895:
+1
> HoS start-up overhead in yarn-client mode
> -
>
> Key: HIVE-13895
> URL: https://issues.apache.org/jira/browse/HIVE-13895
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-13895.1.patch
>
>
> To avoid the too verbose app state report, HIVE-13376 increases the state
> check interval to a default 60s. However, bigger interval brings considerable
> start-up wait time for yarn-client mode.
> Since the state report only exists in yarn-cluster mode, we can disable it
> using {{spark.yarn.submit.waitAppCompletion}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13840) Orc split generation is reading file footers twice
[
https://issues.apache.org/jira/browse/HIVE-13840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306575#comment-15306575
]
Hive QA commented on HIVE-13840:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12806784/HIVE-13840.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10179 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestJdbcWithMiniHA - did not produce a TEST-*.xml file
TestJdbcWithMiniMr - did not produce a TEST-*.xml file
TestOperationLoggingAPIWithTez - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLocksInSubquery
{noformat}
Test results:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/454/testReport
Console output:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/454/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-454/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12806784 - PreCommit-HIVE-MASTER-Build
> Orc split generation is reading file footers twice
> --
>
> Key: HIVE-13840
> URL: https://issues.apache.org/jira/browse/HIVE-13840
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 2.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-13840.1.patch, HIVE-13840.2.patch,
> HIVE-13840.3.patch
>
>
> Recent refactorings to move orc out introduced a regression in split
> generation. This leads to reading the orc file footers twice during split
> generation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13883) WebHCat leaves token crc file never gets deleted
[
https://issues.apache.org/jira/browse/HIVE-13883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306458#comment-15306458
]
Hive QA commented on HIVE-13883:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12806783/HIVE-13883.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 10178 tests
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestJdbcWithMiniHA - did not produce a TEST-*.xml file
TestJdbcWithMiniMr - did not produce a TEST-*.xml file
TestOperationLoggingAPIWithTez - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLocksInSubquery
{noformat}
Test results:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/453/testReport
Console output:
http://ec2-54-177-240-2.us-west-1.compute.amazonaws.com/job/PreCommit-HIVE-MASTER-Build/453/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-453/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12806783 - PreCommit-HIVE-MASTER-Build
> WebHCat leaves token crc file never gets deleted
>
>
> Key: HIVE-13883
> URL: https://issues.apache.org/jira/browse/HIVE-13883
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 1.2.0, 1.1.1, 1.2.1, 2.0.1
>Reporter: niklaus xiao
>Priority: Minor
> Attachments: HIVE-13883.patch
>
>
> In one of our long run environment, there are thousands of
> /tmp/.templeton*.tmp.crc files,
> {quote}
> omm@szxciitslx17645:/> ll /tmp/.templeton*.tmp.crc
> ...
> -rw-r--r-- 1 omm wheel 12 May 26 18:15
> /tmp/.templeton6676048390600607654.tmp.crc
> -rw-r--r-- 1 omm wheel 12 May 26 18:14
> /tmp/.templeton2733383617337556503.tmp.crc
> -rw-r--r-- 1 omm wheel 12 May 26 18:12
> /tmp/.templeton2183121761801669064.tmp.crc
> -rw-r--r-- 1 omm wheel 12 May 26 18:11
> /tmp/.templeton2689764046140543879.tmp.crc
> ...
> {quote}
> {quote}
> omm@szxciitslx17645:/> ll /tmp/.templeton*.tmp.crc | wc -l
> 17986
> {quote}
> It's created by webhcat,
> [https://github.com/apache/hive/blob/master/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/SecureProxySupport.java#L193]
> and never gets deleted
> [https://github.com/apache/hive/blob/master/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/SecureProxySupport.java#L110]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13883) WebHCat leaves token crc file never gets deleted
[
https://issues.apache.org/jira/browse/HIVE-13883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306391#comment-15306391
]
niklaus xiao commented on HIVE-13883:
-
need code review.
> WebHCat leaves token crc file never gets deleted
>
>
> Key: HIVE-13883
> URL: https://issues.apache.org/jira/browse/HIVE-13883
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 1.2.0, 1.1.1, 1.2.1, 2.0.1
>Reporter: niklaus xiao
>Priority: Minor
> Attachments: HIVE-13883.patch
>
>
> In one of our long run environment, there are thousands of
> /tmp/.templeton*.tmp.crc files,
> {quote}
> omm@szxciitslx17645:/> ll /tmp/.templeton*.tmp.crc
> ...
> -rw-r--r-- 1 omm wheel 12 May 26 18:15
> /tmp/.templeton6676048390600607654.tmp.crc
> -rw-r--r-- 1 omm wheel 12 May 26 18:14
> /tmp/.templeton2733383617337556503.tmp.crc
> -rw-r--r-- 1 omm wheel 12 May 26 18:12
> /tmp/.templeton2183121761801669064.tmp.crc
> -rw-r--r-- 1 omm wheel 12 May 26 18:11
> /tmp/.templeton2689764046140543879.tmp.crc
> ...
> {quote}
> {quote}
> omm@szxciitslx17645:/> ll /tmp/.templeton*.tmp.crc | wc -l
> 17986
> {quote}
> It's created by webhcat,
> [https://github.com/apache/hive/blob/master/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/SecureProxySupport.java#L193]
> and never gets deleted
> [https://github.com/apache/hive/blob/master/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/SecureProxySupport.java#L110]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13895) HoS start-up overhead in yarn-client mode
[
https://issues.apache.org/jira/browse/HIVE-13895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306373#comment-15306373
]
Rui Li commented on HIVE-13895:
---
Pinging [~xuefuz] and [~szehon]
> HoS start-up overhead in yarn-client mode
> -
>
> Key: HIVE-13895
> URL: https://issues.apache.org/jira/browse/HIVE-13895
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-13895.1.patch
>
>
> To avoid the too verbose app state report, HIVE-13376 increases the state
> check interval to a default 60s. However, bigger interval brings considerable
> start-up wait time for yarn-client mode.
> Since the state report only exists in yarn-cluster mode, we can disable it
> using {{spark.yarn.submit.waitAppCompletion}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13895) HoS start-up overhead in yarn-client mode
[
https://issues.apache.org/jira/browse/HIVE-13895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-13895:
--
Status: Patch Available (was: Open)
> HoS start-up overhead in yarn-client mode
> -
>
> Key: HIVE-13895
> URL: https://issues.apache.org/jira/browse/HIVE-13895
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-13895.1.patch
>
>
> To avoid the too verbose app state report, HIVE-13376 increases the state
> check interval to a default 60s. However, bigger interval brings considerable
> start-up wait time for yarn-client mode.
> Since the state report only exists in yarn-cluster mode, we can disable it
> using {{spark.yarn.submit.waitAppCompletion}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13895) HoS start-up overhead in yarn-client mode
[
https://issues.apache.org/jira/browse/HIVE-13895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-13895:
--
Attachment: HIVE-13895.1.patch
> HoS start-up overhead in yarn-client mode
> -
>
> Key: HIVE-13895
> URL: https://issues.apache.org/jira/browse/HIVE-13895
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-13895.1.patch
>
>
> To avoid the too verbose app state report, HIVE-13376 increases the state
> check interval to a default 60s. However, bigger interval brings considerable
> start-up wait time for yarn-client mode.
> Since the state report only exists in yarn-cluster mode, we can disable it
> using {{spark.yarn.submit.waitAppCompletion}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13835) TestMiniTezCliDriver.vector_complex_all.q needs golden file update
[ https://issues.apache.org/jira/browse/HIVE-13835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15306288#comment-15306288 ] Matt McCline commented on HIVE-13835: - Do the queries in vector_complex_all.q produce stable results? > TestMiniTezCliDriver.vector_complex_all.q needs golden file update > -- > > Key: HIVE-13835 > URL: https://issues.apache.org/jira/browse/HIVE-13835 > Project: Hive > Issue Type: Task > Components: Test >Affects Versions: 2.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 2.1.0 > > Attachments: HIVE-13835.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
