[jira] [Created] (HIVE-24694) Early connection close to release server resources during creating
Zhihua Deng created HIVE-24694: -- Summary: Early connection close to release server resources during creating Key: HIVE-24694 URL: https://issues.apache.org/jira/browse/HIVE-24694 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Zhihua Deng Assignee: Zhihua Deng If exception happens during we try to get the connection from HiveDriver, the opened transport or session may leave unclosed as the connection returned is null, we cannot call the close method to release the server resources(threads/connection quota), this could make things more worse if the user rearches the connection limit, the following calls to get the connection will be failed until we restart the hs2. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24692) Max number of table hive-metastore can handle
Izek Greenfield created HIVE-24692: -- Summary: Max number of table hive-metastore can handle Key: HIVE-24692 URL: https://issues.apache.org/jira/browse/HIVE-24692 Project: Hive Issue Type: Wish Components: Hive Reporter: Izek Greenfield What the Max/recommended number of table hive meta-store can handle? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24693) Parquet Timestamp Values Read/Write Very Slow
David Mollitor created HIVE-24693:
-
Summary: Parquet Timestamp Values Read/Write Very Slow
Key: HIVE-24693
URL: https://issues.apache.org/jira/browse/HIVE-24693
Project: Hive
Issue Type: Improvement
Reporter: David Mollitor
Assignee: David Mollitor
Parquet {{DataWriteableWriter}} relias on {{NanoTimeUtils}} to convert a
timestamp object into a binary value. The way in which it does this,... it
calls {{toString()}} on the timestamp object, and then parses the String. This
particular timestamp do not carry a timezone, so the string is something like:
{{2021-21-03 12:32:23....}}
The parse code tries to parse the string assuming there is a time zone, and if
not, falls-back and applies the provided "default time zone". As was noted in
[HIVE-24353], if something fails to parse, it is very expensive to try to parse
again. So, for each timestamp in the Parquet file, it:
* Builds a string from the time stamp
* Parses it (throws an exception, parses again)
There is no need to do this kind of string manipulations/parsing, it should
just be using the epoch millis/seconds/time stored internal to the Timestamp
object.
{code:java}
// Converts Timestamp to TimestampTZ.
public static TimestampTZ convert(Timestamp ts, ZoneId defaultTimeZone) {
return parse(ts.toString(), defaultTimeZone);
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24691) Ban commons-logging (again)
Zoltan Matyus created HIVE-24691: Summary: Ban commons-logging (again) Key: HIVE-24691 URL: https://issues.apache.org/jira/browse/HIVE-24691 Project: Hive Issue Type: Bug Components: Logging Affects Versions: 4.0.0 Reporter: Zoltan Matyus Assignee: Zoltan Matyus The usage of commons-logging has been completely removed once from Hive in HIVE-20019. However, new usage has been added since, despite attempts to ban this (bannedDependencies). I'm removing all usage again, and add another way to ban using it (restrictImports). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24690) GlobalLimitOptimizer Fails To Identify Some Queries With LIMIT Operator
Syed Shameerur Rahman created HIVE-24690:
Summary: GlobalLimitOptimizer Fails To Identify Some Queries With
LIMIT Operator
Key: HIVE-24690
URL: https://issues.apache.org/jira/browse/HIVE-24690
Project: Hive
Issue Type: Bug
Components: Query Planning
Affects Versions: 3.1.0, 2.1.0, 1.1.0
Reporter: Syed Shameerur Rahman
Assignee: Syed Shameerur Rahman
As per
[https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GlobalLimitOptimizer.java#L88]
queries like
{code:java}
CREATE TABLE ... AS SELECT col1, col2 FROM tbl LIMIT ..
INSERT OVERWRITE TABLE ... SELECT col1, hash(col2), split(col1) FROM ...
LIMIT...
{code}
falls under the category of qualified list, But after HIVE-9444 it is not.
On investigating this issue, It is found that for
{code:java}
CREATE TABLE ... AS SELECT col1, col2 FROM tbl LIMIT
{code}
query the operator tree looks like *TS -> SEL -> LIM -> RS -> SEL -> LIM -> FS*
Since only only LIMIT operator is allowed as per
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GlobalLimitOptimizer.java#L196
, The *GlobalLimitOptimizer* fails to identify such queries.
*Steps To Reproduce*
{code:java}
set hive.limit.optimize.enable=true;
create table t1 (a int);
create table t2 select * from t1 LIMIT 10;
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-24689) Changing file format (from ORC) is not supported for table xxx
yanhw created HIVE-24689: Summary: Changing file format (from ORC) is not supported for table xxx Key: HIVE-24689 URL: https://issues.apache.org/jira/browse/HIVE-24689 Project: Hive Issue Type: Bug Affects Versions: 3.1.2 Reporter: yanhw use test; drop table if exists test_lzo_partition; alter table test_lzo_partition set FILEFORMAT INPUTFORMAT 'com.hadoop.mapred.DeprecatedLzoTextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' SERDE 'org.openx.data.jsonserde.JsonSerDe' ; alter table test_lzo_partition add if not exists partition (t=2); alter table test_lzo_partition PARTITION (t=2) set FILEFORMAT INPUTFORMAT 'com.hadoop.mapred.DeprecatedLzoTextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' SERDE 'org.openx.data.jsonserde.JsonSerDe' ; It will return: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Changing file format (from ORC) is not supported for table test.test_lzo_partition -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-24688) Optimise ObjectInspectorUtils.copyToStandardObject
László Bodor created HIVE-24688: --- Summary: Optimise ObjectInspectorUtils.copyToStandardObject Key: HIVE-24688 URL: https://issues.apache.org/jira/browse/HIVE-24688 Project: Hive Issue Type: Improvement Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.3.4#803005)
