Hive JDBC complex type objects
According to https://issues.apache.org/jira/browse/HIVE-1378 Hive returns strings for complex types. I couldn’t find anything else on the subject, has there been any more work done in later versions of Hive to return actual objects in the JDBC driver? In NiFi we can have records/results/rows with complex types, but when we translate a Hive ResultSet into fields, complex types like arrays simply become Strings when IMO they’d be more useful as actual arrays, for downstream transformations etc. Thank you in advance, Matt
[jira] [Created] (HIVE-22768) Remove the usages of the Utilities.makeList method call
Ivan Suller created HIVE-22768: -- Summary: Remove the usages of the Utilities.makeList method call Key: HIVE-22768 URL: https://issues.apache.org/jira/browse/HIVE-22768 Project: Hive Issue Type: Improvement Reporter: Ivan Suller The aforementioned org.apache.hadoop.hive.ql.exec.Utilities.makeList(Object...) method is non generic and most of its usages can be replaced by a simple Collections.singletonList(Object) call. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22767) beeline doesn't parse semicolons in comments properly
Zoltan Matyus created HIVE-22767:
Summary: beeline doesn't parse semicolons in comments properly
Key: HIVE-22767
URL: https://issues.apache.org/jira/browse/HIVE-22767
Project: Hive
Issue Type: Bug
Components: Beeline
Reporter: Zoltan Matyus
HIVE-12646 fixed the handling of semicolons in quoted strings, but leaves the
problem of semicolons in comments. E.g. with beeline connected to any
database...
this works: {code:sql}select 1; select /* */ 2; select /* */ 3;{code}
this doesn't work: {code:sql}select 1; select /* ; */ 2; select /* ; */ 3;{code}
This has been fixed and reintroduced before (possibly multiple times). Ideally,
there should be a single utility method somewhere to separate comments, strings
and commands -- with the proper testing in place (q files).
However, I'm trying to make this fix back-portable, so a light touch is needed.
I'm focusing on beeline for now, and only writing (very thorough) unit tests,
as I cannot exclude any new q files from TestCliDriver (which would break,
since it's using a different parsing method).
P.S. excerpt of the error message:
{noformat}
0: jdbc:hive2://...> select 1; select /* ; */ 2; select /* ; */ 3;
INFO : Compiling command(queryId=...): select 1
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
type:int, comment:null)], properties:null)
INFO : Completed compiling command(queryId=...); Time taken: 0.38 seconds
INFO : Executing command(queryId=...): select 1
INFO : Completed executing command(queryId=...); Time taken: 0.004 seconds
INFO : OK
+--+
| _c0 |
+--+
| 1|
+--+
1 row selected (2.007 seconds)
INFO : Compiling command(queryId=...): select /*
ERROR : FAILED: ParseException line 1:9 cannot recognize input near ''
'' '' in select clause
org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize input
near '' '' '' in select clause
at
org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:233)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:79)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:72)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:598)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1505)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1452)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1447)
at
org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
at ...
{noformat}
Similarly, the following query also fails:
{code:sql}select /* ' */ 1; select /* ' */ 2;{code}
I suspect line comments are also not handled properly but I cannot reproduce
this in interactive beeline...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-22766) Upgrade to Kryo4
László Bodor created HIVE-22766: --- Summary: Upgrade to Kryo4 Key: HIVE-22766 URL: https://issues.apache.org/jira/browse/HIVE-22766 Project: Hive Issue Type: Improvement Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22765) Quote table names in CompactionTxnHandler
Laszlo Pinter created HIVE-22765:
Summary: Quote table names in CompactionTxnHandler
Key: HIVE-22765
URL: https://issues.apache.org/jira/browse/HIVE-22765
Project: Hive
Issue Type: Improvement
Affects Versions: 4.0.0
Reporter: Laszlo Pinter
Assignee: Zoltan Chovan
When I run a metastore backed up by a postgres db, I get the following
exception:
{code:java}
MetaException(message:Unable to connect to transaction database
org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does not
existMetaException(message:Unable to connect to transaction database
org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does not
exist Position: 83 at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2468)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2211)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:309) at
org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446) at
org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370) at
org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:311) at
org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:297) at
org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:274) at
org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:225) at
org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
at
org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
at
org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findNextToCompact(CompactionTxnHandler.java:157)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.find_next_compact(HiveMetaStore.java:8061)
at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at com.sun.proxy.$Proxy28.find_next_compact(Unknown Source) at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.findNextCompact(HiveMetaStoreClient.java:4046)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:208)
at com.sun.proxy.$Proxy29.findNextCompact(Unknown Source) at
org.apache.hadoop.hive.ql.txn.compactor.Worker.run(Worker.java:102){code}
This is due to missing quotes around table names.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-22764) Create new command for "optimize" compaction and have basic implementation.
Aditya Shah created HIVE-22764: -- Summary: Create new command for "optimize" compaction and have basic implementation. Key: HIVE-22764 URL: https://issues.apache.org/jira/browse/HIVE-22764 Project: Hive Issue Type: Sub-task Reporter: Aditya Shah Assignee: Aditya Shah Created new blocking compaction (added compaction type "optimize") by adding a lock request on the compaction's transaction. It works mostly like mmMajorCompaction and writes files w/o row_IDs. I have added an additional table property to provide optimize columns that is used by the compactor to cluster the data by. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HIVE-22762) Leap day is incorrectly parsed during cast in Hive
Karen Coppage created HIVE-22762:
Summary: Leap day is incorrectly parsed during cast in Hive
Key: HIVE-22762
URL: https://issues.apache.org/jira/browse/HIVE-22762
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Karen Coppage
Assignee: Karen Coppage
Fix For: 4.0.0
While casting a string to a date with a custom date format having day token
before year and moth tokens, the date is parsed incorrectly for leap days.
h3. How to reproduce
Execute {code}select cast("29 02 0" as date format "dd mm rr"){code} with Hive.
The query results in *2020-02-28*, incorrectly.
Executing the another cast with a slightly modified representation of the date
(day is preceded by year and moth) is however correctly parsed:
{code}select cast("0 02 29" as date format "rr mm dd"){code}
It returns *2020-02-29*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HIVE-22763) 0 is accepted in 12-hour format during timestamp cast
Karen Coppage created HIVE-22763:
Summary: 0 is accepted in 12-hour format during timestamp cast
Key: HIVE-22763
URL: https://issues.apache.org/jira/browse/HIVE-22763
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Karen Coppage
Assignee: Karen Coppage
Fix For: 4.0.0
Having a timestamp string in 12-hour format can be parsed if the hour is 0,
however, based on the [design
document|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit],
it should be rejected.
h3. How to reproduce
Run {code}select cast("2020-01-01 0 am 00" as timestamp format "-mm-dd hh12
p.m. ss"){code}
It shouldn' t be parsed, as the hour component is 0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
