[jira] [Updated] (HIVE-24323) JDBC driver fails when using Kerberos due to missing dependencies

2020-10-28 Thread N Campbell (Jira) 


 [ 
https://issues.apache.org/jira/browse/HIVE-24323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

N Campbell updated HIVE-24323:
--
Description: 
*The Apache Hive web pages historically implied that only 3-JAR files are 
required*

hadoop-auth
 hadoop-common
 hive-jdbc



*If a connection is attempted using Kerberos authentication, it will fail due 
to several missing dependencies*

hadoop-auth-3.1.1.3.1.5.0-152.jar
 hadoop-common-3.1.1.3.1.5.0-152.jar
 hive-jdbc-3.1.0.3.1.5.0-152-standalone.jar

*Dependencies*
commons-collections-3.2.2.jar
 commons-configuration2.jar
 commons-lang-2.6.jar
 guava-29.0-jre.jar
 log4j-1.2.17.jar
 slf4j-api-1.7.25.jar




*It is unclear if the intent of the standalone JAR is to include these 
dependencies or not.* 
But does not seem to be any documentation either way.



*It also appears that dependencies are not being shaded, which can result in 
conflicts with guava or wstx jar files in the class path.* 
Such as noted by ORACLE {color:#00}Doc ID 2650046.1{color}

{color:#00} 
com.ctc.wstx.io.StreamBootstrapper.getInstance(Ljava/lang/String;Lcom/ctc/wstx/io/SystemId;Ljava/io/InputStream;)Lcom/ctc/wstx/io/StreamBootstrapper;
 ]

java.lang.NoSuchMethodError: 
com.ctc.wstx.io.StreamBootstrapper.getInstance(Ljava/lang/String;Lcom/ctc/wstx/io/SystemId;Ljava/io/InputStream;)Lcom/ctc/wstx/io/StreamBootstrapper;
  at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2918)
  at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2901){color}

  was:
The Apache Hive web pages historically implied that only 3-JAR files are 
required

 hadoop-auth
 hadoop-common
 hive-jdbc

If a connection is attempted using Kerberos authentication, it will fail due to 
several missing dependencies

 hadoop-auth-3.1.1.3.1.5.0-152.jar
 hadoop-common-3.1.1.3.1.5.0-152.jar
 hive-jdbc-3.1.0.3.1.5.0-152-standalone.jar

It is unclear if the intent of the standalone JAR is to include these 
dependencies or not. But does not seem to be any documentation either way. 

It also appears that dependencies are not being shaded, which can result in 
conflicts with guava or wstx jar files in the class path. Such as noted by 
ORACLE {color:#00}Doc ID 2650046.1{color}

 commons-collections-3.2.2.jar
 commons-configuration2.jar
 commons-lang-2.6.jar
 guava-29.0-jre.jar
 log4j-1.2.17.jar
 slf4j-api-1.7.25.jar


> JDBC driver fails when using Kerberos due to missing dependencies
> -
>
> Key: HIVE-24323
> URL: https://issues.apache.org/jira/browse/HIVE-24323
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.1.0
>Reporter: N Campbell
>Priority: Major
>
> *The Apache Hive web pages historically implied that only 3-JAR files are 
> required*
> hadoop-auth
>  hadoop-common
>  hive-jdbc
> *If a connection is attempted using Kerberos authentication, it will fail due 
> to several missing dependencies*
> hadoop-auth-3.1.1.3.1.5.0-152.jar
>  hadoop-common-3.1.1.3.1.5.0-152.jar
>  hive-jdbc-3.1.0.3.1.5.0-152-standalone.jar
> *Dependencies*
> commons-collections-3.2.2.jar
>  commons-configuration2.jar
>  commons-lang-2.6.jar
>  guava-29.0-jre.jar
>  log4j-1.2.17.jar
>  slf4j-api-1.7.25.jar
> *It is unclear if the intent of the standalone JAR is to include these 
> dependencies or not.* 
> But does not seem to be any documentation either way.
> *It also appears that dependencies are not being shaded, which can result in 
> conflicts with guava or wstx jar files in the class path.* 
> Such as noted by ORACLE {color:#00}Doc ID 2650046.1{color}
> {color:#00} 
> com.ctc.wstx.io.StreamBootstrapper.getInstance(Ljava/lang/String;Lcom/ctc/wstx/io/SystemId;Ljava/io/InputStream;)Lcom/ctc/wstx/io/StreamBootstrapper;
>  ]
> java.lang.NoSuchMethodError: 
> com.ctc.wstx.io.StreamBootstrapper.getInstance(Ljava/lang/String;Lcom/ctc/wstx/io/SystemId;Ljava/io/InputStream;)Lcom/ctc/wstx/io/StreamBootstrapper;
>   at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2918)
>   at 
> org.apache.hadoop.conf.Configuration.parse(Configuration.java:2901){color}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-15561) JDBC driver does not implement getArray method for array types

2017-01-08 Thread N Campbell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-15561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

N Campbell updated HIVE-15561:
--
Component/s: JDBC

> JDBC driver does not implement getArray method for array types
> --
>
> Key: HIVE-15561
> URL: https://issues.apache.org/jira/browse/HIVE-15561
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 1.2.1
>Reporter: N Campbell
>
> getDriverVersion: 1.2.1000.2.5.0.0-1245
> The JDBC driver returns a JDBC type enum 2003 which is an array type. If an 
> application then attempts to call the getArray() method the driver will throw 
> a not implemented exception.
> Array z = rs.getArray("carrint");
> create table  if not exists TARRSINT (RNUM int , CARRSINT ARRAY< SMALLINT >)
>  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'  COLLECTION ITEMS TERMINATED 
> BY '^' LINES TERMINATED BY '\n'
>  STORED AS textfile ;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-15553) expression to compute running sum fails with Failed to breakup Windowing invocations into Groups

2017-01-06 Thread N Campbell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-15553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

N Campbell updated HIVE-15553:
--
Attachment: TOLAP

> expression to compute running sum fails with Failed to breakup Windowing 
> invocations into Groups
> 
>
> Key: HIVE-15553
> URL: https://issues.apache.org/jira/browse/HIVE-15553
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.1.0
>Reporter: N Campbell
> Attachments: TOLAP
>
>
> The following statement will fail in Hive.Expecting to compute the following 
> as is possible in Oracle, Db2, Teradata, ...
> select c1,   sum ( sum ( c3 ) ) over ( order by c1 )  from certtext.tolap 
> group by c1
> Error: Error while compiling statement: FAILED: SemanticException Failed to 
> breakup Windowing invocations into Groups. At least 1 group must only depend 
> on input columns. Also check for circular dependencies.
> Underlying error: org.apache.hadoop.hive.ql.parse.SemanticException: Line 
> 1:18 Expression not in GROUP BY key 'c3'
> SQLState:  42000
> ErrorCode: 4
> Note: if you use the following statement it can encounter a Tez error when c1 
> is a char vs string type. 
> select c1,  sum( c3 ), sum ( sum ( c3 ) ) over ( order by c1 )  from 
> certtext.tolap group by c1 
> C1SUM(C3) SUM(SUM(C3))OVER(ORDERBYC1)
> AAA   50  50
> BBB   20  70
> CCC   30  100
> DDD   40  140
> 50  190
> Table definition
> create table  if not exists TOLAP (RNUM int , C1 char(3), C2 char(2), C3 int, 
> C4 int)
>  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
>  STORED AS textfile ;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-15548) TEZ exception error when NULL ordering specification used on cursor or window agg

2017-01-06 Thread N Campbell (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15804555#comment-15804555
 ] 

N Campbell commented on HIVE-15548:
---

Another case not involving null order specification

SELECT
`D1`.`C0` AS `rnum`,
`D1`.`C1` AS `c1`,
`D1`.`C2` AS `c2`,
SUM(`D1`.`C3`) OVER( PARTITION BY `D1`.`C4` ) AS `cx`
FROM
(
   SELECT
   `rnum` AS `C0`,
   `c1` AS `C1`,
   `c2` AS `C2`,
   `c3` AS `C3`,
   SUM(`c3`) OVER( PARTITION BY `c1` ORDER BY `c1` ASC ) AS `C4`
   FROM `tolap`
)
`D1`


> TEZ exception error when NULL ordering specification used on cursor or window 
> agg
> -
>
> Key: HIVE-15548
> URL: https://issues.apache.org/jira/browse/HIVE-15548
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.1.0
>Reporter: N Campbell
> Attachments: TOLAP, TSET1
>
>
> Hive server: 2.1.0.2.5.3.0-37
> select c1, c2 from tset1 order by c1 asc nulls last,  c2 asc nulls first
>  select rnum , c1 , c2 , sum( c3 ) over (partition by sum( c3 ) over 
> (partition by c1 order by c1 )) from tolap 
> i.e. 
> Error: Error while processing statement: FAILED: Execution Error, return code 
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, 
> vertexName=Reducer 2, vertexId=vertex_1483461312952_0011_11_01, 
> diagnostics=[Task failed, taskId=task_1483461312952_0011_11_01_00, 
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( 
> failure ) : 
> attempt_1483461312952_0011_11_01_00_0:java.lang.RuntimeException: 
> java.lang.RuntimeException: java.io.EOFException: Detail: 
> "java.io.EOFException" occured for field 0 of 2 fields (INT, CHAR)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
>   at 
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
>   at 
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
>   at 
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at 
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
>   at 
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
>   at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
>   at 
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> table definitions:
> create table  if not exists TSET1 (RNUM int , C1 int, C2 char(3))
>  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
>  STORED AS textfile ;
> create table  if not exists TOLAP (RNUM int , C1 char(3), C2 char(2), C3 int, 
> C4 int)
>  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
>  STORED AS textfile ;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-15548) TEZ exception error when NULL ordering specification used on cursor or window agg

2017-01-05 Thread N Campbell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

N Campbell updated HIVE-15548:
--
Description: 
Hive server: 2.1.0.2.5.3.0-37

select c1, c2 from tset1 order by c1 asc nulls last,  c2 asc nulls first

 select rnum , c1 , c2 , sum( c3 ) over (partition by sum( c3 ) over (partition 
by c1 order by c1 )) from tolap 



i.e. 

Error: Error while processing statement: FAILED: Execution Error, return code 2 
from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, 
vertexName=Reducer 2, vertexId=vertex_1483461312952_0011_11_01, 
diagnostics=[Task failed, taskId=task_1483461312952_0011_11_01_00, 
diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( 
failure ) : 
attempt_1483461312952_0011_11_01_00_0:java.lang.RuntimeException: 
java.lang.RuntimeException: java.io.EOFException: Detail: 
"java.io.EOFException" occured for field 0 of 2 fields (INT, CHAR)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at 
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)


table definitions:


create table  if not exists TSET1 (RNUM int , C1 int, C2 char(3))
 ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
 STORED AS textfile ;

create table  if not exists TOLAP (RNUM int , C1 char(3), C2 char(2), C3 int, 
C4 int)
 ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
 STORED AS textfile ;



  was:
select c1, c2 from tset1 order by c1 asc nulls last,  c2 asc nulls first

 select rnum , c1 , c2 , sum( c3 ) over (partition by sum( c3 ) over (partition 
by c1 order by c1 )) from tolap 



i.e. 

Error: Error while processing statement: FAILED: Execution Error, return code 2 
from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, 
vertexName=Reducer 2, vertexId=vertex_1483461312952_0011_11_01, 
diagnostics=[Task failed, taskId=task_1483461312952_0011_11_01_00, 
diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( 
failure ) : 
attempt_1483461312952_0011_11_01_00_0:java.lang.RuntimeException: 
java.lang.RuntimeException: java.io.EOFException: Detail: 
"java.io.EOFException" occured for field 0 of 2 fields (INT, CHAR)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
at 
org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
at 
org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
at 
org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at

[jira] [Updated] (HIVE-15548) TEZ exception error when NULL ordering specification used on cursor or window agg

2017-01-05 Thread N Campbell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

N Campbell updated HIVE-15548:
--
Attachment: TSET1
TOLAP

> TEZ exception error when NULL ordering specification used on cursor or window 
> agg
> -
>
> Key: HIVE-15548
> URL: https://issues.apache.org/jira/browse/HIVE-15548
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.1.0
>Reporter: N Campbell
> Attachments: TOLAP, TSET1
>
>
> select c1, c2 from tset1 order by c1 asc nulls last,  c2 asc nulls first
>  select rnum , c1 , c2 , sum( c3 ) over (partition by sum( c3 ) over 
> (partition by c1 order by c1 )) from tolap 
> i.e. 
> Error: Error while processing statement: FAILED: Execution Error, return code 
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, 
> vertexName=Reducer 2, vertexId=vertex_1483461312952_0011_11_01, 
> diagnostics=[Task failed, taskId=task_1483461312952_0011_11_01_00, 
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( 
> failure ) : 
> attempt_1483461312952_0011_11_01_00_0:java.lang.RuntimeException: 
> java.lang.RuntimeException: java.io.EOFException: Detail: 
> "java.io.EOFException" occured for field 0 of 2 fields (INT, CHAR)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
>   at 
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
>   at 
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
>   at 
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:422)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at 
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
>   at 
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
>   at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
>   at 
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> table definitions:
> create table  if not exists TSET1 (RNUM int , C1 int, C2 char(3))
>  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
>  STORED AS textfile ;
> create table  if not exists TOLAP (RNUM int , C1 char(3), C2 char(2), C3 int, 
> C4 int)
>  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
>  STORED AS textfile ;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-15547) invalid result set when order by includes nulls sort last

2017-01-05 Thread N Campbell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

N Campbell updated HIVE-15547:
--
Description: 
Query that attempts to sort nulls last produces an incorrect result set.
Data is correct when order by has no null order specification or specification 
is FIRST.

Hive server version: 2.1.0.2.5.3.0-37

Query:
SELECT `tint`.`rnum`, `tint`.`cint` FROM `tint` ORDER BY `tint`.`rnum` ASC 
NULLS LAST

Results:
tint.rnum   tint.cint
  
  -1
  0
  1
  10


Source data
rnumcint
0   
1   -1
2   0
3   1
4   10

Table
create table  if not exists TINT ( RNUM int , CINT int   )
 ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
 STORED AS textfile  ;

  was:
Query that attempts to sort nulls last produces an incorrect result set.
(1) first column contains all null values which is wrong
(2) second column has not sorted the only null value as the last row

Hive server version: 2.1.0.2.5.3.0-37

Query:
SELECT `tint`.`rnum`, `tint`.`cint` FROM `tint` ORDER BY `tint`.`rnum` ASC 
NULLS LAST

Results:
tint.rnum   tint.cint
  
  -1
  0
  1
  10


Source data
rnumcint
0   
1   -1
2   0
3   1
4   10

Table
create table  if not exists TINT ( RNUM int , CINT int   )
 ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
 STORED AS textfile  ;


> invalid result set when order by includes nulls sort last
> -
>
> Key: HIVE-15547
> URL: https://issues.apache.org/jira/browse/HIVE-15547
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.1.0
>Reporter: N Campbell
>
> Query that attempts to sort nulls last produces an incorrect result set.
> Data is correct when order by has no null order specification or 
> specification is FIRST.
> Hive server version: 2.1.0.2.5.3.0-37
> Query:
> SELECT `tint`.`rnum`, `tint`.`cint` FROM `tint` ORDER BY `tint`.`rnum` ASC 
> NULLS LAST
> Results:
> tint.rnum tint.cint
> 
> -1
> 0
> 1
> 10
> Source data
> rnum  cint
> 0 
> 1 -1
> 2 0
> 3 1
> 4 10
> Table
> create table  if not exists TINT ( RNUM int , CINT int   )
>  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
>  STORED AS textfile  ;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-15547) invalid result set when order by includes nulls sort last

2017-01-05 Thread N Campbell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-15547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

N Campbell updated HIVE-15547:
--
Summary: invalid result set when order by includes nulls sort last  (was: 
nulls not sorted last on cursor specification)

> invalid result set when order by includes nulls sort last
> -
>
> Key: HIVE-15547
> URL: https://issues.apache.org/jira/browse/HIVE-15547
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 2.1.0
>Reporter: N Campbell
>
> Query that attempts to sort nulls last produces an incorrect result set.
> (1) first column contains all null values which is wrong
> (2) second column has not sorted the only null value as the last row
> Hive server version: 2.1.0.2.5.3.0-37
> Query:
> SELECT `tint`.`rnum`, `tint`.`cint` FROM `tint` ORDER BY `tint`.`rnum` ASC 
> NULLS LAST
> Results:
> tint.rnum tint.cint
> 
> -1
> 0
> 1
> 10
> Source data
> rnum  cint
> 0 
> 1 -1
> 2 0
> 3 1
> 4 10
> Table
> create table  if not exists TINT ( RNUM int , CINT int   )
>  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
>  STORED AS textfile  ;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9534) incorrect result set for query that projects a windowed aggregate

2015-09-21 Thread N Campbell (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-9534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901809#comment-14901809
 ] 

N Campbell commented on HIVE-9534:
--

IBM Netezza 7.2, IBM Big Insights, SAP Hana SP10, ORACLE 12 etc all return the 
expected result. 

> incorrect result set for query that projects a windowed aggregate
> -
>
> Key: HIVE-9534
> URL: https://issues.apache.org/jira/browse/HIVE-9534
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Reporter: N Campbell
>Assignee: Chaoyu Tang
>
> Result set returned by Hive has one row instead of 5
> {code}
> select avg(distinct tsint.csint) over () from tsint 
> create table  if not exists TSINT (RNUM int , CSINT smallint)
>  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
>  STORED AS TEXTFILE;
> 0|\N
> 1|-1
> 2|0
> 3|1
> 4|10
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9534) incorrect result set for query that projects a windowed aggregate

2015-09-21 Thread N Campbell (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-9534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14901818#comment-14901818
 ] 

N Campbell commented on HIVE-9534:
--

should say Netezza 7.2 and ORACLE 12.

> incorrect result set for query that projects a windowed aggregate
> -
>
> Key: HIVE-9534
> URL: https://issues.apache.org/jira/browse/HIVE-9534
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Reporter: N Campbell
>Assignee: Chaoyu Tang
>
> Result set returned by Hive has one row instead of 5
> {code}
> select avg(distinct tsint.csint) over () from tsint 
> create table  if not exists TSINT (RNUM int , CSINT smallint)
>  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
>  STORED AS TEXTFILE;
> 0|\N
> 1|-1
> 2|0
> 3|1
> 4|10
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-11371) Null pointer exception for nested table query when using ORC versus text

2015-07-25 Thread N Campbell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-11371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

N Campbell updated HIVE-11371:
--
Attachment: TJOIN4
TJOIN3
TJOIN2
TJOIN1

 Null pointer exception for nested table query when using ORC versus text
 

 Key: HIVE-11371
 URL: https://issues.apache.org/jira/browse/HIVE-11371
 Project: Hive
  Issue Type: Bug
Affects Versions: 1.2.0
Reporter: N Campbell
 Attachments: TJOIN1, TJOIN2, TJOIN3, TJOIN4


 Following query will fail if the file format is ORC 
 select tj1rnum, tj2rnum, tjoin3.rnum as rnumt3 from   (select tjoin1.rnum 
 tj1rnum, tjoin2.rnum tj2rnum, tjoin2.c1 tj2c1  from tjoin1 left outer join 
 tjoin2 on tjoin1.c1 = tjoin2.c1 ) tj  left outer join tjoin3 on tj2c1 = 
 tjoin3.c1 
 aused by: java.lang.NullPointerException
   at 
 org.apache.hadoop.hive.ql.exec.vector.VectorCopyRow$LongCopyRow.copy(VectorCopyRow.java:60)
   at 
 org.apache.hadoop.hive.ql.exec.vector.VectorCopyRow.copyByReference(VectorCopyRow.java:260)
   at 
 org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultMultiValue(VectorMapJoinGenerateResultOperator.java:238)
   at 
 org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinOuterGenerateResultOperator.finishOuter(VectorMapJoinOuterGenerateResultOperator.java:495)
   at 
 org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinOuterLongOperator.process(VectorMapJoinOuterLongOperator.java:430)
   ... 22 more
 ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 
 killedTasks:0, Vertex vertex_1437788144883_0004_2_02 [Map 1] killed/failed 
 due to:null]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 
 killedVertices:0
 SQLState:  08S01
 ErrorCode: 2
 getDatabaseProductNameApache Hive
 getDatabaseProductVersion 1.2.1.2.3.0.0-2557
 getDriverName Hive JDBC
 getDriverVersion  1.2.1.2.3.0.0-2557
 getDriverMajorVersion 1
 getDriverMinorVersion 2
 create table  if not exists TJOIN1 (RNUM int , C1 int, C2 int)
 -- ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
  STORED AS orc;
 create table  if not exists TJOIN2 (RNUM int , C1 int, C2 char(2))
 -- ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
  STORED AS orc ;
 create table  if not exists TJOIN3 (RNUM int , C1 int, C2 char(2))
 -- ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
  STORED AS orc ;
 create table  if not exists TJOIN4 (RNUM int , C1 int, C2 char(2))
 -- ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
  STORED AS orc ;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-11372) join with betwee predicate comparing integer types returns no rows when ORC format used

2015-07-25 Thread N Campbell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-11372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

N Campbell updated HIVE-11372:
--
Attachment: TSINT
TINT

 join with betwee predicate comparing integer types returns no rows when ORC 
 format used
 ---

 Key: HIVE-11372
 URL: https://issues.apache.org/jira/browse/HIVE-11372
 Project: Hive
  Issue Type: Bug
Affects Versions: 1.2.0
Reporter: N Campbell
 Attachments: TINT, TSINT


 getDatabaseProductNameApache Hive
 getDatabaseProductVersion 1.2.1.2.3.0.0-2557
 getDriverName Hive JDBC
 getDriverVersion  1.2.1.2.3.0.0-2557
 getDriverMajorVersion 1
 getDriverMinorVersion 2
 select tint.rnum, tsint.rnum from tint , tsint where tint.cint between 
 tsint.csint and tsint.csint
 when ORC used no rows returned versus TEXT
 create table  if not exists TSINT ( RNUM int , CSINT smallint   )
 -- ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
  STORED AS orc  ;
 create table  if not exists TINT ( RNUM int , CINT int   )
 -- ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
  STORED AS orc  ;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-10936) incorrect result set when hive.vectorized.execution.enabled = true with predicate casting to CHAR or VARCHAR

2015-06-04 Thread N Campbell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-10936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

N Campbell updated HIVE-10936:
--
Attachment: GO_TIME_DIM.zip

 incorrect result set when hive.vectorized.execution.enabled = true with 
 predicate casting to CHAR or VARCHAR
 

 Key: HIVE-10936
 URL: https://issues.apache.org/jira/browse/HIVE-10936
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 0.14.0
 Environment: In this case using HDP install of Hive - 0.14.0.2.2.4.2-2
Reporter: N Campbell
 Attachments: GO_TIME_DIM.zip


 Query returns data when set hive.vectorized.execution.enabled = false -or- if 
 target of CAST is STRING and not CHAR/VARCHAR
 set hive.vectorized.execution.enabled = true;
 select 
   `GO_TIME_DIM`.`day_key`
 from 
   `gosalesdw1021`.`go_time_dim` `GO_TIME_DIM` 
 where 
   CAST(`GO_TIME_DIM`.`current_year` AS CHAR(4)) = '2010' 
 group by 
   `GO_TIME_DIM`.`day_key`;
 create table GO_TIME_DIM ( DAY_KEY int , DAY_DATE timestamp , MONTH_KEY int , 
 CURRENT_MONTH smallint , MONTH_NUMBER int , QUARTER_KEY int , CURRENT_QUARTER 
 smallint , CURRENT_YEAR smallint , DAY_OF_WEEK smallint , DAY_OF_MONTH 
 smallint , DAYS_IN_MONTH smallint , DAY_OF_YEAR smallint , WEEK_OF_MONTH 
 smallint , WEEK_OF_QUARTER smallint , WEEK_OF_YEAR smallint , MONTH_EN string 
 , WEEKDAY_EN string , MONTH_DE string , WEEKDAY_DE string , MONTH_FR string , 
 WEEKDAY_FR string , MONTH_JA string , WEEKDAY_JA string , MONTH_AR string , 
 WEEKDAY_AR string , MONTH_CS string , WEEKDAY_CS string , MONTH_DA string , 
 WEEKDAY_DA string , MONTH_EL string , WEEKDAY_EL string , MONTH_ES string , 
 WEEKDAY_ES string , MONTH_FI string , WEEKDAY_FI string , MONTH_HR string , 
 WEEKDAY_HR string , MONTH_HU string , WEEKDAY_HU string , MONTH_ID string , 
 WEEKDAY_ID string , MONTH_IT string , WEEKDAY_IT string , MONTH_KK string , 
 WEEKDAY_KK string , MONTH_KO string , WEEKDAY_KO string , MONTH_MS string , 
 WEEKDAY_MS string , MONTH_NL string , WEEKDAY_NL string , MONTH_NO string , 
 WEEKDAY_NO string , MONTH_PL string , WEEKDAY_PL string , MONTH_PT string , 
 WEEKDAY_PT string , MONTH_RO string , WEEKDAY_RO string , MONTH_RU string , 
 WEEKDAY_RU string , MONTH_SC string , WEEKDAY_SC string , MONTH_SL string , 
 WEEKDAY_SL string , MONTH_SV string , WEEKDAY_SV string , MONTH_TC string , 
 WEEKDAY_TC string , MONTH_TH string , WEEKDAY_TH string , MONTH_TR string , 
 WEEKDAY_TR string )
 ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
  STORED AS TEXTFILE
 LOCATION '../GO_TIME_DIM';
 Then create an ORC equivalent table and load it
 insert overwrite table 
 GO_TIME_DIM
 select * from TEXT.GO_TIME_DIM
 ;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9534) incorrect result set for query that projects a windowed aggregate

2015-05-05 Thread N Campbell (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-9534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14529608#comment-14529608
 ] 

N Campbell commented on HIVE-9534:
--

re your comment about ORACLE

select avg(distinct tsint.csint) over () from tsint

null, -1, 0, 1, 10
ORACLE Oracle Database 12c Enterprise Edition ( 12.1.0.2.0)   returns 2.5,
2.5, 2.5, 2.5, 2.5





 incorrect result set for query that projects a windowed aggregate
 -

 Key: HIVE-9534
 URL: https://issues.apache.org/jira/browse/HIVE-9534
 Project: Hive
  Issue Type: Bug
  Components: SQL
Reporter: N Campbell
Assignee: Chaoyu Tang

 Result set returned by Hive has one row instead of 5
 {code}
 select avg(distinct tsint.csint) over () from tsint 
 create table  if not exists TSINT (RNUM int , CSINT smallint)
  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
  STORED AS TEXTFILE;
 0|\N
 1|-1
 2|0
 3|1
 4|10
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-10488) cast DATE as TIMESTAMP returns incorrect values

2015-04-27 Thread N Campbell (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514263#comment-14514263
 ] 

N Campbell commented on HIVE-10488:
---

Cast was using a column of type date and not literals or value expression.
Variance was the format the data was in.



 cast DATE as TIMESTAMP returns incorrect values
 ---

 Key: HIVE-10488
 URL: https://issues.apache.org/jira/browse/HIVE-10488
 Project: Hive
  Issue Type: Bug
  Components: SQL
Affects Versions: 0.13.1
Reporter: N Campbell
Assignee: Chaoyu Tang

 same data in textfile works
 same data loaded into an ORC table does not
 connection property of tez/mr makes no difference.
 select rnum, cdt, cast (cdt as timestamp) from tdt
 0 null  null
 1 1996-01-01  1969-12-31 19:00:09.496
 2 2000-01-01  1969-12-31 19:00:10.957
 3 2000-12-31  1969-12-31 19:00:11.322
 vs
 0 null  null
 1 1996-01-01  1996-01-01 00:00:00.0
 2 2000-01-01  2000-01-01 00:00:00.0
 3 2000-12-31  2000-12-31 00:00:00.0
 create table  if not exists TDT ( RNUM int , CDT date   )
  STORED AS orc  ;
 insert overwrite table TDT select * from  text.TDT;
 0|\N
 1|1996-01-01
 2|2000-01-01
 3|2000-12-31



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9534) incorrect result set for query that projects a windowed aggregate

2015-04-26 Thread N Campbell (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-9534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14512997#comment-14512997
 ] 

N Campbell commented on HIVE-9534:
--

While Postgres may not support ISO-SQL syntax as shown other vendors do. That 
said, if the engine does not attempt to support distinct aggregation then a 
parsing error should be thrown -and- the documentation improved at

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+WindowingAndAnalytics

 incorrect result set for query that projects a windowed aggregate
 -

 Key: HIVE-9534
 URL: https://issues.apache.org/jira/browse/HIVE-9534
 Project: Hive
  Issue Type: Bug
  Components: SQL
Reporter: N Campbell

 Result set returned by Hive has one row instead of 5
 {code}
 select avg(distinct tsint.csint) over () from tsint 
 create table  if not exists TSINT (RNUM int , CSINT smallint)
  ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
  STORED AS TEXTFILE;
 0|\N
 1|-1
 2|0
 3|1
 4|10
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)