[jira] [Created] (HIVE-13280) Error when more than 1 mapper for HBase storage handler

[Damien Carol (JIRA)]

Damien Carol created HIVE-13280:
---

 Summary: Error when more than 1 mapper for HBase storage handler
 Key: HIVE-13280
 URL: https://issues.apache.org/jira/browse/HIVE-13280
 Project: Hive
  Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Damien Carol






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-11111) Insert on skewed table with STORED AS DIRECTORY is broken

[Damien Carol (JIRA)]

Damien Carol created HIVE-1:
---

 Summary: Insert on skewed table with STORED AS DIRECTORY is broken
 Key: HIVE-1
 URL: https://issues.apache.org/jira/browse/HIVE-1
 Project: Hive
  Issue Type: Bug
Affects Versions: 1.2.0
Reporter: Damien Carol


Doing these queries fails:
{code:sql}
RESET;

DROP TABLE IF EXISTS testskew;

CREATE TABLE IF NOT EXISTS testskew (key int, value STRING)
SKEWED BY (key) ON (1,5,6) STORED AS DIRECTORIES
STORED AS ORC;

insert into testskew VALUES
(1, 'one'),
(1, 'one'),
(1, 'one'),
(1, 'one'),
(1, 'one'),
(1, 'one'),
(2, 'two'),
(3, 'three'),
(5, 'five'),
(5, 'five'),
(5, 'five'),
(5, 'five'),
(5, 'five'),
(6, 'six'),
(6, 'six'),
(6, 'six'),
(6, 'six'),
(6, 'six'),
(6, 'six');
{code}
Stacktrace:
{noformat}
INFO  : Session is already open
INFO  :

INFO  : Status: Running (Executing on YARN cluster with App id 
application_1434957292922_0059)

INFO  : Map 1: 0/1
INFO  : Map 1: 0(+1)/1
INFO  : Map 1: 1/1
INFO  : Loading data to table test.testskew from 
hdfs://nc-h07/user/hive/warehouse/test.db/testskew/.hive-staging_hive_2015-06-25_17-29-34_385_4424227988595852796-14/-ext-1
ERROR : Failed with exception checkPaths: 
hdfs://nc-h07/user/hive/warehouse/test.db/testskew/.hive-staging_hive_2015-06-25_17-29-34_385_4424227988595852796-14/-ext-1
 has nested directory 
hdfs://nc-h07/user/hive/warehouse/test.db/testskew/.hive-staging_hive_2015-06-25_17-29-34_385_4424227988595852796-14/-ext-1/HIVE_DEFAULT_LIST_BUCKETING_DIR_NAME
org.apache.hadoop.hive.ql.metadata.HiveException: checkPaths: 
hdfs://nc-h07/user/hive/warehouse/test.db/testskew/.hive-staging_hive_2015-06-25_17-29-34_385_4424227988595852796-14/-ext-1
 has nested directory 
hdfs://nc-h07/user/hive/warehouse/test.db/testskew/.hive-staging_hive_2015-06-25_17-29-34_385_4424227988595852796-14/-ext-1/HIVE_DEFAULT_LIST_BUCKETING_DIR_NAME
at org.apache.hadoop.hive.ql.metadata.Hive.checkPaths(Hive.java:2466)
at org.apache.hadoop.hive.ql.metadata.Hive.copyFiles(Hive.java:2701)
at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:1645)
at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:297)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at 
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1650)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1409)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1192)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1054)
at 
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:154)
at 
org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:71)
at 
org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:206)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at 
org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:218)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

Error: Error while processing statement: FAILED: Execution Error, return code 1 
from org.apache.hadoop.hive.ql.exec.MoveTask (state=08S01,code=1)
{noformat}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-9769) Error with OperationHandle for each query

[Damien Carol (JIRA)]

Damien Carol created HIVE-9769:
--

 Summary: Error with OperationHandle for each query
 Key: HIVE-9769
 URL: https://issues.apache.org/jira/browse/HIVE-9769
 Project: Hive
  Issue Type: Bug
Affects Versions: 1.0.0
 Environment: Hadoop 2.5.1, Tez 0.5.3
Reporter: Damien Carol
Priority: Minor


For each query, I have this error in logs :
{noformat}
2015-02-24 15:26:13,011 WARN  [HiveServer2-Handler-Pool: Thread-308]: 
thrift.ThriftCLIService (ThriftCLIService.java:FetchResults(596)) - Error 
fetching results:
org.apache.hive.service.cli.HiveSQLException: Couldn't find log associated with 
operation handle: OperationHandle [opType=EXECUTE_STATEMENT, 
getHandleIdentifier()=9b9a9541-a29e-460b-bc8c-344452a02b96]
at 
org.apache.hive.service.cli.operation.OperationManager.getOperationLogRowSet(OperationManager.java:257)
at 
org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:656)
at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:79)
at 
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:37)
at 
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:64)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at 
org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:536)
at 
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:60)
at com.sun.proxy.$Proxy22.fetchResults(Unknown Source)
at 
org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:450)
at 
org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:587)
at 
org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553)
at 
org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at 
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9277) Hybrid Hybrid Grace Hash Join

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9277:
---
Description: 
We are proposing an enhanced hash join algorithm called _“hybrid hybrid grace 
hash join”_.

We can benefit from this feature as illustrated below:
* The query will not fail even if the estimated memory requirement is slightly 
wrong
* Expensive garbage collection overhead can be avoided when hash table grows
* Join execution using a Map join operator even though the small table doesn't 
fit in memory as spilling some data from the build and probe sides will still 
be cheaper than having to shuffle the large fact table

The design was based on Hadoop’s parallel processing capability and significant 
amount of memory available.

  was:
We are proposing an enhanced hash join algorithm called “hybrid hybrid grace 
hash join”. We can benefit from this feature as illustrated below:
o The query will not fail even if the estimated memory requirement is slightly 
wrong
o Expensive garbage collection overhead can be avoided when hash table grows
o Join execution using a Map join operator even though the small table doesn't 
fit in memory as spilling some data from the build and probe sides will still 
be cheaper than having to shuffle the large fact table
The design was based on Hadoop’s parallel processing capability and significant 
amount of memory available.


 Hybrid Hybrid Grace Hash Join
 -

 Key: HIVE-9277
 URL: https://issues.apache.org/jira/browse/HIVE-9277
 Project: Hive
  Issue Type: New Feature
  Components: Physical Optimizer
Reporter: Wei Zheng
Assignee: Wei Zheng
  Labels: join
 Attachments: HIVE-9277.01.patch, HIVE-9277.02.patch, 
 High-leveldesignforHybridHybridGraceHashJoinv1.0.pdf


 We are proposing an enhanced hash join algorithm called _“hybrid hybrid grace 
 hash join”_.
 We can benefit from this feature as illustrated below:
 * The query will not fail even if the estimated memory requirement is 
 slightly wrong
 * Expensive garbage collection overhead can be avoided when hash table grows
 * Join execution using a Map join operator even though the small table 
 doesn't fit in memory as spilling some data from the build and probe sides 
 will still be cheaper than having to shuffle the large fact table
 The design was based on Hadoop’s parallel processing capability and 
 significant amount of memory available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HIVE-9737) Issue come while creating the table in hbase using java Impla API

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol reassigned HIVE-9737:
--

Assignee: Damien Carol

 Issue come while creating the table in hbase using java Impla API
 -

 Key: HIVE-9737
 URL: https://issues.apache.org/jira/browse/HIVE-9737
 Project: Hive
  Issue Type: Bug
  Components: Database/Schema
 Environment: Linux OS, Using Java Impala API with thrift, Cloudera 
 Hbase database
Reporter: Mohit Sharma
Assignee: Damien Carol

 I am trying to create hbase table using this query
 CREATE TABLE foo4(rowkey STRING, a STRING, b STRING) STORED BY 
 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES 
 ('hbase.columns.mapping' = ':key,f:c1,f:c2') TBLPROPERTIES 
 ('hbase.table.name' = 'bar4');
 with the help of java Impala api and I am sharing you code link
 https://github.com/pauldeschacht/impala-java-client
 When I am trying to create table in habse I am facing this issue
 AnalysisException: Syntax error in line 2:
 STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
^
 Encountered: BY
 Expected: AS
 CAUSED BY: Exception: Syntax error,HY000,0,false
 Please help me what I do?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9737) Issue come while creating the table in hbase using java Impla API

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328886#comment-14328886
 ] 

Damien Carol commented on HIVE-9737:


[~mohit.sharmamca09] I don't understand, you're talking about a bug in Impala 
thrift API?
As you said in github :
bq. It does not depend on the HiveServer2.
Why are you creating a bug bug in HIVE JIRA?

Plus, it seems that your bug is more like using HiveQL syntax in Impala than 
a bug in HIVE itself.

 Issue come while creating the table in hbase using java Impla API
 -

 Key: HIVE-9737
 URL: https://issues.apache.org/jira/browse/HIVE-9737
 Project: Hive
  Issue Type: Bug
  Components: Database/Schema
 Environment: Linux OS, Using Java Impala API with thrift, Cloudera 
 Hbase database
Reporter: Mohit Sharma

 I am trying to create hbase table using this query
 CREATE TABLE foo4(rowkey STRING, a STRING, b STRING) STORED BY 
 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES 
 ('hbase.columns.mapping' = ':key,f:c1,f:c2') TBLPROPERTIES 
 ('hbase.table.name' = 'bar4');
 with the help of java Impala api and I am sharing you code link
 https://github.com/pauldeschacht/impala-java-client
 When I am trying to create table in habse I am facing this issue
 AnalysisException: Syntax error in line 2:
 STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
^
 Encountered: BY
 Expected: AS
 CAUSED BY: Exception: Syntax error,HY000,0,false
 Please help me what I do?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9712) Row count and data size are set to LONG.MAX when source table has 0 rows

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9712:
---
Summary: Row count and data size are set to LONG.MAX when source table has 
0 rows  (was: Hive : Row count and data size are set to LONG.MAX when source 
table has 0 rows)

 Row count and data size are set to LONG.MAX when source table has 0 rows
 

 Key: HIVE-9712
 URL: https://issues.apache.org/jira/browse/HIVE-9712
 Project: Hive
  Issue Type: Bug
  Components: Physical Optimizer
Affects Versions: 0.14.0
Reporter: Mostafa Mokhtar
Assignee: Prasanth Jayachandran

 TPC-DS Q66 generates and in-efficient plan because cardinality estimate of 
 dimension table gets set to 9223372036854775807.
 {code}
 Map 10 
 Map Operator Tree:
 TableScan
   alias: ship_mode
   filterExpr: ((sm_carrier) IN ('DIAMOND', 'AIRBORNE') and 
 sm_ship_mode_sk is not null) (type: boolean)
   Statistics: Num rows: 0 Data size: 47 Basic stats: PARTIAL 
 Column stats: COMPLETE
   Filter Operator
 predicate: ((sm_carrier) IN ('DIAMOND', 'AIRBORNE') and 
 sm_ship_mode_sk is not null) (type: boolean)
 Statistics: Num rows: 9223372036854775807 Data size: 
 9223372036854775807 Basic stats: COMPLETE Column stats: COMPLETE
 Select Operator
   expressions: sm_ship_mode_sk (type: int)
   outputColumnNames: _col0
   Statistics: Num rows: 9223372036854775807 Data size: 
 9223372036854775807 Basic stats: COMPLETE Column stats: COMPLETE
   Reduce Output Operator
 key expressions: _col0 (type: int)
 sort order: +
 Map-reduce partition columns: _col0 (type: int)
 Statistics: Num rows: 9223372036854775807 Data size: 
 9223372036854775807 Basic stats: COMPLETE Column stats: COMPLETE
 Execution mode: vectorized
 {code}
 Full plan 
 {code}
 explain  
 select   
  w_warehouse_name
   ,w_warehouse_sq_ft
   ,w_city
   ,w_county
   ,w_state
   ,w_country
 ,ship_carriers
 ,year
   ,sum(jan_sales) as jan_sales
   ,sum(feb_sales) as feb_sales
   ,sum(mar_sales) as mar_sales
   ,sum(apr_sales) as apr_sales
   ,sum(may_sales) as may_sales
   ,sum(jun_sales) as jun_sales
   ,sum(jul_sales) as jul_sales
   ,sum(aug_sales) as aug_sales
   ,sum(sep_sales) as sep_sales
   ,sum(oct_sales) as oct_sales
   ,sum(nov_sales) as nov_sales
   ,sum(dec_sales) as dec_sales
   ,sum(jan_sales/w_warehouse_sq_ft) as jan_sales_per_sq_foot
   ,sum(feb_sales/w_warehouse_sq_ft) as feb_sales_per_sq_foot
   ,sum(mar_sales/w_warehouse_sq_ft) as mar_sales_per_sq_foot
   ,sum(apr_sales/w_warehouse_sq_ft) as apr_sales_per_sq_foot
   ,sum(may_sales/w_warehouse_sq_ft) as may_sales_per_sq_foot
   ,sum(jun_sales/w_warehouse_sq_ft) as jun_sales_per_sq_foot
   ,sum(jul_sales/w_warehouse_sq_ft) as jul_sales_per_sq_foot
   ,sum(aug_sales/w_warehouse_sq_ft) as aug_sales_per_sq_foot
   ,sum(sep_sales/w_warehouse_sq_ft) as sep_sales_per_sq_foot
   ,sum(oct_sales/w_warehouse_sq_ft) as oct_sales_per_sq_foot
   ,sum(nov_sales/w_warehouse_sq_ft) as nov_sales_per_sq_foot
   ,sum(dec_sales/w_warehouse_sq_ft) as dec_sales_per_sq_foot
   ,sum(jan_net) as jan_net
   ,sum(feb_net) as feb_net
   ,sum(mar_net) as mar_net
   ,sum(apr_net) as apr_net
   ,sum(may_net) as may_net
   ,sum(jun_net) as jun_net
   ,sum(jul_net) as jul_net
   ,sum(aug_net) as aug_net
   ,sum(sep_net) as sep_net
   ,sum(oct_net) as oct_net
   ,sum(nov_net) as nov_net
   ,sum(dec_net) as dec_net
  from (
 select 
   w_warehouse_name
   ,w_warehouse_sq_ft
   ,w_city
   ,w_county
   ,w_state
   ,w_country
   ,concat('DIAMOND', ',', 'AIRBORNE') as ship_carriers
 ,d_year as year
   ,sum(case when d_moy = 1 
   then ws_sales_price* ws_quantity else 0 end) as jan_sales
   ,sum(case when d_moy = 2 
   then ws_sales_price* ws_quantity else 0 end) as feb_sales
   ,sum(case when d_moy = 3 
   then ws_sales_price* ws_quantity else 0 end) as mar_sales
   ,sum(case when d_moy = 4 
   then ws_sales_price* ws_quantity else 0 end) as apr_sales
   ,sum(case when d_moy = 5 
   then ws_sales_price* ws_quantity else 0 end) as may_sales
   ,sum(case when d_moy = 6 
   then ws_sales_price* ws_quantity else 0 end) as jun_sales
   ,sum(case when d_moy = 7 
  

[jira] [Commented] (HIVE-9669) selected columns

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14319095#comment-14319095
 ] 

Damien Carol commented on HIVE-9669:


[~smilevasu] Please use dev list instead of creating duplicate bug.

 selected columns
 

 Key: HIVE-9669
 URL: https://issues.apache.org/jira/browse/HIVE-9669
 Project: Hive
  Issue Type: Bug
Reporter: srinivas
Assignee: Damien Carol

 Hi Team,
 In Hive 1.0, selected columns patch is updated?
 Because i am not able to insert selected columns.
 I am using Hive 1.0 bin, how can i apply patch directly in Hive instead of 
 trunk or source.
 Thanks in advance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HIVE-9669) selected columns

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol resolved HIVE-9669.

Resolution: Duplicate

 selected columns
 

 Key: HIVE-9669
 URL: https://issues.apache.org/jira/browse/HIVE-9669
 Project: Hive
  Issue Type: Bug
Reporter: srinivas
Assignee: Damien Carol

 Hi Team,
 In Hive 1.0, selected columns patch is updated?
 Because i am not able to insert selected columns.
 I am using Hive 1.0 bin, how can i apply patch directly in Hive instead of 
 trunk or source.
 Thanks in advance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HIVE-9669) selected columns

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol reassigned HIVE-9669:
--

Assignee: Damien Carol

 selected columns
 

 Key: HIVE-9669
 URL: https://issues.apache.org/jira/browse/HIVE-9669
 Project: Hive
  Issue Type: Bug
Reporter: srinivas
Assignee: Damien Carol

 Hi Team,
 In Hive 1.0, selected columns patch is updated?
 Because i am not able to insert selected columns.
 I am using Hive 1.0 bin, how can i apply patch directly in Hive instead of 
 trunk or source.
 Thanks in advance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9639) Create Index failed in Hive 1.0

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9639:
---
Description: 
Create Index in either type COMPACT or BITMAP does not work on version 1.0. The 
same query can run successfully in 0.13.0 and 0.14.0
{code:sql}
CREATE INDEX idx_id_employee_id
ON TABLE employee_id (employee_id)
AS 'COMPACT'
WITH DEFERRED REBUILD;
{code}
{noformat}
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask. Table not found 
default__employee_id_idx_id_employee_id__
{noformat}

  was:
Create Index in either type COMPACT or BITMAP does not work on version 1.0. The 
same query can run successfully in 0.13.0 and 0.14.0

CREATE INDEX idx_id_employee_id
ON TABLE employee_id (employee_id)
AS 'COMPACT'
WITH DEFERRED REBUILD;
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask. Table not found 
default__employee_id_idx_id_employee_id__


 Create Index failed in Hive 1.0
 ---

 Key: HIVE-9639
 URL: https://issues.apache.org/jira/browse/HIVE-9639
 Project: Hive
  Issue Type: Bug
  Components: Indexing, Parser
Affects Versions: 1.0.0
Reporter: Will Du

 Create Index in either type COMPACT or BITMAP does not work on version 1.0. 
 The same query can run successfully in 0.13.0 and 0.14.0
 {code:sql}
 CREATE INDEX idx_id_employee_id
 ON TABLE employee_id (employee_id)
 AS 'COMPACT'
 WITH DEFERRED REBUILD;
 {code}
 {noformat}
 FAILED: Execution Error, return code 1 from 
 org.apache.hadoop.hive.ql.exec.DDLTask. Table not found 
 default__employee_id_idx_id_employee_id__
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9639) Create Index failed in Hive 1.0

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315000#comment-14315000
 ] 

Damien Carol commented on HIVE-9639:


[~willddy] Could you provide the stack trace?

 Create Index failed in Hive 1.0
 ---

 Key: HIVE-9639
 URL: https://issues.apache.org/jira/browse/HIVE-9639
 Project: Hive
  Issue Type: Bug
  Components: Indexing, Parser
Affects Versions: 1.0.0
Reporter: Will Du

 Create Index in either type COMPACT or BITMAP does not work on version 1.0. 
 The same query can run successfully in 0.13.0 and 0.14.0
 {code:sql}
 CREATE INDEX idx_id_employee_id
 ON TABLE employee_id (employee_id)
 AS 'COMPACT'
 WITH DEFERRED REBUILD;
 {code}
 {noformat}
 FAILED: Execution Error, return code 1 from 
 org.apache.hadoop.hive.ql.exec.DDLTask. Table not found 
 default__employee_id_idx_id_employee_id__
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9616) Hive 0.14

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312044#comment-14312044
 ] 

Damien Carol commented on HIVE-9616:


After reading your description, it seems that it's duplicate of HIVE-9481.
Closing if.

 Hive 0.14
 -

 Key: HIVE-9616
 URL: https://issues.apache.org/jira/browse/HIVE-9616
 Project: Hive
  Issue Type: Bug
Reporter: srinivas

 Hi, 
 I am using hive 0.14 version which will support all crud operation as said by 
 support team
 I am not able to select specific columns to insert, like 
 insert into table table1 id,name,sal select id,name,sal from table2 where 
 table1.id = table2.id



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HIVE-9616) Hive 0.14

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol resolved HIVE-9616.

Resolution: Duplicate

 Hive 0.14
 -

 Key: HIVE-9616
 URL: https://issues.apache.org/jira/browse/HIVE-9616
 Project: Hive
  Issue Type: Bug
Reporter: srinivas
Assignee: Damien Carol

 Hi, 
 I am using hive 0.14 version which will support all crud operation as said by 
 support team
 I am not able to select specific columns to insert, like 
 insert into table table1 id,name,sal select id,name,sal from table2 where 
 table1.id = table2.id



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HIVE-9616) Hive 0.14

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol reassigned HIVE-9616:
--

Assignee: Damien Carol

 Hive 0.14
 -

 Key: HIVE-9616
 URL: https://issues.apache.org/jira/browse/HIVE-9616
 Project: Hive
  Issue Type: Bug
Reporter: srinivas
Assignee: Damien Carol

 Hi, 
 I am using hive 0.14 version which will support all crud operation as said by 
 support team
 I am not able to select specific columns to insert, like 
 insert into table table1 id,name,sal select id,name,sal from table2 where 
 table1.id = table2.id



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9397) SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9397:
---
Affects Version/s: 1.1.0

 SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
  Components: Beeline, Logical Optimizer
Affects Versions: 0.14.0, 1.0.0, 1.1, 1.1.0
Reporter: Damien Carol
Assignee: Navis
 Fix For: 1.2.0

 Attachments: HIVE-9397.1.patch.txt, HIVE-9397.2.patch.txt, 
 HIVE-9397.3.patch.txt


 These queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9397) SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303686#comment-14303686
 ] 

Damien Carol commented on HIVE-9397:


please keep in mind that's this bug is in 0.14 AND 0.15.

Damien CAROL
gopher://dams.zapto.org/1/




 SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.14.0, 0.15.0
Reporter: Damien Carol
Assignee: Navis
 Attachments: HIVE-9397.1.patch.txt, HIVE-9397.2.patch.txt


 These queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9558) [Parquet] support HiveDecimalWritable, HiveCharWritable, HiveVarcharWritable in vectorized mode

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304160#comment-14304160
 ] 

Damien Carol commented on HIVE-9558:


{{TestStreaming.testEndpointConnection}} is not related.
{{TestCliDriver.testCliDriver_vectorized_parquet_types}} is ok, you must modify 
the result of test ouput to have {{Execution mode: vectorized}} in execution 
plan.


 [Parquet] support HiveDecimalWritable, HiveCharWritable, HiveVarcharWritable 
 in vectorized mode
 ---

 Key: HIVE-9558
 URL: https://issues.apache.org/jira/browse/HIVE-9558
 Project: Hive
  Issue Type: Sub-task
Reporter: Dong Chen
Assignee: Dong Chen
 Attachments: HIVE-9558.patch


 When using Parquet in  vectorized mode, 
 {{VectorColumnAssignFactory.buildAssigners(..)}} does not handle 
 HiveDecimalWritable, HiveCharWritable, HiveVarcharWritable. 
 We need fix this and add test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9431) CBO (Calcite Return Path): Removing AST from ParseContext

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14301275#comment-14301275
 ] 

Damien Carol commented on HIVE-9431:


[~jcamachorodriguez] yes, 
{{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion}} is related 
to HIVE-9539

 CBO (Calcite Return Path): Removing AST from ParseContext
 -

 Key: HIVE-9431
 URL: https://issues.apache.org/jira/browse/HIVE-9431
 Project: Hive
  Issue Type: Sub-task
  Components: CBO
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
 Fix For: 0.15.0

 Attachments: HIVE-9431.01.patch, HIVE-9431.02.patch, 
 HIVE-9431.03.patch, HIVE-9431.04.patch, HIVE-9431.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9539) Wrong check of version format in TestWebHCatE2e.getHiveVersion()

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9539:
---
Description: 
Test of {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} 
check that version in that format : {{0.[0-9]+.[0-9]+.*}}
This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}


 Wrong check of version format in TestWebHCatE2e.getHiveVersion()
 

 Key: HIVE-9539
 URL: https://issues.apache.org/jira/browse/HIVE-9539
 Project: Hive
  Issue Type: Bug
Reporter: Damien Carol
Priority: Minor

 Test of 
 {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} check 
 that version in that format : {{0.[0-9]+.[0-9]+.*}}
 This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9538) Exclude thirdparty directory from tarballs

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300215#comment-14300215
 ] 

Damien Carol commented on HIVE-9538:


Failed test org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion is 
related to HIVE-9539.
It's not related to this patch.

 Exclude thirdparty directory from tarballs
 --

 Key: HIVE-9538
 URL: https://issues.apache.org/jira/browse/HIVE-9538
 Project: Hive
  Issue Type: Improvement
Affects Versions: spark-branch, 1.1.0
Reporter: Brock Noland
Assignee: Brock Noland
Priority: Minor
 Attachments: HIVE-9538.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9539) Wrong check of version format in TestWebHCatE2e.getHiveVersion()

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9539:
---
Description: 
Bug caused by HIVE-9485.

Test of {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} 
check that version in that format : {{0.[0-9]+.[0-9]+.*}}
This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}


  was:
Test of {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} 
check that version in that format : {{0.[0-9]+.[0-9]+.*}}
This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}



 Wrong check of version format in TestWebHCatE2e.getHiveVersion()
 

 Key: HIVE-9539
 URL: https://issues.apache.org/jira/browse/HIVE-9539
 Project: Hive
  Issue Type: Bug
  Components: HCatalog
Affects Versions: 1.2.0
Reporter: Damien Carol
Priority: Minor

 Bug caused by HIVE-9485.
 Test of 
 {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} check 
 that version in that format : {{0.[0-9]+.[0-9]+.*}}
 This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9525) Enable constant propagation optimization in few existing tests where it was disabled.

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300214#comment-14300214
 ] 

Damien Carol commented on HIVE-9525:


The last failed test rely on HIVE-9539.
It's not related to this path.

 Enable constant propagation optimization in few existing tests where it was 
 disabled.
 -

 Key: HIVE-9525
 URL: https://issues.apache.org/jira/browse/HIVE-9525
 Project: Hive
  Issue Type: Test
  Components: Logical Optimizer
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
 Attachments: HIVE-9525.1.patch, HIVE-9525.patch


 We have disabled it previously because of issues. But testing again those 
 issues looks like have gone away. We should reenable optimization for these 
 tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-9539) Wrong check of version format in TestWebHCatE2e.getHiveVersion()

[Damien Carol (JIRA)]

Damien Carol created HIVE-9539:
--

 Summary: Wrong check of version format in 
TestWebHCatE2e.getHiveVersion()
 Key: HIVE-9539
 URL: https://issues.apache.org/jira/browse/HIVE-9539
 Project: Hive
  Issue Type: Bug
Reporter: Damien Carol
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9539) Wrong check of version format in TestWebHCatE2e.getHiveVersion()

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9539:
---
Affects Version/s: 1.2.0

 Wrong check of version format in TestWebHCatE2e.getHiveVersion()
 

 Key: HIVE-9539
 URL: https://issues.apache.org/jira/browse/HIVE-9539
 Project: Hive
  Issue Type: Bug
  Components: HCatalog
Affects Versions: 1.2.0
Reporter: Damien Carol
Priority: Minor

 Test of 
 {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} check 
 that version in that format : {{0.[0-9]+.[0-9]+.*}}
 This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9539) Wrong check of version format in TestWebHCatE2e.getHiveVersion()

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9539:
---
Component/s: HCatalog

 Wrong check of version format in TestWebHCatE2e.getHiveVersion()
 

 Key: HIVE-9539
 URL: https://issues.apache.org/jira/browse/HIVE-9539
 Project: Hive
  Issue Type: Bug
  Components: HCatalog
Affects Versions: 1.2.0
Reporter: Damien Carol
Priority: Minor

 Test of 
 {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} check 
 that version in that format : {{0.[0-9]+.[0-9]+.*}}
 This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9539) Wrong check of version format in TestWebHCatE2e.getHiveVersion()

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300206#comment-14300206
 ] 

Damien Carol commented on HIVE-9539:


It's a one line patch in this method :
{code}
@Test
public void getHiveVersion() throws Exception {
MethodCallRetVal p = doHttpCall(templetonBaseUrl + /version/hive,
HTTP_METHOD_TYPE.GET);
Assert.assertEquals(HttpStatus.OK_200, p.httpStatusCode);
MapString, Object props = JsonBuilder.jsonToMap(p.responseBody);
Assert.assertEquals(hive, props.get(module));
Assert.assertTrue(p.getAssertMsg(),
((String) props.get(version)).matches(0.[0-9]+.[0-9]+.*));
}
{code}
Line 244 should be : {code}((String) 
props.get(version)).matches([0-9]+.[0-9]+.[0-9]+.*));{code}
Instead of {code}((String) 
props.get(version)).matches(0.[0-9]+.[0-9]+.*));{code}



 Wrong check of version format in TestWebHCatE2e.getHiveVersion()
 

 Key: HIVE-9539
 URL: https://issues.apache.org/jira/browse/HIVE-9539
 Project: Hive
  Issue Type: Bug
Reporter: Damien Carol
Priority: Minor

 Test of 
 {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} check 
 that version in that format : {{0.[0-9]+.[0-9]+.*}}
 This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HIVE-9539) Wrong check of version format in TestWebHCatE2e.getHiveVersion()

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol reassigned HIVE-9539:
--

Assignee: Damien Carol

 Wrong check of version format in TestWebHCatE2e.getHiveVersion()
 

 Key: HIVE-9539
 URL: https://issues.apache.org/jira/browse/HIVE-9539
 Project: Hive
  Issue Type: Bug
  Components: HCatalog
Affects Versions: 1.2.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Minor

 Bug caused by HIVE-9485.
 Test of 
 {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} check 
 that version in that format : {{0.[0-9]+.[0-9]+.*}}
 This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9539) Wrong check of version format in TestWebHCatE2e.getHiveVersion()

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300212#comment-14300212
 ] 

Damien Carol commented on HIVE-9539:


For record :
{noformat}
GET http://localhost:52505/templeton/v1/version/hive?user.name=johndoe
{noformat}
Returns :
{noformat}
 {module:hive,version:1.2.0-SNAPSHOT}
{noformat}

 Wrong check of version format in TestWebHCatE2e.getHiveVersion()
 

 Key: HIVE-9539
 URL: https://issues.apache.org/jira/browse/HIVE-9539
 Project: Hive
  Issue Type: Bug
  Components: HCatalog
Affects Versions: 1.2.0
Reporter: Damien Carol
Priority: Minor

 Bug caused by HIVE-9485.
 Test of 
 {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} check 
 that version in that format : {{0.[0-9]+.[0-9]+.*}}
 This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9539) Wrong check of version format in TestWebHCatE2e.getHiveVersion()

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9539:
---
Status: Patch Available  (was: Open)

 Wrong check of version format in TestWebHCatE2e.getHiveVersion()
 

 Key: HIVE-9539
 URL: https://issues.apache.org/jira/browse/HIVE-9539
 Project: Hive
  Issue Type: Bug
  Components: HCatalog
Affects Versions: 1.2.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Minor
 Attachments: HIVE-9539.patch


 Bug caused by HIVE-9485.
 Test of 
 {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} check 
 that version in that format : {{0.[0-9]+.[0-9]+.*}}
 This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9539) Wrong check of version format in TestWebHCatE2e.getHiveVersion()

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9539:
---
Attachment: HIVE-9539.patch

 Wrong check of version format in TestWebHCatE2e.getHiveVersion()
 

 Key: HIVE-9539
 URL: https://issues.apache.org/jira/browse/HIVE-9539
 Project: Hive
  Issue Type: Bug
  Components: HCatalog
Affects Versions: 1.2.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Minor
 Attachments: HIVE-9539.patch


 Bug caused by HIVE-9485.
 Test of 
 {{org.apache.hive.hcatalog.templeton.TestWebHCatE2e.getHiveVersion()}} check 
 that version in that format : {{0.[0-9]+.[0-9]+.*}}
 This doesn't works since HIVE version is like {{1.2.0-SNAPHSOT}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9537) string expressions on a fixed length character do not preserve trailing spaces

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9537:
---
Description: 
When a string expression such as upper or lower is applied to a fixed length 
column the trailing spaces of the fixed length character are not preserved.
{code:sql}
CREATE TABLE  if not exists TCHAR ( 
RNUM int, 
CCHAR char(32)
)
ROW FORMAT DELIMITED 
FIELDS TERMINATED BY '|' 
LINES TERMINATED BY '\n' 
STORED AS TEXTFILE;
{code}
{{cchar}} as a {{char(32)}}.
{code:sql}
select cchar, concat(cchar, cchar), concat(lower(cchar), cchar), 
concat(upper(cchar), cchar) 
from tchar;
{code}

0|\N
1|
2| 
3|BB
4|EE
5|FF


  was:
cchar is char(32). when a string expression such as upper or lower is applied 
to a fixed length column the trailing spaces of the fixed length character are 
not preserved.

select cchar, concat(cchar, cchar), concat(lower(cchar), cchar), 
concat(upper(cchar), cchar) from tchar

create table  if not exists TCHAR ( RNUM int , CCHAR char(32 ))
 ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' 
 STORED AS TEXTFILE  ;

0|\N
1|
2| 
3|BB
4|EE
5|FF



 string expressions on a fixed length character do not preserve trailing spaces
 --

 Key: HIVE-9537
 URL: https://issues.apache.org/jira/browse/HIVE-9537
 Project: Hive
  Issue Type: Bug
  Components: SQL
Reporter: N Campbell

 When a string expression such as upper or lower is applied to a fixed length 
 column the trailing spaces of the fixed length character are not preserved.
 {code:sql}
 CREATE TABLE  if not exists TCHAR ( 
 RNUM int, 
 CCHAR char(32)
 )
 ROW FORMAT DELIMITED 
 FIELDS TERMINATED BY '|' 
 LINES TERMINATED BY '\n' 
 STORED AS TEXTFILE;
 {code}
 {{cchar}} as a {{char(32)}}.
 {code:sql}
 select cchar, concat(cchar, cchar), concat(lower(cchar), cchar), 
 concat(upper(cchar), cchar) 
 from tchar;
 {code}
 0|\N
 1|
 2| 
 3|BB
 4|EE
 5|FF



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9510) Throwing null point exception , when get join distinct row count from RelMdUtil.java class

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9510:
---
Description: 
Setting log level in logging.properties file as following:
{noformat}
handlers=java.util.logging.ConsoleHandler.level=INFO
org.apache.calcite.plan.RelOptPlanner.level=ALL
java.util.logging.ConsoleHandler.level=ALL
{noformat}

Running Q3 in TPCH-full  after modifying , in order to  test join reorder,
but running  failed.
QL:
{code:sql}
set  hive.cbo.enable=true;
--ANALYZE TABLE customer COMPUTE STATISTICS for columns;
--ANALYZE TABLE orders COMPUTE STATISTICS for columns;
--ANALYZE TABLE lineitem COMPUTE STATISTICS for columns;

--Q3
-- the query
select 
  l_orderkey, sum(l_extendedprice*(1-l_discount)) as revenue, o_orderdate, 
o_shippriority 
from 
  lineitem l join orders o 
on l.l_orderkey = o.o_orderkey
  join customer c
on c.c_mktsegment = 'BUILDING' and c.c_custkey = o.o_custkey 
where 
  o_orderdate  '1995-03-15' and l_shipdate  '1995-03-15' 
group by l_orderkey, o_orderdate, o_shippriority 
order by revenue desc, o_orderdate 
limit 10;
{code}

LOG:






  was:
Setting log level in logging.properties file as following:
{noformat}
handlers=java.util.logging.ConsoleHandler.level=INFO
org.apache.calcite.plan.RelOptPlanner.level=ALL
java.util.logging.ConsoleHandler.level=ALL
{noformat}

Running Q3 in TPCH-full  after modifying , in order to  test join reorder,
but running  failed.
QL:
set  hive.cbo.enable=true;
--ANALYZE TABLE customer COMPUTE STATISTICS for columns;
--ANALYZE TABLE orders COMPUTE STATISTICS for columns;
--ANALYZE TABLE lineitem COMPUTE STATISTICS for columns;

--Q3
-- the query
{code:sql}
select 
  l_orderkey, sum(l_extendedprice*(1-l_discount)) as revenue, o_orderdate, 
o_shippriority 
from 
  lineitem l join orders o 
on l.l_orderkey = o.o_orderkey
  join customer c
on c.c_mktsegment = 'BUILDING' and c.c_custkey = o.o_custkey 
where 
  o_orderdate  '1995-03-15' and l_shipdate  '1995-03-15' 
group by l_orderkey, o_orderdate, o_shippriority 
order by revenue desc, o_orderdate 
limit 10;
{code}

LOG:
Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner fireRule
FINE: call#15: Apply rule [FilterProjectTransposeRule] to 
[rel#107:HiveFilter.HIVE.[](input=HepRelVertex#106,condition=($2, 
'1995-03-15')), 
rel#105:HiveProject.HIVE.[](input=HepRelVertex#104,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7)]
Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode init
FINEST: new HiveFilter#138
Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode init
FINEST: new HiveProject#139
Jan 29, 2015 11:48:04 AM org.apache.calcite.plan.AbstractRelOptPlanner 
notifyTransformation
FINE: call#15: Rule FilterProjectTransposeRule arguments 
[rel#107:HiveFilter.HIVE.[](input=HepRelVertex#106,condition=($2, 
'1995-03-15')), 
rel#105:HiveProject.HIVE.[](input=HepRelVertex#104,o_orderkey=$0,o_custkey=$1,o_orderdate=$4,o_shippriority=$7)]
 produced HiveProject#139
Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode init
FINEST: new HepRelVertex#140
Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode init
FINEST: new HiveProject#141
Jan 29, 2015 11:48:04 AM org.apache.calcite.rel.AbstractRelNode init
FINEST: new HepRelVertex#142
15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - 
Foreign Key relation:
15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: 
HiveJoin(condition=[=($0, $4)], joinType=[inner])
  HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], 
l_shipdate=[$10])
HiveFilter(condition=[($10, '1995-03-15')])
  HiveTableScan(table=[[default.lineitem]])
  HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], 
o_shippriority=[$7])
HiveFilter(condition=[($4, '1995-03-15')])
  HiveTableScan(table=[[default.orders]])

15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Primary - Foreign Key 
join:
fkSide = 1
FKInfo:FKInfo(rowCount=1.00,ndv=-1.00)
PKInfo:PKInfo(rowCount=1.00,ndv=-1.00,selectivity=1.00)
isPKSideSimple:false
NDV Scaling Factor:1.00

15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: Identified Primary - 
Foreign Key relation:
15/01/29 11:48:04 [main]: DEBUG stats.HiveRelMdRowCount: 
HiveJoin(condition=[=($8, $5)], joinType=[inner])
  HiveJoin(condition=[=($0, $4)], joinType=[inner])
HiveProject(l_orderkey=[$0], l_extendedprice=[$5], l_discount=[$6], 
l_shipdate=[$10])
  HiveFilter(condition=[($10, '1995-03-15')])
HiveTableScan(table=[[default.lineitem]])
HiveProject(o_orderkey=[$0], o_custkey=[$1], o_orderdate=[$4], 
o_shippriority=[$7])
  HiveFilter(condition=[($4, '1995-03-15')])
HiveTableScan(table=[[default.orders]])
  HiveProject(c_custkey=[$0], c_mktsegment=[$6])
HiveFilter(condition=[=($6, 'BUILDING')])
  

[jira] [Updated] (HIVE-9510) Throwing null point exception , when get join distinct row count from RelMdUtil.java class

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9510:
---
Description: 
Setting log level in logging.properties file as following:
{noformat}
handlers=java.util.logging.ConsoleHandler.level=INFO
org.apache.calcite.plan.RelOptPlanner.level=ALL
java.util.logging.ConsoleHandler.level=ALL
{noformat}

Running Q3 in TPCH-full  after modifying , in order to  test join reorder,
but running  failed.
QL:
{code:sql}
set  hive.cbo.enable=true;
--ANALYZE TABLE customer COMPUTE STATISTICS for columns;
--ANALYZE TABLE orders COMPUTE STATISTICS for columns;
--ANALYZE TABLE lineitem COMPUTE STATISTICS for columns;

--Q3
-- the query
select 
  l_orderkey, sum(l_extendedprice*(1-l_discount)) as revenue, o_orderdate, 
o_shippriority 
from 
  lineitem l join orders o 
on l.l_orderkey = o.o_orderkey
  join customer c
on c.c_mktsegment = 'BUILDING' and c.c_custkey = o.o_custkey 
where 
  o_orderdate  '1995-03-15' and l_shipdate  '1995-03-15' 
group by l_orderkey, o_orderdate, o_shippriority 
order by revenue desc, o_orderdate 
limit 10;
{code}

LOG:
see log.txt





  was:
Setting log level in logging.properties file as following:
{noformat}
handlers=java.util.logging.ConsoleHandler.level=INFO
org.apache.calcite.plan.RelOptPlanner.level=ALL
java.util.logging.ConsoleHandler.level=ALL
{noformat}

Running Q3 in TPCH-full  after modifying , in order to  test join reorder,
but running  failed.
QL:
{code:sql}
set  hive.cbo.enable=true;
--ANALYZE TABLE customer COMPUTE STATISTICS for columns;
--ANALYZE TABLE orders COMPUTE STATISTICS for columns;
--ANALYZE TABLE lineitem COMPUTE STATISTICS for columns;

--Q3
-- the query
select 
  l_orderkey, sum(l_extendedprice*(1-l_discount)) as revenue, o_orderdate, 
o_shippriority 
from 
  lineitem l join orders o 
on l.l_orderkey = o.o_orderkey
  join customer c
on c.c_mktsegment = 'BUILDING' and c.c_custkey = o.o_custkey 
where 
  o_orderdate  '1995-03-15' and l_shipdate  '1995-03-15' 
group by l_orderkey, o_orderdate, o_shippriority 
order by revenue desc, o_orderdate 
limit 10;
{code}

LOG:







 Throwing null point exception , when get join distinct row count from 
 RelMdUtil.java class
 --

 Key: HIVE-9510
 URL: https://issues.apache.org/jira/browse/HIVE-9510
 Project: Hive
  Issue Type: Bug
Reporter: asko
Assignee: Julian Hyde
 Attachments: log.txt, log3_cbo5


 Setting log level in logging.properties file as following:
 {noformat}
 handlers=java.util.logging.ConsoleHandler.level=INFO
 org.apache.calcite.plan.RelOptPlanner.level=ALL
 java.util.logging.ConsoleHandler.level=ALL
 {noformat}
 Running Q3 in TPCH-full  after modifying , in order to  test join reorder,
 but running  failed.
 QL:
 {code:sql}
 set  hive.cbo.enable=true;
 --ANALYZE TABLE customer COMPUTE STATISTICS for columns;
 --ANALYZE TABLE orders COMPUTE STATISTICS for columns;
 --ANALYZE TABLE lineitem COMPUTE STATISTICS for columns;
 --Q3
 -- the query
 select 
   l_orderkey, sum(l_extendedprice*(1-l_discount)) as revenue, o_orderdate, 
 o_shippriority 
 from 
   lineitem l join orders o 
 on l.l_orderkey = o.o_orderkey
   join customer c
 on c.c_mktsegment = 'BUILDING' and c.c_custkey = o.o_custkey 
 where 
   o_orderdate  '1995-03-15' and l_shipdate  '1995-03-15' 
 group by l_orderkey, o_orderdate, o_shippriority 
 order by revenue desc, o_orderdate 
 limit 10;
 {code}
 LOG:
 see log.txt



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9510) Throwing null point exception , when get join distinct row count from RelMdUtil.java class

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9510:
---
Attachment: log.txt

 Throwing null point exception , when get join distinct row count from 
 RelMdUtil.java class
 --

 Key: HIVE-9510
 URL: https://issues.apache.org/jira/browse/HIVE-9510
 Project: Hive
  Issue Type: Bug
Reporter: asko
Assignee: Julian Hyde
 Attachments: log.txt, log3_cbo5


 Setting log level in logging.properties file as following:
 {noformat}
 handlers=java.util.logging.ConsoleHandler.level=INFO
 org.apache.calcite.plan.RelOptPlanner.level=ALL
 java.util.logging.ConsoleHandler.level=ALL
 {noformat}
 Running Q3 in TPCH-full  after modifying , in order to  test join reorder,
 but running  failed.
 QL:
 {code:sql}
 set  hive.cbo.enable=true;
 --ANALYZE TABLE customer COMPUTE STATISTICS for columns;
 --ANALYZE TABLE orders COMPUTE STATISTICS for columns;
 --ANALYZE TABLE lineitem COMPUTE STATISTICS for columns;
 --Q3
 -- the query
 select 
   l_orderkey, sum(l_extendedprice*(1-l_discount)) as revenue, o_orderdate, 
 o_shippriority 
 from 
   lineitem l join orders o 
 on l.l_orderkey = o.o_orderkey
   join customer c
 on c.c_mktsegment = 'BUILDING' and c.c_custkey = o.o_custkey 
 where 
   o_orderdate  '1995-03-15' and l_shipdate  '1995-03-15' 
 group by l_orderkey, o_orderdate, o_shippriority 
 order by revenue desc, o_orderdate 
 limit 10;
 {code}
 LOG:



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-9511) Switch Tez to 0.6.0

[Damien Carol (JIRA)]

Damien Carol created HIVE-9511:
--

 Summary: Switch Tez to 0.6.0
 Key: HIVE-9511
 URL: https://issues.apache.org/jira/browse/HIVE-9511
 Project: Hive
  Issue Type: Improvement
Reporter: Damien Carol


Tez 0.6.0 has been released.
Research to switch to version 0.6.0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9510) Throwing null point exception , when get join distinct row count from RelMdUtil.java class

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9510:
---
Component/s: CBO

 Throwing null point exception , when get join distinct row count from 
 RelMdUtil.java class
 --

 Key: HIVE-9510
 URL: https://issues.apache.org/jira/browse/HIVE-9510
 Project: Hive
  Issue Type: Bug
  Components: CBO
Reporter: asko
Assignee: Julian Hyde
 Attachments: log.txt, log3_cbo5


 Setting log level in logging.properties file as following:
 {noformat}
 handlers=java.util.logging.ConsoleHandler.level=INFO
 org.apache.calcite.plan.RelOptPlanner.level=ALL
 java.util.logging.ConsoleHandler.level=ALL
 {noformat}
 Running Q3 in TPCH-full  after modifying , in order to  test join reorder,
 but running  failed.
 QL:
 {code:sql}
 set  hive.cbo.enable=true;
 --ANALYZE TABLE customer COMPUTE STATISTICS for columns;
 --ANALYZE TABLE orders COMPUTE STATISTICS for columns;
 --ANALYZE TABLE lineitem COMPUTE STATISTICS for columns;
 --Q3
 -- the query
 select 
   l_orderkey, sum(l_extendedprice*(1-l_discount)) as revenue, o_orderdate, 
 o_shippriority 
 from 
   lineitem l join orders o 
 on l.l_orderkey = o.o_orderkey
   join customer c
 on c.c_mktsegment = 'BUILDING' and c.c_custkey = o.o_custkey 
 where 
   o_orderdate  '1995-03-15' and l_shipdate  '1995-03-15' 
 group by l_orderkey, o_orderdate, o_shippriority 
 order by revenue desc, o_orderdate 
 limit 10;
 {code}
 LOG:
 see log.txt



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9511) Switch Tez to 0.6.0

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9511:
---
Attachment: HIVE-9511.patch.txt

 Switch Tez to 0.6.0
 ---

 Key: HIVE-9511
 URL: https://issues.apache.org/jira/browse/HIVE-9511
 Project: Hive
  Issue Type: Improvement
Reporter: Damien Carol
 Attachments: HIVE-9511.patch.txt


 Tez 0.6.0 has been released.
 Research to switch to version 0.6.0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9397) SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14295667#comment-14295667
 ] 

Damien Carol commented on HIVE-9397:


These tests pass localy :
* 
{{org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_metadata_only_queries}}
* 
{{org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_histogram_numeric}}

 SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.14.0, 0.15.0
Reporter: Damien Carol
Assignee: Navis
 Attachments: HIVE-9397.1.patch.txt


 These queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HIVE-9476) Beeline fails to start on trunk

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol resolved HIVE-9476.

Resolution: Duplicate

 Beeline fails to start on trunk
 ---

 Key: HIVE-9476
 URL: https://issues.apache.org/jira/browse/HIVE-9476
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.15.0
Reporter: Vaibhav Gumashta
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.15.0


 {code}
 vgumashta:hive vgumashta$ beeline --verbose=true
 [ERROR] Terminal initialization failed; falling back to unsupported
 java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but 
 interface was expected
   at jline.TerminalFactory.create(TerminalFactory.java:101)
   at jline.TerminalFactory.get(TerminalFactory.java:158)
   at org.apache.hive.beeline.BeeLineOpts.init(BeeLineOpts.java:73)
   at org.apache.hive.beeline.BeeLine.init(BeeLine.java:117)
   at 
 org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
   at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 Exception in thread main java.lang.IncompatibleClassChangeError: Found 
 class jline.Terminal, but interface was expected
   at org.apache.hive.beeline.BeeLineOpts.init(BeeLineOpts.java:101)
   at org.apache.hive.beeline.BeeLine.init(BeeLine.java:117)
   at 
 org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
   at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 {code}
 Working fine on 14.1. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9397) SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9397:
---
Affects Version/s: 0.14.0

 SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.14.0, 0.15.0
Reporter: Damien Carol
Assignee: Navis
 Attachments: HIVE-9397.1.patch.txt


 These queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9397) SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293366#comment-14293366
 ] 

Damien Carol commented on HIVE-9397:


Thanks [~navis] for working on it.

I have few comments :
* trailing spaces in enum [[IntegerSubType]]
* trailing spaces x9 in 
[[ql/src/java/org/apache/hadoop/hive/ql/optimizer/StatsOptimizer.java]]

You can create a review board if you want, I will report in it.
Again thanks a lots for working on it.

 SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.14.0, 0.15.0
Reporter: Damien Carol
Assignee: Navis
 Attachments: HIVE-9397.1.patch.txt


 These queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9397) SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293383#comment-14293383
 ] 

Damien Carol commented on HIVE-9397:


Tested with bigint/int/tinyint. seems to work well.

 SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.14.0, 0.15.0
Reporter: Damien Carol
Assignee: Navis
 Attachments: HIVE-9397.1.patch.txt


 These queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9476) Beeline fails to start on trunk

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293485#comment-14293485
 ] 

Damien Carol commented on HIVE-9476:


It's due to incompatible version of JLine between Hadoop and Hive.
Use this quick workaround :
{{noformat}}
export HADOOP_USER_CLASSPATH_FIRST=true
{{noformat}}


 Beeline fails to start on trunk
 ---

 Key: HIVE-9476
 URL: https://issues.apache.org/jira/browse/HIVE-9476
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.15.0
Reporter: Vaibhav Gumashta
Priority: Blocker
 Fix For: 0.15.0


 {code}
 vgumashta:hive vgumashta$ beeline --verbose=true
 [ERROR] Terminal initialization failed; falling back to unsupported
 java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but 
 interface was expected
   at jline.TerminalFactory.create(TerminalFactory.java:101)
   at jline.TerminalFactory.get(TerminalFactory.java:158)
   at org.apache.hive.beeline.BeeLineOpts.init(BeeLineOpts.java:73)
   at org.apache.hive.beeline.BeeLine.init(BeeLine.java:117)
   at 
 org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
   at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 Exception in thread main java.lang.IncompatibleClassChangeError: Found 
 class jline.Terminal, but interface was expected
   at org.apache.hive.beeline.BeeLineOpts.init(BeeLineOpts.java:101)
   at org.apache.hive.beeline.BeeLine.init(BeeLine.java:117)
   at 
 org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
   at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 {code}
 Working fine on 14.1. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9476) Beeline fails to start on trunk

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9476:
---
Component/s: (was: JDBC)
 Beeline

 Beeline fails to start on trunk
 ---

 Key: HIVE-9476
 URL: https://issues.apache.org/jira/browse/HIVE-9476
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.15.0
Reporter: Vaibhav Gumashta
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.15.0


 {code}
 vgumashta:hive vgumashta$ beeline --verbose=true
 [ERROR] Terminal initialization failed; falling back to unsupported
 java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but 
 interface was expected
   at jline.TerminalFactory.create(TerminalFactory.java:101)
   at jline.TerminalFactory.get(TerminalFactory.java:158)
   at org.apache.hive.beeline.BeeLineOpts.init(BeeLineOpts.java:73)
   at org.apache.hive.beeline.BeeLine.init(BeeLine.java:117)
   at 
 org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
   at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 Exception in thread main java.lang.IncompatibleClassChangeError: Found 
 class jline.Terminal, but interface was expected
   at org.apache.hive.beeline.BeeLineOpts.init(BeeLineOpts.java:101)
   at org.apache.hive.beeline.BeeLine.init(BeeLine.java:117)
   at 
 org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
   at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 {code}
 Working fine on 14.1. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9476) Beeline fails to start on trunk

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293489#comment-14293489
 ] 

Damien Carol commented on HIVE-9476:


I think we should close this one as a workaround already exists and it will be 
fixed by YARN-2815.
Also HIVE-9405 will fix this issue.

 Beeline fails to start on trunk
 ---

 Key: HIVE-9476
 URL: https://issues.apache.org/jira/browse/HIVE-9476
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.15.0
Reporter: Vaibhav Gumashta
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.15.0


 {code}
 vgumashta:hive vgumashta$ beeline --verbose=true
 [ERROR] Terminal initialization failed; falling back to unsupported
 java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but 
 interface was expected
   at jline.TerminalFactory.create(TerminalFactory.java:101)
   at jline.TerminalFactory.get(TerminalFactory.java:158)
   at org.apache.hive.beeline.BeeLineOpts.init(BeeLineOpts.java:73)
   at org.apache.hive.beeline.BeeLine.init(BeeLine.java:117)
   at 
 org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
   at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 Exception in thread main java.lang.IncompatibleClassChangeError: Found 
 class jline.Terminal, but interface was expected
   at org.apache.hive.beeline.BeeLineOpts.init(BeeLineOpts.java:101)
   at org.apache.hive.beeline.BeeLine.init(BeeLine.java:117)
   at 
 org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
   at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 {code}
 Working fine on 14.1. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HIVE-9476) Beeline fails to start on trunk

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol reassigned HIVE-9476:
--

Assignee: Damien Carol

 Beeline fails to start on trunk
 ---

 Key: HIVE-9476
 URL: https://issues.apache.org/jira/browse/HIVE-9476
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.15.0
Reporter: Vaibhav Gumashta
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.15.0


 {code}
 vgumashta:hive vgumashta$ beeline --verbose=true
 [ERROR] Terminal initialization failed; falling back to unsupported
 java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but 
 interface was expected
   at jline.TerminalFactory.create(TerminalFactory.java:101)
   at jline.TerminalFactory.get(TerminalFactory.java:158)
   at org.apache.hive.beeline.BeeLineOpts.init(BeeLineOpts.java:73)
   at org.apache.hive.beeline.BeeLine.init(BeeLine.java:117)
   at 
 org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
   at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 Exception in thread main java.lang.IncompatibleClassChangeError: Found 
 class jline.Terminal, but interface was expected
   at org.apache.hive.beeline.BeeLineOpts.init(BeeLineOpts.java:101)
   at org.apache.hive.beeline.BeeLine.init(BeeLine.java:117)
   at 
 org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:469)
   at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 {code}
 Working fine on 14.1. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9397) SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9397:
---
Summary: SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS  
(was: SELECT max(bar) FROM foo is broken with metadata enabled)

 SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.15.0
Reporter: Damien Carol
Assignee: Gopal V

 These queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9397) SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291613#comment-14291613
 ] 

Damien Carol commented on HIVE-9397:


Tested with CLI, it seems to works well.
{noformat}
hive select max(idmagasin) from magasin;
OK
_c0
1871
Time taken: 0.624 seconds, Fetched: 1 row(s)
{noformat}
CLI use only metadata :
{noformat}
hive explain
 select max(idmagasin) from magasin;
OK
Explain
STAGE DEPENDENCIES:
  Stage-0 is a root stage

STAGE PLANS:
  Stage: Stage-0
Fetch Operator
  limit: 1
  Processor Tree:
ListSink

Time taken: 0.233 seconds, Fetched: 10 row(s)
{noformat}

 SELECT max(bar) FROM foo is broken after ANALYZE ... FOR COLUMNS
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.15.0
Reporter: Damien Carol
Assignee: Gopal V

 These queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9385) Sum a Double using an ORC table

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14279973#comment-14279973
 ] 

Damien Carol commented on HIVE-9385:


[~nimartin] Could you provide SHOW CREATE TABLE sales and the query you use 
to store sales amount.
Another thing, if you must have precise value for amount, you should use DECIMAL

 Sum a Double using an ORC table
 ---

 Key: HIVE-9385
 URL: https://issues.apache.org/jira/browse/HIVE-9385
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.13.1
 Environment: HDP 2.x, Hive
Reporter: Nick Martin
Priority: Minor

 I’m storing a sales amount column as a double in an ORC table and when I do:
 {code:sql}
 select sum(x) from sometable
 {code}
 I get a value like {{4.79165141174808E9}}
 A visual inspection of the column values reveals no glaring anomalies…all 
 looks pretty normal. 
 If I do the same thing in a textfile table I get a perfectly fine aggregation 
 of the double field.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-9397) SELECT max(bar) FROM foo is broken with metadata enabled

[Damien Carol (JIRA)]

Damien Carol created HIVE-9397:
--

 Summary: SELECT max(bar) FROM foo is broken with metadata enabled
 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.15.0
Reporter: Damien Carol


This queries produce an error :
{code:sql}

DROP TABLE IF EXISTS foo;

CREATE TABLE foo (id int) STORED AS ORC;

INSERT INTO TABLE foo VALUES (1);
INSERT INTO TABLE foo VALUES (2);
INSERT INTO TABLE foo VALUES (3);
INSERT INTO TABLE foo VALUES (4);
INSERT INTO TABLE foo VALUES (5);

SELECT max(id) FROM foo;

ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;

SELECT max(id) FROM foo;
{code}





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9397) SELECT max(bar) FROM foo is broken with metadata enabled

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9397:
---
Description: 
This queries produce an error :
{code:sql}
DROP TABLE IF EXISTS foo;

CREATE TABLE foo (id int) STORED AS ORC;

INSERT INTO TABLE foo VALUES (1);
INSERT INTO TABLE foo VALUES (2);
INSERT INTO TABLE foo VALUES (3);
INSERT INTO TABLE foo VALUES (4);
INSERT INTO TABLE foo VALUES (5);

SELECT max(id) FROM foo;

ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;

SELECT max(id) FROM foo;
{code}

{noformat}
0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
+-+--+
| _c0 |
+-+--+
org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
0: jdbc:hive2://nc-h04:1/casino
{noformat}

  was:
This queries produce an error :
{code:sql}

DROP TABLE IF EXISTS foo;

CREATE TABLE foo (id int) STORED AS ORC;

INSERT INTO TABLE foo VALUES (1);
INSERT INTO TABLE foo VALUES (2);
INSERT INTO TABLE foo VALUES (3);
INSERT INTO TABLE foo VALUES (4);
INSERT INTO TABLE foo VALUES (5);

SELECT max(id) FROM foo;

ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;

SELECT max(id) FROM foo;
{code}




 SELECT max(bar) FROM foo is broken with metadata enabled
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.15.0
Reporter: Damien Carol

 This queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9397) SELECT max(bar) FROM foo is broken with metadata enabled

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9397:
---
Description: 
This queries produce an error :
{code:sql}
DROP TABLE IF EXISTS foo;

CREATE TABLE foo (id int) STORED AS ORC;

INSERT INTO TABLE foo VALUES (1);
INSERT INTO TABLE foo VALUES (2);
INSERT INTO TABLE foo VALUES (3);
INSERT INTO TABLE foo VALUES (4);
INSERT INTO TABLE foo VALUES (5);

SELECT max(id) FROM foo;

ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;

SELECT max(id) FROM foo;
{code}

The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
{noformat}
0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
+-+--+
| _c0 |
+-+--+
org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
0: jdbc:hive2://nc-h04:1/casino
{noformat}

  was:
This queries produce an error :
{code:sql}
DROP TABLE IF EXISTS foo;

CREATE TABLE foo (id int) STORED AS ORC;

INSERT INTO TABLE foo VALUES (1);
INSERT INTO TABLE foo VALUES (2);
INSERT INTO TABLE foo VALUES (3);
INSERT INTO TABLE foo VALUES (4);
INSERT INTO TABLE foo VALUES (5);

SELECT max(id) FROM foo;

ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;

SELECT max(id) FROM foo;
{code}

{noformat}
0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
+-+--+
| _c0 |
+-+--+
org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
0: jdbc:hive2://nc-h04:1/casino
{noformat}


 SELECT max(bar) FROM foo is broken with metadata enabled
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.15.0
Reporter: Damien Carol

 This queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9397) SELECT max(bar) FROM foo is broken with metadata enabled

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280324#comment-14280324
 ] 

Damien Carol commented on HIVE-9397:


Complete stack :
{noformat}
java.lang.RuntimeException: org.apache.hive.service.cli.HiveSQLException: 
java.lang.ClassCastException
at 
org.apache.hive.beeline.IncrementalRows.hasNext(IncrementalRows.java:84)
at 
org.apache.hive.beeline.TableOutputFormat.print(TableOutputFormat.java:49)
at org.apache.hive.beeline.BeeLine.print(BeeLine.java:1722)
at org.apache.hive.beeline.Commands.execute(Commands.java:793)
at org.apache.hive.beeline.Commands.sql(Commands.java:665)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:938)
at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:778)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:740)
at 
org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:470)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:453)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}

 SELECT max(bar) FROM foo is broken with metadata enabled
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.15.0
Reporter: Damien Carol

 This queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9397) SELECT max(bar) FROM foo is broken with metadata enabled

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9397:
---
Component/s: Beeline

 SELECT max(bar) FROM foo is broken with metadata enabled
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.15.0
Reporter: Damien Carol

 This queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9397) SELECT max(bar) FROM foo is broken with metadata enabled

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9397:
---
Description: 
These queries produce an error :

{code:sql}
DROP TABLE IF EXISTS foo;

CREATE TABLE foo (id int) STORED AS ORC;

INSERT INTO TABLE foo VALUES (1);
INSERT INTO TABLE foo VALUES (2);
INSERT INTO TABLE foo VALUES (3);
INSERT INTO TABLE foo VALUES (4);
INSERT INTO TABLE foo VALUES (5);

SELECT max(id) FROM foo;

ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;

SELECT max(id) FROM foo;
{code}

The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
{noformat}
0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
+-+--+
| _c0 |
+-+--+
org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
0: jdbc:hive2://nc-h04:1/casino
{noformat}

  was:
This queries produce an error :
{code:sql}
DROP TABLE IF EXISTS foo;

CREATE TABLE foo (id int) STORED AS ORC;

INSERT INTO TABLE foo VALUES (1);
INSERT INTO TABLE foo VALUES (2);
INSERT INTO TABLE foo VALUES (3);
INSERT INTO TABLE foo VALUES (4);
INSERT INTO TABLE foo VALUES (5);

SELECT max(id) FROM foo;

ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;

SELECT max(id) FROM foo;
{code}

The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
{noformat}
0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
+-+--+
| _c0 |
+-+--+
org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
0: jdbc:hive2://nc-h04:1/casino
{noformat}


 SELECT max(bar) FROM foo is broken with metadata enabled
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.15.0
Reporter: Damien Carol

 These queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9397) SELECT max(bar) FROM foo is broken with metadata enabled

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14280329#comment-14280329
 ] 

Damien Carol commented on HIVE-9397:


Everything is fine with this one :
{code:sql}
SELECT max(cast(id as bigint)) FROM foo;
{code}
{noformat}
+-+--+
|   _c0   |
+-+--+
| 5   |
+-+--+
1 row selected (15.66 seconds)
{noformat}

 SELECT max(bar) FROM foo is broken with metadata enabled
 

 Key: HIVE-9397
 URL: https://issues.apache.org/jira/browse/HIVE-9397
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 0.15.0
Reporter: Damien Carol

 These queries produce an error :
 {code:sql}
 DROP TABLE IF EXISTS foo;
 CREATE TABLE foo (id int) STORED AS ORC;
 INSERT INTO TABLE foo VALUES (1);
 INSERT INTO TABLE foo VALUES (2);
 INSERT INTO TABLE foo VALUES (3);
 INSERT INTO TABLE foo VALUES (4);
 INSERT INTO TABLE foo VALUES (5);
 SELECT max(id) FROM foo;
 ANALYZE TABLE foo COMPUTE STATISTICS FOR COLUMNS id;
 SELECT max(id) FROM foo;
 {code}
 The last query throws {{org.apache.hive.service.cli.HiveSQLException}}
 {noformat}
 0: jdbc:hive2://nc-h04:1/casino SELECT max(id) FROM foo;
 +-+--+
 | _c0 |
 +-+--+
 org.apache.hive.service.cli.HiveSQLException: java.lang.ClassCastException
 0: jdbc:hive2://nc-h04:1/casino
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9385) Sum a Double using an ORC table

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9385:
---
Description: 
I’m storing a sales amount column as a double in an ORC table and when I do:
{code:sql}
select sum(x) from sometable
{code}
I get a value like {{4.79165141174808E9}}

A visual inspection of the column values reveals no glaring anomalies…all looks 
pretty normal. 

If I do the same thing in a textfile table I get a perfectly fine aggregation 
of the double field.


  was:
I’m storing a sales amount column as a double in an ORC table and when I do:

select sum(x) from sometable

I get a value like 4.79165141174808E9

A visual inspection of the column values reveals no glaring anomalies…all looks 
pretty normal. 

If I do the same thing in a textfile table I get a perfectly fine aggregation 
of the double field.



 Sum a Double using an ORC table
 ---

 Key: HIVE-9385
 URL: https://issues.apache.org/jira/browse/HIVE-9385
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.13.1
 Environment: HDP 2.x, Hive
Reporter: Nick Martin
Priority: Minor

 I’m storing a sales amount column as a double in an ORC table and when I do:
 {code:sql}
 select sum(x) from sometable
 {code}
 I get a value like {{4.79165141174808E9}}
 A visual inspection of the column values reveals no glaring anomalies…all 
 looks pretty normal. 
 If I do the same thing in a textfile table I get a perfectly fine aggregation 
 of the double field.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-9385) Sum a Double using an ORC table

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-9385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14278823#comment-14278823
 ] 

Damien Carol commented on HIVE-9385:


[~nimartin] I don't understand your problem.
Do you complain about double format inaccuracy? Or did you expected another 
result?

 Sum a Double using an ORC table
 ---

 Key: HIVE-9385
 URL: https://issues.apache.org/jira/browse/HIVE-9385
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.13.1
 Environment: HDP 2.x, Hive
Reporter: Nick Martin
Priority: Minor

 I’m storing a sales amount column as a double in an ORC table and when I do:
 {code:sql}
 select sum(x) from sometable
 {code}
 I get a value like {{4.79165141174808E9}}
 A visual inspection of the column values reveals no glaring anomalies…all 
 looks pretty normal. 
 If I do the same thing in a textfile table I get a perfectly fine aggregation 
 of the double field.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HIVE-9384) Sum a Double using an ORC table

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol resolved HIVE-9384.

Resolution: Duplicate

 Sum a Double using an ORC table
 ---

 Key: HIVE-9384
 URL: https://issues.apache.org/jira/browse/HIVE-9384
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.13.1
 Environment: HDP 2.x, Hive
Reporter: Nick Martin
Priority: Minor

 I’m storing a sales amount column as a double in an ORC table and when I do:
 select sum(x) from sometable
 I get a value like 4.79165141174808E9
 A visual inspection of the column values reveals no glaring anomalies…all 
 looks pretty normal. 
 If I do the same thing in a textfile table I get a perfectly fine aggregation 
 of the double field.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9038) Join tests fail on Tez

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9038:
---
Description: 
Tez doesn't run all tests. But, if you run them, following tests fail with runt 
time exception pointing to bugs.
* {{auto_join21.q}}
* {{auto_join29.q}}
* {{auto_join30.q}}
* {{auto_join_filters.q}}
* {{auto_join_nulls.q}} 

  was:Tez doesn't run all tests. But, if you run them, following tests fail 
with runt time exception pointing to bugs. 
{{auto_join21.q,auto_join29.q,auto_join30.q
,auto_join_filters.q,auto_join_nulls.q}} 


 Join tests fail on Tez
 --

 Key: HIVE-9038
 URL: https://issues.apache.org/jira/browse/HIVE-9038
 Project: Hive
  Issue Type: Bug
  Components: Tests, Tez
Reporter: Ashutosh Chauhan
Assignee: Vikram Dixit K
 Attachments: HIVE-9038.1.patch, HIVE-9038.2.patch, HIVE-9038.3.patch


 Tez doesn't run all tests. But, if you run them, following tests fail with 
 runt time exception pointing to bugs.
 * {{auto_join21.q}}
 * {{auto_join29.q}}
 * {{auto_join30.q}}
 * {{auto_join_filters.q}}
 * {{auto_join_nulls.q}} 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-9377) UDF in_file() in WHERE predicate causes NPE.

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-9377:
---
Description: 
Consider the following query:

{code:sql}
SELECT foo, bar from mythdb.foobar where in_file( bar, '/tmp/bar_list.txt' );
{code}

Using {{in_file()}} in a WHERE predicate causes the following NPE:
{noformat}
java.lang.NullPointerException
at 
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getWritableConstantValue(ObjectInspectorUtils.java:1041)
at 
org.apache.hadoop.hive.ql.udf.generic.GenericUDFInFile.getRequiredFiles(GenericUDFInFile.java:93)
at 
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.isDeterministicUdf(ConstantPropagateProcFactory.java:303)
at 
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:226)
at 
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.access$000(ConstantPropagateProcFactory.java:92)
at 
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory$ConstantPropagateFilterProc.process(ConstantPropagateProcFactory.java:623)
at 
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
at 
org.apache.hadoop.hive.ql.optimizer.ConstantPropagate$ConstantPropagateWalker.walk(ConstantPropagate.java:147)
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
at 
org.apache.hadoop.hive.ql.optimizer.ConstantPropagate.transform(ConstantPropagate.java:117)
at 
org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:177)
at 
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10032)
at 
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:189)
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:420)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:306)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1108)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1156)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1045)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1035)
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304)
at 
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:701)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:674)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
I have a tentative fix I need advice on.

  was:
Consider the following query:

{code:sql}
SELECT foo, bar from mythdb.foobar where in_file( bar, '/tmp/bar_list.txt' );
{code}

Using {{in_file()}} in a WHERE predicate causes the following NPE:

java.lang.NullPointerException
at 
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getWritableConstantValue(ObjectInspectorUtils.java:1041)
at 
org.apache.hadoop.hive.ql.udf.generic.GenericUDFInFile.getRequiredFiles(GenericUDFInFile.java:93)
at 
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.isDeterministicUdf(ConstantPropagateProcFactory.java:303)
at 
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:226)
at 
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.access$000(ConstantPropagateProcFactory.java:92)
at 
org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory$ConstantPropagateFilterProc.process(ConstantPropagateProcFactory.java:623)
at 
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at 
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
at 

[jira] [Commented] (HIVE-7674) Update to Spark 1.2 [Spark Branch]

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14264729#comment-14264729
 ] 

Damien Carol commented on HIVE-7674:


Spark 1.2 is out now. Does this ticket still relevant?

 Update to Spark 1.2 [Spark Branch]
 --

 Key: HIVE-7674
 URL: https://issues.apache.org/jira/browse/HIVE-7674
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Brock Noland
Priority: Blocker

 In HIVE-8160 we added a custom repo to use Spark 1.2. Once 1.2 is released we 
 need to remove this repo.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8774) CBO: enable groupBy index

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8774:
---
Component/s: CBO

 CBO: enable groupBy index
 -

 Key: HIVE-8774
 URL: https://issues.apache.org/jira/browse/HIVE-8774
 Project: Hive
  Issue Type: Improvement
  Components: CBO
Reporter: Pengcheng Xiong
Assignee: Pengcheng Xiong
 Attachments: HIVE-8774.1.patch, HIVE-8774.10.patch, 
 HIVE-8774.11.patch, HIVE-8774.12.patch, HIVE-8774.14.patch, 
 HIVE-8774.15.patch, HIVE-8774.2.patch, HIVE-8774.3.patch, HIVE-8774.4.patch, 
 HIVE-8774.5.patch, HIVE-8774.6.patch, HIVE-8774.7.patch, HIVE-8774.8.patch, 
 HIVE-8774.9.patch


 Right now, even when groupby index is build, CBO is not able to use it. In 
 this patch, we are trying to make it use groupby index that we build.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8854) Guava dependency conflict between hive driver and remote spark context[Spark Branch]

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212244#comment-14212244
 ] 

Damien Carol commented on HIVE-8854:


[~vanzin] [~chengxiang li] [~xuefu.w...@kodak.com] I think it's a duplicate of 
HIVE-7387

 Guava dependency conflict between hive driver and remote spark context[Spark 
 Branch]
 

 Key: HIVE-8854
 URL: https://issues.apache.org/jira/browse/HIVE-8854
 Project: Hive
  Issue Type: Sub-task
  Components: Spark
Reporter: Chengxiang Li
  Labels: Spark-M3
 Attachments: hive-dirver-classloader-info.output


 Hive driver would load guava 11.0.2 from hadoop/tez, while remote spark 
 context depends on guava 14.0.1, It should be JobMetrics deserialize failed 
 on Hive driver side since Absent is used in Metrics, here is the hive driver 
 log:
 {noformat}
 java.lang.IllegalAccessError: tried to access method 
 com.google.common.base.Optional.init()V from class 
 com.google.common.base.Absent
 at com.google.common.base.Absent.init(Absent.java:35)
 at com.google.common.base.Absent.clinit(Absent.java:33)
 at sun.misc.Unsafe.ensureClassInitialized(Native Method)
 at 
 sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
 at 
 sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:140)
 at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1057)
 at java.lang.reflect.Field.getFieldAccessor(Field.java:1038)
 at java.lang.reflect.Field.getLong(Field.java:591)
 at 
 java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1663)
 at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:72)
 at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:480)
 at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:468)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.io.ObjectStreamClass.init(ObjectStreamClass.java:468)
 at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:365)
 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:602)
 at 
 java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
 at 
 java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
 at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
 at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
 at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
 at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
 at 
 java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
 at 
 java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
 at 
 java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
 at 
 akka.serialization.JavaSerializer$$anonfun$1.apply(Serializer.scala:136)
 at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
 at akka.serialization.JavaSerializer.fromBinary(Serializer.scala:136)
 at 
 akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)
 at scala.util.Try$.apply(Try.scala:161)
 at 
 akka.serialization.Serialization.deserialize(Serialization.scala:98)
 at 
 akka.remote.serialization.MessageContainerSerializer.fromBinary(MessageContainerSerializer.scala:63)
 at 
 akka.serialization.Serialization$$anonfun$deserialize$1.apply(Serialization.scala:104)
 at scala.util.Try$.apply(Try.scala:161)
 at 
 akka.serialization.Serialization.deserialize(Serialization.scala:98)
 at 
 akka.remote.MessageSerializer$.deserialize(MessageSerializer.scala:23)
 at 
 akka.remote.DefaultMessageDispatcher.payload$lzycompute$1(Endpoint.scala:58)
 at akka.remote.DefaultMessageDispatcher.payload$1(Endpoint.scala:58)
 at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:76)
 at 
 akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:937)
 at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
 at akka.remote.EndpointActor.aroundReceive(Endpoint.scala:415)
 at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
 at 

[jira] [Updated] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8329:
---
Fix Version/s: (was: 0.14.0)
   0.15.0

 Enable postgres for storing stats
 -

 Key: HIVE-8329
 URL: https://issues.apache.org/jira/browse/HIVE-8329
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
 Fix For: 0.15.0

 Attachments: HIVE-8329.1.patch, HIVE-8329.1.patch, HIVE-8329.1.patch


 Simple patch to enable postgresql as JDBC publisher for statistics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14202307#comment-14202307
 ] 

Damien Carol commented on HIVE-8329:


I don't had enough time to fix it for 0.14. Delayed to 0.15. Still a WIP

 Enable postgres for storing stats
 -

 Key: HIVE-8329
 URL: https://issues.apache.org/jira/browse/HIVE-8329
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
 Fix For: 0.15.0

 Attachments: HIVE-8329.1.patch, HIVE-8329.1.patch, HIVE-8329.1.patch


 Simple patch to enable postgresql as JDBC publisher for statistics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14198531#comment-14198531
 ] 

Damien Carol commented on HIVE-8329:


Doesn't compile on the trunk :/ don't know why
{noformat}
[INFO] -
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestSSL.java:[130,57]
 getHost() has protected access in 
org.apache.hive.jdbc.miniHS2.AbstractHiveService
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestSSL.java:[130,83]
 getBinaryPort() has protected access in 
org.apache.hive.jdbc.miniHS2.AbstractHiveService
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestSSL.java:[133,57]
 getHost() has protected access in 
org.apache.hive.jdbc.miniHS2.AbstractHiveService
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestSSL.java:[133,83]
 getBinaryPort() has protected access in 
org.apache.hive.jdbc.miniHS2.AbstractHiveService
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/service/TestHS2ImpersonationWithRemoteMS.java:[54,26]
 cannot find symbol
  symbol:   class Builder
  location: class org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/service/TestHS2ImpersonationWithRemoteMS.java:[97,30]
 cannot find symbol
  symbol:   method getDfs()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java:[329,28]
 cannot find symbol
  symbol:   method getLocalFS()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java:[412,28]
 cannot find symbol
  symbol:   method getLocalFS()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:[87,57]
 cannot find symbol
  symbol:   method getBaseJdbcURL()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:[204,47]
 cannot find symbol
  symbol:   method getBaseJdbcURL()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:[218,47]
 cannot find symbol
  symbol:   method getBaseJdbcURL()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:[230,47]
 cannot find symbol
  symbol:   method getBaseJdbcURL()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:[240,47]
 cannot find symbol
  symbol:   method getBaseJdbcURL()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:[255,47]
 cannot find symbol
  symbol:   method getBaseJdbcURL()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:[282,47]
 cannot find symbol
  symbol:   method getBaseJdbcURL()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:[290,47]
 cannot find symbol
  symbol:   method getBaseJdbcURL()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:[305,76]
 cannot find symbol
  symbol:   method getBaseJdbcURL()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:[317,76]
 cannot find symbol
  symbol:   method getBaseJdbcURL()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 
/home/hduser/hive/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java:[324,76]
 cannot find symbol
  symbol:   method getBaseJdbcURL()
  location: variable miniHS2 of type org.apache.hive.jdbc.miniHS2.MiniHS2
[ERROR] 

[jira] [Commented] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14188792#comment-14188792
 ] 

Damien Carol commented on HIVE-8329:


[~hagleitn] Good catch ! I will re-try with your command. I'm out of the office 
this week. I will try to rebuild a patch tomorrow but I don't have too much 
time.


 Enable postgres for storing stats
 -

 Key: HIVE-8329
 URL: https://issues.apache.org/jira/browse/HIVE-8329
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.14.0

 Attachments: HIVE-8329.1.patch, HIVE-8329.1.patch, HIVE-8329.1.patch


 Simple patch to enable postgresql as JDBC publisher for statistics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14186605#comment-14186605
 ] 

Damien Carol commented on HIVE-8329:


[~hagleitn] I don't know. I'm still working on it.

 Enable postgres for storing stats
 -

 Key: HIVE-8329
 URL: https://issues.apache.org/jira/browse/HIVE-8329
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.14.0

 Attachments: HIVE-8329.1.patch, HIVE-8329.1.patch, HIVE-8329.1.patch


 Simple patch to enable postgresql as JDBC publisher for statistics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14186608#comment-14186608
 ] 

Damien Carol commented on HIVE-8329:


Tests pass locally with :
{code}
 mvn -B -o test -Phadoop-2 -Dtest=TestCliDriver
{code}

 Enable postgres for storing stats
 -

 Key: HIVE-8329
 URL: https://issues.apache.org/jira/browse/HIVE-8329
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.14.0

 Attachments: HIVE-8329.1.patch, HIVE-8329.1.patch, HIVE-8329.1.patch


 Simple patch to enable postgresql as JDBC publisher for statistics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8628) NPE in case of shuffle join in tez

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8628:
---
Description: 
test throws NullPointerException:
{noformat}
Vertex failed, vertexName=Reducer 2, vertexId=vertex_1413774081318_0803_5_03, 
diagnostics=[Task failed, taskId=task_1413774081318_0803_5_03_00, 
diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running 
task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error 
while closing operators: null
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:187)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:142)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Hive Runtime Error while closing 
operators: null
at 
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:218)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:178)
... 13 more
Caused by: java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinFinalLeftData(CommonMergeJoinOperator.java:368)
at 
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.closeOp(CommonMergeJoinOperator.java:310)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:582)
at 
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:200)
... 14 more
], TaskAttempt 1 failed, info=[Error: Failure while running 
task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error 
while closing operators: null
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:187)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:142)
at 
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at 
org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Hive Runtime Error while closing 
operators: null
at 
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:218)
at 
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:178)
... 13 more
Caused by: java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.joinFinalLeftData(CommonMergeJoinOperator.java:368)
at 
org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.closeOp(CommonMergeJoinOperator.java:310)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:582)
at 
org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:200)
... 14 more
], TaskAttempt 2 failed, info=[Error: Failure while running 
task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error 
while 

[jira] [Commented] (HIVE-8543) Compactions fail on metastore using postgres

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182584#comment-14182584
 ] 

Damien Carol commented on HIVE-8543:


[~alangates] You're welcome. I'm sorry I was very busy these last few weeks I 
have not been able to take care of these postgres tickets. You're making a 
good job with these ones.

 Compactions fail on metastore using postgres
 

 Key: HIVE-8543
 URL: https://issues.apache.org/jira/browse/HIVE-8543
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Alan Gates
Assignee: Alan Gates
Priority: Critical
 Fix For: 0.14.0

 Attachments: HIVE-8543.patch


 The worker fails to update the stats when the metastore is using Postgres as 
 the RDBMS.  
 {code}
 org.postgresql.util.PSQLException: ERROR: relation tab_col_stats does not 
 exist
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8543) Compactions fail on metastore using postgres

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179704#comment-14179704
 ] 

Damien Carol commented on HIVE-8543:


[~alangates] I'm using your patch on a real cluster. Major compactions works. I 
reviewed the patch. LGTM

 Compactions fail on metastore using postgres
 

 Key: HIVE-8543
 URL: https://issues.apache.org/jira/browse/HIVE-8543
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Alan Gates
Assignee: Alan Gates
Priority: Critical
 Fix For: 0.14.0

 Attachments: HIVE-8543.patch


 The worker fails to update the stats when the metastore is using Postgres as 
 the RDBMS.  
 {code}
 org.postgresql.util.PSQLException: ERROR: relation tab_col_stats does not 
 exist
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8543) Compactions fail on metastore using postgres

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179707#comment-14179707
 ] 

Damien Carol commented on HIVE-8543:


{{org.apache.hive.minikdc.TestJdbcWithMiniKdc.testNegativeTokenAuth}} fails 
since 15 builds.

 Compactions fail on metastore using postgres
 

 Key: HIVE-8543
 URL: https://issues.apache.org/jira/browse/HIVE-8543
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Alan Gates
Assignee: Alan Gates
Priority: Critical
 Fix For: 0.14.0

 Attachments: HIVE-8543.patch


 The worker fails to update the stats when the metastore is using Postgres as 
 the RDBMS.  
 {code}
 org.postgresql.util.PSQLException: ERROR: relation tab_col_stats does not 
 exist
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179729#comment-14179729
 ] 

Damien Carol commented on HIVE-8329:


Tests pass locally. Re-uploading the *same* patch for third time. 

 Enable postgres for storing stats
 -

 Key: HIVE-8329
 URL: https://issues.apache.org/jira/browse/HIVE-8329
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.14.0

 Attachments: HIVE-8329.1.patch, HIVE-8329.1.patch


 Simple patch to enable postgresql as JDBC publisher for statistics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8329:
---
Attachment: HIVE-8329.1.patch

 Enable postgres for storing stats
 -

 Key: HIVE-8329
 URL: https://issues.apache.org/jira/browse/HIVE-8329
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.14.0

 Attachments: HIVE-8329.1.patch, HIVE-8329.1.patch, HIVE-8329.1.patch


 Simple patch to enable postgresql as JDBC publisher for statistics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8558) CBO: enable n-way joins after CBO join reordering

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8558:
---
Component/s: CBO

 CBO: enable n-way joins after CBO join reordering
 -

 Key: HIVE-8558
 URL: https://issues.apache.org/jira/browse/HIVE-8558
 Project: Hive
  Issue Type: Bug
  Components: CBO
Reporter: Harish Butani
Assignee: Harish Butani
 Attachments: HIVE-8558.1.patch


 With CBO, we disable n-way joins.  But for consecutive tables we can safely 
 collapse into n-way joins. For e.g this is safe to collapse:
 {code}
 select p1.p_name, p2.p_name, p3.p_name from part p1 join part p2 on p1.p_name 
 = p2.p_name join part p3 on p3.p_name = p1.p_name;
 {code}
 whereas, we shouldn't reorder in this case:
 {code}
 select p1.p_name, p2.p_name, p3.p_name 
 from part p1 join part p2 on p1.p_name = p2.p_name join part p3 on p3.p_size 
 = p1.p_size and p3.p_size = p2.p_size join part p4 on p1.p_name = p4.p_name;
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8122) Make use of SearchArgument classes for Parquet SERDE

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8122:
---
Summary: Make use of SearchArgument classes for Parquet SERDE  (was: Make 
use of SearchArgument classes)

 Make use of SearchArgument classes for Parquet SERDE
 

 Key: HIVE-8122
 URL: https://issues.apache.org/jira/browse/HIVE-8122
 Project: Hive
  Issue Type: Sub-task
Reporter: Brock Noland
Assignee: Ferdinand Xu
  Labels: Par

 ParquetSerde could be much cleaner if we used SearchArgument and associated 
 classes like ORC does:
 https://github.com/apache/hive/blob/trunk/serde/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgument.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8122) Make use of SearchArgument classes for Parquet SERDE

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8122:
---
Labels: Par  (was: )

 Make use of SearchArgument classes for Parquet SERDE
 

 Key: HIVE-8122
 URL: https://issues.apache.org/jira/browse/HIVE-8122
 Project: Hive
  Issue Type: Sub-task
Reporter: Brock Noland
Assignee: Ferdinand Xu
  Labels: Par

 ParquetSerde could be much cleaner if we used SearchArgument and associated 
 classes like ORC does:
 https://github.com/apache/hive/blob/trunk/serde/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgument.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8122) Make use of SearchArgument classes for Parquet SERDE

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8122:
---
Labels:   (was: Par)

 Make use of SearchArgument classes for Parquet SERDE
 

 Key: HIVE-8122
 URL: https://issues.apache.org/jira/browse/HIVE-8122
 Project: Hive
  Issue Type: Sub-task
Reporter: Brock Noland
Assignee: Ferdinand Xu

 ParquetSerde could be much cleaner if we used SearchArgument and associated 
 classes like ORC does:
 https://github.com/apache/hive/blob/trunk/serde/src/java/org/apache/hadoop/hive/ql/io/sarg/SearchArgument.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7689) Fix wrong lower case table names in Postgres Metastore back end

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179045#comment-14179045
 ] 

Damien Carol commented on HIVE-7689:


[~alangates] Great! I will push my patch tomorrow. 

 Fix wrong lower case table names in Postgres Metastore back end
 ---

 Key: HIVE-7689
 URL: https://issues.apache.org/jira/browse/HIVE-7689
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
  Labels: metastore, postgres
 Fix For: 0.14.0

 Attachments: HIVE-7689.5.patch, HIVE-7689.6.patch, HIVE-7689.7.patch, 
 HIVE-7689.8.patch, HIVE-7689.9.patch, HIVE-7889.1.patch, HIVE-7889.2.patch, 
 HIVE-7889.3.patch, HIVE-7889.4.patch


 Current 0.14 patch create table with lower case names.
 This patch fix wrong lower case tables names in Postgres Metastore back end.
 Mixing lower case and upper case throws bugs in {{JDBCStatsPublisher}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8543) Compactions fail on metastore using postgres

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179048#comment-14179048
 ] 

Damien Carol commented on HIVE-8543:


I already have a patch for this one. You could assign it to me.

 Compactions fail on metastore using postgres
 

 Key: HIVE-8543
 URL: https://issues.apache.org/jira/browse/HIVE-8543
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Alan Gates
Assignee: Alan Gates
Priority: Critical
 Fix For: 0.14.0


 The worker fails to update the stats when the metastore is using Postgres as 
 the RDBMS.  
 {code}
 org.postgresql.util.PSQLException: ERROR: relation tab_col_stats does not 
 exist
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7689) Fix wrong lower case table names in Postgres Metastore back end

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14170715#comment-14170715
 ] 

Damien Carol commented on HIVE-7689:


[~brocknoland] [~alangates] I discovered a new problem with lower/upper case 
mixing.

{{org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler}} try to access to 
relation {{TAB_COL_STATS}}.

Without identifier quote, the method {{findColumnsWithStats}} throws errors :

{noformat}
2014-10-14 10:50:02,313 DEBUG [nc-h04-22]: txn.CompactionTxnHandler 
(CompactionTxnHandler.java:findColumnsWithStats(554)) - Going to execute 
SELECT COLUMN_NAME FROM TAB_COL_STATS WHERE DB_NAME='casino' AND 
TABLE_NAME='foo7'
2014-10-14 10:50:02,314 ERROR [nc-h04-22]: txn.CompactionTxnHandler 
(CompactionTxnHandler.java:findColumnsWithStats(566)) - Failed to find columns 
to analyze stats on for foo7
org.postgresql.util.PSQLException: ERROR: relation tab_col_stats does not 
exist
  Position: 25
at 
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2096)
at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1829)
at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:372)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:252)
at 
com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464)
at 
org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findColumnsWithStats(CompactionTxnHandler.java:555)
at org.apache.hadoop.hive.ql.txn.compactor.Worker.run(Worker.java:140)
{noformat}

 Fix wrong lower case table names in Postgres Metastore back end
 ---

 Key: HIVE-7689
 URL: https://issues.apache.org/jira/browse/HIVE-7689
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
  Labels: metastore, postgres
 Fix For: 0.14.0

 Attachments: HIVE-7689.5.patch, HIVE-7689.6.patch, HIVE-7689.7.patch, 
 HIVE-7689.8.patch, HIVE-7689.9.patch, HIVE-7889.1.patch, HIVE-7889.2.patch, 
 HIVE-7889.3.patch, HIVE-7889.4.patch


 Current 0.14 patch create table with lower case names.
 This patch fix wrong lower case tables names in Postgres Metastore back end.
 Mixing lower case and upper case throws bugs in {{JDBCStatsPublisher}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8381) Update hive version on trunk to 0.15

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166518#comment-14166518
 ] 

Damien Carol commented on HIVE-8381:


Ok. I understand.

 Update hive version on trunk to 0.15
 

 Key: HIVE-8381
 URL: https://issues.apache.org/jira/browse/HIVE-8381
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.15.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
 Attachments: HIVE-8381.1.patch, HIVE-8381.2.patch


 In view of the branching for 0.14 done, we need to upgrade the version of 
 hive on trunk to 0.15.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8329:
---
Attachment: HIVE-8329.1.patch

Re-upload same patch to trigger tests.

 Enable postgres for storing stats
 -

 Key: HIVE-8329
 URL: https://issues.apache.org/jira/browse/HIVE-8329
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.14.0

 Attachments: HIVE-8329.1.patch, HIVE-8329.1.patch


 Simple patch to enable postgresql as JDBC publisher for statistics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8270) JDBC uber jar is missing some classes required in secure setup.

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164949#comment-14164949
 ] 

Damien Carol commented on HIVE-8270:


[~ashutoshc] I'm working on shading it. If I succeed, I will create a new JIRA.

 JDBC uber jar is missing some classes required in secure setup.
 ---

 Key: HIVE-8270
 URL: https://issues.apache.org/jira/browse/HIVE-8270
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
  Labels: TODOC14
 Fix For: 0.14.0

 Attachments: HIVE-8270.1.patch


 JDBC uber jar is missing some required classes for a secure setup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8329:
---
Attachment: HIVE-8329.1.patch

First patch.

 Enable postgres for storing stats
 -

 Key: HIVE-8329
 URL: https://issues.apache.org/jira/browse/HIVE-8329
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.14.0

 Attachments: HIVE-8329.1.patch


 Simple patch to enable postgresql as JDBC publisher for statistics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8329:
---
Status: Patch Available  (was: Open)

 Enable postgres for storing stats
 -

 Key: HIVE-8329
 URL: https://issues.apache.org/jira/browse/HIVE-8329
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.14.0

 Attachments: HIVE-8329.1.patch


 Simple patch to enable postgresql as JDBC publisher for statistics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8381) Update hive version on trunk to 0.15

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165898#comment-14165898
 ] 

Damien Carol commented on HIVE-8381:


[~brocknoland] Could we wait the release of the RC before changing version on 
trunk?

 Update hive version on trunk to 0.15
 

 Key: HIVE-8381
 URL: https://issues.apache.org/jira/browse/HIVE-8381
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.15.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
 Attachments: HIVE-8381.1.patch, HIVE-8381.2.patch


 In view of the branching for 0.14 done, we need to upgrade the version of 
 hive on trunk to 0.15.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8270) JDBC uber jar is missing some classes required in secure setup.

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163546#comment-14163546
 ] 

Damien Carol commented on HIVE-8270:


There still have a bug in Tomcat.
{noformat}
INFO: 
validateJarFile(/usr/share/apache-tomcat-7.0.50/webapps/SmartDataDev/WEB-INF/lib/hive-jdbc-0.14.0-SNAPSHOT-standalone.jar)
 - jar not loaded. See Servlet Spec 3.0, section 10.7.2. Offending class: 
javax/servlet/Servlet.class
Oct 08, 2014 4:13:02 PM org.apache.catalina.startup.HostConfig deployWAR
{noformat}

After few tests, I see many javax.* and javax.jdo.* classes in the ubber jar. 
It's a real problem.
Avoiding the classes in javax.* namespace will reduce heavily the size of the 
jar.

I think the JDBC standalone jar is poor quality currently. It is unusable in 
J2EE environment.

Also why embedded JDO classes ? 

 JDBC uber jar is missing some classes required in secure setup.
 ---

 Key: HIVE-8270
 URL: https://issues.apache.org/jira/browse/HIVE-8270
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 0.14.0
Reporter: Vaibhav Gumashta
Assignee: Vaibhav Gumashta
  Labels: TODOC14
 Fix For: 0.14.0

 Attachments: HIVE-8270.1.patch


 JDBC uber jar is missing some required classes for a secure setup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-6669) sourcing txn-script from schema script results in failure for mysql oracle

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14162057#comment-14162057
 ] 

Damien Carol commented on HIVE-6669:


[~alangates] Could you add quotes to identifiers in the postgres scripts. This 
way, the lowercase identifiers are explicit?

 sourcing txn-script from schema script results in failure for mysql  oracle
 

 Key: HIVE-6669
 URL: https://issues.apache.org/jira/browse/HIVE-6669
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Prasad Mujumdar
Assignee: Alan Gates
Priority: Blocker
 Attachments: HIVE-6669.patch


 This issues is addressed in 0.13 by in-lining the the transaction schema 
 statements in the schema initialization script (HIVE-6559)
 The 0.14 schema initialization is not fixed. This is the followup ticket for 
 to address the problem in 0.14. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-6669) sourcing txn-script from schema script results in failure for mysql oracle

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14162457#comment-14162457
 ] 

Damien Carol commented on HIVE-6669:


[~alangates] I think we should use char {{}} like in the others upgrade 
scripts of the metastore.
{code:sql}
CREATE TABLE BUCKETING_COLS (
SD_ID bigint NOT NULL,
BUCKET_COL_NAME character varying(256) DEFAULT NULL::character varying,
INTEGER_IDX bigint NOT NULL
);
{code}

Link here 
https://github.com/apache/hive/blob/trunk/metastore/scripts/upgrade/postgres/hive-schema-0.14.0.postgres.sql#L22

 sourcing txn-script from schema script results in failure for mysql  oracle
 

 Key: HIVE-6669
 URL: https://issues.apache.org/jira/browse/HIVE-6669
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Prasad Mujumdar
Assignee: Alan Gates
Priority: Blocker
 Attachments: HIVE-6669.patch


 This issues is addressed in 0.13 by in-lining the the transaction schema 
 statements in the schema initialization script (HIVE-6559)
 The 0.14 schema initialization is not fixed. This is the followup ticket for 
 to address the problem in 0.14. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-6669) sourcing txn-script from schema script results in failure for mysql oracle

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159058#comment-14159058
 ] 

Damien Carol commented on HIVE-6669:


[~alangates] I will test it on Monday 2014-10-13 but at first sight LGTM.

 sourcing txn-script from schema script results in failure for mysql  oracle
 

 Key: HIVE-6669
 URL: https://issues.apache.org/jira/browse/HIVE-6669
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Prasad Mujumdar
Assignee: Alan Gates
Priority: Blocker
 Attachments: HIVE-6669.patch


 This issues is addressed in 0.13 by in-lining the the transaction schema 
 statements in the schema initialization script (HIVE-6559)
 The 0.14 schema initialization is not fixed. This is the followup ticket for 
 to address the problem in 0.14. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8235) Insert into partitioned bucketed sorted tables fails with this file is already being created by

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8235:
---
Description: 
When loading into a partitioned bucketed sorted table the query fails with 
{code}
Caused by: 
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException):
 Failed to create file 
[/tmp/hive/mmokhtar/621d7923-90d1-4d9d-a4c6-b3bb075c7a8c/hive_2014-09-22_23-25-11_678_1598300430132235708-1/_task_tmp.-ext-1/ss_sold_date=1998-01-02/_tmp.00_3/delta_0123305_0123305/bucket_0]
 for [DFSClient_attempt_1406566393272_6085_r_000144_3_-1677753045_12] for 
client [172.21.128.111], because this file is already being created by 
[DFSClient_attempt_1406566393272_6085_r_31_3_-1506661042_12] on 
[172.21.128.122]
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:2543)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2308)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2237)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2190)
at 
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:520)
at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:354)
at 
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)

at org.apache.hadoop.ipc.Client.call(Client.java:1410)
at org.apache.hadoop.ipc.Client.call(Client.java:1363)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at com.sun.proxy.$Proxy15.create(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:190)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:103)
at com.sun.proxy.$Proxy15.create(Unknown Source)
at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.create(ClientNamenodeProtocolTranslatorPB.java:258)
at 
org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1600)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1465)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1390)
at 
org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:394)
at 
org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:390)
at 
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at 
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:390)
at 
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:334)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:906)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:887)
at 
org.apache.hadoop.hive.ql.io.orc.WriterImpl.getStream(WriterImpl.java:1966)
at 
org.apache.hadoop.hive.ql.io.orc.WriterImpl.flushStripe(WriterImpl.java:1983)
at 
org.apache.hadoop.hive.ql.io.orc.WriterImpl.close(WriterImpl.java:2287)
at 
org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.close(OrcRecordUpdater.java:356)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriter
{code}

DDL  
{code:sql}
CREATE TABLE store_sales_sb ( 
ss_sold_date_sk   int,
ss_sold_time_sk   int,
ss_item_skint,
ss_customer_skint,
ss_cdemo_sk   int,
ss_hdemo_sk   int,
ss_addr_skint,
ss_store_sk   int,
ss_promo_sk   int,

[jira] [Commented] (HIVE-7689) Fix wrong lower case table names in Postgres Metastore back end

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156148#comment-14156148
 ] 

Damien Carol commented on HIVE-7689:


[~alangates] ok. I agree. I close this bug as Will Not Fix as I don't want to 
block the 0.14, thanks for your feedbacks.

 Fix wrong lower case table names in Postgres Metastore back end
 ---

 Key: HIVE-7689
 URL: https://issues.apache.org/jira/browse/HIVE-7689
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
  Labels: metastore, postgres
 Fix For: 0.14.0

 Attachments: HIVE-7689.5.patch, HIVE-7689.6.patch, HIVE-7689.7.patch, 
 HIVE-7689.8.patch, HIVE-7689.9.patch, HIVE-7889.1.patch, HIVE-7889.2.patch, 
 HIVE-7889.3.patch, HIVE-7889.4.patch


 Current 0.14 patch create table with lower case names.
 This patch fix wrong lower case tables names in Postgres Metastore back end.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7689) Fix wrong lower case table names in Postgres Metastore back end

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-7689:
---
Resolution: Won't Fix
Status: Resolved  (was: Patch Available)

 Fix wrong lower case table names in Postgres Metastore back end
 ---

 Key: HIVE-7689
 URL: https://issues.apache.org/jira/browse/HIVE-7689
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
  Labels: metastore, postgres
 Fix For: 0.14.0

 Attachments: HIVE-7689.5.patch, HIVE-7689.6.patch, HIVE-7689.7.patch, 
 HIVE-7689.8.patch, HIVE-7689.9.patch, HIVE-7889.1.patch, HIVE-7889.2.patch, 
 HIVE-7889.3.patch, HIVE-7889.4.patch


 Current 0.14 patch create table with lower case names.
 This patch fix wrong lower case tables names in Postgres Metastore back end.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7689) Fix wrong lower case table names in Postgres Metastore back end

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-7689:
---
Description: 
Current 0.14 patch create table with lower case names.
This patch fix wrong lower case tables names in Postgres Metastore back end.

Mixing lower case and upper case throws bugs in {{JDBCStatsPublisher}}

  was:
Current 0.14 patch create table with lower case names.
This patch fix wrong lower case tables names in Postgres Metastore back end.


 Fix wrong lower case table names in Postgres Metastore back end
 ---

 Key: HIVE-7689
 URL: https://issues.apache.org/jira/browse/HIVE-7689
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
  Labels: metastore, postgres
 Fix For: 0.14.0

 Attachments: HIVE-7689.5.patch, HIVE-7689.6.patch, HIVE-7689.7.patch, 
 HIVE-7689.8.patch, HIVE-7689.9.patch, HIVE-7889.1.patch, HIVE-7889.2.patch, 
 HIVE-7889.3.patch, HIVE-7889.4.patch


 Current 0.14 patch create table with lower case names.
 This patch fix wrong lower case tables names in Postgres Metastore back end.
 Mixing lower case and upper case throws bugs in {{JDBCStatsPublisher}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-7689) Fix wrong lower case table names in Postgres Metastore back end

[Damien Carol (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156199#comment-14156199
 ] 

Damien Carol commented on HIVE-7689:


[~alangates] [[~brocknoland] Now I can confirm that without the patch, 
Metastore backend for stats on postgres don't work.

when using jdbc connector for stats, {{JDBCStatsPublisher}} throws errors. Here 
the error in the logs :
{noformat}
14/10/02 09:49:11 ERROR jdbc.JDBCStatsPublisher: Error during JDBC 
initialization.
org.postgresql.util.PSQLException: ERROR: relation partition_stats_v2 already 
exists
at 
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2096)
at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1829)
at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:372)
at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:300)
at 
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.init(JDBCStatsPublisher.java:286)
{noformat}

It is a side effect of mixing lower case and upper case.

To understand this bug :

1. JDBC stats publisher use this code to check if table exists.
{code}
 // Check if the table exists
DatabaseMetaData dbm = conn.getMetaData();
rs = dbm.getTables(null, null, JDBCStatsUtils.getStatTableName(), null);
boolean tblExists = rs.next();
if (!tblExists) { // Table does not exist, create it
  String createTable = JDBCStatsUtils.getCreate();
  stmt.executeUpdate(createTable);  
}
{code}

2. But {{JDBCStatsUtils.getStatTableName()}} value is {{PARTITION_STATS_V2}} in 
*uppercase*.
3. The Metastore try to create an already existing table.

I will create another JIRA that fix *only* this part.


 Fix wrong lower case table names in Postgres Metastore back end
 ---

 Key: HIVE-7689
 URL: https://issues.apache.org/jira/browse/HIVE-7689
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
  Labels: metastore, postgres
 Fix For: 0.14.0

 Attachments: HIVE-7689.5.patch, HIVE-7689.6.patch, HIVE-7689.7.patch, 
 HIVE-7689.8.patch, HIVE-7689.9.patch, HIVE-7889.1.patch, HIVE-7889.2.patch, 
 HIVE-7889.3.patch, HIVE-7889.4.patch


 Current 0.14 patch create table with lower case names.
 This patch fix wrong lower case tables names in Postgres Metastore back end.
 Mixing lower case and upper case throws bugs in {{JDBCStatsPublisher}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-8329) Enable postgres for storing stats

[Damien Carol (JIRA)]

Damien Carol created HIVE-8329:
--

 Summary: Enable postgres for storing stats
 Key: HIVE-8329
 URL: https://issues.apache.org/jira/browse/HIVE-8329
 Project: Hive
  Issue Type: Bug
  Components: Statistics
Affects Versions: 0.14.0
Reporter: Damien Carol
Assignee: Damien Carol
Priority: Blocker
 Fix For: 0.14.0


Simple patch to enable postgresql as JDBC publisher for statistics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)