date:20210709

[jira] [Updated] (CARBONDATA-4243) Select filter query with to_date in filter fails for table with column_meta_cache configured also having SI

2021-07-09 Thread Chetan Bhat (Jira)



 [ 
https://issues.apache.org/jira/browse/CARBONDATA-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chetan Bhat updated CARBONDATA-4243:

Description: 
Create table with column_meta_cache, create secondary indexes and load data to 
table. 

Execute the Select filter query with to_date in filter.

CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, 
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 
decimal(36,36),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 
int) stored as carbondata 
TBLPROPERTIES('COLUMN_META_CACHE'='CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ');
 CREATE INDEX indextable2 ON TABLE uniqdata (DOB) AS 'carbondata';
 CREATE INDEX indextable3 ON TABLE uniqdata (DOJ) AS 'carbondata';
 LOAD DATA INPATH 'hdfs://hacluster/chetan/2000_UniqData.csv' into table 
uniqdata OPTIONS('DELIMITER'=',' , 
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');

 

*Issue: Select filter query with to_date in filter fails for table with 
column_meta_cache configured also having SI*

0: jdbc:hive2://10.21.19.14:23040/default> select 
max(to_date(DOB)),min(to_date(DOB)),count(to_date(DOB)) from uniqdata where 
to_date(DOB)='1975-06-11' or to_date(Dn select 
max(to_date(DOB)),min(to_date(DOB)),count(to_date(DOB)) from uniqdata where 
to_date(DOB)='1975-06-11' or to_date(DOB)='1975-06-23';
 Error: org.apache.hive.service.cli.HiveSQLException: Error running query: 
org.apache.spark.sql.catalyst.errors.package$TreeNodeException: makeCopy, tree:
 !BroadCastSIFilterPushJoin [none#0|#0], [none#1|#1], Inner, BuildRight
 :- *(6) ColumnarToRow
 : +- Scan CarbonDatasourceHadoopRelation chetan.uniqdata[dob#847024|#847024] 
Batched: true, DirectScan: false, PushedFilters: [((cast(input[0] as date) = 
1987) or (cast(in9))], ReadSchema: [dob]
 +- *(8) HashAggregate(keys=[positionReference#847161|#847161], functions=[], 
output=[positionReference#847161|#847161])
 +- ReusedExchange [positionReference#847161|#847161], Exchange 
hashpartitioning(positionReference#847161, 200), ENSURE_REQUIREMENTS, 
[id=#195473|#195473]

at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(Sparation.scala:361)
 at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263)
 at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
 at 
org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78)
 at 
org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62)
 at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:43)
 at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263)
 at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
 at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
 Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: 
makeCopy, tree:
 !BroadCastSIFilterPushJoin [none#0|#0], [none#1|#1], Inner, BuildRight
 :- *(6) ColumnarToRow
 : +- Scan CarbonDatasourceHadoopRelation chetan.uniqdata[dob#847024|#847024] 
Batched: true, DirectScan: false, PushedFilters: [((cast(input[0] as date) = 
1987) or (cast(in9))], ReadSchema: [dob]
 +- *(8) HashAggregate(keys=[positionReference#847161|#847161], functions=[], 
output=[positionReference#847161|#847161])
 +- ReusedExchange [positionReference#847161|#847161], Exchange 
hashpartitioning(positionReference#847161, 200), ENSURE_REQUIREMENTS, 
[id=#195473|#195473]

at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:56)
 at org.apache.spark.sql.catalyst.trees.TreeNode.makeCopy(TreeNode.scala:468)
 at org.apache.spark.sql.catalyst.trees.TreeNode.makeCopy(TreeNode.scala:457)
 at

[jira] [Created] (CARBONDATA-4243) Select filter query with to_date in filter fails for table with column_meta_cache configured also having SI

2021-07-09 Thread Chetan Bhat (Jira)

Chetan Bhat created CARBONDATA-4243:
---

 Summary: Select filter query with to_date in filter fails for 
table with column_meta_cache configured also having SI
 Key: CARBONDATA-4243
 URL: https://issues.apache.org/jira/browse/CARBONDATA-4243
 Project: CarbonData
  Issue Type: Bug
  Components: sql
Affects Versions: 2.2.0
 Environment: Spark 3.1.1
Reporter: Chetan Bhat


Create table with column_meta_cache, create secondary indexes and load data to 
table. 

Execute the Select filter query with to_date in filter.

CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, 
DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 
bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 
decimal(36,36),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 
int) stored as carbondata 
TBLPROPERTIES('COLUMN_META_CACHE'='CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ');
CREATE INDEX indextable2 ON TABLE uniqdata (DOB) AS 'carbondata';
CREATE INDEX indextable3 ON TABLE uniqdata (DOJ) AS 'carbondata';
LOAD DATA INPATH 'hdfs://hacluster/chetan/2000_UniqData.csv' into table 
uniqdata OPTIONS('DELIMITER'=',' , 
'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');

 

Issue: Select filter query with to_date in filter fails for table with 
column_meta_cache configured also having SI

0: jdbc:hive2://10.21.19.14:23040/default> select 
max(to_date(DOB)),min(to_date(DOB)),count(to_date(DOB)) from uniqdata where 
to_date(DOB)='1975-06-11' or to_date(Dn select 
max(to_date(DOB)),min(to_date(DOB)),count(to_date(DOB)) from uniqdata where 
to_date(DOB)='1975-06-11' or to_date(DOB)='1975-06-23';
Error: org.apache.hive.service.cli.HiveSQLException: Error running query: 
org.apache.spark.sql.catalyst.errors.package$TreeNodeException: makeCopy, tree:
!BroadCastSIFilterPushJoin [none#0], [none#1], Inner, BuildRight
:- *(6) ColumnarToRow
: +- Scan CarbonDatasourceHadoopRelation chetan.uniqdata[dob#847024] Batched: 
true, DirectScan: false, PushedFilters: [((cast(input[0] as date) = 1987) or 
(cast(in9))], ReadSchema: [dob]
+- *(8) HashAggregate(keys=[positionReference#847161], functions=[], 
output=[positionReference#847161])
 +- ReusedExchange [positionReference#847161], Exchange 
hashpartitioning(positionReference#847161, 200), ENSURE_REQUIREMENTS, 
[id=#195473]

at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute(Sparation.scala:361)
 at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.$anonfun$run$2(SparkExecuteStatementOperation.scala:263)
 at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
 at 
org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties(SparkOperation.scala:78)
 at 
org.apache.spark.sql.hive.thriftserver.SparkOperation.withLocalProperties$(SparkOperation.scala:62)
 at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation.withLocalProperties(SparkExecuteStatementOperation.scala:43)
 at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:263)
 at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3.run(SparkExecuteStatementOperation.scala:258)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
 at 
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2.run(SparkExecuteStatementOperation.scala:272)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: 
makeCopy, tree:
!BroadCastSIFilterPushJoin [none#0], [none#1], Inner, BuildRight
:- *(6) ColumnarToRow
: +- Scan CarbonDatasourceHadoopRelation chetan.uniqdata[dob#847024] Batched: 
true, DirectScan: false, PushedFilters: [((cast(input[0] as date) = 1987) or 
(cast(in9))], ReadSchema: [dob]
+- *(8) HashAggregate(keys=[positionReference#847161], functions=[], 
output=[positionReference#847161])
 +- ReusedExchange [positionReference#847161], Exchange 
hashpartitioning(positionReference#847161, 200), ENSURE_REQUIREMENTS, 
[id=#195473]

at org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:56)

[GitHub] [carbondata] vikramahuja1001 commented on issue #4168: use java 11 build spark 3.1 failed

2021-07-09 Thread GitBox



vikramahuja1001 commented on issue #4168:
URL: https://github.com/apache/carbondata/issues/4168#issuecomment-877129631


   hi @LiuLarry , you can try using the Oracle Java as given in the [build 
page](https://github.com/apache/carbondata/tree/master/build)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (CARBONDATA-4243) Select filter query with to_date in filter fails for table with column_meta_cache configured also having SI

[jira] [Created] (CARBONDATA-4243) Select filter query with to_date in filter fails for table with column_meta_cache configured also having SI

[GitHub] [carbondata] vikramahuja1001 commented on issue #4168: use java 11 build spark 3.1 failed

3 matches

Site Navigation

Mail list logo

Footer information