[
https://issues.apache.org/jira/browse/CARBONDATA-4024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanna Ravichandran closed CARBONDATA-4024.
---------------------------------------------
Resolution: Duplicate
> Select queries with filter and aggregate queries are not working in Hive
> write - carbon table.
> -----------------------------------------------------------------------------------------------
>
> Key: CARBONDATA-4024
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4024
> Project: CarbonData
> Issue Type: Bug
> Components: hive-integration
> Affects Versions: 2.0.0
> Reporter: Prasanna Ravichandran
> Priority: Major
>
> Select queries with filter and aggregate queries are not working in Hive
> write - carbon table.
> Hive - console:
> 0: /> use t2;
> INFO : State: Compiling.
> INFO : Compiling
> command(queryId=omm_20201008191831_ac10f1ae-8d39-4185-b25a-d690134a94be): use
> t2;
> Current sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : hive.compile.auto.avoid.cbo=true
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling
> command(queryId=omm_20201008191831_ac10f1ae-8d39-4185-b25a-d690134a94be);
> Time taken: 0.122 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : State: Executing.
> INFO : Executing
> command(queryId=omm_20201008191831_ac10f1ae-8d39-4185-b25a-d690134a94be): use
> t2; Current sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing
> command(queryId=omm_20201008191831_ac10f1ae-8d39-4185-b25a-d690134a94be);
> Time taken: 0.019 seconds
> INFO : OK
> INFO : Concurrency mode is disabled, not creating a lock manager
> No rows affected (0.207 seconds)
> 0: /> show tables;
> INFO : State: Compiling.
> INFO : Compiling
> command(queryId=omm_20201008191835_5e1e9469-0054-446f-af82-ec3294ec77b1):
> show tables;
> Current sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : hive.compile.auto.avoid.cbo=true
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name,
> type:string, comment:from deserializer)], properties:null)
> INFO : Completed compiling
> command(queryId=omm_20201008191835_5e1e9469-0054-446f-af82-ec3294ec77b1);
> Time taken: 0.015 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : State: Executing.
> INFO : Executing
> command(queryId=omm_20201008191835_5e1e9469-0054-446f-af82-ec3294ec77b1):
> show tables; Current sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing
> command(queryId=omm_20201008191835_5e1e9469-0054-446f-af82-ec3294ec77b1);
> Time taken: 0.016 seconds
> INFO : OK
> INFO : Concurrency mode is disabled, not creating a lock manager
> +----------------+
> | tab_name |
> +----------------+
> | hive_carbon |
> | hive_table |
> | parquet_table |
> +----------------+
> 3 rows selected (0.114 seconds)
> 0: /> select * from hive_carbon;
> INFO : State: Compiling.
> INFO : Compiling
> command(queryId=omm_20201008191842_9378bab9-181c-455e-aa6d-9b4f787ce6da):
> select * from hive_carbon;
> Current sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : hive.compile.auto.avoid.cbo=true
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Current sql is not contains insert syntax, not need record dest table
> flag
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:hive_carbon.id, type:int,
> comment:null), FieldSchema(name:hive_carbon.name, type:string, comment:null),
> FieldSchema(name:hive_carbon.scale, type:decimal(10,0), comment:null),
> FieldSchema(name:hive_carbon.country, type:string, comment:null),
> FieldSchema(name:hive_carbon.salary, type:double, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=omm_20201008191842_9378bab9-181c-455e-aa6d-9b4f787ce6da);
> Time taken: 0.511 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : State: Executing.
> INFO : Executing
> command(queryId=omm_20201008191842_9378bab9-181c-455e-aa6d-9b4f787ce6da):
> select * from hive_carbon; Current
> sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : Completed executing
> command(queryId=omm_20201008191842_9378bab9-181c-455e-aa6d-9b4f787ce6da);
> Time taken: 0.001 seconds
> INFO : OK
> INFO : Concurrency mode is disabled, not creating a lock manager
> +-----------------+-------------------+--------------------+----------------------+---------------------+
> | hive_carbon.id | hive_carbon.name | hive_carbon.scale | hive_carbon.country
> | hive_carbon.salary |
> +-----------------+-------------------+--------------------+----------------------+---------------------+
> | 1 | Ram | 2 | India | 3500.0 |
> +-----------------+-------------------+--------------------+----------------------+---------------------+
> 1 row selected (0.614 seconds)
> 0: /> select * from hive_carbon where hive_carbon.id=1;
> INFO : State: Compiling.
> INFO : Compiling
> command(queryId=omm_20201008191854_20cca5e2-a9ae-470a-acbc-1d0ceb46f4e2):
> select * from hive_carbon where hive_carbon.id=1; Current
> sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : hive.compile.auto.avoid.cbo=true
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Current sql is not contains insert syntax, not need record dest table
> flag
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:hive_carbon.id, type:int,
> comment:null), FieldSchema(name:hive_carbon.name, type:string, comment:null),
> FieldSchema(name:hive_carbon.scale, type:decimal(10,0), comment:null),
> FieldSchema(name:hive_carbon.country, type:string, comment:null),
> FieldSchema(name:hive_carbon.salary, type:double, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=omm_20201008191854_20cca5e2-a9ae-470a-acbc-1d0ceb46f4e2);
> Time taken: 0.215 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : State: Executing.
> INFO : Executing
> command(queryId=omm_20201008191854_20cca5e2-a9ae-470a-acbc-1d0ceb46f4e2):
> select * from hive_carbon where hive_carbon.id=1; Current
> sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> WARN : Hive-on-MR is deprecated in Hive 2 and may not be available in the
> future versions. Consider using a different execution engine (i.e. spark,
> tez) or using Hive 1.X releases.
> INFO : Query ID = omm_20201008191854_20cca5e2-a9ae-470a-acbc-1d0ceb46f4e2,
> Current sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Number of reduce tasks is set to 0 since there's no reduce operator
> INFO : number of splits:1
> INFO : Submitting tokens for job: job_1601898485220_0037
> INFO : Executing with tokens: [Kind: HDFS_DELEGATION_TOKEN, Service:
> ha-hdfs:hacluster, Ident: (token for carbon: HDFS_DELEGATION_TOKEN
> owner=carbon, renewer=mapred, realUser=hive/hadoop.hadoop....@hadoop.com,
> issueDate=1602164934768, maxDate=1602769734768, sequenceNumber=5596,
> masterKeyId=10), Kind: HIVE_DELEGATION_TOKEN, Service:
> HiveServer2ImpersonationToken, Ident: 00 06 63 61 72 62 6f 6e 06 63 61 72 62
> 6f 6e 21 68 69 76 65 2f 68 61 64 6f 6f 70 2e 68 61 64 6f 6f 70 2e 63 6f 6d 40
> 48 41 44 4f 4f 50 2e 43 4f 4d 8a 01 75 08 78 47 0e 8a 01 75 2c 84 cb 0e 8e 15
> f1 5a]
> INFO : The url to track the job:
> <Job-history-server-URL>/application_1601898485220_0037/
> INFO : Starting Job = job_1601898485220_0037, Tracking URL =
> <Job-history-server-URL>/application_1601898485220_0037/, Current
> sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : Kill Command =
> /opt/huawei/Bigdata/FusionInsight_HD_8.0.2/install/FusionInsight-Hive-3.1.0/hive-3.1.0/bin/..//../hadoop/bin/mapred
> job -kill job_1601898485220_0037
> INFO : Hadoop job information for Stage-1: number of mappers: 1; number of
> reducers: 0
> INFO : 2020-10-08 19:19:11,339 Stage-1 map = 0%, reduce = 0%
> INFO : 2020-10-08 19:19:44,231 Stage-1 map = 100%, reduce = 0%
> ERROR : Ended Job = job_1601898485220_0037 with errors
> INFO : MapReduce Jobs Launched:
> INFO : Stage-Stage-1: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
> INFO : Total MapReduce CPU Time Spent: 0 msec
> INFO : Completed executing
> command(queryId=omm_20201008191854_20cca5e2-a9ae-470a-acbc-1d0ceb46f4e2);
> Time taken: 50.866 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=2)
> 0: /> select count(*) from hive_carbon;
> INFO : State: Compiling.
> INFO : Compiling
> command(queryId=omm_20201008191958_51b13d46-d5f5-4b96-88bf-a953ad339c19):
> select count(*) from hive_carbon; Current
> sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : hive.compile.auto.avoid.cbo=true
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : Current sql is not contains insert syntax, not need record dest table
> flag
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:bigint, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=omm_20201008191958_51b13d46-d5f5-4b96-88bf-a953ad339c19);
> Time taken: 0.238 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> INFO : State: Executing.
> INFO : Executing
> command(queryId=omm_20201008191958_51b13d46-d5f5-4b96-88bf-a953ad339c19):
> select count(*) from hive_carbon; Current
> sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> WARN : Hive-on-MR is deprecated in Hive 2 and may not be available in the
> future versions. Consider using a different execution engine (i.e. spark,
> tez) or using Hive 1.X releases.
> INFO : Query ID = omm_20201008191958_51b13d46-d5f5-4b96-88bf-a953ad339c19,
> Current sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Number of reduce tasks determined at compile time: 1
> INFO : In order to change the average load for a reducer (in bytes):
> INFO : set hive.exec.reducers.bytes.per.reducer=<number>
> INFO : In order to limit the maximum number of reducers:
> INFO : set hive.exec.reducers.max=<number>
> INFO : In order to set a constant number of reducers:
> INFO : set mapreduce.job.reduces=<number>
> INFO : number of splits:1
> INFO : Submitting tokens for job: job_1601898485220_0038
> INFO : Executing with tokens: [Kind: HDFS_DELEGATION_TOKEN, Service:
> ha-hdfs:hacluster, Ident: (token for carbon: HDFS_DELEGATION_TOKEN
> owner=carbon, renewer=mapred, realUser=hive/hadoop.hadoop....@hadoop.com,
> issueDate=1602164998767, maxDate=1602769798767, sequenceNumber=5597,
> masterKeyId=10), Kind: HIVE_DELEGATION_TOKEN, Service:
> HiveServer2ImpersonationToken, Ident: 00 06 63 61 72 62 6f 6e 06 63 61 72 62
> 6f 6e 21 68 69 76 65 2f 68 61 64 6f 6f 70 2e 68 61 64 6f 6f 70 2e 63 6f 6d 40
> 48 41 44 4f 4f 50 2e 43 4f 4d 8a 01 75 08 78 47 0e 8a 01 75 2c 84 cb 0e 8e 15
> f1 5a]
> INFO : The url to track the job:
> <Job-history-server-URL>/application_1601898485220_0038/
> INFO : Starting Job = job_1601898485220_0038, Tracking URL =
> <Job-history-server-URL>/application_1601898485220_0038/, Current
> sessionId=35d8eaaa-6d9f-4e8e-a837-e059b4eb85b4
> INFO : Kill Command =
> /opt/huawei/Bigdata/FusionInsight_HD_8.0.2/install/FusionInsight-Hive-3.1.0/hive-3.1.0/bin/..//../hadoop/bin/mapred
> job -kill job_1601898485220_0038
> INFO : Hadoop job information for Stage-1: number of mappers: 1; number of
> reducers: 1
> INFO : 2020-10-08 19:20:17,684 Stage-1 map = 0%, reduce = 0%
> INFO : 2020-10-08 19:20:50,546 Stage-1 map = 100%, reduce = 100%
> ERROR : Ended Job = job_1601898485220_0038 with errors
> INFO : MapReduce Jobs Launched:
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=2)
> *Common error found in the Job history server URL for filter and aggregate
> queries - In map job for both the above issue:*
> Error: java.io.IOException: java.io.IOException: Database name is not set. at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:414)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:843)
> at
> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:175)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:444) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:349) at
> org.apache.hadoop.mapred.YarnChild$1.run(YarnChild.java:175) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1737)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169) Caused by:
> java.io.IOException: Database name is not set. at
> org.apache.carbondata.hadoop.api.CarbonInputFormat.getDatabaseName(CarbonInputFormat.java:841)
> at
> org.apache.carbondata.hive.MapredCarbonInputFormat.getCarbonTable(MapredCarbonInputFormat.java:80)
> at
> org.apache.carbondata.hive.MapredCarbonInputFormat.getQueryModel(MapredCarbonInputFormat.java:215)
> at
> org.apache.carbondata.hive.MapredCarbonInputFormat.getRecordReader(MapredCarbonInputFormat.java:205)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:411)
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
