[
https://issues.apache.org/jira/browse/KYLIN-3912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nichunen updated KYLIN-3912:
----------------------------
Fix Version/s: (was: v3.0.0)
v3.0.0-alpha2
> Support cube level mapreduce queue config for BeelineHiveClient
> ---------------------------------------------------------------
>
> Key: KYLIN-3912
> URL: https://issues.apache.org/jira/browse/KYLIN-3912
> Project: Kylin
> Issue Type: Improvement
> Components: Job Engine
> Affects Versions: v2.6.1
> Reporter: Liu Shaohui
> Priority: Major
> Fix For: v3.0.0-alpha2
>
>
> To support multi tenants, we set different mapreduce queue config for
> different projects and cubes, but BeelineHiveClient don't use those configs.
> So the getHiveTableRows api always run on same queue in kylin_hive_conf or
> jdbc url, which cause computing resource competition.
>
> {code:java}
> 2018-11-28 15:37:27,261 ERROR [Scheduler 1950398337 Job
> 08b3ee43-c84d-4039-84c5-a36ecb2cff18-228] execution.AbstractExecutable:383 :
> job:08b3ee43-c84d-4039-84c5-a36ecb2cff18-01 execute finished with exception
> java.sql.SQLException: Error while processing statement: FAILED: Execution
> Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> downstreamError is null.
> Query log:
> http://zjy-hadoop-prc-ct14.bj:28911/log?qid=a05e1629-2072-46dd-9d71-b5722d04b2aa
> at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:277)
> at
> org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:376)
> at
> org.apache.kylin.source.hive.BeelineHiveClient.getHiveTableRows(BeelineHiveClient.java:108)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.computeRowCount(HiveMRInput.java:304)
> at
> org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:354)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:165)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
