Re: Debug kylin2.6X with CDH5.15.It didn't build cube.
Hi
You can check the maven profile and will find it uses "hdp" profile, and
there exists "cdh" profile, but it still has some work to do as I used to
try. As Chao Long said, we recommend use HDP sandbox to debug.
--
Regards!
Aron Tao
Lio_Messi 于2019年3月28日周四 下午12:11写道：
> I want to debug kylin by runing DebugTomcat with CDH5.15.When I built a
> cube,the following error appears:
>
> 2019-03-28 19:07:25,957 INFO [BadQueryDetector]
> service.BadQueryDetector:147 : Detect bad query.
> 2019-03-28 19:07:28,935 INFO [FetcherRunner 1204410668-166]
> threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1
> actual
> running, 0 stopped, 0 ready, 3826 already succeed, 6 error, 4 discarded, 0
> others
> 2019-03-28 19:07:34,140 INFO [Scheduler 154250424 Job
> e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] mapred.ClientServiceDelegate:276
> :
> Application state is completed. FinalApplicationStatus=FAILED. Redirecting
> to job history server
> 2019-03-28 19:07:34,163 ERROR [Scheduler 154250424 Job
> e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] common.HadoopJobStatusChecker:58
> :
> error check status
> java.io.IOException: Job status not available
> at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:334)
> at org.apache.hadoop.mapreduce.Job.getStatus(Job.java:341)
> at
> org.apache.kylin.engine.mr
> .common.HadoopJobStatusChecker.checkStatus(HadoopJobStatusChecker.java:38)
> at
> org.apache.kylin.engine.mr
> .common.MapReduceExecutable.doWork(MapReduceExecutable.java:153)
> at
>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
> at
>
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
> at
>
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-03-28 19:07:34,166 ERROR [Scheduler 154250424 Job
> e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] common.MapReduceExecutable:198 :
> error execute
> MapReduceExecutable{id=e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-01,
> name=Extract
> Fact Table Distinct Columns, state=RUNNING}
> java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.Job.getTrackingURL(Job.java:380)
> at
> org.apache.kylin.engine.mr
> .common.HadoopCmdOutput.getInfo(HadoopCmdOutput.java:66)
> at
> org.apache.kylin.engine.mr
> .common.MapReduceExecutable.doWork(MapReduceExecutable.java:163)
> at
>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
> at
>
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
> at
>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
> at
>
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-03-28 19:07:34,176 INFO [Scheduler 154250424 Job
> e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] execution.ExecutableManager:453 :
> job id:e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-01 from RUNNING to ERROR
> 2019-03-28 19:07:34,178 ERROR [Scheduler 154250424 Job
> e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] execution.AbstractExecutable:168
> :
> error running Executable:
> CubingJob{id=e3e62e0e-daf8-e0bb-762d-81bcbddc55ae,
> name=BUILD CUBE - first_cube - 2015010100_2015060100 - GMT+08:00
> 2019-03-27 19:35:43, state=RUNNING}
> 2019-03-28 19:07:34,181 DEBUG [pool-6-thread-1] cachesync.Broadcaster:116 :
> Servers in the cluster: [localhost:7070]
> 2019-03-28 19:07:34,181 DEBUG [pool-6-thread-1] cachesync.Broadcaster:126 :
> Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
> event=update, cacheKey=e3e62e0e-daf8-e0bb-762d-81bcbddc55ae}
> 2019-03-28 19:07:34,185 DEBUG [http-bio-7070-exec-8]
> cachesync.Broadcaster:246 : Broadcasting UPDATE, execute_output,
> e3e62e0e-daf8-e0bb-762d-81bcbddc55ae
> 2019-03-28 19:07:34,185 DEBUG [pool-6-thread-1] cachesync.Broadcaster:116 :
> Servers in the cluster: [localhost:7070]
> 2019-03-28 19:07:34,185 INFO [Scheduler 154250424 Job
> e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] execution.ExecutableManager:453 :
> job id:e3e62e0e-daf8-e0bb-762d-81bcbddc55ae from RUNNING to ERROR
> 2019-03-28 19:07:34,185 DEBUG [pool-6-thread-1] cachesync.Broadcaster:126 :
> Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
>
Re: Debug kylin2.6X with CDH5.15.It didn't build cube.
Hi Lio_Messi,
We usually debug Kylin with hdp sandbox, which is also officially
recommended[http://kylin.apache.org/cn/development/dev_env.html]. Debug with
cdh is not supported yet, maybe you can make it work.
--
Best Regards,
Chao Long
-- Original --
From: "Lio_Messi";
Date: Thu, Mar 28, 2019 08:11 PM
To: "dev";
Subject: Debug kylin2.6X with CDH5.15.It didn't build cube.
I want to debug kylin by runing DebugTomcat with CDH5.15.When I built a
cube,the following error appears:
2019-03-28 19:07:25,957 INFO [BadQueryDetector]
service.BadQueryDetector:147 : Detect bad query.
2019-03-28 19:07:28,935 INFO [FetcherRunner 1204410668-166]
threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual
running, 0 stopped, 0 ready, 3826 already succeed, 6 error, 4 discarded, 0
others
2019-03-28 19:07:34,140 INFO [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] mapred.ClientServiceDelegate:276 :
Application state is completed. FinalApplicationStatus=FAILED. Redirecting
to job history server
2019-03-28 19:07:34,163 ERROR [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] common.HadoopJobStatusChecker:58 :
error check status
java.io.IOException: Job status not available
at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:334)
at org.apache.hadoop.mapreduce.Job.getStatus(Job.java:341)
at
org.apache.kylin.engine.mr.common.HadoopJobStatusChecker.checkStatus(HadoopJobStatusChecker.java:38)
at
org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:153)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-03-28 19:07:34,166 ERROR [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] common.MapReduceExecutable:198 :
error execute
MapReduceExecutable{id=e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-01, name=Extract
Fact Table Distinct Columns, state=RUNNING}
java.lang.NullPointerException
at org.apache.hadoop.mapreduce.Job.getTrackingURL(Job.java:380)
at
org.apache.kylin.engine.mr.common.HadoopCmdOutput.getInfo(HadoopCmdOutput.java:66)
at
org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:163)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-03-28 19:07:34,176 INFO [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] execution.ExecutableManager:453 :
job id:e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-01 from RUNNING to ERROR
2019-03-28 19:07:34,178 ERROR [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] execution.AbstractExecutable:168 :
error running Executable: CubingJob{id=e3e62e0e-daf8-e0bb-762d-81bcbddc55ae,
name=BUILD CUBE - first_cube - 2015010100_2015060100 - GMT+08:00
2019-03-27 19:35:43, state=RUNNING}
2019-03-28 19:07:34,181 DEBUG [pool-6-thread-1] cachesync.Broadcaster:116 :
Servers in the cluster: [localhost:7070]
2019-03-28 19:07:34,181 DEBUG [pool-6-thread-1] cachesync.Broadcaster:126 :
Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
event=update, cacheKey=e3e62e0e-daf8-e0bb-762d-81bcbddc55ae}
2019-03-28 19:07:34,185 DEBUG [http-bio-7070-exec-8]
cachesync.Broadcaster:246 : Broadcasting UPDATE, execute_output,
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae
2019-03-28 19:07:34,185 DEBUG [pool-6-thread-1] cachesync.Broadcaster:116 :
Servers in the cluster: [localhost:7070]
2019-03-28 19:07:34,185 INFO [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] execution.ExecutableManager:453 :
job id:e3e62e0e-daf8-e0bb-762d-81bcbddc55ae from RUNNING to ERROR
2019-03-28 19:07:34,185 DEBUG [pool-6-thread-1] cachesync.Broadcaster:126 :
Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
event=update, cacheKe
Debug kylin2.6X with CDH5.15.It didn't build cube.
I want to debug kylin by runing DebugTomcat with CDH5.15.When I built a
cube,the following error appears:
2019-03-28 19:07:25,957 INFO [BadQueryDetector]
service.BadQueryDetector:147 : Detect bad query.
2019-03-28 19:07:28,935 INFO [FetcherRunner 1204410668-166]
threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual
running, 0 stopped, 0 ready, 3826 already succeed, 6 error, 4 discarded, 0
others
2019-03-28 19:07:34,140 INFO [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] mapred.ClientServiceDelegate:276 :
Application state is completed. FinalApplicationStatus=FAILED. Redirecting
to job history server
2019-03-28 19:07:34,163 ERROR [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] common.HadoopJobStatusChecker:58 :
error check status
java.io.IOException: Job status not available
at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:334)
at org.apache.hadoop.mapreduce.Job.getStatus(Job.java:341)
at
org.apache.kylin.engine.mr.common.HadoopJobStatusChecker.checkStatus(HadoopJobStatusChecker.java:38)
at
org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:153)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-03-28 19:07:34,166 ERROR [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] common.MapReduceExecutable:198 :
error execute
MapReduceExecutable{id=e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-01, name=Extract
Fact Table Distinct Columns, state=RUNNING}
java.lang.NullPointerException
at org.apache.hadoop.mapreduce.Job.getTrackingURL(Job.java:380)
at
org.apache.kylin.engine.mr.common.HadoopCmdOutput.getInfo(HadoopCmdOutput.java:66)
at
org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:163)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-03-28 19:07:34,176 INFO [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] execution.ExecutableManager:453 :
job id:e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-01 from RUNNING to ERROR
2019-03-28 19:07:34,178 ERROR [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] execution.AbstractExecutable:168 :
error running Executable: CubingJob{id=e3e62e0e-daf8-e0bb-762d-81bcbddc55ae,
name=BUILD CUBE - first_cube - 2015010100_2015060100 - GMT+08:00
2019-03-27 19:35:43, state=RUNNING}
2019-03-28 19:07:34,181 DEBUG [pool-6-thread-1] cachesync.Broadcaster:116 :
Servers in the cluster: [localhost:7070]
2019-03-28 19:07:34,181 DEBUG [pool-6-thread-1] cachesync.Broadcaster:126 :
Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
event=update, cacheKey=e3e62e0e-daf8-e0bb-762d-81bcbddc55ae}
2019-03-28 19:07:34,185 DEBUG [http-bio-7070-exec-8]
cachesync.Broadcaster:246 : Broadcasting UPDATE, execute_output,
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae
2019-03-28 19:07:34,185 DEBUG [pool-6-thread-1] cachesync.Broadcaster:116 :
Servers in the cluster: [localhost:7070]
2019-03-28 19:07:34,185 INFO [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] execution.ExecutableManager:453 :
job id:e3e62e0e-daf8-e0bb-762d-81bcbddc55ae from RUNNING to ERROR
2019-03-28 19:07:34,185 DEBUG [pool-6-thread-1] cachesync.Broadcaster:126 :
Announcing new broadcast to all: BroadcastEvent{entity=execute_output,
event=update, cacheKey=e3e62e0e-daf8-e0bb-762d-81bcbddc55ae}
2019-03-28 19:07:34,185 DEBUG [Scheduler 154250424 Job
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae-202] execution.AbstractExecutable:319 :
no need to send email, user list is empty
2019-03-28 19:07:34,186 DEBUG [http-bio-7070-exec-8]
cachesync.Broadcaster:280 : Done broadcasting UPDATE, execute_output,
e3e62e0e-daf8-e0bb-762d-81bcbddc55ae
2019-03-28 19:07:34,188 ERROR [pool-10-thread-1]
threadpool.DefaultScheduler:116 : ExecuteExcept
