Am testing out Kyline with a very basic hive table Kyline - apache-kylin-1.3-HBase-1.1-SNAPSHOT-bin Hbase - 1.1.2 Hadoop - 2.7.1 Hive 1.2.1
Sandbox all local setup. Build Cube Step#1 works well Step#2 fails #2 Step Name: Extract Fact Table Distinct Columns -conf /opt/kylin/conf/kylin_job_conf.xml -cubename sample -output /kylin/kylin_metadata/kylin-6493142b-66a9-4601-bc2f-03324fee1f6d/sample/fact_distinct_columns -jobname Kylin_Fact_Distinct_Columns_sample_Step -tablename default.kylin_intermediate_sample_19700101000000_2922789940817071255_6493142b_66a9_4601_bc2f_03324fee1f6d Exception: null I dont see any job being submitted by kylin on my hadoop cluster either. http://localhost:8088/cluster/apps I suspect i am missing some param in kylin properties to point kylin to the hadoop server/ resource manager ? kylin.log 2016-01-07 16:18:38,078 INFO [pool-20-thread-1] output.FileOutputCommitter: Saved output of task 'attempt_local1628663594_0002_r_000000_0' to hdfs://HOSTXXXX:9000/kylin/kylin_metadata/kylin-6493142b-66a9-4601-bc2f-03324fee1f6d/sample/fact_distinct_columns/_temporary/0/task_local1628663594_0002_r_000000 2016-01-07 16:18:38,079 INFO [pool-20-thread-1] mapred.LocalJobRunner: reduce > reduce 2016-01-07 16:18:38,079 INFO [pool-20-thread-1] mapred.Task: Task 'attempt_local1628663594_0002_r_000000_0' done. 2016-01-07 16:18:38,080 INFO [pool-20-thread-1] mapred.LocalJobRunner: Finishing task: attempt_local1628663594_0002_r_000000_0 [pool-7-thread-6]:[2016-01-07 16:18:38,080][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)] - Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended. [pool-7-thread-6]:[2016-01-07 16:18:38,081][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:110)] - Job job_local1628663594_0002 get status check result. 2016-01-07 16:18:38,082 INFO [Thread-42] mapred.LocalJobRunner: reduce task executor complete. *[pool-7-thread-6]:[2016-01-07 16:18:38,098][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)] - error check status* *java.lang.NullPointerException* at org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:58) at org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56) at org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:144) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) [pool-7-thread-6]:[2016-01-07 16:18:38,100][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] - Saving resource /execute_output/6493142b-66a9-4601-bc2f-03324fee1f6d-01 (Store /opt/apache-kylin-1.3-HBase-1.1-SNAPSHOT-bin/meta) [pool-7-thread-6]:[2016-01-07 16:18:38,108][DEBUG][org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutput.java:94)] - Counters: 38 File System Counters FILE: Number of bytes read=1309954 FILE: Number of bytes written=3045158 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=488 HDFS: Number of bytes written=150 HDFS: Number of read operations=61 HDFS: Number of large read operations=0 HDFS: Number of write operations=17 Map-Reduce Framework Map input records=3 Map output records=3 Map output bytes=20 Map output materialized bytes=42 Input split bytes=1640 Combine input records=3 Combine output records=3 Reduce input groups=1 Reduce shuffle bytes=42 Reduce input records=3 Reduce output records=0 Spilled Records=6 Shuffled Maps =1 Failed Shuffles=0 Merged Map outputs=1 GC time elapsed (ms)=0 CPU time spent (ms)=0 Physical memory (bytes) snapshot=0 Virtual memory (bytes) snapshot=0 Total committed heap usage (bytes)=2008023040 Shuffle Errors BAD_ID=0 CONNECTION=0 IO_ERROR=0 WRONG_LENGTH=0 WRONG_MAP=0 WRONG_REDUCE=0 File Input Format Counters Bytes Read=0 File Output Format Counters Bytes Written=128 [pool-7-thread-6]:[2016-01-07 16:18:38,110][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] - Saving resource /execute_output/6493142b-66a9-4601-bc2f-03324fee1f6d-01 (Store /opt/apache-kylin-1.3-HBase-1.1-SNAPSHOT-bin/meta) [pool-7-thread-6]:[2016-01-07 16:18:38,114][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] - Saving resource /execute_output/6493142b-66a9-4601-bc2f-03324fee1f6d-01 (Store /opt/apache-kylin-1.3-HBase-1.1-SNAPSHOT-bin/meta) [pool-7-thread-6]:[2016-01-07 16:18:38,120][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] - Saving resource /execute_output/6493142b-66a9-4601-bc2f-03324fee1f6d-01 (Store /opt/apache-kylin-1.3-HBase-1.1-SNAPSHOT-bin/meta) [pool-7-thread-6]:[2016-01-07 16:18:38,121][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:241)] - job id:6493142b-66a9-4601-bc2f-03324fee1f6d-01 from RUNNING to ERROR [pool-7-thread-6]:[2016-01-07 16:18:38,124][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] - Saving resource /execute_output/6493142b-66a9-4601-bc2f-03324fee1f6d (Store /opt/apache-kylin-1.3-HBase-1.1-SNAPSHOT-bin/meta) [pool-7-thread-6]:[2016-01-07 16:18:38,128][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] - Saving resource /execute_output/6493142b-66a9-4601-bc2f-03324fee1f6d (Store /opt/apache-kylin-1.3-HBase-1.1-SNAPSHOT-bin/meta) [pool-7-thread-6]:[2016-01-07 16:18:38,131][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:200)] - Saving resource /execute_output/6493142b-66a9-4601-bc2f-03324fee1f6d (Store /opt/apache-kylin-1.3-HBase-1.1-SNAPSHOT-bin/meta) [pool-7-thread-6]:[2016-01-07 16:18:38,132][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:241)] - job id:6493142b-66a9-4601-bc2f-03324fee1f6d from RUNNING to ERROR [pool-6-thread-1]:[2016-01-07 16:18:38,136][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:112)] - Job Fetcher: 0 running, 0 actual running, 0 ready, 3 others
