Hi, I'm not able to build cube. I'm getting the below error. I have added the nameservices property as well. Attached is hdfs-site.xml. Please let me know what is the issue? Here are the logs
[pool-7-thread-2]:[2015-05-26 18:14:46,899][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:222)] - job id:6031ba67-e270-4295-9fa5-cfe3126de275 from READY to RUNNING [pool-7-thread-2]:[2015-05-26 18:14:53,354][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:171)] - Saving resource /execute_output/6031ba67-e270-4295-9fa5-cfe3126de275-01 (Store kylin_metadata@hbase) [pool-7-thread-2]:[2015-05-26 18:14:53,620][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:222)] - job id:6031ba67-e270-4295-9fa5-cfe3126de275-01 from READY to RUNNING [pool-7-thread-2]:[2015-05-26 18:14:55,197][INFO][org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:107)] - parameters of the MapReduceExecutable: [pool-7-thread-2]:[2015-05-26 18:14:55,197][INFO][org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:108)] - -conf C:/Users/mobhatna/Kylin/server/../examples/test_case_data/sandbox/kylin_job_conf.xml -cubename new_star -output /tmp/kylin-6031ba67-e270-4295-9fa5-cfe3126de275/new_star/fact_distinct_columns -jobname Kylin_Fact_Distinct_Columns_new_star_Step -tablename kylin_intermediate_new_star_19700101000000_2922789940817071255_6031ba67_e270_4295_9fa5_cfe3126de275 [pool-7-thread-2]:[2015-05-26 18:14:55,245][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:149)] - append job jar: C:\Users\mobhatna\Kylin\lib\kylin-job-0.7.1-incubating-SNAPSHOT.jar [pool-7-thread-2]:[2015-05-26 18:14:55,245][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:155)] - append kylin.hive.dependency: null to mapreduce.application.classpath [pool-7-thread-2]:[2015-05-26 18:14:55,245][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:170)] - Hadoop job classpath is: $HADOOP_CONF_DIR,/usr/hdp/current/hive-client/conf/,/usr/hdp/2.2.0.0-2041/hive/lib/hive-metastore.jar,/usr/hdp/2.2.0.0-2041/hive/lib/hive-exec.jar,/usr/hdp/2.2.0.0-2041/hive-hcatalog/share/hcatalog/*,$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*,$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*,$PWD/mr-framework/hadoop/share/hadoop/common/*,$PWD/mr-framework/hadoop/share/hadoop/common/lib/*,$PWD/mr-framework/hadoop/share/hadoop/yarn/*,$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*,$PWD/mr-framework/hadoop/share/hadoop/hdfs/*,$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*,/usr/hdp/2.2.0.0-2041/hadoop/lib/hadoop-lzo-0.6.0.2.2.0.0-2041.jar,/etc/hadoop/conf/secure [pool-7-thread-2]:[2015-05-26 18:14:55,386][WARN][org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2472)] - HiveConf of name hive.optimize.mapjoin.mapreduce does not exist [pool-7-thread-2]:[2015-05-26 18:14:55,386][WARN][org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2472)] - HiveConf of name hive.heapsize does not exist [pool-7-thread-2]:[2015-05-26 18:14:55,386][WARN][org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2472)] - HiveConf of name hive.server2.enable.impersonation does not exist [pool-7-thread-2]:[2015-05-26 18:14:55,386][WARN][org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2472)] - HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask does not exist [pool-7-thread-2]:[2015-05-26 18:14:55,432][INFO][org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:319)] - Trying to connect to metastore with URI thrift://sj1dra050.corp.adobe.com:9083 [pool-7-thread-2]:[2015-05-26 18:14:55,713][WARN][org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1492)] - No groups available for user mobhatna [pool-7-thread-2]:[2015-05-26 18:14:55,978][INFO][org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:410)] - Connected to metastore. [pool-6-thread-1]:[2015-05-26 18:14:56,246][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:117)] - Job Fetcher: 0 running, 1 actual running, 1 ready, 12 others [pool-7-thread-2]:[2015-05-26 18:14:56,621][WARN][org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2472)] - HiveConf of name hive.optimize.mapjoin.mapreduce does not exist [pool-7-thread-2]:[2015-05-26 18:14:56,621][WARN][org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2472)] - HiveConf of name hive.heapsize does not exist [pool-7-thread-2]:[2015-05-26 18:14:56,621][WARN][org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2472)] - HiveConf of name hive.server2.enable.impersonation does not exist [pool-7-thread-2]:[2015-05-26 18:14:56,621][WARN][org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:2472)] - HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask does not exist [pool-7-thread-2]:[2015-05-26 18:14:57,245][INFO][org.apache.kylin.common.KylinConfig.createInstanceFromUri(KylinConfig.java:197)] - The URI C:\Users\mobhatna\AppData\Local\Temp\kylin_job_meta2280199537325480270\meta is recognized as LOCAL_FOLDER [pool-7-thread-2]:[2015-05-26 18:14:57,245][INFO][org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:67)] - Using metadata url C:\Users\mobhatna\AppData\Local\Temp\kylin_job_meta2280199537325480270\meta for resource store [pool-7-thread-2]:[2015-05-26 18:14:59,355][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:160)] - Saving resource /cube/new_star.json (Store C:\Users\mobhatna\AppData\Local\Temp\kylin_job_meta2280199537325480270\meta) [pool-7-thread-2]:[2015-05-26 18:15:00,948][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:160)] - Saving resource /model_desc/new_star.json (Store C:\Users\mobhatna\AppData\Local\Temp\kylin_job_meta2280199537325480270\meta) [pool-7-thread-2]:[2015-05-26 18:15:02,557][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:160)] - Saving resource /cube_desc/new_star.json (Store C:\Users\mobhatna\AppData\Local\Temp\kylin_job_meta2280199537325480270\meta) [pool-7-thread-2]:[2015-05-26 18:15:04,734][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:160)] - Saving resource /table/CC_STAR.SESSION.json (Store C:\Users\mobhatna\AppData\Local\Temp\kylin_job_meta2280199537325480270\meta) [pool-7-thread-2]:[2015-05-26 18:15:05,033][INFO][org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.serviceInit(TimelineClientImpl.java:285)] - Timeline service address: http://sj1dra051.corp.adobe.com:8188/ws/v1/timeline/ [pool-7-thread-2]:[2015-05-26 18:15:05,142][INFO][org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:98)] - Connecting to ResourceManager at sj1dra051.corp.adobe.com/10.5.132.174:8021 [pool-7-thread-2]:[2015-05-26 18:15:30,638][INFO][org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:545)] - Cleaning up the staging area /user/mobhatna/.staging/job_1431648296030_22357 [pool-7-thread-2]:[2015-05-26 18:15:30,904][ERROR][org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:83)] - error in FactDistinctColumnsJob java.lang.IllegalArgumentException: java.net.UnknownHostException: nameservice1 at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:374) at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:312) at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:178) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:665) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:601) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:148) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) at org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat.setInputPath(HCatBaseInputFormat.java:318) at org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat.getSplits(HCatBaseInputFormat.java:125) at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:597) at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:614) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:492) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293) at org.apache.kylin.job.hadoop.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:123) at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:80) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:112) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.UnknownHostException: nameservice1 ... 35 more [pool-7-thread-2]:[2015-05-26 18:15:32,481][ERROR][org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:115)] - error execute MapReduceExecutable{id=6031ba67-e270-4295-9fa5-cfe3126de275-01, name=Extract Fact Table Distinct Columns, state=RUNNING} java.lang.IllegalArgumentException: java.net.UnknownHostException: nameservice1 at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:374) at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:312) at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:178) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:665) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:601) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:148) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) at org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat.setInputPath(HCatBaseInputFormat.java:318) at org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat.getSplits(HCatBaseInputFormat.java:125) at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:597) at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:614) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:492) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293) at org.apache.kylin.job.hadoop.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:123) at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:80) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:112) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.UnknownHostException: nameservice1 ... 35 more [pool-7-thread-2]:[2015-05-26 18:15:34,077][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:171)] - Saving resource /execute_output/6031ba67-e270-4295-9fa5-cfe3126de275-01 (Store kylin_metadata@hbase) [pool-7-thread-2]:[2015-05-26 18:15:37,500][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:171)] - Saving resource /execute_output/6031ba67-e270-4295-9fa5-cfe3126de275-01 (Store kylin_metadata@hbase) [pool-7-thread-2]:[2015-05-26 18:15:37,766][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:222)] - job id:6031ba67-e270-4295-9fa5-cfe3126de275-01 from RUNNING to ERROR [pool-7-thread-2]:[2015-05-26 18:15:42,516][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:171)] - Saving resource /execute_output/6031ba67-e270-4295-9fa5-cfe3126de275 (Store kylin_metadata@hbase) [pool-7-thread-2]:[2015-05-26 18:15:45,925][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:171)] - Saving resource /execute_output/6031ba67-e270-4295-9fa5-cfe3126de275 (Store kylin_metadata@hbase) [pool-7-thread-2]:[2015-05-26 18:15:47,783][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:171)] - Saving resource /execute_output/6031ba67-e270-4295-9fa5-cfe3126de275 (Store kylin_metadata@hbase) [pool-7-thread-2]:[2015-05-26 18:15:48,049][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:222)] - job id:6031ba67-e270-4295-9fa5-cfe3126de275 from RUNNING to ERROR [pool-6-thread-1]:[2015-05-26 18:15:50,542][INFO][org.apache.kylin.job.impl.threadpool.DefaultScheduler$FetcherRunner.run(DefaultScheduler.java:117)] - Job Fetcher: 1 running, 1 actual running, 0 ready, 12 others [pool-7-thread-2]:[2015-05-26 18:15:57,503][ERROR][org.apache.kylin.job.execution.AbstractExecutable.notifyUserStatusChange(AbstractExecutable.java:221)] - java.lang.NullPointerException at java.util.regex.Matcher.appendReplacement(Matcher.java:758) at java.util.regex.Matcher.replaceAll(Matcher.java:906) at java.lang.String.replaceAll(String.java:2162) at org.apache.kylin.job.cube.CubingJob.formatNotifications(CubingJob.java:92) at org.apache.kylin.job.execution.AbstractExecutable.notifyUserStatusChange(AbstractExecutable.java:211) at org.apache.kylin.job.execution.DefaultChainedExecutable.onExecuteFinished(DefaultChainedExecutable.java:100) at org.apache.kylin.job.cube.CubingJob.onExecuteFinished(CubingJob.java:118) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Thanks, Mohit
hdfs-site.xml
Description: hdfs-site.xml
