RE: remote hbase cluster
Can you sure that the local Hadoop cluster can use the hdfs://kylin to read/write the hdfs of hbase cluster? -Original Message- From: Tao Li [mailto:tao...@envisioncn.com] Sent: Friday, May 27, 2016 3:31 PM To: dev@kylin.apache.org Subject: 答复: remote hbase cluster The follow is the config: kylin.hbase.cluster.fs=hdfs://kylin hbase-site.xml hbase.rootdir hdfs://kylin/hbase -邮件原件- 发件人: Yapu Jia [mailto:yapu...@microsoft.com] 发送时间: 2016年5月27日 14:57 收件人: dev@kylin.apache.org 主题: RE: remote hbase cluster What's details config about the remote hbase cluster? It seems that the kylin.hbase.cluster.fs is not right. Thanks, Yapu -Original Message- From: Tao Li [mailto:tao...@envisioncn.com] Sent: Friday, May 27, 2016 2:13 PM To: dev@kylin.apache.org Subject: remote hbase cluster Hi, I want to use local hadoop cluster with the remote hbase cluster, and I get an exception: Caused by: java.lang.IllegalArgumentException: java.net.UnknownHostException: What should I do ? Best regards, Tao Li ???(,? This email message (including any attachments) is confidential and may be legally privileged. If you have received it by mistake, please notify the sender by return email and delete this message from your system. Any unauthorized use or dissemination of this message in whole or in part is strictly prohibited. Envision Energy Limited and all its subsidiaries shall not be liable for the improper or incomplete transmission of the information contained in this email nor for any delay in its receipt or damage to your system. Envision Energy Limited does not guarantee the integrity of this email message, nor that this email message is free of viruses, interceptions, or interference. 本邮件(包括任何附件)内容机密并受法律保护。如果您意外地收到这封邮件,请回复通知发件人并从当前系统中删除本邮件。任何未经授权者,严禁使用并部分或者全部的转发本条信息。任何未经授权的使用或传播都是被严格禁止的。远景能源与其分公司不对因不正确和不完整的转发此邮件包含的信息以及因任何因邮件延迟或对你的系统造成的损害而负责。远景能源不能保证此邮件的真实完整性,也不能确定此邮件是否含有病毒或者监听程序。 This email message (including any attachments) is confidential and may be legally privileged. If you have received it by mistake, please notify the sender by return email and delete this message from your system. Any unauthorized use or dissemination of this message in whole or in part is strictly prohibited. Envision Energy Limited and all its subsidiaries shall not be liable for the improper or incomplete transmission of the information contained in this email nor for any delay in its receipt or damage to your system. Envision Energy Limited does not guarantee the integrity of this email message, nor that this email message is free of viruses, interceptions, or interference.
RE: remote hbase cluster
What's details config about the remote hbase cluster? It seems that the kylin.hbase.cluster.fs is not right. Thanks, Yapu -Original Message- From: Tao Li [mailto:tao...@envisioncn.com] Sent: Friday, May 27, 2016 2:13 PM To: dev@kylin.apache.org Subject: remote hbase cluster Hi, I want to use local hadoop cluster with the remote hbase cluster, and I get an exception: Caused by: java.lang.IllegalArgumentException: java.net.UnknownHostException: What should I do ? Best regards, Tao Li ???(,? This email message (including any attachments) is confidential and may be legally privileged. If you have received it by mistake, please notify the sender by return email and delete this message from your system. Any unauthorized use or dissemination of this message in whole or in part is strictly prohibited. Envision Energy Limited and all its subsidiaries shall not be liable for the improper or incomplete transmission of the information contained in this email nor for any delay in its receipt or damage to your system. Envision Energy Limited does not guarantee the integrity of this email message, nor that this email message is free of viruses, interceptions, or interference.
RE: RE: RE: kylin question
Which step you get this exception? Can you get more details information? And, I recommend use Ambari HDP to deploy your environment. Thanks. -Original Message- From: 水。。。海 [mailto:549066...@qq.com] Sent: Friday, May 20, 2016 4:48 PM To: dev <dev@kylin.apache.org> Subject: Re: RE: RE: kylin question jackson version in hadoop is 1.8.8, I try 1.8.8 and 1.9.13 ,all have problems -- Original -- From: "Yapu Jia";<yapu...@microsoft.com>; Date: Fri, May 20, 2016 04:41 PM To: "dev@kylin.apache.org"<dev@kylin.apache.org>; Subject: RE: RE: kylin question Hi, You can check your jackson version. I think that if your jackson version not match your Hadoop verison, it would cause this exception. Thanks. -Original Message- From: 水。。。海 [mailto:549066...@qq.com] Sent: Friday, May 20, 2016 4:29 PM To: dev <dev@kylin.apache.org> Subject: Re: RE: kylin question I put the Hadoop-yarn* jars to $KYLIN_HOME/lib ,resolved the exception,but I have other exception like this java.lang.NoSuchMethodError: org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper; at org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59) at org.apache.hadoop.yarn.util.timeline.TimelineUtils.(TimelineUtils.java:47) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:151) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:94) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.mapred.ResourceMgrDelegate.(ResourceMgrDelegate.java:88) at org.apache.hadoop.mapred.YARNRunner.(YARNRunner.java:111) at org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34) at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95) at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:82) at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:75) at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1255) at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1251) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) at org.apache.hadoop.mapreduce.Job.connect(Job.java:1250) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1279) at org.apache.kylin.engine.mr.common.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:147) at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:96) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:118) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) and I put jackson-* 1.9.13 jars to $KYLIN_HOME/bin,still have exception Thanks! ------ Original -- From: "Yapu Jia";<yapu...@microsoft.com>; Date: Fri, May 20, 2016 02:56 PM To: "dev@kylin.apache.org"<dev@kylin.apache.org>; Subject: RE: kylin question I think this issue maybe caused by that your mapred classpath not have Hadoop-yarn* jars. Thanks. -Original Message- From: 水。。。海 [mailto:549066...@qq.com] Sent: Friday, May 20, 2016 2:28 PM To: dev <dev@kylin.apache.org> Subject: kylin question Hi: When I Quick play with a sample cube get some exception like this org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job.exception.ExecuteException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.yarn.util.timeline.TimelineUtils at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(Defaul
RE: RE: kylin question
Hi, You can check your jackson version. I think that if your jackson version not match your Hadoop verison, it would cause this exception. Thanks. -Original Message- From: 水。。。海 [mailto:549066...@qq.com] Sent: Friday, May 20, 2016 4:29 PM To: dev <dev@kylin.apache.org> Subject: Re: RE: kylin question I put the Hadoop-yarn* jars to $KYLIN_HOME/lib ,resolved the exception,but I have other exception like this java.lang.NoSuchMethodError: org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper; at org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59) at org.apache.hadoop.yarn.util.timeline.TimelineUtils.(TimelineUtils.java:47) at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:151) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:94) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.mapred.ResourceMgrDelegate.(ResourceMgrDelegate.java:88) at org.apache.hadoop.mapred.YARNRunner.(YARNRunner.java:111) at org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34) at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95) at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:82) at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:75) at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1255) at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1251) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) at org.apache.hadoop.mapreduce.Job.connect(Job.java:1250) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1279) at org.apache.kylin.engine.mr.common.AbstractHadoopJob.waitForCompletion(AbstractHadoopJob.java:147) at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:96) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:118) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) and I put jackson-* 1.9.13 jars to $KYLIN_HOME/bin,still have exception Thanks! -- Original -- From: "Yapu Jia";<yapu...@microsoft.com>; Date: Fri, May 20, 2016 02:56 PM To: "dev@kylin.apache.org"<dev@kylin.apache.org>; Subject: RE: kylin question I think this issue maybe caused by that your mapred classpath not have Hadoop-yarn* jars. Thanks. -Original Message- From: 水。。。海 [mailto:549066...@qq.com] Sent: Friday, May 20, 2016 2:28 PM To: dev <dev@kylin.apache.org> Subject: kylin question Hi: When I Quick play with a sample cube get some exception like this org.apache.kylin.job.exception.ExecuteException: org.apache.kylin.job.exception.ExecuteException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.yarn.util.timeline.TimelineUtils at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.kylin.job.exception.ExecuteException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.yarn.util.timeline.TimelineUtils at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124) at org.apache.kylin.job.execution.DefaultChainedExecut
RE: why can't I see the project which is created by other kylin server when I visit a kylin server ?
I think your kylin server not share one metadata. Maybe you kylin configuration have some problems. -Original Message- From: 杨海乐 [mailto:yangha...@letv.com] Sent: Tuesday, May 17, 2016 3:53 PM To: dev@kylin.apache.org Subject: why can't I see the project which is created by other kylin server when I visit a kylin server ? Hello all, After creating a project in a kylin server,I visit other kylin serevr and can't see the project even if I clear browser cache。If I click "reload metadata",It work. I want to know why and how to auto update the metadata。 thanks! -- View this message in context: https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fapache-kylin.74782.x6.nabble.com%2fwhy-can-t-I-see-the-project-which-is-created-by-other-kylin-server-when-I-visit-a-kylin-server-tp4575.html=01%7c01%7cyapujia%40064d.mgd.microsoft.com%7c66d2e46d2da94efa3b9708d37e2a52ba%7c72f988bf86f141af91ab2d7cd011db47%7c1=JKwP0IKDjlfw2RwugpTxQcrXp%2fcqc3eUPJaTGkFmTYE%3d Sent from the Apache Kylin mailing list archive at Nabble.com.
RE: Kylin configuration requirements
Please refer to : http://kylin.apache.org/docs15/install/kylin_cluster.html -Original Message- From: yaoxiao...@huan.tv [mailto:yaoxiao...@huan.tv] Sent: Monday, May 16, 2016 11:11 AM To: devSubject: Kylin configuration requirements Hello , the 600 million data with kylin, four machines (3 slave node), need what configuration. yaoxiao...@huan.tv
RE: expansion rate so high
Which kylin version do you use? The latest version have a good improvement in data expansion. -Original Message- From: Peng [mailto:pengli0...@outlook.com] Sent: Monday, May 16, 2016 10:21 AM To: dev@kylin.apache.org Subject: expansion rate so high Hi, Is it normal when the expansion rate is about 1600% ? My cube : about one hundred million data; fact table has one lookup table; 5 dimensions , in which 4 dimensions' encoding are fixed length, separately the length are 9, 8, 8, 33 ; 2 measures, finally the expansion rate is about 1600. Thanks Peng
RE: Can't build and refresh job
I had checked it in the job list. It not have any running or error job for the cube. If there have any running or error job about this cube. I think I can't use others Kylin server to build and refresh it. In this case, it may be a Kylin bug. -Original Message- From: ShaoFeng Shi [mailto:shaofeng...@apache.org] Sent: Friday, May 13, 2016 10:27 PM To: dev@kylin.apache.org Subject: Re: Can't build and refresh job Please check whether you have a running or error job for this cube; If true, you need discard it first; only after that you can start another build job, otherwise you will get "There is already a building segment" error. 2016-05-13 19:54 GMT+08:00 Yapu Jia <yapu...@microsoft.com>: > Hi, all, > > > > I can build and refresh a cube successful in the past. > > After a segment build success, I can’t refresh or build the > cube and query this segment data. > > The web tip is that: > > > > > > And, the kylin.log is that: > > > > java.lang.IllegalStateException: There is already a > building segment! > > at > org.apache.kylin.cube.CubeManager.checkNoBuildingSegment(CubeManager.j > ava:541) > > at > org.apache.kylin.cube.CubeManager.appendSegments(CubeManager.java:438) > > at > org.apache.kylin.cube.CubeManager.appendSegments(CubeManager.java:433) > > at > org.apache.kylin.cube.CubeManager.appendSegments(CubeManager.java:418) > > at > org.apache.kylin.rest.service.JobService.submitJob(JobService.java:208 > ) > > at > org.apache.kylin.rest.service.JobService$$FastClassByCGLIB$$83a44b2a.i > nvoke() > > at > net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > > at > org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation > .invokeJoinpoint(Cglib2AopProxy.java:689) > > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(R > eflectiveMethodInvocation.java:150) > > at > org.springframework.security.access.intercept.aopalliance.MethodSecuri > tyInterceptor.invoke(MethodSecurityInterceptor.java:64) > > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(R > eflectiveMethodInvocation.java:172) > > at > org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterce > ptor.intercept(Cglib2AopProxy.java:622) > > at > org.apache.kylin.rest.service.JobService$$EnhancerByCGLIB$$bfa162cd.su > bmitJob() > > at > org.apache.kylin.rest.controller.CubeController.rebuild(CubeController > .java:251) > > at > sun.reflect.GeneratedMethodAccessor37.invoke(Unknown > Source) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess > orImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:606) > > at > org.springframework.web.method.support.InvocableHandlerMethod.invoke(I > nvocableHandlerMethod.java:213) > > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeFo > rRequest(InvocableHandlerMethod.java:126) > > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocable > HandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96) > > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHa > ndlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617 > ) > > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHa > ndlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578) > > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapte > r.handle(AbstractHandlerMethodAdapter.java:80) > > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(Dispatche > rServlet.java:923) > > at > org.springframework.web.servlet.DispatcherServlet.doService(Dispatcher > Servlet.java:852) > > at > org.springframework.web.servlet.FrameworkServlet.processRequest(Framew > orkServlet.java:882) > > at > org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServle > t.java:800) > > at > javax.servlet.http.HttpServlet.service(HttpServlet.java:649) > > at > javax.servlet.http.HttpServlet.service(HttpServlet.java:727) > > > > I can use others kylin server to build or refresh this cube. > > And, when I restart this kylin server, I can refresh and > build this cube. So, I think that maybe a bug. > > Can you help me to fix it. > > > > Thanks, > > Yapu > > > -- Best regards, Shaofeng Shi
problem about build development env
Hi, I use the Hortonworks Sandbox 2.2.4 to bulid my development env. I refer to the offical doc about development env and success in step(mvn clean install -DskipTests). But, Failure in step(Run unit trests), include: mvn test -Dtest=org.apache.kylin.job.BuildCubeWithEngineTest -DfailIfNoTests=false -Dhdp.version=2.2 -P sandbox mvn test -Dtest=org.apache.kylin.job.BuildIIWithEngineTest -DfailIfNoTests=false -Dhdp.version=2.2 -P sandbox The failure info is [INFO] [INFO] Reactor Summary: [INFO] [INFO] Kylin:HadoopOLAPEngine . SUCCESS [ 4.458 s] [INFO] Kylin:AtopCalcite .. SUCCESS [ 5.050 s] [INFO] Kylin:Common ... SUCCESS [ 21.077 s] [INFO] Kylin:Metadata . SUCCESS [ 3.027 s] [INFO] Kylin:Dictionary ... SUCCESS [ 2.193 s] [INFO] Kylin:Cube . SUCCESS [ 6.438 s] [INFO] Kylin:InvertedIndex SUCCESS [ 0.966 s] [INFO] Kylin:Job .. FAILURE [01:02 min] [INFO] Kylin:Storage .. SKIPPED [INFO] Kylin:Query SKIPPED [INFO] Kylin:JDBC . SKIPPED [INFO] Kylin:RESTServer ... SKIPPED [INFO] Kylin:Monitor .. SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 01:48 min [INFO] Finished at: 2016-02-26T02:39:12+00:00 [INFO] Final Memory: 40M/722M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on project kylin-job: There are test failures. [ERROR] [ERROR] Please refer to /root/kylin/job/../target/surefire-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :kylin-job Therefore, I don't kown how fix it. I skip the step (Run unit trests) and go to next step (lunch Kylin Web Server). There have a problem is that I can't lunch org.apache.kylin.rest.DebugTomcat . The error is "Selection does not contain a main type". So, how to fix these problem and build development env about Kylin. Thanks, Yapu