Is the old Kylin instance still running? You need shutdown that as Kylin only allow one instance to run the job engine;
On 5/4/15, 3:31 PM, "Tobias Bockrath" <[email protected]> wrote: >Hi Luke, >thank your for your response. >I deployed Kylin at a single machine beside Hadoop Cluster now. I also >installed Hadoop Client and set up cluster env on that machine. >Next I switched to an account which has privileges to operate >HDFS/Hive/HBase/JobHistory and started Kylin via kylin.sh start. >But Kylin isn't starting correctly because of an "fail to acquire lock, >scheduler has not been started" Error. >Here is the content from the Kylin.log file: > >[Thread-9]:[2015-04-30 >11:14:31,210][INFO][org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:4 >38)] >- Initiating client connection, connectString=nn1.local, dn2.local, >dn3.local:2181 sessionTimeout=60000 >watcher=org.apache.curator.ConnectionState@6910d398 > >[Thread-9-SendThread(192.168.2.25:2181)]:[2015-04-30 >11:14:31,326][INFO][org.apache.zookeeper.ClientCnxn$SendThread.logStartCon >nect(ClientCnxn.java:975)] >- Opening socket connection to server 192.168.2.25/192.168.2.25:2181. Will >not attempt to authenticate using SASL (unknown error) > >[Thread-9-SendThread(192.168.2.25:2181)]:[2015-04-30 >11:14:31,328][INFO][org.apache.zookeeper.ClientCnxn$SendThread.primeConnec >tion(ClientCnxn.java:852)] >- Socket connection established to 192.168.2.25/192.168.2.25:2181, >initiating session > >[Thread-9-SendThread(192.168.2.25:2181)]:[2015-04-30 >11:14:31,343][INFO][org.apache.zookeeper.ClientCnxn$SendThread.onConnected >(ClientCnxn.java:1235)] >- Session establishment complete on server 192.168.2.25/192.168.2.25:2181, >sessionid = 0x14ccf450f87003f, negotiated timeout = 40000 > >[Thread-9-EventThread]:[2015-04-30 >11:14:31,362][INFO][org.apache.curator.framework.state.ConnectionStateMana >ger.postState(ConnectionStateManager.java:228)] >- State change: CONNECTED > >[Thread-9]:[2015-04-30 >11:14:34,251][WARN][org.apache.kylin.job.impl.threadpool.DefaultScheduler. >init(DefaultScheduler.java:205)] >- fail to acquire lock, scheduler has not been started > >[Thread-9]:[2015-04-30 >11:14:34,298][INFO][org.apache.zookeeper.ZooKeeper.close(ZooKeeper.java:68 >4)] >- Session: 0x14ccf450f87003f closed > >[Thread-9-EventThread]:[2015-04-30 >11:14:34,299][INFO][org.apache.zookeeper.ClientCnxn$EventThread.run(Client >Cnxn.java:512)] >- EventThread shut down > >[Thread-9]:[2015-04-30 >11:14:34,302][ERROR][org.apache.kylin.rest.controller.JobController$1.run( >JobController.java:87)] >- scheduler has not been started >I can't figure out why the scheduler is not starting. What does this error >mean? Does somebody had similiar Issues? > >Kind regards >Tobias > > > > >2015-04-29 15:21 GMT+02:00 Luke Han <[email protected]>: > >> There's maybe some conflict between Kylin's tomcat instance and others. >> You can install Tomcat at single machine beside Hadoop Cluster (as our >> deployment). But please make sure that machine has hadoop client >>installed >> and set correctly with your cluster env. >> The account should has privileges to operate HDFS/Hive/HBase/JobHistory >>and >> others. >> >> Please refer to this installation guide for more detail: >> >> >>https://github.com/apache/incubator-kylin/blob/master/docs/Installation/I >>nstallation.md >> >> Thanks. >> >> >> Best Regards! >> --------------------- >> >> Luke Han >> >> 2015-04-29 20:57 GMT+08:00 Tobias Bockrath <[email protected]>: >> >> > Hi Luke, >> > >> > its a CentOS release 6.6 Server with a 2665 Mhz Dualcore CPU and 16 GB >> Ram. >> > >> > I am using the binary package without changing tomcats memory >>settings. I >> > just untared kylin, set the path to kylin home and started kylin. >> > >> > I deployed it on the NameNode of a HDP cluster, so there is the whole >>HDP >> > Stack running on the the machine. I would prefer running Kylin on a >> single >> > machine, but deploying it on the Namenode seemed to be the easiest >>way to >> > get started with Kylin. Is it possible to run Kylin on a Single >>Machine >> > "outside" the HDP Cluster, where no hadoop is preinstalled? >> > >> > I figured out that there are some other tomcat processes running on >>the >> > Namenode. Maybe these processes have not been stopped correctly while >>i >> was >> > playing with Kylin a few days ago.. And maybe they are now causing the >> > memory leak Exception. >> > >> > So I think the best deployment for Kylin in our environment would be >> > running Kylin on a single machine and not on the Name- or the >>Datanode, >> but >> > i didnt found an installation guide yet. >> > >> > Kind regards >> > Tobias >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > 2015-04-29 14:10 GMT+02:00 Luke Han <[email protected]>: >> > >> > > Could you please let's know your server's spec? >> > > >> > > OS and version, CPU, Memory... >> > > >> > > Did you changed any setting about tomcat's memory? >> > > >> > > Is there any other application running on that machine? >> > > >> > > And, are you use our binary package or generated by yourself? >> > > >> > > Thanks. >> > > >> > > >> > > Best Regards! >> > > --------------------- >> > > >> > > Luke Han >> > > >> > > 2015-04-29 20:03 GMT+08:00 Tobias Bockrath <[email protected]>: >> > > >> > > > Yes I use http://host:7070/kylin/ >> > > > >> > > > The GUI with the Login screen is reachable for a few seconds, but >> then >> > > > Tomcat is crashing completely and the GUI isn't reachable any >>more. >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > 2015-04-29 11:05 GMT+02:00 Li Yang <[email protected]>: >> > > > >> > > > > The mentioned leak is about a dependency of Kylin: >> > > metrics-core-3.0.1.jar >> > > > > >> > > > > Our deployment has the same log, so it's not critical, and shall >> not >> > > > block >> > > > > GUI from start up. >> > > > > >> > > > > What url have you tried? http://host:7070/kylin/ ? >> > > > > >> > > > > On Tue, Apr 28, 2015 at 6:01 PM, Tobias Bockrath < >> > [email protected]> >> > > > > wrote: >> > > > > >> > > > > > Hello, >> > > > > > >> > > > > > I deployed Kylin 0.7.1 Snapshot in a HDP 2.2 Cluster with 1 >> > NameNode >> > > > and >> > > > > 2 >> > > > > > DataNodes. I exported Kylin Home and started Kylin via >>kylin.sh >> > > start. >> > > > > > >> > > > > > The Startup seems to be normal, but the Webinterface isn't >> > reachable. >> > > > > > >> > > > > > So I checked the Tomcat logs. In the catalina log I found >> following >> > > > > > Exception: >> > > > > > >> > > > > > >> > > > > > >> > > > > > Apr 28, 2015 10:48:18 AM >> > > org.apache.catalina.core.AprLifecycleListener >> > > > > > lifecycleEvent >> > > > > > INFORMATION: The APR based Apache Tomcat Native library which >> > allows >> > > > > > optimal performance in production environments was not found >>on >> the >> > > > > > java.library.path: >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >>:/usr/hdp/2.2.0.0-2041/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.2.0.0- >>2041/hadoop/lib/native >> > > > > > Apr 28, 2015 10:48:19 AM org.apache.coyote.AbstractProtocol >>init >> > > > > > INFORMATION: Initializing ProtocolHandler ["http-bio-7070"] >> > > > > > Apr 28, 2015 10:48:19 AM org.apache.coyote.AbstractProtocol >>init >> > > > > > INFORMATION: Initializing ProtocolHandler ["ajp-bio-9009"] >> > > > > > Apr 28, 2015 10:48:19 AM org.apache.catalina.startup.Catalina >> load >> > > > > > INFORMATION: Initialization processed in 1674 ms >> > > > > > Apr 28, 2015 10:48:19 AM >>org.apache.catalina.core.StandardService >> > > > > > startInternal >> > > > > > INFORMATION: Starting service Catalina >> > > > > > Apr 28, 2015 10:48:19 AM >>org.apache.catalina.core.StandardEngine >> > > > > > startInternal >> > > > > > INFORMATION: Starting Servlet Engine: Apache Tomcat/7.0.59 >> > > > > > Apr 28, 2015 10:48:19 AM >>org.apache.catalina.startup.HostConfig >> > > > deployWAR >> > > > > > INFORMATION: Deploying web application archive >> > > > > > /home/kylin/kylin-0.7.1/tomcat/webapps/kylin.war >> > > > > > Apr 28, 2015 10:48:20 AM >> > org.apache.catalina.loader.WebappClassLoader >> > > > > > validateJarFile >> > > > > > INFORMATION: >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >>validateJarFile(/home/kylin/kylin-0.7.1/tomcat/webapps/kylin/WEB-INF/lib/ >>servlet-api-2.5.jar) >> > > > > > - jar not loaded. See Servlet Spec 3.0, section 10.7.2. >>Offending >> > > > class: >> > > > > > javax/servlet/Servlet.class >> > > > > > Apr 28, 2015 10:48:44 AM >> org.apache.catalina.startup.TaglibUriRule >> > > body >> > > > > > INFORMATION: TLD skipped. URI: urn:com:sun:jersey:api:view is >> > already >> > > > > > defined >> > > > > > Apr 28, 2015 10:48:44 AM >> org.apache.catalina.startup.TaglibUriRule >> > > body >> > > > > > INFORMATION: TLD skipped. URI: urn:com:sun:jersey:api:view is >> > already >> > > > > > defined >> > > > > > Apr 28, 2015 10:48:44 AM >> org.apache.catalina.startup.TaglibUriRule >> > > body >> > > > > > INFORMATION: TLD skipped. URI: urn:com:sun:jersey:api:view is >> > already >> > > > > > defined >> > > > > > Apr 28, 2015 10:48:44 AM >> org.apache.catalina.startup.TaglibUriRule >> > > body >> > > > > > INFORMATION: TLD skipped. URI: urn:com:sun:jersey:api:view is >> > already >> > > > > > defined >> > > > > > Apr 28, 2015 10:48:44 AM >> org.apache.catalina.startup.TaglibUriRule >> > > body >> > > > > > INFORMATION: TLD skipped. URI: urn:com:sun:jersey:api:view is >> > already >> > > > > > defined >> > > > > > Apr 28, 2015 10:48:44 AM >> org.apache.catalina.startup.TaglibUriRule >> > > body >> > > > > > INFORMATION: TLD skipped. URI: urn:com:sun:jersey:api:view is >> > already >> > > > > > defined >> > > > > > Apr 28, 2015 10:48:44 AM >> org.apache.catalina.startup.TaglibUriRule >> > > body >> > > > > > INFORMATION: TLD skipped. URI: urn:com:sun:jersey:api:view is >> > already >> > > > > > defined >> > > > > > Apr 28, 2015 1:48:55 AM org.apache.catalina.startup.HostConfig >> > > > deployWAR >> > > > > > INFORMATION: Deployment of web application archive >> > > > > > /home/kylin/kylin-0.7.1/tomcat/webapps/kylin.war has finished >>in >> > > 36,259 >> > > > > ms >> > > > > > Apr 28, 2015 1:48:55 AM org.apache.coyote.AbstractProtocol >>start >> > > > > > INFORMATION: Starting ProtocolHandler ["http-bio-7070"] >> > > > > > Apr 28, 2015 1:48:55 AM org.apache.coyote.AbstractProtocol >>start >> > > > > > INFORMATION: Starting ProtocolHandler ["ajp-bio-9009"] >> > > > > > Apr 28, 2015 1:48:55 AM org.apache.catalina.startup.Catalina >> start >> > > > > > INFORMATION: Server startup in 36417 ms >> > > > > > Apr 28, 2015 1:48:57 AM org.apache.coyote.AbstractProtocol >>pause >> > > > > > INFORMATION: Pausing ProtocolHandler ["http-bio-7070"] >> > > > > > Apr 28, 2015 1:48:57 AM org.apache.coyote.AbstractProtocol >>pause >> > > > > > INFORMATION: Pausing ProtocolHandler ["ajp-bio-9009"] >> > > > > > Apr 28, 2015 1:48:57 AM >>org.apache.catalina.core.StandardService >> > > > > > stopInternal >> > > > > > INFORMATION: Stopping service Catalina >> > > > > > Apr 28, 2015 1:48:57 AM >> > org.apache.catalina.loader.WebappClassLoader >> > > > > > clearReferencesThreads >> > > > > > SCHWERWIEGEND: The web application [/kylin] appears to have >> > started a >> > > > > > thread named [Thread-5] but has failed to stop it. This is >>very >> > > likely >> > > > to >> > > > > > create a memory leak. >> > > > > > Apr 28, 2015 1:48:57 AM >> > org.apache.catalina.loader.WebappClassLoader >> > > > > > clearReferencesThreads >> > > > > > SCHWERWIEGEND: The web application [/kylin] appears to have >> > started a >> > > > > > thread named [Thread-9] but has failed to stop it. This is >>very >> > > likely >> > > > to >> > > > > > create a memory leak. >> > > > > > Apr 28, 2015 1:48:57 AM >> > org.apache.catalina.loader.WebappClassLoader >> > > > > > checkThreadLocalMapForLeaks >> > > > > > SCHWERWIEGEND: The web application [/kylin] created a >>ThreadLocal >> > > with >> > > > > key >> > > > > > of type [com.codahale.metrics.ThreadLocalRandom$1] (value >> > > > > > [com.codahale.metrics.ThreadLocalRandom$1@46d8c695]) and a >>value >> > of >> > > > type >> > > > > > [com.codahale.metrics.ThreadLocalRandom] (value >> > > > > > [com.codahale.metrics.ThreadLocalRandom@41d58489]) but failed >>to >> > > > remove >> > > > > it >> > > > > > when the web application was stopped. Threads are going to be >> > renewed >> > > > > over >> > > > > > time to try and avoid a probable memory leak. >> > > > > > Apr 28, 2015 1:48:57 AM >> > org.apache.catalina.loader.WebappClassLoader >> > > > > > checkThreadLocalMapForLeaks >> > > > > > SCHWERWIEGEND: The web application [/kylin] created a >>ThreadLocal >> > > with >> > > > > key >> > > > > > of type [com.codahale.metrics.ThreadLocalRandom$1] (value >> > > > > > [com.codahale.metrics.ThreadLocalRandom$1@46d8c695]) and a >>value >> > of >> > > > type >> > > > > > [com.codahale.metrics.ThreadLocalRandom] (value >> > > > > > [com.codahale.metrics.ThreadLocalRandom@28f06023]) but failed >>to >> > > > remove >> > > > > it >> > > > > > when the web application was stopped. Threads are going to be >> > renewed >> > > > > over >> > > > > > time to try and avoid a probable memory leak. >> > > > > > Apr 28, 2015 1:48:57 AM >> > org.apache.catalina.loader.WebappClassLoader >> > > > > > checkThreadLocalMapForLeaks >> > > > > > SCHWERWIEGEND: The web application [/kylin] created a >>ThreadLocal >> > > with >> > > > > key >> > > > > > of type [com.codahale.metrics.ThreadLocalRandom$1] (value >> > > > > > [com.codahale.metrics.ThreadLocalRandom$1@46d8c695]) and a >>value >> > of >> > > > type >> > > > > > [com.codahale.metrics.ThreadLocalRandom] (value >> > > > > > [com.codahale.metrics.ThreadLocalRandom@3d37aed4]) but failed >>to >> > > > remove >> > > > > it >> > > > > > when the web application was stopped. Threads are going to be >> > renewed >> > > > > over >> > > > > > time to try and avoid a probable memory leak. >> > > > > > Apr 28, 2015 1:48:57 AM >> > org.apache.catalina.loader.WebappClassLoader >> > > > > > checkThreadLocalMapForLeaks >> > > > > > SCHWERWIEGEND: The web application [/kylin] created a >>ThreadLocal >> > > with >> > > > > key >> > > > > > of type [com.codahale.metrics.ThreadLocalRandom$1] (value >> > > > > > [com.codahale.metrics.ThreadLocalRandom$1@46d8c695]) and a >>value >> > of >> > > > type >> > > > > > [com.codahale.metrics.ThreadLocalRandom] (value >> > > > > > [com.codahale.metrics.ThreadLocalRandom@12b6ffb2]) but failed >>to >> > > > remove >> > > > > it >> > > > > > when the web application was stopped. Threads are going to be >> > renewed >> > > > > over >> > > > > > time to try and avoid a probable memory leak. >> > > > > > Apr 28, 2015 1:48:57 AM org.apache.coyote.AbstractProtocol >>stop >> > > > > > INFORMATION: Stopping ProtocolHandler ["http-bio-7070"] >> > > > > > Apr 28, 2015 1:48:57 AM org.apache.coyote.AbstractProtocol >>stop >> > > > > > INFORMATION: Stopping ProtocolHandler ["ajp-bio-9009"] >> > > > > > Apr 28, 2015 1:48:57 AM org.apache.coyote.AbstractProtocol >> destroy >> > > > > > INFORMATION: Destroying ProtocolHandler ["http-bio-7070"] >> > > > > > Apr 28, 2015 1:48:57 AM org.apache.coyote.AbstractProtocol >> destroy >> > > > > > INFORMATION: Destroying ProtocolHandler ["ajp-bio-9009"] >> > > > > > >> > > > > > >> > > > > > >> > > > > > Anyone familiar with that Failure? Is it a failure in the >> > > Configuration >> > > > > or >> > > > > > maybe a bug? >> > > > > > >> > > > > > Kind regards >> > > > > > Tobias >> > > > > > >> > > > > >> > > > >> > > >> > >>
