logs.dir is a kafka broker configuration you can override for where the broker data is stored (in case you want to write it outside the sandbox).
If you don't have any active/failed/killed/staged broker tasks then you should check the mesos logs to see why maybe offers are not going out, you should be seeing offers in the kafka scheduler logs e.g. 2015-06-04 22:53:29,641 [Thread-16] INFO ly.stealth.mesos.kafka.Scheduler$ - [resourceOffers] slave1#O1966 cpus:1.70 mem:1792.00 disk:35164.00 ports:[31839..32000][31000..31630][31633..31774][31778..31834] slave2#O1967 cpus:1.80 mem:2304.00 disk:35164.00 ports:[31000..31230][31235..31597][31600..32000] slave3#O1968 cpus:1.60 mem:1536.00 disk:35164.00 ports:[31045..31056][31061..31103][31000..31042][31105..31589][31592..32000] ~ Joe Stein - - - - - - - - - - - - - - - - - http://www.stealth.ly - - - - - - - - - - - - - - - - - On Thu, Jun 4, 2015 at 6:50 PM, William Francis <[email protected]> wrote: > > Hi Joe - > > Everything is the latest version as checked out from github today. At this > point I'm just following the directions in the README.md to see if I can > get it working. > > I see the scheduler registers itself as a framework on the mesos master: > > IDHostUserNameActive TasksCPUsMemMax ShareRegisteredRe-Registered > …5050-25434-0004 > <http://wfrancis-centos7-master1:5050/#/frameworks/20150603-233921-184621996-5050-25434-0004> > 2440709ddd51rootkafka000 B0%an hour ago- > but I don't see anything else launched. > > However, I did update my broker 0 to more memory/heap in the hopes that > maybe that would rectify the issue, but it did not: > > [root@wfrancis-centos7-master1 kafka]# ./kafka-mesos.sh update 0 --heap > 512 --mem 1024 --api=http://slave2:7000 > Broker updated > > broker: > id: 0 > active: false > state: stopped > resources: cpus:0.50, mem:1024, heap:512 > options: log.dirs=/tmp/log > failover: delay:10s, max-delay:60s > > [root@wfrancis-centos7-master1 kafka]# ./kafka-mesos.sh start 0 --api= > http://slave2:7000 > Error: Broker 0 scheduled to start. Got timeout > > I also set a log.dirs in the hopes of getting some useful output, but > don't see any in the container or outside of it. > > other ideas welcome! thanks. > > > > On Thu, Jun 4, 2015 at 5:54 PM, Joe Stein <[email protected]> wrote: > >> Hi William, do you have staging/running/failed/killed broker tasks? First >> thought looking at the status the issue could be that you have the heap == >> mem, when setting it so small for testing usually twice the size (assuming >> you have cgroups for your isolator). What version are you using? >> >> ~ Joe Stein >> - - - - - - - - - - - - - - - - - >> >> http://www.stealth.ly >> - - - - - - - - - - - - - - - - - >> >> On Thu, Jun 4, 2015 at 5:38 PM, William Francis <[email protected]> >> wrote: >> >>> >>> Hello - I have Mesos and Marathon running on centos 7. I'm trying to get >>> Kafka running but am having difficulty starting the brokers. I can launch >>> the scheduler directly as a framework or as a docker container and that is >>> fine (I think) but when I try to launch the brokers they time out and I'm >>> not aware of any logs I can look through to find out what's going on. >>> >>> In this example I start up a kafa-mesos docker container on my slave2 >>> and then attempt to control it from master1. >>> >>> This is all with java 1.7.0 >>> >>> docker run -it -p 7000:7000 --add-host=master1:172.27.1.11 >>> root/kafka-mesos ./kafka-mesos.sh scheduler --master=master1:5050 >>> --zk=master1:2181 --api=http://172.27.2.12:7000 >>> --storage=file:kafka-mesos.json >>> >>> 2015-06-04 20:46:04,664 [main] INFO ly.stealth.mesos.kafka.Scheduler$ >>> - Starting Scheduler$: >>> debug: false, storage: file:kafka-mesos.json >>> mesos: master=master1:5050, user=<default>, principal=<none>, >>> secret=<none> >>> framework: name=kafka, role=*, timeout=30d >>> api: http://172.27.2.12:7000, zk: master1:2181, jre: <none> >>> 2015-06-04 20:46:04,780 [main] INFO org.eclipse.jetty.server.Server - >>> jetty-9.0.z-SNAPSHOTWrappedArray() >>> 2015-06-04 20:46:04,821 [main] INFO >>> org.eclipse.jetty.server.handler.ContextHandler - Started >>> WrappedArray(o.e.j.s.ServletContextHandler@13a15f06{/,null,AVAILABLE}) >>> 2015-06-04 20:46:04,850 [main] INFO >>> org.eclipse.jetty.server.ServerConnector - Started >>> WrappedArray(ServerConnector@2f37e8bc{HTTP/1.1}{0.0.0.0:7000}) >>> 2015-06-04 20:46:04,851 [main] INFO ly.stealth.mesos.kafka.HttpServer$ >>> - started on port 7000 >>> I0604 20:46:04.943804 16 sched.cpp:157] Version: 0.22.1 >>> I0604 20:46:04.947131 43 sched.cpp:254] New master detected at >>> [email protected]:5050 >>> I0604 20:46:04.947571 43 sched.cpp:264] No credentials provided. >>> Attempting to register without authentication >>> >>> then back on my master1: >>> >>> [root@wfrancis-centos7-master1 kafka]# ./kafka-mesos.sh status --api= >>> http://slave2:7000 >>> Cluster status received >>> >>> cluster: >>> brokers: >>> >>> [root@wfrancis-centos7-master1 kafka]# ./kafka-mesos.sh add 0 --api= >>> http://slave2:7000 >>> Broker added >>> >>> broker: >>> id: 0 >>> active: false >>> state: stopped >>> resources: cpus:0.50, mem:128, heap:128 >>> failover: delay:10s, max-delay:60s >>> >>> [root@wfrancis-centos7-master1 kafka]# ./kafka-mesos.sh start 0 --api= >>> http://slave2:7000 >>> Error: Broker 0 scheduled to start. Got timeout >>> >>> [root@wfrancis-centos7-master1 kafka]# ./kafka-mesos.sh status --api= >>> http://slave2:7000 >>> Cluster status received >>> >>> cluster: >>> brokers: >>> id: 0 >>> active: true >>> state: starting >>> resources: cpus:0.50, mem:128, heap:128 >>> failover: delay:10s, max-delay:60s >>> >>> However the broker never starts. Ideas of how to debug this? >>> >>> >>> >> >
