Can you just use the Mesos cluster name? We'd end up preferring that to a configuration tweak anyway.
On Mar 10, 2015, at 2:40 PM, Jeff Schroeder <[email protected]> wrote: > Alright. You've convinced me I should put this online, but I'll have to clean > it up first. I would have originally but had one weird question. > > In my company, we almost certainly will have multiple potentially large Mesos > clusters as different failure domains. The issue I've got is relatively > simple, the solution is not. The collectd daemon runs on every single Mesos > node (master or slave) in the firm. If it is running on a master, it will get > data from localhost:$master_port/metrics/snapshot. It then checked if > json_data["master/elected"] == 1 and ONLY reports the data if it is the > elected master. By default it uses a namespace like: > > mesos.master.* > > If you have multiple clusters this obviously won't work, so I added an > optional cluster name attribute to the collectd config. That way it will > report to: > > mesos.archive.* where your mesos cluster name is "archive". > > How would you propose a decent way to namespace master stats that are cluster > wide. I had to lie (in the collectd plugin code) and change the host as a > metric such as: > > collectd.$HOSTNAME.master.* seems to make absolutely no sense when there is a > single elected master per cluster. > > > On Tuesday, March 10, 2015, Steven Schlansker <[email protected]> > wrote: > We would use (and probably contribute back to!) such an improved plugin as > well, if you do polish it up be sure to announce to this list :) > > On Mar 10, 2015, at 2:05 PM, Dan Dong <[email protected]> wrote: > > > Hi, Jeff, > > Thanks, is your plugin working together with collectd? It would be great > > to publish it! > > > > A general question for Mesos: Is there a method to monitor CPU/Memory/Disk > > usages of jobs from different frameworks(e.g: Hadoop/Mapreduce, Spark etc)? > > (Not necessarily to generate figures, text format numbers are quite enough.) > > So e.g for a hadoop job, when it's finished, we can collect the general > > metrics of it? Ideally although there are many jobs from different > > frameworks running at the same time on mesos, > > we still could get their metrics respectively. > > > > Cheers, > > Dan > > > > 2015-03-10 15:46 GMT-05:00 Jeff Schroeder <[email protected]>: > > I installed it and played with it for a bit but was somewhat underwhelmed > > with it. It doesn't support slaves and all of the hardcoding with > > duplication isn't my favorite. I ended up writing a single plugin to > > support both masters and slaves and putting it on every node in my Mesos > > cluster. > > > > Would it be worth polishing up a bit and throwing on github? > > > > > > On Tuesday, March 10, 2015, Dan Dong <[email protected]> wrote: > > Hi, All, > > Does anybody use this mesos-collectd-plugin: > > https://github.com/rayrod2030/collectd-mesos > > > > I have installed collectd and this plugin, then configured it as > > instructions and restarted the collectd daemon, why seems nothing happens > > on the mesos:5050 web UI( python plugin has been turned on in > > collectd.conf). > > > > My question is: > > 1. Should I install collectd and this mesos-collectd-plugin on each master > > and slave nodes and restart collectd daemon? (This is what I have done.) > > 2. Should the config file mesos-master.conf only configured on master node > > and > > mesos-slave.conf only configured on slave node?(This is what I have > > done.) > > Or both of them should only appear on master node? > > 3. Is there an example( or a figure) of what output one is expected to see > > by this plugin? > > > > Cheers, > > Dan > > > > > > > > -- > > Text by Jeff, typos by iPhone > > > > > > -- > Text by Jeff, typos by iPhone
