Roy, if you are in fact talking about implementation of monitoring system for ES then the following are my 2 cents:
- regarding how to poll data from ES nodes, I would not implement it as a "plugin" or anything similar that is dependent on ES platform itself. It should be an external process and possibly very light process (Java process is not light). The impact on ES should be as minimal as possible and it should not rely on plugin system of ES either. I would consider looking at something like RRD tools (and extensions on top of it). Really lightweight process that sits on each node, kicks once a while, grabs all metrics via REST API and stores it into local storage. It should then try to send these data into central store but it may not be always available thus storing it locally is important in case the connection to central store is restored at later time. The other argument why not to use plugin system of ES is that you should be able to upgrade monitoring system independently on ES cluster - you can not do this with ES plugins. - should it poll the same stats from every single node? I think it should! If ES cluster gets into trouble is can be because not every part of the cluster sees "the same picture". IMO the only way how to learn about this is collecting individual "pictures from every node". It is also the most simple way of doing it (and you should definitely aim for simplicity). Solving duplicities on the central storage side later should be possible but you might not even need to do it if the size of collected data is not a problem for you. The part where you need to be very careful when polling the same stats from each node is making sure this is not putting high load on the ES cluster. If it is possible to poll from the local node only (like using _local in URL) then opt for it. Though I am no sure if this option is still available post 1.0.0.RC1 release. Regards, Lukas On Tue, Jan 21, 2014 at 10:09 PM, [email protected] < [email protected]> wrote: > You must deploy plugins to all nodes, so you will still have the plugin > available when nodes come and go. You could add an action so that you can > turn on/off a plugin explicitly by remote command. Or you can define a > condition so a plugin can decide when to run. For example if you want a > singleton, you can check if you run on the master node, and execute only > then. > > Jörg > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFQUHak0-PQ%3DLW3C0_0GQy8EwJThOKC__RhGgzGJk4MvA%40mail.gmail.com > . > > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAO9cvUbcy%2B-F1EdrOAJE%2Bh8Ud8JcbnpWpAf78ijZ4yO0jn6RYA%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
