I've also run into this situation (multiple sites with their own ES clusters) and have solved it in a somewhat hacky way specific to the information I need. Most searches I need to do are for one cluster only, and only a few specific types of searches need to be done on all of them, so I made a small javascript tool to do those specific cases using the REST API. Basically, it just sends the same query to each graylog and then combines the results onto a simple web page. Making this kind of tool should be straight-forward for somone familiar with REST.
That said, I think a better solution would be to allow each web interface to duplicate requests across the network to another web interface and then combine the results with its own. Perhaps this could be accomplished with a plugin? -Michael On Thursday, January 8, 2015 at 11:32:25 AM UTC-5, Fábio Tramasoli wrote: > > Hello, John. > > Maybe I'm crazy, BUT, if you could search without the web interface, > meaning directly through the REST API, I think you could do this: > > Server1 and Server2 are both from the same ElasticSearch cluster. > BUT you set the parameters for "forced-awareness" ( > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html#forced-awareness), > > and shard-allocation-filtering ( > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html#allocation-filtering). > > Then you create two indexes, let's say graylog2server1 and graylog2server2 > and tie them to their respective servers. > > graylog2-server running on Server1 writes to index "graylog2server1" and > Server2 to index "graylog2server2". > > From there you could use multi-index-search (ref.: > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/multi-index.html > ). > > It's just an idea, but I think it would work and network for the E S > cluster usage in this case is just for cluster-state related bussiness. > > > "Brainstorming" :) > > Em quinta-feira, 19 de junho de 2014 05h20min28s UTC-3, [email protected] > escreveu: >> >> Thank you for your reply. >> >> I understand what you are saying. >> When I configure both Graylog nodes to use their own prefix, the slave >> server (Server2) is still saving it's data on Server1's storage. >> (ie. Server1 > Graylog21. Server2 > Graylog22). >> >> From what I have read, a master/master situation is not possible. How >> would you do this? >> >> John >> >> Op donderdag 19 juni 2014 09:46:03 UTC+2 schreef Martin René Mortensen: >>> >>> I dont think so at present. >>> >>> You can have several graylog2 server, but only 1 ES index, you cannot >>> search in more than 1 ES cluster. >>> >>> I understand your strategy, disconnected indexing with distributed >>> search, but graylog2 cannot search in more than 1 ES gluster. >>> >>> I was thinking about using 1 ES cluster with 2 nodes, 2 graylog2 >>> instances with each their own index prefix in the same ES index. Graylog2 >>> searching might just search in all graylog2_* indices and therefore might >>> just search through them all. This is not a recommended strategy though, >>> just a thought. >>> >>> /Martin >>> >>> On Wednesday, 18 June 2014 13:16:35 UTC+2, [email protected] wrote: >>>> >>>> I have two Graylog2 servers at two locations; Server1 and Server2. >>>> >>>> Server1 holds the Mongo database, Both servers use the Mongo database >>>> on Server1. >>>> >>>> Both servers however also store their data in Elasticsearch on Server1. >>>> If Server1 goes down, Server2 will stop receiving messages. >>>> >>>> Server1 should store its data in ES on Server1 >>>> Server2 should store its data in ES on Server2 >>>> >>>> So when Server1 goes down, Server2 should still be receiving messages. >>>> >>>> I would rather not replicate the ES indices or by some other way use >>>> double disk space or cause massive network load. >>>> >>>> Main goal is to have a dedicated Graylog2+ES server on each location; >>>> receiving message from hosts on their respective location and being able >>>> to >>>> search via 1 webinterface in both ES indices. >>>> >>>> Is this possible? And if so; how? >>>> >>> -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
