[graylog2] Re: Graylog indicies
Personally I changed all the references to graylog in the conf files back to graylog2, and so far no issues with that stuff. All my indices came back as expected. On Thursday, May 12, 2016 at 11:52:22 PM UTC-7, kaiser wrote: > > Hello, > > I have updated graylog with current version 2.0 > > After the update new indices are prefixed with graylog. > > My indices prefixed by graylog2 from graylog 1.3.4 are not displayed in > graylog. > > Is there a way to add them? > > regards. > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/4d7e2896-00ce-4a84-b221-79bb3081d239%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Graylog nodes unable to communicate with each other
I now have both servers using the same mongo, and as far as I can tell everything works. But I'm back to the same problem with an admin logged into the slave having the ability to accidentally or intentionally delete indices. The reader account is pretty much useless. I realize it's possible to create dashboards and streams to return some functionality. Up to now I had no reason or desire to do so. I have no reason to limit anyone from what they can search, and I want them to see the sources and stats. I would much prefer an account that looks almost identical to admin, but prevents one from changing various inputs/settings or deleting indices. I think we need a third superuser account type. I have seen similar feedback from others here. What to do? On Thursday, May 12, 2016 at 3:50:28 PM UTC-7, Mark Moorcroft wrote: > > > I'm having a similar issue. I have things to a point where neither > instance sees more than one "node". Both are seeing the elasticsearch > indicies (one local, one not). The master node seems mostly operational. I > set up a "slave" node for only one reason. The Graylog user levels made it > necessary to add another instance so users have full search capability but > no way to delete an index by mistake. It appears things have changed and > that strategy won't work anymore. The only step you mention that I haven't > done is clone the mongo. Right now my slave instance sees the indices, but > none of the searches ever load, and I see errors that no master is > selected, along with can't retrieve retention or rotation config. I presume > I'm reaching elasticsearch, but not the master graylog? I see no connection > errors in either mongo log. > >> >> -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/75475c80-4734-4a44-bb49-64946014901f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Graylog nodes unable to communicate with each other
I'm having a similar issue. I have things to a point where neither instance sees more than one "node". Both are seeing the elasticsearch indicies (one local, one not). The master node seems mostly operational. I set up a "slave" node for only one reason. The Graylog user levels made it necessary to add another instance so users have full search capability but no way to delete an index by mistake. It appears things have changed and that strategy won't work anymore. The only step you mention that I haven't done is clone the mongo. Right now my slave instance sees the indices, but none of the searches ever load, and I see errors that no master is selected, along with can't retrieve retention or rotation config. I presume I'm reaching elasticsearch, but not the master graylog? I see no connection errors in either mongo log. On Wednesday, May 11, 2016 at 12:32:27 AM UTC-7, Jochen Schalanda wrote: > > Hi Ross, > > make sure that elasticsearch_network_host (see > https://github.com/Graylog2/graylog2-server/blob/2.0.0/misc/graylog.conf#L187-L194 > and > http://docs.graylog.org/en/2.0/pages/upgrade.html#default-network-host) > is set to an IP address (or host name) which the other Elasticsearch and > Graylog nodes can access. > > Additionally make sure that the two Graylog nodes are using the same > MongoDB database and the same password_secret (see > https://github.com/Graylog2/graylog2-server/blob/2.0.0/misc/graylog.conf#L9-L11 > ). > > Cheers, > Jochen > > > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/ba8884d7-041f-4021-9b7a-ecbbf28f76f2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Received by deleted input on outdated node?
ALL messages are relevant to every user. And unless I don't have a firm grasp of Streams, I found that option unacceptable. So I set up a second VM with full search but no way to mess with the archived data or delete inputs by mistake. On Tuesday, June 16, 2015 at 1:18:53 AM UTC-7, Jochen Schalanda wrote: Hi Mark, you could probably create read-only users and assign them to a stream with messages relevant to them. Cheers, Jochen -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Received by deleted input on outdated node?
And if I could link to the master mongoDB then obviously that would defeat the point of giving search ability to users without making them an admin on the master? On Monday, June 15, 2015 at 6:17:23 AM UTC-7, Jochen Schalanda wrote: Hi Mark, input configurations are being stored inside MongoDB and are linked to the node ID. If your slave Graylog instance is either using another node ID or isn't able to access the MongoDB with the input configurations, you'll see the message (deleted input on outdated node) in the web interface. Cheers, Jochen On Friday, 12 June 2015 21:52:50 UTC+2, Mark Moorcroft wrote: I asked this back in April and I'm still looking for an answer. I have a protected VM running graylog/mongo/elastic, and all of our actual graylog usage takes place on a slave VM due to the way user accounts work. My question is about the slave graylog log events. They all show Received by deleted input on outdated node presumably because none of the inputs are local, and the elastic index is also remote. Is this a configuration error on my part, or is this just a consequence of using this arrangement? Is there any way to have them appear with the input and node on the remote? -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Received by deleted input on outdated node?
I asked this back in April and I'm still looking for an answer. I have a protected VM running graylog/mongo/elastic, and all of our actual graylog usage takes place on a slave VM due to the way user accounts work. My question is about the slave graylog log events. They all show Received by deleted input on outdated node presumably because none of the inputs are local, and the elastic index is also remote. Is this a configuration error on my part, or is this just a consequence of using this arrangement? Is there any way to have them appear with the input and node on the remote? -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] 1.1.2 kudos
When I did the 1.1.0 update it was essentially unusable. 1.1.1 at least eliminated the null pointer errors in search but I couldn't figure out how to see any detail on log entries. After installing 1.1.2 I am frankly WOW'ed by the new interface now that it actually seems to be working. Kudos to the developers on a tool that just seems to get better and better. I may be more cautious about updates in the future. Is there much difference between elastic 1.5.1 and 1.5.2? The repo update doesn't seem to offer the 1.5.2 update now that I'm on 1.5.1. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Re: Graylog 1.1 rpm update issue on 1 of 2
Yep, 1.1.1 solved the null pointer. On Saturday, June 6, 2015 at 3:12:35 AM UTC-7, Bernd Ahlers wrote: Mark, we released version 1.1.1 to fix some urgent issues. One of them was a NullPointerException during search. https://www.graylog.org/graylog-v1-1-1-is-now-available/ Can you please update to 1.1.1 and check if your problems are solved? Bernd Mark Moorcroft [Fri, Jun 05, 2015 at 04:13:52PM -0700] wrote: BTW and FWIW I am running the Oracle 8U45 JRE on both servers. In case that matters. On Thursday, June 4, 2015 at 8:42:08 PM UTC-7, Mark Moorcroft wrote: I yum updated both of my CentOS6 graylog servers to 1.1. The primary server where all the ES indexes reside seemed to have worked no problem. The second one that connects to the 1st seems to work perfectly in every way, BUT any attempt to Search results in the Oops message. I see no errors in the logs or the System Overview. Even my Dashboard with statistics on source message qty values works. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- Developer Tel.: +49 (0)40 609 452 077 Fax.: +49 (0)40 609 452 078 TORCH GmbH - A Graylog company Steckelhörn 11 20457 Hamburg Germany Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 Geschäftsführer: Lennart Koopmann (CEO) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Graylog 1.1 rpm update issue on 1 of 2
) ~[graylog-web-interface.graylog-web-interface-1.1.0.jar:na] at Routes$$anonfun$routes$1$$anonfun$applyOrElse$7$$anonfun$apply$459.apply(routes_routing.scala:1659) ~[graylog-web-interface.graylog-web-interface-1.1.0.jar:na] at play.core.Router$HandlerInvokerFactory$$anon$4.resultCall(Router.scala:264) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.core.Router$HandlerInvokerFactory$JavaActionInvokerFactory$$anon$15$$anon$1.invocation(Router.scala:255) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.core.j.JavaAction$$anon$1.call(JavaAction.scala:55) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.GlobalSettings$1.call(GlobalSettings.java:67) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.mvc.Security$AuthenticatedAction.call(Security.java:44) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.core.j.JavaAction$$anonfun$11.apply(JavaAction.scala:82) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.core.j.JavaAction$$anonfun$11.apply(JavaAction.scala:82) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) ~[org.scala-lang.scala-library-2.10.4.jar:na] at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) ~[org.scala-lang.scala-library-2.10.4.jar:na] at play.core.j.HttpExecutionContext$$anon$2.run(HttpExecutionContext.scala:40) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Execution$trampoline$.execute(Execution.scala:46) [com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.core.j.HttpExecutionContext.execute(HttpExecutionContext.scala:32) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at scala.concurrent.impl.Future$.apply(Future.scala:31) ~[org.scala-lang.scala-library-2.10.4.jar:na] at scala.concurrent.Future$.apply(Future.scala:485) ~[org.scala-lang.scala-library-2.10.4.jar:na] at play.core.j.JavaAction$class.apply(JavaAction.scala:82) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.core.Router$HandlerInvokerFactory$JavaActionInvokerFactory$$anon$15$$anon$1.apply(Router.scala:252) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:130) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:130) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.utils.Threads$.withContextClassLoader(Threads.scala:21) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:129) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:128) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at scala.Option.map(Option.scala:145) [org.scala-lang.scala-library-2.10.4.jar:na] at play.api.mvc.Action$$anonfun$apply$1.apply(Action.scala:128) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.mvc.Action$$anonfun$apply$1.apply(Action.scala:121) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$mapM$1.apply(Iteratee.scala:483) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$mapM$1.apply(Iteratee.scala:483) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$flatMapM$1.apply(Iteratee.scala:519) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$flatMapM$1.apply(Iteratee.scala:519) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$flatMap$1$$anonfun$apply$14.apply(Iteratee.scala:496) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$flatMap$1$$anonfun$apply$14.apply(Iteratee.scala:496) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) ~[org.scala-lang.scala-library-2.10.4.jar:na] at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) ~[org.scala-lang.scala-library-2.10.4.jar:na] ... 6 common frames omitted On Thursday, June 4, 2015 at 8:42:08 PM UTC-7, Mark Moorcroft wrote: I yum updated both of my CentOS6 graylog servers to 1.1. The primary server where all the ES indexes reside seemed to have worked no problem. The second one that connects to the 1st seems to work perfectly in every way, BUT any attempt to Search results in the Oops message. I see no errors in the logs or the System Overview. Even my Dashboard with statistics on source message qty values works. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr
[graylog2] Re: Graylog 1.1 rpm update issue on 1 of 2
So the problem was that the only local input on our alternate server was internal metrics. The only reason I even have a second server is because you don't allow searches for a non-admin. I added the Random HTTP input and the error disappeared. I don't want to have that either, but it seems I have no choice but to have some sort of local input now? So I guess the question is, what is the best throw-away input to have, since there is no reason for it to exist? On Thursday, June 4, 2015 at 8:42:08 PM UTC-7, Mark Moorcroft wrote: I yum updated both of my CentOS6 graylog servers to 1.1. The primary server where all the ES indexes reside seemed to have worked no problem. The second one that connects to the 1st seems to work perfectly in every way, BUT any attempt to Search results in the Oops message. I see no errors in the logs or the System Overview. Even my Dashboard with statistics on source message qty values works. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Graylog 1.1 rpm update issue on 1 of 2
Interestingly, if I increase the sleep period between random http messages I still get the null pointer exception. I'm at 3000 milliseconds now and I'm still getting the Oops. On Friday, June 5, 2015 at 12:03:29 PM UTC-7, Mark Moorcroft wrote: So the problem was that the only local input on our alternate server was internal metrics. The only reason I even have a second server is because you don't allow searches for a non-admin. I added the Random HTTP input and the error disappeared. I don't want to have that either, but it seems I have no choice but to have some sort of local input now? So I guess the question is, what is the best throw-away input to have, since there is no reason for it to exist? On Thursday, June 4, 2015 at 8:42:08 PM UTC-7, Mark Moorcroft wrote: I yum updated both of my CentOS6 graylog servers to 1.1. The primary server where all the ES indexes reside seemed to have worked no problem. The second one that connects to the 1st seems to work perfectly in every way, BUT any attempt to Search results in the Oops message. I see no errors in the logs or the System Overview. Even my Dashboard with statistics on source message qty values works. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Graylog 1.1 rpm update issue on 1 of 2
Hmm, OK, it seems that any time a Search includes anything found on the remote server it generates a null pointer. As long as all results exist in the local index it works. So either it's a bug or I have something configured wrong. Or something got screwed up during the update. I see statistics about the remote index values. I see details about the remote index size in Indices. Nodes mentions only the local index. Sources shows me info about all sources in the remote index. On Thursday, June 4, 2015 at 8:42:08 PM UTC-7, Mark Moorcroft wrote: I yum updated both of my CentOS6 graylog servers to 1.1. The primary server where all the ES indexes reside seemed to have worked no problem. The second one that connects to the 1st seems to work perfectly in every way, BUT any attempt to Search results in the Oops message. I see no errors in the logs or the System Overview. Even my Dashboard with statistics on source message qty values works. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Graylog 1.1 rpm update issue on 1 of 2
.jar:na] at play.core.Router$HandlerInvokerFactory$$anon$4.resultCall(Router.scala:264) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.core.Router$HandlerInvokerFactory$JavaActionInvokerFactory$$anon$15$$anon$1.invocation(Router.scala:255) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.core.j.JavaAction$$anon$1.call(JavaAction.scala:55) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.GlobalSettings$1.call(GlobalSettings.java:67) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.mvc.Security$AuthenticatedAction.call(Security.java:44) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.core.j.JavaAction$$anonfun$11.apply(JavaAction.scala:82) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.core.j.JavaAction$$anonfun$11.apply(JavaAction.scala:82) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) ~[org.scala-lang.scala-library-2.10.4.jar:na] at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) ~[org.scala-lang.scala-library-2.10.4.jar:na] at play.core.j.HttpExecutionContext$$anon$2.run(HttpExecutionContext.scala:40) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Execution$trampoline$.execute(Execution.scala:46) [com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.core.j.HttpExecutionContext.execute(HttpExecutionContext.scala:32) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at scala.concurrent.impl.Future$.apply(Future.scala:31) ~[org.scala-lang.scala-library-2.10.4.jar:na] at scala.concurrent.Future$.apply(Future.scala:485) ~[org.scala-lang.scala-library-2.10.4.jar:na] at play.core.j.JavaAction$class.apply(JavaAction.scala:82) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.core.Router$HandlerInvokerFactory$JavaActionInvokerFactory$$anon$15$$anon$1.apply(Router.scala:252) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:130) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:130) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.utils.Threads$.withContextClassLoader(Threads.scala:21) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:129) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:128) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at scala.Option.map(Option.scala:145) [org.scala-lang.scala-library-2.10.4.jar:na] at play.api.mvc.Action$$anonfun$apply$1.apply(Action.scala:128) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.mvc.Action$$anonfun$apply$1.apply(Action.scala:121) ~[com.typesafe.play.play_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$mapM$1.apply(Iteratee.scala:483) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$mapM$1.apply(Iteratee.scala:483) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$flatMapM$1.apply(Iteratee.scala:519) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$flatMapM$1.apply(Iteratee.scala:519) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$flatMap$1$$anonfun$apply$14.apply(Iteratee.scala:496) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at play.api.libs.iteratee.Iteratee$$anonfun$flatMap$1$$anonfun$apply$14.apply(Iteratee.scala:496) ~[com.typesafe.play.play-iteratees_2.10-2.3.9.jar:2.3.9] at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) ~[org.scala-lang.scala-library-2.10.4.jar:na] at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) ~[org.scala-lang.scala-library-2.10.4.jar:na] ... 6 common frames omitted On Thursday, June 4, 2015 at 8:42:08 PM UTC-7, Mark Moorcroft wrote: I yum updated both of my CentOS6 graylog servers to 1.1. The primary server where all the ES indexes reside seemed to have worked no problem. The second one that connects to the 1st seems to work perfectly in every way, BUT any attempt to Search results in the Oops message. I see no errors in the logs or the System Overview. Even my Dashboard with statistics on source message qty values works. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Graylog 1.1 rpm update issue on 1 of 2
BTW and FWIW I am running the Oracle 8U45 JRE on both servers. In case that matters. On Thursday, June 4, 2015 at 8:42:08 PM UTC-7, Mark Moorcroft wrote: I yum updated both of my CentOS6 graylog servers to 1.1. The primary server where all the ES indexes reside seemed to have worked no problem. The second one that connects to the 1st seems to work perfectly in every way, BUT any attempt to Search results in the Oops message. I see no errors in the logs or the System Overview. Even my Dashboard with statistics on source message qty values works. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Graylog 1.1 rpm update issue on 1 of 2
I yum updated both of my CentOS6 graylog servers to 1.1. The primary server where all the ES indexes reside seemed to have worked no problem. The second one that connects to the 1st seems to work perfectly in every way, BUT any attempt to Search results in the Oops message. I see no errors in the logs or the System Overview. Even my Dashboard with statistics on source message qty values works. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Read Only Users and Search and/or Stream which matches all messages
FWIW my solution to this was to create a second graylog virtual machine where all users are admin level. The second instance uses the elasticsearch index of the primary. This gives users full search ability without any way to go deleting the inputs by mistake. So far it appears to be a workable solution. For the most part the primary server is never logged into. It exists for compliance and archiving, and all of my audit logs go there too. Most of the time I probably don't even need to keep the graylog-web service running on the primary. On Monday, May 18, 2015 at 10:40:19 AM UTC-7, Roddy Rodstein wrote: Greetings, Could you please assist us with our GrayLog 1.0 read only users setup and allowing them to search? We realize that by default read only users are not able to search, but they can use streams. *Option 1: create a stream which matches all messages* This post below and a couple others mentioned creating a stream which matches all messages, and give access to this stream to read only users. https://groups.google.com/forum/#!searchin/graylog2/read$20only$20users$20search/graylog2/Iv7x3BKnhPI/3F_EIXCmCPUJ Could you please assist with a the steps to create a stream which matches all messages? We have been unsuccessful in all our attempts to create a stream which matches all messages. *Option 2: update the non-admin user permissions through the USER API. * This post has a solution but does not really show how to implement. https://github.com/Graylog2/graylog2-web-interface/issues/620 This option really looks great, could you please assist with the steps to implement? Thank you in advance for you support! -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] com.fasterxml.jackson.core.JsonParseException:
This morning I was seeing bunches of errors in the server.log. I think I
tracked them to a syslog/tcp input. My rsyslog entry on the client is as
follows.
# Graylog
$template GRAYLOGRFC5424,%PRI%%PROTOCOL-VERSION%
%TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID%
%STRUCTURED-DATA% %msg%\n
*.* @@xxx.xxx.xxx.xxx:12204;GRAYLOGRFC5424
It seems the cause was memory errors on a compute node. The question is if
this is a graylog bug or expected behavior. There were a series of these
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'xxx':
was expecting ('true', 'false' or 'null') at [Source: errors. I'm running
the current versions of graylog-server and elasticsearch. The token 'xxx'
is a random character(s) and then a massive bunch of garbage characters
will follow the error.
From /var/log/messages:
May 1 13:08:56 compute-0-21 kernel: flush-8:0: page allocation failure.
order:2, mode:0x20
May 1 13:08:56 compute-0-21 kernel: Pid: 444, comm: flush-8:0 Not tainted
2.6.32-431.11.2.el6.x86_64 #1
May 1 13:08:56 compute-0-21 kernel: Call Trace:
May 1 13:08:56 compute-0-21 kernel: IRQ [8112f9da] ?
__alloc_pages_nodemask+0x74a/0x8d0
May 1 13:08:56 compute-0-21 kernel: [8116e492] ?
kmem_getpages+0x62/0x170
May 1 13:08:56 compute-0-21 kernel: [8116f0aa] ?
fallback_alloc+0x1ba/0x270
May 1 13:08:56 compute-0-21 kernel: [8116eaff] ?
cache_grow+0x2cf/0x320
May 1 13:08:56 compute-0-21 kernel: [8116ee29] ?
cache_alloc_node+0x99/0x160
May 1 13:08:56 compute-0-21 kernel: [8116fff0] ?
kmem_cache_alloc_node_trace+0x90/0x200
May 1 13:08:56 compute-0-21 kernel: [8117020d] ?
__kmalloc_node+0x4d/0x60
May 1 13:08:56 compute-0-21 kernel: [8145033a] ?
__alloc_skb+0x7a/0x180
May 1 13:08:56 compute-0-21 kernel: [8145090d] ?
dev_alloc_skb+0x1d/0x40
May 1 13:08:56 compute-0-21 kernel: [a025c728] ?
nv_alloc_rx_optimized+0x198/0x270 [forcedeth]
May 1 13:08:56 compute-0-21 kernel: [a025bc76] ?
nv_rx_process_optimized+0x126/0x2a0 [forcedeth]
May 1 13:08:56 compute-0-21 kernel: [a025d80c] ?
nv_napi_poll+0x8c/0x610 [forcedeth]
May 1 13:08:56 compute-0-21 kernel: [8105dd5c] ?
scheduler_tick+0xcc/0x260
May 1 13:08:56 compute-0-21 kernel: [81460fb3] ?
net_rx_action+0x103/0x2f0
May 1 13:08:56 compute-0-21 kernel: [8112eef2] ?
free_pcppages_bulk+0x392/0x460
May 1 13:08:56 compute-0-21 kernel: [8107a8e1] ?
__do_softirq+0xc1/0x1e0
May 1 13:08:56 compute-0-21 kernel: [810e6eb0] ?
handle_IRQ_event+0x60/0x170
May 1 13:08:56 compute-0-21 kernel: [8100c30c] ?
call_softirq+0x1c/0x30
May 1 13:08:56 compute-0-21 kernel: [8100fa75] ?
do_softirq+0x65/0xa0
May 1 13:08:56 compute-0-21 kernel: [8107a795] ?
irq_exit+0x85/0x90
May 1 13:08:56 compute-0-21 kernel: [81531605] ? do_IRQ+0x75/0xf0
May 1 13:08:56 compute-0-21 kernel: [8100b9d3] ?
ret_from_intr+0x0/0x11
May 1 13:08:56 compute-0-21 kernel: EOI [811bdd20] ?
submit_bh+0x60/0x1f0
May 1 13:08:56 compute-0-21 kernel: [811c0598] ?
__block_write_full_page+0x1c8/0x330
May 1 13:08:56 compute-0-21 kernel: [811bf560] ?
end_buffer_async_write+0x0/0x190
May 1 13:08:56 compute-0-21 kernel: [811c07e0] ?
block_write_full_page_endio+0xe0/0x120
May 1 13:08:56 compute-0-21 kernel: [a02c4b30] ?
buffer_unmapped+0x0/0x20 [ext3]
May 1 13:08:56 compute-0-21 kernel: [811c0835] ?
block_write_full_page+0x15/0x20
May 1 13:08:56 compute-0-21 kernel: [a02c56dd] ?
ext3_ordered_writepage+0x1ed/0x240 [ext3]
May 1 13:08:56 compute-0-21 kernel: [811336c7] ?
__writepage+0x17/0x40
May 1 13:08:56 compute-0-21 kernel: [8113498d] ?
write_cache_pages+0x1fd/0x4c0
May 1 13:08:56 compute-0-21 kernel: [a0203e28] ?
__ext4_journal_stop+0x68/0xa0 [ext4]
May 1 13:08:56 compute-0-21 kernel: [811336b0] ?
__writepage+0x0/0x40
May 1 13:08:56 compute-0-21 kernel: [81134c74] ?
generic_writepages+0x24/0x30
May 1 13:08:56 compute-0-21 kernel: [81134cb5] ?
do_writepages+0x35/0x40
May 1 13:08:56 compute-0-21 kernel: [811b50cd] ?
writeback_single_inode+0xdd/0x290
May 1 13:08:56 compute-0-21 kernel: [811b54cd] ?
writeback_sb_inodes+0xbd/0x170
May 1 13:08:56 compute-0-21 kernel: [811b562b] ?
writeback_inodes_wb+0xab/0x1b0
May 1 13:08:56 compute-0-21 kernel: [811b5a23] ?
wb_writeback+0x2f3/0x410
May 1 13:08:56 compute-0-21 kernel: [81527f30] ?
thread_return+0x4e/0x76e
May 1 13:08:56 compute-0-21 kernel: [81084d92] ?
del_timer_sync+0x22/0x30
May 1 13:08:56 compute-0-21 kernel: [811b5bfb] ?
wb_do_writeback+0xbb/0x240
May 1 13:08:56 compute-0-21 kernel: [811b5de3] ?
bdi_writeback_task+0x63/0x1b0
May 1 13:08:56 compute-0-21 kernel: [8109b117] ?
bit_waitqueue+0x17/0xd0
May 1 13:08:56
[graylog2] Re: com.fasterxml.jackson.core.JsonParseException:
I found a year or more old reference to the same error and the solution back then was to switch the NXlog ouput from tcp to udp. The same change seems to have stopped the errors now. So it seems I can't currently use om_tcp in NXlog to send Windows logs. I'm not sure why udp works and tcp doesn't. On Friday, May 1, 2015 at 4:29:49 PM UTC-7, Mark Moorcroft wrote: This morning I was seeing bunches of errors in the server.log. I think I tracked them to a syslog/tcp input. My rsyslog entry on the client is as follows. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: com.fasterxml.jackson.core.JsonParseException:
Hmm, oh bugger, it seems the kernel errors are not the issue. The question
remains what is. I still see the errors every few minutes. The errors make
reference to GELF, and I only have one GELF tcp input from 2 Windows boxes
running NXlog. The errors seem to have started with the last graylog-server
update.
On Friday, May 1, 2015 at 4:29:49 PM UTC-7, Mark Moorcroft wrote:
This morning I was seeing bunches of errors in the server.log. I think I
tracked them to a syslog/tcp input. My rsyslog entry on the client is as
follows.
# Graylog
$template GRAYLOGRFC5424,%PRI%%PROTOCOL-VERSION%
%TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID%
%STRUCTURED-DATA% %msg%\n
*.* @@xxx.xxx.xxx.xxx:12204;GRAYLOGRFC5424
It seems the cause was memory errors on a compute node. The question is if
this is a graylog bug or expected behavior. There were a series of these
com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'xxx':
was expecting ('true', 'false' or 'null') at [Source: errors. I'm running
the current versions of graylog-server and elasticsearch. The token 'xxx'
is a random character(s) and then a massive bunch of garbage characters
will follow the error.
From /var/log/messages:
May 1 13:08:56 compute-0-21 kernel: flush-8:0: page allocation failure.
order:2, mode:0x20
May 1 13:08:56 compute-0-21 kernel: Pid: 444, comm: flush-8:0 Not tainted
2.6.32-431.11.2.el6.x86_64 #1
May 1 13:08:56 compute-0-21 kernel: Call Trace:
May 1 13:08:56 compute-0-21 kernel: IRQ [8112f9da] ?
__alloc_pages_nodemask+0x74a/0x8d0
May 1 13:08:56 compute-0-21 kernel: [8116e492] ?
kmem_getpages+0x62/0x170
May 1 13:08:56 compute-0-21 kernel: [8116f0aa] ?
fallback_alloc+0x1ba/0x270
May 1 13:08:56 compute-0-21 kernel: [8116eaff] ?
cache_grow+0x2cf/0x320
May 1 13:08:56 compute-0-21 kernel: [8116ee29] ?
cache_alloc_node+0x99/0x160
May 1 13:08:56 compute-0-21 kernel: [8116fff0] ?
kmem_cache_alloc_node_trace+0x90/0x200
May 1 13:08:56 compute-0-21 kernel: [8117020d] ?
__kmalloc_node+0x4d/0x60
May 1 13:08:56 compute-0-21 kernel: [8145033a] ?
__alloc_skb+0x7a/0x180
May 1 13:08:56 compute-0-21 kernel: [8145090d] ?
dev_alloc_skb+0x1d/0x40
May 1 13:08:56 compute-0-21 kernel: [a025c728] ?
nv_alloc_rx_optimized+0x198/0x270 [forcedeth]
May 1 13:08:56 compute-0-21 kernel: [a025bc76] ?
nv_rx_process_optimized+0x126/0x2a0 [forcedeth]
May 1 13:08:56 compute-0-21 kernel: [a025d80c] ?
nv_napi_poll+0x8c/0x610 [forcedeth]
May 1 13:08:56 compute-0-21 kernel: [8105dd5c] ?
scheduler_tick+0xcc/0x260
May 1 13:08:56 compute-0-21 kernel: [81460fb3] ?
net_rx_action+0x103/0x2f0
May 1 13:08:56 compute-0-21 kernel: [8112eef2] ?
free_pcppages_bulk+0x392/0x460
May 1 13:08:56 compute-0-21 kernel: [8107a8e1] ?
__do_softirq+0xc1/0x1e0
May 1 13:08:56 compute-0-21 kernel: [810e6eb0] ?
handle_IRQ_event+0x60/0x170
May 1 13:08:56 compute-0-21 kernel: [8100c30c] ?
call_softirq+0x1c/0x30
May 1 13:08:56 compute-0-21 kernel: [8100fa75] ?
do_softirq+0x65/0xa0
May 1 13:08:56 compute-0-21 kernel: [8107a795] ?
irq_exit+0x85/0x90
May 1 13:08:56 compute-0-21 kernel: [81531605] ?
do_IRQ+0x75/0xf0
May 1 13:08:56 compute-0-21 kernel: [8100b9d3] ?
ret_from_intr+0x0/0x11
May 1 13:08:56 compute-0-21 kernel: EOI [811bdd20] ?
submit_bh+0x60/0x1f0
May 1 13:08:56 compute-0-21 kernel: [811c0598] ?
__block_write_full_page+0x1c8/0x330
May 1 13:08:56 compute-0-21 kernel: [811bf560] ?
end_buffer_async_write+0x0/0x190
May 1 13:08:56 compute-0-21 kernel: [811c07e0] ?
block_write_full_page_endio+0xe0/0x120
May 1 13:08:56 compute-0-21 kernel: [a02c4b30] ?
buffer_unmapped+0x0/0x20 [ext3]
May 1 13:08:56 compute-0-21 kernel: [811c0835] ?
block_write_full_page+0x15/0x20
May 1 13:08:56 compute-0-21 kernel: [a02c56dd] ?
ext3_ordered_writepage+0x1ed/0x240 [ext3]
May 1 13:08:56 compute-0-21 kernel: [811336c7] ?
__writepage+0x17/0x40
May 1 13:08:56 compute-0-21 kernel: [8113498d] ?
write_cache_pages+0x1fd/0x4c0
May 1 13:08:56 compute-0-21 kernel: [a0203e28] ?
__ext4_journal_stop+0x68/0xa0 [ext4]
May 1 13:08:56 compute-0-21 kernel: [811336b0] ?
__writepage+0x0/0x40
May 1 13:08:56 compute-0-21 kernel: [81134c74] ?
generic_writepages+0x24/0x30
May 1 13:08:56 compute-0-21 kernel: [81134cb5] ?
do_writepages+0x35/0x40
May 1 13:08:56 compute-0-21 kernel: [811b50cd] ?
writeback_single_inode+0xdd/0x290
May 1 13:08:56 compute-0-21 kernel: [811b54cd] ?
writeback_sb_inodes+0xbd/0x170
May 1 13:08:56 compute-0-21 kernel: [811b562b] ?
writeback_inodes_wb+0xab/0x1b0
May 1 13:08:56 compute-0-21 kernel
[graylog2] Re: Filter or Drop messages from a specific source
So this is an undocumented (as of yet) method to have graylog filter an input as it feeds the elasticsearch index? If I do a search on the graylog site for drool I get nothing. On Thursday, April 30, 2015 at 10:43:38 PM UTC-7, temo tsurtsumia wrote: import org.graylog2.plugin.Message rule Drop host x when m : Message( source == 10.0.3.x ) then m.setFilterOut(true); System.out.println( [Drop host y] : + m.toString() ); end import org.graylog2.plugin.Message rule Drop host y when m : Message( source == 10.0.3.y ) then m.setFilterOut(true); System.out.println( [Drop host y] : + m.toString() ); end import org.graylog2.plugin.Message rule Drop host z when m : Message( source == 10.0.3.z ) then m.setFilterOut(true); System.out.println( [Drop host z] : + m.toString() ); end change host accordingly -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Graylog 1.0.2 blacklist
I asked a similar question recently (title Exclude strategy), but I never got any reply. On Thursday, April 30, 2015 at 12:59:21 PM UTC-7, temo tsurtsumia wrote: How to apply simply blacklist rules for dropping unnecessary messages -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] graylog-server startup failing on boot
I have graylog/mongo/elastic installed via repo (RPM) on CentOS6. What I'm seeing is any time I reboot the VM graylog-server fails to start. It seems it tries to start up before elasticsearch has a chance to stabilize, because if I service graylog-server restart later it will work. The problem is this is a protected VM that I don't have root on, so I have to get the system owner to restart the service for me. I'm not sure if elasticsearch is taking too long to start, or if graylog-server needs a test so it waits for the elasticsearch service to be running. As it is there seems to be no wait loop, so graylog-server just dies, but it appears to leave the lock file behind. None of this is good. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Exclude strategy?
I'm wondering if anyone can suggest a strategy for eliminating certain classes of collected logged events. In particular I have 3 compute clusters. Each one does NAT DHCP for the compute nodes. I prefer that the head nodes continue to collect logged compute node traffic, but I have no need to see them on the log collector. Nor do I want the Sources list clogged up with 200 compute node names. It seems to me there are several possible strategies for getting rid of them. I was hoping someone might suggest the best practice. One possible issue is on the older cluster running CentOS5 the nodes run syslogd (not rsyslogd), which is much less flexible to configure. I actually had to replace syslogd on the head node with rsyslog to get the output I wanted for graylog. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Oracle java updates?
The elasticsearch wisdom seems to be to use the Oracle JRE. But has anyone figured out how to keep the Oracle JRE updated on a standalone elastic server that never runs a browser. I can't seem to find any documentation about this. And I can't find any reference to a java command that checks for pending updates on the command line. I don't see any sign that the linux JRE has a control panel, and according to the documentation I found Windows is the only platform the supports auto-update. Obviously if you use the CentOS yum installed java then yum update handles the updates. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Re: Increase JVM heap space
So most of the performance tuning should take place in the /etc/elasticsearch settings then (local or not)? The graylog elastic index doesn't appear to store anything anyway. Looks like we peak at about 2.5k messages per minute (very rare) from a dozen sources. More commonly we see 500 message per minute spikes. On Thursday, April 16, 2015 at 1:23:39 AM UTC-7, Jochen Schalanda wrote: Hi, while you can certainly increase the maximum heap size for Graylog, it shouldn't be necessary for most workloads. As a matter of fact, increasing the heap size is counter-productive most of the time, as it increases the garbage collection time. What are you trying to achieve with this? Cheers, Jochen On Thursday, 16 April 2015 00:06:03 UTC+2, Mark Moorcroft wrote: From my kickstart: sed -i -e 's/-Xms1g -Xmx1g -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server/-Xms4g -Xmx4g -XX:NewRatio=1 -server/' /etc/sysconfig/graylog-server I increased from 1G to 4G here. On Wednesday, April 15, 2015 at 6:42:40 AM UTC-7, Alejandro Cabrera Obed wrote: OK, but how can I increase the heap space in the Node tab of the graylog web intrerface??? Because I see this message: The JVM is using *764 of 972 MB* heap space and will not attempt to use more than *972 MB* *Is it possible to grow up from 972 MB to 4 GB ??? How ???* *Thanks* 2015-04-15 4:54 GMT-03:00 Jochen Schalanda joc...@graylog.com: Hi Alejandro, starting with Graylog 1.0.0, incoming messages are always written to the disk journal (which is generally a good thing). You can disable the disk journal entirely (see https://github.com/Graylog2/graylog2-server/blob/1.0.1/misc/graylog2.conf#L245-246), but I would not recommend doing this. Do you see any problems with your current Graylog setup? Cheers, Jochen On Tuesday, 14 April 2015 21:29:24 UTC+2, Alejandro Cabrera Obed wrote: People, in my graylog server I have a lot of incoming logs and in the Node tab of Graylog web (version 1.0.1) I can see too many processing messages and the processing status bar is always near the maximum. How can I increase the JVM heap space in order to avoid journaling??? At the moment the JVM heap space is 972 MB. Thanks a lot, Alejandro -- You received this message because you are subscribed to a topic in the Google Groups graylog2 group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/graylog2/Srt7uXJDdpY/unsubscribe. To unsubscribe from this group and all its topics, send an email to graylog2+u...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Alejandro Cabrera Obed aco...@gmail.com -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Search advise
This is probably a dumb newb question, but at this moment it's not obvious to me. If I have a saved search like: dropping event AND queue is full Is it possible to see the list of Sources with the number of logged events per source ONLY, instead of 10 pages of results? I guess you could call that a summary. It's often the case I don't care about the details. I'm just in a hurry to go look at the clients in question so I just want to see which ones are spitting out the messages. Part of the problem is probably that I have yet to embrace Streams because it's really not clear to me how they work. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Best ElasticSearch version ?
Not exactly a Graylog issue, but yum update elasticsearch seems to fail entirely. It simply never finds any updates. I never noticed until just now. I updated the repo file to the 1.5 series, and it still found no updates pending. Finally I just downloaded the 1.4.4 and 1.5.1 RPM's and rpm -Uvh installed them. Maybe in future the 1.5 updates will appear in yum? I hope I don't have to edit the repo file every time they have a minor version number change. Perhaps yum update will only work from minor version to minor version (1.3.1.to 1.3.2), and from 1.3 to 1.4 only works outside of yum? On Thursday, April 9, 2015 at 12:58:40 AM UTC-7, Jochen Schalanda wrote: Hi Florent, Graylog works fine with Elasticsearch 1.5.0 and you can upgrade, if you want to. I would recommend upgrading to Elasticsearch 1.4.4 at least. Cheers, Jochen On Thursday, 9 April 2015 09:46:27 UTC+2, Florent B wrote: Hi, I'm running Graylog 1 with ES 1.4.1. I would like to know if I can safely upgrade to ES 1.5.0, and should I do ? Thank you :) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Questions about strategy
When I initially set out to replace free Splunk with Graylog the requirements were as follows: Create a central log collector with write access granted to only one person (non-tech manager) for compliance and forensics. The collected data includes about 8 CentOS boxes sending auditd and syslog, and 2 Windows servers sending Win logs via NXlog. Grant read access (i.e. search) to the sysadmin staff. Initially I set up 2 completely separate Graylog VM's with one access limited and one not. This was only necessary because of the perplexing way Graylog requires me to use Streams to limit access, which I found totally unapproachable. This demanded all senders to send streams to both VM's and it doubles the storage requirements. It occurred to me last week that I should be able to have both VM's using the same elastic storage. It seemed I could have the protected VM store all the data, and just have the admin access VM parse it for search. I presume I can't just run graylog-web on the second VM because that must use different authentication/access lists. But I'm having some trouble figuring out how to get the secondary graylog VM to share the search data. I have it connecting and I see the Index name from the other VM under indices, but the numbers don't correlate at all. And I don't see any events unless I collect them locally, so I presume the second VM would store it's own collected events just fine, but they are not sharing them (the whole point). So I'm clearly missing an obvious large piece of the puzzle to close the loop. That or I'm barking up the wrong tree entirely. Actually, at the end of the day, the protected VM that collects that data doesn't even need graylog-web at all. It just needs to be a data collector. Which I guess means it needs elastic collecting data. But of course the inputs are created using graylog-web. I'm still not totally wrapping my head around how the graylog-server and elasticsearch pieces fit together. I'm hoping someone has done something similar that can offer some insight. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Questions about strategy
I found out why my second Graylog VM was seeing a different Elastic index.. so problem solved there. Still hoping for feedback on the whole strategy though. How do I make the secondary graylog-server/web stop warning me there are no configured inputs? How should these be set on the primary and secondary graylog VM's: # we don't want the graylog2 server to store any data, or be master node elasticsearch_node_master = false elasticsearch_node_data = false On Monday, March 30, 2015 at 12:15:39 PM UTC-7, Mark Moorcroft wrote: Initially I set up 2 completely separate Graylog VM's with one access limited and one not. This was only necessary because of the perplexing way Graylog requires me to use Streams to limit access, which I found totally unapproachable. This demanded all senders to send streams to both VM's and it doubles the storage requirements. It occurred to me last week that I should be able to have both VM's using the same elastic storage. It seemed I could have the protected VM store all the data, and just have the admin access VM parse it for search. I presume I can't just run graylog-web on the second VM because that must use different authentication/access lists. But I'm having some trouble figuring out how to get the secondary graylog VM to share the search data. I have it connecting and I see the Index name from the other VM under indices, but the numbers don't correlate at all. And I don't see any events unless I collect them locally, so I presume the second VM would store it's own collected events just fine, but they are not sharing them (the whole point). So I'm clearly missing an obvious large piece of the puzzle to close the loop. That or I'm barking up the wrong tree entirely. Actually, at the end of the day, the protected VM that collects that data doesn't even need graylog-web at all. It just needs to be a data collector. Which I guess means it needs elastic collecting data. But of course the inputs are created using graylog-web. I'm still not totally wrapping my head around how the graylog-server and elasticsearch pieces fit together. I'm hoping someone has done something similar that can offer some insight. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: More Graylog/Elastic questions from the cheap seats
Next question... Why do all of the elastic stored records appear to reside in the default dynamic named node, but the apparently empty graylog2-server elastic node is the one gobbling up heap memory? According to my elastic node diags the empty graylog2-server node, that according to the graylog interface isn't used, the more memory I give it, the more it will use. Also, I switched from OpenJDK to Oracle today. It complains that -XX:PermSize=128m -XX:MaxPermSize=256m from /etc/sysconfig/graylog-server are no longer supported. On Wednesday, March 25, 2015 at 7:31:38 PM UTC-7, Mark Moorcroft wrote: In looking at trying to increase the heap size today after a general overhaul of our logging system I was reminded about a few things I never seemed to get answers to in the past. Some of these statements are in fact questions. Setting mlockall in elasticsearch apparently does NOT set it for graylog? I can't seem to find a way to increase the heap size for the graylog index beyond 972MB. From the beginning I have wondered why I need the default elastic index (node with the dynamic naming) that never seems to be used, as well as the graylog index(node). The default elastic index seems to have all of the recommended tweaks (like mlockall), but the graylog index doesn't. Where exactly am I supposed to be changing them? Many times today on both of my graylog systems clicking on System:Nodes produces This exception has been logged with id 6libgij97.. I don't see any other issues. If I run curl http://localhost:9200/_nodes/process?pretty; when I look at the nodes parameters the graylog node is version 1.3.7 but the default node is 1.3.4 with different build numbers. More dumb questions to follow if I can remember them ;-) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] [ANN] Graylog 1.0.1 has been released
Yeah, well I'm seeing Caused by: java.util.concurrent.TimeoutException: No response received after 5000 so clearly the server is going out to lunch, possibly due to incoming traffic? I'm going to guess I need to allocate more resources to various things, and possibly give the VM more memory. But as you can see from my other threads I'm having some issues understanding how to convince Graylog/Elastic to use more resources. I'm still chipping away at it since nobody has responded. On Thu, Mar 26, 2015 at 2:49 AM, Edmundo Alvarez edmu...@graylog.com wrote: Hi Mark, Please check your Graylog server and web interface logs for more information. If you need help with it, we will need to know the error that is logged there when you access the nodes page, and please open a new thread for it :). Regards, Edmundo -- Developer Tel.: +49 (0)40 609 452 077 Fax.: +49 (0)40 609 452 078 TORCH GmbH - A Graylog company Steckelhörn 11 20457 Hamburg Germany Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 Geschäftsführer: Lennart Koopmann (CEO) On 26 Mar 2015, at 01:28, Mark Moorcroft plak...@gmail.com wrote: Nice... BTW, I have been getting This exception has been logged with id 6libgij97. quite a bit today when I click on the nodes link. This is happening on both of my graylog servers. On Monday, March 16, 2015 at 8:00:44 AM UTC-7, Jochen Schalanda wrote: Hi, I'm delighted to announce the release of Graylog 1.0.1 into the wild. This is purely a bug-fix release and doesn't bring any new features. The changes since Graylog 1.0.0 are: • Properly log stack traces (#970) • Update REST API browser to new Graylog logo • Avoid spamming the logs if the original input of a message in the disk journal can't be loaded (#1005) • Allows reader users to see the journal status (#1009) • Compatibility with MongoDB 3.0 and Wired Tiger storage engine (#1024) • Respect rest_transport_uri when generating entity URLs in REST API (#1020) • Properly map NodeNotFoundException (#1137) • Allow replacing all existing Grok patterns on bulk import (#1150) • Configuration option for discarding messages on error in AMQP inputs (#1018) • Configuration option of maximum HTTP chunk size for HTTP-based inputs (#1011) • Clone alarm callbacks when cloning a stream (#990) • Add hasField() and getField() methods to MessageSummary class (#923) • Add per input parse time metrics (#1106) • Allow the use of log4j-extras classes in log4j configuration (#1042) • Fix updating of input statistics for Radio nodes (#1022) • Emit proper error message when a regular expression in an Extractor doesn't match example message (#1157) • Add additional information to system jobs (#920) • Fix false positive message on LDAP login test (#1138) • Calculate saved search resolution dynamically (#943) • Only enable LDAP test buttons when data is present (#1097) • Load more than 1 message on Extractor form (#1105) • Fix NPE when listing alarm callback using non-existent plugin (#1152) • Redirect to nodes overview when node is not found (#1137) • Fix documentation links to integrations and data sources (#1136) • Prevent accidental indexing of web interface by web crawlers (#1151) • Validate grok pattern name on the client to avoid duplicate names (#937) • Add message journal usage to nodes overview page (#1083) • Properly format numbers according to locale (#1128, #1129) Thanks to everyone who helped creating this release by using Graylog and reporting bugs and regressions to the mailing list and on GitHub. The official RPM and DEB packages, as well as the virtual machine images have been updated with the new version. As always, if you find any bugs in this release, please open an issue on GitHub at https://github.com/Graylog2/graylog2-server/issues. Best regards, Jochen (in the name of the Graylog team) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to a topic in the Google Groups graylog2 group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/graylog2/liulcbtvnuk/unsubscribe. To unsubscribe from this group and all its topics, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr
[graylog2] Re: More Graylog/Elastic questions from the cheap seats
Still flailing without guidance I have some more questions. I changed elasticsearch_discovery_zen_ping_unicast_hosts = 127.0.0.1:9300, and with 9300 it appears that the default index (node) is being filled. Graylog creates a second node that I don't believe I need (port 9350 which I didn't set or chose), but if I look at the elasticsearch parameters after adjusting heap size in sysconfig, Graylog Nodes is showing me the heap size for the index that isn't being used (the one set in /etc/sysconfig/graylog-server). The memory usage fluctuates as though something is happening, but that index is totally empty. The default dynamically named index is filling, and I have increased the heap size there in /etc/sysconfig/elasticsearch. So the web interface is showing me status on the unused index (node). On Wednesday, March 25, 2015 at 7:31:38 PM UTC-7, Mark Moorcroft wrote: In looking at trying to increase the heap size today after a general overhaul of our logging system I was reminded about a few things I never seemed to get answers to in the past. Some of these statements are in fact questions. Setting mlockall in elasticsearch apparently does NOT set it for graylog? I can't seem to find a way to increase the heap size for the graylog index beyond 972MB. From the beginning I have wondered why I need the default elastic index (node with the dynamic naming) that never seems to be used, as well as the graylog index(node). The default elastic index seems to have all of the recommended tweaks (like mlockall), but the graylog index doesn't. Where exactly am I supposed to be changing them? -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] More Graylog/Elastic questions from the cheap seats
In looking at trying to increase the heap size today after a general overhaul of our logging system I was reminded about a few things I never seemed to get answers to in the past. Some of these statements are in fact questions. Setting mlockall in elasticsearch apparently does NOT set it for graylog? I can't seem to find a way to increase the heap size for the graylog index beyond 972MB. From the beginning I have wondered why I need the default elastic index (node with the dynamic naming) that never seems to be used, as well as the graylog index(node). The default elastic index seems to have all of the recommended tweaks (like mlockall), but the graylog index doesn't. Where exactly am I supposed to be changing them? Many times today on both of my graylog systems clicking on System:Nodes produces This exception has been logged with id 6libgij97.. I don't see any other issues. If I run curl http://localhost:9200/_nodes/process?pretty; when I look at the nodes parameters the graylog node is version 1.3.7 but the default node is 1.3.4 with different build numbers. More dumb questions to follow if I can remember them ;-) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: [ANN] Graylog 1.0.1 has been released
Nice... BTW, I have been getting This exception has been logged with id 6libgij97. quite a bit today when I click on the nodes link. This is happening on both of my graylog servers. On Monday, March 16, 2015 at 8:00:44 AM UTC-7, Jochen Schalanda wrote: Hi, I'm delighted to announce the release of Graylog 1.0.1 into the wild. This is purely a bug-fix release and doesn't bring any new features. The changes since Graylog 1.0.0 are: - Properly log stack traces (#970 https://github.com/Graylog2/graylog2-server/issues/970) - Update REST API browser to new Graylog logo - Avoid spamming the logs if the original input of a message in the disk journal can't be loaded (#1005 https://github.com/Graylog2/graylog2-server/issues/1005) - Allows reader users to see the journal status (#1009 https://github.com/Graylog2/graylog2-server/issues/1009) - Compatibility with MongoDB 3.0 and Wired Tiger storage engine (#1024 https://github.com/Graylog2/graylog2-server/issues/1024) - Respect rest_transport_uri when generating entity URLs in REST API ( #1020 https://github.com/Graylog2/graylog2-server/issues/1020) - Properly map NodeNotFoundException (#1137 https://github.com/Graylog2/graylog2-web-interface/issues/1137) - Allow replacing all existing Grok patterns on bulk import (#1150 https://github.com/Graylog2/graylog2-web-interface/pull/1150) - Configuration option for discarding messages on error in AMQP inputs (#1018 https://github.com/Graylog2/graylog2-server/issues/1018) - Configuration option of maximum HTTP chunk size for HTTP-based inputs (#1011 https://github.com/Graylog2/graylog2-server/issues/1011 ) - Clone alarm callbacks when cloning a stream (#990 https://github.com/Graylog2/graylog2-server/issues/990) - Add hasField() and getField() methods to MessageSummary class (#923 https://github.com/Graylog2/graylog2-server/issues/923) - Add per input parse time metrics (#1106 https://github.com/Graylog2/graylog2-web-interface/issues/1106) - Allow the use of log4j-extras https://logging.apache.org/log4j/extras/ classes in log4j configuration (#1042 https://github.com/Graylog2/graylog2-server/issues/1042) - Fix updating of input statistics for Radio nodes (#1022 https://github.com/Graylog2/graylog2-web-interface/issues/1122) - Emit proper error message when a regular expression in an Extractor doesn't match example message (#1157 https://github.com/Graylog2/graylog2-web-interface/issues/1157) - Add additional information to system jobs (#920 https://github.com/Graylog2/graylog2-server/issues/920) - Fix false positive message on LDAP login test (#1138 https://github.com/Graylog2/graylog2-web-interface/issues/1138) - Calculate saved search resolution dynamically (#943 https://github.com/Graylog2/graylog2-web-interface/issues/943) - Only enable LDAP test buttons when data is present (#1097 https://github.com/Graylog2/graylog2-web-interface/issues/1097) - Load more than 1 message on Extractor form (#1105 https://github.com/Graylog2/graylog2-web-interface/issues/1105) - Fix NPE when listing alarm callback using non-existent plugin (#1152 https://github.com/Graylog2/graylog2-web-interface/issues/1152) - Redirect to nodes overview when node is not found (#1137 https://github.com/Graylog2/graylog2-web-interface/issues/1137) - Fix documentation links to integrations and data sources (#1136 https://github.com/Graylog2/graylog2-web-interface/issues/1136) - Prevent accidental indexing of web interface by web crawlers (#1151 https://github.com/Graylog2/graylog2-web-interface/issues/1151) - Validate grok pattern name on the client to avoid duplicate names ( #937 https://github.com/Graylog2/graylog2-server/issues/937) - Add message journal usage to nodes overview page (#1083 https://github.com/Graylog2/graylog2-web-interface/issues/1083) - Properly format numbers according to locale (#1128 https://github.com/Graylog2/graylog2-web-interface/issues/1128, #1129 https://github.com/Graylog2/graylog2-web-interface/issues/1129) Thanks to everyone who helped creating this release by using Graylog and reporting bugs and regressions to the mailing list and on GitHub. The official RPM and DEB packages, as well as the virtual machine images have been updated with the new version. As always, if you find any bugs in this release, please open an issue on GitHub at https://github.com/Graylog2/graylog2-server/issues. Best regards, Jochen (in the name of the Graylog team) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: More Graylog/Elastic questions from the cheap seats
I'm not sure if it's considered a best practice to tweak the default /etc/sysconfig/graylog-server? GRAYLOG_SERVER_JAVA_OPTS=-Xms2g -Xmx2g -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled-XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow But this at least seems to give you double the heap space. It's still not obvious how you should set mlockall. Or if I should even try. On Wednesday, March 25, 2015 at 7:31:38 PM UTC-7, Mark Moorcroft wrote: In looking at trying to increase the heap size today after a general overhaul of our logging system I was reminded about a few things I never seemed to get answers to in the past. Some of these statements are in fact questions. Setting mlockall in elasticsearch apparently does NOT set it for graylog? I can't seem to find a way to increase the heap size for the graylog index beyond 972MB. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: [ANN] Graylog 1.0.1 has been released
It still says 1.0.0 for graylog-web at the bottom of the interface despite yum reporting 1.0.1. FYI On Monday, March 16, 2015 at 8:00:44 AM UTC-7, Jochen Schalanda wrote: Hi, I'm delighted to announce the release of Graylog 1.0.1 into the wild. This is purely a bug-fix release and doesn't bring any new features. The changes since Graylog 1.0.0 are: - Properly log stack traces (#970 https://github.com/Graylog2/graylog2-server/issues/970) - Update REST API browser to new Graylog logo - Avoid spamming the logs if the original input of a message in the disk journal can't be loaded (#1005 https://github.com/Graylog2/graylog2-server/issues/1005) - Allows reader users to see the journal status (#1009 https://github.com/Graylog2/graylog2-server/issues/1009) - Compatibility with MongoDB 3.0 and Wired Tiger storage engine (#1024 https://github.com/Graylog2/graylog2-server/issues/1024) - Respect rest_transport_uri when generating entity URLs in REST API ( #1020 https://github.com/Graylog2/graylog2-server/issues/1020) - Properly map NodeNotFoundException (#1137 https://github.com/Graylog2/graylog2-web-interface/issues/1137) - Allow replacing all existing Grok patterns on bulk import (#1150 https://github.com/Graylog2/graylog2-web-interface/pull/1150) - Configuration option for discarding messages on error in AMQP inputs (#1018 https://github.com/Graylog2/graylog2-server/issues/1018) - Configuration option of maximum HTTP chunk size for HTTP-based inputs (#1011 https://github.com/Graylog2/graylog2-server/issues/1011 ) - Clone alarm callbacks when cloning a stream (#990 https://github.com/Graylog2/graylog2-server/issues/990) - Add hasField() and getField() methods to MessageSummary class (#923 https://github.com/Graylog2/graylog2-server/issues/923) - Add per input parse time metrics (#1106 https://github.com/Graylog2/graylog2-web-interface/issues/1106) - Allow the use of log4j-extras https://logging.apache.org/log4j/extras/ classes in log4j configuration (#1042 https://github.com/Graylog2/graylog2-server/issues/1042) - Fix updating of input statistics for Radio nodes (#1022 https://github.com/Graylog2/graylog2-web-interface/issues/1122) - Emit proper error message when a regular expression in an Extractor doesn't match example message (#1157 https://github.com/Graylog2/graylog2-web-interface/issues/1157) - Add additional information to system jobs (#920 https://github.com/Graylog2/graylog2-server/issues/920) - Fix false positive message on LDAP login test (#1138 https://github.com/Graylog2/graylog2-web-interface/issues/1138) - Calculate saved search resolution dynamically (#943 https://github.com/Graylog2/graylog2-web-interface/issues/943) - Only enable LDAP test buttons when data is present (#1097 https://github.com/Graylog2/graylog2-web-interface/issues/1097) - Load more than 1 message on Extractor form (#1105 https://github.com/Graylog2/graylog2-web-interface/issues/1105) - Fix NPE when listing alarm callback using non-existent plugin (#1152 https://github.com/Graylog2/graylog2-web-interface/issues/1152) - Redirect to nodes overview when node is not found (#1137 https://github.com/Graylog2/graylog2-web-interface/issues/1137) - Fix documentation links to integrations and data sources (#1136 https://github.com/Graylog2/graylog2-web-interface/issues/1136) - Prevent accidental indexing of web interface by web crawlers (#1151 https://github.com/Graylog2/graylog2-web-interface/issues/1151) - Validate grok pattern name on the client to avoid duplicate names ( #937 https://github.com/Graylog2/graylog2-server/issues/937) - Add message journal usage to nodes overview page (#1083 https://github.com/Graylog2/graylog2-web-interface/issues/1083) - Properly format numbers according to locale (#1128 https://github.com/Graylog2/graylog2-web-interface/issues/1128, #1129 https://github.com/Graylog2/graylog2-web-interface/issues/1129) Thanks to everyone who helped creating this release by using Graylog and reporting bugs and regressions to the mailing list and on GitHub. The official RPM and DEB packages, as well as the virtual machine images have been updated with the new version. As always, if you find any bugs in this release, please open an issue on GitHub at https://github.com/Graylog2/graylog2-server/issues. Best regards, Jochen (in the name of the Graylog team) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: [ANN] Graylog2 0.92.0 released
How long until I can yum update? On Monday, December 1, 2014 1:58:12 AM UTC-8, Jochen Schalanda wrote: Hi everyone, after an extended beta and release candidate phase we just released Graylog2 0.92.0. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: [ANN] Graylog2 0.92.0 released
I edited the repo file and changed 0.91 to 0.92 On Wednesday, December 3, 2014 6:13:15 PM UTC-8, Zi Dvbelju wrote: You'll need to remove the old repository and add the new one (old repository references 91). At least that's what I had to do. yum remove graylog2-0.91-repository-el6-1.1.0-1.noarch Then you can add the new one: rpm -Uvh https://packages.graylog2.org/repo/packages/graylog2-0.92-repository-el6_latest.rpm Finally, yum update graylog2-server On Wednesday, December 3, 2014 9:07:13 PM UTC-5, Mark Moorcroft wrote: How long until I can yum update? On Monday, December 1, 2014 1:58:12 AM UTC-8, Jochen Schalanda wrote: Hi everyone, after an extended beta and release candidate phase we just released Graylog2 0.92.0. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Mirror server?
Is the GELF data stream encrypted? Probably 95% of the reason we even use fluentd/elastic/graylog is to meet the requirement to encrypt the data over the wire. I pretty much do all the filtering and extraction in fluentd on the secure_senders. I think pretty much any government or corporate entity these days has a requirement to encrypt everything over the wire. So I'm a little confused why encryption always seems to be an afterthought or an optional add-on. Even Splunk does a lousy job handling encryption. On Thursday, November 13, 2014 3:23:19 AM UTC-8, Jochen Schalanda wrote: Hi Mark, I think the easiest setup for your requirements would be to forward the messages processed by the locked down Graylog2 server to the user-facing Graylog2 server via the GELF output. This way you could filter messages or run extractors in exactly one place and just forward the final messages to the instance users can run searches on. If you were sending the log messages to both Graylog2 instances directly, you would need to set up filters and extractors on both of them and keep them in sync. Cheers, Jochen Am Mittwoch, 12. November 2014 22:06:48 UTC+1 schrieb Mark Moorcroft: Question for the room: If I have a need to provide a LOCKED down graylog server for compliance, and second one that someone can actually use to do searches and monitor our systems. Is it considered a best practice to mirror the outputs from all of the systems to two nearly identical VM's? We currently use fluentd to push the logs. Or is it better to have one graylog server push (rebroadcast) all of it's data to a second one. This is not for failover, but mostly because the current graylog authentication setup so severely limits what a read only user can do unless someone sets up Streams, which I'm virtually certain nobody here will take the time to do. I hope this isn't an RTFM situation. If so I apologize in advance. It doesn't appear to me that Radio has anything to do with this need. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Mirror server?
Question for the room: If I have a need to provide a LOCKED down graylog server for compliance, and second one that someone can actually use to do searches and monitor our systems. Is it considered a best practice to mirror the outputs from all of the systems to two nearly identical VM's? We currently use fluentd to push the logs. Or is it better to have one graylog server push (rebroadcast) all of it's data to a second one. This is not for failover, but mostly because the current graylog authentication setup so severely limits what a read only user can do unless someone sets up Streams, which I'm virtually certain nobody here will take the time to do. I hope this isn't an RTFM situation. If so I apologize in advance. It doesn't appear to me that Radio has anything to do with this need. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Re: Root password shasum change fails
Thanks, in my haste I had failed to single quote the input. And changing the password allowed me to get away without doing so. Obviously PEBKAC though, and not a bug. Apologies On Friday, November 7, 2014 1:19:53 AM UTC-8, Jochen Schalanda wrote: Hi Mark, I just tried to reproduce this bug in Graylog2 but without success. If you've used the shell to generate the SHA256 sum of your admin password, please make sure to properly escape the input. Example: $ echo -n 'my$password' | shasum -a 256 63b9a3f67f9d896dd7f52fdeb283fab2aa2d692521673bd6caf0bf04c2a842d2 - Without the single quotes around the password, the shell would try to interpolate the string with the environment variable *$password* and the resulting hash would be for the string my (because there usually is no such environment variable). Now I get to go back and change it in mongo and other places :-( The password for the authentication against MongoDB is not related to the admin password of Graylog2 and you usually should use different credentials for these things. Cheers, Jochne -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Re: Root password shasum change fails
Generally true, but when you are setting something up to hand off to a manager the game changes. So I just use a long random hash that he can store in case it's needed some day. On Fri, Nov 7, 2014 at 1:19 AM, Jochen Schalanda joc...@torch.sh wrote: The password for the authentication against MongoDB is not related to the admin password of Graylog2 and you usually should use different credentials for these things. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Root password shasum change fails
I am in the process of resetting all the passwords on our graylog server to hand over to the system owner. My old password works with the shasum instructions provided, but the new 14 character random one fails every time. Both the old and the new have special characters, but the new one will never authenticate. Any idea what ay be going on? I don't believe you can uses delimiters or quotes with shasum? The new one has $ and * in it, where the old one does not. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Root password shasum change fails
Looks like you can't use $. On Thursday, November 6, 2014 1:40:01 PM UTC-8, Mark Moorcroft wrote: I am in the process of resetting all the passwords on our graylog server to hand over to the system owner. My old password works with the shasum instructions provided, but the new 14 character random one fails every time. Both the old and the new have special characters, but the new one will never authenticate. Any idea what ay be going on? I don't believe you can uses delimiters or quotes with shasum? The new one has $ and * in it, where the old one does not. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Re: Root password shasum change fails
I had a dollar in the password itself. Since removing the dollar I have it working. Now I get to go back and change it in mongo and other places :-( On Thu, Nov 6, 2014 at 2:03 PM, Jochen Schalanda joc...@schalanda.name wrote: Hi Mark On 06.11.2014 22:46, Mark Moorcroft wrote: Looks like you can't use $. Just to clarify this, do you have a '$' in your password or in the SHA256 of your password? Cheers, Jochen On Thursday, November 6, 2014 1:40:01 PM UTC-8, Mark Moorcroft wrote: I am in the process of resetting all the passwords on our graylog server to hand over to the system owner. My old password works with the shasum instructions provided, but the new 14 character random one fails every time. Both the old and the new have special characters, but the new one will never authenticate. Any idea what ay be going on? I don't believe you can uses delimiters or quotes with shasum? The new one has $ and * in it, where the old one does not. -- You received this message because you are subscribed to a topic in the Google Groups graylog2 group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/graylog2/sjCVaaM3B94/unsubscribe. To unsubscribe from this group and all its topics, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Graylog2 capabilities
You have to be an admin to configure or save a dashboard. There seems to be no way to have control of the search without having access to disable or remove inputs. It makes no sense to me at all. On Monday, November 3, 2014 2:15:46 PM UTC-8, Mave Zero wrote: Hello, we are looking into how we can best use Graylog vs some of the existing technology in place. In order to understand this better, I need some information about some features: - Logging of multiple sources/channels - Splitting various text logs into fields - Support of JSON records in logs - Search by split fields of text logs and/or any JSON fields - Easy stats (facets) on search results (list of unique/top values in each field) - Graphs/Charts - histograms and other charts by different fields (e.g. average response-time by page-type by day) - User Ability to configure and save dashboards -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Rsync backup?
OK, disregard, I will be reporting to the backuppc forum since it appears any file in /var/log may abort the process. If I filter out /var/log I get success. On Tuesday, October 21, 2014 1:57:46 PM UTC-7, Mark Moorcroft wrote: I am just now discovering that I can't rsync backup my graylog2/elasticsearch/fluentd system unless I filter out /var/log. There must be simply too much file activity for rsync to get a handle on it. Presumably it could be possible to use some sort of file system snapshot strategy (like ZFS might use), but I was wondering what methods people may be using. I was planning to cron a scheduled mongodb dump to back up. But we also need to back up some stuff in /var/log. It will be an issue to kill services to get this done, but I really don't want to go all the way to some sort of failover setup either. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Server fails to start
I rebooted my graylog2 box today and now I get the following: [root@graylog ~]# service graylog2-server start Starting graylog2-server: [ OK ] [root@graylog ~]# Exception in thread main java.lang.AssertionError: data were read beyond record size, check your serializer Followed by 2 pages of java errors. Anybody have any ideas? -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Server fails to start
Thanks, I reverted my VM image and solved it that way. On Wednesday, October 22, 2014 3:58:50 PM UTC-7, lennart wrote: Hey Mark, can you post those Java errors/stacktraces? Thanks, Lennart On Thu, Oct 23, 2014 at 12:10 AM, Mark Moorcroft pla...@gmail.com javascript: wrote: I rebooted my graylog2 box today and now I get the following: [root@graylog ~]# service graylog2-server start Starting graylog2-server: [ OK ] [root@graylog ~]# Exception in thread main java.lang.AssertionError: data were read beyond record size, check your serializer Followed by 2 pages of java errors. Anybody have any ideas? -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Export log
Amen, I agree 100%. On Monday, July 28, 2014 11:44:44 PM UTC-7, Dennis Brouwer wrote: Hi All, We are seriously looking into Graylog but for archiving purposes we would like to export the logging in Graylog back to normal Syslog format so we can GZIP it (we need to save logging for a year). But, there isn't an option to export it. Do you guys have any idea on how te realize this? Dennis -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] No Search in non-admin account?
Running the repo RPM version of GL2 from yesterday.
I finally got around to adding our non-admin accounts in GL. When you log
in there is no “Search” function anywhere to be found. And if you enter a
search URL:
http://xxx.xxx.nasa.gov:9000/search?rangetype=relativefields=width=1280relative=300from=to=q=#?fields=source%2Cmessage
http://graylog.ndc.nasa.gov:9000/search?rangetype=relativefields=width=1280relative=300from=to=q=#?fields=source%2Cmessage
You get:
(You caused a org.graylog2.restclient.lib.APIException. API call failed GET
http://@127.0.0.1:12900/search/universal/relative?
range=300range_type=relativequery=*limit=100offset=0filter=*sort=timestamp:desc
returned 403 Forbidden body: {type:ApiError,message:Not authorized”})
Reason: There was a problem with your search. We expected HTTP 200, but got
a HTTP 403.
Th documentation says nothing about user level differences that I could
find. I presume a non-admin is supposed to be able to search, but I don’t
see “Search” on the top menu at all. And the default page in the admin
account seems to be Search. The default in a non-admin account appears to
be Streams.
graylog2-web-interface v0.21.0-beta4 (Oracle Corporation 1.7.0_65 / Linux
2.6.32-431.23.3.el6.x86_64) on xxx.xxx.nasa.gov
--
You received this message because you are subscribed to the Google Groups
graylog2 group.
To unsubscribe from this group and stop receiving emails from it, send an email
to graylog2+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Re: Newbie to graylog2
I have wondered that myself. On Friday, August 22, 2014 7:48:33 AM UTC-7, Foobar Geez wrote: A few questions: - What is the typical release cycle or how soon GL2 typically supports new Elasticsearch versions? I see from GL2 release notes that it supports v0.90 of Elasticsearch and the latest version seems to be v1.3.2 (seems like a big delta). -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] 443 as non-root?
All CentOS here. On Tue, Aug 26, 2014 at 11:05 AM, Lennart Koopmann lenn...@torch.sh wrote: Another think to look at when on Ubuntu: http://manpages.ubuntu.com/manpages/hardy/man1/authbind.1.html On Tue, Aug 26, 2014 at 8:02 PM, Mark Moorcroft plak...@gmail.com wrote: I have read various strategies here to run the web interface with 443 access as non-root, such as iptables redirects etc. Apache and postfix both manage to run as non-root on low ports. So I was wondering if it's on the radar to allow this with GL2? I realize apache and postfix manage this trick through various hoops jumped through. But at the end of the day I wonder if you will eventually be able to install GL2 web with 443 enabled and it just works? privileged low port access discussion -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to a topic in the Google Groups graylog2 group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/graylog2/L-Zag1e0ob4/unsubscribe. To unsubscribe from this group and all its topics, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
