Hi! You can also check this things: 1. Is logstash process user has properly permissions to read Tomcat's log? 2. Is SELinux enabled? 2. Is iptables running? Is there accept rules for GELF input?
-- Sincerely, Arkadiy Shinkarev e-mail: [email protected] Cell.: +7 (926) 147-51-87 2014-05-25 22:30 GMT+04:00 Joseph DJOMEDA <[email protected]>: > Hello I have tried your example and even tried java log of a different > application on a totally different box and still I can't see anything on > graylog. Is there really something I am not doing right? I have this time > used the grokdebug.herokuapp.com application to check my patterns. > > with debug on , this is what I have : > _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] > {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} > _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] > {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} > _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] > {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} > _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] > {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} > _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] > {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} > _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] > {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} > > > my log looks like something like this: > 25 May 2014 ;17:37:56.159 [ajp-bio-8009-exec-8] INFO > c.d.m.web.rest.RestServiceController - [activity][login][ > email:[email protected]] > 25 May 2014 ;17:42:24.425 [ajp-bio-8009-exec-9] INFO > c.d.m.web.rest.RestServiceController - [activity][login][ > email:[email protected]] > 25 May 2014 ;17:42:28.988 [ajp-bio-8009-exec-9] INFO > c.d.m.web.rest.RestServiceController - [activity][login][ > email:[email protected]] > > my filter also looks like this: > > %{MONTHDAY:day} %{MONTH:month} %{YEAR:year} ;%{TIME} \[%{DATA:thread}\] > %{LOGLEVEL:loglevel} %{GREEDYDATA:class} - %{GREEDYDATA:stageinfo} > > > I am really desperate and out of solution. my graylog frontend system page > looks like this: > > > <https://lh5.googleusercontent.com/-nyYLA0Q4DNQ/U4I0jRQVx6I/AAAAAAAAA5U/i7-2Vw9eVxU/s1600/graylog2.png> > > > <https://lh3.googleusercontent.com/-mrQ051tt6uk/U4I0C-T97II/AAAAAAAAA5I/Gn1IE6zOmmE/s1600/graylog1.png> > > > <https://lh5.googleusercontent.com/-N0LykjNmwBU/U4I1ryzLa0I/AAAAAAAAA5k/d8RodsxaiQc/s1600/graylog3.png> > > > <https://lh3.googleusercontent.com/-EIm-j1Dp728/U4I2lw-15sI/AAAAAAAAA54/OXUFc1t_82U/s1600/grokdebug.png> > > > > On Thursday, May 15, 2014 10:58:41 AM UTC, Arkadiy Shinkarev wrote: >> >> You have an error in grok pattern, try this one: >> %{DATESTAMP:datestamp} %{LOGLEVEL:loglevel} \[%{GREEDYDATA:thread}\] >> \[%{GREEDYDATA:classinfo}\] %{GREEDYDATA:loginfo} >> >> Later, you can use Grok Debugger - http://grokdebug.herokuapp.com/ >> >> >> On Saturday, April 19, 2014 4:19:57 PM UTC+4, Joseph DJOMEDA wrote: >>> >>> Hello Good People, >>> >>> I am coming from splunk background with even little experience on it. >>> But I am having issue getting basic stuff done. I have graylog2 server and >>> web interface running fine let's say on IP :112. I have a java application >>> running on a server IP : 27. the log of the app is of the type shown below. >>> I know it needs some cleanups but I am more concerned about sending >>> something to graylog2: >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> *2014-04-01 21:54:17,398 INFO [Thread-2] >>> [org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider] - >>> HHH000006: Autocommit mode: true2014-04-01 21:54:17,399 WARN [Thread-2] >>> [org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider] - >>> HHH000148: No JDBC Driver class was specified by property >>> hibernate.connection.driver_class2014-04-01 21:54:17,425 INFO [Thread-2] >>> [com.mchange.v2.log.MLog] - MLog clients using log4j logging.2014-04-01 >>> 21:54:17,545 INFO [Thread-2] [com.mchange.v2.c3p0.C3P0Registry] - >>> Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true; >>> trace: 10]2014-04-01 21:54:17,930 INFO [Thread-2] >>> [com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource] - Initializing c3p0 >>> pool... com.mchange.v2.c3p0.PoolBackedDataSource@d678e16f [ >>> connectionPoolDataSource -> >>> com.mchange.v2.c3p0.WrapperConnectionPoolDataSource@7bb4a24 [ >>> acquireIncrement -> 2, acquireRetryAttempts -> 30, acquireRetryDelay -> >>> 1000, autoCommitOnClose -> false, automaticTestTable -> null, >>> breakAfterAcquireFailure -> false, checkoutTimeout -> 0, >>> connectionCustomizerClassName -> null, connectionTesterClassName -> >>> com.mchange.v2.c3p0.impl.DefaultConnectionTester, >>> debugUnreturnedConnectionStackTraces -> false, factoryClassLocation -> >>> null, forceIgnoreUnresolvedTransactions -> false, identityToken -> >>> nm1r17918k7ta81op67fy|71b3cc1f, idleConnectionTestPeriod -> 300, >>> initialPoolSize -> 3, maxAdministrativeTaskTime -> 0, maxConnectionAge -> >>> 0, maxIdleTime -> 60000, maxIdleTimeExcessConnections -> 0, maxPoolSize -> >>> 40, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 2, >>> nestedDataSource -> com.mchange.v2.c3p0.DriverManagerDataSource@2ba1b5de [ >>> description -> null, driverClass -> null, factoryClassLocation -> null, >>> identityToken -> nm1r17918k7ta81op67fy|67f095ba, jdbcUrl -> >>> jdbc:mysql://localhost:3306/do_my_app, properties -> {user=******, >>> password=******, autocommit=true, driverClassName=com.mysql.jdbc.Driver, >>> release_mode=auto} ], preferredTestQuery -> null, propertyCycle -> 0, >>> testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, >>> unreturnedConnectionTimeout -> 0, usesTraditionalReflectiveProxies -> >>> false; userOverrides: {} ], dataSourceName -> null, factoryClassLocation -> >>> null, identityToken -> nm1r17918k7ta81op67fy|51af67f9, numHelperThreads -> >>> 3 ]2014-04-01 21:54:18,453 INFO [Thread-2] [org.hibernate.dialect.Dialect] >>> - HHH000400: Using dialect: org.hibernate.dialect.MySQLDialect * >>> >>> I got logstash on the server IP: 27 and have its configuration shown >>> below: >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> *#logstash for IP 27 server# logstash.conf fileinput { file { type >>> => my_app path => ["/opt/tomcatinstances/my_app/logs/catalina.out"] >>> }} filter { grok { match => {"message" => %{DATESTAMP_EVENTLOG:datestamp} >>> %{LOGLEVEL:loglevel} \[%{WORD:thread}\] \[%{GREEDYDATA:classinfo}\] >>> %{WORD:loginfo}"} } } output { gelf { host => "xxx.xxx.xxx.112" facility => >>> "%{@type}" custom_fields => ["environment", "production"] }}* >>> >>> when I run bin/logstash --debug -f logstasb.conf I have bunch of the >>> following >>> >>> >>> >>> >>> >>> >>> >>> *=>"my_server_name", >>> "path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", >>> "tags"=>["_grokparsefailure"]}, "tags"]}>, @data={"message"=>" >>> voucher_type vouchertyp0_", "@version"=>"1", >>> "@timestamp"=>"2014-04-19T10:55:17.333Z", "type"=>"my_app", >>> "host"=>"my_server_name", >>> "path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", >>> "tags"=>["_grokparsefailure"]}, @cancelled=false>, :level=>:debug, >>> :file=>"logstash/filters/grok.rb", :line=>"310"}["Sending GELF event", >>> {"short_message"=>" from", "full_message"=>" from", >>> "host"=>"my_server_name", "facility"=>"%{@type}", "_type"=>"my_app", >>> "_path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", >>> "_tags"=>"_grokparsefailure", "_environment"=>"production", "level"=>6}] >>> {:level=>:debug, :file=>"logstash/outputs/gelf.rb", :line=>"203"}output >>> received {:event=>{"message"=>" voucher_type vouchertyp0_", >>> "@version"=>"1", "@timestamp"=>"2014-04-19T10:55:17.333Z", >>> "type"=>"my_app", "host"=>"my_server_name", >>> "path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", >>> "tags"=>["_grokparsefailure"]}, :level=>:debug, :file=>"(eval)", >>> :line=>"34"}["Sending GELF event", {"short_message"=>" voucher_type >>> vouchertyp0_", "full_message"=>" voucher_type vouchertyp0_", >>> "host"=>"my_server_name", "facility"=>"%{@type}", "_type"=>"my_app", >>> "_path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", >>> "_tags"=>"_grokparsefailure", "_environment"=>"production", "level"=>6}] >>> {:level=>:debug, :file=>"logstash/outputs/gelf.rb", >>> :line=>"203"}_discover_file_glob: >>> /opt/tomcatinstances/my_app/logs/catalina.out: glob is: >>> ["/opt/tomcatinstances/my_app/logs/catalina.out"] {:level=>:debug, >>> :file=>"filewatch/watch.rb", :line=>"117"}_discover_file_glob: >>> /opt/tomcatinstances/my_app/logs/catalina.out: glob is: >>> ["/opt/tomcatinstances/my_app/logs/catalina.out"] {:level=>:debug, >>> :file=>"filewatch/watch.rb", :line=>"117"}_discover_file_glob: >>> /opt/tomcatinstances/my_app/logs/catalina.out: glob is: >>> ["/opt/tomcatinstances/my_app/logs/catalina.out"] {:level=>:debug, >>> :file=>"filewatch/watch.rb", :line=>"117"}* >>> >>> I do not think anything good is happening , besides graylog still >>> complaining about not receiving any input and it's annoyingly blinking >>> >>> *There is a node without any running inputs. 5 days ago * >>> * There is a node without any running inputs. This means that you are >>> not receiving any messages from this node at this point in time. This is >>> most probably an indication of an error or misconfiguration. You can click >>> here <http://188.138.98.112:9000/system/inputs> to solve this* >>> >>> Can anyone please shed some light? Thank you >>> >> -- > You received this message because you are subscribed to a topic in the > Google Groups "graylog2" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/graylog2/uC4bV_WZXDo/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
