Hello Arkadiy , thanks for your effort my answers *inline*
On Monday, 26 May 2014 06:31:13 UTC, Arkadiy Shinkarev wrote: > > Hi! > You can also check this things: > 1. Is logstash process user has properly permissions to read Tomcat's log? > *Yes * > 2. Is SELinux enabled? > *~/graylog2-web-interface-0.20.1# sestatusThe program 'sestatus' is currently not installed. You can install it by typing:apt-get install policycoreutils* > 2. Is iptables running? Is there accept rules for GELF input? > *~/graylog2-web-interface-0.20.1# netstat -ano | grep 12201udp 0 0 0.0.0.0:12201 0.0.0.0:* off (0.00/0/0)To Action From-- ------ ----80 ALLOW IN Anywhere443 ALLOW IN Anywhere3306 ALLOW IN Anywhere9200 ALLOW IN Anywhere53 ALLOW IN Anywhere12900/tcp ALLOW IN 127.0.0.19300/tcp ALLOW IN 127.0.0.112201/udp ALLOW IN Anywhere80 ALLOW IN Anywhere (v6)443 ALLOW IN Anywhere (v6)3306 ALLOW IN Anywhere (v6)9200 ALLOW IN Anywhere (v6)53 ALLOW IN Anywhere (v6)12201/udp ALLOW IN Anywhere (v*6) Only thing is that telnet doens't seem to connect. I have no Idea how logstash talks to graylog etc *~$ telnet xxx.xxx.xxx.xxx 12201Trying xxx.xxx.xxx.xxx...telnet: Unable to connect to remote host: Connection timed out* Thanks > -- > Sincerely, > Arkadiy Shinkarev > e-mail: [email protected] <javascript:> > Cell.: +7 (926) 147-51-87 > > > 2014-05-25 22:30 GMT+04:00 Joseph DJOMEDA <[email protected]<javascript:> > >: > >> Hello I have tried your example and even tried java log of a different >> application on a totally different box and still I can't see anything on >> graylog. Is there really something I am not doing right? I have this time >> used the grokdebug.herokuapp.com application to check my patterns. >> >> with debug on , this is what I have : >> _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] >> {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} >> _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] >> {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} >> _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] >> {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} >> _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] >> {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} >> _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] >> {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} >> _discover_file_glob: /var/lib/tomcat7/b_log.log: glob is: [] >> {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"} >> >> >> my log looks like something like this: >> 25 May 2014 ;17:37:56.159 [ajp-bio-8009-exec-8] INFO >> c.d.m.web.rest.RestServiceController - [activity][login][ >> email:[email protected] <javascript:>] >> 25 May 2014 ;17:42:24.425 [ajp-bio-8009-exec-9] INFO >> c.d.m.web.rest.RestServiceController - [activity][login][ >> email:[email protected] <javascript:>] >> 25 May 2014 ;17:42:28.988 [ajp-bio-8009-exec-9] INFO >> c.d.m.web.rest.RestServiceController - [activity][login][ >> email:[email protected] <javascript:>] >> >> my filter also looks like this: >> >> %{MONTHDAY:day} %{MONTH:month} %{YEAR:year} ;%{TIME} \[%{DATA:thread}\] >> %{LOGLEVEL:loglevel} %{GREEDYDATA:class} - %{GREEDYDATA:stageinfo} >> >> >> I am really desperate and out of solution. my graylog frontend system >> page looks like this: >> >> >> <https://lh5.googleusercontent.com/-nyYLA0Q4DNQ/U4I0jRQVx6I/AAAAAAAAA5U/i7-2Vw9eVxU/s1600/graylog2.png> >> >> >> <https://lh3.googleusercontent.com/-mrQ051tt6uk/U4I0C-T97II/AAAAAAAAA5I/Gn1IE6zOmmE/s1600/graylog1.png> >> >> >> <https://lh5.googleusercontent.com/-N0LykjNmwBU/U4I1ryzLa0I/AAAAAAAAA5k/d8RodsxaiQc/s1600/graylog3.png> >> >> >> <https://lh3.googleusercontent.com/-EIm-j1Dp728/U4I2lw-15sI/AAAAAAAAA54/OXUFc1t_82U/s1600/grokdebug.png> >> >> >> >> On Thursday, May 15, 2014 10:58:41 AM UTC, Arkadiy Shinkarev wrote: >>> >>> You have an error in grok pattern, try this one: >>> %{DATESTAMP:datestamp} %{LOGLEVEL:loglevel} \[%{GREEDYDATA:thread}\] >>> \[%{GREEDYDATA:classinfo}\] %{GREEDYDATA:loginfo} >>> >>> Later, you can use Grok Debugger - http://grokdebug.herokuapp.com/ >>> >>> >>> On Saturday, April 19, 2014 4:19:57 PM UTC+4, Joseph DJOMEDA wrote: >>>> >>>> Hello Good People, >>>> >>>> I am coming from splunk background with even little experience on it. >>>> But I am having issue getting basic stuff done. I have graylog2 server and >>>> web interface running fine let's say on IP :112. I have a java application >>>> running on a server IP : 27. the log of the app is of the type shown >>>> below. >>>> I know it needs some cleanups but I am more concerned about sending >>>> something to graylog2: >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *2014-04-01 21:54:17,398 INFO [Thread-2] >>>> [org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider] - >>>> HHH000006: Autocommit mode: true 2014-04-01 21:54:17,399 WARN [Thread-2] >>>> [org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider] - >>>> HHH000148: No JDBC Driver class was specified by property >>>> hibernate.connection.driver_class 2014-04-01 21:54:17,425 INFO [Thread-2] >>>> [com.mchange.v2.log.MLog] - MLog clients using log4j logging.2014-04-01 >>>> 21:54:17,545 INFO [Thread-2] [com.mchange.v2.c3p0.C3P0Registry] - >>>> Initializing c3p0-0.9.1 [built 16-January-2007 14:46:42; debug? true; >>>> trace: 10] 2014-04-01 21:54:17,930 INFO [Thread-2] >>>> [com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource] - Initializing >>>> c3p0 >>>> pool... com.mchange.v2.c3p0.PoolBackedDataSource@d678e16f [ >>>> connectionPoolDataSource -> >>>> com.mchange.v2.c3p0.WrapperConnectionPoolDataSource@7bb4a24 [ >>>> acquireIncrement -> 2, acquireRetryAttempts -> 30, acquireRetryDelay - > >>>> 1000, autoCommitOnClose -> false, automaticTestTable -> null, >>>> breakAfterAcquireFailure -> false, checkoutTimeout -> 0, >>>> connectionCustomizerClassName -> null, connectionTesterClassName -> >>>> com.mchange.v2.c3p0.impl.DefaultConnectionTester, >>>> debugUnreturnedConnectionStackTraces -> false, factoryClassLocation -> >>>> null, forceIgnoreUnresolve dTransactions -> false, identityToken -> >>>> nm1r17918k7ta81op67fy|71b3cc1f, idleConnectionTestPeriod -> 300, >>>> initialPoolSize -> 3, maxAdministrativeTaskTime -> 0, maxConnectionAge -> >>>> 0, maxIdleTime -> 60000, maxIdleTimeExcessConnections -> 0, maxPoolSize -> >>>> 40, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 2, >>>> nested DataSource -> com.mchange.v2.c3p0.DriverManagerDataSource@2ba1b5de >>>> [ >>>> description -> null, driverClass -> null, factoryClassLocation -> null, >>>> identityToken -> nm1r17918k7ta81op67fy|67f095ba, jdbcUrl -> >>>> jdbc:mysql://localhost:3306/do_my_app, properties -> {user=******, >>>> password=******, autocommit=true, driverClassName=com.mysql.jdbc.Dri ver, >>>> release_mode=auto} ], preferredTestQuery -> null, propertyCycle -> 0, >>>> testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, >>>> unreturnedConnectionTimeout -> 0, usesTraditionalReflectiveProxies -> >>>> false; userOverrides: {} ], dataSourceName -> null, factoryClassLocation >>>> -> >>>> null, identityToken -> nm1r17918k7ta81op67fy|5 1af67f9, numHelperThreads >>>> -> >>>> 3 ]2014-04-01 21:54:18,453 INFO [Thread-2] [org.hibernate.dialect.Dialect] >>>> - HHH000400: Using dialect: org.hibernate.dialect.MySQLDialect * >>>> >>>> I got logstash on the server IP: 27 and have its configuration shown >>>> below: >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *#logstash for IP 27 server# logstash.conf fileinput { file { type >>>> => my_app path => ["/opt/tomcatinstances/my_app/logs/catalina.out"] >>>> }} filter { grok { match => {"message" => >>>> %{DATESTAMP_EVENTLOG:datestamp} >>>> %{LOGLEVEL:loglevel} \[%{WORD:thread}\] \[%{GREEDYDATA:classinfo}\] >>>> %{WORD:loginfo}"} } } output { gelf { host => "xxx.xxx.xxx.112" facility >>>> => "%{@type}" custom_fields => ["environment", "production"] }}* >>>> >>>> when I run bin/logstash --debug -f logstasb.conf I have bunch of the >>>> following >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *=>"my_server_name", >>>> "path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", >>>> "tags"=>["_grokparsefailure"]}, "tags"]}>, @data={"message"=>" >>>> voucher_type vouchertyp0_", "@version"=>"1", >>>> "@timestamp"=>"2014-04-19T10:55:17.333Z", "type"=>"my_app", >>>> "host"=>"my_server_name", >>>> "path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", >>>> "tags"=>["_grokparsefailure"]}, @cancelled=false>, :level=>:debug, >>>> :file=>"logstash/filters/grok.rb", :line=>"310"} ["Sending GELF event", >>>> {"short_message"=>" from", "full_message"=>" from", >>>> "host"=>"my_server_name", "facility"=>"%{@type}", "_type"=>"my_app", >>>> "_path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", >>>> "_tags"=>"_grokparsefailure", "_environment"=>"production", "level"=>6}] >>>> {:level=>:debug, :file=>"logstash/outputs/gelf.rb", :line=>"203"} output >>>> received {:event=>{"message"=>" voucher_type vouchertyp0_", >>>> "@version"=>"1", "@timestamp"=>"2014-04-19T10:55:17.333Z", >>>> "type"=>"my_app", "host"=>"my_server_name", >>>> "path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", >>>> "tags"=>["_grokparsefailure"]}, :level=>:debug, :file=>"(eval)", >>>> :line=>"34"} ["Sending GELF event", {"short_message"=>" >>>> voucher_type >>>> vouchertyp0_", "full_message"=>" voucher_type vouchertyp0_", >>>> "host"=>"my_server_name", "facility"=>"%{@type}", "_type"=>"my_app", >>>> "_path"=>"/opt/tomcatinstances/my_app/logs/catalina.out", >>>> "_tags"=>"_grokparsefailure", "_environment"=>"production", "level"=>6}] >>>> {:level=>:debug, :file=>"logstash/outputs/gelf.rb", :line=>"203"} >>>> _discover_file_glob: /opt/tomcatinstances/my_app/logs/catalina.out: glob >>>> is: ["/opt/tomcatinstances/my_app/logs/catalina.out"] {:level=>:debug, >>>> :file=>"filewatch/watch.rb", :line=>"117"} _discover_file_glob: >>>> /opt/tomcatinstances/my_app/logs/catalina.out: glob is: >>>> ["/opt/tomcatinstances/my_app/logs/catalina.out"] {:level=>:debug, >>>> :file=>"filewatch/watch.rb", :line=>"117"} _discover_file_glob: >>>> /opt/tomcatinstances/my_app/logs/catalina.out: glob is: >>>> ["/opt/tomcatinstances/my_app/logs/catalina.out"] {:level=>:debug, >>>> :file=>"filewatch/watch.rb", :line=>"117"}* >>>> >>>> I do not think anything good is happening , besides graylog still >>>> complaining about not receiving any input and it's annoyingly blinking >>>> >>>> *There is a node without any running inputs. 5 days ago * >>>> * There is a node without any running inputs. This means that you are >>>> not receiving any messages from this node at this point in time. This is >>>> most probably an indication of an error or misconfiguration. You can click >>>> here <http://188.138.98.112:9000/system/inputs> to solve this* >>>> >>>> Can anyone please shed some light? Thank you >>>> >>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "graylog2" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/graylog2/uC4bV_WZXDo/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "graylog2" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
