Hi Jochen,
this bad behavior of graylog started when I activated the Cisco Catalyst
plugin, without this component there are no messages to Elasticsearch so
why this plugin introduce this kind of exceptions ? Any ideas ?
Graylog is very powerful but it is also very complex and at the moment I can
not make new indexes, but look inside /var/log/graylog/elasticsearch
I find graylog.log file:
Caused by: java.lang.NumberFormatException: For input string: "190>1318998:
cipg01.pg.infn.it: [syslog@9 s_id ="cipg01.pg.infn.it:11001"]: 1319110: Aug
31 15:34:35.876: %SEC-6-IPACCESSLOGP: list 101 denied tcp
116.27.84.102(5752) -"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:589)
at java.lang.Long.parseLong(Long.java:631)
at
org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:145)
at
org.elasticsearch.index.mapper.core.LongFieldMapper.innerParseCreateField(LongFieldMapper.java:275)
at
org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:241)
at
org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:321)
... 22 more
[2016-08-31 15:34:39,459][DEBUG][action.bulk ] [Horus]
[graylog_2][0] failed to execute bulk item (index) index
{[graylog_deflector][message][a9ae21f1-6f7f-11e6-b25f-001a4ab8e90b],
source[{"level":"190>1318999: cipg01.pg.infn.it: [syslog@9 s_id
=\"cipg01.pg.infn.it:11001\"]: 1319111: Aug 31 15:34:36.958:
%SEC-6-IPACCESSLOGP: list 101 denied tcp 128.65.184.100(36183)
-","gl2_remote_ip":"10.0.0.9","gl2_remote_port":65416,"streams":[],"source":"cipg01.management","local_level":6,"message":"list
101 denied tcp 128.65.184.100(36183) -> 193.205.222.83(23), 1
packet","gl2_source_input":"57ab1eb493802a03b1c9da67","local_facility":"sec","mnemonic":"ipaccesslogp","gl2_source_node":"20fb3024-572f-411c-a58e-b2cbba370bf9","facility":"190>1318999:
cipg01.pg.infn.it: [syslog@9 s_id =\"cipg01.pg.infn.it:11001\"]: 1319111:
Aug 31 15:34:36.958: %SEC-6-IPACCESSLOGP: list 101 denied tcp
128.65.184.100(36183) -","timestamp":"2016-08-31 13:34:37.967"}]}
MapperParsingException[failed to parse [level]]; nested:
NumberFormatException[For input string: "190>1318999: cipg01.pg.infn.it:
[syslog@9 s_id ="cipg01.pg.infn.it:11001"]: 1319111: Aug 31 15:34:36.958:
%SEC-6-IPACCESSLOGP: list 101 denied tcp 128.65.184.100(36183) -"];
at
org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:329)
at
org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:309)
at
org.elasticsearch.index.mapper.DocumentParser.parseValue(DocumentParser.java:436)
at
org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:262)
at
org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:122)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:309)
at
org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:580)
at
org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:559)
at
org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:212)
at
org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:224)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:326)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:119)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
at
org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:639)
at
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at
org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
at
org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
at
org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
at
org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
at
org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NumberFormatException: For input string: "190>1318999:
cipg01.pg.infn.it: [syslog@9 s_id ="cipg01.pg.infn.it:11001"]: 1319111: Aug
31 15:34:36.958: %SEC-6-IPACCESSLOGP: list 101 denied tcp
128.65.184.100(36183) -"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:589)
at java.lang.Long.parseLong(Long.java:631)
at
org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:145)
at
org.elasticsearch.index.mapper.core.LongFieldMapper.innerParseCreateField(LongFieldMapper.java:275)
at
org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:241)
at
org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:321)
... 22 more
[2016-08-31 15:34:39,476][DEBUG][action.bulk ] [Horus]
[graylog_2][2] failed to execute bulk item (index) index
{[graylog_deflector][message][aa63dd60-6f7f-11e6-b25f-001a4ab8e90b],
source[{"level":"190>1319000: cipg01.pg.infn.it: [syslog@9 s_id
=\"cipg01.pg.infn.it:11001\"]: 1319112: Aug 31 15:34:38.149:
%SEC-6-IPACCESSLOGP: list 101 denied tcp 1.55.57.166(24103)
-","gl2_remote_ip":"10.0.0.9","gl2_remote_port":65416,"streams":[],"source":"cipg01.management","local_level":6,"message":"list
101 denied tcp 1.55.57.166(24103) -> 193.205.222.106(23), 1
packet","gl2_source_input":"57ab1eb493802a03b1c9da67","local_facility":"sec","mnemonic":"ipaccesslogp","gl2_source_node":"20fb3024-572f-411c-a58e-b2cbba370bf9","facility":"190>1319000:
cipg01.pg.infn.it: [syslog@9 s_id =\"cipg01.pg.infn.it:11001\"]: 1319112:
Aug 31 15:34:38.149: %SEC-6-IPACCESSLOGP: list 101 denied tcp
1.55.57.166(24103) -","timestamp":"2016-08-31 13:34:39.157"}]}
MapperParsingException[failed to parse [level]]; nested:
NumberFormatException[For input string: "190>1319000: cipg01.pg.infn.it:
[syslog@9 s_id ="cipg01.pg.infn.it:11001"]: 1319112: Aug 31 15:34:38.149:
%SEC-6-IPACCESSLOGP: list 101 denied tcp 1.55.57.166(24103) -"];
it's seem to be a failed error in parsing "...nested:
NumberFormatException" . CIPG01 is a Cisco Catalyst 3750X and there is a
configuration for syslog:
logging origin-id hostname
logging host 10.0.0.60 transport udp port 11001 session-id hostname
Any other suggestions ?
Thanks you so much for help !
Best Regards
Enrico
On Wednesday, August 31, 2016 at 9:36:36 AM UTC+2, Jochen Schalanda wrote:
> Hi Enrico,
>
> please check the logs of your Elasticsearch node(s) for errors (or use
> Graylog to view the complete error messages, if they are indexed anyway).
>
> There will probably be some mapping exceptions which will also tell you
> the offending field.
>
> If you have found the offending field, you might want to create a custom
> Elasticsearch index mapping:
> http://docs.graylog.org/en/2.0/pages/configuration/elasticsearch.html#custom-index-mappings
>
> Cheers,
> Jochen
>
> On Tuesday, 30 August 2016 15:28:26 UTC+2, Enrico wrote:
>>
>> Dear All,
>> I'm using the version graylog virtual machine for managing all messagges of
>> servers and network equipment.
>> To log all the hostname names in the messages from cisco equipment I had
>> to add an local input named Cisco Catalyst,
>> that I've downloaded from market place.
>>
>> After this installation I noticed that the number of recorded messages has
>> increased a lot and the Top Sources is became
>> Elasticserach. for example I see a lot of this messages:
>>
>>
>> Timestamp
>> <http://10.0.0.60/search?rangetype=keyword&fields=message%2Csource&width=1920&highlightMessage=&keyword=Last+Hour&q=source%3Aelasticsearch#>
>> source
>> <http://10.0.0.60/search?rangetype=keyword&fields=message%2Csource&width=1920&highlightMessage=&keyword=Last+Hour&q=source%3Aelasticsearch#>
>>
>> <http://10.0.0.60/search?rangetype=keyword&fields=message%2Csource&width=1920&highlightMessage=&keyword=Last+Hour&q=source%3Aelasticsearch#>
>> *2016-08-30 15:25:31.546* elasticsearch
>>
>> ... 22 more
>> *2016-08-30 15:25:31.546* elasticsearch
>>
>> at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:321)
>> *2016-08-30 15:25:31.545* elasticsearch
>>
>> at
>> org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:241)
>> *2016-08-30 15:25:31.544* elasticsearch
>>
>> at
>> org.elasticsearch.index.mapper.core.LongFieldMapper.innerParseCreateField(LongFieldMapper.java:275)
>> *2016-08-30 15:25:31.542* elasticsearch
>>
>> at
>> org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:145)
>> *2016-08-30 15:25:31.541* elasticsearch
>>
>> at java.lang.Long.parseLong(Long.java:631)
>> *2016-08-30 15:25:31.540* elasticsearch
>>
>> at java.lang.Long.parseLong(Long.java:589)
>>
>>
>> Does anyone exaplain that behaviour ? How Can I drop these messages ?
>> Thanks a lot !
>> Best Regards
>> Enrico
>>
>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/802fbc51-375b-4bdb-a800-69d913f2603e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
