On Tue, Dec 8, 2015 at 1:44 PM, Peter Portante <[email protected]> wrote: > On Tue, Dec 8, 2015 at 6:38 AM, Brian Knox <[email protected]> wrote: > >> As a short term solution I'm working on a small service (in golang) that >> accepts logs over tcp, can replace characters in JSON field names in a @cee >> syslog line, and then forward the line to another syslog destination. In >> tests on my laptop it handles modifying ~ 50,000 reasonably sized log lines >> a second per connection. It gracefully handles tcp connection issues and >> I'll test it under adverse circumstances to make sure it's reasonably >> robust. I personally find this preferable to deploying logstash just to >> substitute one character. I'll release it open source this week in case >> any one else needs an immediate solution to this problem like I do. >> >> It's less than ideal - ideally elasticsearch would support JSON rather than >> a subset of characters JSON allows - but it solves the immediate problem >> for us. >> > > Have you brought this up with the ElasticSearch community to see what they > say?
I think Brian touched on this already - previously dots were allowed but it was confusing (both for users and for Elasticsearch itself) when you ran a query or indexed a document with a dot in the field name. For example, is "user.name" an actual field or do we have a "user" object with "name" as the actual field that contains data? Lucene (the search engine library on which Elasticsearch is built) doesn't support real hierarchies. So when you index a "user" object with a "name" field, the data you put in there is actually indexed in a "user.name" field in Lucene. This means you can't have both "user.name" as a separate field and the "name" field in the "user" object in the same index. Actually, you could have that before, and that caused quite some bugs (imagine what happens if you have different definitions for those fields which are supposed to be identical). With this in mind, I doubt this change would be reverted in future, it was done on purpose to work around issues that came up in previous versions. One could think of workarounds, but I don't see one that wouldn't come with its own problems. Unfortunately, Elasticsearch changes quite a lot in backwards-incompatible ways between major versions. This hurts the ecosystem as you can see here, but it helps drive the project itself forward at a quicker pace. Best regards, Radu -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
