Is there any additional serializers in 1.5SNAPSHOT which can resolve the JSON issue. I noticed that with the 1.5 SNAPSHOT it was not storing timestamp field in elasticsearch. Is the default serializer changed from Logstash or is it a bug. I was getting timestamp in 1.4 version .
On Fri, Apr 11, 2014 at 12:56 PM, Deepak Subhramanian < [email protected]> wrote: > > I tried using the latest flume elastic search sink jar build from > 1.5SNAPSHOT ,but still no luck. I will try to see if it is an issue with > elastic search api . When I loaded json data using hive it loaded JSON > properly. But we have to pass a property es.input.json in hive. Is there a > way to pass a property in Flume.There is another user also posted the same > issue. Any inputs ? > > > On Wed, Apr 9, 2014 at 2:18 PM, Ashish <[email protected]> wrote: > >> IMHO, checkout the trunk, build it and then just use the agent config >> from old setup. Keep the old setup as is. >> >> thanks >> ashish >> >> >> On Wed, Apr 9, 2014 at 6:17 PM, Deepak Subhramanian < >> [email protected]> wrote: >> >>> Thanks Otis. I will give it a try. Do I have to replace the flume jars >>> or add the flume jar to the plugins directory to override the current flume >>> version. >>> >>> >>> On Wed, Apr 9, 2014 at 2:19 AM, Otis Gospodnetic < >>> [email protected]> wrote: >>> >>>> Hi, >>>> >>>> The ES Sink has been modified quite a bit in the trunk. Would it make >>>> sense to give that a try? >>>> >>>> Otis >>>> -- >>>> Performance Monitoring * Log Analytics * Search Analytics >>>> Solr & Elasticsearch Support * http://sematext.com/ >>>> >>>> >>>> On Mon, Apr 7, 2014 at 12:08 PM, Deepak Subhramanian < >>>> [email protected]> wrote: >>>> >>>>> I am posting JSON data to ElasticSearch using Flume . But for some >>>>> reason it is not loading the correct data in the message. Instead of JSON >>>>> message it is storing message data as >>>>> "org.elasticsearch.common.xcontent.XContentBuilder@32f4122e" >>>>> >>>>> I tried using the DynamicSerializer which determines the content-type. >>>>> But no luck. >>>>> >>>>> Here is the sample data posted on Elastic Search. >>>>> >>>>> { >>>>> >>>>> "_index": "test_flume-2014-04-07", >>>>> >>>>> "_type": "logs", >>>>> >>>>> "_id": "M9E-33RQTy2kA6QhW6mSUw", >>>>> >>>>> "_score": null, >>>>> >>>>> "_source": { >>>>> >>>>> "@message": " >>>>> org.elasticsearch.common.xcontent.XContentBuilder@58bf76d2", >>>>> >>>>> "@timestamp": "2014-04-07T09:49:26.490Z", >>>>> >>>>> "@fields": { >>>>> >>>>> "timestamp": "1396864166490" >>>>> >>>>> } >>>>> >>>>> }, >>>>> >>>>> "sort": [ >>>>> >>>>> 1396864166490, >>>>> >>>>> 1396864166490 >>>>> >>>>> ] >>>>> >>>>> } >>>>> >>>>> org.elasticsearch.common.xcontent.XContentBuilder >>>>> -- >>>>> Deepak Subhramanian >>>>> >>>> >>>> >>> >>> >>> -- >>> Deepak Subhramanian >>> >> >> >> >> -- >> thanks >> ashish >> >> Blog: http://www.ashishpaliwal.com/blog >> My Photo Galleries: http://www.pbase.com/ashishpaliwal >> > > > > -- > Deepak Subhramanian > -- Deepak Subhramanian
