I tried using the latest flume elastic search sink jar build from 1.5SNAPSHOT ,but still no luck. I will try to see if it is an issue with elastic search api . When I loaded json data using hive it loaded JSON properly. But we have to pass a property es.input.json in hive. Is there a way to pass a property in Flume.There is another user also posted the same issue. Any inputs ?
On Wed, Apr 9, 2014 at 2:18 PM, Ashish <[email protected]> wrote: > IMHO, checkout the trunk, build it and then just use the agent config from > old setup. Keep the old setup as is. > > thanks > ashish > > > On Wed, Apr 9, 2014 at 6:17 PM, Deepak Subhramanian < > [email protected]> wrote: > >> Thanks Otis. I will give it a try. Do I have to replace the flume jars or >> add the flume jar to the plugins directory to override the current flume >> version. >> >> >> On Wed, Apr 9, 2014 at 2:19 AM, Otis Gospodnetic < >> [email protected]> wrote: >> >>> Hi, >>> >>> The ES Sink has been modified quite a bit in the trunk. Would it make >>> sense to give that a try? >>> >>> Otis >>> -- >>> Performance Monitoring * Log Analytics * Search Analytics >>> Solr & Elasticsearch Support * http://sematext.com/ >>> >>> >>> On Mon, Apr 7, 2014 at 12:08 PM, Deepak Subhramanian < >>> [email protected]> wrote: >>> >>>> I am posting JSON data to ElasticSearch using Flume . But for some >>>> reason it is not loading the correct data in the message. Instead of JSON >>>> message it is storing message data as >>>> "org.elasticsearch.common.xcontent.XContentBuilder@32f4122e" >>>> >>>> I tried using the DynamicSerializer which determines the content-type. >>>> But no luck. >>>> >>>> Here is the sample data posted on Elastic Search. >>>> >>>> { >>>> >>>> "_index": "test_flume-2014-04-07", >>>> >>>> "_type": "logs", >>>> >>>> "_id": "M9E-33RQTy2kA6QhW6mSUw", >>>> >>>> "_score": null, >>>> >>>> "_source": { >>>> >>>> "@message": " >>>> org.elasticsearch.common.xcontent.XContentBuilder@58bf76d2", >>>> >>>> "@timestamp": "2014-04-07T09:49:26.490Z", >>>> >>>> "@fields": { >>>> >>>> "timestamp": "1396864166490" >>>> >>>> } >>>> >>>> }, >>>> >>>> "sort": [ >>>> >>>> 1396864166490, >>>> >>>> 1396864166490 >>>> >>>> ] >>>> >>>> } >>>> >>>> org.elasticsearch.common.xcontent.XContentBuilder >>>> -- >>>> Deepak Subhramanian >>>> >>> >>> >> >> >> -- >> Deepak Subhramanian >> > > > > -- > thanks > ashish > > Blog: http://www.ashishpaliwal.com/blog > My Photo Galleries: http://www.pbase.com/ashishpaliwal > -- Deepak Subhramanian
