Lakshmanan, We looked over elasticsearch-hadoop when we adopted storm-elasticsearch, and elasticsearch-hadoop has only basic feature at that time. (No trident implementation) Please refer https://github.com/apache/storm/pull/573 to track our discussion on this module.
Thanks! Jungtaek Lim (HeartSaVioR) 2016년 3월 27일 (일) 오전 3:12, Lakshmanan Muthuraman <[email protected]>님이 작성: > We have been using elasticsearch-hadoop to write to ElasticSearch from > Storm. It looks good so far. Any thoughts on adopting ElasticSearch Hadoop > as part of Storm external rather than trying to write and maintain our own > bolt in Storm Project. > > Any thoughts? > > On Tue, Mar 22, 2016 at 2:19 PM, Aaron.Dossett <[email protected]> > wrote: > >> No, we hadn’t looked at that, but once we are streaming into elastic at >> scale (not there yet) I would be interested to compare. >> >> From: Tech Id <[email protected]> >> Reply-To: "[email protected]" <[email protected]> >> Date: Monday, March 21, 2016 at 1:21 PM >> To: "[email protected]" <[email protected]> >> Subject: Re: external/storm-elasticsearch - upgrade requested >> >> Thanks Aaron, >> >> Did you have a chance to compare the elasticsearch-hadoop ( >> https://www.elastic.co/guide/en/elasticsearch/hadoop/current/storm.html) >> with the storm-elasticsearch ( >> https://github.com/apache/storm/tree/master/external/storm-elasticsearch) >> ? >> >> Former uses REST client while the latter uses TransportClient. >> >> It would be interesting to know some performance numbers between them. >> >> Thanks ! >> >> On Mon, Mar 21, 2016 at 5:40 AM, Aaron.Dossett <[email protected]> >> wrote: >> >>> Hi, >>> >>> The best way to make feature requests like this is via the Apache Jira. ( >>> https://issues.apache.org/jira/browse/STORM/) >>> >>> As it happens, my team (at Target) has adapted the elastic search bolt >>> for 2.X and we are using it in production. We opened a JIRA ( >>> https://issues.apache.org/jira/browse/STORM-1475) to track that and >>> contribute our change back to the project. We’re cleaning a couple of >>> things up right now, but I’m hopeful we will contribute it back soon. >>> >>> Your second request could be a nice feature enhancement, can you open a >>> JIRA for it? In my experience, these external bolts are a really nice way >>> to get involved with the project as well. You could try adding a couple of >>> features and contributing those. >>> >>> Thanks! >>> >>> From: Tech Id <[email protected]> >>> Reply-To: "[email protected]" <[email protected]> >>> Date: Friday, March 18, 2016 at 9:45 PM >>> To: "[email protected]" <[email protected]> >>> Subject: external/storm-elasticsearch - upgrade requested >>> >>> Hi, >>> >>> I see that the version of elastic-search used >>> in external/storm-elasticsearch/pom.xml is quite old (1.6.0) while the >>> latest elastic-search is 2.2.0 >>> >>> 2.x version is not compatible with 1.x version of elastic-search and so >>> I request you to upgrade. >>> >>> Also, current storm-elasticsearch uses TransportClient of elastic-search >>> but does not expose many of the useful options in the TransportClient (like >>> routing). >>> >>> Request you to expose them to the users too. >>> >>> >>> Thanks >>> Tid >>> >> >> >
