[
https://issues.apache.org/jira/browse/FLUME-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638512#comment-13638512
]
Edward Sargisson commented on FLUME-2015:
-----------------------------------------
Hi Tim,
Great! Being able to control the indexing in more detail is something that had
occurred to me too so thank you for putting this together.
As I see it, we have a few issues to resolve to bring this in.
Firstly, the basic defect in FLUME-1782 isn't fixed so the incorrect index name
is calculated in many conditions.
Secondly, in order to fix that defect in a way that doesn't break existing
implementations I've created an ElasticSearchEventSerializer2 interface which
takes the timestamp the sink decided to provide - if it didn't find one in the
event (I only put that patch up about an hour ago).
Thirdly, FLUME-1972 contains the patch to provide the ID to elasticsearch if a
user wishes to (instead of relying on elasticsearch generating it).
Fourthly, I don't see any update to the Flume User Guide.
So, how about we do this:
* if you could possibly modify your patch to uses ElasticSearchEventSerializer2
and ElasticSearchIdProvider then that would be really great.
* hopefully the Flume committers can either get FLUME 1782 and FLUME 1972
reviewed and committed - or wait on your work.
Nit: Builderer isn't English... ;-) How about
ElasticSearchIndexRequestBuilderFactory?
> ElasticSearchSink: need access to IndexRequestBuilder instance during flume
> event processing
> --------------------------------------------------------------------------------------------
>
> Key: FLUME-2015
> URL: https://issues.apache.org/jira/browse/FLUME-2015
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.3.0, v1.3.1
> Reporter: Tim Bacon
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I need more control over the indexing performed by the {{ElasticSearchSink}}
> -- in particular I need access to an {{IndexRequestBuilder}} instance during
> flume event processing. The interactions between {{ElasticSearchSink}} and
> {{ElasticSearchEventSerializer}} currently do not make this possible.
> I have authored a patch that meets my needs and maintains
> backwards-compatibility for existing users of the sink. It is available at
> [https://github.com/prime8/flume/commit/1056c129b10c95cee50e0c3f77e309668f82bfc6]
> -- please let me know if it can be pulled into the main ASF source tree!
> Thanks :-)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
