[
https://issues.apache.org/jira/browse/FLUME-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640276#comment-13640276
]
Tim Bacon commented on FLUME-2015:
----------------------------------
Mike: thanks for the pointer to Edward's work and the info on how to properly
attach patches.
My motivating use case is that
* my flume events have no structured headers and are not logging-related
* there is a structured avro blob in the event bytes and I want to use avro to
turn those bytes into the JSON that will be stored in ElasticSearch (i.e.
bypass {{XContentBuilder}} entirely)
* I need to be able to set the ElasticSearch ID, index name and possibly index
type from the actual event content (similar to [FLUME-1972])
Edward: thanks for the info about the work you have already done and the
comments on my patch.
I'd be delighted to merge the existing patch
[https://issues.apache.org/jira/secure/attachment/12579897/FLUME-1782-FLUME-1972-1.4.0-1.patch]
with the work I've done -- and rename the Builder-er to Factory ;-) -- then
attach the result to this issue for review. For starters I'll add the timestamp
as a method param along with the index name/type. I'll probably need 24 hours
or so for turnaround.
Let me know if you would like me to also take a stab at the user guide changes.
Thanks!
Tim
> ElasticSearchSink: need access to IndexRequestBuilder instance during flume
> event processing
> --------------------------------------------------------------------------------------------
>
> Key: FLUME-2015
> URL: https://issues.apache.org/jira/browse/FLUME-2015
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.3.0, v1.3.1
> Reporter: Tim Bacon
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I need more control over the indexing performed by the {{ElasticSearchSink}}
> -- in particular I need access to an {{IndexRequestBuilder}} instance during
> flume event processing. The interactions between {{ElasticSearchSink}} and
> {{ElasticSearchEventSerializer}} currently do not make this possible.
> I have authored a patch that meets my needs and maintains
> backwards-compatibility for existing users of the sink. It is available at
> [https://github.com/prime8/flume/commit/1056c129b10c95cee50e0c3f77e309668f82bfc6]
> -- please let me know if it can be pulled into the main ASF source tree!
> Thanks :-)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
