[
https://issues.apache.org/jira/browse/FLUME-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257610#comment-13257610
]
Inder SIngh commented on FLUME-1097:
------------------------------------
Folks,
Please review
https://issues.apache.org/jira/secure/attachment/12523368/FLUME-1097-1.patch
which attempts at adding CustomHeaderDecorator support's to Sinks through Sink
configuration.
Overview of Changes
1.EventHeaderDecorator -> generic class having processHeader()
2.SinkEventHeaderDecorator is a EventHeaderDecorator which overrides timestamp
header at this hop.
3.EventHeaderDecoratorFactory provides EventHeaderDecorator's based on
<sink>.headerDecorator config
4.AbstractSink -> has a eventHeaderDecortaor
5.HDFSEventSink -> checks for decorator through config and calls
processHeader() for each event which modifies/add headers based on event
decorator defined through config.
Sample Configuration for testing
----------------------------------
agent1.sinks.log-sink1.hdfs.headerDecorator = sinkDecorator
Testing Done
-------------
1. Unit tests
2. manual tests done - agent1.sinks.log-sink1.hdfs.filePrefix = flume-data/%D
results in publishing data with timestamp at agent's hop running HDFSEventSink
> Support modification of event Headers through configuration
> -----------------------------------------------------------
>
> Key: FLUME-1097
> URL: https://issues.apache.org/jira/browse/FLUME-1097
> Project: Flume
> Issue Type: New Feature
> Reporter: Inder SIngh
> Assignee: Inder SIngh
> Priority: Minor
> Attachments: FLUME-1097-1.patch
>
>
> As discussed on the flume threads we want to support adding/modifying headers
> at flume agents through configuration.
> Primary use-case as of now -
> Use HDFSEventSink to publish data in paths like /flume-data/YYYY/mm/DD/HH/MM
> wherein timestamp is of agent running HDFSEventSink.
> What's not there today
> -------------------------
> 1. headers are put by clientLIB and can't be added/modified in flight by
> agents.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
