[
https://issues.apache.org/jira/browse/FLUME-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258761#comment-13258761
]
[email protected] commented on FLUME-1097:
------------------------------------------------------
bq. On 2012-04-20 20:07:27, Hari Shreedharan wrote:
bq. >
flume-ng-core/src/main/java/org/apache/flume/event/SinkEventHeaderDecorator.java,
line 29
bq. > <https://reviews.apache.org/r/4826/diff/1/?file=103469#file103469line29>
bq. >
bq. > This should be named something like TimeStampEventDecorator or
something. We might want to add decorators that add other stuff at the sink.
Thanks Hari. I'll work towards this feedback and come back.
- Inder
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4826/#review7092
-----------------------------------------------------------
On 2012-04-20 10:33:14, Inder Singh wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4826/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-20 10:33:14)
bq.
bq.
bq. Review request for Flume, Arvind Prabhakar, Mike Percy, and Hari
Shreedharan.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Overview of Changes
bq.
bq. 1.EventHeaderDecorator -> generic class having processHeader()
bq. 2.SinkEventHeaderDecorator is a EventHeaderDecorator which overrides
timestamp header at this hop.
bq. 3.EventHeaderDecoratorFactory provides EventHeaderDecorator's based on
<sink>.headerDecorator config
bq. 4.AbstractSink -> has a eventHeaderDecortaor
bq. 5.HDFSEventSink -> checks for decorator through config and calls
processHeader() for each event which modifies/add headers based on event
decorator defined through config.
bq.
bq. Sample Configuration for testing
bq. ----------------------------------
bq. agent1.sinks.log-sink1.hdfs.headerDecorator = sinkDecorator
bq.
bq.
bq. This addresses bug FLUME-1097.
bq. https://issues.apache.org/jira/browse/FLUME-1097
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
flume-ng-core/src/main/java/org/apache/flume/event/EventHeaderDecorator.java
PRE-CREATION
bq.
flume-ng-core/src/main/java/org/apache/flume/event/EventHeaderDecoratorFactory.java
PRE-CREATION
bq.
flume-ng-core/src/main/java/org/apache/flume/event/SinkEventHeaderDecorator.java
PRE-CREATION
bq. flume-ng-core/src/main/java/org/apache/flume/sink/AbstractSink.java
2334059
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
1b61cad
bq.
bq. Diff: https://reviews.apache.org/r/4826/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Testing Done
bq. -------------
bq. 1. Unit tests
bq. 2. manual tests done - agent1.sinks.log-sink1.hdfs.filePrefix =
flume-data/%D results in publishing data with timestamp at agent's hop running
HDFSEventSink
bq.
bq.
bq. Thanks,
bq.
bq. Inder
bq.
bq.
> Support modification of event Headers through configuration
> -----------------------------------------------------------
>
> Key: FLUME-1097
> URL: https://issues.apache.org/jira/browse/FLUME-1097
> Project: Flume
> Issue Type: New Feature
> Reporter: Inder SIngh
> Assignee: Inder SIngh
> Priority: Minor
> Attachments: FLUME-1097-1.patch
>
>
> As discussed on the flume threads we want to support adding/modifying headers
> at flume agents through configuration.
> Primary use-case as of now -
> Use HDFSEventSink to publish data in paths like /flume-data/YYYY/mm/DD/HH/MM
> wherein timestamp is of agent running HDFSEventSink.
> What's not there today
> -------------------------
> 1. headers are put by clientLIB and can't be added/modified in flight by
> agents.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
