> On Aug. 24, 2016, 4:30 p.m., Attila Simon wrote: > > w/flume-ng-core/src/main/java/org/apache/flume/interceptor/RemoveHeaderInterceptor.java, > > line 139 > > <https://reviews.apache.org/r/51244/diff/6/?file=1482094#file1482094line139> > > > > Please don't log out the whole Event for each removed header. Also > > please collect all the headers which were removed and print them out as a > > single log entry per Event. > > Balázs Donát Bessenyei wrote: > I think there is a benefit to keeping the event in the log, but I changed > the code to collect the headers for logging > > If you think the whole event logging is a blocking issue, please re-open
Fair enough. Then could you please consider wrapping the logging of event with a org.apache.flume.conf.LogPrivacyUtil#allowLogRawData() check? - Attila ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/51244/#review146665 ----------------------------------------------------------- On Aug. 24, 2016, 9:08 p.m., Balázs Donát Bessenyei wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/51244/ > ----------------------------------------------------------- > > (Updated Aug. 24, 2016, 9:08 p.m.) > > > Review request for Flume. > > > Bugs: FLUME-2171 > https://issues.apache.org/jira/browse/FLUME-2171 > > > Repository: flume-git > > > Description > ------- > > I found Flume OG's decorators to handle event headers useful and some to be > missing from Flume NG. More specifically, we could have an interceptor to > remove headers from an event. > > > Diffs > ----- > > > c/flume-ng-core/src/main/java/org/apache/flume/interceptor/InterceptorType.java > fe341e9 > c/flume-ng-doc/sphinx/FlumeUserGuide.rst 5e677c6 > > w/flume-ng-core/src/main/java/org/apache/flume/interceptor/RemoveHeaderInterceptor.java > PRE-CREATION > > w/flume-ng-core/src/test/java/org/apache/flume/interceptor/RemoveHeaderInterceptorTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/51244/diff/ > > > Testing > ------- > > All tests (besides the FLUME-2974-related ones) in flume-ng-core run > successfully > > I've used this config for manual testing: > > a1.sources = r1 > a1.sources.r1.type = netcat > a1.sources.r1.bind = 0.0.0.0 > a1.sources.r1.port = 6666 > a1.sources.r1.channels = c1 > > a1.channels = c1 > a1.channels.c1.type = memory > a1.channels.c1.capacity = 10000 > a1.channels.c1.transactionCapacity = 10000 > a1.channels.c1.byteCapacityBufferPercentage = 20 > a1.channels.c1.byteCapacity = 800000 > > a1.channels = c1 > a1.sinks = k1 > a1.sinks.k1.type = logger > a1.sinks.k1.channel = c1 > > > a1.sources.r1.interceptors = i1 i2 i3 > a1.sources.r1.interceptors.i1.type = timestamp > > a1.sources.r1.interceptors.i2.type = host > a1.sources.r1.interceptors.i2.hostHeader = hostname > > a1.sources.r1.interceptors.i3.type = remove_header > a1.sources.r1.interceptors.i3.with.name = timestamp > > > Thanks, > > Balázs Donát Bessenyei > >
