----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47098/#review132183 -----------------------------------------------------------
I'm getting the following when I tried to compile flume after applying the patch: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile (default-compile) on project flume-ng-core: Compilation failure [ERROR] /Users/jarcec/apache/flume/flume-ng-core/src/main/java/org/apache/flume/source/http/BLOBHandler.java:[43,7] error: BLOBHandler is not abstract and does not override abstract method setNullHeaderReplacement(String) in HTTPSourceHandler [ERROR] -> [Help 1] Which points to: flume-ng-core/src/main/java/org/apache/flume/source/http/HTTPSourceHandler.java (line 44) <https://reviews.apache.org/r/47098/#comment196350> This will be a backward in-compatible change. Can we perhaps convert this interface to abstract class, move handling of the nullReplacement here, so that we will break the contract only once? E.g. that we won't have to break it somewhere in the future when we will need to add yet another argument. - Jarek Cecho On May 8, 2016, 7:48 p.m., neerja khattar wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47098/ > ----------------------------------------------------------- > > (Updated May 8, 2016, 7:48 p.m.) > > > Review request for Flume. > > > Repository: flume-git > > > Description > ------- > > The issue is when the header value is null it throws null pointer exception > and flume stops processing further events. > For example: > [{ > "headers" : { > "timestamp" : "434324343", > "host" : null > }, > "body" : "random_body" > }] > > The solution to fix this is: > > 1. If the header has a null value in the json, flume will replace it with a > replacement string. > 2. The default value for a replacement string is an empty string. > 3. To overwrite default string, set "handler.nullReplacementHeader" > property in flume config. > > > Diffs > ----- > > flume-ng-core/src/main/java/org/apache/flume/source/http/HTTPSource.java > b520b03 > > flume-ng-core/src/main/java/org/apache/flume/source/http/HTTPSourceConfigurationConstants.java > 86caf7d > > flume-ng-core/src/main/java/org/apache/flume/source/http/HTTPSourceHandler.java > 726bf0c > flume-ng-core/src/main/java/org/apache/flume/source/http/JSONHandler.java > 197f66a > > flume-ng-core/src/test/java/org/apache/flume/source/http/TestJSONHandler.java > 455781c > > Diff: https://reviews.apache.org/r/47098/diff/ > > > Testing > ------- > > The following are the test cases: > > 1. Header has null value in json and handler.nullReplacementHeader is not set > in flume config. The default value will be used to replace null. > > [{ > "headers" : { > "timestamp" : "434324343", > "host" : null > }, > "body" : "random_body" > }] > > Output in hdfs : {timestamp=434324343, host=} random_body > > 2. Header is not null in json and handler.nullReplacementHeader is not set > in flume config. The replacement implementation doesnt come in to > consideration. > > [{ > "headers" : { > "timestamp" : "434324343", > "host" : 1 > }, > "body" : "random_body" > }] > Output in hdfs : {timestamp=434324343, host=1} random_body > > 3. Header has null value in json and handler.nullReplacementHeader=abc is > set in flume config. The null value in header will be replaced by abc. > > > [{ > "headers" : { > "timestamp" : "434324343", > "host" : null > }, > "body" : "random_body" > }] > > > Output in hdfs {timestamp=434324343, host=abc} random_body > > 4. Header has null value in json and handler.nullReplacementHeader=1 is set > in flume config. The null value in header will be replaced by 1 as a string . > > [{ > "headers" : { > "timestamp" : "434324343", > "host" : null > }, > "body" : "random_body" > }] > > > Output in hdfs: {timestamp=434324343, host=1} random_body > > 5. Header is not null in json and handler.nullReplacementHeader is also set > in flume config. The replacement implementation doesnt come in to > consideration. > > [{ > "headers" : { > "timestamp" : "434324343", > "host" : 1 > }, > "body" : "random_body" > }] > Output in hdfs : {timestamp=434324343, host=1} random_body > > > Thanks, > > neerja khattar > >
