Bryan Bende created NIFI-3191: --------------------------------- Summary: HDFS Processors Should Allow Choosing LZO Compression Key: NIFI-3191 URL: https://issues.apache.org/jira/browse/NIFI-3191 Project: Apache NiFi Issue Type: Improvement Affects Versions: 0.7.1, 1.1.0 Reporter: Bryan Bende
The HDFS processors have a CompressionType enum that maps the enum to a class name, currently it has the following: {code} case NONE: return "NONE"; case DEFAULT: return DefaultCodec.class.getName(); case BZIP: return BZip2Codec.class.getName(); case GZIP: return GzipCodec.class.getName(); case LZ4: return Lz4Codec.class.getName(); case SNAPPY: return SnappyCodec.class.getName(); case AUTOMATIC: return "Automatically Detected"; {code} The Lzo codec class is GPL (http://code.google.com/p/hadoop-gpl-compression and https://github.com/twitter/hadoop-lzo) so it can not be shipped with NiFi or Hadoop, but if someone adds it to their Hadoop distribution there should be a way to select it from NiFi. Seems LZO could be added to the enumeration with a mapping of: {code} case LZO: return "com.hadoop.compression.lzo.LzoCodec"; {code} The libraries would still need to be linked through the LD_LIBRARY_PATH such as: {code} java.arg.15=-Djava.libaray.path=/path/to/your/lzocodec.so {code} and possibly point the additional classpath resource to the jar containing LzoCodec.java. -- This message was sent by Atlassian JIRA (v6.3.4#6332)