[
https://issues.apache.org/jira/browse/NIFI-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16023056#comment-16023056
]
ASF GitHub Bot commented on NIFI-3191:
--------------------------------------
Github user bbende commented on the issue:
https://github.com/apache/nifi/pull/1802
+1 Was able to successfully use Twitter's LZO codec with the HDFS
processors with this change, will merge to master
> HDFS Processors Should Allow Choosing LZO Compression
> -----------------------------------------------------
>
> Key: NIFI-3191
> URL: https://issues.apache.org/jira/browse/NIFI-3191
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 1.1.0, 0.7.1
> Reporter: Bryan Bende
> Assignee: Pierre Villard
>
> The HDFS processors have a CompressionType enum that maps the enum to a class
> name, currently it has the following:
> {code}
> case NONE: return "NONE";
> case DEFAULT: return DefaultCodec.class.getName();
> case BZIP: return BZip2Codec.class.getName();
> case GZIP: return GzipCodec.class.getName();
> case LZ4: return Lz4Codec.class.getName();
> case SNAPPY: return SnappyCodec.class.getName();
> case AUTOMATIC: return "Automatically Detected";
> {code}
> The Lzo codec class is GPL (http://code.google.com/p/hadoop-gpl-compression
> and https://github.com/twitter/hadoop-lzo) so it can not be shipped with NiFi
> or Hadoop, but if someone adds it to their Hadoop distribution there should
> be a way to select it from NiFi.
> Seems LZO could be added to the enumeration with a mapping of:
> {code}
> case LZO: return "com.hadoop.compression.lzo.LzoCodec";
> {code}
> The libraries would still need to be linked through the LD_LIBRARY_PATH such
> as:
> {code}
> java.arg.15=-Djava.libaray.path=/path/to/your/lzocodec.so
> {code}
> and possibly point the additional classpath resource to the jar containing
> LzoCodec.java.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
