[
https://issues.apache.org/jira/browse/NIFI-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011004#comment-16011004
]
ASF GitHub Bot commented on NIFI-3191:
--------------------------------------
GitHub user pvillard31 opened a pull request:
https://github.com/apache/nifi/pull/1802
NIFI-3191 - HDFS Processors Should Allow Choosing LZO Compression
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [X] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [X] Is your initial contribution a single, squashed commit?
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pvillard31/nifi NIFI-3191
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/1802.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1802
----
commit 84ea06238ef99df4bef0d5b4e4721565234442e5
Author: Pierre Villard <[email protected]>
Date: 2017-05-15T17:55:48Z
NIFI-3191 - HDFS Processors Should Allow Choosing LZO Compression
----
> HDFS Processors Should Allow Choosing LZO Compression
> -----------------------------------------------------
>
> Key: NIFI-3191
> URL: https://issues.apache.org/jira/browse/NIFI-3191
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 1.1.0, 0.7.1
> Reporter: Bryan Bende
>
> The HDFS processors have a CompressionType enum that maps the enum to a class
> name, currently it has the following:
> {code}
> case NONE: return "NONE";
> case DEFAULT: return DefaultCodec.class.getName();
> case BZIP: return BZip2Codec.class.getName();
> case GZIP: return GzipCodec.class.getName();
> case LZ4: return Lz4Codec.class.getName();
> case SNAPPY: return SnappyCodec.class.getName();
> case AUTOMATIC: return "Automatically Detected";
> {code}
> The Lzo codec class is GPL (http://code.google.com/p/hadoop-gpl-compression
> and https://github.com/twitter/hadoop-lzo) so it can not be shipped with NiFi
> or Hadoop, but if someone adds it to their Hadoop distribution there should
> be a way to select it from NiFi.
> Seems LZO could be added to the enumeration with a mapping of:
> {code}
> case LZO: return "com.hadoop.compression.lzo.LzoCodec";
> {code}
> The libraries would still need to be linked through the LD_LIBRARY_PATH such
> as:
> {code}
> java.arg.15=-Djava.libaray.path=/path/to/your/lzocodec.so
> {code}
> and possibly point the additional classpath resource to the jar containing
> LzoCodec.java.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
