[jira] [Commented] (NIFI-5872) Add compression option on JsonRecordSetWriter
[ https://issues.apache.org/jira/browse/NIFI-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714374#comment-16714374 ] ASF GitHub Bot commented on NIFI-5872: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3208 > Add compression option on JsonRecordSetWriter > - > > Key: NIFI-5872 > URL: https://issues.apache.org/jira/browse/NIFI-5872 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.9.0 > > > Add a compression option on the JsonRecordSetWriter controller service as we > have on the AvroRecordSetWriter controller service. This can be useful in > case of very IO intensive workflows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5872) Add compression option on JsonRecordSetWriter
[ https://issues.apache.org/jira/browse/NIFI-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714373#comment-16714373 ] ASF subversion and git services commented on NIFI-5872: --- Commit a6f91a197516128a1809c7563efaaf3e5fe63a17 in nifi's branch refs/heads/master from [~pvillard] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=a6f91a1 ] NIFI-5872 - Added compression option to JsonRecordSetWriter This closes #3208. Signed-off-by: Koji Kawamura > Add compression option on JsonRecordSetWriter > - > > Key: NIFI-5872 > URL: https://issues.apache.org/jira/browse/NIFI-5872 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > > Add a compression option on the JsonRecordSetWriter controller service as we > have on the AvroRecordSetWriter controller service. This can be useful in > case of very IO intensive workflows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5872) Add compression option on JsonRecordSetWriter
[ https://issues.apache.org/jira/browse/NIFI-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16714372#comment-16714372 ] ASF GitHub Bot commented on NIFI-5872: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/3208 The performance improvement result is impressing! Code looks good and I was able to compress records. +1 merging. Thanks @pvillard31 > Add compression option on JsonRecordSetWriter > - > > Key: NIFI-5872 > URL: https://issues.apache.org/jira/browse/NIFI-5872 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > > Add a compression option on the JsonRecordSetWriter controller service as we > have on the AvroRecordSetWriter controller service. This can be useful in > case of very IO intensive workflows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5872) Add compression option on JsonRecordSetWriter
[ https://issues.apache.org/jira/browse/NIFI-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712086#comment-16712086 ] ASF GitHub Bot commented on NIFI-5872: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/3208 Ran a test on my laptop with a workflow generating a lot of data: - without compression, I reached 129M events / 11.15GB out per 5 minutes - with compression (Snappy), I reached 161M events / 1GB out per 5 minutes It improves the number of processed eps while reducing the amount of data on disk. > Add compression option on JsonRecordSetWriter > - > > Key: NIFI-5872 > URL: https://issues.apache.org/jira/browse/NIFI-5872 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > > Add a compression option on the JsonRecordSetWriter controller service as we > have on the AvroRecordSetWriter controller service. This can be useful in > case of very IO intensive workflows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5872) Add compression option on JsonRecordSetWriter
[ https://issues.apache.org/jira/browse/NIFI-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712084#comment-16712084 ] ASF GitHub Bot commented on NIFI-5872: -- GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/3208 NIFI-5872 - Added compression option to JsonRecordSetWriter Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi NIFI-5872 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3208.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3208 commit 2a71d632f75c704b07f04a2a4fe9596e2a56cb74 Author: Pierre Villard Date: 2018-12-06T08:50:20Z NIFI-5872 - Added compression option to JsonRecordSetWriter > Add compression option on JsonRecordSetWriter > - > > Key: NIFI-5872 > URL: https://issues.apache.org/jira/browse/NIFI-5872 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > > Add a compression option on the JsonRecordSetWriter controller service as we > have on the AvroRecordSetWriter controller service. This can be useful in > case of very IO intensive workflows. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
