[
https://issues.apache.org/jira/browse/NIFI-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17577689#comment-17577689
]
Matthew Hawkins commented on NIFI-2827:
---------------------------------------
Thanks for taking a look [~joewitt]; adding compression levels as described
above is 3 lines (see attachment) over the original patch.
(patch zstd then patch clvl)
This uses the hack as described above as I can't see a clean way to add
explicit support for zstd's 22 levels; the property is declared final but would
need to be set based on some switch on what the compression format is to
overcome the difference between this and other compression methods.
Perhaps Commons has a declaration of this, but a generic compression type
object could have a property of what compression levels are supported by the
underlying format so it's not an explicit 0-9 in that one-liner method
conjunction when the property is declared.
It's not actually quite as hacky to do what I've done here; the zstd 22 levels
are explicitly defined in the standard to be increasing in the same sort of
manner to gzip's 0-9, and since they don't recommend using levels over 20
anyway it's not so bad to just double the set value which will achieve the
desired effect.
It's worthwhile to drop this in as-is IMO because the compression versus gzip
is insane (see ods attachment). That's lorem ipsum in batches of 10000 into
Merge and Compress; so yes easily compressible with a good algorithm. In/Out
sizes in bytes, time in seconds (5 min mean)
> CompressContent should support brotli (MIT) and zstd (BSD)
> ----------------------------------------------------------
>
> Key: NIFI-2827
> URL: https://issues.apache.org/jira/browse/NIFI-2827
> Project: Apache NiFi
> Issue Type: New Feature
> Reporter: Andre F de Miranda
> Priority: Major
> Attachments: NIFI-2827-zstd.patch
>
>
> brotli and zstd are gaining lots of momentum across the IoT sphere and I
> suspect it would be a nice to have in NiFi to be able to support those two
> emerging compression algorithms.
> Both brotli and zstd JAVA implementations are JNIs
> https://github.com/MeteoGroup/jbrotli (ASL 2)
> https://github.com/luben/zstd-jni (3 clause BSD)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
