[
https://issues.apache.org/jira/browse/NIFI-10273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-10273:
------------------------------------
Status: Patch Available (was: Open)
> MergeContent tar compression fails on FlowFiles > 8.5GB
> -------------------------------------------------------
>
> Key: NIFI-10273
> URL: https://issues.apache.org/jira/browse/NIFI-10273
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.16.3
> Reporter: Michael Hogue
> Assignee: Michael Hogue
> Priority: Minor
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When attempting to tar compress FlowFiles > 8.5GB, MergeContent fails with
> the following error:
> {noformat}
> java.lang.RuntimeException: entry size '<FlowFileSize>' is too big (>
> 8589934591){noformat}
> This is because apache commons-compress library has a hard maximum :
> [https://commons.apache.org/proper/commons-compress/apidocs/src-html/org/apache/commons/compress/archivers/tar/TarConstants.html#line.94]
> 077777777777 octal bytes is 8589934591 decimal bytes (~8.5GB).
> This can be addressed by configuring a BigNumberMode on the
> {{{}TarArchiveOutputStream{}}}:
> [https://commons.apache.org/proper/commons-compress/javadocs/api-1.10/org/apache/commons/compress/archivers/tar/TarArchiveOutputStream.html#setBigNumberMode(int)]
> {{Here's an example of another project doing this:}}
> {{[https://github.com/DataConservancy/dcs-packaging-tool/pull/36/files]}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
