[ https://issues.apache.org/jira/browse/NIFI-10273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Hogue updated NIFI-10273: --------------------------------- Description: When attempting to tar compress FlowFiles > 8.5GB, MergeContent fails with the following error: {noformat} java.lang.RuntimeException: entry size '<FlowFileSize>' is too big (> 8589934591){noformat} This is because apache commons-compress library has a hard maximum : [https://commons.apache.org/proper/commons-compress/apidocs/src-html/org/apache/commons/compress/archivers/tar/TarConstants.html#line.94] 077777777777 octal bytes is 8589934591 decimal bytes (~8.5GB). This can be addressed by configuring a BigNumberMode on the {{{}TarArchiveOutputStream{}}}: [https://commons.apache.org/proper/commons-compress/javadocs/api-1.10/org/apache/commons/compress/archivers/tar/TarArchiveOutputStream.html#setBigNumberMode(int)] }} {{Here's an example of another project doing this:}} {{[https://github.com/DataConservancy/dcs-packaging-tool/pull/36/files]}} was: When attempting to tar compress FlowFiles > 8.5GB, MergeContent fails with the following error: {noformat} java.lang.RuntimeException: entry size '<FlowFileSize>' is too big (> 8589934591){noformat} This is because apache commons-compress library has a hard maximum : [https://commons.apache.org/proper/commons-compress/apidocs/src-html/org/apache/commons/compress/archivers/tar/TarConstants.html#line.94] 077777777777 octal bytes is 8589934591 decimal bytes (~8.5GB). This can be addressed by configuring a BigNumberMode on the {{TarArchiveOutputStream: [https://commons.apache.org/proper/commons-compress/javadocs/api-1.10/org/apache/commons/compress/archivers/tar/TarArchiveOutputStream.html#setBigNumberMode(int)] }} {{Here's an example of another project doing this:}} {{https://github.com/DataConservancy/dcs-packaging-tool/pull/36/files}} > MergeContent tar compression fails on FlowFiles > 8.5GB > ------------------------------------------------------- > > Key: NIFI-10273 > URL: https://issues.apache.org/jira/browse/NIFI-10273 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Affects Versions: 1.16.3 > Reporter: Michael Hogue > Priority: Minor > > When attempting to tar compress FlowFiles > 8.5GB, MergeContent fails with > the following error: > {noformat} > java.lang.RuntimeException: entry size '<FlowFileSize>' is too big (> > 8589934591){noformat} > This is because apache commons-compress library has a hard maximum : > [https://commons.apache.org/proper/commons-compress/apidocs/src-html/org/apache/commons/compress/archivers/tar/TarConstants.html#line.94] > 077777777777 octal bytes is 8589934591 decimal bytes (~8.5GB). > This can be addressed by configuring a BigNumberMode on the > {{{}TarArchiveOutputStream{}}}: > [https://commons.apache.org/proper/commons-compress/javadocs/api-1.10/org/apache/commons/compress/archivers/tar/TarArchiveOutputStream.html#setBigNumberMode(int)] > }} > {{Here's an example of another project doing this:}} > {{[https://github.com/DataConservancy/dcs-packaging-tool/pull/36/files]}} -- This message was sent by Atlassian Jira (v8.20.10#820010)