[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17291585#comment-17291585
]
Evgenii Bovykin commented on COMPRESS-565:
--
Any estimate on when this fix will be released? It's a quite important problem
to fix for us.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289908#comment-17289908
]
Evgenii Bovykin commented on COMPRESS-565:
--
[~peterlee] thanks, did a check with Zip64Mode.AlwaysWithComestibles,
everything seems to be ok.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289867#comment-17289867
]
Peter Lee commented on COMPRESS-565:
Hi [~evgenii.bovykin]
Please have a look if my PR works or not :
[https://github.com/PeterAlfredLee/commons-compress.git] with branch
COMPRESS-565
I tested on my Windows with Expand-Archive/7z and was successfully extracted.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17288128#comment-17288128
]
Peter Lee commented on COMPRESS-565:
> Using the attached jar helps my issue. Expand-Archive successfully extracts
>the archive, 7z doesn't complain about headers.
Great. I didn't expact that Expand-Archive and 7z are reading CFH.
> [~peterlee]no problem if you want to port it, but I can do so as well.
I can do this with a GH pull request.:)
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287878#comment-17287878
]
Stefan Bodewig commented on COMPRESS-565:
-
[~peterlee]no problem if you want to port it, but I can do so as well.
What I would do:
* introduce another constant to Zip64Mode that works like ALWAYS for the LFH
and AS_NEEDED for CFH. This would be a compromise that would allow Powershell
(and likely Excel) to read the archive.
* ask [~evgenii.bovykin] to try a new snapshot using that new value rather
than ALWAYS.
* document that at the zip page on the site
* not change any defaults WRT Zip64Mode
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287865#comment-17287865
]
Evgenii Bovykin commented on COMPRESS-565:
--
Using the attached jar helps my issue. Expand-Archive successfully extracts the
archive, 7z doesn't complain about headers.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287847#comment-17287847
]
Peter Lee commented on COMPRESS-565:
> this wouldn't be sensitive to the unicode extra field, but I really suspect
>the lfh offset inside of the central directory to be what causes the problem.
Interesting. I can give it a try tomorrow, and I would port out the result then.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287800#comment-17287800
]
Stefan Bodewig commented on COMPRESS-565:
-
[~evgenii.bovykin] could you please give the jar I've attached a try? This is
identical to current git master but handles the ZIP64 entry of the central
directory in a different way - this wouldn't be sensitive to the unicode extra
field, but I really suspect the lfh offset inside of the central directory to
be what causes the problem.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287771#comment-17287771
]
Stefan Bodewig commented on COMPRESS-565:
-
Too bad, thank you.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287737#comment-17287737
]
Evgenii Bovykin commented on COMPRESS-565:
--
With -bb3 it still prints the same.
{{}}
{{7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21}}
{{Scanning the drive for archives:}}
{{1 file, 5096123 bytes (4977 KiB)}}
{{Testing archive: artifacts_artifacts_3_artifacts.zip}}{{WARNINGS:}}
{{Headers Error}}
{{--}}
{{Path = artifacts_artifacts_3_artifacts.zip}}
{{Type = zip}}
{{WARNINGS:}}
{{Headers Error}}
{{Physical Size = 5096123}}
{{64-bit = +}}
{{Characteristics = Zip64}}{{T big.txt}}
{{Everything is Ok}}
{{Archives with Warnings: 1}}
{{Warnings: 1}}
{{Size: 524288}}
{{Compressed: 5096123}}
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287727#comment-17287727
]
Stefan Bodewig commented on COMPRESS-565:
-
If I read the command line options correctly, throwing in -bb3 as switch might
provide more extensive information.
{{7z t -bb3 ..}}.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287722#comment-17287722
]
Evgenii Bovykin commented on COMPRESS-565:
--
Unfortunately, the output is not very informative
{{7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21}}
{{Scanning the drive for archives:}}
{{1 file, 5096123 bytes (4977 KiB)}}{{Testing archive:
artifacts_artifacts_3_artifacts.zip}}{{WARNINGS:}}
{{Headers Error}}
{{--}}
{{Path = artifacts_artifacts_3_artifacts.zip}}
{{Type = zip}}
{{WARNINGS:}}
{{Headers Error}}
{{Physical Size = 5096123}}
{{64-bit = +}}
{{Characteristics = Zip}}{{64}}
{{Everything is Ok}}
{{Archives with Warnings: 1}}
{{Warnings: 1}}
{{Size: 524288}}
{{Compressed: 5096123}}
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287718#comment-17287718
]
Stefan Bodewig commented on COMPRESS-565:
-
Does 7z on Windows provide you with more details? I'm not using Windows
normally, so checking would take some time ...
What we have changed between 1.9 and 1.20 is the handling of the "relative
offset of local header" inside the central directory entry - at least this is
where I see the created archives differ. This is one of the problems Excel has
had with our archives as well. To be honest I believe our implementation is
correct and Excel is wrong - this is why I'd really like to know what bothers
7z.
Finally are any of the consumers of your archive actually using the Unicode
extra fields? If not and if leaving this out fixes the problem then I'd just
drop it. But I must admit I'm not really sure how/why the unicode extra fields
changes the outcome in any way.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287705#comment-17287705
]
Stefan Bodewig commented on COMPRESS-565:
-
just as a data-point, p7z 16 for Linux (the latest version available as apt
package) doesn't see any problems with the archive.
I've run Peter's test code in Evgenii's repo with both 1.20 and 1.9 and the
LFHs written for archive/input.bin are identical (except for the timestamps
contained within).
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287701#comment-17287701
]
Evgenii Bovykin commented on COMPRESS-565:
--
I'm happy to provide you with any additional information, as this issue is
quite critical for us and stops us from fully upgrading to the new version of
the library.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287700#comment-17287700
]
Stefan Bodewig commented on COMPRESS-565:
-
Yes, it might be open source (actually it most probably leaves the archive
handling to the classes in .NET Framework or Core which may delegate to to
Windows native libraries on Windows and zlib on non-Windows platforms), but
that doesn't mean figuring out where the errors is would be easy ;)
Unfortunately the full error isn't really helpful, I was hoping it would say
what exactly it doesn't like about the local file header. Sorry for bothering
you.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287687#comment-17287687
]
Evgenii Bovykin commented on COMPRESS-565:
--
Expand-Archive is open sourced, you can find its implementation here -
[https://github.com/PowerShell/Microsoft.PowerShell.Archive/blob/master/Microsoft.PowerShell.Archive/Microsoft.PowerShell.Archive.psm1#L249]
The full error is
Exception calling "ExtractToFile" with "3" argument(s): "A local file header is
corrupt."
At
C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Archive\Microsoft.PowerShell.Archive.psm1:1056
char:25
+ ... [System.IO.Compression.ZipFileExtensions]::ExtractToFile( ...
+ ~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : InvalidDataException
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287682#comment-17287682
]
Stefan Bodewig commented on COMPRESS-565:
-
There are known incompatibilities with our Zip64 implementation and what Excel
expects, see https://issues.apache.org/jira/browse/COMPRESS-474 and I wouldn't
be surprised if PowerShell used the same code. But this should only cause
problems if the archive contains files smaller than 4GB and not be related to
Unicode Extra Fields at all.
Please include the exact error messages when reporting an interoperability
issue not just "error about corrupted headers". If a tool fails to extract
archives created by Commons Compress than it could be the fault of either side
- given the spec is vague enough and implementations differ in their
interpretation of it.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287676#comment-17287676
]
Stefan Bodewig commented on COMPRESS-565:
-
[~peterlee] can you post the warnings 7z emits here, please?
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287589#comment-17287589
]
Peter Lee commented on COMPRESS-565:
I'm not familiar with *Expand-Archive Powershell utility*. Is it open sourced
or not? I can't find anything on google.
7zip is open sourced but I'm not familiar with its code.:(
The difference between using
_output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)_
or not using it is:
whether we are adding the extra field _Info-ZIP Unicode Path Extra Field_ in
the extra field or not. And I think the reason why 7z is complaining and
*Expand-Archive Powershell utility* on Windows can't extract the archive is :
*_Info-ZIP Unicode Path Extra Field_ is not supported by them*.
See also: sector 4.6.9 of [zip
APPNOTE|https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] for more
detailed information
With _ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS_ being set, we will
always add the _Info-ZIP Unicode Path Extra Field_, which can be seen in the
generated zip:
!image-2021-02-20-15-51-21-747.png!
I can make some simple explanations :
First of all, zip format is using little endian.
The first 2 bytes 0x7075 is the signature of _Info-ZIP Unicode Path Extra
Field_.And the 0x000e is the size of this field, which is 14.
The 0x01 is the version of this extra field, which is always 1 now(according to
the [zip APPNOTE|https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT]).
The 4 bytes 0x7df6c07c is the CRC32 checksum of the file name(which can be
easily checked with any CRC32 check tools using the name _input.bin_).
The 9 bytes 0x69 6e 70 75 74 2e 62 69 6e is the UTF-8 value of the file name,
which is _input.bin_.
You can see that 9 + 4 + 1 = 14 is exactly the length of this field I
mentioned. So I think we have built a correct _Info-ZIP Unicode Path Extra
Field._
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287572#comment-17287572
]
Peter Lee commented on COMPRESS-565:
I see. I can reproduce this issue now.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286988#comment-17286988
]
Evgeny Bovykin commented on COMPRESS-565:
-
??Is the input.bin an empty file with size of 5GB???
Yes. Your code seems to be ok.
??I'm testing on Ubuntu and I can successfully extract the generated output.zip
using _unzip_ command without reporting any errors.??
Yes, some tools ignore the incorrect headers and just extract the archive. As I
said, *Expand-Archive Powershell utility* on Windows can't extract the archive.
With 7zip 19.0 you can use *7z t archive_name* command, it will report the
incorrect headers.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286987#comment-17286987
]
Evgeny Bovykin commented on COMPRESS-565:
-
??Is the input.bin an empty file with size of 5GB???
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286905#comment-17286905
]
Peter Lee commented on COMPRESS-565:
I also tested in windows using 7zip 19.00, and it successfully extracted the
_output.zip_ without reporting any errors.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286884#comment-17286884
]
Peter Lee commented on COMPRESS-565:
And I'm testing with UnZip 6.00.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286882#comment-17286882
]
Peter Lee commented on COMPRESS-565:
I'm testing like this:
{code:java}
RandomAccessFile randomAccessFile = new RandomAccessFile("archive/input.bin",
"rw");
randomAccessFile.setLength(1024L * 1024 * 1024 * 5);
File outputFile = new File("archive/output.zip");
outputFile.createNewFile();
try(ZipArchiveOutputStream zipArchiveOutputStream = new
ZipArchiveOutputStream(new BufferedOutputStream(new
FileOutputStream(outputFile {
zipArchiveOutputStream.setUseZip64(Zip64Mode.Always);
zipArchiveOutputStream.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS);
zipArchiveOutputStream.putArchiveEntry(new ZipArchiveEntry("input.bin"));
InputStream inputStream = new FileInputStream("archive/input.bin");
IOUtils.copy(inputStream, zipArchiveOutputStream);
zipArchiveOutputStream.closeArchiveEntry();
}
{code}
And I think this test is exactly the same as yours.
I'm testing on Ubuntu and I can successfully extract the generated output.zip
using _unzip_ command without reporting any errors.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286828#comment-17286828
]
Peter Lee commented on COMPRESS-565:
Not familiar with gradle and kotlin. :(
Is the input.bin an empty file with size of 5GB?
Or could you provide an example with Java using maven?
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285732#comment-17285732
]
Evgenii Bovykin commented on COMPRESS-565:
--
[~peterlee] isn't the 1.20 the latest version? The problem reproduces on it. If
not, could you please point me to the latest version?
`archive/input.bin` file is generated on the fly when runnig the main.kt file.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285685#comment-17285685
]
Peter Lee commented on COMPRESS-565:
Hi, Evgenii. Thank you for reporting this.
Could you reproduce this issue using the latest version of Compress?
And I didn't find a 'archive/input.bin' in your GH project. Is there any way
you can share it(if this file doesn't contain anything important)?
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
