[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287847#comment-17287847
]
Peter Lee commented on COMPRESS-565:
> this wouldn't be sensitive to the unicode extra field, but I really suspect
>the lfh offset inside of the central directory to be what causes the problem.
Interesting. I can give it a try tomorrow, and I would port out the result then.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IO-663) FileUtils.copyDirectory(File srcDir, File destDir) fails on Windows
[
https://issues.apache.org/jira/browse/IO-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287840#comment-17287840
]
Gary D. Gregory commented on IO-663:
Hi [~melloware]
May you please be more precise in your error description?
> FileUtils.copyDirectory(File srcDir, File destDir) fails on Windows
> ---
>
> Key: IO-663
> URL: https://issues.apache.org/jira/browse/IO-663
> Project: Commons IO
> Issue Type: Bug
>Reporter: Elliotte Rusty Harold
>Priority: Critical
>
> This bug is shared (likely because of code copied from one place to another)
> between the similar methods in commons IO, codehaus-plexus-utils, and
> maven-shared-utils.
> I don't have an isolated test case because this bug is platform specific and
> I've only seen it in Jenkins CI builds on Windows using JDK 7 through 15. I
> don't have a Windows system handy to test it. However it is reproducible.
> Typical code that triggers it is in RestoreBackupPomsPhaseTest in
> maven-release:
> {{// copy poms so tests are valid without clean}}
> {{ File sourceDir = getTestFile( "src/test/resources" + projectPath );}}
> {{ File testDir = getTestFile( "target/test-classes" + projectPath );}}
> {{ FileUtils.copyDirectoryStructure( sourceDir, testDir );}}
> I don't know whether there might be something weird in the setup of those two
> directories that's involved.
> Typical error message is:
> Caused by: java.nio.file.FileSystemException:
>
> F:\jenkins\jenkins-slave\workspace\maven-box_maven-release_windows@2@2\windows-jdk8-m3.6.x_build\maven-release-manager\target\test-classes\projects\restore-backup-poms\basic-pom\pom.xml:
> The process cannot access the file because it is being used by another
> process
> "The process cannot access the file because it is being used by another
> process" I think points to the root of the bug. This is a Windows file system
> error message.
> Some history is here where I noticed it:
> [https://github.com/apache/maven-release/pull/42]
> In this case, I started with plexus-utils 3.1.0 which worked, upgraded to
> plexus-utils 3.3.0, which didn't. And then tried the FileUtils.copyDirectory
> from both maven-shared-utils and commons-io, all of which failed in the same
> way.
> I think this is caused by the use of NIO, which doesn't work quite the same
> when copying files on Windows as on Linux and Mac OS X.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-statistics] mmalenic commented on pull request #26: Add truncated normal distribution.
mmalenic commented on pull request #26: URL: https://github.com/apache/commons-statistics/pull/26#issuecomment-782765017 Hi Alex, thanks for the feedback, I've implemented those changes. I was thinking about the test class as well and while I was writing it I was attempting to use parameterized tests, however I wasn't able to come up with an elegant solution. Perhaps the abstract base class could somehow use parameterized tests? In terms of adding distribution exceptions, how do you feel about adding a similar exception here: https://github.com/apache/commons-statistics/blob/1339b4ce6e217525bee893f1365b93a68acf/commons-statistics-distribution/src/main/java/org/apache/commons/statistics/distribution/TruncatedNormalDistribution.java#L66-L68 Which indicates that the condition is too large or equal to? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work logged] (IO-651) Provide method to retrieve data from DeferredFileOutputStream as InputStream
[
https://issues.apache.org/jira/browse/IO-651?focusedWorklogId=555277=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-555277
]
ASF GitHub Bot logged work on IO-651:
-
Author: ASF GitHub Bot
Created on: 20/Feb/21 22:21
Start Date: 20/Feb/21 22:21
Worklog Time Spent: 10m
Work Description: garydgregory commented on a change in pull request #206:
URL: https://github.com/apache/commons-io/pull/206#discussion_r579716893
##
File path:
src/main/java/org/apache/commons/io/output/DeferredFileOutputStream.java
##
@@ -272,4 +274,31 @@ public void writeTo(final OutputStream outputStream)
throws IOException {
}
}
}
+
+/**
+ * Gets the current contents of this byte stream as an {@link InputStream}.
+ * If the data for this output stream has been retained in memory, the
+ * returned stream is backed by buffers of {@code this} stream,
+ * avoiding memory allocation and copy, thus saving space and time.
+ * Otherwise, the returned stream will be one that is created from the data
+ * that has been committed to disk.
+ *
+ * @return the current contents of this output stream.
+ * @throws IOException if this stream is not yet closed or an error occurs.
+ * @see org.apache.commons.io.output.ByteArrayOutputStream#toInputStream()
Review comment:
Ping. FYI, you should run the _default_ Maven build: 'mvn'.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 555277)
Time Spent: 2h 10m (was: 2h)
> Provide method to retrieve data from DeferredFileOutputStream as InputStream
>
>
> Key: IO-651
> URL: https://issues.apache.org/jira/browse/IO-651
> Project: Commons IO
> Issue Type: Improvement
> Components: Streams/Writers
>Affects Versions: 2.6
>Reporter: John Mark
>Priority: Major
> Labels: easyfix, performance, pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> It would be very helpful to have to {{toInputStream()}} method on the
> {{DeferredFileOutputStream}} class. Besides for convenience, it would allow
> for improved efficiency since in the case of in-memory data the byte buffer
> would not need to be copied (as opposed to the current {{getData()}} method).
> The implementation is pretty simple and can be something like the following
> (based on the current {{writeTo(OutputStream)}} method):
> {code:java}
> public InputStream toInputStream() throws IOException {
> if (!closed) {
> throw new IOException("Stream not closed");
> }
> if (isInMemory()) {
> return memoryOutputStream.toInputStream();
> } else {
> return Files.newInputStream(outputFile.toPath());
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-io] garydgregory commented on a change in pull request #206: IO-651 Add DeferredFileOutputStream.getInputStream()
garydgregory commented on a change in pull request #206:
URL: https://github.com/apache/commons-io/pull/206#discussion_r579716893
##
File path:
src/main/java/org/apache/commons/io/output/DeferredFileOutputStream.java
##
@@ -272,4 +274,31 @@ public void writeTo(final OutputStream outputStream)
throws IOException {
}
}
}
+
+/**
+ * Gets the current contents of this byte stream as an {@link InputStream}.
+ * If the data for this output stream has been retained in memory, the
+ * returned stream is backed by buffers of {@code this} stream,
+ * avoiding memory allocation and copy, thus saving space and time.
+ * Otherwise, the returned stream will be one that is created from the data
+ * that has been committed to disk.
+ *
+ * @return the current contents of this output stream.
+ * @throws IOException if this stream is not yet closed or an error occurs.
+ * @see org.apache.commons.io.output.ByteArrayOutputStream#toInputStream()
Review comment:
Ping. FYI, you should run the _default_ Maven build: 'mvn'.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Work logged] (IO-720) fix error about usage of DirectBuffer in jdk 16/17
[ https://issues.apache.org/jira/browse/IO-720?focusedWorklogId=555276=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-555276 ] ASF GitHub Bot logged work on IO-720: - Author: ASF GitHub Bot Created on: 20/Feb/21 22:16 Start Date: 20/Feb/21 22:16 Worklog Time Spent: 10m Work Description: garydgregory merged pull request #205: URL: https://github.com/apache/commons-io/pull/205 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 555276) Time Spent: 50m (was: 40m) > fix error about usage of DirectBuffer in jdk 16/17 > -- > > Key: IO-720 > URL: https://issues.apache.org/jira/browse/IO-720 > Project: Commons IO > Issue Type: Improvement >Reporter: Jin Xu >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > https://github.com/apache/commons-io/pull/205 > usage of DirectBuffer will cause IllegalAccessError. > This pr aims to fix it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [commons-io] garydgregory merged pull request #205: [IO-720] Fix error about usage of DirectBuffer in JRE 16/17
garydgregory merged pull request #205: URL: https://github.com/apache/commons-io/pull/205 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [commons-lang] garydgregory commented on pull request #712: Fix broken Javadoc links to commons-text
garydgregory commented on pull request #712: URL: https://github.com/apache/commons-lang/pull/712#issuecomment-782756947 Merged. TY @smowton ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [commons-lang] garydgregory merged pull request #712: Fix broken Javadoc links to commons-text
garydgregory merged pull request #712: URL: https://github.com/apache/commons-lang/pull/712 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [commons-lang] garydgregory merged pull request #714: true & false String constant
garydgregory merged pull request #714: URL: https://github.com/apache/commons-lang/pull/714 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287800#comment-17287800
]
Stefan Bodewig edited comment on COMPRESS-565 at 2/20/21, 9:24 PM:
---
[~evgenii.bovykin] could you please give the jar I've attached a try? This is
identical to current git master but handles the ZIP64 entry of the central
directory in a different way - this wouldn't be sensitive to the unicode extra
field, but I really suspect the lfh offset inside of the central directory to
be what causes the problem.
If oyu prefer t build the jar yourself, it is the branch
compress-565-experiment I've just pushed.
was (Author: bodewig):
[~evgenii.bovykin] could you please give the jar I've attached a try? This is
identical to current git master but handles the ZIP64 entry of the central
directory in a different way - this wouldn't be sensitive to the unicode extra
field, but I really suspect the lfh offset inside of the central directory to
be what causes the problem.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287800#comment-17287800
]
Stefan Bodewig edited comment on COMPRESS-565 at 2/20/21, 9:24 PM:
---
[~evgenii.bovykin] could you please give the jar I've attached a try? This is
identical to current git master but handles the ZIP64 entry of the central
directory in a different way - this wouldn't be sensitive to the unicode extra
field, but I really suspect the lfh offset inside of the central directory to
be what causes the problem.
If you prefer to build the jar yourself, it is the branch
compress-565-experiment I've just pushed.
was (Author: bodewig):
[~evgenii.bovykin] could you please give the jar I've attached a try? This is
identical to current git master but handles the ZIP64 entry of the central
directory in a different way - this wouldn't be sensitive to the unicode extra
field, but I really suspect the lfh offset inside of the central directory to
be what causes the problem.
If oyu prefer t build the jar yourself, it is the branch
compress-565-experiment I've just pushed.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287800#comment-17287800
]
Stefan Bodewig commented on COMPRESS-565:
-
[~evgenii.bovykin] could you please give the jar I've attached a try? This is
identical to current git master but handles the ZIP64 entry of the central
directory in a different way - this wouldn't be sensitive to the unicode extra
field, but I really suspect the lfh offset inside of the central directory to
be what causes the problem.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Bodewig updated COMPRESS-565:
Attachment: commons-compress-1.21-SNAPSHOT.jar
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: commons-compress-1.21-SNAPSHOT.jar,
> image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287771#comment-17287771
]
Stefan Bodewig commented on COMPRESS-565:
-
Too bad, thank you.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (COMPRESS-566) make gzip deflate buffer size configurable
[
https://issues.apache.org/jira/browse/COMPRESS-566?focusedWorklogId=555239=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-555239
]
ASF GitHub Bot logged work on COMPRESS-566:
---
Author: ASF GitHub Bot
Created on: 20/Feb/21 17:51
Start Date: 20/Feb/21 17:51
Worklog Time Spent: 10m
Work Description: bokken commented on a change in pull request #168:
URL: https://github.com/apache/commons-compress/pull/168#discussion_r579686912
##
File path:
src/main/java/org/apache/commons/compress/compressors/gzip/GzipParameters.java
##
@@ -118,4 +120,28 @@ public int getOperatingSystem() {
public void setOperatingSystem(final int operatingSystem) {
this.operatingSystem = operatingSystem;
}
+
+/**
+ * Gets size of the buffer used to retrieve compressed data.
+ * @return The size of the buffer used to retrieve compressed data.
+ * @since 1.21
+ * @see #setBufferSize(int)
+ */
+public int getBufferSize() {
+return this.bufferSize;
+}
+
+/**
+ * Sets size of the buffer used to retrieve compressed data from
+ * {@link Deflater} and write to underlying {@link OutputStream}.
+ *
+ * @param bufferSize the bufferSize to set. Must be a positive value.
+ * @since 1.21
+ */
+public void setBufferSize(int bufferSize) {
+if (bufferSize <= 0) {
Review comment:
I personally do not see the value, but can add if desired.
There is certainly some threshold where larger values are counter
productive, but my opinion is consumers can determine that on their own.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 555239)
Time Spent: 1h 10m (was: 1h)
> make gzip deflate buffer size configurable
> --
>
> Key: COMPRESS-566
> URL: https://issues.apache.org/jira/browse/COMPRESS-566
> Project: Commons Compress
> Issue Type: Improvement
> Components: Compressors
>Reporter: Brett Okken
>Priority: Minor
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The deflateBuffer in GzipCompressorOutputStream is hardcoded to 512.
> It would be good if this could be configurable in GzipParameters.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-compress] bokken commented on a change in pull request #168: COMPRESS-566 allow gzip buffer size to be configured
bokken commented on a change in pull request #168:
URL: https://github.com/apache/commons-compress/pull/168#discussion_r579686912
##
File path:
src/main/java/org/apache/commons/compress/compressors/gzip/GzipParameters.java
##
@@ -118,4 +120,28 @@ public int getOperatingSystem() {
public void setOperatingSystem(final int operatingSystem) {
this.operatingSystem = operatingSystem;
}
+
+/**
+ * Gets size of the buffer used to retrieve compressed data.
+ * @return The size of the buffer used to retrieve compressed data.
+ * @since 1.21
+ * @see #setBufferSize(int)
+ */
+public int getBufferSize() {
+return this.bufferSize;
+}
+
+/**
+ * Sets size of the buffer used to retrieve compressed data from
+ * {@link Deflater} and write to underlying {@link OutputStream}.
+ *
+ * @param bufferSize the bufferSize to set. Must be a positive value.
+ * @since 1.21
+ */
+public void setBufferSize(int bufferSize) {
+if (bufferSize <= 0) {
Review comment:
I personally do not see the value, but can add if desired.
There is certainly some threshold where larger values are counter
productive, but my opinion is consumers can determine that on their own.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287737#comment-17287737
]
Evgenii Bovykin commented on COMPRESS-565:
--
With -bb3 it still prints the same.
{{}}
{{7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21}}
{{Scanning the drive for archives:}}
{{1 file, 5096123 bytes (4977 KiB)}}
{{Testing archive: artifacts_artifacts_3_artifacts.zip}}{{WARNINGS:}}
{{Headers Error}}
{{--}}
{{Path = artifacts_artifacts_3_artifacts.zip}}
{{Type = zip}}
{{WARNINGS:}}
{{Headers Error}}
{{Physical Size = 5096123}}
{{64-bit = +}}
{{Characteristics = Zip64}}{{T big.txt}}
{{Everything is Ok}}
{{Archives with Warnings: 1}}
{{Warnings: 1}}
{{Size: 524288}}
{{Compressed: 5096123}}
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287727#comment-17287727
]
Stefan Bodewig commented on COMPRESS-565:
-
If I read the command line options correctly, throwing in -bb3 as switch might
provide more extensive information.
{{7z t -bb3 ..}}.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287722#comment-17287722
]
Evgenii Bovykin edited comment on COMPRESS-565 at 2/20/21, 3:40 PM:
Unfortunately, the output is not very informative
{{7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21}}
{{Scanning the drive for archives:}}
{{1 file, 5096123 bytes (4977 KiB)}}{{Testing archive:
artifacts_artifacts_3_artifacts.zip}}
{{WARNINGS:}}
{{Headers Error}}
{{--}}
{{Path = artifacts_artifacts_3_artifacts.zip}}
{{Type = zip}}
{{WARNINGS:}}
{{Headers Error}}
{{Physical Size = 5096123}}
{{64-bit = +}}
{{Characteristics = Zip}}{{64}}
{{Everything is Ok}}
{{Archives with Warnings: 1}}
{{Warnings: 1}}
{{Size: 524288}}
{{Compressed: 5096123}}
was (Author: evgenii.bovykin):
Unfortunately, the output is not very informative
{{7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21}}
{{Scanning the drive for archives:}}
{{1 file, 5096123 bytes (4977 KiB)}}{{Testing archive:
artifacts_artifacts_3_artifacts.zip}}{{WARNINGS:}}
{{Headers Error}}
{{--}}
{{Path = artifacts_artifacts_3_artifacts.zip}}
{{Type = zip}}
{{WARNINGS:}}
{{Headers Error}}
{{Physical Size = 5096123}}
{{64-bit = +}}
{{Characteristics = Zip}}{{64}}
{{Everything is Ok}}
{{Archives with Warnings: 1}}
{{Warnings: 1}}
{{Size: 524288}}
{{Compressed: 5096123}}
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287722#comment-17287722
]
Evgenii Bovykin commented on COMPRESS-565:
--
Unfortunately, the output is not very informative
{{7-Zip 19.00 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2019-02-21}}
{{Scanning the drive for archives:}}
{{1 file, 5096123 bytes (4977 KiB)}}{{Testing archive:
artifacts_artifacts_3_artifacts.zip}}{{WARNINGS:}}
{{Headers Error}}
{{--}}
{{Path = artifacts_artifacts_3_artifacts.zip}}
{{Type = zip}}
{{WARNINGS:}}
{{Headers Error}}
{{Physical Size = 5096123}}
{{64-bit = +}}
{{Characteristics = Zip}}{{64}}
{{Everything is Ok}}
{{Archives with Warnings: 1}}
{{Warnings: 1}}
{{Size: 524288}}
{{Compressed: 5096123}}
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287718#comment-17287718
]
Stefan Bodewig commented on COMPRESS-565:
-
Does 7z on Windows provide you with more details? I'm not using Windows
normally, so checking would take some time ...
What we have changed between 1.9 and 1.20 is the handling of the "relative
offset of local header" inside the central directory entry - at least this is
where I see the created archives differ. This is one of the problems Excel has
had with our archives as well. To be honest I believe our implementation is
correct and Excel is wrong - this is why I'd really like to know what bothers
7z.
Finally are any of the consumers of your archive actually using the Unicode
extra fields? If not and if leaving this out fixes the problem then I'd just
drop it. But I must admit I'm not really sure how/why the unicode extra fields
changes the outcome in any way.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287705#comment-17287705
]
Stefan Bodewig commented on COMPRESS-565:
-
just as a data-point, p7z 16 for Linux (the latest version available as apt
package) doesn't see any problems with the archive.
I've run Peter's test code in Evgenii's repo with both 1.20 and 1.9 and the
LFHs written for archive/input.bin are identical (except for the timestamps
contained within).
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287701#comment-17287701
]
Evgenii Bovykin commented on COMPRESS-565:
--
I'm happy to provide you with any additional information, as this issue is
quite critical for us and stops us from fully upgrading to the new version of
the library.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287700#comment-17287700
]
Stefan Bodewig commented on COMPRESS-565:
-
Yes, it might be open source (actually it most probably leaves the archive
handling to the classes in .NET Framework or Core which may delegate to to
Windows native libraries on Windows and zlib on non-Windows platforms), but
that doesn't mean figuring out where the errors is would be easy ;)
Unfortunately the full error isn't really helpful, I was hoping it would say
what exactly it doesn't like about the local file header. Sorry for bothering
you.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-net] garydgregory merged pull request #60: Bump commons-io from 2.6 to 2.8.0
garydgregory merged pull request #60: URL: https://github.com/apache/commons-net/pull/60 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [commons-net] garydgregory merged pull request #74: Bump junit from 4.13.1 to 4.13.2
garydgregory merged pull request #74: URL: https://github.com/apache/commons-net/pull/74 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287687#comment-17287687
]
Evgenii Bovykin commented on COMPRESS-565:
--
Expand-Archive is open sourced, you can find its implementation here -
[https://github.com/PowerShell/Microsoft.PowerShell.Archive/blob/master/Microsoft.PowerShell.Archive/Microsoft.PowerShell.Archive.psm1#L249]
The full error is
Exception calling "ExtractToFile" with "3" argument(s): "A local file header is
corrupt."
At
C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Archive\Microsoft.PowerShell.Archive.psm1:1056
char:25
+ ... [System.IO.Compression.ZipFileExtensions]::ExtractToFile( ...
+ ~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : InvalidDataException
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287687#comment-17287687
]
Evgenii Bovykin edited comment on COMPRESS-565 at 2/20/21, 1:08 PM:
Expand-Archive is open sourced, you can find its implementation here -
[https://github.com/PowerShell/Microsoft.PowerShell.Archive/blob/master/Microsoft.PowerShell.Archive/Microsoft.PowerShell.Archive.psm1#L249]
The full error is
{{Exception calling "ExtractToFile" with "3" argument(s): "A local file header
is corrupt."}}
{{ At}}
{{
C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Archive\Microsoft.PowerShell.Archive.psm1:1056}}
{{ char:25}}
{{ + ... [System.IO.Compression.ZipFileExtensions]::ExtractToFile( ...}}
{{ + ~}}
{{ + CategoryInfo : NotSpecified: (:) [], MethodInvocationException}}
{{ + FullyQualifiedErrorId : InvalidDataException}}
was (Author: evgenii.bovykin):
Expand-Archive is open sourced, you can find its implementation here -
[https://github.com/PowerShell/Microsoft.PowerShell.Archive/blob/master/Microsoft.PowerShell.Archive/Microsoft.PowerShell.Archive.psm1#L249]
The full error is
Exception calling "ExtractToFile" with "3" argument(s): "A local file header is
corrupt."
At
C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\Microsoft.PowerShell.Archive\Microsoft.PowerShell.Archive.psm1:1056
char:25
+ ... [System.IO.Compression.ZipFileExtensions]::ExtractToFile( ...
+ ~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : InvalidDataException
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287682#comment-17287682
]
Stefan Bodewig commented on COMPRESS-565:
-
There are known incompatibilities with our Zip64 implementation and what Excel
expects, see https://issues.apache.org/jira/browse/COMPRESS-474 and I wouldn't
be surprised if PowerShell used the same code. But this should only cause
problems if the archive contains files smaller than 4GB and not be related to
Unicode Extra Fields at all.
Please include the exact error messages when reporting an interoperability
issue not just "error about corrupted headers". If a tool fails to extract
archives created by Commons Compress than it could be the fault of either side
- given the spec is vague enough and implementations differ in their
interpretation of it.
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287676#comment-17287676
]
Stefan Bodewig commented on COMPRESS-565:
-
[~peterlee] can you post the warnings 7z emits here, please?
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (LANG-1593) Common behaviour for StringUtils join APIs when called with char or String delimiter
[
https://issues.apache.org/jira/browse/LANG-1593?focusedWorklogId=555125=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-555125
]
ASF GitHub Bot logged work on LANG-1593:
Author: ASF GitHub Bot
Created on: 20/Feb/21 10:12
Start Date: 20/Feb/21 10:12
Worklog Time Spent: 10m
Work Description: HubertWo closed pull request #635:
URL: https://github.com/apache/commons-lang/pull/635
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 555125)
Time Spent: 1h 50m (was: 1h 40m)
> Common behaviour for StringUtils join APIs when called with char or String
> delimiter
>
>
> Key: LANG-1593
> URL: https://issues.apache.org/jira/browse/LANG-1593
> Project: Commons Lang
> Issue Type: Improvement
>Affects Versions: 3.4, 3.11
>Reporter: Kiruahxh
>Priority: Minor
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> For now, join(int[], char) is working well.
> However, the same join method called with a string delimiter behaves
> differently : it returns a single memory address which is not the desired
> behavior.
> I think that, for coherence, calling StringUtils with a char or String
> delimiter should return the exact same value.
> Ex :
> {code:java}
> CLASSPATH="./commons-lang3-3.11.jar" jshell
> | Welcome to JShell -- Version 11.0.8
> jshell> import org.apache.commons.lang3.StringUtils
> jshell> int[] arr = {1, 2, 3, 4, 5, 6, 7};
> jshell> String result = StringUtils.join(arr, '-');
> result ==> "1-2-3-4-5-6-7"
> jshell> String result = StringUtils.join(arr, "-");
> result ==> "[I@69663380-"
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-lang] HubertWo closed pull request #635: [LANG-1593] - Common behaviour for StringUtils join APIs when called with char or String delimiter
HubertWo closed pull request #635: URL: https://github.com/apache/commons-lang/pull/635 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work logged] (LANG-1593) Common behaviour for StringUtils join APIs when called with char or String delimiter
[
https://issues.apache.org/jira/browse/LANG-1593?focusedWorklogId=555119=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-555119
]
ASF GitHub Bot logged work on LANG-1593:
Author: ASF GitHub Bot
Created on: 20/Feb/21 10:03
Start Date: 20/Feb/21 10:03
Worklog Time Spent: 10m
Work Description: HubertWo commented on pull request #635:
URL: https://github.com/apache/commons-lang/pull/635#issuecomment-782599104
Pulled changes from apache:master for the third time.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 555119)
Time Spent: 1h 40m (was: 1.5h)
> Common behaviour for StringUtils join APIs when called with char or String
> delimiter
>
>
> Key: LANG-1593
> URL: https://issues.apache.org/jira/browse/LANG-1593
> Project: Commons Lang
> Issue Type: Improvement
>Affects Versions: 3.4, 3.11
>Reporter: Kiruahxh
>Priority: Minor
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> For now, join(int[], char) is working well.
> However, the same join method called with a string delimiter behaves
> differently : it returns a single memory address which is not the desired
> behavior.
> I think that, for coherence, calling StringUtils with a char or String
> delimiter should return the exact same value.
> Ex :
> {code:java}
> CLASSPATH="./commons-lang3-3.11.jar" jshell
> | Welcome to JShell -- Version 11.0.8
> jshell> import org.apache.commons.lang3.StringUtils
> jshell> int[] arr = {1, 2, 3, 4, 5, 6, 7};
> jshell> String result = StringUtils.join(arr, '-');
> result ==> "1-2-3-4-5-6-7"
> jshell> String result = StringUtils.join(arr, "-");
> result ==> "[I@69663380-"
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-lang] HubertWo commented on pull request #635: [LANG-1593] - Common behaviour for StringUtils join APIs when called with char or String delimiter
HubertWo commented on pull request #635: URL: https://github.com/apache/commons-lang/pull/635#issuecomment-782599104 Pulled changes from apache:master for the third time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (IMAGING-282) NullPointerException in org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata() when GraphicControlExtension is null
[ https://issues.apache.org/jira/browse/IMAGING-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno P. Kinoshita updated IMAGING-282: --- Assignee: Bruno P. Kinoshita > NullPointerException in > org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata() when > GraphicControlExtension is null > - > > Key: IMAGING-282 > URL: https://issues.apache.org/jira/browse/IMAGING-282 > Project: Commons Imaging > Issue Type: Bug > Components: Format: GIF >Affects Versions: 1.0-alpha2 >Reporter: Samuel Monsarrat >Assignee: Bruno P. Kinoshita >Priority: Minor > Attachments: problem.gif > > > The attached gif file produces a NullPointerException in > org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata() l553 due > to variable id not having a GraphicControlExtension component (id.gce = null). > This is set to null in findAllImageData l715. > Elsewhere in org.apache.commons.imaging.formats.gif.GifImageParser is tested > for null (l746). > This gif seems to contain incorrect metadata. Simply re-saving it with > ImageMagick correct the issue but since the gif is correctly displayed by all > viewers at my disposal it would seem that org.apache.commons.imaging needs to > be more permissive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IMAGING-282) NullPointerException in org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata() when GraphicControlExtension is null
[ https://issues.apache.org/jira/browse/IMAGING-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno P. Kinoshita updated IMAGING-282: --- Fix Version/s: 1.0-alpha3 > NullPointerException in > org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata() when > GraphicControlExtension is null > - > > Key: IMAGING-282 > URL: https://issues.apache.org/jira/browse/IMAGING-282 > Project: Commons Imaging > Issue Type: Bug > Components: Format: GIF >Affects Versions: 1.0-alpha2 >Reporter: Samuel Monsarrat >Assignee: Bruno P. Kinoshita >Priority: Minor > Fix For: 1.0-alpha3 > > Attachments: problem.gif > > > The attached gif file produces a NullPointerException in > org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata() l553 due > to variable id not having a GraphicControlExtension component (id.gce = null). > This is set to null in findAllImageData l715. > Elsewhere in org.apache.commons.imaging.formats.gif.GifImageParser is tested > for null (l746). > This gif seems to contain incorrect metadata. Simply re-saving it with > ImageMagick correct the issue but since the gif is correctly displayed by all > viewers at my disposal it would seem that org.apache.commons.imaging needs to > be more permissive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (COMPRESS-565) Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
[
https://issues.apache.org/jira/browse/COMPRESS-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287589#comment-17287589
]
Peter Lee commented on COMPRESS-565:
I'm not familiar with *Expand-Archive Powershell utility*. Is it open sourced
or not? I can't find anything on google.
7zip is open sourced but I'm not familiar with its code.:(
The difference between using
_output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)_
or not using it is:
whether we are adding the extra field _Info-ZIP Unicode Path Extra Field_ in
the extra field or not. And I think the reason why 7z is complaining and
*Expand-Archive Powershell utility* on Windows can't extract the archive is :
*_Info-ZIP Unicode Path Extra Field_ is not supported by them*.
See also: sector 4.6.9 of [zip
APPNOTE|https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] for more
detailed information
With _ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS_ being set, we will
always add the _Info-ZIP Unicode Path Extra Field_, which can be seen in the
generated zip:
!image-2021-02-20-15-51-21-747.png!
I can make some simple explanations :
First of all, zip format is using little endian.
The first 2 bytes 0x7075 is the signature of _Info-ZIP Unicode Path Extra
Field_.And the 0x000e is the size of this field, which is 14.
The 0x01 is the version of this extra field, which is always 1 now(according to
the [zip APPNOTE|https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT]).
The 4 bytes 0x7df6c07c is the CRC32 checksum of the file name(which can be
easily checked with any CRC32 check tools using the name _input.bin_).
The 9 bytes 0x69 6e 70 75 74 2e 62 69 6e is the UTF-8 value of the file name,
which is _input.bin_.
You can see that 9 + 4 + 1 = 14 is exactly the length of this field I
mentioned. So I think we have built a correct _Info-ZIP Unicode Path Extra
Field._
> Regression - Corrupted headers when using 64 bit ZipArchiveOutputStream
> ---
>
> Key: COMPRESS-565
> URL: https://issues.apache.org/jira/browse/COMPRESS-565
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
>Reporter: Evgenii Bovykin
>Assignee: Peter Lee
>Priority: Major
> Attachments: image-2021-02-20-15-51-21-747.png
>
>
> We've recently updated commons-compress library from version 1.9 to 1.20 and
> now experiencing the problem that didn't occur before.
>
> When using ZipArchiveOutputStream to archive 5Gb file and setting the
> following fields
> {{output.setUseZip64(Zip64Mode.Always)}}
>
> {{output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)}}
> resulting archive contains corrupted headers.
> *Expand-Archive Powershell utility cannot extract the archive at all with the
> error about corrupted header. 7zip also complains about it, but can extract
> the archive.*
>
> The problem didn't appear when using library version 1.9.
>
> I've created a sample project that reproduces the error -
> [https://github.com/missingdays/commons-compress-example]
> Issue doesn't reproduce if you do any of the following:
>
> # Downgrade library to version 1.9
> # Remove
> output.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.ALWAYS)
> # Remove output.setUseZip64(Zip64Mode.Always) and zip smaller file (e.g. 1Gb)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
