[ 
https://issues.apache.org/jira/browse/COMPRESS-623?focusedWorklogId=800262&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-800262
 ]

ASF GitHub Bot logged work on COMPRESS-623:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 12/Aug/22 10:26
            Start Date: 12/Aug/22 10:26
    Worklog Time Spent: 10m 
      Work Description: dweiss commented on code in PR #306:
URL: https://github.com/apache/commons-compress/pull/306#discussion_r944253806


##########
src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java:
##########
@@ -595,11 +598,16 @@ public InputStream getRawInputStream(final 
ZipArchiveEntry ze) {
         if (!(ze instanceof Entry)) {
             return null;
         }
-        final long start = ze.getDataOffset();
-        if (start == EntryStreamOffsets.OFFSET_UNKNOWN) {
-            return null;
+
+        try {
+            final long start = getDataOffset(ze);
+            if (start == EntryStreamOffsets.OFFSET_UNKNOWN) {
+                return null;
+            }
+            return createBoundedInputStream(start, ze.getCompressedSize());
+        } catch (IOException e) {
+            throw new UncheckedIOException(e);

Review Comment:
   It'd be probably cleaner to change the signature of getRawInputStream to 
throw IOException but this would be backward-incompatible.



##########
src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java:
##########
@@ -595,11 +598,16 @@ public InputStream getRawInputStream(final 
ZipArchiveEntry ze) {
         if (!(ze instanceof Entry)) {
             return null;
         }
-        final long start = ze.getDataOffset();
-        if (start == EntryStreamOffsets.OFFSET_UNKNOWN) {
-            return null;
+
+        try {
+            final long start = getDataOffset(ze);
+            if (start == EntryStreamOffsets.OFFSET_UNKNOWN) {
+                return null;
+            }
+            return createBoundedInputStream(start, ze.getCompressedSize());
+        } catch (IOException e) {
+            throw new UncheckedIOException(e);

Review Comment:
   I also don't know but it seems like the returned offset from getDataOffset 
should never be unknown so the check there maybe removed.



##########
src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java:
##########
@@ -613,7 +621,7 @@ public InputStream getRawInputStream(final ZipArchiveEntry 
ze) {
      * @throws IOException on error
      */
     public void copyRawEntries(final ZipArchiveOutputStream target, final 
ZipArchiveEntryPredicate predicate)
-            throws IOException {
+        throws IOException {

Review Comment:
   Sorry for whitespace changes - I can revert these (any way to reformat 
automatically to conform to the convention used?).





Issue Time Tracking
-------------------

    Worklog Id:     (was: 800262)
    Time Spent: 40m  (was: 0.5h)

> Speed up ZipFile's raw stream copying by not seeking for local headers until 
> they're needed
> -------------------------------------------------------------------------------------------
>
>                 Key: COMPRESS-623
>                 URL: https://issues.apache.org/jira/browse/COMPRESS-623
>             Project: Commons Compress
>          Issue Type: Improvement
>            Reporter: Dawid Weiss
>            Priority: Minor
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> I have a somewhat odd use case with gigabytes of ZIP files, each with 
> thousands of documents (on comparatively slow, network drives). We need to 
> restructure these ZIPs without the need to recompress files.
> The above turns out to work almost perfectly with raw-data copying ZipFile 
> offers but empirical tests showed a major slowdown in the initial opening of 
> zip files, linked to multiple reads/seeks for local file headers. If an 
> option is passed to ignore those headers, raw streams are inaccessible.
> I've taken a look at the code and the code in getRawInputStream could 
> basically do the same thing that getInputStream does - lazily load the 
> missing offset via getDataOffset(ZipEntry). In fact, getInputStream could 
> just call getRawInputStream directly, which avoids some code duplication. 
> I also restructured the code that maintains LinkedList for duplicated names a 
> bit. I think this bit could be removed altogether and replaced with a 
> pointer-chain list directly on Entry class instances (the map would then 
> point at the first entry; duplicated entries are an oddity and infrequent in 
> the wild).
> I see speedups for opening and copying random raw streams in the order of 
> 3-4x and all the current tests pass. I filed a PR at github - happy to 
> discuss it there.
> https://github.com/apache/commons-compress/pull/306



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to