[jira] [Commented] (COMPRESS-562) ZipArchiveInputStream fails with unexpected record signature while ZipInputStream from java.util.zip succeeds
[
https://issues.apache.org/jira/browse/COMPRESS-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17349780#comment-17349780
]
Stefan Bodewig commented on COMPRESS-562:
-
well the zero bytes look like padding, unfortunately padding isn't mentioned
anywhere. But it looks as if the signing header wanted to start at a 512 byte
block boundary. We could try to skip all-0 bytes until we reach a block
boundary if we don't recognize the suspect header and it starts with 0s. There
we could try to see whether we find a good signature. But like so often we'd
only be fighting against limitations of trying to stream a ZIP archive.
In general {{ZipArchiveInputStream}} really is a very limited tool. The ZIP
archive format is not one that lends itself to streaming, it works best with
random access this is why {{ZipFile}} is so much superior to it. If you can use
{{ZipFile}}, do so. Not just because the APK signature block problem goes away,
it is so much "more correct", see
https://commons.apache.org/proper/commons-compress/zip.html#ZipArchiveInputStream_vs_ZipFile
> ZipArchiveInputStream fails with unexpected record signature while
> ZipInputStream from java.util.zip succeeds
> -
>
> Key: COMPRESS-562
> URL: https://issues.apache.org/jira/browse/COMPRESS-562
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
> Environment: Zip 3.0 (July 5th 2008), by Info-ZIP, Compiled with gcc
> 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14) for Unix (Mac OS X)
> on Feb 22 2019.
> osx 10.14.6, AdoptOpenJDK 11.0.7
>Reporter: Oleksii Khomchenko
>Priority: Major
> Attachments: apk.PNG, test-services-1.1.0.apk
>
>
> Thank you a lot for the library.
>
> I recently encountered next issue:
> {code:java}
> Exception in thread "main" java.util.zip.ZipException: Unexpected record
> signature: 0X0
> {code}
> is thrown when reading test-services-1.1.0.apk from
> [https://maven.google.com/web/index.html?q=test-ser#androidx.test.services:test-services:1.1.0]
> via commons-compress:1.20 while java.util.zip reads it without the exception.
>
> {code:java}
> public class UnzipTestServicesSample {
> public static void main(String[] args) throws Exception {
> Path p = Paths.get("test-services-1.1.0.apk");
> System.out.println("\n=== java std zip ===\n");
> try (InputStream is = Files.newInputStream(p); ZipInputStream zis =
> new ZipInputStream(is)) {
> ZipEntry entry;
> while ((entry = zis.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> System.out.println("\n=== apache compress zip ===\n");
> try (InputStream is = Files.newInputStream(p); ArchiveInputStream ais
> = new ZipArchiveInputStream(is)) {
> ArchiveEntry entry;
> while ((entry = ais.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> }
> }{code}
>
> zip -T says that archive is fine:
>
> {code:java}
> $ zip -T test-services-1.1.0.apk
> test of test-services-1.1.0.apk OK{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-562) ZipArchiveInputStream fails with unexpected record signature while ZipInputStream from java.util.zip succeeds
[
https://issues.apache.org/jira/browse/COMPRESS-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282311#comment-17282311
]
Oleksii Khomchenko commented on COMPRESS-562:
-
Good evening.
I apologize for a late reply, got overwhelmed at work. I really appreciate your
findings, thank you a lot for insights.
I am curious what do you think about potential lenient option for
ZipArchiveInputStream. I assume it can be tricky as random zero bytes before
signing block is only one case out of many of them.
I will consider using ZipFile for my case.
Thank you again.
> ZipArchiveInputStream fails with unexpected record signature while
> ZipInputStream from java.util.zip succeeds
> -
>
> Key: COMPRESS-562
> URL: https://issues.apache.org/jira/browse/COMPRESS-562
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
> Environment: Zip 3.0 (July 5th 2008), by Info-ZIP, Compiled with gcc
> 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14) for Unix (Mac OS X)
> on Feb 22 2019.
> osx 10.14.6, AdoptOpenJDK 11.0.7
>Reporter: Oleksii Khomchenko
>Priority: Major
> Attachments: apk.PNG, test-services-1.1.0.apk
>
>
> Thank you a lot for the library.
>
> I recently encountered next issue:
> {code:java}
> Exception in thread "main" java.util.zip.ZipException: Unexpected record
> signature: 0X0
> {code}
> is thrown when reading test-services-1.1.0.apk from
> [https://maven.google.com/web/index.html?q=test-ser#androidx.test.services:test-services:1.1.0]
> via commons-compress:1.20 while java.util.zip reads it without the exception.
>
> {code:java}
> public class UnzipTestServicesSample {
> public static void main(String[] args) throws Exception {
> Path p = Paths.get("test-services-1.1.0.apk");
> System.out.println("\n=== java std zip ===\n");
> try (InputStream is = Files.newInputStream(p); ZipInputStream zis =
> new ZipInputStream(is)) {
> ZipEntry entry;
> while ((entry = zis.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> System.out.println("\n=== apache compress zip ===\n");
> try (InputStream is = Files.newInputStream(p); ArchiveInputStream ais
> = new ZipArchiveInputStream(is)) {
> ArchiveEntry entry;
> while ((entry = ais.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> }
> }{code}
>
> zip -T says that archive is fine:
>
> {code:java}
> $ zip -T test-services-1.1.0.apk
> test of test-services-1.1.0.apk OK{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-562) ZipArchiveInputStream fails with unexpected record signature while ZipInputStream from java.util.zip succeeds
[
https://issues.apache.org/jira/browse/COMPRESS-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266538#comment-17266538
]
Peter Lee commented on COMPRESS-562:
The attached test-services-1.1.0.apk I upload is the one I mentioned - I
removed the redundant bytes of zero and it could be successfully read with
ZipArchiveInputStream.
> ZipArchiveInputStream fails with unexpected record signature while
> ZipInputStream from java.util.zip succeeds
> -
>
> Key: COMPRESS-562
> URL: https://issues.apache.org/jira/browse/COMPRESS-562
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
> Environment: Zip 3.0 (July 5th 2008), by Info-ZIP, Compiled with gcc
> 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14) for Unix (Mac OS X)
> on Feb 22 2019.
> osx 10.14.6, AdoptOpenJDK 11.0.7
>Reporter: Oleksii Khomchenko
>Priority: Major
> Attachments: apk.PNG, test-services-1.1.0.apk
>
>
> Thank you a lot for the library.
>
> I recently encountered next issue:
> {code:java}
> Exception in thread "main" java.util.zip.ZipException: Unexpected record
> signature: 0X0
> {code}
> is thrown when reading test-services-1.1.0.apk from
> [https://maven.google.com/web/index.html?q=test-ser#androidx.test.services:test-services:1.1.0]
> via commons-compress:1.20 while java.util.zip reads it without the exception.
>
> {code:java}
> public class UnzipTestServicesSample {
> public static void main(String[] args) throws Exception {
> Path p = Paths.get("test-services-1.1.0.apk");
> System.out.println("\n=== java std zip ===\n");
> try (InputStream is = Files.newInputStream(p); ZipInputStream zis =
> new ZipInputStream(is)) {
> ZipEntry entry;
> while ((entry = zis.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> System.out.println("\n=== apache compress zip ===\n");
> try (InputStream is = Files.newInputStream(p); ArchiveInputStream ais
> = new ZipArchiveInputStream(is)) {
> ArchiveEntry entry;
> while ((entry = ais.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> }
> }{code}
>
> zip -T says that archive is fine:
>
> {code:java}
> $ zip -T test-services-1.1.0.apk
> test of test-services-1.1.0.apk OK{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-562) ZipArchiveInputStream fails with unexpected record signature while ZipInputStream from java.util.zip succeeds
[
https://issues.apache.org/jira/browse/COMPRESS-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266536#comment-17266536
]
Peter Lee commented on COMPRESS-562:
Disclaimer : not familiar with zpk
I checked the apk file(test-services-1.1.0.apk) and found something strange :
There are 237 bytes of zero before the actual apk signing block.
!apk.PNG!
These redundant bytes of zero broke the read of apk signing block - that's why
we are throwing the unexpected record signature exception. And I can
successfully read this apk file with these bytes removed.
Accoarding to the [apk signing block
specification|[https://source.android.com/security/apksigning/v2]
,|https://source.android.com/security/apksigning/v2],]these bytes are not
mentioned. Please feel free to tell me if they are reasonable.
In short words, I believe the apk file is corrupted and could not be
successfully read using ZipArchiveInputStream(but can be read with ZipFile).
BTW : Why java standard zip(ZipInputStream) can successfully read this apk?
I check the code of ZipInputStream and found they didn't check if a Central
Directory File or APK signing block is met. They simply return null if the
signature is not the one of Local File Header. That's why they didn't report
any exceptions.
See also : [ZipInputStream in
OpenJDK|https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/util/zip/ZipInputStream.java#L284]
> ZipArchiveInputStream fails with unexpected record signature while
> ZipInputStream from java.util.zip succeeds
> -
>
> Key: COMPRESS-562
> URL: https://issues.apache.org/jira/browse/COMPRESS-562
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
> Environment: Zip 3.0 (July 5th 2008), by Info-ZIP, Compiled with gcc
> 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14) for Unix (Mac OS X)
> on Feb 22 2019.
> osx 10.14.6, AdoptOpenJDK 11.0.7
>Reporter: Oleksii Khomchenko
>Priority: Major
> Attachments: apk.PNG
>
>
> Thank you a lot for the library.
>
> I recently encountered next issue:
> {code:java}
> Exception in thread "main" java.util.zip.ZipException: Unexpected record
> signature: 0X0
> {code}
> is thrown when reading test-services-1.1.0.apk from
> [https://maven.google.com/web/index.html?q=test-ser#androidx.test.services:test-services:1.1.0]
> via commons-compress:1.20 while java.util.zip reads it without the exception.
>
> {code:java}
> public class UnzipTestServicesSample {
> public static void main(String[] args) throws Exception {
> Path p = Paths.get("test-services-1.1.0.apk");
> System.out.println("\n=== java std zip ===\n");
> try (InputStream is = Files.newInputStream(p); ZipInputStream zis =
> new ZipInputStream(is)) {
> ZipEntry entry;
> while ((entry = zis.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> System.out.println("\n=== apache compress zip ===\n");
> try (InputStream is = Files.newInputStream(p); ArchiveInputStream ais
> = new ZipArchiveInputStream(is)) {
> ArchiveEntry entry;
> while ((entry = ais.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> }
> }{code}
>
> zip -T says that archive is fine:
>
> {code:java}
> $ zip -T test-services-1.1.0.apk
> test of test-services-1.1.0.apk OK{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-562) ZipArchiveInputStream fails with unexpected record signature while ZipInputStream from java.util.zip succeeds
[
https://issues.apache.org/jira/browse/COMPRESS-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266534#comment-17266534
]
Peter Lee commented on COMPRESS-562:
I tested to read this .apk with ZipFile and can be successfully read - maybe
you can try to read it with ZipFile instead of ZipArchiveInputStream.
> ZipArchiveInputStream fails with unexpected record signature while
> ZipInputStream from java.util.zip succeeds
> -
>
> Key: COMPRESS-562
> URL: https://issues.apache.org/jira/browse/COMPRESS-562
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
> Environment: Zip 3.0 (July 5th 2008), by Info-ZIP, Compiled with gcc
> 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14) for Unix (Mac OS X)
> on Feb 22 2019.
> osx 10.14.6, AdoptOpenJDK 11.0.7
>Reporter: Oleksii Khomchenko
>Priority: Major
>
> Thank you a lot for the library.
>
> I recently encountered next issue:
> {code:java}
> Exception in thread "main" java.util.zip.ZipException: Unexpected record
> signature: 0X0
> {code}
> is thrown when reading test-services-1.1.0.apk from
> [https://maven.google.com/web/index.html?q=test-ser#androidx.test.services:test-services:1.1.0]
> via commons-compress:1.20 while java.util.zip reads it without the exception.
>
> {code:java}
> public class UnzipTestServicesSample {
> public static void main(String[] args) throws Exception {
> Path p = Paths.get("test-services-1.1.0.apk");
> System.out.println("\n=== java std zip ===\n");
> try (InputStream is = Files.newInputStream(p); ZipInputStream zis =
> new ZipInputStream(is)) {
> ZipEntry entry;
> while ((entry = zis.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> System.out.println("\n=== apache compress zip ===\n");
> try (InputStream is = Files.newInputStream(p); ArchiveInputStream ais
> = new ZipArchiveInputStream(is)) {
> ArchiveEntry entry;
> while ((entry = ais.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> }
> }{code}
>
> zip -T says that archive is fine:
>
> {code:java}
> $ zip -T test-services-1.1.0.apk
> test of test-services-1.1.0.apk OK{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (COMPRESS-562) ZipArchiveInputStream fails with unexpected record signature while ZipInputStream from java.util.zip succeeds
[
https://issues.apache.org/jira/browse/COMPRESS-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17266530#comment-17266530
]
Peter Lee commented on COMPRESS-562:
This is associated with COMPRESS-455 and COMPRESS-461.
I'm trying to find out the problem. Considering I'm not familiar with APK
specification, a patch or PR is always welcome. :)
> ZipArchiveInputStream fails with unexpected record signature while
> ZipInputStream from java.util.zip succeeds
> -
>
> Key: COMPRESS-562
> URL: https://issues.apache.org/jira/browse/COMPRESS-562
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.20
> Environment: Zip 3.0 (July 5th 2008), by Info-ZIP, Compiled with gcc
> 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14) for Unix (Mac OS X)
> on Feb 22 2019.
> osx 10.14.6, AdoptOpenJDK 11.0.7
>Reporter: Oleksii Khomchenko
>Priority: Major
>
> Thank you a lot for the library.
>
> I recently encountered next issue:
> {code:java}
> Exception in thread "main" java.util.zip.ZipException: Unexpected record
> signature: 0X0
> {code}
> is thrown when reading test-services-1.1.0.apk from
> [https://maven.google.com/web/index.html?q=test-ser#androidx.test.services:test-services:1.1.0]
> via commons-compress:1.20 while java.util.zip reads it without the exception.
>
> {code:java}
> public class UnzipTestServicesSample {
> public static void main(String[] args) throws Exception {
> Path p = Paths.get("test-services-1.1.0.apk");
> System.out.println("\n=== java std zip ===\n");
> try (InputStream is = Files.newInputStream(p); ZipInputStream zis =
> new ZipInputStream(is)) {
> ZipEntry entry;
> while ((entry = zis.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> System.out.println("\n=== apache compress zip ===\n");
> try (InputStream is = Files.newInputStream(p); ArchiveInputStream ais
> = new ZipArchiveInputStream(is)) {
> ArchiveEntry entry;
> while ((entry = ais.getNextEntry()) != null) {
> System.out.println("entry: " + entry.getName());
> }
> }
> }
> }{code}
>
> zip -T says that archive is fine:
>
> {code:java}
> $ zip -T test-services-1.1.0.apk
> test of test-services-1.1.0.apk OK{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
