Hi I use common-compression to with with large tar files (~50 GB). I saw huge performance degradation (10 times) when search for particular file using recent version (starting from 1.26) compared to 1.25 and below . the issue is compression IOUtils.skip now uses common-io IOUtils skip. old implementation was using skip on underlying stream to skip as much as possible, then read the remaining data. now, common-io IOUtils reads all the data instead of skipping it.
It makes searching in large TAR about 10 times slower and saturates my SSD IO. I use buffered stream, and huge buffer size helps a bit - it makes it 5 times slower instead of 10 times. I'd like to go back to old behavior and just skip the entries I don't need instead of reading them , but there's no proper way to modify the behavior in my app itself - skip is called from private methods My proposal is to make it configurable in TarArchiveInputStream , common-compression IOUtils or even common-io Utils how to skip - with pure read or with skip and read combination, or add protected method skip(InputStream in, long bytesToSkip) method in TarArchiveInputStream so that I could extend the class and modify the behavior. Eugene CONFIDENTIALIT? : Ce document est destin? uniquement ? la personne ou ? l'entit? ? qui il est adress?. L'information apparaissant dans ce document est de nature l?galement privil?gi?e et confidentielle. Si vous n'?tes pas le destinataire vis? ou la personne charg?e de le remettre ? son destinataire, vous ?tes, par la pr?sente, avis? que toute lecture, usage, copie ou communication du contenu de ce document est strictement interdit. De plus, vous ?tes pri? de communiquer avec l'exp?diteur sans d?lai et de d?truire ce document imm?diatement. CONFIDENTIALITY: This document is intended solely for the individual or entity to whom it is addressed. The information contained in this document is legally privileged and confidential. If you are not the intended recipient or the person responsible for delivering it to the intended recipient, you are hereby advised that you are strictly prohibited from reading, using, copying or disseminating the contents of this document. Please inform the sender immediately and delete this document immediately.
