Hi wrotycz, wrotycz wrote:
How does plzip find members, especially for parallel decompression?
When decompressing from a regular file, plzip builds an index containing the position and size of each member in the file. The index is built by traversing the members backwards from the end of the file. The worker threads then use the index to find the members.
When decompressing from a non-seekable source (like a pipe), plzip starts a splitter thread that looks for the 'ID string' ("LZIP") of the next header and validates it by using the 'Member size' field in the trailer preceding either the header found or the end of file if no header is found. The splitter distributes the members it finds to the worker threads.
Best regards, Antonio.
