I have been using ZipFile.jl and it worked quite well. I am not sure what 
you mean that "zip" is not available. ZipFile relies on the zlib library 
and works "in memory" using ccalls.

One issue that I am not sure about is if ZipFile supports zip64 files. 
There is a TODO here 
https://github.com/fhs/ZipFile.jl/blob/master/src/ZipFile.jl#L54. Don't 
know how much work it would be to implement it if missing.

Am Dienstag, 10. Juni 2014 09:37:10 UTC+2 schrieb Tomas Lycken:
>
> In my thesis, I'm working on a project that produces huge amounts of 
> output in text files - about 25-30GB spread across a million or more files 
> per simulation run. If I compress the files using e.g. `tar --xz --create 
> -f archive.tar.gz tracefiles/` I can reduce the size on disk by a factor 
> 5-6 or even more. I postprocess all this data in Julia, and reading the 
> data files seems to be a major bottleneck.
>
> Has any effort been made toward reading files in these formats in Julia? 
> I've seen [ZipFile](https://github.com/fhs/ZipFile.jl) for handling the 
> .zip format, but unfortunately `zip` isn't available on our cluster, while 
> `tar` is.
>
> If there hasn't been any work on this, I might take a stab at it sometime 
> - but first I must finish my thesis...
>
> // T
>

Reply via email to