Hi,

I have a very large gziped text file (all ASCII characters and ~500GB) that I want to stream and process line-by-line, and I thought the iopipe library would be perfect for this, but I can't seem to get it to work. So far, this is the closest I have to getting it to work:

import iopipe.textpipe;
import iopipe.zip;
import iopipe.bufpipe;
import iopipe.stream;

void main()
{

auto fileToRead = openDev("file.gz").bufd.unzip(CompressionFormat.gzip);

  foreach (line; fileToRead.assumeText.byLineRange!false)
  {
     \\ do stuff
  }
}

but this only processes the first ~200 odd lines (I guess the initial read into the buffer). Can anyone help me out?

Thanks very much

Andrew

Reply via email to