Re: How to open a compressed file in gz format ?

[Steven Schveighoffer via Digitalmars-d-learn]

On 3/14/21 9:36 PM, sharkloc wrote:
I want to read the content(file.gz) line by line,the following code is 
not friendly to large files of hundreds of Gb, and the memory overhead 
is also very large.


import std.stdio;
import std.process;
import std.string;

void main(string[] args){

     string fileName = args[1];
     string command = "gzip -dc " ~ fileName ;
     auto dmd = executeShell(command);

     if(dmd.status != 0){
         writeln("Compilation failed:\n", dmd.output);
     }
     else{
         auto all=chomp(dmd.output).split("\n");
         writeln(typeid(all));
         for(int i=0; i<all.length; i++){
             writeln(all[i]);
         }
     }

}

It's not super-user-friendly, but iopipe excels at this kind of stuff 
(untested):

// dub dependencies: [iopipe, io]
import iopipe.bufpipe;
import iopipe.textpipe;
import iopipe.zip;
import iopipe.refc;
import std.io;

import std.stdio;

void main(string[] args) {
   string fileName = args[1];
   auto lineRange = File(fileName) // open file
       .refCounted // make it copyable
       .bufd // buffer it
       .unzip // unzip it
       .assumeText // assume the binary data is utf8 text
       .byLineRange!true; // true = discard newlines

   foreach(line; lineRange)
       writeln(line);
}

-Steve