18Feb2009 (UTC +8) On Wed, Feb 18, 2009 at 7:08 PM, Pablo Manalastas <[email protected]> wrote: > uintmax_t can't count to 100 billion. But since you have coreutils > source, you can always recompile a version of wc using type > long long for total_lines. Galing talaga ng open source!
Just an idea: why not re-compile wc.c to include code that calloc a buffer and read from it? Just like what "less" does. That way, you don't needs GB's of RAM to load and process the large text files in its entirety. Compare what "less' does to how "more" performs, and you can see what I'm imagining. > --- On Mon, 2/16/09, Ramil Galib <[email protected]> wrote: >> From: Ramil Galib <[email protected]> >> Subject: Re: [plug] Maximum number of lines wc can count >> To: "Philippine Linux Users' Group (PLUG) Technical Discussion List" >> <[email protected]> >> Date: Monday, February 16, 2009, 4:31 PM >> >> Since Google can't answer me, I download the coreutils >> source package in which wc belongs. >> the variable total_lines is of type uintmax_t. >> Now I'm getting somewhere. >> >> On 2/16/09, Roger Filomeno <[email protected]> wrote: >> I should since the whole file is not actually loaded into memory. Its jst >> going to take a lot of I/O >> >> > On Mon, Feb 16, 2009 at 12:27 PM, Ramil Galib <[email protected]> wrote: >> > >> >> I have text files ranging from 5 to 10 GB. I want to use wc -l to >> >> count the number of lines of these files. I expect the number of lines >> >> to be from 50 to 100 billion. >> >> Can wc handle this? >> >> Hope you can help. >> >> Thanks! Drexx Laggui -- CISA, CISSP, CFE Associate, ISO27001 LA, CCSI, CSA http://www.laggui.com ( Singapore / Manila / California ) Computer forensics; Penetration testing; QMS & ISMS developers; K-Transfer PGP fingerprint = 6E62 A089 E3EA 1B93 BFB4 8363 FFEC 3976 FF31 8A4E _________________________________________________ Philippine Linux Users' Group (PLUG) Mailing List http://lists.linux.org.ph/mailman/listinfo/plug Searchable Archives: http://archives.free.net.ph
