hi, you can try this: first get only that field (sed/awk/perl) whihc you want
to sort on in a file. sort that file which i assume would be lot less in size
then your current file/table. then run a loop on the main file using sorted
file as variable.
here is the logic in shell:
awk '{print $<filed-to-be-sorted-on>}' <large-file> > tmp-file
sort <tmp-file>
for id in `cat <sorted-temp-file>`;do grep $id <large-file> >>
sorted-large-file;done
From: Ramprasad Prasad <ramprasad...@gmail.com>
To: Shawn H Corey <shawnhco...@gmail.com>
Cc: Perl Beginners <beginners@perl.org>
Sent: Sunday, August 7, 2011 11:01 AM
Subject: Re: Sorting an extremely LARGE file
On 7 August 2011 21:24, Shawn H Corey <shawnhco...@gmail.com> wrote:
> On 11-08-07 11:46 AM, Ramprasad Prasad wrote:
>
>> I used a mysql database , but the order by clause used to hang the
>> process indefinitely
>> If I sort files in smaller chunks how can I merge them back ??
>>
>>
> Please use "Reply All" when responding to a message on this list.
>
> You need two temporary files and lots of disk space.
>
> 1. Open the first and second sorted files.
> 2. Read one record from each.
> 3. Write the lesser record to the first temporary file.
> 4. Read another record from the file where you got the record you wrote.
> 5. If not eof, goto 3.
> 6. Write the remaining of the other file to the end of the temporary file.
>
> Repeat the above with the first temporary file and the third sorted file,
> writing the result to the second temporary file.
>
> Repeat the above with the second temporary file and the fourth sorted file,
> writing the result to the first temporary file.
>
> And so on...
>
> Rename the final temporary file to your sorted file name.
>
>
>
There would be a CPAN module already doing this ??
