Using the system linux sort ... Does not help.
On my dual quad core machine , (8 gb ram) sort -n file takes 10
minutes and in the end produces no output.
when I put this data in mysql , there is an index on the order by
field ... But I guess keys don't help when you are selecting the
entire table.
I guess there is a serious need for re-architecting , rather than
create such monstrous files, but when people work with legacy systems
which worked fine when there was lower usage and now you tell then you
need a overhaul because the current system doesn't scale ... That
takes a lot of convincing
On 8/8/11, Uri Guttman <u...@stemsystems.com> wrote:
>>>>>> "RP" == Rajeev Prasad <rp.ne...@yahoo.com> writes:
>
> RP> hi, you can try this: first get only that field (sed/awk/perl)
> RP> whihc you want to sort on in a file. sort that file which i assume
> RP> would be lot less in size then your current file/table. then run a
> RP> loop on the main file using sorted file as variable.
>
> RP>
> RP> here is the logic in shell:
> RP>
> RP> awk '{print $<filed-to-be-sorted-on>}' <large-file> > tmp-file
> RP>
> RP> sort <tmp-file>
> RP>
>
> RP> for id in `cat <sorted-temp-file>`;do grep $id <large-file> >>
> sorted-large-file;done
>
> have you thought about the time this will take? you are doing an O( N**2
> ) grep there. you are looping over all N keys and then scanning the file
> N lines for each key. that will take a very long time for such a large
> file. as others have said, either use the sort utility or do a
> merge/sort on the records. your way is effectively a slow bubble sort!
>
> uri
>
> --
> Uri Guttman -- uri AT perlhunter DOT com --- http://www.perlhunter.com
> --
> ------------ Perl Developer Recruiting and Placement Services
> -------------
> ----- Perl Code Review, Architecture, Development, Training, Support
> -------
>
--
Sent from my mobile device
Thanks
Ram
<http://www.netcore.co.in/>
n <http://pragatee.com>
--
To unsubscribe, e-mail: beginners-unsubscr...@perl.org
For additional commands, e-mail: beginners-h...@perl.org
http://learn.perl.org/
