I routinely use sort on very large files and generally use a buffer size of 1500M to accomplish this. I have some large memory 64-bit machines but am not able to utilize this memory to improve sorting efficiency. In other words, I would like to be able to set -S to allow sort to consume much more memory to save the necessity of writing large temp files to disk (unless necessary). I presume there is a way to compile sort to allow it to utilize more than 4G of memory on an x86_64 machine but have not been able to determine how. Here is what I am currently encountering:
ls -l temp -rw-rw-r-- 1 rmorin users 807714374 May 29 16:15 temp sort -S 4000M temp > temp.sort sort(45777) malloc: *** mmap(size=4194308096) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug I work on x86_64 Macs and also x86_64 machines running RHEL. I would appreciate any advice on the matter. Kindest regards, Ryan Morin _______________________________________________ Bug-coreutils mailing list [email protected] http://lists.gnu.org/mailman/listinfo/bug-coreutils
