oyekomova wrote: > Thanks to everyone for their excellent suggestions. I was able to > acheive the following results with all your suggestions. However, I am > unable to cross file size of 6 million rows. I would appreciate any > helpful suggestions on avoiding memory errors. None of the solutions > posted was able to cross this limit.
Did you try using numpy.fromfile ? This will not require you to allocate more memory than needed. If you specify a count, it will also not have to re-allocate memory in blocks as the array size grows. It's limitation is that it is not a very sophisticated csv reader (it only understands a single separator (plus line-feeds are typically seen as a separator). -Travis -- http://mail.python.org/mailman/listinfo/python-list
