On 08/10/13 21:46, Leena Gupta wrote:
I need to process a large csv file every 5-10 minutes. The file could contain 3mill to 10 mill rows and size could be 6MB to 10MB(+).
Assuming you have a modern multi-core PC with >2G RAM then those sizes should not be a problem. If you have an older PC then you may need to be a bit more clever about how you load the files.
... I need to sum up a number value by grouping on certain attributes and store the output in a datastore.
Again that sounds fairly normal. There are modules that can help with that too.
I wanted to know if Python is recommended and can it be used
> for processing data in csv files of this size? Yes, for sure.
Python has a csv library as well.
Yes and it is quite well documented but if you have specific questions we can try to help. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.flickr.com/photos/alangauldphotos _______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
