mark a écrit :
Hi there,
I need to extract data from text files (~4 GB) on this data some
operations are performed like avg, max, min, group etc. The result is
formated and written in some other text files (some KB).
I currently think about database tools might be suitable for this. I
would just write the import from the text files and ... the tool does
the rest. The only problem I can imagine is that this would not be
fast enough.
Is this an a priori, or did you actually benchmark and found out it
would not fit your requirements ?
But I would give it a shoot.
Unfortunately I have only some knowledge of SQLite which is not an
option here.
>
Some additional requirements I can think of are:
- Python (I want to hone my programming skills too)
- Python-only (no C-lib) for simplicity (installation, portability).
Therefore SQLite is not an option
- must be fast
These two requirements can conflict for some values of "fast".
- I like SQL (select a, b from ...) this would be nice (row[..] + ...
is a little hard getting used to)
So far I found PyDBLite, PyTables, Buzhug but they are difficult to
compare for a beginner.
Never used any of them - I have sqlite, mysql and pgsql installed on all
my machines -, so I can't help here.
--
http://mail.python.org/mailman/listinfo/python-list
