Hi, >I do my batch processing daily using a python script I've written. I >found that trying to do it with pl/pgsql took more than 24 hours to >process 24 hours worth of logs. I then used C# and in memory hash >tables to drop the time to 2 hours, but I couldn't get mono installed >on some of my older servers. Python proved the fastest and I can >process 24 hours worth of logs in about 15 minutes. Common reports run >in < 1 sec and custom reports run in < 15 seconds (usually). > >
Advertising
When you say you do your batch processing in a Python script do you mean a you are using 'plpython' inside PostgreSQL or using Python to execut select statements and crunch the data 'outside' PostgreSQL? Your reply is very interesting. Thanks. Regards, Rudi. ---------------------------(end of broadcast)--------------------------- TIP 7: don't forget to increase your free space map settings
