Re: Tuning PostgreSQL for bulk imports

2007-01-08 Thread Kirk Strauser
On Monday 08 January 2007 1:51 am, Abdullah Al-Marrie wrote: Why did you choose PostgreSQL over MySQL 5.0.x? We value our data, ruling out MyISAM. PostgreSQL is much faster than InnoDB for many concurrent reads and complex queries. Is the latest PostgreSQL release performance much better

Re: Tuning PostgreSQL for bulk imports

2007-01-08 Thread John Levine
Why did you choose PostgreSQL over MySQL 5.0.x? We value our data, ruling out MyISAM. Huh? I thought you said that the SQL database is just a mirror of the stuff from Foxpro. R's, John k ___ freebsd-questions@freebsd.org mailing list

Re: Tuning PostgreSQL for bulk imports

2007-01-08 Thread Kirk Strauser
On Monday 08 January 2007 9:05 am, John Levine wrote: Why did you choose PostgreSQL over MySQL 5.0.x? We value our data, ruling out MyISAM. Huh? I thought you said that the SQL database is just a mirror of the stuff from Foxpro. Not *all* of it. We're migrating over to it as the native

Re: Tuning PostgreSQL for bulk imports

2007-01-07 Thread Ivan Voras
Kirk Strauser wrote: I have an hourly job that converts our legacy Foxpro database into PostgreSQL tables so that our web applications, etc. can run reports off the data in a reasonable amount of time. Believe it or not, this has been running perfectly in production for over a year. The only

Re: Tuning PostgreSQL for bulk imports

2007-01-07 Thread Kirk Strauser
On Sunday 07 January 2007 15:15, Ivan Voras wrote: There's a dedicated mailing list for PostgreSQL performance: pgsql-performance/at/postgresql.org, which can give you really good advice, but here's some tips: I've read, read, and re-read the general tuning tips, and done as much as seemed

Re: Tuning PostgreSQL for bulk imports

2007-01-07 Thread Ivan Voras
Kirk Strauser wrote: On Sunday 07 January 2007 15:15, Ivan Voras wrote: - What might help you is to keep the WAL (write-ahead-log, i.e. journal) files on a completely separate (and fast) drive from the rest of the database, to allow parallelism and speed. For best results, format it with 32k

Re: Tuning PostgreSQL for bulk imports

2007-01-07 Thread Norberto Meijome
On Sun, 7 Jan 2007 16:18:06 -0600 Kirk Strauser [EMAIL PROTECTED] wrote: There's a dedicated mailing list for PostgreSQL performance: pgsql-performance/at/postgresql.org, which can give you really good advice, but here's some tips: I've read, read, and re-read the general tuning tips,

Re: Tuning PostgreSQL for bulk imports

2007-01-07 Thread Kirk Strauser
On Sunday 07 January 2007 17:15, Ivan Voras wrote: What are your current IO rates? Since you have only two drives you might be restricted by available disk bandwidth... So says gstat. According to top, the import only takes about 40% of one CPU, but both of the drives are saturated. I'm

Re: Tuning PostgreSQL for bulk imports

2007-01-07 Thread Kirk Strauser
On Sunday 07 January 2007 19:22, Norberto Meijome wrote: I could be wrong and this only apply to generating indexes? That's what we're doing now. By dropping the table and recreating it, all the index maintenance gets deferred until one batch at the end (which is vastly faster in practice).

Re: Tuning PostgreSQL for bulk imports

2007-01-07 Thread Abdullah Al-Marrie
Kirk Strauser wrote: I have an hourly job that converts our legacy Foxpro database into PostgreSQL tables so that our web applications, etc. can run reports off the data in a reasonable amount of time. Believe it or not, this has been running perfectly in production for over a year. The only