Re: [Bizgres-general] Re: [PERFORM] faster INSERT with possible

2005-07-27 Thread Josh Berkus
Luke, Well - now that I test it, it appears you are correct, temp table COPY bypasses WAL - thanks for pointing it out! RIght. The problem is bypassing WAL for loading new scratch tables which aren't TEMPORARY tables. We need to do this for multi-threaded ETL, since: a) Temp tables can't

Re: [Bizgres-general] Re: [PERFORM] faster INSERT with possible

2005-07-27 Thread Kris Jurka
On Wed, 27 Jul 2005, Josh Berkus wrote: b) you can't index a temp table. jurka# create temp table t (a int); CREATE jurka# create index myi on t(a); CREATE ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an

Re: [Bizgres-general] Re: [PERFORM] faster INSERT with possible

2005-07-27 Thread Simon Riggs
On Wed, 2005-07-27 at 09:29 -0700, Josh Berkus wrote: Luke, Well - now that I test it, it appears you are correct, temp table COPY bypasses WAL - thanks for pointing it out! RIght. The problem is bypassing WAL for loading new scratch tables which aren't TEMPORARY tables. We need to

Re: [Bizgres-general] Re: [PERFORM] faster INSERT with possible

2005-07-27 Thread PFC
I had in mind the extra tables that an application sometimes needs to operate faster. Denormalisations, pre-joined tables, pre-calculated results, aggregated data. These are not temporary tables, just part of the application - multi-user tables that stay across shutdown/restart. You could

Re: [Bizgres-general] Re: [PERFORM] faster INSERT with possible

2005-07-27 Thread Tom Lane
Josh Berkus josh@agliodbs.com writes: RIght. The problem is bypassing WAL for loading new scratch tables which aren't TEMPORARY tables. We need to do this for multi-threaded ETL, since: a) Temp tables can't be shared by several writers, and b) you can't index a temp table. This may not

Re: [Bizgres-general] Re: [PERFORM] faster INSERT with possible

2005-07-26 Thread Luke Lonergan
Hannu, On 7/26/05 11:56 AM, Hannu Krosing [EMAIL PROTECTED] wrote: On T, 2005-07-26 at 11:46 -0700, Luke Lonergan wrote: Yah - that's a typical approach, and it would be excellent if the COPY bypassed WAL for the temp table load. Don't *all* operations on TEMP tables bypass WAL ? Good