Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-11-06 Thread Tom Lane
"Robert Haas" <[EMAIL PROTECTED]> writes: > OK, here's an updated version... Applied with some small stylistic revisions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.o

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-11-05 Thread Robert Haas
OK, here's an updated version... 1. Use IncrBufferRefCount() so that we can do unconditional ReleaseBuffers elsewhere. I'm not sure this is really any simpler, and although IncrBufferRefCount() is pretty cheap, it's certainly not as cheap as a NULL pointer test. 2. Consolidate a bunch of logic i

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-11-04 Thread Tom Lane
"Robert Haas" <[EMAIL PROTECTED]> writes: >> 2. The logic changes in RelationGetBufferForTuple seem bizarre and >> overcomplicated. ISTM that the buffer saved by the bistate ought to >> be about equivalent to relation->rd_targblock, ie, it's your first >> trial location and also a place to save th

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-11-04 Thread Robert Haas
> 2. The logic changes in RelationGetBufferForTuple seem bizarre and > overcomplicated. ISTM that the buffer saved by the bistate ought to > be about equivalent to relation->rd_targblock, ie, it's your first > trial location and also a place to save the located buffer on the way > out. I'd sugges

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-11-04 Thread Tom Lane
"Robert Haas" <[EMAIL PROTECTED]> writes: > Patch resnapped to HEAD, with straightforward adjustments to > compensate for Heikki's changes to the ReadBuffer interface. See > attached. I looked this over a bit. A couple of suggestions: 1. You could probably simplify life a bit by treating the Bu

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-11-01 Thread Simon Riggs
On Sat, 2008-11-01 at 13:23 -0400, Robert Haas wrote: > > Can you test whether using the buffer access strategy is a win or a > > loss? Most of that gain is probably coming from the reduction in > > pinning. > > --PATCHED-- > Time: 13869.724 ms (median) > --PATCHED with BAS disabled-- > Time: 14

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-11-01 Thread Robert Haas
> Can you test whether using the buffer access strategy is a win or a > loss? Most of that gain is probably coming from the reduction in > pinning. Patch resnapped to HEAD, with straightforward adjustments to compensate for Heikki's changes to the ReadBuffer interface. See attached. New testing

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-10-31 Thread Simon Riggs
On Thu, 2008-10-30 at 22:46 -0400, Robert Haas wrote: > > You should try profiling the patch. You can count the invocations of the > > buffer access routines to check its all working in the right ratios. > > *goes and learns how to do profile PostgreSQL* > > OK, that was a good suggestion. It l

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-10-30 Thread Simon Riggs
On Thu, 2008-10-30 at 23:05 -0400, Robert Haas wrote: > > Whatever timings you have are worth publishing. > > Here are the timings for copying the first ten million integers into a > one-column table created in the same transaction, with and without the > patch. As you can see, now that I've cor

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-10-30 Thread Robert Haas
> Whatever timings you have are worth publishing. Here are the timings for copying the first ten million integers into a one-column table created in the same transaction, with and without the patch. As you can see, now that I've corrected my previous error of not putting CREATE TABLE and COPY in

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-10-30 Thread Robert Haas
> You should try profiling the patch. You can count the invocations of the > buffer access routines to check its all working in the right ratios. *goes and learns how to do profile PostgreSQL* OK, that was a good suggestion. It looks like part of my problem here is that I didn't put the CREATE T

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-10-29 Thread Simon Riggs
On Wed, 2008-10-29 at 21:58 -0400, Robert Haas wrote: > > If you say its a loss you should publish timings to support that. Using > > a BAS for VACUUM was a performance gain, not a loss. > > Well, I can dig up and publish the timings from my laptop, but I'm not > sure where that will get us. Tru

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-10-29 Thread Robert Haas
> If you say its a loss you should publish timings to support that. Using > a BAS for VACUUM was a performance gain, not a loss. Well, I can dig up and publish the timings from my laptop, but I'm not sure where that will get us. Trust me, the numbers were higher with BAS, otherwise I wouldn't be

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-10-29 Thread Simon Riggs
On Tue, 2008-10-28 at 23:45 -0400, Robert Haas wrote: > One concern that I have about this approach is that the situation in > which people are probably most concerned about COPY performance is > restoring a dump. In that case, the COPY will be the only thing > running, and using a BufferAccessS

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-10-28 Thread Robert Haas
And here's the patch, which based on comments thus far does the following: - Replaces the use_wal, use_fsm arguments in various places with a single options argument. - Creates a BAS_BULKWRITE buffer access strategy. - Creates a BulkInsertState object so that COPY and CTAS can use BAS_BULKWRITE an

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-10-26 Thread Robert Haas
> Seems sane to me. I don't see the point of the HEAP_INSERT_BULK flag > bit --- providing or not providing bistate would cover that, and if > you have a bit as well then you have to define what the inconsistent > combinations mean. I concur with making all-zeroes be the typical > state of the fl

Re: [HACKERS] BufferAccessStrategy for bulk insert

2008-10-26 Thread Tom Lane
"Robert Haas" <[EMAIL PROTECTED]> writes: > I am kind of inclined to define flags like this: > #define HEAP_INSERT_SKIP_WAL 0x0001 > #define HEAP_INSERT_SKIP_FSM 0x0002 > #define HEAP_INSERT_BULK 0x0004 /* do we even need this one? */ > And then: > Oid heap_insert(Relation relation, HeapTuple tu

[HACKERS] BufferAccessStrategy for bulk insert

2008-10-25 Thread Robert Haas
I'm taking a look at doing the refactoring Tom Lane and Simon Riggs discussed here: http://archives.postgresql.org/pgsql-patches/2008-02/msg00155.php In terms of the buffer manager, I think we can simply introduce a new strategy type BAS_BULKWRITE and make it behave identically to BAS_VACUUM. An