On Wed, Mar 2, 2016 at 10:31 AM, Dilip Kumar <dilipbal...@gmail.com> wrote:
> 1. One option can be as you suggested like ProcArrayGroupClearXid, With > some modification, because when we wait for the request and extend w.r.t > that, may be again we face the Context Switch problem, So may be we can > extend in some multiple of the Request. > (But we need to take care whether to give block directly to requester or > add it into FSM or do both i.e. give at-least one block to requester and > put some multiple in FSM) > > > 2. Other option can be that we analyse the current Load on the extend and > then speed up or slow down the extending speed. > Simple algorithm can look like this > I have tried the approach of group extend, 1. We convert the extension lock to TryLock and if we get the lock then extend by one block.2. 2. If don't get the Lock then use the Group leader concep where only one process will extend for all, Slight change from ProcArrayGroupClear is that here other than satisfying the requested backend we Add some extra blocks in FSM, say GroupSize*10. 3. So Actually we can not get exact load but still we have some factor like group size tell us exactly the contention size and we extend in multiple of that. Performance Analysis: --------------------- Performance is scaling with this approach, its slightly less compared to previous patch where we directly give extend_by_block parameter and extend in multiple blocks, and I think that's obvious because in group extend case only after contention happen on lock we add extra blocks, but in former case it was extending extra blocks optimistically. Test1(COPY) ----- ./psql -d postgres -c "COPY (select g.i::text FROM generate_series(1, 10000) g(i)) TO '/tmp/copybinary' WITH BINARY"; echo COPY data from '/tmp/copybinary' WITH BINARY; > copy_script ./pgbench -f copy_script -T 300 -c$ -j$ postgres Shared Buffer:8GB max_wal_size:10GB Storage:Magnetic Disk WAL:SSD ----------------------------------------------------------------------------------------------- Client Base multi_extend by 20 page group_extend_patch(groupsize*10) 1 105 157 149 2 217 255 282 4 210 494 452 8 166 702 629 16 145 563 561 32 124 480 566 Test2(INSERT) -------- ./psql -d postgres -c "create table test_data(a int, b text)" ./psql -d postgres -c "insert into test_data values(generate_series(1,1000),repeat('x', 1024))" ./psql -d postgres -c "create table data (a int, b text) echo "insert into data select * from test_data;" >> insert_script shared_buffers=512GB max_wal_size=20GB checkpoint_timeout=10min ./pgbench -c $ -j $ -f insert_script -T 300 postgres Client Base Multi-extend by 1000 *group extend (group*10) *group extend (group*100) 1 117 118 125 122 2 111 140 161 159 4 51 190 141 187 8 43 254 148 172 16 40 243 150 173 * (group*10)-> means inside the code, Group leader will see how many members are in the group who want blocks, so we will satisfy request of all member + will put extra blocks in FSM extra block to extend are = (group*10) --> 10 is just some constant. Summary: ------------ 1. Here with group extend patch, there is no configuration to tell that how many block to extend, so that should be decided by current load in the system (contention on the extension lock). 2. With small multiplier i.e. 10 we can get fairly good improvement compare to base code, but when load is high like record size is 1K, improving the multiplier size giving better results. * Note: Currently this is POC patch, It has only one group Extend List, so currently can handle only one relation Group extend. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com
diff --git a/src/backend/access/heap/hio.c b/src/backend/access/heap/hio.c index 8140418..adb82ba 100644 --- a/src/backend/access/heap/hio.c +++ b/src/backend/access/heap/hio.c @@ -23,6 +23,7 @@ #include "storage/freespace.h" #include "storage/lmgr.h" #include "storage/smgr.h" +#include "storage/proc.h" /* @@ -168,6 +169,160 @@ GetVisibilityMapPins(Relation relation, Buffer buffer1, Buffer buffer2, } } + +static BlockNumber +GroupExtendRelation(PGPROC *proc, Relation relation, BulkInsertState bistate) +{ + volatile PROC_HDR *procglobal = ProcGlobal; + uint32 nextidx; + uint32 wakeidx; + int extraWaits = -1; + BlockNumber targetBlock; + int count = 0; + + /* Add ourselves to the list of processes needing a group extend. */ + proc->groupExtendMember = true; + + while (true) + { + nextidx = pg_atomic_read_u32(&procglobal->extendGroupFirst); + pg_atomic_write_u32(&proc->extendGroupNext, nextidx); + + if (pg_atomic_compare_exchange_u32(&procglobal->extendGroupFirst, + &nextidx, + (uint32) proc->pgprocno)) + break; + } + + /* + * If the list was not empty, the leader will clear our XID. It is + * impossible to have followers without a leader because the first process + * that has added itself to the list will always have nextidx as + * INVALID_PGPROCNO. + */ + if (nextidx != INVALID_PGPROCNO) + { + /* Sleep until the leader clears our XID. */ + for (;;) + { + /* acts as a read barrier */ + PGSemaphoreLock(&proc->sem); + if (!proc->groupExtendMember) + break; + extraWaits++; + } + + Assert(pg_atomic_read_u32(&proc->extendGroupNext) == INVALID_PGPROCNO); + + /* Fix semaphore count for any absorbed wake ups */ + while (extraWaits-- > 0) + PGSemaphoreUnlock(&proc->sem); + + targetBlock = proc->blockNum; + + proc->blockNum = InvalidBlockNumber; + + return targetBlock; + } + + /* We are the leader. Acquire the lock on behalf of everyone. */ + LockRelationForExtension(relation, ExclusiveLock); + + /* + * Now that we've got the lock, clear the list of processes waiting for + * group extending + */ + while (true) + { + nextidx = pg_atomic_read_u32(&procglobal->extendGroupFirst); + if (pg_atomic_compare_exchange_u32(&procglobal->extendGroupFirst, + &nextidx, + PG_INT32_MAX)) + break; + } + + /* Remember head of list so we can perform wakeups after dropping lock. */ + wakeidx = nextidx; + + + /* Walk the list and clear all XIDs. */ + while (nextidx != INVALID_PGPROCNO) + { + PGPROC *proc = &ProcGlobal->allProcs[nextidx]; + Buffer buffer; + + buffer = ReadBufferBI(relation, P_NEW, bistate); + proc->blockNum = BufferGetBlockNumber(buffer); + + ReleaseBuffer(buffer); + + /* Move to next proc in list. */ + nextidx = pg_atomic_read_u32(&proc->extendGroupNext); + + count ++; + } + + count = count*10; + + do + { + Buffer buffer; + Page page; + Size freespace; + BlockNumber blockNum; + + buffer = ReadBufferBI(relation, P_NEW, bistate); + + /* + * Now acquire lock on the new page. + */ + LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); + + page = BufferGetPage(buffer); + PageInit(page, BufferGetPageSize(buffer), 0); + + freespace = PageGetHeapFreeSpace(page); + MarkBufferDirty(buffer); + blockNum = BufferGetBlockNumber(buffer); + UnlockReleaseBuffer(buffer); + + RecordPageWithFreeSpace(relation, blockNum, freespace); + } while (count--); + + /* We're done with the lock now. */ + UnlockRelationForExtension(relation, ExclusiveLock); + + /* + * Now that we've released the lock, go back and wake everybody up. We + * don't do this under the lock so as to keep lock hold times to a + * minimum. The system calls we need to perform to wake other processes + * up are probably much slower than the simple memory writes we did while + * holding the lock. + */ + while (wakeidx != INVALID_PGPROCNO) + { + PGPROC *proc = &ProcGlobal->allProcs[wakeidx]; + + wakeidx = pg_atomic_read_u32(&proc->extendGroupNext); + pg_atomic_write_u32(&proc->extendGroupNext, INVALID_PGPROCNO); + + /* ensure all previous writes are visible before follower continues. */ + pg_write_barrier(); + + proc->groupExtendMember = false; + + if (proc != MyProc) + PGSemaphoreUnlock(&proc->sem); + else + { + targetBlock = proc->blockNum; + proc->blockNum = InvalidBlockNumber; + } + } + + return targetBlock; +} + /* * RelationGetBufferForTuple * @@ -238,6 +393,7 @@ RelationGetBufferForTuple(Relation relation, Size len, BlockNumber targetBlock, otherBlock; bool needLock; + int extraBlocks; len = MAXALIGN(len); /* be conservative */ @@ -308,6 +464,7 @@ RelationGetBufferForTuple(Relation relation, Size len, } } +loop: while (targetBlock != InvalidBlockNumber) { /* @@ -441,27 +598,95 @@ RelationGetBufferForTuple(Relation relation, Size len, needLock = !RELATION_IS_LOCAL(relation); if (needLock) - LockRelationForExtension(relation, ExclusiveLock); + { + if (TryLockRelationForExtension(relation, ExclusiveLock)) + { + buffer = ReadBufferBI(relation, P_NEW, bistate); + /* + * We can be certain that locking the otherBuffer first is OK, since + * it must have a lower page number. + */ + if ((otherBuffer != InvalidBuffer) && !extraBlocks) + LockBuffer(otherBuffer, BUFFER_LOCK_EXCLUSIVE); + + /* + * Now acquire lock on the new page. + */ + LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); + + UnlockRelationForExtension(relation, ExclusiveLock); + } + else + { + targetBlock = GroupExtendRelation(MyProc, relation, bistate); + goto loop; + } + } + else + { + buffer = ReadBufferBI(relation, P_NEW, bistate); + /* + * We can be certain that locking the otherBuffer first is OK, since + * it must have a lower page number. + */ + if ((otherBuffer != InvalidBuffer) && !extraBlocks) + LockBuffer(otherBuffer, BUFFER_LOCK_EXCLUSIVE); + /* + * Now acquire lock on the new page. + */ + LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); + + } + +#if 0 + if (LOCKACQUIRE_NOT_AVAIL) + + if (use_fsm) + extraBlocks = RelationGetExtendBlocks(relation) - 1; + else + extraBlocks = 0; /* * XXX This does an lseek - rather expensive - but at the moment it is the * only way to accurately determine how many blocks are in a relation. Is * it worth keeping an accurate file length in shared memory someplace, * rather than relying on the kernel to do it for us? */ - buffer = ReadBufferBI(relation, P_NEW, bistate); - /* - * We can be certain that locking the otherBuffer first is OK, since it - * must have a lower page number. - */ - if (otherBuffer != InvalidBuffer) - LockBuffer(otherBuffer, BUFFER_LOCK_EXCLUSIVE); + do + { + buffer = ReadBufferBI(relation, P_NEW, bistate); + + /* + * We can be certain that locking the otherBuffer first is OK, since + * it must have a lower page number. + */ + if ((otherBuffer != InvalidBuffer) && !extraBlocks) + LockBuffer(otherBuffer, BUFFER_LOCK_EXCLUSIVE); + + /* + * Now acquire lock on the new page. + */ + LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); + + if (extraBlocks) + { + Page page; + Size freespace; + BlockNumber blockNum; + + page = BufferGetPage(buffer); + PageInit(page, BufferGetPageSize(buffer), 0); + + freespace = PageGetHeapFreeSpace(page); + MarkBufferDirty(buffer); + blockNum = BufferGetBlockNumber(buffer); + UnlockReleaseBuffer(buffer); + RecordPageWithFreeSpace(relation, blockNum, freespace); + } + + } while (extraBlocks--); - /* - * Now acquire lock on the new page. - */ - LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); /* * Release the file-extension lock; it's now OK for someone else to extend @@ -471,7 +696,7 @@ RelationGetBufferForTuple(Relation relation, Size len, */ if (needLock) UnlockRelationForExtension(relation, ExclusiveLock); - +#endif /* * We need to initialize the empty new page. Double-check that it really * is empty (this should never happen, but if it does we don't want to diff --git a/src/backend/storage/lmgr/lmgr.c b/src/backend/storage/lmgr/lmgr.c index 9d16afb..08e5e07 100644 --- a/src/backend/storage/lmgr/lmgr.c +++ b/src/backend/storage/lmgr/lmgr.c @@ -340,6 +340,21 @@ LockRelationForExtension(Relation relation, LOCKMODE lockmode) (void) LockAcquire(&tag, lockmode, false, false); } +LockAcquireResult +TryLockRelationForExtension(Relation relation, LOCKMODE lockmode) +{ + LOCKTAG tag; + + SET_LOCKTAG_RELATION_EXTEND(tag, + relation->rd_lockInfo.lockRelId.dbId, + relation->rd_lockInfo.lockRelId.relId); + + return LockAcquire(&tag, lockmode, false, true); +} + + + + /* * UnlockRelationForExtension */ diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c index 6453b88..a823eda 100644 --- a/src/backend/storage/lmgr/proc.c +++ b/src/backend/storage/lmgr/proc.c @@ -182,6 +182,7 @@ InitProcGlobal(void) ProcGlobal->walwriterLatch = NULL; ProcGlobal->checkpointerLatch = NULL; pg_atomic_init_u32(&ProcGlobal->procArrayGroupFirst, INVALID_PGPROCNO); + pg_atomic_init_u32(&ProcGlobal->extendGroupFirst, INVALID_PGPROCNO); /* * Create and initialize all the PGPROC structures we'll need. There are diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h index dbcdd3f..5a0e60b 100644 --- a/src/include/storage/proc.h +++ b/src/include/storage/proc.h @@ -152,6 +152,11 @@ struct PGPROC */ TransactionId procArrayGroupMemberXid; + bool groupExtendMember; + pg_atomic_uint32 extendGroupNext; + uint32 blockNum; + + /* Per-backend LWLock. Protects fields below (but not group fields). */ LWLock backendLock; @@ -223,6 +228,8 @@ typedef struct PROC_HDR PGPROC *bgworkerFreeProcs; /* First pgproc waiting for group XID clear */ pg_atomic_uint32 procArrayGroupFirst; + pg_atomic_uint32 extendGroupFirst; + /* WALWriter process's latch */ Latch *walwriterLatch; /* Checkpointer process's latch */
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers