On Sat, 2005-01-01 at 17:47, Simon Riggs wrote:
> On Sat, 2005-01-01 at 17:01, Bruce Momjian wrote:
> > Simon Riggs wrote:
> >  
> > > Well, I think we're saying: its not in 8.0 now, and we take our time to
> > > consider patches for 8.1 and accept the situation that the parameter
> > > names/meaning will change in next release.
> > 
> > I have no problem doing something for 8.0 if we can find something that
> > meets all the items I mentioned.
> > 
> > One idea would be to just remove bgwriter_percent.  Beta/RC users would
> > still have it in their postgresql.conf, but it is commented out so it
> > should be OK.  If they uncomment it their server would not start but we
> > could just tell testers to remove it.  I see that as better than having
> > conflicting parameters.
> 
> Can't say I like that at first thought. I'll think some more though...
> 
> > Another idea is to have bgwriter_percent be the percent of the buffer it
> > will scan.  
> 
> Hmmm....well that was my original suggestion (bg2.patch on 12 Dec)
> (...though with a bug, as Neil pointed out)
> 
> > We could default that to 50% or 100%, but we then need to
> > make sure all beta/RC users update their postgresql.conf with the new
> > default because the commented-out default will not be correct.
> 
> ...we just differ/ed on what the default should be...
> 
> > At this point I see these as our only two viable options, aside from
> > doing nothing.
> 
> > I realize our current behavior requires a full scan of the buffer cache,
> > but how often is the bgwriter_maxpages limit met?  If it is not a full
> > scan is done anyway, right?  
> 
> Well, if you heavy a very heavy read workload then that would be a
> problem. I was more worried about concurrency in a heavy write
> situation, but I can see your point, and agree.
> 
> (Idea #1 still suffers from this, so we should rule it out...)
> 
> > It seems the only way to really add
> > functionality is to change bgwriter_precent to control how much of the
> > buffer is scanned.
> 
> OK. I think you've persuaded me on idea #2, if I understand you right:
> 
> bgwriter_percent = 50 (default)
> bgwriter_maxpages = 100 (default)
> 
> percent is the number of shared_buffers we scan, limited by maxpages.
> 
> (I'll code it up in a couple of hours when the kids are in bed)

Here's the basic patch - no changes to current default values or docs.

Not sure if this is still interesting or not...

-- 
Best Regards, Simon Riggs
Index: src/backend/storage/buffer/bufmgr.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/storage/buffer/bufmgr.c,v
retrieving revision 1.182
diff -d -c -r1.182 bufmgr.c
*** src/backend/storage/buffer/bufmgr.c	24 Nov 2004 02:56:17 -0000	1.182
--- src/backend/storage/buffer/bufmgr.c	1 Jan 2005 21:03:16 -0000
***************
*** 682,717 ****
  	BufferDesc **dirty_buffers;
  	BufferTag  *buftags;
  	int			num_buffer_dirty;
  	int			i;
  
  	/* If either limit is zero then we are disabled from doing anything... */
  	if (percent == 0 || maxpages == 0)
  		return 0;
  
  	/*
! 	 * Get a list of all currently dirty buffers and how many there are.
  	 * We do not flush buffers that get dirtied after we started. They
! 	 * have to wait until the next checkpoint.
  	 */
! 	dirty_buffers = (BufferDesc **) palloc(NBuffers * sizeof(BufferDesc *));
! 	buftags = (BufferTag *) palloc(NBuffers * sizeof(BufferTag));
  
  	LWLockAcquire(BufMgrLock, LW_EXCLUSIVE);
- 	num_buffer_dirty = StrategyDirtyBufferList(dirty_buffers, buftags,
- 											   NBuffers);
  
! 	/*
! 	 * If called by the background writer, we are usually asked to only
! 	 * write out some portion of dirty buffers now, to prevent the IO
! 	 * storm at checkpoint time.
! 	 */
! 	if (percent > 0)
! 	{
! 		Assert(percent <= 100);
! 		num_buffer_dirty = (num_buffer_dirty * percent + 99) / 100;
! 	}
! 	if (maxpages > 0 && num_buffer_dirty > maxpages)
! 		num_buffer_dirty = maxpages;
  
  	/* Make sure we can handle the pin inside the loop */
  	ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
--- 682,728 ----
  	BufferDesc **dirty_buffers;
  	BufferTag  *buftags;
  	int			num_buffer_dirty;
+     int         max_buffer_dirty = 1;
+     int         max_buffer_scan = 1;
  	int			i;
  
  	/* If either limit is zero then we are disabled from doing anything... */
  	if (percent == 0 || maxpages == 0)
  		return 0;
  
+     /* Set number of buffers we will scan from LRUs of buffer lists */
+     if (percent > 0 ) {
+     	Assert(percent <= 100);
+    	    max_buffer_scan = (NBuffers * percent + 99) / 100;
+     }
+ 
+     /* at checkpoint time we scan the whole buffer list */
+     if (percent < 0)
+     	max_buffer_scan = NBuffers;
+ 
+     if (maxpages < 0 || maxpages > NBuffers)
+     	max_buffer_dirty = NBuffers;
+     else
+         max_buffer_dirty = maxpages;
+ 
+     /* we cannot find more dirty buffers than we scan */
+     if (max_buffer_dirty > max_buffer_scan)
+         max_buffer_dirty = max_buffer_scan;
+ 
  	/*
! 	 * Get a list of dirty buffers to clean and how many there are.
  	 * We do not flush buffers that get dirtied after we started. They
! 	 * have to wait until the next call of this function
  	 */
! 	dirty_buffers = 
!          (BufferDesc **) palloc(max_buffer_dirty * sizeof(BufferDesc *));
! 	buftags = (BufferTag *) palloc(max_buffer_dirty * sizeof(BufferTag));
  
  	LWLockAcquire(BufMgrLock, LW_EXCLUSIVE);
  
!    	num_buffer_dirty = StrategyDirtyBufferList(dirty_buffers, buftags,
! 											   max_buffer_dirty,
!                                                max_buffer_scan);
  
  	/* Make sure we can handle the pin inside the loop */
  	ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
Index: src/backend/storage/buffer/freelist.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/storage/buffer/freelist.c,v
retrieving revision 1.48
diff -d -c -r1.48 freelist.c
*** src/backend/storage/buffer/freelist.c	16 Sep 2004 16:58:31 -0000	1.48
--- src/backend/storage/buffer/freelist.c	1 Jan 2005 21:03:17 -0000
***************
*** 735,756 ****
   * StrategyDirtyBufferList
   *
   * Returns a list of dirty buffers, in priority order for writing.
-  * Note that the caller may choose not to write them all.
   *
   * The caller must beware of the possibility that a buffer is no longer dirty,
   * or even contains a different page, by the time he reaches it.  If it no
   * longer contains the same page it need not be written, even if it is (again)
   * dirty.
   *
!  * Buffer pointers are stored into buffers[], and corresponding tags into
!  * buftags[], both of size max_buffers.  The function returns the number of
!  * buffer IDs stored.
   */
  int
  StrategyDirtyBufferList(BufferDesc **buffers, BufferTag *buftags,
! 						int max_buffers)
  {
  	int			num_buffer_dirty = 0;
  	int			cdb_id_t1;
  	int			cdb_id_t2;
  	int			buf_id;
--- 735,757 ----
   * StrategyDirtyBufferList
   *
   * Returns a list of dirty buffers, in priority order for writing.
   *
   * The caller must beware of the possibility that a buffer is no longer dirty,
   * or even contains a different page, by the time he reaches it.  If it no
   * longer contains the same page it need not be written, even if it is (again)
   * dirty.
   *
!  * We scan the buffer lists T1 and T2 for at most max_buffer_scan buffers, 
!  * recording any dirty buffer pointers in buffers[], and corresponding tags into
!  * buftags[], both of size max_buffer_dirty. The function returns the number of
!  * dirty buffer IDs stored.
   */
  int
  StrategyDirtyBufferList(BufferDesc **buffers, BufferTag *buftags,
! 						int max_buffer_dirty, int max_buffer_scan)
  {
  	int			num_buffer_dirty = 0;
+ 	int			num_buffer_scan = 0;
  	int			cdb_id_t1;
  	int			cdb_id_t2;
  	int			buf_id;
***************
*** 779,790 ****
  					buffers[num_buffer_dirty] = buf;
  					buftags[num_buffer_dirty] = buf->tag;
  					num_buffer_dirty++;
! 					if (num_buffer_dirty >= max_buffers)
  						break;
  				}
  			}
  
  			cdb_id_t1 = StrategyCDB[cdb_id_t1].next;
  		}
  
  		if (cdb_id_t2 >= 0)
--- 780,794 ----
  					buffers[num_buffer_dirty] = buf;
  					buftags[num_buffer_dirty] = buf->tag;
  					num_buffer_dirty++;
! 					if (num_buffer_dirty >= max_buffer_dirty)
  						break;
  				}
  			}
  
  			cdb_id_t1 = StrategyCDB[cdb_id_t1].next;
+ 			num_buffer_scan++;
+ 			if (num_buffer_scan >= max_buffer_scan)
+ 				break;
  		}
  
  		if (cdb_id_t2 >= 0)
***************
*** 799,810 ****
  					buffers[num_buffer_dirty] = buf;
  					buftags[num_buffer_dirty] = buf->tag;
  					num_buffer_dirty++;
! 					if (num_buffer_dirty >= max_buffers)
  						break;
  				}
  			}
  
  			cdb_id_t2 = StrategyCDB[cdb_id_t2].next;
  		}
  	}
  
--- 803,817 ----
  					buffers[num_buffer_dirty] = buf;
  					buftags[num_buffer_dirty] = buf->tag;
  					num_buffer_dirty++;
! 					if (num_buffer_dirty >= max_buffer_dirty)
  						break;
  				}
  			}
  
  			cdb_id_t2 = StrategyCDB[cdb_id_t2].next;
+ 			num_buffer_scan++;
+ 			if (num_buffer_scan >= max_buffer_scan)
+ 				break;
  		}
  	}
  
Index: src/include/storage/buf_internals.h
===================================================================
RCS file: /projects/cvsroot/pgsql/src/include/storage/buf_internals.h,v
retrieving revision 1.74
diff -d -c -r1.74 buf_internals.h
*** src/include/storage/buf_internals.h	16 Oct 2004 18:05:07 -0000	1.74
--- src/include/storage/buf_internals.h	1 Jan 2005 21:03:18 -0000
***************
*** 184,190 ****
  extern void StrategyInvalidateBuffer(BufferDesc *buf);
  extern void StrategyHintVacuum(bool vacuum_active);
  extern int StrategyDirtyBufferList(BufferDesc **buffers, BufferTag *buftags,
! 						int max_buffers);
  extern void StrategyInitialize(bool init);
  
  /* buf_table.c */
--- 184,190 ----
  extern void StrategyInvalidateBuffer(BufferDesc *buf);
  extern void StrategyHintVacuum(bool vacuum_active);
  extern int StrategyDirtyBufferList(BufferDesc **buffers, BufferTag *buftags,
! 						int max_buffer_dirty, int max_buffer_scan);
  extern void StrategyInitialize(bool init);
  
  /* buf_table.c */
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Reply via email to