Re: [HACKERS] pg_rewarm status

2013-12-19 Thread Jeff Janes
On Wed, Dec 18, 2013 at 6:08 PM, Robert Haas wrote: > > Yeah, absolutely. The first backend that needs a prefetch probably > isn't going to get it in time. I think that's OK though. Once the > background process is started, response times will be quicker... > although possibly still not quick

Re: [HACKERS] pg_rewarm status

2013-12-19 Thread Cédric Villemain
Le jeudi 19 décembre 2013 03:08:59, Robert Haas a écrit : > On Wed, Dec 18, 2013 at 6:07 PM, Cédric Villemain wrote: > >> When the prefetch process starts up, it services requests from the > >> queue by reading the requested blocks (or block ranges). When the > >> queue is empty, it sleeps. If

Re: [HACKERS] pg_rewarm status

2013-12-19 Thread Andres Freund
On 2013-12-19 09:16:59 -0500, Robert Haas wrote: > There's no uint32 type at the SQL level, and int32 is no good because > it can't represent sufficiently large positive values to cover the > largest possible block number. Well, pg_class.relpages is an int32, so I think that limit is already kind

Re: [HACKERS] pg_rewarm status

2013-12-19 Thread Robert Haas
On Thu, Dec 19, 2013 at 8:37 AM, Amit Kapila wrote: > Few observations: > 1. > pg_prewarm.control > +# pg_buffercache extension > Wrong name. Oops. > 2. > +pg_prewarm(regclass, mode text default 'buffer', fork text default 'main', > + first_block int8 default null, > + last_b

Re: [HACKERS] pg_rewarm status

2013-12-19 Thread Amit Kapila
On Wed, Dec 18, 2013 at 8:33 PM, Robert Haas wrote: > On Tue, Dec 17, 2013 at 12:35 PM, Jeff Janes wrote: > All right, here is an updated patch. I swapped the second and third > arguments, because I think overriding the prewarm mode will be a lot > more common than overriding the relation fork.

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Wed, Dec 18, 2013 at 6:07 PM, Cédric Villemain wrote: > In the case of effective_io_concurrency, however, this may not work as well as > expected, IIRC it is used to prefetch heap blocks, hopefully the requested > blocks are contiguous but if there are too much holes it is enough to fill the >

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Alvaro Herrera
Robert Haas escribió: > I'm not inclined to wait for the next CommitFest to commit this, > because it's a very simple patch and has already had a lot more field > testing than most patches get before they're committed. And it's just > a contrib module, so the damage it can do if there is in fact

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Cédric Villemain
Le mercredi 18 décembre 2013 18:40:09, Robert Haas a écrit : > Now that we have dynamic background workers, I've been thinking that > it might be possible to write a background worker to do asynchronous > prefetch on systems where we don't have OS support. We could store a > small ring buffer in s

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 7:05 PM, Cédric Villemain wrote: > Le mardi 17 décembre 2013 21:14:44, Josh Berkus a écrit : >> On 12/17/2013 06:34 AM, Robert Haas wrote: >> > On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila > wrote: >> >> I have used pg_prewarm during some of work related to Buffer Managem

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Cédric Villemain
Le mardi 17 décembre 2013 21:14:44, Josh Berkus a écrit : > On 12/17/2013 06:34 AM, Robert Haas wrote: > > On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila wrote: > >> I have used pg_prewarm during some of work related to Buffer Management > >> and other performance related work. It is quite useful

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Cédric Villemain
Le mardi 17 décembre 2013 17:45:51, Robert Haas a écrit : > On Tue, Dec 17, 2013 at 11:02 AM, Jim Nasby wrote: > > On 12/17/13, 8:34 AM, Robert Haas wrote: > >> On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila > >> > >> wrote: > >>> I have used pg_prewarm during some of work related to Buffer Manag

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 9:03 PM, KONDO Mitsumasa wrote: > (2013/12/18 5:33), Robert Haas wrote: >> Sounds like it might be worth dusting the patch off again... > > I'd like to request you to add all_index option and usage_count option. > When all_index option is selected, all index become rewarm n

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 12:35 PM, Jeff Janes wrote: > Since it doesn't use directIO, you can't warm the PG buffers without also > warming FS cache as a side effect. That is why I like 'buffer' as the > default--if the data fits in shared_buffers, it warm those, otherwise it at > least warms the F

Re: [HACKERS] pg_rewarm status

2013-12-17 Thread KONDO Mitsumasa
(2013/12/18 5:33), Robert Haas wrote: Sounds like it might be worth dusting the patch off again... I'd like to request you to add all_index option and usage_count option. When all_index option is selected, all index become rewarm nevertheless user doesn't input relation name. And usage_count op

Re: [HACKERS] pg_rewarm status

2013-12-17 Thread MauMau
From: "Robert Haas" On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila wrote: I have used pg_prewarm during some of work related to Buffer Management and other performance related work. It is quite useful utility. +1 for reviving this patch for 9.4 Any other votes? +1 Some customers requested:

Re: [HACKERS] pg_rewarm status

2013-12-17 Thread Josh Berkus
On 12/17/2013 06:34 AM, Robert Haas wrote: > On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila wrote: >> I have used pg_prewarm during some of work related to Buffer Management and >> other performance related work. It is quite useful utility. >> +1 for reviving this patch for 9.4 > > Any other votes

Re: [HACKERS] pg_rewarm status

2013-12-17 Thread Robert Haas
On Tue, Dec 17, 2013 at 3:31 PM, MauMau wrote: >> Any other votes? > > +1 > Some customers requested: > > 1. fill the database cache with frequently accessed data before starting or > resuming service for their users (for the first time or after maintenance > work), so that they can provide steady

Re: [HACKERS] pg_rewarm status

2013-12-17 Thread Jeff Janes
On Tue, Dec 17, 2013 at 8:02 AM, Jim Nasby wrote: > On 12/17/13, 8:34 AM, Robert Haas wrote: > >> On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila >> wrote: >> >>> I have used pg_prewarm during some of work related to Buffer Management >>> and >>> other performance related work. It is quite useful

Re: [HACKERS] pg_rewarm status

2013-12-17 Thread Robert Haas
On Tue, Dec 17, 2013 at 11:02 AM, Jim Nasby wrote: > On 12/17/13, 8:34 AM, Robert Haas wrote: >> >> On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila >> wrote: >>> >>> I have used pg_prewarm during some of work related to Buffer Management >>> and >>> other performance related work. It is quite usefu

Re: [HACKERS] pg_rewarm status

2013-12-17 Thread Kevin Grittner
Robert Haas wrote: > Amit Kapila wrote: >> I have used pg_prewarm during some of work related to Buffer >> Management and other performance related work. It is quite >> useful utility. >> +1 for reviving this patch for 9.4 > > Any other votes? Where I would have used a prewarm utility is followi

Re: [HACKERS] pg_rewarm status

2013-12-17 Thread Jim Nasby
On 12/17/13, 8:34 AM, Robert Haas wrote: On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila wrote: I have used pg_prewarm during some of work related to Buffer Management and other performance related work. It is quite useful utility. +1 for reviving this patch for 9.4 Any other votes? We've had

Re: [HACKERS] pg_rewarm status

2013-12-17 Thread Robert Haas
On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila wrote: > I have used pg_prewarm during some of work related to Buffer Management and > other performance related work. It is quite useful utility. > +1 for reviving this patch for 9.4 Any other votes? -- Robert Haas EnterpriseDB: http://www.enterpri

Re: [HACKERS] pg_rewarm status

2013-12-16 Thread Amit Kapila
On Tue, Dec 17, 2013 at 12:04 AM, Jeff Janes wrote: > On Mon, Dec 16, 2013 at 10:02 AM, Robert Haas wrote: >> On Mon, Dec 16, 2013 at 12:02 PM, Jeff Amiel >> wrote: >> Well, the patch was rejected, more or less because people felt it >> overlapped with pgfincore too much. I don't particularly

Re: [HACKERS] pg_rewarm status

2013-12-16 Thread Robert Haas
On Mon, Dec 16, 2013 at 1:34 PM, Jeff Janes wrote: > On Mon, Dec 16, 2013 at 10:02 AM, Robert Haas wrote: >> On Mon, Dec 16, 2013 at 12:02 PM, Jeff Amiel >> wrote: >> > Trying to follow the threads and other references - but I can't >> > determine where this patch ended up. >> > >> > (http://www

Re: [HACKERS] pg_rewarm status

2013-12-16 Thread Jeff Janes
On Mon, Dec 16, 2013 at 10:02 AM, Robert Haas wrote: > On Mon, Dec 16, 2013 at 12:02 PM, Jeff Amiel > wrote: > > Trying to follow the threads and other references - but I can't > determine where this patch ended up. > > ( > http://www.postgresql.org/message-id/ca+tgmobrrrxco+t6gcqrw_djw+uf9zedwf

Re: [HACKERS] pg_rewarm status

2013-12-16 Thread Jeff Janes
On Mon, Dec 16, 2013 at 9:02 AM, Jeff Amiel wrote: > Trying to follow the threads and other references - but I can't determine > where this patch ended up. > ( > http://www.postgresql.org/message-id/ca+tgmobrrrxco+t6gcqrw_djw+uf9zedwf9bejnu+rb5teb...@mail.gmail.com > ) > > I'm > trying to experi

Re: [HACKERS] pg_rewarm status

2013-12-16 Thread Robert Haas
On Mon, Dec 16, 2013 at 12:02 PM, Jeff Amiel wrote: > Trying to follow the threads and other references - but I can't determine > where this patch ended up. > (http://www.postgresql.org/message-id/ca+tgmobrrrxco+t6gcqrw_djw+uf9zedwf9bejnu+rb5teb...@mail.gmail.com) Well, the patch was rejected, m

[HACKERS] pg_rewarm status

2013-12-16 Thread Jeff Amiel
Trying to follow the threads and other references - but I can't determine where this patch ended up. (http://www.postgresql.org/message-id/ca+tgmobrrrxco+t6gcqrw_djw+uf9zedwf9bejnu+rb5teb...@mail.gmail.com) I'm trying to experiment with some new hardware - and the functionality to add specific