Re: [HACKERS] Synchronized Scan update

2007-03-14 Thread Zeugswetter Andreas ADI SD
The advantage of sync_scan_offset is that, in some situations, a second scan can actually finish faster than if it were the only query executing, because a previous scan has already caused some blocks to be cached. However, 16 is a small number because that benefit would only be

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Mon, 2007-03-12 at 13:21 +, Simon Riggs wrote: So based on those thoughts, sync_scan_offset should be fixed at 16, rather than being variable. In addition, ss_report_loc() should only report its position every 16 blocks, rather than do this every time, which will reduce overhead of this

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Tom Lane
Jeff Davis [EMAIL PROTECTED] writes: I agree that ss_report_loc() doesn't need to report on every call. If there's any significant overhead I agree that it should report less often. Do you think that the overhead is significant on such a simple function? One extra LWLock cycle per page

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Tue, 2007-03-13 at 12:53 -0400, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: I agree that ss_report_loc() doesn't need to report on every call. If there's any significant overhead I agree that it should report less often. Do you think that the overhead is significant on such a

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Simon Riggs
On Mon, 2007-03-12 at 17:46 -0700, Jeff Davis wrote: On Mon, 2007-03-12 at 13:21 +, Simon Riggs wrote: So based on those thoughts, sync_scan_offset should be fixed at 16, rather than being variable. In addition, ss_report_loc() should only report its position every 16 blocks, rather

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Tue, 2007-03-13 at 10:08 -0700, Jeff Davis wrote: One extra LWLock cycle per page processed definitely *is* a significant overhead ... can you say context swap storm? I'd think about doing it once every 100 or so pages. No lock is needed to store the hint. If somehow the hint

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Simon Riggs
On Tue, 2007-03-13 at 10:08 -0700, Jeff Davis wrote: On Tue, 2007-03-13 at 12:53 -0400, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: I agree that ss_report_loc() doesn't need to report on every call. If there's any significant overhead I agree that it should report less often.

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Tue, 2007-03-13 at 17:17 +, Simon Riggs wrote: On Tue, 2007-03-13 at 10:08 -0700, Jeff Davis wrote: On Tue, 2007-03-13 at 12:53 -0400, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: I agree that ss_report_loc() doesn't need to report on every call. If there's any

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Hannu Krosing
Ühel kenal päeval, T, 2007-03-13 kell 12:53, kirjutas Tom Lane: Jeff Davis [EMAIL PROTECTED] writes: I agree that ss_report_loc() doesn't need to report on every call. If there's any significant overhead I agree that it should report less often. Do you think that the overhead is significant

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Tue, 2007-03-13 at 17:11 +, Simon Riggs wrote: On Mon, 2007-03-12 at 17:46 -0700, Jeff Davis wrote: On Mon, 2007-03-12 at 13:21 +, Simon Riggs wrote: So based on those thoughts, sync_scan_offset should be fixed at 16, rather than being variable. In addition, ss_report_loc()

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Simon Riggs
On Tue, 2007-03-13 at 11:28 -0700, Jeff Davis wrote: On Tue, 2007-03-13 at 17:11 +, Simon Riggs wrote: On Mon, 2007-03-12 at 17:46 -0700, Jeff Davis wrote: On Mon, 2007-03-12 at 13:21 +, Simon Riggs wrote: So based on those thoughts, sync_scan_offset should be fixed at 16,

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Jeff Davis
On Tue, 2007-03-13 at 19:24 +, Simon Riggs wrote: The advantage of sync_scan_offset is that, in some situations, a second scan can actually finish faster than if it were the only query executing, because a previous scan has already caused some blocks to be cached. However, 16 is a

Re: [HACKERS] Synchronized Scan update

2007-03-13 Thread Simon Riggs
On Tue, 2007-03-13 at 13:39 -0700, Jeff Davis wrote: Do you have an opinion about sync_scan_threshold versus a simple sync_scan_enable? enable_sync_scan? After looking at other GUC names, I suggest that it's either sync_scan (for on/off) or sync_scan_threshold (if we do want to

Re: [HACKERS] Synchronized Scan update

2007-03-12 Thread Simon Riggs
On Fri, 2007-03-02 at 15:03 -0800, Jeff Davis wrote: Is there any consensus about whether to include these two parameters as GUCs or constants if my patch is to be accepted? (1) sync_scan_threshold: Use synchronized scanning for tables greater than this many pages; smaller tables will not be

Re: [HACKERS] Synchronized Scan update

2007-03-12 Thread Luke Lonergan
Simon, On 3/12/07 6:21 AM, Simon Riggs [EMAIL PROTECTED] wrote: So based on those thoughts, sync_scan_offset should be fixed at 16, rather than being variable. In addition, ss_report_loc() should only report its position every 16 blocks, rather than do this every time, which will reduce

Re: [HACKERS] Synchronized Scan update

2007-03-12 Thread Simon Riggs
On Mon, 2007-03-12 at 08:42 -0700, Luke Lonergan wrote: On 3/12/07 6:21 AM, Simon Riggs [EMAIL PROTECTED] wrote: So based on those thoughts, sync_scan_offset should be fixed at 16, rather than being variable. In addition, ss_report_loc() should only report its position every 16 blocks,

Re: [HACKERS] Synchronized Scan update

2007-03-07 Thread Jim Nasby
On Mar 6, 2007, at 9:43 AM, Josh Berkus wrote: Don't get me wrong, I want things to be easily understandable as well but the reason you site above pretty much makes us need to remove most of the postgresql.conf, including all bgwriter, vacuum cost delay, and autovac settings. Not to mention

Re: [HACKERS] Synchronized Scan update

2007-03-06 Thread Josh Berkus
JD, Don't get me wrong, I want things to be easily understandable as well but the reason you site above pretty much makes us need to remove most of the postgresql.conf, including all bgwriter, vacuum cost delay, and autovac settings. Not to mention commit delay and others ;). Wouldn't that

Re: [HACKERS] Synchronized Scan update

2007-03-05 Thread Jeff Davis
On Sun, 2007-03-04 at 11:54 +, Simon Riggs wrote: (2) sync_scan_offset: Start a new scan this many pages before a currently running scan to take advantage of the pages that are likely already in cache. I'm somewhat dubious about this parameter, I have to say, even though I am eager

Re: [HACKERS] Synchronized Scan update

2007-03-04 Thread Simon Riggs
On Fri, 2007-03-02 at 15:03 -0800, Jeff Davis wrote: Is there any consensus about whether to include these two parameters as GUCs or constants if my patch is to be accepted? (1) sync_scan_threshold: Use synchronized scanning for tables greater than this many pages; smaller tables will not be

Re: [HACKERS] Synchronized Scan update

2007-03-04 Thread Joshua D. Drake
(2) sync_scan_offset: Start a new scan this many pages before a currently running scan to take advantage of the pages that are likely already in cache. I'm somewhat dubious about this parameter, I have to say, even though I am eager for this feature. It seems like a magic parameter that

Re: [HACKERS] Synchronized Scan update

2007-03-02 Thread Jeff Davis
Is there any consensus about whether to include these two parameters as GUCs or constants if my patch is to be accepted? (1) sync_scan_threshold: Use synchronized scanning for tables greater than this many pages; smaller tables will not be affected. (2) sync_scan_offset: Start a new scan this

Re: [HACKERS] Synchronized Scan update

2007-03-02 Thread Josh Berkus
Jeff, Right now they are just constants defined in a header, but a GUC might make sense. I'd like to know which version is more acceptable when I submit my final patch. As much as I hate the thought of more GUCs, until we have a solid performance profile for synch scan we probably need them.

Re: [HACKERS] Synchronized Scan update

2007-03-02 Thread Jeff Davis
On Fri, 2007-03-02 at 15:49 -0800, Josh Berkus wrote: Jeff, Right now they are just constants defined in a header, but a GUC might make sense. I'd like to know which version is more acceptable when I submit my final patch. As much as I hate the thought of more GUCs, until we have a

Re: [HACKERS] Synchronized Scan update

2007-03-02 Thread Josh Berkus
Jeff, PS: Did you happen to get my patch for testing (sent off-list)? If testing will take a while, that's OK, I'd just like to know whether to expect the results before feature freeze. I'm not sure. We have a bunch to patches in our queue to test, and the benchmark guys don't really expect

[HACKERS] Synchronized Scan update

2007-02-27 Thread Jeff Davis
I have found some interesting results from my tests with the Synchronized Scan patch I'm working on. The two benefits that I hope to achieve with the patch are: (1) Better caching behavior with multiple sequential scans running in parallel (2) Faster sequential reads from disk and less seeking