Re: [HACKERS] idea for concurrent seqscans

2005-03-14 Thread Bruce Momjian
Added to TODO list: * Allow sequential scans to take advantage of other concurrent sequentiqal scans, also called Synchronised Scanning --- Jeff Davis wrote: I had an idea that might improve parallel seqscans on the

Re: [HACKERS] idea for concurrent seqscans

2005-02-28 Thread Gaetano Mendola
Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: Assuming you're talkning about You might wonder why we don't order all the regression test queries explicitly to get rid of this issue once and for all. The reason is that that would make the regression tests less useful, not more, since

Re: [HACKERS] idea for concurrent seqscans

2005-02-28 Thread Simon Riggs
On Sat, 2005-02-26 at 10:47 -0500, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: Assuming you're talkning about You might wonder why we don't order all the regression test queries explicitly to get rid of this issue once and for all. The reason is that that would make the

Re: [HACKERS] idea for concurrent seqscans

2005-02-27 Thread Neil Conway
Jeff Davis wrote: The only reason I did that was because the original source was difficult to read and work on. Is there a reason that code and comments wrap around to the next line throughout the file? I'm not sure what you mean. Assuming your editor expand tabs to 4 spaces (which is the

Re: [HACKERS] idea for concurrent seqscans

2005-02-26 Thread Jeff Davis
I have a newer version of my Synchronized Scanning patch which hopefully makes it closer to a real patch, the first one was more of a proof of concept. DONE: * I added a proper bounds check for the result it gets from shared memory. * I expanded the shared memory to be a static hash table

Re: [HACKERS] idea for concurrent seqscans

2005-02-26 Thread William Volkman
On Fri, 2005-02-25 at 22:49, Jim C. Nasby wrote: On Fri, Feb 25, 2005 at 11:51:40PM -0500, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: but I also hate to burden the developers with rewriting a lot of regression tests when their time could be better spent elsewhere. The patch

Re: [HACKERS] idea for concurrent seqscans

2005-02-26 Thread Andrew Dunstan
Tom Lane said: Jim C. Nasby [EMAIL PROTECTED] writes: but I also hate to burden the developers with rewriting a lot of regression tests when their time could be better spent elsewhere. Certainly, but I suspect it's just a matter of adding ORDER BY to everything, which just about anyone (even

Re: [HACKERS] idea for concurrent seqscans

2005-02-26 Thread Neil Conway
Jeff Davis wrote: I have a newer version of my Synchronized Scanning patch which hopefully makes it closer to a real patch, the first one was more of a proof of concept. A few minor comments: - context diffs (diff -c) are the preferred format. Also, folks usually send patches as a single diff; an

Re: [HACKERS] idea for concurrent seqscans

2005-02-26 Thread Neil Conway
William Volkman wrote: The patch is for *concurrent* seqscans, would the regression tests even be affected by this (IIRC they are single user, single process)? No; `make installcheck' is serial, but `make check' executes tests in parallel in multiple backends concurrently. -Neil

Re: [HACKERS] idea for concurrent seqscans

2005-02-26 Thread Jeff Davis
Thanks for the information. On Sat, 2005-02-26 at 23:39 +1100, Neil Conway wrote: Jeff Davis wrote: I have a newer version of my Synchronized Scanning patch which hopefully makes it closer to a real patch, the first one was more of a proof of concept. A few minor comments: - context

Re: [HACKERS] idea for concurrent seqscans

2005-02-26 Thread Jim C. Nasby
On Sat, Feb 26, 2005 at 12:22:45AM -0800, Jeff Davis wrote: * I expanded the shared memory to be a static hash table (currently set to a size of 8KB, but that is a one line change if that's too big). Now it can keep track of many scans on many tables at once. I know you're still early in this,

Re: [HACKERS] idea for concurrent seqscans

2005-02-26 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: Assuming you're talkning about You might wonder why we don't order all the regression test queries explicitly to get rid of this issue once and for all. The reason is that that would make the regression tests less useful, not more, since they'd tend to

Re: [HACKERS] idea for concurrent seqscans

2005-02-26 Thread Jeff Davis
On Sat, 2005-02-26 at 10:16 -0600, Jim C. Nasby wrote: On Sat, Feb 26, 2005 at 12:22:45AM -0800, Jeff Davis wrote: * I expanded the shared memory to be a static hash table (currently set to a size of 8KB, but that is a one line change if that's too big). Now it can keep track of many scans

[HACKERS] idea for concurrent seqscans

2005-02-25 Thread Jeff Davis
I had an idea that might improve parallel seqscans on the same relation. If you have lots of concurrent seqscans going on a large relation, the cache hit ratio is very low. But, if the seqscans are concurrent on the same relation, there may be something to gain by starting a seqscan near the page

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Simon Riggs
On Fri, 2005-02-25 at 00:34 -0800, Jeff Davis wrote: I had an idea that might improve parallel seqscans on the same relation. If you have lots of concurrent seqscans going on a large relation, the cache hit ratio is very low. But, if the seqscans are concurrent on the same relation, there

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Jeff Davis
On Fri, 2005-02-25 at 13:38 +, Simon Riggs wrote: On Fri, 2005-02-25 at 00:34 -0800, Jeff Davis wrote: I had an idea that might improve parallel seqscans on the same relation. If you have lots of concurrent seqscans going on a large relation, the cache hit ratio is very low. But, if

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Tom Lane
Jeff Davis [EMAIL PROTECTED] writes: (1) Do we care about reverse scans being done with synchronized scanning? If so, is there a good way to know in advance whether it is going to be a forward or reverse scan (i.e. before heap_getnext())? There are no reverse heapscans --- the only case where

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Jeff Davis
On Fri, 2005-02-25 at 12:54 -0500, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: (1) Do we care about reverse scans being done with synchronized scanning? If so, is there a good way to know in advance whether it is going to be a forward or reverse scan (i.e. before heap_getnext())?

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Tom Lane
Jeff Davis [EMAIL PROTECTED] writes: I didn't consider that. Is there a reason the regression tests assume the results will be returned in a certain order (or a consistent order)? We use diff as the checking tool. regards, tom lane ---(end of

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Jim C. Nasby
On Fri, Feb 25, 2005 at 01:30:57PM -0500, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: I didn't consider that. Is there a reason the regression tests assume the results will be returned in a certain order (or a consistent order)? We use diff as the checking tool. Doesn't the SQL

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Jeff Davis
On Fri, 2005-02-25 at 13:30 -0500, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: I didn't consider that. Is there a reason the regression tests assume the results will be returned in a certain order (or a consistent order)? We use diff as the checking tool. Well, that does make

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Jeff Davis
On Fri, 2005-02-25 at 18:03 -0600, Jim C. Nasby wrote: On Fri, Feb 25, 2005 at 01:30:57PM -0500, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: I didn't consider that. Is there a reason the regression tests assume the results will be returned in a certain order (or a consistent

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Bruce Momjian
Sorry, please disregard my ramblings. I thought it was a different question. --- pgman wrote: Jim C. Nasby wrote: On Fri, Feb 25, 2005 at 01:30:57PM -0500, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: I

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Bruce Momjian
Jim C. Nasby wrote: On Fri, Feb 25, 2005 at 01:30:57PM -0500, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: I didn't consider that. Is there a reason the regression tests assume the results will be returned in a certain order (or a consistent order)? We use diff as the

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Jim C. Nasby
On Fri, Feb 25, 2005 at 04:30:17PM -0800, Jeff Davis wrote: On Fri, 2005-02-25 at 18:03 -0600, Jim C. Nasby wrote: On Fri, Feb 25, 2005 at 01:30:57PM -0500, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: I didn't consider that. Is there a reason the regression tests assume the

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: but I also hate to burden the developers with rewriting a lot of regression tests when their time could be better spent elsewhere. Certainly, but I suspect it's just a matter of adding ORDER BY to everything, which just about anyone (even myself!) should

Re: [HACKERS] idea for concurrent seqscans

2005-02-25 Thread Jim C. Nasby
On Fri, Feb 25, 2005 at 11:51:40PM -0500, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: but I also hate to burden the developers with rewriting a lot of regression tests when their time could be better spent elsewhere. Certainly, but I suspect it's just a matter of adding ORDER