Re: [HACKERS] Sequential scans

2007-05-03 Thread Simon Riggs
On Wed, 2007-05-02 at 23:59 +0100, Heikki Linnakangas wrote: Umm, you naturally have just entry per relation, but we were talking about how many entries the table needs to hold.. You're patch had a hard-coded value of 1000 which is quite arbitrary. We need to think of the interaction with

Re: [HACKERS] Sequential scans

2007-05-03 Thread Heikki Linnakangas
Jeff Davis wrote: On Wed, 2007-05-02 at 23:59 +0100, Heikki Linnakangas wrote: Jeff Davis wrote: On Wed, 2007-05-02 at 20:58 +0100, Heikki Linnakangas wrote: Jeff Davis wrote: What should be the maximum size of this hash table? Good question. And also, how do you remove entries from it? I

Re: [HACKERS] Sequential scans

2007-05-03 Thread Heikki Linnakangas
Simon Riggs wrote: We need to think of the interaction with partitioning here. People will ask whether we would recommend that individual partitions of a large table should be larger/smaller than a particular size, to allow these optimizations to kick in. My thinking is that database designers

Re: [HACKERS] Sequential scans

2007-05-03 Thread Jeff Davis
On Thu, 2007-05-03 at 09:25 +0100, Heikki Linnakangas wrote: The hash table keeps track of ongoing seq scans. That's presumably related to number of backends; I can't imagine a plan where a backend is executing more than one seq scan at a time, so that gives us an upper bound of NBackends

Re: [HACKERS] Sequential scans

2007-05-03 Thread Jeff Davis
On Thu, 2007-05-03 at 08:01 +0100, Simon Riggs wrote: On Wed, 2007-05-02 at 23:59 +0100, Heikki Linnakangas wrote: Umm, you naturally have just entry per relation, but we were talking about how many entries the table needs to hold.. You're patch had a hard-coded value of 1000 which is

Re: [HACKERS] Sequential scans

2007-05-03 Thread Heikki Linnakangas
Jeff Davis wrote: What I was trying to say before is that, in my design, it keeps track of relations being scanned, not scans happening. So 5 backends scanning the same table would result in one entry that consists of the most recently- read block by any of those backends for that relation.

Re: [HACKERS] Sequential scans

2007-05-03 Thread Jeff Davis
On Thu, 2007-05-03 at 19:27 +0100, Heikki Linnakangas wrote: I understand that the data structure keeps track of relations being scanned, with one entry per such relation. I think that's very good design, and I'm not suggesting to change that. But what's the right size for that? We don't

[HACKERS] Sequential scans

2007-05-02 Thread Heikki Linnakangas
Hi, I'm starting to review the synchronized scans and scan-resistant buffer cache patches. The patches have complex interactions so I'm taking a holistic approach. There's four outstanding issues with the sync scans in particular: 1. The simplistic hash approach. While it's nice to not have

Re: [HACKERS] Sequential scans

2007-05-02 Thread Jeff Davis
On Wed, 2007-05-02 at 14:26 +0100, Heikki Linnakangas wrote: Hi, I'm starting to review the synchronized scans and scan-resistant buffer cache patches. The patches have complex interactions so I'm taking a holistic approach. There's four outstanding issues with the sync scans in

Re: [HACKERS] Sequential scans

2007-05-02 Thread Gregory Stark
Heikki Linnakangas [EMAIL PROTECTED] writes: Let's use a normal hash table instead, and use a lock to protect it. If we only update it every 10 pages or so, the overhead should be negligible. To further reduce contention, we could modify ReadBuffer to let the caller know if the read

Re: [HACKERS] Sequential scans

2007-05-02 Thread Heikki Linnakangas
Gregory Stark wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Let's use a normal hash table instead, and use a lock to protect it. If we only update it every 10 pages or so, the overhead should be negligible. To further reduce contention, we could modify ReadBuffer to let the caller know

Re: [HACKERS] Sequential scans

2007-05-02 Thread Heikki Linnakangas
Jeff Davis wrote: On Wed, 2007-05-02 at 14:26 +0100, Heikki Linnakangas wrote: Let's use a normal hash table instead, and use a lock to protect it. If we only update it every 10 pages or so, the overhead should be negligible. To further reduce contention, we could modify ReadBuffer to let the

Re: [HACKERS] Sequential scans

2007-05-02 Thread Jeff Davis
On Wed, 2007-05-02 at 20:02 +0100, Heikki Linnakangas wrote: Gregory Stark wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Let's use a normal hash table instead, and use a lock to protect it. If we only update it every 10 pages or so, the overhead should be negligible. To

Re: [HACKERS] Sequential scans

2007-05-02 Thread Heikki Linnakangas
Jeff Davis wrote: On Wed, 2007-05-02 at 20:02 +0100, Heikki Linnakangas wrote: Gregory Stark wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Let's use a normal hash table instead, and use a lock to protect it. If we only update it every 10 pages or so, the overhead should be negligible.

Re: [HACKERS] Sequential scans

2007-05-02 Thread Jeff Davis
On Wed, 2007-05-02 at 20:58 +0100, Heikki Linnakangas wrote: Jeff Davis wrote: What should be the maximum size of this hash table? Good question. And also, how do you remove entries from it? I guess the size should somehow be related to number of backends. Each backend will

Re: [HACKERS] Sequential scans

2007-05-02 Thread Heikki Linnakangas
Jeff Davis wrote: On Wed, 2007-05-02 at 20:58 +0100, Heikki Linnakangas wrote: Jeff Davis wrote: What should be the maximum size of this hash table? Good question. And also, how do you remove entries from it? I guess the size should somehow be related to number of backends. Each backend

Re: [HACKERS] Sequential scans

2007-05-02 Thread Jeff Davis
On Wed, 2007-05-02 at 23:59 +0100, Heikki Linnakangas wrote: Jeff Davis wrote: On Wed, 2007-05-02 at 20:58 +0100, Heikki Linnakangas wrote: Jeff Davis wrote: What should be the maximum size of this hash table? Good question. And also, how do you remove entries from it? I guess the