Re: [HACKERS] would hw acceleration help postgres (databases in general) ?

2011-03-26 Thread 3dmashup
Yes! Probably very much so. There is good evidence that using multiple CPU's and GPU's will speed sorting and many other database operations too. See http://www.cs.cmu.edu/afs/cs.cmu.edu/Web/People/ngm/15-823/project/Final.pdf The question become how practical is it? There are numerous iss

Re: [HACKERS] Libpq PGRES_COPY_BOTH - version compatibility

2011-03-26 Thread Tom Lane
Robert Haas writes: > On Mon, Jan 3, 2011 at 6:55 AM, Magnus Hagander wrote: >>> ISTM that the correct fix is to increment to protocol version number to >>> 3.1 and send PGRES_COPY_OUT if the client requests version 3.0.  That's >>> what the version numbers are for, no? >> It still won't help in

Re: [HACKERS] DO hint update?

2011-03-26 Thread Tom Lane
Robert Haas writes: > Does this need updating now that languages are more exensiony? Or is > it OK? > if (!HeapTupleIsValid(languageTuple)) > ereport(ERROR, > (errcode(ERRCODE_UNDEFINED_OBJECT), > errmsg("la

Re: [HACKERS] Libpq PGRES_COPY_BOTH - version compatibility

2011-03-26 Thread Robert Haas
On Mon, Jan 3, 2011 at 6:55 AM, Magnus Hagander wrote: >> ISTM that the correct fix is to increment to protocol version number to >> 3.1 and send PGRES_COPY_OUT if the client requests version 3.0.  That's >> what the version numbers are for, no? > > In a way - yes. I assume we didn't do that becau

Re: [HACKERS] Performance bug in DO blocks

2011-03-26 Thread Robert Haas
On Tue, Jan 18, 2011 at 1:46 PM, Tom Lane wrote: > I just noticed that if you execute the same DO command over and over > within a session, it gets slower and slower.  And if you keep it up > you'll notice the backend's RAM consumption bloating too.  The cause > appears to be that we leak the cach

[HACKERS] DO hint update?

2011-03-26 Thread Robert Haas
Does this need updating now that languages are more exensiony? Or is it OK? if (!HeapTupleIsValid(languageTuple)) ereport(ERROR, (errcode(ERRCODE_UNDEFINED_OBJECT), errmsg("language \"%s\" does not exist", la

Re: [HACKERS] Typed-tables patch broke pg_upgrade

2011-03-26 Thread Robert Haas
On Fri, Mar 11, 2011 at 8:28 AM, Bruce Momjian wrote: > Is this still an open bug? Is anyone working on fixing this? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] psql \dt and table size

2011-03-26 Thread Robert Haas
On Sat, Mar 26, 2011 at 9:42 PM, Robert Haas wrote: > On Wed, Mar 23, 2011 at 4:33 PM, Alvaro Herrera > wrote: >> Excerpts from Robert Haas's message of mié mar 23 17:24:59 -0300 2011: >>> On Mon, Mar 21, 2011 at 1:44 PM, Bernd Helmle wrote: >>> > It stroke me today again, that \dt+ isn't displa

Re: [HACKERS] psql \dt and table size

2011-03-26 Thread Robert Haas
On Wed, Mar 23, 2011 at 4:33 PM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of mié mar 23 17:24:59 -0300 2011: >> On Mon, Mar 21, 2011 at 1:44 PM, Bernd Helmle wrote: >> > It stroke me today again, that \dt+ isn't displaying the acurate table size >> > for tables, since it uses p

[HACKERS] alpha5

2011-03-26 Thread Robert Haas
Per previous discussion, I'm going to wrap alpha5 Monday morning Eastern time, barring objections. This time, I'm going to try to make sure that announcements actually go out; I dropped the ball on that last time. If anyone wants to push time zone updates, translation updates, etc., I suppose now

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-03-26 Thread Robert Haas
On Sat, Mar 26, 2011 at 5:19 PM, Dimitri Fontaine wrote: > I think the best choice is to only accept qualified parameter names in > SQL functions (function_name.parameter_name).  If a referenced table > share the function's name, ERROR out and HINT to alias the table name. > > If we allow more tha

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-26 Thread Jeff Janes
On Fri, Mar 25, 2011 at 8:07 AM, Gurjeet Singh wrote: > On Tue, Mar 22, 2011 at 3:53 PM, Robert Haas wrote: >> >> On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes wrote: >> > On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas >> > wrote: >> >> >> >> A related area that could use some looking at is why pe

Re: [HACKERS] SSI bug?

2011-03-26 Thread Dan Ports
On Fri, Mar 25, 2011 at 04:06:30PM -0400, Tom Lane wrote: > Up to now, I believe the lockmgr's lock table is the only shared hash > table that is expected to grow past the declared size; that can happen > anytime a session exceeds max_locks_per_transaction, which we consider > to be only a soft lim

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-03-26 Thread Pavel Stehule
2011/3/26 Dimitri Fontaine : > Joshua Berkus writes: >>> Personally I'd vote for *not* having any such dangerous semantics as >>> that. We should have learned better by now from plpgsql experience. >>> I think the best idea is to throw error for ambiguous references, >>> period. >> >> As a likely

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-03-26 Thread Dimitri Fontaine
Joshua Berkus writes: >> Personally I'd vote for *not* having any such dangerous semantics as >> that. We should have learned better by now from plpgsql experience. >> I think the best idea is to throw error for ambiguous references, >> period. > > As a likely heavy user of this feature, I agree

Re: [HACKERS] resolving SQL ambiguity (was Re: WIP: Allow SQL-lang funcs to ref params by param name)

2011-03-26 Thread Darren Duncan
Pavel Stehule wrote: sorry - I dislike this. The design is correct, but it is against to SQL verbosity. A reader must to thinking about missing tablenames. I dont't think so it is good solution, because it doesn't solve a backing compatibility problem - somebody must to fix a function still, and

Re: [HACKERS] resolving SQL ambiguity (was Re: WIP: Allow SQL-lang funcs to ref params by param name)

2011-03-26 Thread Pavel Stehule
2011/3/26 Darren Duncan : > Pavel Stehule wrote: >> >> 2011/3/26 Darren Duncan : >>> >>> I mention 2 possible solutions here, both which involve syntax >>> alterations, >>> each between the -- lines.  I personally like the second/lower >>> option more. >> >> -1 >> >> this is not based on an

Re: [HACKERS] resolving SQL ambiguity (was Re: WIP: Allow SQL-lang funcs to ref params by param name)

2011-03-26 Thread Darren Duncan
Pavel Stehule wrote: 2011/3/26 Darren Duncan : I mention 2 possible solutions here, both which involve syntax alterations, each between the -- lines. I personally like the second/lower option more. -1 this is not based on any pattern on SQL. It's not simple, and it introduce a reserv

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Jan Wieck
On 3/26/2011 3:17 PM, Robert Haas wrote: On Mar 26, 2011, at 1:44 PM, Itagaki Takahiro wrote: On Sun, Mar 27, 2011 at 01:12, Simon Riggs wrote: At the same time I would change count_nondeletable_pages() so that it uses a forward scan direction (if that leads to a speedup). +1. Hmm.

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Jan Wieck
On 3/26/2011 12:12 PM, Simon Riggs wrote: On Sat, Mar 26, 2011 at 2:30 PM, Jan Wieck wrote: My current idea for a fix is to modify lazy_truncate_heap(). It does acquire and release the exclusive lock, so it should be possible to do this in smaller chunks, releasing and reacquiring the lock

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Robert Haas
On Mar 26, 2011, at 1:44 PM, Itagaki Takahiro wrote: > On Sun, Mar 27, 2011 at 01:12, Simon Riggs wrote: >>> At the same time I would >>> change count_nondeletable_pages() so that it uses a forward scan direction >>> (if that leads to a speedup). > > +1. Hmm. That would speed up truncations t

Re: [HACKERS] 9.1 Beta

2011-03-26 Thread Robert Haas
On Sat, Mar 26, 2011 at 11:46 AM, Tom Lane wrote: > Greg Stark writes: >> There's not much point in releasing a beta with behaviour that we know >> we intend to change. All it will do is elicit bug reports that we have >> to respond to saying "we know, we were going to change that anyways". > >>

Re: [HACKERS] race condition in sync rep

2011-03-26 Thread Tom Lane
Simon Riggs writes: > On Sat, Mar 26, 2011 at 4:24 PM, Tom Lane wrote: >> Well, in that case, it should be on the open-items list.  If the system >> is still behaving that way, it's a bug. > Is it? Sync rep requires fsync on the standby. If you then explicitly > turn off fsync on the standby the

Re: [HACKERS] race condition in sync rep

2011-03-26 Thread Robert Haas
On Sat, Mar 26, 2011 at 12:41 PM, Simon Riggs wrote: > Is it? Sync rep requires fsync on the standby. If you then explicitly > turn off fsync on the standby then it has a performance impact, as > documented. Actually, it doesn't, now that you fixed this. Before: [rhaas@office ~]$ pgbench -T 10

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Itagaki Takahiro
On Sun, Mar 27, 2011 at 01:12, Simon Riggs wrote: >> At the same time I would >> change count_nondeletable_pages() so that it uses a forward scan direction >> (if that leads to a speedup). +1. > Do we need that? Linux readahead works in both directions doesn't it? > Guess it wouldn't hurt too mu

Re: [HACKERS] Open issues for collations

2011-03-26 Thread Martijn van Oosterhout
On Sat, Mar 26, 2011 at 12:36:43AM -0400, Tom Lane wrote: > ** Selecting a field from a record-returning function's output. > Currently, we'll use the field's declared collation; except that > if the field has default collation, we'll replace that with the common > collation of the function's input

Re: [HACKERS] Open issues for collations

2011-03-26 Thread Robert Haas
On Mar 26, 2011, at 12:34 PM, Greg Stark wrote: > The ones with the collation expressions inside the casts seem very > strange and the behaviour following the domain don't seem > unreasonable. The behaviour with the collate clauses outside the cast > should definitely be follow the explicit collat

Re: [HACKERS] race condition in sync rep

2011-03-26 Thread Simon Riggs
On Sat, Mar 26, 2011 at 4:24 PM, Tom Lane wrote: > Simon Riggs writes: >> On Sat, Mar 26, 2011 at 4:04 PM, Tom Lane wrote: >>> In particular, in view of today's fix, shouldn't this commit be reverted? >>> >>> http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=6e8e7cc580665ddd43c8ca

Re: [HACKERS] race condition in sync rep

2011-03-26 Thread Robert Haas
On Mar 26, 2011, at 12:24 PM, Tom Lane wrote: > Simon Riggs writes: >> On Sat, Mar 26, 2011 at 4:04 PM, Tom Lane wrote: >>> In particular, in view of today's fix, shouldn't this commit be reverted? >>> >>> http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=6e8e7cc580665ddd43c8ca2a

Re: [HACKERS] 9.1 Beta

2011-03-26 Thread Simon Riggs
On Sat, Mar 26, 2011 at 3:46 PM, Tom Lane wrote: > Greg Stark writes: >> There's not much point in releasing a beta with behaviour that we know >> we intend to change. All it will do is elicit bug reports that we have >> to respond to saying "we know, we were going to change that anyways". > >> I

Re: [HACKERS] Open issues for collations

2011-03-26 Thread Greg Stark
On Sat, Mar 26, 2011 at 3:16 PM, Robert Haas wrote: >> ** What to do with domains whose declaration includes a COLLATE clause? >> Currently, we'll impute that collation to the result of a cast to the >> domain type --- even if the cast's input expression includes an >> explicit COLLATE clause. > >

Re: [HACKERS] race condition in sync rep

2011-03-26 Thread Simon Riggs
On Sat, Mar 26, 2011 at 3:00 PM, Robert Haas wrote: > On Mar 26, 2011, at 6:16 AM, Simon Riggs wrote: >> That seems very unlikely even with fsync=off in a real config where we >> have network path to consider. >> >> Your definition of a "nasty" race condition seems off. >> >> I've added code for

Re: [HACKERS] race condition in sync rep

2011-03-26 Thread Tom Lane
Simon Riggs writes: > On Sat, Mar 26, 2011 at 4:04 PM, Tom Lane wrote: >> In particular, in view of today's fix, shouldn't this commit be reverted? >> >> http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=6e8e7cc580665ddd43c8ca2acc6d60f345570a57 >> >> I thought at the time that th

Re: [HACKERS] race condition in sync rep

2011-03-26 Thread Simon Riggs
On Sat, Mar 26, 2011 at 4:04 PM, Tom Lane wrote: > Robert Haas writes: >>> I've added code for you. > >> Your skepticism seems out of place. I actually hit this problem in testing. >> We could debate how realistic my test setup was, but why?  It is not our >> policy - and bad practice in genera

Re: [HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Simon Riggs
On Sat, Mar 26, 2011 at 2:30 PM, Jan Wieck wrote: > My current idea for a fix is to modify lazy_truncate_heap(). It does acquire > and release the exclusive lock, so it should be possible to do this in > smaller chunks, releasing and reacquiring the lock so that client > transactions can get thei

Re: [HACKERS] race condition in sync rep

2011-03-26 Thread Tom Lane
Robert Haas writes: >> I've added code for you. > Your skepticism seems out of place. I actually hit this problem in testing. > We could debate how realistic my test setup was, but why? It is not our > policy - and bad practice in general - to ship code with race conditions. > The code you a

Re: [HACKERS] 9.1 Beta

2011-03-26 Thread Tom Lane
Greg Stark writes: > There's not much point in releasing a beta with behaviour that we know > we intend to change. All it will do is elicit bug reports that we have > to respond to saying "we know, we were going to change that anyways". > I think the goal of a beta is to be able to say "we think

Re: [HACKERS] Open issues for collations

2011-03-26 Thread Robert Haas
On Mar 26, 2011, at 12:36 AM, Tom Lane wrote: > ** Selecting a field from a record-returning function's output. > Currently, we'll use the field's declared collation; except that > if the field has default collation, we'll replace that with the common > collation of the function's inputs, if any.

Re: [HACKERS] Open issues for collations

2011-03-26 Thread Greg Stark
On Sat, Mar 26, 2011 at 4:36 AM, Tom Lane wrote: > ** Selecting a field from a record-returning function's output. > Currently, we'll use the field's declared collation; except that > if the field has default collation, we'll replace that with the common > collation of the function's inputs, if an

Re: [HACKERS] When and how many times does ExecSetParamPlan executes?

2011-03-26 Thread Vaibhav Kaushal
Thanks a lot Mr. Tom. I understand it better now. I see that the function ExecSetParamPlan is indeed called only once when executing a query which would have a paramkind = PARAM_EXEC. The query helped me see the run in debugger, making things clearer (a lot clearer in fact, especially reading your

Re: [HACKERS] race condition in sync rep

2011-03-26 Thread Robert Haas
On Mar 26, 2011, at 6:16 AM, Simon Riggs wrote: > That seems very unlikely even with fsync=off in a real config where we > have network path to consider. > > Your definition of a "nasty" race condition seems off. > > I've added code for you. Your skepticism seems out of place. I actually hit th

Re: [HACKERS] 9.1 Beta

2011-03-26 Thread Andrew Dunstan
On 03/25/2011 06:18 PM, Simon Riggs wrote: Judging by the number of new threads about development for 9.2, I think its time we declared 9.1 Beta. I just had a conversation with some Debian developers about how PostgreSQL 9.0 got pulled out of their release because we delayed by 3 weeks. So we m

Re: [HACKERS] 9.1 Beta

2011-03-26 Thread Robert Haas
On Mar 26, 2011, at 4:27 AM, Simon Riggs wrote: > The basic point of this post was this: If we wait for the Open Items > list to drop to zero, many people are unable to contribute and that > means delay. Also, waiting for the Open Items list to drop to zero > puts the schedule in the hands of one

Re: [HACKERS] 9.1 Beta

2011-03-26 Thread Greg Stark
On Sat, Mar 26, 2011 at 9:22 AM, Simon Riggs wrote: > First, you are presuming that the state of those patches must hold up > the whole release process. I don't think it should There's not much point in releasing a beta with behaviour that we know we intend to change. All it will do is elicit bug

[HACKERS] Lock problem with autovacuum truncating heap

2011-03-26 Thread Jan Wieck
We have run across a problem with autovacuum that occurs when it can truncate off a large amount of empty blocks. It behaves different in version 9.0 than previous versions. Both behaviors are bad. Consider a relation receives constant inserts/updates that are satisfied using freespace at the

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-03-26 Thread Robert Haas
On Mar 25, 2011, at 11:23 PM, Tom Lane wrote: > If this were PL/perl, or PL/almost-anything-except-SQL, I could get > behind such a proposal. But it's not, it's SQL; and SQL doesn't do > things that way. SQL's idea of disambiguation is qualified names. > > And even more to the point: to the ext

Re: [HACKERS] race condition in sync rep

2011-03-26 Thread Simon Riggs
On Sat, Mar 26, 2011 at 1:11 AM, Robert Haas wrote: > I believe I've figured out why synchronous replication has such > terrible performance with fsync=off: it has a nasty race condition. > It may happen - if the standby responds very quickly - that the > standby acks the commit record and awaken

Re: [HACKERS] Open issues for collations

2011-03-26 Thread Simon Riggs
On Sat, Mar 26, 2011 at 4:36 AM, Tom Lane wrote: > Robert Haas writes: >> I think some discussion of which of the things on the open >> item lists need to be done before beta might be helpful, and we ought >> to add any items that are not there but are blockers. > > Here's a quick enumeration of

Re: [HACKERS] 9.1 Beta

2011-03-26 Thread Simon Riggs
On Sat, Mar 26, 2011 at 1:48 AM, Robert Haas wrote: > On Fri, Mar 25, 2011 at 6:18 PM, Simon Riggs wrote: > >> The sooner we declare Beta, the sooner people will test. Then we will >> have user feedback, bugs to fix etc.. Everybody is very clearly >> sitting idle. With a longer bug list we will m

Re: [HACKERS] 9.1 Beta

2011-03-26 Thread Simon Riggs
On Sat, Mar 26, 2011 at 12:33 AM, Tom Lane wrote: > > The correct question is whether we're ready for beta, and I would say > the answer is clearly no, unless you have a pretty low standard for what > "ready for beta" means.  Perhaps it would be suitable to discuss what > the standard for that rea

Re: [HACKERS] resolving SQL ambiguity (was Re: WIP: Allow SQL-lang funcs to ref params by param name)

2011-03-26 Thread Pavel Stehule
2011/3/26 Darren Duncan : > Robert Haas wrote: >> >> On Mar 25, 2011, at 9:22 PM, Joshua Berkus wrote: >>> >>> Tom, >>> Personally I'd vote for *not* having any such dangerous semantics as that. We should have learned better by now from plpgsql experience. I think the best idea is t