Re: [HACKERS] heap vacuum cleanup locks

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 12:19 AM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Sun, Jun 5, 2011 at 12:03, Robert Haas robertmh...@gmail.com wrote: If other buffer pins do exist, then we can't defragment the page, but that doesn't mean no useful work can be done: we can still mark used

Re: [HACKERS] heap vacuum cleanup locks

2011-06-06 Thread Jim Nasby
On Jun 6, 2011, at 1:00 AM, Robert Haas wrote: On Mon, Jun 6, 2011 at 12:19 AM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Sun, Jun 5, 2011 at 12:03, Robert Haas robertmh...@gmail.com wrote: If other buffer pins do exist, then we can't defragment the page, but that doesn't mean no

Re: [HACKERS] heap vacuum cleanup locks

2011-06-06 Thread Pavan Deolasee
On Sun, Jun 5, 2011 at 8:33 AM, Robert Haas robertmh...@gmail.com wrote: We've occasionally seen problems with VACUUM getting stuck for failure to acquire a cleanup lock due to, for example, a cursor holding a pin on the buffer page.  In the worst case, this can cause an undetected deadlock,

Re: [HACKERS] storing TZ along timestamps

2011-06-06 Thread Jim Nasby
On Jun 4, 2011, at 3:56 AM, Greg Stark wrote: On Thu, Jun 2, 2011 at 8:58 PM, Jim Nasby j...@nasby.net wrote: I'm torn between whether the type should store the original time or the original time converted to GMT. This is the wrong way to think about it. We *never* store time converted

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Heikki Linnakangas
On 06.06.2011 07:12, Robert Haas wrote: I did some further investigation of this. It appears that more than 99% of the lock manager lwlock traffic that remains with this patch applied has locktag_type == LOCKTAG_VIRTUALTRANSACTION. Every SELECT statement runs in a separate transaction, and for

Re: [HACKERS] heap vacuum cleanup locks

2011-06-06 Thread Heikki Linnakangas
On 06.06.2011 09:35, Jim Nasby wrote: I've had a related idea that I haven't looked into... if you're scanning a relation (ie: index scan, seq scan) I've wondered if it would be more efficient to deal with the entire page at once, possibly be making a copy of it. This would reduce the number

[HACKERS] WIP: AuthenticationMD5 protocol documentation clarification

2011-06-06 Thread Cyan Ogilvie
This is my first patch, so I hope I've got the process right for submitting patches. I'm building a driver to talk version 3.0 of the protocol, and generally I've found the documentation to be excellent. One are I had trouble with was responding to the AuthenticationMD5Password challenge. After

Re: [HACKERS] Vacuum, visibility maps and SKIP_PAGES_THRESHOLD

2011-06-06 Thread Heikki Linnakangas
On 03.06.2011 22:16, Bruce Momjian wrote: I realize we just read the pages from the kernel to maintain sequential I/O, but do we actually read the contents of the page if we know it doesn't need vacuuming? Yes. If so, do we need to? Not necessarily, but it allows us to freeze old tuples,

[HACKERS] gdb with postgres

2011-06-06 Thread HuangQi
Hi, I was using gdb to debug postgres. In order to debug the backend of running query, I start postgres first and use select * from pg_backend_pid() to ask for backend pid. Then I start gdb in another bash window with gdb postgres and attach the pid obtained above and set the breakpoint. Then I

Re: [HACKERS] gdb with postgres

2011-06-06 Thread Pavan Deolasee
On Mon, Jun 6, 2011 at 1:13 PM, HuangQi huangq...@gmail.com wrote: Hi,    I was using gdb to debug postgres. In order to debug the backend of running query, I start postgres first and use select * from pg_backend_pid() to ask for backend pid. Then I start gdb in another bash window with gdb

Re: [HACKERS] gdb with postgres

2011-06-06 Thread HuangQi
Sorry, but recompile with this flag still doesn't work. On 6 June 2011 15:46, Pavan Deolasee pavan.deola...@gmail.com wrote: On Mon, Jun 6, 2011 at 1:13 PM, HuangQi huangq...@gmail.com wrote: Hi, I was using gdb to debug postgres. In order to debug the backend of running query, I start

Re: [HACKERS] BLOB support

2011-06-06 Thread Radosław Smogura
On Sun, 05 Jun 2011 22:16:41 +0200, Dimitri Fontaine wrote: Tom Lane t...@sss.pgh.pa.us writes: Yes. I think the appropriate problem statement is provide streaming access to large field values, as an alternative to just fetching/storing the entire value at once. I see no good reason to

Re: [HACKERS] heap vacuum cleanup locks

2011-06-06 Thread Simon Riggs
On Sun, Jun 5, 2011 at 4:03 AM, Robert Haas robertmh...@gmail.com wrote: We've occasionally seen problems with VACUUM getting stuck for failure to acquire a cleanup lock due to, for example, a cursor holding a pin on the buffer page.  In the worst case, this can cause an undetected deadlock,

Re: [HACKERS] BLOB support

2011-06-06 Thread Pavel Stehule
2011/6/6 Radosław Smogura rsmog...@softperience.eu: On Sun, 05 Jun 2011 22:16:41 +0200, Dimitri Fontaine wrote: Tom Lane t...@sss.pgh.pa.us writes: Yes.  I think the appropriate problem statement is provide streaming access to large field values, as an alternative to just fetching/storing

Re: [HACKERS] SAVEPOINTs and COMMIT performance

2011-06-06 Thread Heikki Linnakangas
On 06.02.2011 23:09, Simon Riggs wrote: On Sun, 2011-02-06 at 12:11 -0500, Bruce Momjian wrote: Did this ever get addressed? Patch attached. Seems like the easiest fix I can come up with. @@ -2518,7 +2518,7 @@ CommitTransactionCommand(void) case TBLOCK_SUBEND:

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Simon Riggs
On Sat, Jun 4, 2011 at 5:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: The approach looks sound to me. It's a fairly isolated patch and we should be considering this for inclusion in 9.1, not wait another year. That suggestion is completely insane.  The

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Heikki Linnakangas
On 06.06.2011 12:40, Simon Riggs wrote: On Sat, Jun 4, 2011 at 5:55 PM, Tom Lanet...@sss.pgh.pa.us wrote: Simon Riggssi...@2ndquadrant.com writes: The approach looks sound to me. It's a fairly isolated patch and we should be considering this for inclusion in 9.1, not wait another year.

Re: [HACKERS] WIP: Fast GiST index build

2011-06-06 Thread Heikki Linnakangas
On 06.06.2011 10:42, Heikki Linnakangas wrote: On 03.06.2011 14:02, Alexander Korotkov wrote: Hackers, WIP patch of fast GiST index build is attached. Code is dirty and comments are lacking, but it works. Now it is ready for first benchmarks, which should prove efficiency of selected

Re: [HACKERS] DOMAINs and CASTs

2011-06-06 Thread Peter Eisentraut
On tis, 2011-05-17 at 14:11 -0500, Jaime Casanova wrote: On Tue, May 17, 2011 at 12:19 PM, Robert Haas robertmh...@gmail.com wrote: The more controversial question is what to do if someone tries to create such a cast anyway. We could just ignore that as we do now, or we could throw a

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 2:54 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Ah, I remember I saw that vxid lock pop up quite high in an oprofile profile recently. I think it was the case of executing a lot of very simple prepared queries. So it would be nice to address that,

Re: [HACKERS] WIP: Fast GiST index build

2011-06-06 Thread Alexander Korotkov
Hi! On Mon, Jun 6, 2011 at 2:51 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 06.06.2011 10:42, Heikki Linnakangas wrote: I ran another test with a simple table generated with: CREATE TABLE pointtest (p point); INSERT INTO pointtest SELECT point(random(), random())

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Heikki Linnakangas
On 06.06.2011 07:12, Robert Haas wrote: I did some further investigation of this. It appears that more than 99% of the lock manager lwlock traffic that remains with this patch applied has locktag_type == LOCKTAG_VIRTUALTRANSACTION. Every SELECT statement runs in a separate transaction, and for

Re: [HACKERS] heap vacuum cleanup locks

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 2:36 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Do we know if this is really a problem though ? The deadlock for example, can happen only when a backend tries to get a table level conflicting lock while holding the buffer pin and I am not sure if we do that. The

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 8:02 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 06.06.2011 07:12, Robert Haas wrote: I did some further investigation of this.  It appears that more than 99% of the lock manager lwlock traffic that remains with this patch applied has

Re: [HACKERS] Error in PQsetvalue

2011-06-06 Thread Pavel Golub
Hello, guys. You wrote: MM On Fri, Jun 3, 2011 at 10:36 PM, Andrew Chernow a...@esilo.com wrote: On 6/3/2011 10:26 PM, Andrew Chernow wrote: I disagree -- I think the fix is a one-liner. line 446: if (tup_num == res-ntups !res-tuples[tup_num]) should just become if (tup_num == res-ntups)

Re: [HACKERS] WIP: Fast GiST index build

2011-06-06 Thread Alexander Korotkov
On Mon, Jun 6, 2011 at 2:51 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Do you think there's some worst-case data distributions where this algorithm would perform particularly badly? I think there could be some worst-case GiST applications. Just now gist fast build

Re: [HACKERS] WIP: Fast GiST index build

2011-06-06 Thread Alexander Korotkov
On Mon, Jun 6, 2011 at 4:14 PM, Alexander Korotkov aekorot...@gmail.comwrote: If I succeed then it will invoke even more such calls. I meant here that if I succeed in enhancements which improve index quality then fast build algorithm will invoke even more such calls. -- With best regards,

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Heikki Linnakangas
On 06.06.2011 14:59, Robert Haas wrote: BTW, how do you identify from oprofile that *vxid* locks were the problem? I didn't think it could produce that level of detail. It can show the call stack of each call, with --callgraph=n option, where you can see what percentage of the calls to

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Dimitri Fontaine
Jeff Davis pg...@j-davis.com writes: I'd like to take another look at Range Types and whether part of it should be an extension. Some of these issues relate to extensions in general, not just range types. That's a good question :) I think the way things are going to be organised now is that

Re: [HACKERS] [PATCH] Bug in XPATH() if expression returns a scalar value

2011-06-06 Thread Peter Eisentraut
On tis, 2011-05-31 at 16:19 +0200, Florian Pflug wrote: If people deem this to be a problem, we could instead add a separate function XPATH_VALUE() that returns VARCHAR, and make people use that for scalar-value-returning expressions. Why not replicate what contrib/xml2 provides, namely

Re: [HACKERS] Domains versus polymorphic functions, redux

2011-06-06 Thread Peter Eisentraut
On fre, 2011-06-03 at 13:53 -0500, Kevin Grittner wrote: Another long-range nicety would be something which I have seen in some other databases, and which is consistent with the inheritance theme, is that you can't compare or assign dissimilar domains -- an error is thrown. So if you try to

[HACKERS] [v9.2] Fix leaky-view problem, part 1

2011-06-06 Thread Kohei Kaigai
This patch enables to fix up leaky-view problem using functions with tiny cost estimation scenario. The point of this scenario is criteria to reorder qualifiers of scanning plan in order_qual_clauses(). The optimizer may pull up simple subqueries into upper level, then its qualifier will get

Re: [HACKERS] gdb with postgres

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 3:43 AM, HuangQi huangq...@gmail.com wrote: Hi,    I was using gdb to debug postgres. In order to debug the backend of running query, I start postgres first and use select * from pg_backend_pid() to ask for backend pid. Then I start gdb in another bash window with gdb

Re: [HACKERS] gdb with postgres

2011-06-06 Thread HuangQi
If I enter c, gdb will directly finish executing this process and current query will finish. Furthermore, if I enter next query, gdb will not debug it and stay in continue status. On 6 June 2011 21:44, Robert Haas robertmh...@gmail.com wrote: On Mon, Jun 6, 2011 at 3:43 AM, HuangQi

Re: [HACKERS] gdb with postgres

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 9:47 AM, HuangQi huangq...@gmail.com wrote: If I enter c, gdb will directly finish executing this process and current query will finish. Furthermore, if I enter next query, gdb will not debug it and stay in continue status. Hmm, that must mean your breakpoint isn't

Re: [HACKERS] gdb with postgres

2011-06-06 Thread Kevin Grittner
HuangQi huangq...@gmail.com wrote: (gdb) b qp_add_paths_to_joinrel Breakpoint 1 at 0x1a6744: file joinpath.c, line 67. (gdb) attach 23903 If I enter c, gdb will directly finish executing this process and current query will finish. Are you absolutely sure that running your query will

Re: [HACKERS] WIP: AuthenticationMD5 protocol documentation clarification

2011-06-06 Thread Robert Haas
On Sun, Jun 5, 2011 at 11:26 AM, Cyan Ogilvie cyan.ogil...@gmail.com wrote: This is my first patch, so I hope I've got the process right for submitting patches. You're doing great. I suspect we do want to either (1) reword what you've done in English, rather than writing it as code, or at

Re: [HACKERS] BLOB support

2011-06-06 Thread Tom Lane
=?UTF-8?Q?Rados=C5=82aw_Smogura?= rsmog...@softperience.eu writes: I think more about this with contrast to sent references, but I still have in my mind construct Blob myWeddingDvd = conn.createBlob(myWeddingStream, size); //A bit outdated we have BlueRay conn.prepareStatemnt(INSERT

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Simon Riggs
On Mon, Jun 6, 2011 at 11:19 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 06.06.2011 12:40, Simon Riggs wrote: On Sat, Jun 4, 2011 at 5:55 PM, Tom Lanet...@sss.pgh.pa.us  wrote: Simon Riggssi...@2ndquadrant.com  writes: The approach looks sound to me. It's a fairly

[HACKERS] Different execution time for same plan

2011-06-06 Thread Nick Raj
Hi, I am using postgresql 8.4.6. I have made an index on my data-type that is working fine. I mean output is coming properly. When i execute the query first time, query takes a quite longer time but second time execution of the same query takes very less time (despite execution plan is same)

Re: [HACKERS] Domains versus polymorphic functions, redux

2011-06-06 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: What you are looking for is the SQL feature called distinct types. The makers of the SQL standard have sort of deprecated domains in favor of distinct types, because distinct types address your sort of use case better, and prescribing the behavior of

Re: [HACKERS] BLOB support

2011-06-06 Thread Radosław Smogura
Tom Lane t...@sss.pgh.pa.us Monday 06 of June 2011 16:13:26 =?UTF-8?Q?Rados=C5=82aw_Smogura?= rsmog...@softperience.eu writes: I think more about this with contrast to sent references, but I still have in my mind construct Blob myWeddingDvd = conn.createBlob(myWeddingStream, size); //A

Re: [HACKERS] BLOB support

2011-06-06 Thread Tom Lane
=?utf-8?q?Rados=C5=82aw_Smogura?= rsmog...@softperience.eu writes: Introducing streaming for TOAST is little useless, sorry just for cite from my, mentoined document: (This is generally about on demand stream of TOASTed value, in context of LOBs is acceptable, as long not transactional

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Alvaro Herrera
Excerpts from Tom Lane's message of sáb jun 04 12:49:05 -0400 2011: Alvaro Herrera alvhe...@commandprompt.com writes: What surprises me is that the open references remain after a database drop. Surely this means that no backends keep open file descriptors to any table in that database,

Re: [HACKERS] BLOB support

2011-06-06 Thread Heikki Linnakangas
On 06.06.2011 17:13, Tom Lane wrote: =?UTF-8?Q?Rados=C5=82aw_Smogura?=rsmog...@softperience.eu writes: I think more about this with contrast to sent references, but I still have in my mind construct Blob myWeddingDvd = conn.createBlob(myWeddingStream, size); //A bit outdated we have

Re: [HACKERS] Different execution time for same plan

2011-06-06 Thread Kevin Grittner
First off, this is posted to the wrong list -- this list is for discussion of development of the PostgreSQL product. There is a list for performance questions where this belongs: pgsql-performa...@postgresql.org. I'm moving this to the performance list with a blind copy to the -hackers list so

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 10:49 AM, Simon Riggs si...@2ndquadrant.com wrote: My point was that we have in the past implemented performance changes to increase scalability at the last minute, and also that our personal risk perspectives are not always set in stone. Robert has highlighted the

Re: [HACKERS] WIP: AuthenticationMD5 protocol documentation clarification

2011-06-06 Thread Heikki Linnakangas
On 06.06.2011 16:58, Robert Haas wrote: On Sun, Jun 5, 2011 at 11:26 AM, Cyan Ogilviecyan.ogil...@gmail.com wrote: This is my first patch, so I hope I've got the process right for submitting patches. You're doing great. I suspect we do want to either (1) reword what you've done in English,

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: What we found out after more careful investigation is that the file is kept open by a backend connected to a different database. I have a suspicion that what happened here is that this backend was forced to flush out a page from shared

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Alvaro Herrera
Excerpts from Kevin Grittner's message of lun jun 06 11:58:51 -0400 2011: Alvaro Herrera alvhe...@commandprompt.com wrote: What we found out after more careful investigation is that the file is kept open by a backend connected to a different database. I have a suspicion that what

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Kevin Grittner's message of lun jun 06 11:58:51 -0400 2011: Alvaro Herrera alvhe...@commandprompt.com wrote: What we found out after more careful investigation is that the file is kept open by a backend connected to a different

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: IMHO, it's better to just have a deadline, and stuff either makes it or it doesn't. I realize we haven't always adhered to the principle in the past, but at least IMV that's not a mistake we want to continue repeating. +1 I've said it before, but

[HACKERS] WALInsertLock tuning

2011-06-06 Thread Simon Riggs
In earlier discussions of how to improve WALInsertLock contention, it was observed that we must zero each new page before we advance the WAL insertion point. http://postgresql.1045698.n5.nabble.com/Reworking-WAL-locking-td1983647.html IMHO the page zeroing is completely unnecessary, and

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Alvaro Herrera
Excerpts from Tom Lane's message of lun jun 06 12:10:24 -0400 2011: Alvaro Herrera alvhe...@commandprompt.com writes: Hmm interesting. I don't think the placement suggested by Tom would be useful, because the Zabbix backends are particularly busy all the time, so they wouldn't run

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Jeff Davis
On Mon, 2011-06-06 at 06:56 +0200, Pavel Stehule wrote: 2011/6/6 Darren Duncan dar...@darrenduncan.net: Jeff Davis wrote: I'd like to take another look at Range Types and whether part of it should be an extension. Some of these issues relate to extensions in general, not just range

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: That doesn't solve the WAL problem Kevin found, of course ... I wouldn't worry about that too much -- the WAL issue is self-limiting and not likely to ever cause a failure. The biggest risk is that every now and then some new individual will

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Jeff Davis
On Sun, 2011-06-05 at 21:51 -0700, Darren Duncan wrote: Jeff Davis wrote: I'd like to take another look at Range Types and whether part of it should be an extension. Some of these issues relate to extensions in general, not just range types. First of all, what are the advantages to

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Pavel Stehule
2011/6/6 Jeff Davis pg...@j-davis.com: On Mon, 2011-06-06 at 06:56 +0200, Pavel Stehule wrote: 2011/6/6 Darren Duncan dar...@darrenduncan.net: Jeff Davis wrote: I'd like to take another look at Range Types and whether part of it should be an extension. Some of these issues relate to

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of lun jun 06 12:10:24 -0400 2011: Yeah, I wasn't that thrilled with the suggestion either. But we can't just have backends constantly closing every open FD they hold, or performance will suffer. I don't see

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 12:30 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of lun jun 06 12:10:24 -0400 2011: Yeah, I wasn't that thrilled with the suggestion either.  But we can't just have backends constantly closing

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Jeff Davis
On Mon, 2011-06-06 at 14:42 +0200, Dimitri Fontaine wrote: I think the way things are going to be organised now is that we will have core-blessed extensions: don't mix the mechanism and the policy. I like that idea. non-issue if we had a good type interface system (that works on

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Christopher Browne
On Sun, Jun 5, 2011 at 6:59 PM, Jeff Davis pg...@j-davis.com wrote: There might also be some middle ground, where its like the minimalist approach, but with a few very basic constructors and accessors. That would at least make it easier to test, but then to be actually useful (with index

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Jeff Davis
On Mon, 2011-06-06 at 18:28 +0200, Pavel Stehule wrote: we can define a step FOREACH x IN RANGE . BY That wouldn't need any of the range infrastructure at all -- it would be purely syntactic, right? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list

Re: [HACKERS] WALInsertLock tuning

2011-06-06 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: In earlier discussions of how to improve WALInsertLock contention, it was observed that we must zero each new page before we advance the WAL insertion point. http://postgresql.1045698.n5.nabble.com/Reworking-WAL-locking-td1983647.html IMHO the page

Re: [HACKERS] heap vacuum cleanup locks

2011-06-06 Thread Alvaro Herrera
Excerpts from Robert Haas's message of lun jun 06 08:06:10 -0400 2011: But the problem of vacuum stalling out because it can't get the cleanup lock is a very real one. I've seen at least one customer hit this in production, and it was pretty painful. Now, granted, you need some bad

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Jun 6, 2011 at 12:30 PM, Tom Lane t...@sss.pgh.pa.us wrote: On reflection I think this behavior is probably limited to the case where we've done what we used to call a blind write of a block that is unrelated to our database or tables.  For

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Jeff Davis
On Mon, 2011-06-06 at 16:45 +, Christopher Browne wrote: How to slice it apart into an appropriate admixture of core and extensions is a good question, though it seems pretty likely that having an extension for each data type that is to be mixed into a range is a reasonable way to go. ...

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Robert Haas
On Sun, Jun 5, 2011 at 2:59 PM, Jeff Davis pg...@j-davis.com wrote: So, where on this spectrum should range types fall? I think the most minimalist would be to only support #1 (and the necessary type IO functions); and leave all other functions, operators, and opclasses to an extension. That

Re: [HACKERS] heap vacuum cleanup locks

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 12:49 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of lun jun 06 08:06:10 -0400 2011: But the problem of vacuum stalling out because it can't get the cleanup lock is a very real one.  I've seen at least one customer hit this

Re: [HACKERS] patch: Allow \dd to show constraint comments

2011-06-06 Thread Alvaro Herrera
Excerpts from Josh Kupershmidt's message of dom jun 05 16:36:57 -0400 2011: On Tue, May 24, 2011 at 10:31 PM, Josh Kupershmidt schmi...@gmail.com wrote: Attached is a rebased patch. From a quick look, it seems that most of the object types missing from \dd are already covered by pg_comments

Re: [HACKERS] Postmaster holding unlinked files for pg_largeobject table

2011-06-06 Thread Alvaro Herrera
Excerpts from Tom Lane's message of lun jun 06 12:49:46 -0400 2011: Robert Haas robertmh...@gmail.com writes: On Mon, Jun 6, 2011 at 12:30 PM, Tom Lane t...@sss.pgh.pa.us wrote: On reflection I think this behavior is probably limited to the case where we've done what we used to call a blind

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Simon Riggs
On Mon, Jun 6, 2011 at 5:14 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Perhaps the best way to describe the suggestion that this be included in 9.1 isn't that it's an insane suggestion; but that it's a suggestion which, if adopted, would be likely to drive those who are striving

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Pavel Stehule
2011/6/6 Jeff Davis pg...@j-davis.com: On Mon, 2011-06-06 at 18:28 +0200, Pavel Stehule wrote: we can define a step FOREACH x IN RANGE . BY That wouldn't need any of the range infrastructure at all -- it would be purely syntactic, right? I don't think. For lot of types the

Re: [HACKERS] patch: Allow \dd to show constraint comments

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 1:03 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Josh Kupershmidt's message of dom jun 05 16:36:57 -0400 2011: On Tue, May 24, 2011 at 10:31 PM, Josh Kupershmidt schmi...@gmail.com wrote: Attached is a rebased patch. From a quick look, it seems

Re: [HACKERS] BLOB support

2011-06-06 Thread Radoslaw Smogura
I just started with some image as blob works. And I think topic of this will come back. As well many other problems will arise. Flattering tuple, etc. I will send scretches of streaming in this way, I hope, as I want go back to clusterization work. Sorry, for top reply, windows phone 7.

Re: [HACKERS] permissions of external PID file

2011-06-06 Thread David Fetter
On Fri, Jun 03, 2011 at 09:51:45PM +0300, Peter Eisentraut wrote: The external PID file, if configured, is currently generated with 600 permissions, which results from the umask setting in the postmaster. I think it would be nicer if we could make that 644. I have often dealt with scripts

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Ross J. Reedstrom
On Mon, Jun 06, 2011 at 12:53:49PM -0400, Robert Haas wrote: I don't have clear feeling on this question in general, but if we're going to break this up into pieces, it's important that they be logical pieces. Putting half the feature in core and half into an extension just because we can

[HACKERS] heap_hot_search_buffer refactoring

2011-06-06 Thread Robert Haas
The attached patch refactors heap_hot_search_buffer() so that index_getnext() can use it, and modifies index_getnext() to do so. The idea is based on one of Heikki's index-only scan patches, from 2009: http://archives.postgresql.org/pgsql-hackers/2009-07/msg00676.php I believe, however, that

Re: [HACKERS] patch: Allow \dd to show constraint comments

2011-06-06 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Jun 6, 2011 at 1:03 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Hmm, if we're going to have pg_comments as a syntactic sugar kind of thing, it should output things in format immediately useful to the user, i.e. relation/column/etc

Re: [HACKERS] patch: Allow \dd to show constraint comments

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 2:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jun 6, 2011 at 1:03 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Hmm, if we're going to have pg_comments as a syntactic sugar kind of thing, it should output things in

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: That means that this is, IMHO, the right approach. Have core support that enables user defined RANGE types with indexing and planner support, etc, like we have OPERATOR CLASS and FAMILY and all the jazz. Yes, we do, however.. And the useful

Re: [HACKERS] WALInsertLock tuning

2011-06-06 Thread Simon Riggs
On Mon, Jun 6, 2011 at 5:47 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: In earlier discussions of how to improve WALInsertLock contention, it was observed that we must zero each new page before we advance the WAL insertion point.

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Merlin Moncure
On Sun, Jun 5, 2011 at 1:59 PM, Jeff Davis pg...@j-davis.com wrote: In the several talks that I've given, a common question is related to multiranges (ranges with holes). These get a little complex, and I don't have a complete answer. However, multiranges can be approximated with ordered

Re: [HACKERS] Error in PQsetvalue

2011-06-06 Thread Merlin Moncure
On Mon, Jun 6, 2011 at 7:09 AM, Pavel Golub pa...@microolap.com wrote: Sorry for delay in answer. Yeah, I'm glad to. Should I apply this patch by myself? sure, run it against your test case and make sure it works. if so, we can pass it up the chain of command (hopefully as context diff).

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Josh Berkus
That's an improvement of about ~3.5x. According to the vmstat output, when running without the patch, the CPU state was about 40% idle. With the patch, it dropped down to around 6%. Wow! That's fantastic. Jignesh, are you in a position to test any of Robert's work using DBT or other

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: IMHO, it's better to just have a deadline, Well, that's the fine point we're now talking about. I still think that we should try at making the best release possible. And if that means including changes at beta time because that's when someone got

Re: [HACKERS] SAVEPOINTs and COMMIT performance

2011-06-06 Thread Simon Riggs
On Mon, Jun 6, 2011 at 10:33 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 06.02.2011 23:09, Simon Riggs wrote: On Sun, 2011-02-06 at 12:11 -0500, Bruce Momjian wrote: Did this ever get addressed? Patch attached. Seems like the easiest fix I can come up with. @@

Re: [HACKERS] Range Types and extensions

2011-06-06 Thread Dimitri Fontaine
Stephen Frost sfr...@snowman.net writes: I don't like the idea of having a capability which is not utilized in core. We should make it so extensions can *also* have access to define their own, but we should have the basics covered in core. Well if another part of core depends on the feature

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Dave Page
On Mon, Jun 6, 2011 at 8:12 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: So, to the question “do we want hard deadlines?” I think the answer is “no”, to “do we need hard deadlines?”, my answer is still “no”, and to the question “does this very change should be considered this late?” my

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Stefan Kaltenbrunner
On 06/06/2011 09:24 PM, Dave Page wrote: On Mon, Jun 6, 2011 at 8:12 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: So, to the question “do we want hard deadlines?” I think the answer is “no”, to “do we need hard deadlines?”, my answer is still “no”, and to the question “does this very

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Stephen Frost
* Dave Page (dp...@pgadmin.org) wrote: Much as I hate to say it (I too want to keep our schedule as predictable and organised as possible), I have to agree. Assuming the patch is good, I think this is something we should push into 9.1. It really could be a game changer. So, with folks putting

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Dave Page
On Mon, Jun 6, 2011 at 8:40 PM, Stefan Kaltenbrunner ste...@kaltenbrunner.cc wrote: On 06/06/2011 09:24 PM, Dave Page wrote: On Mon, Jun 6, 2011 at 8:12 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: So, to the question “do we want hard deadlines?” I think the answer is “no”, to “do we

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Josh Berkus
On 6/6/11 12:12 PM, Dimitri Fontaine wrote: So, to the question “do we want hard deadlines?” I think the answer is “no”, to “do we need hard deadlines?”, my answer is still “no”, and to the question “does this very change should be considered this late?” my answer is yes. I could not disagree

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Andrew Dunstan
On 06/06/2011 03:24 PM, Dave Page wrote: On Mon, Jun 6, 2011 at 8:12 PM, Dimitri Fontainedimi...@2ndquadrant.fr wrote: So, to the question “do we want hard deadlines?” I think the answer is “no”, to “do we need hard deadlines?”, my answer is still “no”, and to the question “does this very

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Dave Page
On Mon, Jun 6, 2011 at 8:44 PM, Stephen Frost sfr...@snowman.net wrote: * Dave Page (dp...@pgadmin.org) wrote: Much as I hate to say it (I too want to keep our schedule as predictable and organised as possible), I have to agree. Assuming the patch is good, I think this is something we should

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Jignesh Shah
On Mon, Jun 6, 2011 at 2:49 PM, Josh Berkus j...@agliodbs.com wrote: That's an improvement of about ~3.5x.  According to the vmstat output, when running without the patch, the CPU state was about 40% idle. With the patch, it dropped down to around 6%. Wow!  That's fantastic. Jignesh, are

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Christopher Browne
On Mon, Jun 6, 2011 at 5:13 PM, Simon Riggs si...@2ndquadrant.com wrote: The cost to us is a few days work and the benefit is a whole year's worth of increased performance for our user base, which has a hardware equivalent well into the millions of dollars. I doubt that this is an accurate

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Robert Haas
On Mon, Jun 6, 2011 at 3:59 PM, Christopher Browne cbbro...@gmail.com wrote: On Mon, Jun 6, 2011 at 5:13 PM, Simon Riggs si...@2ndquadrant.com wrote: The cost to us is a few days work and the benefit is a whole year's worth of increased performance for our user base, which has a hardware

[HACKERS] contrib/citext versus collations

2011-06-06 Thread Tom Lane
I've been looking into bug #6053, in which Regina Obe complains that hash-based DISTINCT queries fail for type citext. The cause is not far to seek: the header comment for execGrouping.c states * Note: we currently assume that equality and hashing functions are not * collation-sensitive, so

Re: [HACKERS] reducing the overhead of frequent table locks - now, with WIP patch

2011-06-06 Thread Kevin Grittner
Stephen Frost sfr...@snowman.net wrote: if people really want this in, then we need to figure out what the new schedule is going to be. I suggest June, 2012. That way we can get a whole bunch more really cool patches in, and the users won't have to wait for 9.2 to get them. -Kevin --

  1   2   >