Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-28 Thread Dimitri Fontaine
Craig Ringer cr...@postnewspapers.com.au writes: 1) People preparing statements to save on parse+plan time; and 2) People preparing statements to get convenenient param placement. I suspect that most of (1) also want (2), but many of (2) don't care much about (1) and are just preparing

Re: [HACKERS] Anyone know if Alvaro is OK?

2010-02-28 Thread Devrim GÜNDÜZ
I'm happy that you and your family are fine, Alvaro. Same from my wife, too. On Sat, 2010-02-27 at 19:45 -0800, Alvaro Herrera wrote: Hi. We're out of town right now, and it seems I can't get to my home machine (probably just a loose cable). Our building was shaken badly enough that we'll

Re: [HACKERS] Anyone know if Alvaro is OK?

2010-02-28 Thread Boszormenyi Zoltan
It must be that someone has dropped a small amount of cheese into a lactose-intolerant volcano god... Marc G. Fournier írta: Is there a higher then normal amount of earthquakes happening recently? haiti, japan just had one for 6.9, there was apparently one in illinos a few weeks back, one on

[HACKERS] dedebugging and a functions that just don't work on debian flavour

2010-02-28 Thread Ivan Sergio Borgonovo
I wrote a function that turns tsvectors into tsqueries. I can't see any evident error and actually the function return the expected results but just for a strange set of queries. Otherwise it causes tsqueryout to exit with errors when the result is returned. It seems that tsqueryout is running

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Greg Stark
On Sun, Feb 28, 2010 at 6:07 AM, Greg Smith g...@2ndquadrant.com wrote: Not forced to--have the option of.  There are obviously workloads where you wouldn't want this.  At the same time, I think there are some pretty common ones people are going to expect HS+SR to work on transparently where

Re: [HACKERS] A thought on Index Organized Tables

2010-02-28 Thread Greg Stark
On Sun, Feb 28, 2010 at 6:02 AM, Gokulakannan Somasundaram gokul...@gmail.com wrote: So just with a addition of 8 bytes per tuple, we can have the snapshot stored with the index. Can someone please comment on this? The transaction information on tuples take 18 bytes plus several info bits. It's

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-28 Thread Greg Stark
On Fri, Feb 26, 2010 at 4:01 AM, Robert Haas robertmh...@gmail.com wrote: It's not going to be easier to implement.  Yeah, it would be easy to provide a global switch via a GUC setting, but that's not going to be helpful, because this is the sort of thing that really needs to be managed

Re: [HACKERS] [GENERAL] trouble with to_char('L')

2010-02-28 Thread Bruce Momjian
Bruce Momjian wrote: Hiroshi Inoue wrote: Bruce Momjian wrote: Hiroshi Inoue wrote: Bruce Momjian wrote: Where are we on this issue? Oops I forgot it completely. I have a little improved version and would post it tonight. Ah, very good. Thanks. Attached is an

Re: [HACKERS] A thought on Index Organized Tables

2010-02-28 Thread Tom Lane
Gokulakannan Somasundaram gokul...@gmail.com writes: a) We are already going from table to index to do unique checks. This is the same thing, which we will do to go and update the snapshot in the indexes. No, it is not the same thing. Updating index snapshots requires being able to *re-find* a

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Joachim Wieland
On Sun, Feb 28, 2010 at 2:54 PM, Greg Stark gsst...@mit.edu wrote: Really? I think we get lots of suprised wows from the field from the idea that a long-running read-only query can cause your database to bloat. I think the only reason that's obvious to us is that we've been grappling with that

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-28 Thread Greg Stark
On Tue, Feb 23, 2010 at 3:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: The plan I was thinking of was to pass a flag indicating if it's a directory to fsync_fname() and open it RD_ONLY if it's a directory and RDRW if it's a file. Then ignore any error returns (or at least EBADF and EINVAL) iff

[HACKERS] Simple fix for contrib/xml2

2010-02-28 Thread Tom Lane
I'm beginning to think nobody is going to step up and fix contrib/xml2, so I had a look at it myself. It strikes me that there is a pretty simple fix for it: just get rid of the attempt to manage libxml memory via palloc, and don't do anything else. The implication of this will be that if an

Re: [HACKERS] [COMMITTERS] Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-28 Thread Tom Lane
Greg Stark st...@mit.edu writes: So I'm thinking of something like this. Ignore ESDIR when opening a directory (and return immediately) and ignore EBADF when trying to fsync a directory. Seems reasonable, but get rid of the comment However we can't do this just yet, it has portability issues;

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Josh Berkus
All, First, from the nature of the arguments, we need to eventually have both versions of SR: delay-based and xmin-pub. And it would be fantastic if Greg Smith and Tom Lane could work on xmin-pub to see if we can get it ready as well. I also think, based on the discussion and Greg's test case,

Re: [HACKERS] Simple fix for contrib/xml2

2010-02-28 Thread Bruce Momjian
Tom Lane wrote: I'm beginning to think nobody is going to step up and fix contrib/xml2, so I had a look at it myself. It strikes me that there is a pretty simple fix for it: just get rid of the attempt to manage libxml memory via palloc, and don't do anything else. The implication of this

Re: [HACKERS] Anyone know if Alvaro is OK?

2010-02-28 Thread Ron Mayer
Jaime Casanova wrote: At Saturday, 02/27/2010 on 4:21 pm Marc G. Fournier scra...@hub.org wrote: On Sat, Feb 27, 2010 at 10:45 PM, Alvaro Herrera alvhe...@alvh.no-ip.org wrote: Is there a higher then normal amount of earthquakes happening recently? Re: the more frequent earthquakes, yeah I

Re: [HACKERS] Simple fix for contrib/xml2

2010-02-28 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: FYI, I created a web page of all the open xml items: http://momjian.us/cgi-bin/pgsql/mbox?xml Oh, thanks, I was just about to go trawling through the archives to verify all the reported failure cases really do get fixed this way. I don't think you

Re: [HACKERS] contrib/xml2 pfree bug

2010-02-28 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Didn't we just clean up a mess in our XML handling to do with memory handlers? It looks like contrib/xml2 might have similar problems. BTW, I couldn't duplicate this because I don't know what cb_ob_invoice_xml(1,1) refers to. Can you provide a

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Greg Smith
Joachim Wieland wrote: Instead, I assume that most people who will grab 9.0 and use HS+SR do already have a database with a certain query profile. Now with HS+SR they will try to put the most costly and longest read-only queries to the standby but in the end will run the same number of queries

[HACKERS] full text search index scan query plan changed in 8.4.2?

2010-02-28 Thread xu fei
Hi, hackers: I am testing the index used by full text search recently. I have install 8.3.9 and 8.4.2 separately. In 8.3.9, the query plan is like: postgres=# explain SELECT s.name as source , t.name as target FROM element as s, element as t WHERE to_tsvector('testcfg',s.name)  @@

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Greg Smith
Josh Berkus wrote: First, from the nature of the arguments, we need to eventually have both versions of SR: delay-based and xmin-pub. And it would be fantastic if Greg Smith and Tom Lane could work on xmin-pub to see if we can get it ready as well. As I see it, the main technical obstacle

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Dimitri Fontaine
Josh Berkus j...@agliodbs.com writes: 2) A more usable vacuum_defer_cleanup_age. If it was feasible for a user to configure the master to not vacuum records less than, say, 5 minutes dead, then that would again offer the choice to the user of slightly degraded performance on the master

Re: [HACKERS] contrib/xml2 pfree bug

2010-02-28 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Didn't we just clean up a mess in our XML handling to do with memory handlers? It looks like contrib/xml2 might have similar problems. BTW, I couldn't duplicate this because I don't know what cb_ob_invoice_xml(1,1) refers

Re: [HACKERS] contrib/xml2 pfree bug

2010-02-28 Thread Andrew Dunstan
Andrew Dunstan wrote: Almost any XML will do for the first param. e.g.: It looks like you need to make sure the XML library is called first to induce the crash, so before doing what's below, do: select query_to_xml('select 1 as x',true,false,''): select xslt_process(

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Joachim Wieland
On Sun, Feb 28, 2010 at 8:47 PM, Josh Berkus j...@agliodbs.com wrote: 1) Automated retry of cancelled queries on the slave.  I have no idea how hard this would be to implement, but it makes the difference between writing lots of exception-handling code for slave connections (unacceptable) to

Re: [HACKERS] remove contrib/xml2

2010-02-28 Thread Tom Lane
I believe I have fixed all the reported crashes in contrib/xml2. However there is still this issue pointed out by Robert: CREATE TABLE xpath_test (id integer NOT NULL, t xml); INSERT INTO xpath_test VALUES (1, 'rowlistrow a=1/row a=2 b=oops//rowlist'); SELECT * FROM xpath_table('id', 't',

Re: [HACKERS] Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Simon Riggs
On Fri, 2010-02-26 at 16:44 -0500, Tom Lane wrote: Greg Stark gsst...@mit.edu writes: On Fri, Feb 26, 2010 at 9:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: There's *definitely* not going to be enough information in the WAL stream coming from a master that doesn't think it has HS slaves. We

Re: [HACKERS] Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Simon Riggs
On Fri, 2010-02-26 at 03:33 -0500, Greg Smith wrote: I really hope this discussion can say focused on if and how it's possible to improve this area, with the goal being to deliver a product everyone can be proud of with the full feature set that makes this next release a killer one. The

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Josh Berkus
Greg, Joachim, As I see it, the main technical obstacle here is that a subset of a feature already on the SR roadmap needs to get built earlier than expected to pull this off. I don't know about Tom, but I have no expectation it's possible for me to get up to speed on that code fast enough

Re: [HACKERS] remove contrib/xml2

2010-02-28 Thread Andrew Dunstan
Tom Lane wrote: I believe I have fixed all the reported crashes in contrib/xml2. Yay! Well done! That at least removes any possibly urgency about removing the module. However there is still this issue pointed out by Robert: CREATE TABLE xpath_test (id integer NOT NULL, t xml);

[HACKERS] contrib/xml2 vs core xml in 8.3

2010-02-28 Thread Tom Lane
So I started to work on back-patching the no-pallocs change in xml2. It seems to work fine in 8.4, but it crashes in 8.3. The reason is that we never back-patched the 8.4 change to make utils/adt/xml.c not change the libxml2 memory allocation functions. That means that if you try to intermix

Re: [HACKERS] contrib/xml2 vs core xml in 8.3

2010-02-28 Thread David Fetter
On Sun, Feb 28, 2010 at 06:57:33PM -0500, Tom Lane wrote: So I started to work on back-patching the no-pallocs change in xml2. It seems to work fine in 8.4, but it crashes in 8.3. The reason is that we never back-patched the 8.4 change to make utils/adt/xml.c not change the libxml2 memory

[HACKERS] contrib/xml2 vs core xml, more issues

2010-02-28 Thread Tom Lane
Some other things struck me on further code-reading. These are not problems if core xml functions and contrib/xml2 functions are used independently, but they would be issues if one is called within the other --- and both modules have functions that invoke arbitrary SQL inside their execution, so

[HACKERS] Could we do pgindent on just utils/adt/xml.c in the 8.3 branch?

2010-02-28 Thread Tom Lane
I just committed a backpatch of http://archives.postgresql.org/pgsql-committers/2009-05/msg00229.php to 8.3. As in that patch, I did not try to manually reindent the code that's now wrapped in PG_TRY blocks. It might be a good idea to run pgindent on just that one file so that it's

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-28 Thread Robert Haas
On Sat, Feb 27, 2010 at 11:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Feb 26, 2010 at 7:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Wouldn't it be better if it just did the right thing automatically? The sort of heuristic I'm envisioning would

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-28 Thread Robert Haas
On Sun, Feb 28, 2010 at 2:52 AM, Mark Mielke m...@mark.mielke.cc wrote: On 02/27/2010 11:20 PM, Craig Ringer wrote: Essentially, you have: 1) People preparing statements to save on parse+plan time; and 2) People preparing statements to get convenenient param placement. I suspect that most

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Greg Smith
Josh Berkus wrote: Well, we could throw this on the user if we could get them some information on how to calculate that number. For example, some way for them to calculate the number of XIDs per minute via a query, and then set vacuum_defer_cleanup_age appropriately on the master. Sure, it's

Re: [HACKERS] Re: Hot Standby query cancellation and Streaming Replication integration

2010-02-28 Thread Robert Haas
On Sun, Feb 28, 2010 at 5:38 PM, Josh Berkus j...@agliodbs.com wrote: Greg, Joachim, As I see it, the main technical obstacle here is that a subset of a feature already on the SR roadmap needs to get built earlier than expected to pull this off.  I don't know about Tom, but I have no

[HACKERS] Re: Could we do pgindent on just utils/adt/xml.c in the 8.3 branch?

2010-02-28 Thread Bruce Momjian
Tom Lane wrote: I just committed a backpatch of http://archives.postgresql.org/pgsql-committers/2009-05/msg00229.php to 8.3. As in that patch, I did not try to manually reindent the code that's now wrapped in PG_TRY blocks. It might be a good idea to run pgindent on just that one file so

Re: [HACKERS] Lock Wait Statistics (next commitfest)

2010-02-28 Thread Robert Haas
On Sun, Feb 28, 2010 at 1:06 AM, Mark Kirkwood mark.kirkw...@catalyst.net.nz wrote: Robert Haas wrote: This might be my fault, so I apologize for killing your enthusiasm.  I think when I get wrapped up in a CommitFest (and especially during the second half) I get wound up in determining

Re: [HACKERS] Could we do pgindent on just utils/adt/xml.c in the 8.3 branch?

2010-02-28 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Sure, I can easily do that, but I am not sure how to generate a typedef list for that release, except to use the old typedef collection method for that file. Is that OK? One of the things I noticed while patching was that the 8.3 typedef list was missing

Re: [HACKERS] dedebugging and a functions that just don't work on debian flavour

2010-02-28 Thread Robert Haas
On Sun, Feb 28, 2010 at 8:43 AM, Ivan Sergio Borgonovo m...@webthatworks.it wrote: I've no idea what to do next other than asking if someone can give a look to the code and check my comprehension of what a tsquery should be in memory. Have you tried valgrind? ...Robert -- Sent via

Re: [HACKERS] remove contrib/xml2

2010-02-28 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: ... The reason for that behavior is that xpath_table runs through the XPATH_NODESET results generated by the various XPaths and dumps the k'th one of each into the k'th output row generated for the current input row. Damn that's

Re: [HACKERS] psql with Function Type in \df

2010-02-28 Thread David Fetter
On Thu, Feb 25, 2010 at 07:20:58PM -0500, Bruce Momjian wrote: Did we ever get tab completion support for these backslash commands? Nope :/ Not sure if I'll be able to get to it this week, either. Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778

Re: [HACKERS] A thought on Index Organized Tables

2010-02-28 Thread Gokulakannan Somasundaram
No, it is not the same thing. Updating index snapshots requires being able to *re-find* a previously made index entry for the current row. And it has to be done 100% reliably. The worst that happens if an index entry is not found when it should be during a uniqueness check is that the

Re: [HACKERS] A thought on Index Organized Tables

2010-02-28 Thread Gokulakannan Somasundaram
The transaction information on tuples take 18 bytes plus several info bits. It's possible just storing a subset of that would be useful but it's unclear. And I think it would complicate the code if it had to sometimes fetch the heap tuple to get the rest and sometimes doesn't. Visibility