Re: [HACKERS] posix_fadvsise in base backups

2011-09-24 Thread Greg Stark
On Sat, Sep 24, 2011 at 4:16 PM, Magnus Hagander wrote: > I was assuming the kernel was smart enough to read this as "*this* > process is not going to be using this file anymore", not "nobody in > the whole machine is going to use this file anymore". And the process > running the base backup is ce

Re: [HACKERS] Hot Backup with rsync fails at pg_clog if under load

2011-09-24 Thread Daniel Farina
On Fri, Sep 23, 2011 at 9:45 AM, Robert Haas wrote: > On Fri, Sep 23, 2011 at 11:43 AM, Aidan Van Dyk wrote: >> On Fri, Sep 23, 2011 at 4:41 AM, Heikki Linnakangas >> wrote: >> Unfortunately, it's impossible, because the error message "Could not read from file "pg_clog/0001" at offset

Re: [HACKERS] [v9.2] Fix Leaky View Problem

2011-09-24 Thread Noah Misch
On Fri, Sep 23, 2011 at 06:25:01PM -0400, Robert Haas wrote: > On Mon, Sep 12, 2011 at 3:31 PM, Kohei KaiGai wrote: > > The Part-1 implements corresponding SQL syntax stuffs which are > > "security_barrier" > > reloption of views, and "LEAKPROOF" option on creation of functions to be > > stored >

Re: [HACKERS] posix_fadvsise in base backups

2011-09-24 Thread Cédric Villemain
2011/9/24 Andres Freund : > Hi, > > On Saturday, September 24, 2011 05:16:48 PM Magnus Hagander wrote: >> On Sat, Sep 24, 2011 at 17:14, Andres Freund wrote: >> > On Saturday, September 24, 2011 05:08:17 PM Magnus Hagander wrote: >> >> Attached patch adds a simple call to posix_fadvise with >> >>

Re: [HACKERS] psql setenv command

2011-09-24 Thread Jeff Janes
On Thu, Sep 15, 2011 at 7:46 AM, Andrew Dunstan wrote: > On Thu, September 15, 2011 10:44 am, Andrew Dunstan wrote: >> >> As discussed, patch attached. >> > > > this time with patch. Hi Andrew, A description of the \setenv command should show up in the output of \?. Should there be a regression

Re: [HACKERS] Satisfy extension dependency by one of multiple extensions

2011-09-24 Thread Joshua Berkus
All, > >> We might want to have a system where an extension can declare that > >> it > >> "provides" capabilites, and then have another extension "require" > >> those > >> capabilities. That would be a neater solution to the case that > >> there are > >> multiple extensions that all provide the sa

Re: [HACKERS] [PATCH] Log crashed backend's query (activity string)

2011-09-24 Thread Marti Raudsepp
On Sat, Sep 24, 2011 at 22:23, Robert Haas wrote: > It's not the reviewer's job to convince Tom of anything in particular, > but I think it's helpful for them to state their opinion, whatever it > may be (agreeing with Tom, disagreeing with Tom, or whatever). My opinion is that this can be made s

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-24 Thread Joshua Berkus
> Since we haven't yet come up with a reasonable way of machine-editing > postgresql.conf, this seems like a fairly serious objection to > getting > rid of recovery.conf. I wonder if there's a way we can work around > that... Well, we *did* actually come up with a reasonable way, but it died und

Re: [HACKERS] [pgsql-advocacy] Unlogged vs. In-Memory

2011-09-24 Thread Dimitri Fontaine
Robert Haas writes: > Basically, for every unlogged table, you get an empty _init fork, and > for every index of an unlogged table, you get an _init fork > initialized to an empty index. The _init forks are copied over the > main forks by the startup process before entering normal running. Let's

Re: [HACKERS] [PATCH] Log crashed backend's query (activity string)

2011-09-24 Thread Robert Haas
On Sat, Sep 24, 2011 at 1:51 PM, Jeff Janes wrote: > It seems like everyone agrees that this feature is wanted, but Tom is > still very much opposed to the general approach to implement it, as > being too dangerous. > Is it the reviewer's job to try to convince him otherwise? It's not the reviewe

Re: [HACKERS] Satisfy extension dependency by one of multiple extensions

2011-09-24 Thread Dimitri Fontaine
Yeb Havinga writes: >> We might want to have a system where an extension can declare that it >> "provides" capabilites, and then have another extension "require" those >> capabilities. That would be a neater solution to the case that there are >> multiple extensions that all provide the same capab

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-24 Thread Simon Riggs
On Sat, Sep 24, 2011 at 6:01 PM, Tom Lane wrote: > Simon Riggs writes: >> ... The time to replace it is now and I >> welcome that day and have already agreed to it. > > Okay, so you do agree that eventually we want to be rid of > recovery.conf?  I think everyone else agrees on that.  But if we ar

Re: [HACKERS] [PATCH] Log crashed backend's query (activity string)

2011-09-24 Thread Jeff Janes
On Fri, Sep 9, 2011 at 2:51 AM, Marti Raudsepp wrote: > On Thu, Sep 8, 2011 at 03:22, Robert Haas wrote: >> On Wed, Sep 7, 2011 at 5:24 PM, Alvaro Herrera >> wrote: >>> I remember we had bugs whereby an encoding conversion would fail, >>> leading to elog trying to report this problem, but this a

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-24 Thread Robert Haas
On Sep 24, 2011, at 1:04 PM, Tom Lane wrote: > I don't exactly buy this argument. If postgresql.conf is hard to > machine-edit, why is recovery.conf any easier? Because you generally just write a brand-new file, without worrying about preserving existing settings. You aren't really editing at a

Re: [HACKERS] Re: memory barriers (was: Yes, WaitLatch is vulnerable to weak-memory-ordering bugs)

2011-09-24 Thread Martijn van Oosterhout
On Sat, Sep 24, 2011 at 12:46:48PM -0400, Robert Haas wrote: > > I found the Linux kernel document on this topic quite readable. I think > > the main lesson here is that processors track data dependancies (other > > than the Alpha apparently), but not control dependancies.  So in the > > example, t

Re: [HACKERS] Large C files

2011-09-24 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Tom Lane wrote: > >> Actually, I believe that the *main* problem with pgrminclude is that > >> it fails to account for combinations of build options other than those > >> that Bruce uses. In the previous go-round, the reason we were still > >> squashing

Re: [HACKERS] Large C files

2011-09-24 Thread Andrew Dunstan
On 09/24/2011 01:10 PM, Peter Geoghegan wrote: On 24 September 2011 16:41, Tom Lane wrote: Frankly, with the tool in its current state I'd rather not run it at all, ever. The value per man-hour expended is too low. The mess it made out of the xlog-related includes this time around makes me

Re: [HACKERS] Adding CORRESPONDING to Set Operations

2011-09-24 Thread Kerem Kat
On Sat, Sep 24, 2011 at 19:51, Tom Lane wrote: > Kerem Kat writes: >> In the parser while analyzing SetOperationStmt, larg and rarg needs to be >> transformed as subqueries. SetOperationStmt can have two fields representing >> larg and rarg with projected columns according to corresponding: >> la

Re: [HACKERS] Large C files

2011-09-24 Thread Peter Geoghegan
On 24 September 2011 16:41, Tom Lane wrote: > Frankly, with the tool in its current state I'd rather not run it at > all, ever.  The value per man-hour expended is too low.  The mess it > made out of the xlog-related includes this time around makes me question > whether it's even a net benefit, re

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-24 Thread Tom Lane
Robert Haas writes: > On Fri, Sep 23, 2011 at 6:55 PM, Tatsuo Ishii wrote: >> I'm not sure what you mean by "not deal with" but part of pgpool-II's >> functionality assumes that we can easily generate recovery.conf. If >> reconf.conf is integrated into postgresql.conf, we need to edit >> postgres

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-24 Thread Tom Lane
Simon Riggs writes: > ... The time to replace it is now and I > welcome that day and have already agreed to it. Okay, so you do agree that eventually we want to be rid of recovery.conf? I think everyone else agrees on that. But if we are going to remove recovery.conf eventually, what is the ben

Re: [HACKERS] Large C files

2011-09-24 Thread Tom Lane
Bruce Momjian writes: > Tom Lane wrote: >> Actually, I believe that the *main* problem with pgrminclude is that >> it fails to account for combinations of build options other than those >> that Bruce uses. In the previous go-round, the reason we were still >> squashing bugs months later is that i

Re: [HACKERS] Re: memory barriers (was: Yes, WaitLatch is vulnerable to weak-memory-ordering bugs)

2011-09-24 Thread Robert Haas
On Sat, Sep 24, 2011 at 9:45 AM, Martijn van Oosterhout wrote: > I think memory accesses are also fantastically expensive, so it's worth > some effort to optimise that. This is definitely true. > I found the Linux kernel document on this topic quite readable. I think > the main lesson here is th

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-24 Thread Robert Haas
On Fri, Sep 23, 2011 at 6:55 PM, Tatsuo Ishii wrote: >>> There are many. Tools I can name include pgpool, 2warm, PITRtools, but >>> there are also various tools from Sun, an IBM reseller I have >>> forgotten the name of, OmniTI and various other backup software >>> providers. Those are just the on

Re: [HACKERS] posix_fadvsise in base backups

2011-09-24 Thread Andres Freund
Hi, On Saturday, September 24, 2011 05:16:48 PM Magnus Hagander wrote: > On Sat, Sep 24, 2011 at 17:14, Andres Freund wrote: > > On Saturday, September 24, 2011 05:08:17 PM Magnus Hagander wrote: > >> Attached patch adds a simple call to posix_fadvise with > >> POSIX_FADV_DONTNEED on all the file

Re: [HACKERS] Adding CORRESPONDING to Set Operations

2011-09-24 Thread Kerem Kat
On Sat, Sep 24, 2011 at 18:49, Tom Lane wrote: > > Kerem Kat writes: > > There is a catch inserting subqueries for corresponding in the planner. > > Parser expects to see equal number of columns in both sides of the > > UNION query. If there is corresponding however we cannot guarantee that. > >

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-24 Thread Robert Haas
On Sat, Sep 24, 2011 at 9:02 AM, Simon Riggs wrote: > The semantics are clear: recovery.conf is read first, then > postgresql.conf. It's easy to implement (1 line of code) and easy to > understand. Eh, well, if you can implement it in one line of code, consider my objection withdrawn. I can't se

Re: [HACKERS] Large C files

2011-09-24 Thread Bruce Momjian
Tom Lane wrote: > Actually, I believe that the *main* problem with pgrminclude is that > it fails to account for combinations of build options other than those > that Bruce uses. In the previous go-round, the reason we were still > squashing bugs months later is that it took that long for people t

Re: [HACKERS] Adding CORRESPONDING to Set Operations

2011-09-24 Thread Tom Lane
Kerem Kat writes: > There is a catch inserting subqueries for corresponding in the planner. > Parser expects to see equal number of columns in both sides of the > UNION query. If there is corresponding however we cannot guarantee that. Well, you certainly need the parse analysis code to be aware

Re: [HACKERS] Large C files

2011-09-24 Thread Tom Lane
Bruce Momjian writes: > Robert Haas wrote: >> I'm also curious to see how much more fallout we're going to see from >> that run. We had a few glitches when it was first done, but it didn't >> seem like they were really all that bad. It might be that we'd be >> better off running pgrminclude a lo

Re: [HACKERS] posix_fadvsise in base backups

2011-09-24 Thread Magnus Hagander
On Sat, Sep 24, 2011 at 17:14, Andres Freund wrote: > Hi, > > On Saturday, September 24, 2011 05:08:17 PM Magnus Hagander wrote: >> Attached patch adds a simple call to posix_fadvise with >> POSIX_FADV_DONTNEED on all the files being read when doing a base >> backup, to help the kernel not to tras

Re: [HACKERS] posix_fadvsise in base backups

2011-09-24 Thread Andres Freund
Hi, On Saturday, September 24, 2011 05:08:17 PM Magnus Hagander wrote: > Attached patch adds a simple call to posix_fadvise with > POSIX_FADV_DONTNEED on all the files being read when doing a base > backup, to help the kernel not to trash the filesystem cache. > > Seems like a simple enough fix -

[HACKERS] posix_fadvsise in base backups

2011-09-24 Thread Magnus Hagander
Attached patch adds a simple call to posix_fadvise with POSIX_FADV_DONTNEED on all the files being read when doing a base backup, to help the kernel not to trash the filesystem cache. Seems like a simple enough fix - in fact, I don't remember why I took it out of the original patch :O Any reason

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-24 Thread Tatsuo Ishii
>> I'm not sure what you mean by "not deal with" but part of pgpool-II's >> functionality assumes that we can easily generate recovery.conf. If >> reconf.conf is integrated into postgresql.conf, we need to edit >> postgresql.conf, which is a little bit harder than generating >> recovery.conf, I thi

Re: [HACKERS] Range Types - symmetric

2011-09-24 Thread Bruce Momjian
Jeff Davis wrote: > On Tue, 2011-09-13 at 12:34 -0400, Christopher Browne wrote: > > > select int4range(5,2); > > > ERROR: range lower bound must be less than or equal to range upper bound > > > > > > Of course, I won't argue this is a bug, but I was wondering if it > > > wouldn't be handy to all

Re: [HACKERS] fix for pg_upgrade

2011-09-24 Thread Bruce Momjian
\panam wrote: > Hi, just tried to upgrade from 9.0 to 9.1 and got this error during > pg_upgrade : > Mismatch of relation id: database "xyz", old relid 465783, new relid 16494 > It seems, I get this error on every table as I got it on another table > (which I did not need and deleted) before as wel

Re: [HACKERS] What Would You Like To Do?

2011-09-24 Thread Andrew Dunstan
On 09/24/2011 09:51 AM, Bruce Momjian wrote: Joshua D. Drake wrote: On 09/13/2011 11:51 AM, Michael Nolan wrote: The ability to restore a table from a backup file to a different table name in the same database and schema. This can be done but agreed it is no

Re: [HACKERS] What Would You Like To Do?

2011-09-24 Thread Bruce Momjian
Joshua D. Drake wrote: > > On 09/13/2011 11:51 AM, Michael Nolan wrote: > > > > > The ability to restore a table from a backup file to a different > > table > > name in the same database and schema. > > > > > > This can be done but agreed it is not intuitive. > > > > >

Re: [HACKERS] Re: memory barriers (was: Yes, WaitLatch is vulnerable to weak-memory-ordering bugs)

2011-09-24 Thread Martijn van Oosterhout
On Fri, Sep 23, 2011 at 04:22:09PM +0100, Greg Stark wrote: > So you have two memory fetches which I guess I still imagine have to > be initiated in the right order but they're both in flight at the same > time. I have no idea how the memory controller works and I could > easily imagine either one

Re: [HACKERS] Large C files

2011-09-24 Thread Bruce Momjian
Robert Haas wrote: > On Fri, Sep 9, 2011 at 11:28 PM, Peter Geoghegan > wrote: > > It's very difficult or impossible to anticipate how effective the tool > > will be in practice, but when you consider that it works and does not > > produce false positives for the first 3 real-world cases tested,

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-24 Thread Simon Riggs
On Fri, Sep 23, 2011 at 6:17 PM, Robert Haas wrote: > On Fri, Sep 23, 2011 at 12:51 PM, Josh Berkus wrote: >> I'm happy to make upgrades easier, but I want a path which eventually >> ends in recovery.conf going away.  It's a bad API, confuses our users, >> and is difficult to support and maintain

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-09-24 Thread Simon Riggs
On Fri, Sep 23, 2011 at 5:51 PM, Josh Berkus wrote: > Simon, > >> There are many. Tools I can name include pgpool, 2warm, PITRtools, but >> there are also various tools from Sun, an IBM reseller I have >> forgotten the name of, OmniTI and various other backup software >> providers. Those are just

Re: [HACKERS] Re: [BUGS] BUG #6189: libpq: sslmode=require verifies server certificate if root.crt is present

2011-09-24 Thread Magnus Hagander
On Fri, Sep 23, 2011 at 16:44, Alvaro Herrera wrote: > > Excerpts from Magnus Hagander's message of vie sep 23 11:31:37 -0300 2011: >> >> On Fri, Sep 23, 2011 at 15:55, Alvaro Herrera >> wrote: > >> > This seems strange to me.  Why not have a second option to let the user >> > indicate the desire

Re: [HACKERS] Large C files

2011-09-24 Thread Hannu Krosing
On Wed, 2011-09-07 at 01:22 +0200, Jan Urbański wrote: > On 07/09/11 01:13, Peter Geoghegan wrote: > > On 6 September 2011 08:29, Peter Eisentraut wrote: > >> I was thinking about splitting up plpython.c, but it's not even on that > >> list. ;-) > > > > IIRC the obesity of that file is something

[HACKERS] TABLE tab completion

2011-09-24 Thread Magnus Hagander
TABLE tab completion in psql only completes to tables, not views. but the TABLE command works fine for both tables and views (and also sequences). Seems we should just complete it to relations and not tables - or can anyone see a particular reason why we shouldn't? Trivial patch attached. --  M

Re: [HACKERS] CUDA Sorting

2011-09-24 Thread Hannu Krosing
On Mon, 2011-09-19 at 10:36 -0400, Greg Smith wrote: > On 09/19/2011 10:12 AM, Greg Stark wrote: > > With the GPU I'm curious to see how well > > it handles multiple processes contending for resources, it might be a > > flashy feature that gets lots of attention but might not really be > > very use

Re: [HACKERS] CUDA Sorting

2011-09-24 Thread Hannu Krosing
On Mon, 2011-09-19 at 15:12 +0100, Greg Stark wrote: > On Mon, Sep 19, 2011 at 1:11 PM, Vitor Reus wrote: > > Since I'm new to pgsql development, I replaced the code of pgsql > > qsort_arg to get used with the way postgres does the sort. The problem > > is that I can't use the qsort_arg_comparator

Re: [HACKERS] Adding CORRESPONDING to Set Operations

2011-09-24 Thread Kerem Kat
I am looking into perpunion.c and analyze.c There is a catch inserting subqueries for corresponding in the planner. Parser expects to see equal number of columns in both sides of the UNION query. If there is corresponding however we cannot guarantee that. Target columns, collations and types for t