Re: [HACKERS] Finer Extension dependencies

2012-04-03 Thread Dimitri Fontaine
Tom Lane writes: >> On Apr 2, 2012, at 11:24 AM, Peter Eisentraut wrote: >>> Or an extension could specify itself which version numbering scheme it >>> uses. This just has to be a reference to a type, which in turn could be >>> semver, debversion, or even just numeric or text (well, maybe name).

Re: [HACKERS] Publish checkpoint timing and sync files summary data to pg_stat_bgwriter

2012-04-03 Thread Robert Haas
On Sat, Mar 31, 2012 at 5:34 PM, Peter Geoghegan wrote: >> Since any backend write is necessarily the result of that backend >> trying to allocate a buffer, I think maybe we should just count >> whether the number of times it was trying to allocate a buffer *using >> a BAS* vs. the number of times

Re: [HACKERS] Command Triggers patch v18

2012-04-03 Thread Robert Haas
On Sun, Apr 1, 2012 at 7:22 AM, Dimitri Fontaine wrote: >> See above example: I am pretty sure you need a stack. > > In next version, certainly. As of now I'm willing to start a new stack > in each command executed in a command trigger. That means 9.2 will only > expose the first level of the stac

Re: [HACKERS] Patch pg_is_in_backup()

2012-04-03 Thread Gilles Darold
Hi Gabriele, Le 31/03/2012 14:25, Gabriele Bartolini a écrit : > Hi Gilles, > >first and foremost, sorry for jumping in this thread so late. I > read all previous discussions and I'd be happy to help you with this > patch. > >> Agreed and sorry for the response delay. I've attached 2 patches >

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-03 Thread Robert Haas
On Mon, Apr 2, 2012 at 5:23 AM, Dave Page wrote: > If homebrew intentionally creates a hole like that, then for as long > as I'm one of the PostgreSQL webmasters it will *never* be listed on > our download pages. I think that's a bit harsh. It's not as if the PostgreSQL package creates the secur

Re: [HACKERS] measuring lwlock-related latency spikes

2012-04-03 Thread Kevin Grittner
> Robert Haas wrote: > Kevin Grittner wrote: > >> I can't help thinking that the "background hinter" I had ideas >> about writing would prevent many of the reads of old CLOG pages, >> taking a lot of pressure off of this area. It just occurred to me >> that the difference between that idea and

Re: [HACKERS] Command Triggers patch v18

2012-04-03 Thread Dimitri Fontaine
Robert Haas writes: > I think we're talking past each other. If someone executes DDL > command A and the command trigger executes DDL command B which fires > another command trigger, then the command trigger for A needs to see > the information relevant to A both before and after the command > tr

[HACKERS] parallel pg_dump

2012-04-03 Thread Andrew Dunstan
I haven't finished reviewing this yet - but there are some things that need to be fixed. First, either the creation of the destination directory needs to be delayed until all the sanity checks have passed and we're sure we're actually going to write something there, or it needs to be removed i

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-04-03 Thread Albe Laurenz
Shigeru HANADA wrote: > Attached are latest version of pgsql_fdw patches. Note that > pgsql_fdw_analyze.patch is only for test the effect of local statistics. > Please apply patches in the order below: > > (1) pgsql_fdw_v18.patch > (2) pgsql_fdw_pushdown_v11.patch > (3) pgsql_fdw_analyze.patch (

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Robert Haas
On Sun, Apr 1, 2012 at 12:35 PM, Joachim Wieland wrote: > On Wed, Mar 28, 2012 at 2:20 PM, Alvaro Herrera > wrote: >> My main comment about the current patch is that it looks like it's >> touching pg_restore parallel code by moving some stuff into parallel.c. >> If that's really the case and its

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Joachim Wieland
On Tue, Apr 3, 2012 at 9:34 AM, Robert Haas wrote: > On Sun, Apr 1, 2012 at 12:35 PM, Joachim Wieland wrote: >> Unfortunately this is not really the case. What is being moved out of >> pg_backup_archiver.c and into parallel.c is either the shutdown logic >> that has been applied only a few days a

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Alvaro Herrera
Excerpts from Joachim Wieland's message of mar abr 03 11:40:31 -0300 2012: > > On Tue, Apr 3, 2012 at 9:34 AM, Robert Haas wrote: > > Hmm.  It looks to me like the part-two patch still contains a bunch of > > code rearrangement.  For example, the current code for > > pg_backup_archiver.c patch

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Robert Haas
On Tue, Apr 3, 2012 at 10:40 AM, Joachim Wieland wrote: >> On a similar note, what's the point of changing struct Archive to have >> int numWorkers instead of int number_of_jobs, and furthermore >> shuffling the declaration around to a different part of the struct? > > number_of_jobs was in the st

[HACKERS] invalid search_path complaints

2012-04-03 Thread Robert Haas
If you use ALTER ROLE/DATABASE SET to configure an invalid search_path, PostgreSQL 9.1 issues a complaint about the invalid setting on each new connection. This is a behavior change relatively to previous releases, which did not. git bisect blames this commit: 2594cf0e8c0440619b1651c5a406d37

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 3, 2012 at 10:40 AM, Joachim Wieland wrote: >> I completely agree. Assertions helped a lot dealing with concurrent >> code. How do you want to tackle this for now? Want me to create a >> separate header pg_assert.h as part of my patch? Or is it okay to >> factor

Re: [HACKERS] invalid search_path complaints

2012-04-03 Thread Tom Lane
Robert Haas writes: > If you use ALTER ROLE/DATABASE SET to configure an invalid > search_path, PostgreSQL 9.1 issues a complaint about the invalid > setting on each new connection. This is a behavior change relatively > to previous releases, which did not. I would say that's an improvement. Do

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Robert Haas
On Tue, Apr 3, 2012 at 11:38 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Apr 3, 2012 at 10:40 AM, Joachim Wieland wrote: >>> I completely agree. Assertions helped a lot dealing with concurrent >>> code. How do you want to tackle this for now? Want me to create a >>> separate header pg_a

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-03 Thread Christopher Browne
On Tue, Apr 3, 2012 at 8:22 AM, Robert Haas wrote: > On Mon, Apr 2, 2012 at 5:23 AM, Dave Page wrote: >> If homebrew intentionally creates a hole like that, then for as long >> as I'm one of the PostgreSQL webmasters it will *never* be listed on >> our download pages. > > I think that's a bit har

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mar abr 03 12:38:20 -0300 2012: > > Robert Haas writes: > > On Tue, Apr 3, 2012 at 10:40 AM, Joachim Wieland wrote: > >> I completely agree. Assertions helped a lot dealing with concurrent > >> code. How do you want to tackle this for now? Want me to create a

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 3, 2012 at 11:38 AM, Tom Lane wrote: >> Possibly we could move assert.c into src/port/ and make it part of >> libpgport? > The trouble is that it calls write_stderr(), which has a non-trivial > implementation on Windows that I don't believe will be suitable for

Re: [HACKERS] invalid search_path complaints

2012-04-03 Thread Robert Haas
On Tue, Apr 3, 2012 at 11:49 AM, Tom Lane wrote: > Robert Haas writes: >> If you use ALTER ROLE/DATABASE SET to configure an invalid >> search_path, PostgreSQL 9.1 issues a complaint about the invalid >> setting on each new connection.  This is a behavior change relatively >> to previous releases

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Robert Haas
On Tue, Apr 3, 2012 at 11:59 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Apr 3, 2012 at 11:38 AM, Tom Lane wrote: >>> Possibly we could move assert.c into src/port/ and make it part of >>> libpgport? > >> The trouble is that it calls write_stderr(), which has a non-trivial >> implementa

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Tom Lane
Alvaro Herrera writes: > That only leaves assert_enabled to be handled. In the backend it lives > in guc.c; what to do about frontend code? There's no mechanism for turning such a switch on or off in most frontend code anyway. I'd think it could just be assumed to be on in the frontend implemen

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 3, 2012 at 11:59 AM, Tom Lane wrote: >> Well, if we don't have a solution to that problem then it's premature >> to propose making Assert available to frontend code.  So my opinion >> is that that idea is too half-baked to be pushing into 9.2 at this >> time.  Le

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-03 Thread Robert Haas
On Tue, Apr 3, 2012 at 11:56 AM, Christopher Browne wrote: > It's pretty typical for MacOS applications to require "enter your > password; I need to su to root to install this!" in plenty of places > where the UI does not actually tell you what is being done as root. > After enough iterations of "

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Robert Haas
On Tue, Apr 3, 2012 at 12:17 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Apr 3, 2012 at 11:59 AM, Tom Lane wrote: >>> Well, if we don't have a solution to that problem then it's premature >>> to propose making Assert available to frontend code.  So my opinion >>> is that that idea is to

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 3, 2012 at 12:17 PM, Tom Lane wrote: >> No, the reason for write_stderr() is that fprintf(stderr) is unreliable >> on Windows.  If memory serves, it can actually crash in some situations. > Dude, we're already doing fprintf(stderr) all over pg_dump. If it's > u

Re: [HACKERS] patch for parallel pg_dump

2012-04-03 Thread Robert Haas
On Tue, Apr 3, 2012 at 12:37 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Apr 3, 2012 at 12:17 PM, Tom Lane wrote: >>> No, the reason for write_stderr() is that fprintf(stderr) is unreliable >>> on Windows.  If memory serves, it can actually crash in some situations. > >> Dude, we're alr

Re: [HACKERS] invalid search_path complaints

2012-04-03 Thread Robert Haas
On Tue, Apr 3, 2012 at 12:05 PM, Robert Haas wrote: > On Tue, Apr 3, 2012 at 11:49 AM, Tom Lane wrote: >> Robert Haas writes: >>> If you use ALTER ROLE/DATABASE SET to configure an invalid >>> search_path, PostgreSQL 9.1 issues a complaint about the invalid >>> setting on each new connection.  T

Re: [HACKERS] invalid search_path complaints

2012-04-03 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 3, 2012 at 12:05 PM, Robert Haas wrote: >> On Tue, Apr 3, 2012 at 11:49 AM, Tom Lane wrote: >>> I would say that's an improvement.  Do you think it isn't? >> It seems like a log spam hazard at high connection rates. [ shrug... ] Failing to report a problem is

Re: [HACKERS] invalid search_path complaints

2012-04-03 Thread Peter Geoghegan
On 3 April 2012 19:16, Tom Lane wrote: > Robert Haas writes: >> On Tue, Apr 3, 2012 at 12:05 PM, Robert Haas wrote: >>> On Tue, Apr 3, 2012 at 11:49 AM, Tom Lane wrote: I would say that's an improvement.  Do you think it isn't? > >>> It seems like a log spam hazard at high connection rates

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-04-03 Thread Marko Kreen
On Sun, Apr 01, 2012 at 07:23:06PM -0400, Tom Lane wrote: > Marko Kreen writes: > > So the proper approach would be to have new API call, designed to > > handle it, and allow early-exit only from there. > > > That would also avoid any breakage of old APIs. Also it would avoid > > any accidental

Re: [HACKERS] invalid search_path complaints

2012-04-03 Thread Robert Haas
On Tue, Apr 3, 2012 at 2:16 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Apr 3, 2012 at 12:05 PM, Robert Haas wrote: >>> On Tue, Apr 3, 2012 at 11:49 AM, Tom Lane wrote: I would say that's an improvement.  Do you think it isn't? > >>> It seems like a log spam hazard at high connect

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-03 Thread Jay Levitt
Robert Haas wrote: On Mon, Apr 2, 2012 at 5:23 AM, Dave Page wrote: If homebrew intentionally creates a hole like that, then for as long as I'm one of the PostgreSQL webmasters it will *never* be listed on our download pages. I think that's a bit harsh. It's not as if the PostgreSQL package

Re: [HACKERS] log chunking broken with large queries under load

2012-04-03 Thread Andrew Dunstan
On 04/02/2012 01:03 PM, Tom Lane wrote: Andrew Dunstan writes: On 04/02/2012 12:44 PM, Tom Lane wrote: You could do something like having a list of pending chunks for each value of (pid mod 256). The length of each such list ought to be plenty short under ordinary circumstances. Yeah, ok,

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-04-03 Thread Tom Lane
Marko Kreen writes: > The fact remains that upper-level code must cooperate with callback. > Why is it useful to hijack PQgetResult() to do so? Because that's the defined communications channel. We're not "hijacking" it. If we're going to start using pejorative words here, I will assert that yo

Re: [HACKERS] invalid search_path complaints

2012-04-03 Thread Tom Lane
Robert Haas writes: > So we have an established precedent that it is right to warn about > things that are sketchy at the time that they are defined, but not > every time they are used. Sure, but we don't have that option available to us here --- or more accurately, ALTER USER/DATABASE SET *does*

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-04-03 Thread Marko Kreen
On Tue, Apr 03, 2012 at 05:32:25PM -0400, Tom Lane wrote: > Marko Kreen writes: > > The fact remains that upper-level code must cooperate with callback. > > Why is it useful to hijack PQgetResult() to do so? > > Because that's the defined communications channel. We're not > "hijacking" it. If w

Re: [HACKERS] HTTP Frontend? (and a brief thought on materialized views)

2012-04-03 Thread Josh Berkus
> While I was doing this I always thought this would have been a better > approach for my previous project, an accounting application. If I could > just have stored entities like invoice & customer as a single document that > is inserted, updated, etc. atomically it would be a lot simpler and fas

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-03 Thread Greg Smith
On 04/01/2012 04:19 PM, Jay Levitt wrote: POSSIBLE OBJECTIONS/PREREQUISITES 10. There is no homebrew support for multiple versions, and no current plans to add it (though it's on the wishlist). This means homebrew is only useful if "I want to install a PostgreSQL thingie" is the common Mac use

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-03 Thread Josh Berkus
On 4/3/12 5:22 AM, Robert Haas wrote: > On Mon, Apr 2, 2012 at 5:23 AM, Dave Page wrote: >> If homebrew intentionally creates a hole like that, then for as long >> as I'm one of the PostgreSQL webmasters it will *never* be listed on >> our download pages. I don't agree. Listed with a warning, sur

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-03 Thread Dave Page
On Tue, Apr 3, 2012 at 7:48 PM, Josh Berkus wrote: > On 4/3/12 5:22 AM, Robert Haas wrote: >> On Mon, Apr 2, 2012 at 5:23 AM, Dave Page wrote: >>> If homebrew intentionally creates a hole like that, then for as long >>> as I'm one of the PostgreSQL webmasters it will *never* be listed on >>> our

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-03 Thread Dave Page
On Tue, Apr 3, 2012 at 6:58 PM, Greg Smith wrote: > > --Documentation > > Homebrew will have to become more complicated if it's going to try and > wander down this path.  With complexity and backward compatibility come > increased needs for documentation. One more to add: --QA When PostgreSQL u

Re: [HACKERS] invalid search_path complaints

2012-04-03 Thread Scott Mead
On Tue, Apr 3, 2012 at 2:37 PM, Tom Lane wrote: > Robert Haas writes: > > So we have an established precedent that it is right to warn about > > things that are sketchy at the time that they are defined, but not > > every time they are used. > > Sure, but we don't have that option available to u

Re: [HACKERS] Switching to Homebrew as recommended Mac install?

2012-04-03 Thread Greg Stark
On Wed, Apr 4, 2012 at 1:19 AM, Dave Page wrote: > then, we're talking about making parts of the filesystem > world-writeable so it doesn't even matter if the user is running as an > admin for a trojan or some other nasty to attack the system. The argument is that a trojan or other nasty doesn't

Re: [HACKERS] performance-test farm

2012-04-03 Thread Greg Smith
On 03/05/2012 05:20 PM, Tomas Vondra wrote: What is the current state of this effort? Is there someone else working on that? If not, I propose this (for starters): * add a new page "Performance results" to the menu, with a list of members that uploaded the perfomance-results * for ea

Re: [HACKERS] checkpoint patches

2012-04-03 Thread Greg Smith
On 03/25/2012 04:29 PM, Jim Nasby wrote: Another $0.02: I don't recall the community using pg_bench much at all to measure latency... I believe it's something fairly new. I point this out because I believe there are differences in analysis that you need to do for TPS vs latency. I think Robert'

[HACKERS] psql: tab completions for 'WITH'

2012-04-03 Thread Josh Kupershmidt
Hi all, I noticed psql's tab-completion for 'WITH' is a bit overeager. If you try to tab-complete commands like: ALTER ROLE jsmith WITH [TAB] COPY tbl FROM 'filename' WITH [TAB] you'll get 'RECURSIVE' unhelpfully filled in. I think 'RECURSIVE' should only be suggested if 'WITH' is the first a

Re: [HACKERS] Faster compression, again

2012-04-03 Thread Huchev
For a C implementation, it could interesting to consider LZ4 algorithm, since it is written natively in this language. In contrast, Snappy has been ported to C by Andy from the original C++ Google code, which lso translate into less extensive usage and tests. http://code.google.com/p/lz4/ Further

[HACKERS] Scalability micro-conference topic proposals (LPC2012)

2012-04-03 Thread Mathieu Desnoyers
Dear Postgresql hackers, We are organizing a micro-conference on scaling both upwards (many cores) and downwards (low footprint, energy efficiency) that targets all layers of the software stack. Our intent is to bring together application, libraries and kernel developers to discuss the scalability