[HACKERS] Re: Review of "pg_basebackup and pg_receivexlog to use non-blocking socket communication", was: Re: Re: [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2013-01-15 Thread Abhijit Menon-Sen
Hi. This patch was marked "Needs review" with no reviewers in the ongoing CF, so I decided to take a look at it. I see that Zoltan has posted a review, so I've added him to the list. But I took a look at the latest patch in any case. Here are some comments, mostly cosmetic ones. > diff -dcrpN po

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Tom Lane
Alvaro Herrera writes: > There are still 34 items needing attention in CF3. I suggest that, if > you have some spare time, your help would be very much appreciated > there. The commitfest that started on Jan 15th has 65 extra items. > Anything currently listed in CF3 can rightfully be considered

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Tom Lane
It's a compiler bug. icc 11.1 apparently thinks that this loop in doPickSplit: /* * Update nodes[] array to point into the newly formed innerTuple, so that * we can adjust their downlinks below. */ SGITITERATE(innerTuple, i, node) { nodes[i] = node; } is go

Re: [HACKERS] pg_dump transaction's read-only mode

2013-01-15 Thread Pavan Deolasee
On Wed, Jan 9, 2013 at 6:42 AM, Gurjeet Singh wrote: > >> > I have updated the commitfest submission to link to the correct patch > email. > > Thanks Gurjeet. > I initially thought that this patch deserves accompanying documentation > because pg_dump's serializable transaction may error out beca

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Jeff Janes
On Tuesday, January 15, 2013, Simon Riggs wrote: > On 15 January 2013 22:55, Bruce Momjian > > wrote: > > >> Why is this being discussed now? > > > > It is for 9.4 and will take months. I didn't think there was a better > > time. We don't usually discuss features during beta testing. > > Bruce,

Re: [HACKERS] string escaping in tutorial/syscat.source

2013-01-15 Thread Josh Kupershmidt
On Tue, Jan 15, 2013 at 6:35 PM, Jeff Janes wrote: > Do you propose back-patching this? You could argue that this is a bug in > 9.1 and 9.2. Before that, they generate deprecation warnings, but do not > give the wrong answer. I think that backpatching to 9.1 would be reasonable, though I won'

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Alvaro Herrera
Bruce Momjian escribió: > On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote: > > > > > > On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote: > > > > Claudio, Stephen, > > > > It really seems like the areas where we could get the most "bang for the > > buck" in paralle

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Claudio Freire
On Wed, Jan 16, 2013 at 12:55 AM, Stephen Frost wrote: >> If memory serves me correctly (and it does, I suffered it a lot), the >> performance hit is quite considerable. Enough to make it "a lot worse" >> rather than "not as good". > > I feel like we must not be communicating very well. > > If the

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Michael Paquier
On Wed, Jan 16, 2013 at 1:32 PM, Bruce Momjian wrote: > On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote: > > > > > > On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote: > > > > Claudio, Stephen, > > > > It really seems like the areas where we could get the most "bang for

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote: > > > On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote: > > Claudio, Stephen, > > It really seems like the areas where we could get the most "bang for the > buck" in parallelism would be: > > 1. Parallel sort >

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Michael Paquier
On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote: > Claudio, Stephen, > > It really seems like the areas where we could get the most "bang for the > buck" in parallelism would be: > > 1. Parallel sort > 2. Parallel aggregation (for commutative aggregates) > 3. Parallel nested loop join (especia

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Josh Berkus
Claudio, Stephen, It really seems like the areas where we could get the most "bang for the buck" in parallelism would be: 1. Parallel sort 2. Parallel aggregation (for commutative aggregates) 3. Parallel nested loop join (especially for expression joins, like GIS) -- Josh Berkus PostgreSQL Exp

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: > Actually, thanks to much faster storage (think SSD, SAN), it's easily > possible for PostgreSQL to become CPU-limited on a seq scan query, even > when reading from disk. Particularly with a complex filter being applied or if it's feeding into something ab

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Josh Berkus
>> but there will be >> quite a few cases where it's much, much better. > > Just cached segments. Actually, thanks to much faster storage (think SSD, SAN), it's easily possible for PostgreSQL to become CPU-limited on a seq scan query, even when reading from disk. -- Josh Berkus PostgreSQL Expe

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Stephen Frost
* Claudio Freire (klaussfre...@gmail.com) wrote: > On Wed, Jan 16, 2013 at 12:13 AM, Stephen Frost wrote: > > Sequentially scanning the *same* data over and over is certainly > > counterprouctive. Synchroscans fixed that, yes. That's not what we're > > talking about though- we're talking about s

Re: [HACKERS] log_lock_waits to identify transaction's relation

2013-01-15 Thread Stephen Frost
Simon, * Simon Riggs (si...@2ndquadrant.com) wrote: > Attached patch passes through further information about the lock wait, > so we can display the following message instead >LOG: process %d acquired %s on transaction %u on relation %u of > database %u after %ld.%03d ms I love this idea. Pl

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Claudio Freire
On Wed, Jan 16, 2013 at 12:13 AM, Stephen Frost wrote: > * Claudio Freire (klaussfre...@gmail.com) wrote: >> On Tue, Jan 15, 2013 at 8:19 PM, Bruce Momjian wrote: >> > The 1GB idea is interesting. I found in pg_upgrade that file copy would >> > just overwhelm the I/O channel, and that doing mult

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Stephen Frost
* Claudio Freire (klaussfre...@gmail.com) wrote: > On Tue, Jan 15, 2013 at 8:19 PM, Bruce Momjian wrote: > > The 1GB idea is interesting. I found in pg_upgrade that file copy would > > just overwhelm the I/O channel, and that doing multiple copies on the > > same device had no win, but those were

Re: [HACKERS] transforms

2013-01-15 Thread Peter Eisentraut
S X behave like other platforms in this regard. There might be other portability issues on other platforms. pg-transforms-20130115.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Claudio Freire
On Tue, Jan 15, 2013 at 7:46 PM, Tom Lane wrote: >> Compressing every small packet seems like it'd be overkill and might >> surprise people by actually reducing performance in the case of lots of >> small requests. > > Yeah, proper selection and integration of a compression method would be > criti

[HACKERS] log_lock_waits to identify transaction's relation

2013-01-15 Thread Simon Riggs
When there is contention between concurrent transactions it shows up as with log_lock_waits as LOG: process %d acquired %s on %s transaction %u after %ld.%03d ms Which is mostly useless for identifying and eliminating the contention in the application since you can't tell which table is causin

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Claudio Freire
On Tue, Jan 15, 2013 at 8:19 PM, Bruce Momjian wrote: >> Given our row-based storage architecture, I can't imagine we'd do >> anything other than take a row-based approach to this.. I would think >> we'd do two things: parallelize based on partitioning, and parallelize >> seqscan's across the ind

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 09:11:20AM +0900, Michael Paquier wrote: > > > On Wed, Jan 16, 2013 at 7:14 AM, Bruce Momjian wrote: > > I mentioned last year that I wanted to start working on parallelism: > > https://wiki.postgresql.org/wiki/Parallel_Query_Execution > > Years ago

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Tom Lane
Andres Freund writes: > -O0 passes Grumble... suspect we're chasing another compiler bug now, but ... You might try -O1; if that shows the bug it'll probably be a tad easier to debug in. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Andres Freund
On 2013-01-16 02:34:52 +0100, Andres Freund wrote: > On 2013-01-16 02:13:26 +0100, Andres Freund wrote: > > On 2013-01-15 19:56:52 -0500, Tom Lane wrote: > > > Andres Freund writes: > > > > FWIW its also triggerable if two other function calls are places inside > > > > the above if() (I tried fpri

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Andres Freund
On 2013-01-15 20:32:00 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2013-01-15 19:56:52 -0500, Tom Lane wrote: > >> At this point I'm more interested in his report in > >> about > >> the Assert at spgdoinsert.c:1222 failing. That's pretty new code, so > >> more likely to have a genuine

[HACKERS] string escaping in tutorial/syscat.source

2013-01-15 Thread Jeff Janes
On Sun, Oct 14, 2012 at 8:53 PM, Josh Kupershmidt > wrote: > Hi all, > It seems the queries in ./src/tutorial/syscat.source use string > escaping with the assumption that standard_conforming_strings is off, > and thus give wrong results with modern versions. A simple fix is > attached. Hi Josh, D

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Andres Freund
On 2013-01-16 02:13:26 +0100, Andres Freund wrote: > On 2013-01-15 19:56:52 -0500, Tom Lane wrote: > > Andres Freund writes: > > > FWIW its also triggerable if two other function calls are places inside > > > the above if() (I tried fprintf(stderr, "argh") and kill(0, 0)). > > > > [ confused... ]

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Tom Lane
Andres Freund writes: > On 2013-01-15 19:56:52 -0500, Tom Lane wrote: >> At this point I'm more interested in his report in >> about >> the Assert at spgdoinsert.c:1222 failing. That's pretty new code, so >> more likely to have a genuine bug, and I wonder if it's related to >> the spgist issue i

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Andres Freund
On 2013-01-15 19:56:52 -0500, Tom Lane wrote: > Andres Freund writes: > > FWIW its also triggerable if two other function calls are places inside > > the above if() (I tried fprintf(stderr, "argh") and kill(0, 0)). > > [ confused... ] You mean replacing the abort() in the elog macro with > one o

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Tom Lane
Andres Freund writes: > FWIW its also triggerable if two other function calls are places inside > the above if() (I tried fprintf(stderr, "argh") and kill(0, 0)). [ confused... ] You mean replacing the abort() in the elog macro with one of these functions? Or something else? > It seems the cha

Re: [HACKERS] Enabling Checksums

2013-01-15 Thread Greg Smith
First rev of a simple corruption program is attached, in very C-ish Python. The parameters I settled on are to accept a relation name, byte offset, byte value, and what sort of operation to do: overwrite, AND, OR, XOR. I like XOR here because you can fix it just by running the program again.

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Michael Paquier
On Wed, Jan 16, 2013 at 7:14 AM, Bruce Momjian wrote: > I mentioned last year that I wanted to start working on parallelism: > > https://wiki.postgresql.org/wiki/Parallel_Query_Execution > > Years ago I added thread-safety to libpq. Recently I added two parallel > execution paths to pg_u

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Andres Freund
On 2013-01-16 00:26:01 +0100, Andres Freund wrote: > On 2013-01-15 17:56:40 -0500, Tom Lane wrote: > > Andres Freund writes: > > > I played a bit arround (thanks Sergey!) and it seems to be some rather > > > strange optimization issue around the fsync request queue. > > > > > Namely changing > >

[HACKERS] Patch: log level change in gistsplit.c

2013-01-15 Thread Josh Hansen
Hello all, We've been seeing these same messages flying by our logs for years: 2012-12-13 22:28:07 GMT LOG: picksplit method for column 2 of index "xxx_idx" doesn't support secondary split I see from the linked message that Tom Lane made the recommendation of changing the loglevel

Re: [HACKERS] pkg-config files for libpq and ecpg

2013-01-15 Thread Tom Lane
Peter Eisentraut writes: > I'll take another stab at providing pkg-config files for the client-side > libraries. This bit: > + echo 'Libs.private: $(filter-out > $(PKG_CONFIG_REQUIRES_PRIVATE:lib%=-l%),$(filter-out -L..%, $(SHLIB_LINK)))' > >>$@ appears to assume that SHLIB_LINK contains

Re: [HACKERS] pg_ctl idempotent option

2013-01-15 Thread Michael Paquier
On Wed, Jan 16, 2013 at 4:29 AM, Bruce Momjian wrote: > That's Japanese for idempotent. ;-) LOL > +1. -- Michael Paquier http://michael.otacoo.com

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Andrew Dunstan
On 01/15/2013 06:22 PM, Tom Lane wrote: Stephen Frost writes: Perhaps I'm taking a bit too narrow view of the world, but my thinking is OLTP won't want things compressed, as it increases latency of requests, while OLAP users are operating with enough data that they'll go through the effort to

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Andres Freund
On 2013-01-15 17:56:40 -0500, Tom Lane wrote: > Andres Freund writes: > > I played a bit arround (thanks Sergey!) and it seems to be some rather > > strange optimization issue around the fsync request queue. > > > Namely changing > > request->rnode = rnode; > > into > > request->rnode.sp

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Tom Lane
Stephen Frost writes: > Perhaps I'm taking a bit too narrow view of the world, but my thinking > is OLTP won't want things compressed, as it increases latency of > requests, while OLAP users are operating with enough data that they'll > go through the effort to use COPY. Also, if there are so man

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Bruce Momjian
On Tue, Jan 15, 2013 at 06:15:57PM -0500, Stephen Frost wrote: > * Gavin Flower (gavinflo...@archidevsys.co.nz) wrote: > > How about being aware of multiple spindles - so if the requested > > data covers multiple spindles, then data could be extracted in > > parallel. This may, or may not, involve

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Really? Given that libpq provides no useful support for doing anything >> with COPY data, much less higher-level packages such as Perl DBI, I'd >> venture that the real-world ratio is more like 90/10. If not 99/1. > Perhaps I'm t

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Stephen Frost
* Gavin Flower (gavinflo...@archidevsys.co.nz) wrote: > How about being aware of multiple spindles - so if the requested > data covers multiple spindles, then data could be extracted in > parallel. This may, or may not, involve multiple I/O channels? Yes, this should dovetail with partitioning and

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 12:03:50PM +1300, Gavin Flower wrote: > On 16/01/13 11:14, Bruce Momjian wrote: > > I mentioned last year that I wanted to start working on parallelism: > > https://wiki.postgresql.org/wiki/Parallel_Query_Execution > > Years ago I added thread-safety t

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Really? Given that libpq provides no useful support for doing anything > with COPY data, much less higher-level packages such as Perl DBI, I'd > venture that the real-world ratio is more like 90/10. If not 99/1. Perhaps I'm taking a bit too narrow view of

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Bruce Momjian
On Tue, Jan 15, 2013 at 11:01:04PM +, Simon Riggs wrote: > On 15 January 2013 22:55, Bruce Momjian wrote: > > >> Why is this being discussed now? > > > > It is for 9.4 and will take months. I didn't think there was a better > > time. We don't usually discuss features during beta testing. >

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Gavin Flower
On 16/01/13 11:14, Bruce Momjian wrote: I mentioned last year that I wanted to start working on parallelism: https://wiki.postgresql.org/wiki/Parallel_Query_Execution Years ago I added thread-safety to libpq. Recently I added two parallel execution paths to pg_upgrade. The first paral

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Simon Riggs
On 15 January 2013 22:55, Bruce Momjian wrote: >> Why is this being discussed now? > > It is for 9.4 and will take months. I didn't think there was a better > time. We don't usually discuss features during beta testing. Bruce, there are many, many patches on the queue. How will we ever get to

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Tom Lane
Andres Freund writes: > I played a bit arround (thanks Sergey!) and it seems to be some rather > strange optimization issue around the fsync request queue. > Namely changing > request->rnode = rnode; > into > request->rnode.spcNode = rnode.spcNode; > request->rnode.dbNode = rno

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Simon Riggs
On 15 January 2013 22:14, Bruce Momjian wrote: > I mentioned last year that I wanted to start working on parallelism: We don't normally begin discussing topics for next release just as a CF is starting. Why is this being discussed now? -- Simon Riggs http://www.2ndQuadrant.

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Bruce Momjian
On Tue, Jan 15, 2013 at 10:53:29PM +, Simon Riggs wrote: > On 15 January 2013 22:14, Bruce Momjian wrote: > > > I mentioned last year that I wanted to start working on parallelism: > > We don't normally begin discussing topics for next release just as a > CF is starting. > > Why is this bei

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> I find the argument that this supports compression-over-the-wire to be >> quite weak, because COPY is only one form of bulk data transfer, and >> one that a lot of applications don't ever use. If we think we need to >> support tran

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Bruce Momjian
On Tue, Jan 15, 2013 at 10:39:10PM +, Peter Geoghegan wrote: > On 15 January 2013 22:14, Bruce Momjian wrote: > > I believe it is time to start adding parallel execution to the backend. > > We already have some parallelism in the backend: > > effective_io_concurrency and helper processes. I t

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Peter Geoghegan
On 15 January 2013 22:14, Bruce Momjian wrote: > I believe it is time to start adding parallel execution to the backend. > We already have some parallelism in the backend: > effective_io_concurrency and helper processes. I think it is time we > start to consider additional options. A few months

Re: [HACKERS] Parallel query execution

2013-01-15 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > Parallelism isn't going to help all queries, in fact it might be just a > small subset, but it will be the larger queries. The pg_upgrade > parallelism only helps clusters with multiple databases or tablespaces, > but the improvements are significant. T

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Andres Freund
On 2013-01-15 14:40:11 -0500, Tom Lane wrote: > Sergey Koposov writes: > > And I do see the tblspc file left after the finish of "make check": > > tmp_check/data/pg_tblspc/16385/PG_9.3_201212081/16384/16387 > > Interesting. If the tests are run immediately after initdb, 16387 > is the relfil

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > COPY with a file name is super-user-only. I am unclear how you would > use STDIN/STDOUT in any meaningful way with binary data produced by > compression. I guess you could with libpq. The patch that I posted provided this: psql -h myhost -c "COPY myta

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Bruce Momjian
On Tue, Jan 15, 2013 at 04:22:48PM -0500, Stephen Frost wrote: > * Bruce Momjian (br...@momjian.us) wrote: > > Well, COPY is super-user only, so it seems only useful for FDW, no? We > > already have lots of user-configuration FDW commands, so I can see > > adding this one too. > > COPY is most ce

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > I find the argument that this supports compression-over-the-wire to be > quite weak, because COPY is only one form of bulk data transfer, and > one that a lot of applications don't ever use. If we think we need to > support transmission compression for ours

[HACKERS] Parallel query execution

2013-01-15 Thread Bruce Momjian
I mentioned last year that I wanted to start working on parallelism: https://wiki.postgresql.org/wiki/Parallel_Query_Execution Years ago I added thread-safety to libpq. Recently I added two parallel execution paths to pg_upgrade. The first parallel path allows execution of external bina

Re: [HACKERS] unlogged tables vs. GIST

2013-01-15 Thread Robert Haas
On Tue, Jan 15, 2013 at 4:26 PM, Tom Lane wrote: > Robert Haas writes: >> I think that might be acceptable from a performance point of view - >> after all, if the index is unlogged, you're saving the cost of WAL - >> but I guess I still prefer a generic solution to this problem (a >> generalizati

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Tom Lane
Stephen Frost writes: > * Bruce Momjian (br...@momjian.us) wrote: >> And this leads to support-my-compression-binary-of-the-day mess. Why >> not just allow them to do '|compression-binary'? > The popen patch doesn't support the '|compression-binary' option through > the FE protocol. Even if it

Re: [HACKERS] json api WIP patch

2013-01-15 Thread Gavin Flower
On 16/01/13 08:04, David Fetter wrote: On Mon, Jan 14, 2013 at 07:52:56PM -0500, Andrew Dunstan wrote: On 01/14/2013 07:36 PM, Merlin Moncure wrote: While testing this I noticed that integer based 'get' routines are zero based -- was this intentional? Virtually all other aspects of SQL are 1 b

Re: [HACKERS] json api WIP patch

2013-01-15 Thread David E. Wheeler
On Jan 15, 2013, at 12:17 PM, Andrew Dunstan wrote: > I doubt I'm very representative either. People like David Wheeler, Taras > Mitran, Joe Van Dyk, and the Heroku guys would be better people to ask than > me. I'm quite prepared to change it if that's the consensus. They’re JSON arrays, not S

[HACKERS] Re: [BUGS] BUG #7809: Running pg_dump on slave w/ streaming replication fails if there are unlogged tables

2013-01-15 Thread Magnus Hagander
On Tue, Jan 15, 2013 at 12:13 AM, wrote: > The following bug has been logged on the website: > > Bug reference: 7809 > Logged by: Joe Van Dyk > Email address: j...@tanga.com > PostgreSQL version: 9.2.2 > Operating system: Ubuntu > Description: > > Running pg_dump on a streami

Re: [HACKERS] unlogged tables vs. GIST

2013-01-15 Thread Tom Lane
Robert Haas writes: > I think that might be acceptable from a performance point of view - > after all, if the index is unlogged, you're saving the cost of WAL - > but I guess I still prefer a generic solution to this problem (a > generalization of GetXLogRecPtrForTemp) rather than a special-purpos

Re: [HACKERS] json api WIP patch

2013-01-15 Thread Daniel Farina
On Tue, Jan 15, 2013 at 12:17 PM, Andrew Dunstan wrote: > > On 01/15/2013 02:47 PM, Merlin Moncure wrote: >> >> On Tue, Jan 15, 2013 at 1:04 PM, David Fetter wrote: >>> >>> On Mon, Jan 14, 2013 at 07:52:56PM -0500, Andrew Dunstan wrote: On 01/14/2013 07:36 PM, Merlin Moncure wrote:

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > Well, COPY is super-user only, so it seems only useful for FDW, no? We > already have lots of user-configuration FDW commands, so I can see > adding this one too. COPY is most certainly not superuser-only.. COPY w/ popen, if that popen can call anythin

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Stephen Frost
* Christopher Browne (cbbro...@gmail.com) wrote: > How about having a "pg_filters" table in pg_catalog which allows capturing > labels and names of known-to-be-safe binary filters: I was considering that (though I was thinking they'd be "transformations" rather than filters; filter implies that yo

Re: [HACKERS] unlogged tables vs. GIST

2013-01-15 Thread Robert Haas
On Tue, Jan 15, 2013 at 1:58 PM, Heikki Linnakangas wrote: >> I think what Heikki had in mind was that the copy in the index would be >> the authoritative one, not some image in shared memory. This'd imply >> dirtying the root page on every insert, as well as increased contention >> for the root

Re: [HACKERS] pg_ctl idempotent option

2013-01-15 Thread Josh Berkus
On 01/15/2013 07:55 AM, Peter Eisentraut wrote: > On 1/14/13 10:22 AM, Tom Lane wrote: >> Idempotent is a ten-dollar word. Can we find something that average >> people wouldn't need to consult a dictionary to understand? > > My suggestion in the original thread was --oknodo, but people didn't > l

[HACKERS] [sepgsql 3/3] Add db_procedure:execute permission checks

2013-01-15 Thread Kohei KaiGai
This patch adds sepgsql support for permission checks almost equivalent to the existing FUNCTION EXECUTE privilege. This feature is constructed on new OAT_FUNCTION_EXEC event type being invoked around pg_proc_aclcheck() except for cases when function's permissions are checked during CREATE or ALTE

Re: [HACKERS] count(*) of zero rows returns 1

2013-01-15 Thread Robert Haas
On Tue, Jan 15, 2013 at 2:26 PM, Shaun Thomas wrote: > Let's see what EnterpriseDB produces: > > test=> select * from dual; > > dummy > --- > X > (1 row) Hey, don't blame us. We didn't come up with this bad idea ... just trying to make life easier for those who are used to it. -- Robert

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Bruce Momjian
On Tue, Jan 15, 2013 at 03:37:07PM -0500, Christopher Browne wrote: > On Tue, Jan 15, 2013 at 2:53 PM, Stephen Frost wrote: > > The popen patch doesn't support the '|compression-binary' option through > > the FE protocol. Even if it did, it would only be available for > > superusers as we can't a

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-01-15 Thread Robert Haas
On Tue, Jan 15, 2013 at 3:22 PM, Tom Lane wrote: > Robert Haas writes: >> Or perhaps there is some other way to make sure that the user "really >> meant it", like refusing to create in pg_catalog unless the schema >> name is given explicitly. I kind of like that idea, actually. > > That does see

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Christopher Browne
On Tue, Jan 15, 2013 at 2:53 PM, Stephen Frost wrote: > The popen patch doesn't support the '|compression-binary' option through > the FE protocol. Even if it did, it would only be available for > superusers as we can't allow regular users to run arbitrary commands on > the server-side. That poi

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Sergey Koposov
On Tue, 15 Jan 2013, Tom Lane wrote: BTW, I just finished trying to reproduce this on an IA64 machine belonging to Red Hat, without success. So that seems to eliminate any possibility of the machine architecture being the trigger issue. The compiler's still a likely cause though. Anybody have a

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-01-15 Thread Tom Lane
Robert Haas writes: > Or perhaps there is some other way to make sure that the user "really > meant it", like refusing to create in pg_catalog unless the schema > name is given explicitly. I kind of like that idea, actually. That does seem attractive at first glance. Did you have an implementat

Re: [HACKERS] json api WIP patch

2013-01-15 Thread Andrew Dunstan
On 01/15/2013 02:47 PM, Merlin Moncure wrote: On Tue, Jan 15, 2013 at 1:04 PM, David Fetter wrote: On Mon, Jan 14, 2013 at 07:52:56PM -0500, Andrew Dunstan wrote: On 01/14/2013 07:36 PM, Merlin Moncure wrote: While testing this I noticed that integer based 'get' routines are zero based -- wa

Re: [HACKERS] [PERFORM] Slow query: bitmap scan troubles

2013-01-15 Thread Tom Lane
Bruce Momjian writes: > On Mon, Jan 14, 2013 at 12:56:37PM -0500, Tom Lane wrote: >> Remember also that "enable_seqscan=off" merely adds 1e10 to the >> estimated cost of seqscans. For sufficiently large tables this is not >> exactly a hard disable, just a thumb on the scales. But I don't know >>

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-01-15 Thread Robert Haas
On Mon, Jan 14, 2013 at 2:07 PM, Alvaro Herrera wrote: > Tom Lane escribió: >> Alvaro Herrera writes: > >> > alvherre=# create extension adminpack; >> > ERROR: permission denied for schema pg_catalog >> >> Um. I knew that that module's desire to shove stuff into pg_catalog >> would bite us some

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-01-15 Thread Robert Haas
On Sun, Jan 13, 2013 at 4:09 PM, Tom Lane wrote: > Right, that is the argument for ignoring missing schemas, and I think it > is entirely sensible for *search* activities. But allowing *creation* > to occur in an indeterminate schema is a horrid idea. But the default search path is $user, public

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-15 Thread Tom Lane
Robert Haas writes: > There are an awful lot of places in our source tree where the error > level is fixed. We could invent a new construct, say ereport_error or > so, that is just like ereport except that it takes no error-level > parameter because it's hard-coded to ERROR. > It would be a bit

[HACKERS] [sepgsql 1/3] add name qualified creation label

2013-01-15 Thread Kohei KaiGai
This patch adds sepgsql the feature of name qualified creation label. Background, on creation of a certain database object, sepgsql assigns a default security label according to the security policy that has a set of rules to determine a label of new object. Usually, a new object inherits its paren

Re: [HACKERS] Get current query in a trigger function

2013-01-15 Thread Robert Haas
On Mon, Jan 14, 2013 at 9:38 PM, Vlad Arkhipov wrote: >>> Is there any simple way of getting a query for which a trigger was >>> executed? >>> debug_query_string and ActivePortal->sourceText return the top query when >>> there are nested triggers. >> >> I believe - only if the trigger is written i

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > On Tue, Jan 15, 2013 at 10:55:04AM -0500, Stephen Frost wrote: > > * Peter Eisentraut (pete...@gmx.net) wrote: > > > Operating on compressed files transparently in file_fdw is obviously > > > useful, but why only gzip? > > > > This isn't really an argume

[HACKERS] Re: [pgsql-cluster-hackers] Save The Date: Cluster-Hackers meeting May 21st

2013-01-15 Thread Josh Berkus
> Given last year we didn't have more than 30 mins to spend on logical > replication, how much time will we have to discuss non-open source > systems? Mostly the idea is to discuss what some of the proprietary systems need from core postgres, especially where it overlaps with what OSS systems nee

Re: [HACKERS] passing diff options to pg_regress

2013-01-15 Thread Robert Haas
On Mon, Jan 14, 2013 at 6:57 AM, Peter Eisentraut wrote: > I sometimes find it useful to view a regression test difference using > other diff options, such as -u -w or more context. There is currently > no easy way to accomplish that. > > I suggest allowing to override the diff options using an e

Re: [HACKERS] json api WIP patch

2013-01-15 Thread Merlin Moncure
On Tue, Jan 15, 2013 at 1:04 PM, David Fetter wrote: > On Mon, Jan 14, 2013 at 07:52:56PM -0500, Andrew Dunstan wrote: >> >> On 01/14/2013 07:36 PM, Merlin Moncure wrote: >> >While testing this I noticed that integer based 'get' routines are >> >zero based -- was this intentional? Virtually all o

Re: [HACKERS] [PERFORM] Slow query: bitmap scan troubles

2013-01-15 Thread Bruce Momjian
On Mon, Jan 14, 2013 at 12:56:37PM -0500, Tom Lane wrote: > > The reported behavior was that the planner would prefer to > > sequential-scan the table rather than use the index, even if > > enable_seqscan=off. I'm not sure what the query looked like, but it > > could have been something best imple

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-15 Thread Robert Haas
On Sun, Jan 13, 2013 at 4:16 PM, Tom Lane wrote: > Andres Freund writes: >>> Basically, the aspects of this that I think are likely to be >>> reproducible wins across different platforms are (a) teaching the >>> compiler that elog(ERROR) doesn't return, and (b) reducing code size as >>> much as p

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-15 Thread Tom Lane
Sergey Koposov writes: > And I do see the tblspc file left after the finish of "make check": > tmp_check/data/pg_tblspc/16385/PG_9.3_201212081/16384/16387 Interesting. If the tests are run immediately after initdb, 16387 is the relfilenode assigned to table "foo" in the tablespace regressi

Re: [HACKERS] pg_ctl idempotent option

2013-01-15 Thread Boszormenyi Zoltan
2013-01-15 20:28 keltezéssel, Bruce Momjian írta: On Tue, Jan 15, 2013 at 10:25:23AM -0500, Tom Lane wrote: Alvaro Herrera writes: Vik Reykja escribi�: On Mon, Jan 14, 2013 at 4:22 PM, Tom Lane wrote: Idempotent is a ten-dollar word. Can we find something that average people wouldn't need

Re: [HACKERS] pg_ctl idempotent option

2013-01-15 Thread Bruce Momjian
On Tue, Jan 15, 2013 at 10:55:41AM -0500, Peter Eisentraut wrote: > On 1/14/13 10:22 AM, Tom Lane wrote: > > Idempotent is a ten-dollar word. Can we find something that average > > people wouldn't need to consult a dictionary to understand? > > My suggestion in the original thread was --oknodo, b

Re: [HACKERS] pg_ctl idempotent option

2013-01-15 Thread Bruce Momjian
On Tue, Jan 15, 2013 at 10:25:23AM -0500, Tom Lane wrote: > Alvaro Herrera writes: > > Vik Reykja escribi�: > >> On Mon, Jan 14, 2013 at 4:22 PM, Tom Lane wrote: > >>> Idempotent is a ten-dollar word. Can we find something that average > >>> people wouldn't need to consult a dictionary to unders

Re: [HACKERS] count(*) of zero rows returns 1

2013-01-15 Thread Shaun Thomas
On 01/15/2013 01:18 PM, Bruce Momjian wrote: AFAICR, 'dual' table has one column named 'DUMMY' and one row with value, single character X. How elegant. :-( Let's see what EnterpriseDB produces: test=> select * from dual; dummy --- X (1 row) Yep, elegant gets my vote. ;) But then ag

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-15 Thread Bruce Momjian
On Tue, Jan 15, 2013 at 10:55:04AM -0500, Stephen Frost wrote: > * Peter Eisentraut (pete...@gmx.net) wrote: > > Operating on compressed files transparently in file_fdw is obviously > > useful, but why only gzip? > > This isn't really an argument, imv. It's only gzip *right this moment* > because

Re: [HACKERS] count(*) of zero rows returns 1

2013-01-15 Thread Bruce Momjian
On Mon, Jan 14, 2013 at 10:25:39PM -0500, Gurjeet Singh wrote: > On Mon, Jan 14, 2013 at 3:09 PM, David Johnston wrote: > > What does "SELECT * FROM dual" in Oracle yield? > > > AFAICR, 'dual' table has one column named 'DUMMY' and one row with value, > single character X. How elegant. :-

Re: [HACKERS] count(*) of zero rows returns 1

2013-01-15 Thread Bruce Momjian
On Mon, Jan 14, 2013 at 10:47:58PM -0500, Gurjeet Singh wrote: > Interesting to note that SELECT * FROM table_with_zero_cols does not complain > of anything. > > postgres=# select * from test1; > -- > (0 rows) > > This I believe result of the fact that we allow user to drop all columns of a > ta

[HACKERS] pkg-config files for libpq and ecpg

2013-01-15 Thread Peter Eisentraut
I'll take another stab at providing pkg-config files for the client-side libraries. The main reason this time around is that this works a lot better (or at all) for multi-arch library installations. Another is that pkg-config has become a lot smarter and flexible over the years, and it's probably

  1   2   >