Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Jim Nasby
On 9/29/15 4:13 PM, Alvaro Herrera wrote: Joe Conway wrote: On 09/29/2015 01:48 PM, Alvaro Herrera wrote: I remember it, but I'm not sure it would have helped you. As I recall, your trouble was that after a reboot the init script decided to initdb the mount point -- postmaster wouldn't have

Re: [HACKERS] Use pg_rewind when target timeline was switched

2015-09-29 Thread Michael Paquier
On Sat, Sep 19, 2015 at 8:27 AM, Michael Paquier wrote: > On Fri, Sep 18, 2015 at 6:25 PM, Michael Paquier wrote: >> The refactoring of getTimelineHistory as you propose looks like a good >> idea to me, I tried to remove by myself the difference between source >> and target in copy_fetch.c and frie

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-29 Thread Fabien COELHO
Hello Tatsuo, So on second thought the formula should rather be: ... / (is_connect? nclients: nthreads) I don't think this is quite correct. If is_connect is false, then following loop is executed in threadRun(): /* make connections to the database */ for

Re: [HACKERS] Re: BRIN Scan: Optimize memory allocation in function 'bringetbitmap'

2015-09-29 Thread Simon Riggs
On 28 September 2015 at 15:47, zhangjinyu wrote: > Sorry, I forgot attaching patch. But I have send path in another thread. > Please see this thread "Patch: Optimize memory allocation in function > 'bringetbitmap' ". > Please don't submit a patch on a separate thread. Thanks. Let's close this

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-29 Thread Tatsuo Ishii
>> FWIW, I vote with Tatsuo-san. Such a change will break comparability of >> results with all previous versions, which means it's not something to do >> in minor releases, even if we now believe the previous results were >> somewhat bogus. Arguing that it's "not a behavioral change" seems >> qui

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Michael Paquier
On Wed, Sep 30, 2015 at 7:19 AM, Tom Lane wrote: > I wrote: >> Josh Berkus writes: >>> Give me source with the change, and I'll put it on a cheap, low-bandwith >>> AWS instance and hammer the heck out of it. That should raise any false >>> positives we can expect. > >> Here's a draft patch again

Re: [HACKERS] pageinspect patch, for showing tuple data

2015-09-29 Thread Michael Paquier
On Tue, Sep 29, 2015 at 11:39 PM, Nikolay Shaplov wrote: > But since now we actually parse data with tuple_data_split, we can provide a > precisely formed fake information, so you are not limited with how it is > actually stored in page. You just pass any arguments you want. So you does not > need

Re: [HACKERS] Doubt in pgbench TPS number

2015-09-29 Thread Tatsuo Ishii
>> Here conn_total_time is the sum of time to establish connection to >> PostgreSQL. Since establishing connections to PostgreSQL is done in >> serial rather than in parallel, conn_total_time should have been >> divided by nclients. > > After some more thinking and looking again at the connection

Re: [HACKERS] Arguable RLS security bug, EvalPlanQual() paranoia

2015-09-29 Thread Adam Brightwell
> I'm not convinced this is the right place, but at a minimum it should be > referenced from the RLS documentation. Further, it should be noted that > users who have direct SQL access can control what the isolation level > is for their transaction. I agree that it should be referenced by the RLS

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Pavan Deolasee
On Wed, Sep 30, 2015 at 2:16 AM, Alvaro Herrera wrote: > > > That's a very good point. I think Github and other sites are already > blocked in countries like India and Cuba. Github is not blocked in India and was never as far as I know. Well our government recently blocked some porn sites, but

Re: [HACKERS] LISTEN denial of service with aborted transaction

2015-09-29 Thread Matt Newell
On Tuesday, September 29, 2015 09:58:35 PM Tom Lane wrote: > Matt Newell writes: > > On Monday, September 28, 2015 07:22:26 PM Tom Lane wrote: > >> Possibly. sinval catchup notification works like that, so you could > >> maybe > >> invent a similar mechanism for notify. Beware that we've had to

Re: [HACKERS] [PATCH 5/6] pg_basebackup: allow GetConnection() to make non-replication connections.

2015-09-29 Thread Michael Paquier
On Wed, Sep 30, 2015 at 10:47 AM, Robert Haas wrote: > It would probably be a good idea to review > https://wiki.postgresql.org/wiki/Submitting_a_Patch -- the Linux style > of patch submission is generally not preferred here; we like to > discuss closely-related patches as a group, on a single thr

Re: [HACKERS] LISTEN denial of service with aborted transaction

2015-09-29 Thread Tom Lane
Matt Newell writes: > On Monday, September 28, 2015 07:22:26 PM Tom Lane wrote: >> Possibly. sinval catchup notification works like that, so you could maybe >> invent a similar mechanism for notify. Beware that we've had to fix >> performance issues with sinval catchup; you don't want to release

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-29 Thread Kyotaro HORIGUCHI
Hello, At Tue, 29 Sep 2015 16:57:03 +0200, Tomas Vondra wrote in <560aa6bf.5030...@2ndquadrant.com> > >>> The patch does not change the check_index_only implementation - it > >>> still needs to check the clauses, just like in v1 of the patch. To > >>> make this re-check unnecessary, we'd have to

Re: [HACKERS] [PATCH 5/6] pg_basebackup: allow GetConnection() to make non-replication connections.

2015-09-29 Thread Robert Haas
On Tue, Sep 29, 2015 at 6:16 PM, Joshua Elsasser wrote: > --- Hi! Thanks for submitting patches to the PostgreSQL community. We need more developers, and appreciate contributions. However, I'm somewhat flummoxed by this particular patch series, because (1) there's no real explanation of what t

Re: [HACKERS] Comment update to pathnode.c

2015-09-29 Thread Etsuro Fujita
On 2015/09/29 20:51, Robert Haas wrote: On Tue, Sep 29, 2015 at 1:55 AM, Etsuro Fujita wrote: Thanks for the comments! Attached is an updated version of the patch. Committed and back-patched to 9.5. Thanks! Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] Parallel Seq Scan

2015-09-29 Thread Robert Haas
On Thu, Sep 24, 2015 at 2:31 PM, Amit Kapila wrote: > [ parallel_seqscan_partialseqscan_v18.patch ] I spent a bit of time reviewing the heapam.c changes in this patch this evening, and I think that your attempt to add support for synchronized scans has some problems. - In both heapgettup() and h

[HACKERS] Removing max_connection requirement on hot_standby

2015-09-29 Thread Chris Winslett
I'm orchestrating Postgres to behave as a leader-follower cluster. I've run into issues when I am scaling down a connection count for a cluster (scaling up is fine -- scaling down results in fatal errors). I use an open source tool I've written to orchestrate the cluster called Governor ( http://g

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Joshua D. Drake
On 09/29/2015 03:46 PM, Tom Lane wrote: Alvaro Herrera writes: Merlin Moncure wrote: I've read this email about three times now and it's not clear at all to me what a issue/bug tracker brings to the table. In my opinion, this thread is about a bug tracker, not a patch tracker. We already ha

Re: [HACKERS] PATCH: use foreign keys to improve join estimates v1

2015-09-29 Thread David Rowley
On 29 September 2015 at 01:59, Tomas Vondra wrote: > Hi, > > On 09/27/2015 02:00 PM, David Rowley wrote: > >> I've been working on this again. I've put back the code that you wrote >> for the looping over each combination of relations from either side of >> the join. >> >> I've also added some co

Re: [HACKERS] LISTEN denial of service with aborted transaction

2015-09-29 Thread Matt Newell
On Monday, September 28, 2015 07:22:26 PM Tom Lane wrote: > Matt Newell writes: > > 1. When a connection issues it's first LISTEN command, in > > Exec_ListenPreCommit QUEUE_BACKEND_POS(MyBackendId) = QUEUE_TAIL; > > this causes the connection to iterate through every notify queued in the > > slru,

Re: [HACKERS] Add pg_basebackup single tar output format

2015-09-29 Thread Michael Paquier
On Wed, Sep 30, 2015 at 7:16 AM, Joshua Elsasser wrote: > On Tue, Sep 29, 2015 at 11:50:37PM +0200, Andres Freund wrote: >> On 2015-09-29 14:38:11 -0700, Josh Elsasser wrote: >> > I've put my changes up as a series of relatively small commits on this >> > branch of a github fork: >> > >> > https:/

[HACKERS] Re: BRIN Scan: Optimize memory allocation in function 'bringetbitmap'

2015-09-29 Thread zhangjinyu
I forgot disable-c-assert last test. The following show the test result when disable-c-assert. After optimize code (warm run) postgres=# select count(*) from lineitem where l_orderkey=1; count --- 6 (1 row) Time: 327.143 ms Before optimizing code (warm run) postgres=# select count(*) f

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Tom Lane
Alvaro Herrera writes: > Merlin Moncure wrote: >> I've read this email about three times now and it's not clear at all >> to me what a issue/bug tracker brings to the table. > In my opinion, this thread is about a bug tracker, not a patch tracker. > We already have a patch tracking system which w

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Josh Berkus
On 09/29/2015 12:47 PM, Tom Lane wrote: > Josh Berkus writes: >> In general, having the postmaster survive deletion of PGDATA is >> suboptimal. In rare cases of having it survive installation of a new >> PGDATA (via PITR restore, for example), I've even seen the zombie >> postmaster corrupt the d

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Josh Berkus
On 09/29/2015 03:08 PM, Merlin Moncure wrote: > I've read this email about three times now and it's not clear at all > to me what a issue/bug tracker brings to the table. Here are the problems I'd like to solve: 1. "Was this issue fixed in a Postgres update? Which one?" 2. Not losing track of m

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Alvaro Herrera
Merlin Moncure wrote: > I've read this email about three times now and it's not clear at all > to me what a issue/bug tracker brings to the table. In my opinion, this thread is about a bug tracker, not a patch tracker. We already have a patch tracking system which works very well. (We are not ab

[HACKERS] [PATCH 6/6] pg_basebackup: add a single-tar output format.

2015-09-29 Thread Joshua Elsasser
This will write one single tar file containing all tablespaces, and can be written to stdout. --- src/bin/pg_basebackup/pg_basebackup.c | 282 -- 1 file changed, 269 insertions(+), 13 deletions(-) diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_bas

Re: [HACKERS] CustomScan support on readfuncs.c

2015-09-29 Thread Kouhei Kaigai
> On Mon, Sep 28, 2015 at 8:31 PM, Kouhei Kaigai wrote: > >> Instead of doing this: > >> > >> +/* Dump library and symbol name instead of raw pointer */ > >> appendStringInfoString(str, " :methods "); > >> -_outToken(str, node->methods->CustomName); > >> +_outToken(str, node->meth

Re: [HACKERS] Add pg_basebackup single tar output format

2015-09-29 Thread Joshua Elsasser
On Tue, Sep 29, 2015 at 11:50:37PM +0200, Andres Freund wrote: > On 2015-09-29 14:38:11 -0700, Josh Elsasser wrote: > > I've put my changes up as a series of relatively small commits on this > > branch of a github fork: > > > > https://github.com/jre/postgres/commits/single-tar > > > > Comments a

[HACKERS] [PATCH 2/6] pg_basebackup: factor out tar open/close code for reusability.

2015-09-29 Thread Joshua Elsasser
This adds a simple struct and open and close functions to abstract and isolate the zlib vs. stdio output logic and allow it to be reused. --- src/bin/pg_basebackup/pg_basebackup.c | 300 +- 1 file changed, 154 insertions(+), 146 deletions(-) diff --git a/src/bin/pg

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
I wrote: > Josh Berkus writes: >> Give me source with the change, and I'll put it on a cheap, low-bandwith >> AWS instance and hammer the heck out of it. That should raise any false >> positives we can expect. > Here's a draft patch against HEAD (looks like it will work on 9.5 or > 9.4 without m

Re: [HACKERS] PATCH: use foreign keys to improve join estimates v1

2015-09-29 Thread David Rowley
On 29 September 2015 at 01:59, Tomas Vondra wrote: > > CREATE TABLE f (id1 INT, id2 INT, PRIMARY KEY (id1, id2)); > > CREATE TABLE d1 (id1 INT, id2 INT, FOREIGN KEY (id1, id2) REFERENCES > f(id1, id2)); > CREATE TABLE d2 (id1 INT, id2 INT, FOREIGN KEY (id1, id2) REFERENCES > f(id1, id2)); > > INS

[HACKERS] [PATCH 4/6] pg_basebackup: don't lose a zero-length file at the end of a tablespace.

2015-09-29 Thread Joshua Elsasser
After a complete tar header was buffered it would only be processed during the next iteration of the read loop. A zero-length file such as a directory had no data to read, so the loop would exit without ever having processed the file. --- src/bin/pg_basebackup/pg_basebackup.c | 238 +++

Re: [HACKERS] unclear about row-level security USING vs. CHECK

2015-09-29 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 28 September 2015 at 20:15, Stephen Frost wrote: > > I listed out the various alternatives but didn't end up getting any > > responses to it. I'm still of the opinion that the documentation is the > > main thing which needs improving here, but

[HACKERS] [PATCH 5/6] pg_basebackup: allow GetConnection() to make non-replication connections.

2015-09-29 Thread Joshua Elsasser
--- src/bin/pg_basebackup/pg_basebackup.c | 4 ++-- src/bin/pg_basebackup/pg_receivexlog.c | 4 ++-- src/bin/pg_basebackup/pg_recvlogical.c | 4 ++-- src/bin/pg_basebackup/streamutil.c | 6 +++--- src/bin/pg_basebackup/streamutil.h | 2 +- 5 files changed, 10 insertions(+), 10 deletions(-

[HACKERS] [PATCH 1/6] Add support for longer filenames in tar headers (up to 254 bytes).

2015-09-29 Thread Joshua Elsasser
New functions tarHeaderRename() and tarHeaderGetName() are exposed to store and retrieve the longer filenames. tarCreateHeader() continues to limit filenames to 99 bytes to preserve compatability with existing consumers. --- src/include/pgtar.h | 2 + src/port/tar.c | 134 +

[HACKERS] [PATCH 3/6] pg_basebackup: factor out code to add a recovery.conf file to the tar file.

2015-09-29 Thread Joshua Elsasser
This is just a simple refactor for readability and reusability. --- src/bin/pg_basebackup/pg_basebackup.c | 46 --- 1 file changed, 26 insertions(+), 20 deletions(-) diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c index f

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Merlin Moncure
On Tue, Sep 29, 2015 at 12:42 PM, Joshua D. Drake wrote: > On 09/29/2015 07:25 AM, Merlin Moncure wrote: >> >> On Wed, Sep 23, 2015 at 1:18 PM, Kam Lasater wrote: >>> >>> Hello, >>> >>> Last night I heard that Postgres had no issue/bug tracker. At first I >>> thought the guy was trolling me. Ser

Re: [HACKERS] Arguable RLS security bug, EvalPlanQual() paranoia

2015-09-29 Thread Stephen Frost
* Adam Brightwell (adam.brightw...@crunchydatasolutions.com) wrote: > On Mon, Aug 3, 2015 at 6:21 PM, Peter Geoghegan wrote: > > On Mon, Aug 3, 2015 at 3:07 PM, Stephen Frost wrote: > >> Thoughts? Trying to keep it straight-forward and provide a simple > >> solution for users to be able to addre

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-29 Thread Robert Haas
On Mon, Sep 28, 2015 at 11:15 PM, Etsuro Fujita wrote: > I thought the same thing [1]. While I thought it was relatively easy to > make changes to RefetchForeignRow that way for the foreign table case > (scanrelid>0), I was not sure how hard it would be to do so for the foreign > join case (scanr

Re: [HACKERS] Add pg_basebackup single tar output format

2015-09-29 Thread Andres Freund
Hi Josh, On 2015-09-29 14:38:11 -0700, Josh Elsasser wrote: > As a side effect I've refactored the some of the pg_basebackup code > for readability and reusability Cool, that's desperately needed. I've been trying to bug Magnus into doing that for a bunch of conferences now ;) > I've put my chan

[HACKERS] Add pg_basebackup single tar output format

2015-09-29 Thread Josh Elsasser
Hi. I have a need to pipe the output from pg_basebackup for a multi-tablespace cluster into another program without spooling to disk. Seeing as the current -F tar output format can't do that, I've made an attempt at implementing that myself. As a side effect I've refactored the some of the pg_base

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
Josh Berkus writes: > Give me source with the change, and I'll put it on a cheap, low-bandwith > AWS instance and hammer the heck out of it. That should raise any false > positives we can expect. Here's a draft patch against HEAD (looks like it will work on 9.5 or 9.4 without modifications, too)

Re: [HACKERS] Arguable RLS security bug, EvalPlanQual() paranoia

2015-09-29 Thread Adam Brightwell
On Mon, Aug 3, 2015 at 6:21 PM, Peter Geoghegan wrote: > On Mon, Aug 3, 2015 at 3:07 PM, Stephen Frost wrote: >> Thoughts? Trying to keep it straight-forward and provide a simple >> solution for users to be able to address the issue, if they're worried >> about it. Perhaps this, plus an additio

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Alvaro Herrera
Joe Conway wrote: > On 09/29/2015 01:48 PM, Alvaro Herrera wrote: > > I remember it, but I'm not sure it would have helped you. As I recall, > > your trouble was that after a reboot the init script decided to initdb > > the mount point -- postmaster wouldn't have been running at all ... > > Righ

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Simon Riggs
On 29 September 2015 at 20:52, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Tue, Sep 29, 2015 at 8:34 PM, Simon Riggs > wrote: > >> On 29 September 2015 at 12:52, Shulgin, Oleksandr < >> oleksandr.shul...@zalando.de> wrote: >> >> >>> Hitting a process with a signal and hoping it

Re: [HACKERS] Less than ideal error reporting in pg_stat_statements

2015-09-29 Thread Peter Geoghegan
On Fri, Sep 25, 2015 at 8:51 AM, Marti Raudsepp wrote: > I've also been seeing lots of log messages saying "LOG: out of > memory" on a server that's hosting development databases. I put off > debugging this until now because it didn't seem to have any adverse > effects on the system. That's unfo

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Joe Conway
On 09/29/2015 01:48 PM, Alvaro Herrera wrote: > Joe Conway wrote: >> On 09/29/2015 12:47 PM, Tom Lane wrote: >>> We could possibly add additional checks, like trying to verify that >>> pg_control has the same inode number it used to. But I'm afraid that >>> would add portability issues and false-p

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Alvaro Herrera
Joe Conway wrote: > On 09/29/2015 12:47 PM, Tom Lane wrote: > > We could possibly add additional checks, like trying to verify that > > pg_control has the same inode number it used to. But I'm afraid that > > would add portability issues and false-positive hazards that would > > outweigh the value

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Alvaro Herrera
Oleg Bartunov wrote: > me too, but I have to mention one problem we should have in mind - it's > independency from political games (sanctions). Think about developers from > Russia, who one day may be blocked by Github, for example. That's a very good point. I think Github and other sites are

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Joe Conway
On 09/29/2015 12:47 PM, Tom Lane wrote: > We could possibly add additional checks, like trying to verify that > pg_control has the same inode number it used to. But I'm afraid that > would add portability issues and false-positive hazards that would > outweigh the value. Not sure you remember the

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Oleg Bartunov
On Tue, Sep 29, 2015 at 6:36 PM, Andrew Dunstan wrote: > > > On 09/29/2015 10:55 AM, Steve Crawford wrote: > >> On Tue, Sep 29, 2015 at 7:16 AM, David Fetter > da...@fetter.org>> wrote: >> >> ...What we're not fine with is depending on a proprietary system, no >> matter what type of licen

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Alvaro Herrera
Tom Lane wrote: > Testing accessibility of "global/pg_control" would be enough to catch this > case, but only if we do it before you create a new one. So that seems > like an argument for making the test relatively often. The once-a-minute > option is sounding better and better. If we weren't a

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Shulgin, Oleksandr
On Tue, Sep 29, 2015 at 8:34 PM, Simon Riggs wrote: > On 29 September 2015 at 12:52, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > > >> Hitting a process with a signal and hoping it will produce a meaningful >> response in all circumstances without disrupting its current task was

Re: [HACKERS] ON CONFLICT issues around whole row vars,

2015-09-29 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: > On Tue, Sep 29, 2015 at 8:24 AM, Andres Freund wrote: > > So, took a bit longer than "tomorrow. I fought for a long while with a > > mysterious issue, which turned out to be separate bug: The excluded > > relation was affected by row level security poli

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
Josh Berkus writes: > On 09/29/2015 11:48 AM, Tom Lane wrote: >> But today I thought of another way: suppose that we teach the postmaster >> to commit hara-kiri if the $PGDATA directory goes away. Since the >> buildfarm script definitely does remove all the temporary data directories >> it create

Re: [HACKERS] ON CONFLICT issues around whole row vars,

2015-09-29 Thread Andres Freund
On September 29, 2015 8:52:14 PM GMT+02:00, Peter Geoghegan wrote: >On Tue, Sep 29, 2015 at 8:24 AM, Andres Freund >wrote: >> So, took a bit longer than "tomorrow. I fought for a long while with >a >> mysterious issue, which turned out to be separate bug: The excluded >> relation was affected by

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Josh Berkus
On 09/29/2015 12:18 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 09/29/2015 02:48 PM, Tom Lane wrote: >>> Also, perhaps we'd only enable this behavior in --enable-cassert builds, >>> to avoid any risk of a postmaster incorrectly choosing to suicide in a >>> production scenario. Or maybe tha

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
Andrew Dunstan writes: > On 09/29/2015 02:48 PM, Tom Lane wrote: >> Also, perhaps we'd only enable this behavior in --enable-cassert builds, >> to avoid any risk of a postmaster incorrectly choosing to suicide in a >> production scenario. Or maybe that's overly conservative. > Not every buildfar

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> I wouldn't want to do this every time through the postmaster's main loop, > >> but we could do this once an hour for no added cost by adding the check > >> where it does TouchSocketLockFi

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Andrew Dunstan
On 09/29/2015 02:48 PM, Tom Lane wrote: A problem the buildfarm has had for a long time is that if for some reason the scripts fail to stop a test postmaster, the postmaster process will hang around and cause subsequent runs to fail because of socket conflicts. This seems to have gotten a lot w

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> I wouldn't want to do this every time through the postmaster's main loop, >> but we could do this once an hour for no added cost by adding the check >> where it does TouchSocketLockFiles; or once every few minutes if we >> carried a

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Josh Berkus
On 09/29/2015 11:48 AM, Tom Lane wrote: > But today I thought of another way: suppose that we teach the postmaster > to commit hara-kiri if the $PGDATA directory goes away. Since the > buildfarm script definitely does remove all the temporary data directories > it creates, this ought to get the jo

Re: [HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > But today I thought of another way: suppose that we teach the postmaster > to commit hara-kiri if the $PGDATA directory goes away. Since the > buildfarm script definitely does remove all the temporary data directories > it creates, this ought to get the job

Re: [HACKERS] ON CONFLICT issues around whole row vars,

2015-09-29 Thread Peter Geoghegan
On Tue, Sep 29, 2015 at 8:24 AM, Andres Freund wrote: > So, took a bit longer than "tomorrow. I fought for a long while with a > mysterious issue, which turned out to be separate bug: The excluded > relation was affected by row level security policies, which doesn't make > sense. Why? You certain

[HACKERS] Idea for improving buildfarm robustness

2015-09-29 Thread Tom Lane
A problem the buildfarm has had for a long time is that if for some reason the scripts fail to stop a test postmaster, the postmaster process will hang around and cause subsequent runs to fail because of socket conflicts. This seems to have gotten a lot worse lately due to the influx of very slow b

Re: [HACKERS] BRIN indexes for MAX, MIN, ORDER BY?

2015-09-29 Thread Simon Riggs
On 29 September 2015 at 13:20, Jeremy Harris wrote: > On 27/09/15 21:58, Gavin Wahl wrote: > > Somewhat harder but still possible would be using BRIN indexes to > > accelerate ORDER BY. This would require a sorting algorithm that can take > > advantage of mostly-sorted inputs. You would sort the

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Simon Riggs
On 29 September 2015 at 12:52, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > Hitting a process with a signal and hoping it will produce a meaningful > response in all circumstances without disrupting its current task was way > too naive. > Hmm, I would have to disagree, sorry. For

Re: [HACKERS] BRIN indexes for MAX, MIN, ORDER BY?

2015-09-29 Thread Jeremy Harris
On 27/09/15 21:58, Gavin Wahl wrote: > Somewhat harder but still possible would be using BRIN indexes to > accelerate ORDER BY. This would require a sorting algorithm that can take > advantage of mostly-sorted inputs. You would sort the page ranges by their > minimum or maximum value, then feed the

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

2015-09-29 Thread Alvaro Herrera
Hi Fujii, thanks for the review. I have pushed the patch to 9.5 and master. Fujii Masao wrote: > @@ -5321,7 +5333,7 @@ xact_redo_commit(xl_xact_parsed_commit *parsed, > /* Set the transaction commit timestamp and metadata */ > TransactionTreeSetCommitTsData(xid, parsed->nsubxacts, pars

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Shulgin, Oleksandr
On Tue, Sep 29, 2015 at 7:01 PM, Simon Riggs wrote: > On 25 September 2015 at 12:13, Shulgin, Oleksandr < > oleksandr.shul...@zalando.de> wrote: > > >> I now believe that use of ProcessInterrupts() in the recently proposed >> design as well as manipulation of proc latch due to use of shared memor

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-09-29 Thread Robert Haas
On Tue, Sep 29, 2015 at 4:49 AM, Kouhei Kaigai wrote: > Also note that EvalPlanQualFetchRowMarks() will raise an error > if RefetchForeignRow callback returned NULL tuple. > Is it right or expected behavior? That's not how I read the code. If RefetchForeignRow returns NULL, we just ignore the ro

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Tom Lane
Andres Freund writes: > On 2015-09-29 13:40:28 -0400, Tom Lane wrote: >> I think you missed my point: gitlab would then believe it's in charge of, >> eg, granting write access to that repo. We could perhaps whack it over >> the head till it only does what we want and not ten other things, but >>

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Andres Freund
On 2015-09-29 13:40:28 -0400, Tom Lane wrote: > Andres Freund writes: > > I don't have any opinion WRT gitlab, but I'm fairly certain it'd be > > unproblematic to configure automatic mirroring into it from > > gitmaster. > > I think you missed my point: gitlab would then believe it's in charge of

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Joshua D. Drake
On 09/29/2015 07:25 AM, Merlin Moncure wrote: On Wed, Sep 23, 2015 at 1:18 PM, Kam Lasater wrote: Hello, Last night I heard that Postgres had no issue/bug tracker. At first I thought the guy was trolling me. Seriously, how could this be. Certainly a mature open source project that is the datab

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Tom Lane
Andres Freund writes: > On 2015-09-29 13:27:15 -0400, Tom Lane wrote: >> Not that so much as that the gitlab code really wants to be connected up >> to our code repo. That makes complete sense in terms of its goals as >> stated by Torsten upthread, namely to be a git repo manager. But it's >> no

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Andres Freund
On 2015-09-29 13:27:15 -0400, Tom Lane wrote: > Corey Huinker writes: > >>> And they'd sure love to be in charge of our code repo. > > >> Mh - i'm not a native speaker. I didn't understand this line. > > > Tom was saying that the JIRA/Atlassian people would happily volunteer to > > host our code

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Jeff Anton
Seems to me that there are a bunch of agendas here. I read about not wanting to be trapped into a proprietary system. You can be trapped in any software you depend upon. Compilers, Platforms, SCM, issue tracking are all places to be trapped. Postgres and Postgresql have been around a very l

Re: [HACKERS] CustomScan support on readfuncs.c

2015-09-29 Thread Robert Haas
On Mon, Sep 28, 2015 at 8:31 PM, Kouhei Kaigai wrote: >> Instead of doing this: >> >> +/* Dump library and symbol name instead of raw pointer */ >> appendStringInfoString(str, " :methods "); >> -_outToken(str, node->methods->CustomName); >> +_outToken(str, node->methods->methods_l

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Tom Lane
Corey Huinker writes: >>> And they'd sure love to be in charge of our code repo. >> Mh - i'm not a native speaker. I didn't understand this line. > Tom was saying that the JIRA/Atlassian people would happily volunteer to > host our code repository for no cost (in money) to us. Not that so much

Re: [HACKERS] Parallel Seq Scan

2015-09-29 Thread Robert Haas
On Tue, Sep 29, 2015 at 12:39 AM, Amit Kapila wrote: > Attached patch is a rebased patch based on latest commit (d1b7c1ff) > for Gather node. > > - I have to reorganize the defines in execParallel.h and .c. To keep > ParallelExecutorInfo, in GatherState node, we need to include execParallel.h > in

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Corey Huinker
> > > And they'd sure love to be in charge of our code repo. >> > > Mh - i'm not a native speaker. I didn't understand this line. > > Tom was saying that the JIRA/Atlassian people would happily volunteer to host our code repository for no cost (in money) to us. The implication being that they have

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-09-29 Thread Simon Riggs
On 25 September 2015 at 12:13, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > I now believe that use of ProcessInterrupts() in the recently proposed > design as well as manipulation of proc latch due to use of shared memory > queue are major blockers. > > In order to simplify things

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-29 Thread Tomas Vondra
On 09/29/2015 04:57 PM, Tomas Vondra wrote: Hello, On 09/29/2015 12:27 PM, Kyotaro HORIGUCHI wrote: ... cost_index() seems to need to be fixed. It would count excluded clauses in estimate. Hmm, good point. The problem is that extract_nonindex_conditions uses baserel->baserestrictinfo agai

Re: [HACKERS] Box type equality

2015-09-29 Thread Jeff Anton
The Box type is the oldest non-linear type in the Postgres system. We used it as the template for extensibility in the original system about thirty years ago. We had R-trees for box indexing. If you want fuzzy box matching, that seems possible with R-trees and some creativity by say matching

Re: [HACKERS] GinPageIs* don't actually return a boolean

2015-09-29 Thread Alvaro Herrera
Andres Freund wrote: > I went through all headers in src/include and checked for macros > containing [^&]&[^&] and checked whether they have this hazard. Found a > fair number. > > That patch also changes !! tests into != 0 style. I don't think you pushed this, did you? -- Álvaro Herrera

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Andrew Dunstan
On 09/29/2015 10:55 AM, Steve Crawford wrote: On Tue, Sep 29, 2015 at 7:16 AM, David Fetter > wrote: ...What we're not fine with is depending on a proprietary system, no matter what type of license, as infrastructure... Exactly. Which is why I was warning abo

[HACKERS] Box type equality

2015-09-29 Thread Stanislav Kelvich
Hi. I've faced an issue with Box type comparison that exists almost for a five years. > create table t (b box); CREATE TABLE > select count(*), b from t group by b; ERROR: could not identify an equality operator for type box As mentioned in http://www.postgresql.org/message-id/pine.lnx.4.64.1

Re: [HACKERS] Box type equality

2015-09-29 Thread Tom Lane
Stanislav Kelvich writes: > I've faced an issue with Box type comparison that exists almost for a five > years. Try twenty-five years. The code's been like that since Berkeley. > That can be fixed by b-tree equality for boxes, but we need some > decisions there. The problem with inventing

Re: [HACKERS] ON CONFLICT issues around whole row vars,

2015-09-29 Thread Andres Freund
On 2015-09-24 17:25:21 +0200, Andres Freund wrote: > Stuff I want to fix by tomorrow: > * Whole row var references to exclude > * wrong offsets for columns after dropped ones > * INSTEAD DO UPDATE for tables with oids > > Do you know of anything else? So, took a bit longer than "tomorrow. I fought

Re: [HACKERS] unclear about row-level security USING vs. CHECK

2015-09-29 Thread Adam Brightwell
> My vote would be to keep it as-is. Same for me. > It feels perfectly natural to me. USING clauses add to the query's > WHERE clause controlling which existing rows you can SELECT, UPDATE or > DELETE. WITH CHECK clauses control what new data you can add via > INSERT or UPDATE. UPDATE allows both

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-29 Thread Tom Lane
Asif Naeem writes: > Thank you Tom. The issue seems not reproducible anymore with latest PG95 > source code (commit 60fcee9e5e77dc748a9787fae34328917683b95e) Windows build Thanks for testing! I've marked this issue resolved in the 9.5 open-items list. regards, tom lane

Re: [HACKERS] PATCH: index-only scans with partial indexes

2015-09-29 Thread Tomas Vondra
Hello, On 09/29/2015 12:27 PM, Kyotaro HORIGUCHI wrote: Hi, At Sat, 26 Sep 2015 18:00:33 +0200, Tomas Vondra wrote in <5606c121.10...@2ndquadrant.com> Hi, On 09/26/2015 01:28 PM, Tomas Vondra wrote: The patch does not change the check_index_only implementation - it still needs to check th

Re: [HACKERS] row_security GUC, BYPASSRLS

2015-09-29 Thread Adam Brightwell
> I've attached a patch to implement it. It's not fully polished but it's > sufficient for comment, I believe. Additional comments, documentation > and regression tests are to be added, if we have agreement on the > grammer and implementation approach. I have given the first patch a quick review

Re: [HACKERS] No Issue Tracker - Say it Ain't So!

2015-09-29 Thread Steve Crawford
On Tue, Sep 29, 2015 at 7:16 AM, David Fetter wrote: > ...What we're not fine with is depending on a proprietary system, no > matter what type of license, as infrastructure... > > Exactly. Which is why I was warning about latching onto features only available in the closed enterprise version. Ch

Re: [HACKERS] [PATCH v2] GSSAPI encryption support

2015-09-29 Thread Robbie Harwood
Robbie Harwood writes: Michael Paquier writes: > Well, the issue is still here: login through gssapi fails with > your patch, not with HEAD. This patch is next on my review list by > the way so I'll see what I can do about it soon even if I am in > the US for Postgres O

Re: [HACKERS] 9.5: Can't connect with PGSSLMODE=require on Windows

2015-09-29 Thread Asif Naeem
Thank you Tom. The issue seems not reproducible anymore with latest PG95 source code (commit 60fcee9e5e77dc748a9787fae34328917683b95e) Windows build i.e. C:\PG\postgresql\pg95_with_openssl>bin\psql.exe -d postgres -h > 172.16.141.232 > psql (9.5alpha2) > WARNING: Console code page (437) differs fr

Re: [HACKERS] track_commit_timestamp and COMMIT PREPARED

2015-09-29 Thread Alvaro Herrera
Petr Jelinek wrote: > On 2015-09-29 13:44, Fujii Masao wrote: > >On Tue, Sep 29, 2015 at 12:05 PM, Alvaro Herrera > > wrote: > >-#define RecoveryRequiresBoolParameter(param_name, currValue, masterValue) \ > >-do { \ > >-bool _currValue = (currValue); \ > >-bool _masterValue = (masterValue)

Re: [HACKERS] pageinspect patch, for showing tuple data

2015-09-29 Thread Nikolay Shaplov
В письме от 26 сентября 2015 20:57:25 пользователь Michael Paquier написал: > > So I would consider two options: Either move t_infomask/t_infomask2 > > details > > into storage.sgml or leave as it is. > > The documentation redirects the reader to look at htup_details.h (the > documentation is act

  1   2   >