Re: [HACKERS] inherit support for foreign tables

2015-04-16 Thread Etsuro Fujita
On 2015/03/23 2:57, Tom Lane wrote: Etsuro Fujita fujita.ets...@lab.ntt.co.jp writes: [ fdw-inh-8.patch ] I've committed this with some substantial rearrangements, notably: * As I mentioned earlier, I got rid of a few unnecessary restrictions on foreign tables so as to avoid introducing

[HACKERS] Supporting src/test/modules in MSVC builds

2015-04-16 Thread Michael Paquier
Hi all, As mentioned previously (cab7npqscphafxs2rzeb-fbccjqiknqxjhloztkggim1mf5x...@mail.gmail.com), attached are patches to add support for src/test/modules in MSVC builds, modules whose tests are not supported since they have been moved from contrib/: - 0001 adds support for the build portion.

Re: [HACKERS] inherit support for foreign tables

2015-04-16 Thread Kyotaro HORIGUCHI
Hello, At Thu, 16 Apr 2015 12:20:47 +0900, Etsuro Fujita fujita.ets...@lab.ntt.co.jp wrote in 552f2a8f.2090...@lab.ntt.co.jp On 2015/04/15 3:52, Alvaro Herrera wrote: On 4/14/15 5:49 AM, Etsuro Fujita wrote: postgres=# create foreign table ft1 (c1 int) server myserver options (table_name

Re: [HACKERS] show xl_prev in xlog.c errcontext

2015-04-16 Thread Heikki Linnakangas
On 04/15/2015 11:35 PM, Alvaro Herrera wrote: I found this patch in my local repo that I wrote some weeks or months ago while debugging some XLog corruption problem: it was difficult to pinpoint what XLog record in a long sequence of WAL files was causing a problem, and the displaying the prev

Re: [HACKERS] EvalPlanQual behaves oddly for FDW queries involving system columns

2015-04-16 Thread Etsuro Fujita
On 2015/04/15 2:27, Jim Nasby wrote: On 4/14/15 1:05 AM, Kyotaro HORIGUCHI wrote: As an example, the following operations cause an unexpected result. Those results are indeed surprising, but since we allow it in a direct connection I don't see why we wouldn't allow it in the Postgres FDW...

Re: [HACKERS] show xl_prev in xlog.c errcontext

2015-04-16 Thread Michael Paquier
On Thu, Apr 16, 2015 at 3:25 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 04/15/2015 11:35 PM, Alvaro Herrera wrote: I found this patch in my local repo that I wrote some weeks or months ago while debugging some XLog corruption problem: it was difficult to pinpoint what XLog record in a

Re: [HACKERS] Fix broken Install.bat when target directory contains a space

2015-04-16 Thread Asif Naeem
Hi Michael, I spend spend time look into the patch. Good catch, I am also surprised to see that current Windows install script don’t support spaces in the path. Please see my findings as following i.e. *Without the patch* 1. C:\PG\postgresql\src\tools\msvcinstall C:\PG\postgresql\inst with

[HACKERS] Minor improvement to config.sgml

2015-04-16 Thread Etsuro Fujita
Hi, Attached is a small patch to mark up on with literal in doc/src/sgml/config.sgml. Best regards, Etsuro Fujita diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index b30c68d..0d8624a 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -6752,7 +6752,7 @@

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-16 Thread Andres Freund
On 2015-04-15 17:58:54 +0300, Heikki Linnakangas wrote: When the speculative insertion is finished, write a new kind of a WAL record for that. The record only needs to contain the ctid of the tuple. Replaying that record will clear the flag on the heap tuple that said that it was a speculative

Re: [HACKERS] PATCH: default_index_tablespace

2015-04-16 Thread Greg Stark
On 15 Apr 2015 19:12, Tom Lane t...@sss.pgh.pa.us wrote: I'm afraid this idea is a nonstarter, because it will break existing applications, and in particular existing pg_dump output files, which expect to be able to determine an index's tablespace by setting default_tablespace. (It is *not*

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-16 Thread Greg Stark
On 15 Apr 2015 15:43, Simon Riggs si...@2ndquadrant.com wrote: It all depends upon who is being selfish. Why is a user selfish for not wanting to clean every single block they scan, when the people that made the mess do nothing and go faster 10 minutes from now? Randomly and massively

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-16 Thread Andres Freund
On 2015-04-15 18:53:15 +0300, Heikki Linnakangas wrote: Hmm, ok, I've read the INSERT ... ON CONFLICT UPDATE and logical decoding thread now, and I have to say that IMHO it's a lot more sane to handle this in ReorderBufferCommit() like Peter first did, than to make the main insertion path more

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-16 Thread Pavan Deolasee
On Thu, Apr 16, 2015 at 2:47 PM, Greg Stark st...@mit.edu wrote: On 15 Apr 2015 15:43, Simon Riggs si...@2ndquadrant.com wrote: It all depends upon who is being selfish. Why is a user selfish for not wanting to clean every single block they scan, when the people that made the mess do

Re: [HACKERS] FPW compression leaks information

2015-04-16 Thread Michael Paquier
On Wed, Apr 15, 2015 at 9:42 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Apr 15, 2015 at 9:20 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Apr 15, 2015 at 2:22 PM, Fujii Masao wrote: On Wed, Apr 15, 2015 at 11:55 AM, Michael Paquier wrote: 1) Doc patch to

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2015-04-16 Thread Etsuro Fujita
On 2015/03/05 21:08, Etsuro Fujita wrote: Here is an updated version. The EXPLAIN output has also been improved as discussed in [1]. I noticed that the EXPLAIN for a pushed-down update (delete) on inheritance childs doubly displays Foreign Update (Foreign Delete), one for ForeignScan and

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2015-04-16 Thread Etsuro Fujita
On 2015/04/16 19:57, Amit Langote wrote: On 16-04-2015 PM 07:50, Etsuro Fujita wrote: The EXPLAIN output has also been improved as discussed in [1]. I noticed that the EXPLAIN for a pushed-down update (delete) on inheritance childs doubly displays Foreign Update (Foreign Delete), one for

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2015-04-16 Thread Amit Langote
On 16-04-2015 PM 07:50, Etsuro Fujita wrote: The EXPLAIN output has also been improved as discussed in [1]. I noticed that the EXPLAIN for a pushed-down update (delete) on inheritance childs doubly displays Foreign Update (Foreign Delete), one for ForeignScan and the other for ModifyTable.

Re: [HACKERS] Streaming replication and WAL archive interactions

2015-04-16 Thread Heikki Linnakangas
On 03/01/2015 12:36 AM, Venkata Balaji N wrote: Patch did get applied successfully to the latest master. Can you please rebase. Here you go. On 01/31/2015 03:07 PM, Andres Freund wrote: On 2014-12-19 22:56:40 +0200, Heikki Linnakangas wrote: This add two new archive_modes, 'shared' and

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-16 Thread Alvaro Herrera
Pavan Deolasee wrote: On Thu, Apr 16, 2015 at 2:47 PM, Greg Stark st...@mit.edu wrote: From a holistic point of view the question is how many times is a given hit chain going to need to be followed before it's pruned. Or to put it another way, how expensive is creating a hot chain. Does it

Re: [HACKERS] PATCH: default_index_tablespace

2015-04-16 Thread David Steele
On 4/15/15 11:33 PM, Amit Kapila wrote: On Thu, Apr 16, 2015 at 8:01 AM, Bruce Momjian br...@momjian.us mailto:br...@momjian.us wrote: On Wed, Apr 15, 2015 at 07:12:11PM -0400, Tom Lane wrote: jltal...@adv-solutions.net mailto:jltal...@adv-solutions.net writes: This small patch implements

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-16 Thread Andres Freund
On 2015-04-16 10:20:20 -0300, Alvaro Herrera wrote: I think you're failing to consider that in the patch there is a distinction between read-only page accesses and page updates. During a page update, HOT cleanup is always done even with the patch, so there won't be any additional bloat that

Re: [HACKERS] [COMMITTERS] pgsql: Move pg_upgrade from contrib/ to src/bin/

2015-04-16 Thread Bernd Helmle
--On 15. April 2015 15:02:05 -0400 Andrew Dunstan and...@dunslane.net wrote: We've handled the buildfarm being red for a few days before. People are usually good about applying fixes fairly quickly. Took me some time to get that due to my mail backlog, but i've done the hotfix for dotterel

Re: [HACKERS] Supporting src/test/modules in MSVC builds

2015-04-16 Thread Andrew Dunstan
On 04/16/2015 02:46 AM, Michael Paquier wrote: Hi all, As mentioned previously (cab7npqscphafxs2rzeb-fbccjqiknqxjhloztkggim1mf5x...@mail.gmail.com), attached are patches to add support for src/test/modules in MSVC builds, modules whose tests are not supported since they have been moved from

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-16 Thread Peter Geoghegan
On Thu, Apr 16, 2015 at 2:23 AM, Andres Freund and...@anarazel.de wrote: I'm, completely independent of logical decoding, of the *VERY* strong opinion that 'speculative insertions' should never be visible when looking with normal snapshots. For one it allows to simplify considerations around

Re: [HACKERS] PATCH: default_index_tablespace

2015-04-16 Thread jltallon
I'm afraid this idea is a nonstarter, because it will break existing applications, and in particular existing pg_dump output files, which expect to be able to determine an index's tablespace by setting default_tablespace. (It is *not* adequate that the code falls back to

Re: [HACKERS] initdb -S and tablespaces

2015-04-16 Thread Abhijit Menon-Sen
Hi. Here's a variation of the earlier patch that follows all links in PGDATA. Does this look more like what you had in mind? -- Abhijit From d86888b0d2f5a3a57027d26ce050a3bbb58670d3 Mon Sep 17 00:00:00 2001 From: Abhijit Menon-Sen a...@2ndquadrant.com Date: Thu, 6 Nov 2014 00:45:56 +0530

[HACKERS] Disabling trust/ident authentication configure option

2015-04-16 Thread Bernd Helmle
We have a customer using a patch to harden their PostgreSQL installation (see attached) they would like to contribute. This patch adds the ability to disable trust and ident authentication at compile time via configure options, thus making it impossible to use these authentication methods for

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-16 Thread Pavan Deolasee
On Thu, Apr 16, 2015 at 6:50 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Pavan Deolasee wrote: On Thu, Apr 16, 2015 at 2:47 PM, Greg Stark st...@mit.edu wrote: From a holistic point of view the question is how many times is a given hit chain going to need to be followed before

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-16 Thread Heikki Linnakangas
On 04/16/2015 12:18 PM, Andres Freund wrote: On 2015-04-15 18:53:15 +0300, Heikki Linnakangas wrote: Hmm, ok, I've read the INSERT ... ON CONFLICT UPDATE and logical decoding thread now, and I have to say that IMHO it's a lot more sane to handle this in ReorderBufferCommit() like Peter first

Re: [HACKERS] INSERT ... ON CONFLICT IGNORE (and UPDATE) 3.0

2015-04-16 Thread Peter Geoghegan
On Thu, Apr 16, 2015 at 2:18 AM, Andres Freund and...@anarazel.de wrote: On 2015-04-15 18:53:15 +0300, Heikki Linnakangas wrote: Hmm, ok, I've read the INSERT ... ON CONFLICT UPDATE and logical decoding thread now, and I have to say that IMHO it's a lot more sane to handle this in

Re: [HACKERS] FILTER/WITHIN GROUP vs. expressions; is a HINT possible here?

2015-04-16 Thread Josh Berkus
On 04/16/2015 01:01 PM, David G. Johnston wrote: If this is not covered adequately enough in the documentation then that should be remedied. Did you evaluate the documentation in that light while preparing your blog post? Your response seems very defensive. I'm unclear on why discussing

[HACKERS] FILTER/WITHIN GROUP vs. expressions; is a HINT possible here?

2015-04-16 Thread Josh Berkus
Folks: SELECT device_id, count(*)::INT as present, count(*)::INT FILTER (WHERE valid) as valid_count, mode()::INT WITHIN GROUP (order by val) as mode, percentile_disc(0.5)::INT WITHIN GROUP (order by val) as median FROM dataflow_0913

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-16 Thread Alvaro Herrera
Andres Freund wrote: On 2015-04-16 10:20:20 -0300, Alvaro Herrera wrote: I think you're failing to consider that in the patch there is a distinction between read-only page accesses and page updates. During a page update, HOT cleanup is always done even with the patch, so there won't be

Re: [HACKERS] FILTER/WITHIN GROUP vs. expressions; is a HINT possible here?

2015-04-16 Thread David G. Johnston
On Thu, Apr 16, 2015 at 1:06 PM, Josh Berkus j...@agliodbs.com wrote: On 04/16/2015 01:01 PM, David G. Johnston wrote: If this is not covered adequately enough in the documentation then that should be remedied. Did you evaluate the documentation in that light while preparing your blog

Re: [HACKERS] inherit support for foreign tables

2015-04-16 Thread David Fetter
On Wed, Apr 15, 2015 at 09:35:05AM +0900, Kyotaro HORIGUCHI wrote: Hi, Before suppressing the symptom, I doubt the necessity and/or validity of giving foreign tables an ability to be a parent. Is there any reasonable usage for the ability? I think we should choose to inhibit foreign

Re: [HACKERS] Supporting src/test/modules in MSVC builds

2015-04-16 Thread Alvaro Herrera
Andrew Dunstan wrote: Thanks for doing this. Yes, much appreciated. It looks good, and if you've tested it I'm satisfied. I suggest that we apply patches 1 and 2 immediately. AUIU they don't require any changes to the buildfarm, as the MSVC build process will automatically build and

Re: [HACKERS] Assertion failure when streaming logical changes

2015-04-16 Thread Heikki Linnakangas
On 04/07/2015 03:54 PM, Andres Freund wrote: On 2015-04-07 17:22:12 +0800, Craig Ringer wrote: It might be a good idea to apply this if nothing better is forthcoming. Logical decoding in WALsenders is broken at the moment. Yes. Committed. - Heikki -- Sent via pgsql-hackers mailing list

Re: [HACKERS] FILTER/WITHIN GROUP vs. expressions; is a HINT possible here?

2015-04-16 Thread Josh Berkus
On 04/16/2015 01:18 PM, David G. Johnston wrote: On Thu, Apr 16, 2015 at 1:06 PM, Josh Berkus j...@agliodbs.com mailto:j...@agliodbs.comwrote: On 04/16/2015 01:01 PM, David G. Johnston wrote: If this is not covered adequately enough in the documentation then that should be

Re: [HACKERS] FILTER/WITHIN GROUP vs. expressions; is a HINT possible here?

2015-04-16 Thread David G. Johnston
On Thu, Apr 16, 2015 at 12:22 PM, Josh Berkus j...@agliodbs.com wrote: Folks: SELECT device_id, count(*)::INT as present, count(*)::INT FILTER (WHERE valid) as valid_count, mode()::INT WITHIN GROUP (order by val) as mode,

Re: [HACKERS] Turning off HOT/Cleanup sometimes

2015-04-16 Thread Simon Riggs
On 16 April 2015 at 15:21, Andres Freund and...@anarazel.de wrote: On 2015-04-16 10:20:20 -0300, Alvaro Herrera wrote: I think you're failing to consider that in the patch there is a distinction between read-only page accesses and page updates. During a page update, HOT cleanup is always

[HACKERS] Performance tuning assisted by a GUI application

2015-04-16 Thread Jacek Wielemborek
Hello, (Please pardon me if this is offtopic and I should send it to another mailing list instead) I had a brief discussion on #postgresql and thought that perhaps there might be a need for a tool that would enable a fine-tuning of PostgreSQL performance settings by conveniently testing them

Re: [HACKERS] Supporting src/test/modules in MSVC builds

2015-04-16 Thread Alvaro Herrera
Andrew Dunstan wrote: On 04/16/2015 07:42 PM, Michael Paquier wrote: Then if all goes well we can apply the third patch and I'll fix the buildfarm client for the forthcoming release to run the tests on MSVC builds. Nothing will break in the meantime - the tests just won't get run until

[HACKERS] pg_upgrade in 9.5 broken for adminpack

2015-04-16 Thread Jeff Janes
pg_upgrade was recently broken for use upgrading from a system with adminpack installed. Breaking commit is: commit 30982be4e5019684e1772dd9170aaa53f5a8e894 Author: Peter Eisentraut pete...@gmx.net Integrate pg_upgrade_support module into backend from pg_upgrade_dump_12870.log

Re: Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2015-04-16 Thread Kouhei Kaigai
Hanada-san, I merged explain patch into foreign_join patch. Now v12 is the latest patch. It contains many garbage lines... Please ensure the patch is correctly based on the latest master + custom_join patch. Thanks, -- NEC Business Creation Division / PG-Strom Project KaiGai Kohei

Re: [HACKERS] Supporting src/test/modules in MSVC builds

2015-04-16 Thread Michael Paquier
On Fri, Apr 17, 2015 at 4:47 AM, Alvaro Herrera wrote: Andrew Dunstan wrote: It looks good, and if you've tested it I'm satisfied. I suggest that we apply patches 1 and 2 immediately. AUIU they don't require any changes to the buildfarm, as the MSVC build process will automatically build and

Re: [HACKERS] Supporting src/test/modules in MSVC builds

2015-04-16 Thread Andrew Dunstan
On 04/16/2015 07:42 PM, Michael Paquier wrote: Then if all goes well we can apply the third patch and I'll fix the buildfarm client for the forthcoming release to run the tests on MSVC builds. Nothing will break in the meantime - the tests just won't get run until the new client version is

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

2015-04-16 Thread Amit Langote
Fujita-san, On 16-04-2015 PM 08:40, Etsuro Fujita wrote: From what I see in Tom's commit message[0] for FTI patch, this shouldn't be, right? To be specific, there should be Foreign Scan there as per the commit. Am I missing something? As shown in the below example, this patch doesn't

Re: [HACKERS] pg_upgrade in 9.5 broken for adminpack

2015-04-16 Thread Bruce Momjian
On Thu, Apr 16, 2015 at 07:33:50PM -0700, Jeff Janes wrote: pg_upgrade was recently broken for use upgrading from a system with adminpack installed. Breaking commit is: commit 30982be4e5019684e1772dd9170aaa53f5a8e894 Author: Peter Eisentraut pete...@gmx.net     Integrate

Re: [HACKERS] reparsing query

2015-04-16 Thread Tatsuo Ishii
It is not difficult to output parsed query in some tool readable format but it comes with a maintain overhead: once tools rely on it, we have to conform to some schema continuously, like the xml/xmlns. Do we want to take this? Depends on how far the tools can go with this exposed information.

Re: [HACKERS] FILTER/WITHIN GROUP vs. expressions; is a HINT possible here?

2015-04-16 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: ERROR: syntax error at or near FILTER LINE 4: count(*)::INT FILTER (WHERE valid) as valid_count, The error is right, that's invalid syntax. I can't insert a ::INT between the aggregate() and FILTER. However, the error message is also

Re: [HACKERS] FILTER/WITHIN GROUP vs. expressions; is a HINT possible here?

2015-04-16 Thread Josh Berkus
On 04/16/2015 04:36 PM, Tom Lane wrote: You could imagine teaching yyerror() to have some SQL-specific knowledge that it could apply by comparing the current lookahead token to the current parse state stack ... but neither of those things are exposed to it by Bison. We could probably get the

Re: [HACKERS] [PERFORM] pushing order by + limit to union subqueries

2015-04-16 Thread Qingqing Zhou
On Sat, Feb 28, 2015 at 8:24 AM, Tom Lane t...@sss.pgh.pa.us wrote: There would be cases where that would be a win, and there would be cases where it wouldn't be, so I'd not be in favor of making the transformation blindly. Unfortunately, given the current state of the planner that's all we