Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-08-27 Thread Etsuro Fujita
On 2015/08/27 11:08, Kouhei Kaigai wrote: On 2015/08/26 18:01, Kouhei Kaigai wrote: You may think execution of alternative plan is the best way for EPQ rechecks, however, other folks may think their own implementation is the best for EPQ rechecks. I never argue which approach is better. What I p

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-08-27 Thread Kouhei Kaigai
> > As I have said repeatedly, it is software design decision by the author > > of extension. Even if it consumes 100 times larger memory and 1000 times > > slower, it is his decision and responsibility. > > Why he has to be forced to use a particular logic despite his intension? > > I don't under

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-08-27 Thread Etsuro Fujita
On 2015/08/27 16:52, Kouhei Kaigai wrote: I wrote: I don't understand what you proposed, What I'm talking about is philosophy of software/interface design. I understand EPQ recheck by alternative plan is "one" reasonable way, however, people often have different ideas and may be better than yo

Re: [HACKERS] psql - better support pipe line

2015-08-27 Thread Shulgin, Oleksandr
On Wed, Aug 26, 2015 at 5:44 PM, Pavel Stehule wrote: > Hi > > 2015-08-25 17:21 GMT+02:00 Tom Lane : > >> Jim Nasby writes: >> > What I've had problems with is trying to correlate psql specified >> > connection attributes with things like DBI. It would be nice if there >> > was a way to get a fu

Re: [HACKERS] Custom Scans and private data

2015-08-27 Thread Andres Freund
On 2015-08-26 23:55:16 +, Kouhei Kaigai wrote: > - _copyCustomScan() needs to allow to allocate larger than sizeof(CustomScan), > according to the requirement by custom-scan provider. I think it's somewhat nice to allow larger objects, but I don't think it's absolutely necessary. > - We may

Re: [HACKERS] checkpointer continuous flushing

2015-08-27 Thread Andres Freund
On 2015-08-18 09:08:43 +0200, Fabien COELHO wrote: > Please find attached a v10, where I have reviewed comments for style & > contents, and also slightly extended the documentation about the flush > option to hint that it is essentially useful for high write loads. Without > further data, I think i

Re: [HACKERS] missing locking in at least INSERT INTO view WITH CHECK

2015-08-27 Thread Andres Freund
On 2013-10-24 20:58:55 +0100, Dean Rasheed wrote: > On 24 October 2013 18:28, Andres Freund wrote: > > On 2013-10-23 21:20:58 +0100, Dean Rasheed wrote: > >> On 23 October 2013 21:08, Andres Freund wrote: > >> > On 2013-10-23 20:51:27 +0100, Dean Rasheed wrote: > >> >> Hmm, my first thought is th

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

2015-08-27 Thread Etsuro Fujita
On 2015/08/27 17:30, Etsuro Fujita wrote: I think we would probably need others' opinions about this issue. Attached is an updated version of the patch [1]. I'd be happy if it helps people discuss about this issue. Changes: * rebased to HEAD. * add some more docs and comments. * fix a bug i

Re: [HACKERS] checkpointer continuous flushing

2015-08-27 Thread Fabien COELHO
Hello Andres, Please find attached a v10, where I have reviewed comments for style & contents, and also slightly extended the documentation about the flush option to hint that it is essentially useful for high write loads. Without further data, I think it is not obvious to give more definite

Re: [HACKERS] checkpointer continuous flushing

2015-08-27 Thread Andres Freund
On 2015-08-27 14:32:39 +0200, Fabien COELHO wrote: > >v10b misses the checkpoint_sort part of the patch, and thus cannot be > >applied. > > Yes, indeed, the second part is expected to be applied on top of v10a. Oh, sorry. I'd somehow assumed they were two variants of the same patch (one with "sli

[HACKERS] AcquireRewriteLocks/acquireLocksOnSubLinks vs. rowsecurity

2015-08-27 Thread Andres Freund
Hi, The locking around rowsecurity policy expressions seems to be insufficient: SELECT * FROM document WHERE f_leak(dtitle) ORDER BY did; WARNING: RelationIdGetRelation(247984) without holding lock on the relation WARNING: relation_open(247984, NoLock) of relation "uaccount" without previously

[HACKERS] What does RIR as in fireRIRrules stand for?

2015-08-27 Thread Andres Freund
Hi, I searched the current code, all diffs to the current code, and the mailing list, but still haven't got an actual clue what RIR is supposed to stand for. There's a few things that come to mind (Rewrite Instead Rule?), but none of them seem to make too much sense. Greetings, Andres Freund -

Re: [HACKERS] WIP: Rework access method interface

2015-08-27 Thread Alexander Korotkov
On Wed, Aug 26, 2015 at 7:20 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Wed, Aug 26, 2015 at 6:50 PM, Tom Lane wrote: > >> Alexander Korotkov writes: >> > OK. So, as we mentioned before, if we need to expose something of am >> > parameters at SQL-level then we need to write

Re: [HACKERS] What does RIR as in fireRIRrules stand for?

2015-08-27 Thread David G. Johnston
Pure speculation but the word fire makes me think of triggers, not rule. Fire Referential Integrity Rules (Triggers)? David J. On Thu, Aug 27, 2015 at 9:13 AM, Andres Freund wrote: > Hi, > > I searched the current code, all diffs to the current code, and the > mailing list, but still haven't g

Re: [HACKERS] psql - better support pipe line

2015-08-27 Thread Pavel Stehule
2015-08-27 11:54 GMT+02:00 Shulgin, Oleksandr : > > > On Wed, Aug 26, 2015 at 5:44 PM, Pavel Stehule > wrote: > >> Hi >> >> 2015-08-25 17:21 GMT+02:00 Tom Lane : >> >>> Jim Nasby writes: >>> > What I've had problems with is trying to correlate psql specified >>> > connection attributes with thi

Re: [HACKERS] What does RIR as in fireRIRrules stand for?

2015-08-27 Thread Tom Lane
Andres Freund writes: > I searched the current code, all diffs to the current code, and the > mailing list, but still haven't got an actual clue what RIR is supposed > to stand for. There's a few things that come to mind (Rewrite Instead > Rule?), but none of them seem to make too much sense. htt

Re: [HACKERS] psql - better support pipe line

2015-08-27 Thread Pavel Stehule
2015-08-27 11:54 GMT+02:00 Shulgin, Oleksandr : > > > On Wed, Aug 26, 2015 at 5:44 PM, Pavel Stehule > wrote: > >> Hi >> >> 2015-08-25 17:21 GMT+02:00 Tom Lane : >> >>> Jim Nasby writes: >>> > What I've had problems with is trying to correlate psql specified >>> > connection attributes with thi

Re: [HACKERS] What does RIR as in fireRIRrules stand for?

2015-08-27 Thread Andres Freund
On 2015-08-27 15:13:52 +0200, Andres Freund wrote: > I searched the current code, all diffs to the current code, and the > mailing list, but still haven't got an actual clue what RIR is supposed > to stand for. There's a few things that come to mind (Rewrite Instead > Rule?), but none of them seem

Re: [HACKERS] checkpointer continuous flushing

2015-08-27 Thread Fabien COELHO
v10b misses the checkpoint_sort part of the patch, and thus cannot be applied. Yes, indeed, the second part is expected to be applied on top of v10a. Oh, sorry. I'd somehow assumed they were two variants of the same patch (one with "slim" sorting and the other without). The idea is that as

Re: [HACKERS] Reduce ProcArrayLock contention

2015-08-27 Thread Amit Kapila
On Tue, Aug 25, 2015 at 5:21 PM, Amit Kapila wrote: > > On Thu, Aug 20, 2015 at 3:49 PM, Andres Freund wrote: > > How hard did you try checking whether this causes regressions? This > > increases the number of atomics in the commit path a fair bit. I doubt > > it's really bad, but it seems like a

Re: [HACKERS] What does RIR as in fireRIRrules stand for?

2015-08-27 Thread Tom Lane
Andres Freund writes: > On 2015-08-27 09:43:09 -0400, Tom Lane wrote: >> http://www.postgresql.org/message-id/3e887762.5b68f...@yahoo.com > Oops. I saw that message but thought, based on the subject, it'd a scam > mail... > Can we either add a comment to the effect of Jan's message, or just > re

Re: [HACKERS] What does RIR as in fireRIRrules stand for?

2015-08-27 Thread Andres Freund
On 2015-08-27 09:43:09 -0400, Tom Lane wrote: > Andres Freund writes: > > I searched the current code, all diffs to the current code, and the > > mailing list, but still haven't got an actual clue what RIR is supposed > > to stand for. There's a few things that come to mind (Rewrite Instead > > Ru

Re: [HACKERS] [BUGS] Compile fails on AIX 6.1

2015-08-27 Thread Tom Lane
A year ago we had a thread about assembler syntax errors on AIX, but it died off when the original complainant stopped responding. I was recently contacted off-list by Steve Underwood, who was seeing the same symptoms on AIX 7.1. After some investigation, we found that the problem is that IBM's a

Re: [HACKERS] What does RIR as in fireRIRrules stand for?

2015-08-27 Thread Steve Crawford
Candidate for Appendix K? Cheers, Steve On Thu, Aug 27, 2015 at 6:52 AM, Tom Lane wrote: > Andres Freund writes: > > On 2015-08-27 09:43:09 -0400, Tom Lane wrote: > >> http://www.postgresql.org/message-id/3e887762.5b68f...@yahoo.com > > > Oops. I saw that message but thought, based on the subj

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-27 Thread Andres Freund
On 2015-08-16 05:58:17 +0200, Andres Freund wrote: > On 2015-08-15 23:50:09 -0400, Noah Misch wrote: > > The policy would then be > > (already is?) to wrap in "#ifdef FRONTEND" any inline function that uses a > > backend symbol. When a header is dedicated to such functions, we might > > avoid > >

Re: [HACKERS] What does RIR as in fireRIRrules stand for?

2015-08-27 Thread Tom Lane
Steve Crawford writes: > Candidate for Appendix K? Meh ... it's not a user-visible notation, in fact it only appears in rewriteHandler.c AFAICS. I think an explanation in fireRIRrules' header comment would be fine. regards, tom lane -- Sent via pgsql-hackers mailing l

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-27 Thread Tom Lane
Andres Freund writes: > As Noah I think it'd be good if we, over time, started to document a few > more things one currently have to pick up over time. I'm wondering > whether these should be subsections under a new sect1 ('Code Structure'? > Don't like that much), or all independent sect1s. "Str

Re: [HACKERS] Resource Owner reassign Locks

2015-08-27 Thread Tom Lane
I went ahead and pushed this into 9.2 and 9.1. It did not apply at all to 9.0, though, as there evidently was some refactoring affecting LockReassignCurrentOwner() between 9.0 and 9.1. We could possibly have applied some additional patches to 9.0, but I think that would be stretching the argument

[HACKERS] Re: [COMMITTERS] pgsql: psql: show proper row count in \x mode for zero-column output

2015-08-27 Thread Bruce Momjian
On Thu, Jun 25, 2015 at 10:42:54AM -0400, Peter Eisentraut wrote: > On 3/24/15 9:04 PM, Bruce Momjian wrote: > > psql: show proper row count in \x mode for zero-column output > > > > Also, fix pager enable selection for such cases, and other cleanups for > > zero-column output. > > > > Report by

Re: [HACKERS] missing locking in at least INSERT INTO view WITH CHECK

2015-08-27 Thread Dean Rasheed
On 27 August 2015 at 12:18, Andres Freund wrote: > On 2013-10-24 20:58:55 +0100, Dean Rasheed wrote: >> So I >> think the only thing missing from rewriteTargetView() is to lock any >> relations from any sublink subqueries in viewquery using >> acquireLocksOnSubLinks() -- patch attached. > > This i

Re: [HACKERS] missing locking in at least INSERT INTO view WITH CHECK

2015-08-27 Thread Andres Freund
On 2015-08-27 18:37:10 +0100, Dean Rasheed wrote: > On 27 August 2015 at 12:18, Andres Freund wrote: > >> > >> /* > >> + * If the view query contains any sublink subqueries, we should also > >> + * acquire locks on any relations they refer to. We know that there > >> won't > >> +

Re: [HACKERS] AcquireRewriteLocks/acquireLocksOnSubLinks vs. rowsecurity

2015-08-27 Thread Dean Rasheed
On 27 August 2015 at 13:49, Andres Freund wrote: > Hi, > > The locking around rowsecurity policy expressions seems to be > insufficient: > SELECT * FROM document WHERE f_leak(dtitle) ORDER BY did; > WARNING: RelationIdGetRelation(247984) without holding lock on the relation > WARNING: relation_o

Re: [HACKERS] missing locking in at least INSERT INTO view WITH CHECK

2015-08-27 Thread Dean Rasheed
On 27 August 2015 at 18:44, Andres Freund wrote: > On 2015-08-27 18:37:10 +0100, Dean Rasheed wrote: >> I have a feeling that RLS might suffer from the same issue, but I >> haven't looked yet. > > There's a similar issue there, yes: > http://archives.postgresql.org/message-id/20150827124931.GD159

Re: [HACKERS] missing locking in at least INSERT INTO view WITH CHECK

2015-08-27 Thread Andres Freund
On 2015-08-27 19:19:35 +0100, Dean Rasheed wrote: > It also seems to me that this warning has proved its worth Same here - I plan to re-submit it. Perhaps the number of bugs it found convinces Tom, after I address some of his points. > although I don't think it's something a production build shou

Re: [HACKERS] missing locking in at least INSERT INTO view WITH CHECK

2015-08-27 Thread Dean Rasheed
On 27 August 2015 at 19:29, Andres Freund wrote: > On 2015-08-27 19:19:35 +0100, Dean Rasheed wrote: >> It also seems to me that this warning has proved its worth > > Same here - I plan to re-submit it. Perhaps the number of bugs it found > convinces Tom, after I address some of his points. > >> a

Re: [HACKERS] Our trial to TPC-DS but optimizer made unreasonable plan

2015-08-27 Thread Qingqing Zhou
On Wed, Aug 26, 2015 at 5:28 PM, Tom Lane wrote: > > After looking at the code a bit, IMO the most reasonable thing to do is to > include this transformation in inline_set_returning_functions(), perhaps > renaming it to something like inline_srfs_and_ctes(). > This is essentially the same as my c

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-27 Thread Robert Haas
On Thu, Aug 27, 2015 at 4:31 PM, Tom Lane wrote: >> That's not yet perfect, but shows what I'm thinking of. Comments? > > Needs a bit of copy-editing in places, but +1 overall. Yeah. I bet there's a lot more useful stuff we could include also, but everything Andres mentioned is certainly good to

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-27 Thread Tom Lane
Robert Haas writes: > Yeah. I bet there's a lot more useful stuff we could include also, > but everything Andres mentioned is certainly good to put in there. > Alternatively, some of this stuff could go into a README file instead > of the documentation, but I think we've been leaning toward > doc

[HACKERS] Spurious standby query cancellations

2015-08-27 Thread Jeff Janes
In ResolveRecoveryConflictWithLock, there is the comment: /* * If blowing away everybody with conflicting locks doesn't work, after * the first two attempts then we just start blowing everybody away until * it does work. But what it does is something different than that. At l

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-08-27 Thread Bruce Momjian
On Wed, Jul 1, 2015 at 11:21:47AM -0700, Josh Berkus wrote: > All: > > Replying to multiple people below. > > On 07/01/2015 07:15 AM, Fujii Masao wrote: > > On Tue, Jun 30, 2015 at 2:40 AM, Josh Berkus wrote: > >> You're confusing two separate things. The primary manageability problem > >> has

Re: [HACKERS] 9.5 feature count

2015-08-27 Thread Bruce Momjian
On Fri, Jun 26, 2015 at 11:13:19AM -0700, Peter Geoghegan wrote: > On Fri, Jun 26, 2015 at 11:09 AM, Alvaro Herrera > wrote: > > Maybe there's a reasonable way to measure releases (my 8.0 is bigger > > than your 9.1!), but I don't think this is it. > > > I agree with the sentiment, but I don't t

Re: [HACKERS] Custom Scans and private data

2015-08-27 Thread Robert Haas
On Wed, Aug 26, 2015 at 11:23 PM, Andres Freund wrote: >> > Looking at >> > postgres_fdw and the pg-strom example linked upthread imo pretty clearly >> > shows how ugly this is. There's also the rather noticeable difference >> > that we already have callbacks in the node for custom scans (used by

Re: [HACKERS] Custom Scans and private data

2015-08-27 Thread Tom Lane
Robert Haas writes: > On Wed, Aug 26, 2015 at 11:23 PM, Andres Freund wrote: >> While that comment made me laugh, I'm not really sure why my examples >> are bait. One is the probably most used fdw, the other the only user of >> the custom scan interface I know of. > I suspect what Tom is getting

Re: [HACKERS] PostgreSQL for VAX on NetBSD/OpenBSD

2015-08-27 Thread Greg Stark
For completeness, here's the regression tests from the conrttrib modules. I haven't looked into why earthdistance is coming up with such odd results but I suspect it all comes from the same arithmetic source. I don't see any surprising internal dependencies on ieee floating point. For what it's wo

Re: [HACKERS] PostgreSQL for VAX on NetBSD/OpenBSD

2015-08-27 Thread Tom Lane
Greg Stark writes: > For completeness, here's the regression tests from the conrttrib > modules. I haven't looked into why earthdistance is coming up with > such odd results but I suspect it all comes from the same arithmetic > source. I don't see any surprising internal dependencies on ieee > flo

Re: [HACKERS] Custom Scans and private data

2015-08-27 Thread Andres Freund
Hi, On 2015-08-27 18:59:13 -0400, Tom Lane wrote: > But I do think that letting custom-scan extensions fool about with > the semantics of plan-copying (and plan-serialization for that matter) > is a risky choice that is not justified by some marginal gains in code > readability To me the likeliho

Re: [HACKERS] PostgreSQL for VAX on NetBSD/OpenBSD

2015-08-27 Thread Tom Lane
I wrote: > Greg Stark writes: >> For what it's worth there are a number of mentions in the docs of >> platforms that have non-ieee semantics behaving differently so I >> wouldn't say we don't support such platforms. If we could avoid the >> test failures without weakening the tests for other platf

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-27 Thread Alvaro Herrera
Tom Lane wrote: > It strikes me that the information in backend/utils/mmgr/README would be > a good candidate to move into the Internals SGML, too. Almost none of > that is "stuff you only care about when reading utils/mmgr/". Completely agreed. -- Álvaro Herrerahttp://www.2ndQ

Re: [HACKERS] Custom Scans and private data

2015-08-27 Thread Andres Freund
On 2015-08-27 23:23:54 +0100, Robert Haas wrote: > I think it's great that you are experimenting with the feature, and I > think we ought to try to make it better, even if that requires > incompatible changes to the API. Yea, I think it's too early to consider this API stable at all. That's imo ju

[HACKERS] Core dump with nested CREATE TEMP TABLE

2015-08-27 Thread Jim Nasby
I don't have an independent reproduction yet (though make test in [1] should reproduce this). I haven't actually been able to reproduce by hand yet, but pgtap has something to do with this. This is affecting at least 9.4 and a fairly recent HEAD. -- Bits from top of test/sql/base.sql \i test/

[HACKERS] pg_extension_config_dump() doesn't register owned sequences

2015-08-27 Thread Jim Nasby
Is it intentional that pg_extension_config_dump() doesn't also add sequences owned by a table? If not, I'll work on a patch. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.

Re: [HACKERS] psql - better support pipe line

2015-08-27 Thread Jim Nasby
On 8/27/15 8:37 AM, Pavel Stehule wrote: I don't think we can detect and remove the default values from this output in a reliable way? This is pretty difficult - any parameter can be important, and hard to identify default values on client side without connect to server side. I don't s

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-08-27 Thread Jim Nasby
On 8/4/15 9:18 PM, Michael Paquier wrote: >And using a new file with multi line support would be good. This file just contains a JSON blob, hence we just need to fetch its content entirely and then let the server parse it using the existing facilities. It sounds like there's other places where

Re: [HACKERS] pg_extension_config_dump() doesn't register owned sequences

2015-08-27 Thread Michael Paquier
On Fri, Aug 28, 2015 at 10:16 AM, Jim Nasby wrote: > Is it intentional that pg_extension_config_dump() doesn't also add sequences > owned by a table? If not, I'll work on a patch. I recall this behavior to be intentional. They are taken into account only if they are explicitly marked with pg_exte

[HACKERS] patch: version_stamp.pl: Add Git commit info to version if 'git' is specified

2015-08-27 Thread Øyvind A . Holm
Hi, this is a patch for src/tools/version_stamp.pl that adds the current Git commit information from "git describe --tags --long" to the current version string if "git" is specified on the command line. I've been testing for regressions and such lately (none found, yay), and the current output

Re: [HACKERS] patch: version_stamp.pl: Add Git commit info to version if 'git' is specified

2015-08-27 Thread Michael Paquier
On Fri, Aug 28, 2015 at 12:14 PM, Øyvind A. wrote: > this is a patch for src/tools/version_stamp.pl that adds the current Git > commit information from "git describe --tags --long" to the current > version string if "git" is specified on the command line. I've been > testing for regressions and su

Re: [HACKERS] patch: version_stamp.pl: Add Git commit info to version if 'git' is specified

2015-08-27 Thread Tom Lane
=?utf-8?Q?=C3=98yvind_A=2E?= Holm writes: > this is a patch for src/tools/version_stamp.pl that adds the current Git > commit information from "git describe --tags --long" to the current > version string if "git" is specified on the command line. Salesforce did something similar in their intern

Re: [HACKERS] patch: version_stamp.pl: Add Git commit info to version if 'git' is specified

2015-08-27 Thread Tom Lane
Michael Paquier writes: > ... you can already append > a custom string after the version string with --with-extra-version in > configure. Here is for example one I use for development: > GIT_CURRENT=`cd $PGSOURCE && git rev-parse --short HEAD` > ./configure --with-extra-version=-$GIT_CURRENT Yeah

Re: [HACKERS] patch: version_stamp.pl: Add Git commit info to version if 'git' is specified

2015-08-27 Thread Øyvind A . Holm
On 28 August 2015 at 06:17, Tom Lane wrote: > Michael Paquier writes: > > ... you can already append > > a custom string after the version string with --with-extra-version > > in configure. Here is for example one I use for development: > > GIT_CURRENT=`cd $PGSOURCE && git rev-parse --short HEAD`

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-27 Thread Noah Misch
On Thu, Aug 27, 2015 at 11:31:44AM -0400, Tom Lane wrote: > Andres Freund writes: > > As Noah I think it'd be good if we, over time, started to document a few > > more things one currently have to pick up over time. I'm wondering > > whether these should be subsections under a new sect1 ('Code Str

Re: [HACKERS] patch: version_stamp.pl: Add Git commit info to version if 'git' is specified

2015-08-27 Thread Fabien COELHO
Salesforce did something similar in their internal build, and TBH I do not find it a good idea. The basic problem is it's completely misleading to equate the last commit with the source you actually built from, because that might not have been an unmodified file set. Indeed. What I've done