[HACKERS] checkpointer continuous flushing

2015-06-01 Thread Fabien COELHO
Hello pg-devs, This patch is a simplified and generalized version of Andres Freund's August 2014 patch for flushing while writing during checkpoints, with some documentation and configuration warnings added. For the initial patch, see:

Re: [HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-01 Thread Andres Freund
On 2015-05-31 07:51:59 -0400, Robert Haas wrote: 1) We continue determining the oldest SlruScanDirectory(SlruScanDirCbFindEarliest) on the master to find the oldest offsets segment to truncate. Alternatively, if we determine it to be safe, we could use oldestMulti to find that.

Re: [HACKERS] [NOVICE] psql readline Tab insert tab

2015-06-01 Thread Albe Laurenz
Hans Ginzel wrote: how to make psql (readline) to insert tab when Tab is pressed? E.g. for pasting. I know, there is -n option. But then the history is not accessible. It could be done by adding the following lines to your ~/.inputrc file: $if Psql TAB: tab-insert $endif Great! Thank

Re: [HACKERS] problems on Solaris

2015-06-01 Thread Andres Freund
On 2015-05-31 08:00:44 -0400, Robert Haas wrote: On Sat, May 30, 2015 at 7:09 PM, Andres Freund and...@anarazel.de wrote: On 2015-05-27 21:23:34 -0400, Robert Haas wrote: Oh wow, that's bad, and could explain a couple of the problems we're seing. One possible way to fix is to replace the

Re: [HACKERS] Memory leak with XLogFileCopy since de768844 (WAL file with .partial)

2015-06-01 Thread Michael Paquier
On Thu, May 28, 2015 at 9:09 PM, Michael Paquier michael.paqu...@gmail.com wrote: Since commit de768844, XLogFileCopy of xlog.c returns to caller a pstrdup'd string that can be used afterwards for other things. XLogFileCopy is used in only one place, and it happens that the result string is

[HACKERS] Arguable RLS security bug, EvalPlanQual() paranoia

2015-06-01 Thread Peter Geoghegan
Very minor concern about RLS This existing ExecUpdate() comment seems a little inaccurate: /* * Check any RLS UPDATE WITH CHECK policies * * If we generate a new candidate tuple after EvalPlanQual testing, we * must loop back here and recheck any RLS policies and

Re: [HACKERS] [COMMITTERS] pgsql: Row-Level Security Policies (RLS)

2015-06-01 Thread Dean Rasheed
On 28 May 2015 at 08:51, Stephen Frost sfr...@snowman.net wrote: * Dean Rasheed (dean.a.rash...@gmail.com) wrote: Actually I think a new boolean field is unnecessary, and probably the policy_id field is too. Re-reading the code in rowsecurity.c, I think it could use a bit of refactoring.

Re: [HACKERS] [patch] PL/Python is too lossy with floats

2015-06-01 Thread Marko Kreen
On Wed, Mar 11, 2015 at 9:49 PM, Peter Eisentraut pete...@gmx.net wrote: On 3/3/15 9:32 AM, Marko Kreen wrote: PL/Python uses str(v) to convert float data, but is lossy by design. Only repr(v) is guaranteed to have enough precision to make floats roundtrip properly: committed In 9.3 and

Re: [HACKERS] psql :: support for \ev viewname and \sv viewname

2015-06-01 Thread Jeevan Chalke
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, failed I have reviewed this patch. Most of the code is just

Re: [CORE] [HACKERS] postpone next week's release

2015-06-01 Thread Jim Nasby
On 5/29/15 5:28 PM, Bruce Momjian wrote: could expect that anyone committing a user-visible semantics change should update the release notes themselves. Yes, that would be nice. FWIW, I've always wondered why we don't create an empty next-version release notes as part of stamping a major

Re: [CORE] [HACKERS] postpone next week's release

2015-06-01 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2015-06-01 12:32:21 -0400, Tom Lane wrote: There are good reasons to write the release notes all in one batch: otherwise you don't get any uniformity of editorial style. I agree that that's a good reason for major releases, I do however wonder if

Re: [CORE] [HACKERS] postpone next week's release

2015-06-01 Thread Tom Lane
Jim Nasby jim.na...@bluetreble.com writes: FWIW, I've always wondered why we don't create an empty next-version release notes as part of stamping a major release and expect patch authors to add to it. I realize that likely creates merge conflicts, but that seems less work than doing it all

Re: [CORE] [HACKERS] postpone next week's release

2015-06-01 Thread Andres Freund
On 2015-06-01 12:32:21 -0400, Tom Lane wrote: There are good reasons to write the release notes all in one batch: otherwise you don't get any uniformity of editorial style. I agree that that's a good reason for major releases, I do however wonder if it'd not be a good idea to do differently for

Re: [HACKERS] [CORE] postpone next week's release

2015-06-01 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Fri, May 29, 2015 at 8:02 PM, Robert Haas robertmh...@gmail.com wrote: I think we should postpone next week's release. I'm a bit split on this. We *definitely* don't want to release the multixact fix without it being carefully reviewed, that's

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-06-01 Thread Peter Eisentraut
On 5/31/15 1:29 PM, Joel Jacobson wrote: My suggestion is to use pg_xjournal instead of pg_xlog If we're going to make any changes in this area, I would like to see a more comprehensive solution for separating user-editable files from internal files. (E.g., make subdirectories etc and var; just

[HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-01 Thread Robert Haas
On Mon, Jun 1, 2015 at 12:46 AM, Noah Misch n...@leadboat.com wrote: Incomplete review, done in a relative rush: Thanks. On Fri, May 29, 2015 at 03:08:11PM -0400, Robert Haas wrote: OK, here's a patch. Actually two patches, differing only in whitespace, for 9.3 and for master (ha!). I now

Re: [HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-01 Thread Robert Haas
On Mon, Jun 1, 2015 at 4:58 AM, Andres Freund and...@anarazel.de wrote: I'm probably biased here, but I think we should finish reviewing, testing, and committing my patch before we embark on designing this. Probably, yes. I am wondering whether doing this immediately won't end up making some

[HACKERS] Release notes committed

2015-06-01 Thread Tom Lane
I've committed the release notes for today's releases, if anyone wants to take a look. Fortunately, there's not much to them. Unfortunately, I need to start the wrap process in about an hour, so there's not much time to review either :-( regards, tom lane -- Sent via

Re: [HACKERS] [NOVICE] psql readline Tab insert tab

2015-06-01 Thread Peter Eisentraut
On 6/1/15 7:00 AM, Albe Laurenz wrote: Hans Ginzel wrote: how to make psql (readline) to insert tab when Tab is pressed? E.g. for pasting. I know, there is -n option. But then the history is not accessible. It could be done by adding the following lines to your ~/.inputrc file: $if Psql

Re: [HACKERS] Join Filter vs. Index Cond (performance regression 9.1-9.2+/HEAD)

2015-06-01 Thread Andrew Gierth
Tom == Tom Lane t...@sss.pgh.pa.us writes: Tom Once you're down to an estimate of one row retrieved, adding Tom additional index conditions simply increases the cost (not by Tom much, but it increases) without delivering any visible benefit. OK, but this is a serious problem because estimate

Re: [CORE] [HACKERS] postpone next week's release

2015-06-01 Thread Josh Berkus
All, Just my $0.02 on PR: it has never been a PR problem to do multiple update releases, as long as we could provide a good reason for doing so (like: fix A is available now and we didn't want to hold it back waiting for fix B). It's always a practical question of (a) packaging and (b)

Re: [HACKERS] Really bad blowups with hash outer join and nulls

2015-06-01 Thread Andrew Gierth
Jim == Jim Nasby jim.na...@bluetreble.com writes: Jim Anything happen with this, or the patch Andrew posted? No. And my attention has just been drawn to this, which looks like the same issue: http://www.postgresql.org/message-id/52b47b47-0926-4e15-b25e-212df52fe...@oseberg.io -- Andrew

Re: [HACKERS] Join Filter vs. Index Cond (performance regression 9.1-9.2+/HEAD)

2015-06-01 Thread Tom Lane
Andrew Gierth and...@tao11.riddles.org.uk writes: Tom == Tom Lane t...@sss.pgh.pa.us writes: Tom Once you're down to an estimate of one row retrieved, adding Tom additional index conditions simply increases the cost (not by Tom much, but it increases) without delivering any visible benefit.

Re: [HACKERS] Multixid hindsight design

2015-06-01 Thread Thomas Munro
On Tue, May 12, 2015 at 9:20 AM, Heikki Linnakangas hlinn...@iki.fi wrote: The beauty of this would be that the TED entries can be zapped at restart, just like pg_subtrans, and pg_multixact before 9.3. It doesn't need to be WAL-logged, and we are free to change its on-disk layout even in a

Re: [HACKERS] Join Filter vs. Index Cond (performance regression 9.1-9.2+/HEAD)

2015-06-01 Thread Robert Haas
On Sun, May 31, 2015 at 12:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Gierth and...@tao11.riddles.org.uk writes: Obviously it makes little sense to use an (a,b,c) index to look up just (a,b) and then filter on c; the question is, what is the planner doing that leads it to get this so

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-06-01 Thread Josh Berkus
On 05/31/2015 10:46 AM, Tom Lane wrote: Joel Jacobson j...@trustly.com writes: If we could turn back time, would we have picked pg_xlog as the most optimal name for this important directory, or would we have come up with a more user-friendly name? Yeah... My suggestion is to use

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-06-01 Thread Joel Jacobson
On Mon, Jun 1, 2015 at 10:17 PM, Josh Berkus j...@agliodbs.com wrote: If we symlink pg_xlog, then it will still trip up anyone who does rm -rf *log*/* or deletes files directly from inside the directory, both of which I've seen. Deleting the directory itself is comparatively rare in my

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-01 Thread Alvaro Herrera
Peter Eisentraut wrote: But it is a valid point that we'd need to build up more extension API tests before we start cutting back significantly on the maybe-example-maybe-real extensions that we ship in contrib. We need to find a good middle ground. Nowadays we can test concurrent behavior

Re: [HACKERS] nested loop semijoin estimates

2015-06-01 Thread Tom Lane
Tomas Vondra tomas.von...@2ndquadrant.com writes: On 06/01/15 00:08, Tom Lane wrote: Attached is an incremental patch (on top of the previous one) to allow startup cost of parameterized paths to be considered when the relation is the RHS of a semi or anti join. It seems reasonably clean

Re: [HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-01 Thread Alvaro Herrera
Thomas Munro wrote: - There's a third possible problem related to boundary cases in SlruScanDirCbRemoveMembers, but I don't understand that one well enough to explain it. Maybe Thomas can jump in here and explain the concern. I noticed something in passing which is probably not

Re: [HACKERS] RFC: Remove contrib entirely

2015-06-01 Thread Peter Eisentraut
On 5/29/15 5:35 PM, Tom Lane wrote: But let's get to the point: the real reason for keeping most of these contrib modules in the core distribution is that they are essential test cases for core's extensibility features. contrib/isn is actually a good example of that. It made us realize that

Re: [HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-01 Thread Alvaro Herrera
Alvaro Herrera wrote: Robert Haas wrote: In the process of investigating this, we found a few other things that seem like they may also be bugs: - As noted upthread, replaying an older checkpoint after a newer checkpoint has already happened may lead to similar problems. This may

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-06-01 Thread Bruce Momjian
On Mon, Jun 1, 2015 at 05:57:27PM -0400, David Steele wrote: On 6/1/15 4:42 PM, Joel Jacobson wrote: Also ... if we were to rename it, it should be pg_wal or pg_xact. Please let's not add yet another term for the WAL. I like pg_wal. It's correct and suitably mysterious. +1 -- Bruce

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-06-01 Thread David Steele
On 6/1/15 4:42 PM, Joel Jacobson wrote: Also ... if we were to rename it, it should be pg_wal or pg_xact. Please let's not add yet another term for the WAL. I like pg_wal. It's correct and suitably mysterious. -- - David Steele da...@pgmasters.net signature.asc Description: OpenPGP

Re: [HACKERS] Multixid hindsight design

2015-06-01 Thread Greg Stark
On Mon, Jun 1, 2015 at 8:53 PM, Thomas Munro thomas.mu...@enterprisedb.com wrote: What about prepared transactions? They can lock rows FOR SHARE that survive server restarts. And they can make update chains that are still uncommitted after a restart. I think we should think separately about

Re: [HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-01 Thread Alvaro Herrera
Alvaro Herrera wrote: Anyway here's a quick script to almost-reproduce the problem. Meh. Really attached now. I also wanted to post the error messages we got: 2015-05-27 16:15:17 UTC [4782]: [3-1] user=,db= LOG: entering standby mode 2015-05-27 16:15:18 UTC [4782]: [4-1] user=,db= LOG:

Re: [HACKERS] nested loop semijoin estimates

2015-06-01 Thread Josh Berkus
On 06/01/2015 02:18 PM, Tom Lane wrote: Tomas Vondra tomas.von...@2ndquadrant.com writes: On 06/01/15 00:08, Tom Lane wrote: Attached is an incremental patch (on top of the previous one) to allow startup cost of parameterized paths to be considered when the relation is the RHS of a semi or

Re: [HACKERS] nested loop semijoin estimates

2015-06-01 Thread Tomas Vondra
On 06/01/15 23:47, Josh Berkus wrote: On 06/01/2015 02:18 PM, Tom Lane wrote: Anybody else want to speak for or against back-patching the patch as posted? I intentionally didn't push it in before today's releases, but I will push it later this week if there are not objections. I would like

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-06-01 Thread Thomas Munro
On Tue, Jun 2, 2015 at 8:17 AM, Josh Berkus j...@agliodbs.com wrote: Also ... if we were to rename it, it should be pg_wal or pg_xact. Please let's not add yet another term for the WAL. +1 for pg_wal if it has to be renamed. If pg_clog also has to be renamed, how about using your other

Re: [HACKERS] nested loop semijoin estimates

2015-06-01 Thread Josh Berkus
On 06/01/2015 03:22 PM, Tomas Vondra wrote: On 06/01/15 23:47, Josh Berkus wrote: On 06/01/2015 02:18 PM, Tom Lane wrote: Anybody else want to speak for or against back-patching the patch as posted? I intentionally didn't push it in before today's releases, but I will push it later this

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-06-01 Thread Josh Berkus
On 06/01/2015 04:22 PM, Thomas Munro wrote: On Tue, Jun 2, 2015 at 8:17 AM, Josh Berkus j...@agliodbs.com wrote: Also ... if we were to rename it, it should be pg_wal or pg_xact. Please let's not add yet another term for the WAL. +1 for pg_wal if it has to be renamed. If pg_clog also has

[HACKERS] Minor improvement to func.sgml

2015-06-01 Thread Etsuro Fujita
Hi, Here is a doc patch to add materialized views and foreign tables to database objects that pg_table_is_visible() can be used with. Best regards, Etsuro Fujita *** a/doc/src/sgml/func.sgml --- b/doc/src/sgml/func.sgml *** *** 15537,15544 SELECT relname FROM pg_class WHERE

[HACKERS] Minor issue with BRIN regression tests

2015-06-01 Thread Peter Geoghegan
Attached patch adjusts BRIN regression tests to make a non-obvious dependency on tuple order explicit. Currently, an index-only scan plan is used by the query that I've adjusted. I'd rather be sure that that continues. This was spotted while running the regression tests with the Postgres default

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-06-01 Thread Michael Nolan
Why not take a simpler approach and create a zero length file in directories that should not be fiddled with by non-experts using a file name something like DO.NOT.DELETE.THESE.FILES? No, it won't prevent the incredibly stupid from doing incredibly stupid things, nothing will. -- Mike Nolan

Re: [HACKERS] pg_xlog - pg_xjournal?

2015-06-01 Thread Michael Paquier
On Tue, Jun 2, 2015 at 8:48 AM, Josh Berkus j...@agliodbs.com wrote: On 06/01/2015 04:22 PM, Thomas Munro wrote: On Tue, Jun 2, 2015 at 8:17 AM, Josh Berkus j...@agliodbs.com wrote: Also ... if we were to rename it, it should be pg_wal or pg_xact. Please let's not add yet another term for the

Re: [HACKERS] about lob(idea)

2015-06-01 Thread Jim Nasby
On 5/27/15 5:02 AM, Guillaume Lelarge wrote: 2015-05-27 0:01 GMT+02:00 Martín Marqués mar...@2ndquadrant.com mailto:mar...@2ndquadrant.com: El 25/05/15 a las 06:13, alex2010 escribió: Maybe it makes sense to add ability to store large objects in the same table space as the table.

Re: [HACKERS] [CORE] postpone next week's release

2015-06-01 Thread Noah Misch
On Sun, May 31, 2015 at 12:09:16PM -0400, David Steele wrote: On 5/31/15 11:49 AM, Noah Misch wrote: On Sat, May 30, 2015 at 09:51:04PM -0400, David Steele wrote: Sure - I can write code to do that. But then why release a beta at all? It's largely for the benefit of folks planning

Re: [HACKERS] PostgreSQL 8.3 index page count clarification

2015-06-01 Thread Jim Nasby
On 5/26/15 7:55 AM, Andrew Dunstan wrote: On 05/26/2015 08:45 AM, Srinivas Karthik V wrote: For the user created indexes in PostgreSQL 8.3.6, I would like to know which file or function populates the index data structure. Specifically, where in the code is index-tuples and index-pages

[HACKERS] Re: [GENERAL] 9.4.1 - 9.4.2 problem: could not access status of transaction 1

2015-06-01 Thread Noah Misch
On Mon, Jun 01, 2015 at 02:06:05PM -0400, Robert Haas wrote: On Mon, Jun 1, 2015 at 12:46 AM, Noah Misch n...@leadboat.com wrote: On Fri, May 29, 2015 at 03:08:11PM -0400, Robert Haas wrote: SetMultiXactIdLimit() bracketed certain parts of its logic with if (!InRecovery), but those guards