Re: [HACKERS] incomplete removal of not referenced CTEs

2016-09-01 Thread Tomas Vondra
On 09/01/2016 09:58 PM, Andres Freund wrote: > On 2016-09-01 15:46:45 -0400, Tom Lane wrote: >> Tomas Vondra writes: >>> While investigating a CTE-related query, I've noticed that we don't >>> really remove all unreachable CTEs. >> >> We expend a grand total of

Re: [HACKERS] \timing interval

2016-09-01 Thread Tom Lane
[ This patch is marked Ready For Committer, and discussion seems to have died off, so let's get on with committing something ... ] Corey Huinker writes: > Generally speaking, people disliked the third mode for \timing, and were > generally fine with AndrewG's idea of

Re: [HACKERS] \timing interval

2016-09-01 Thread Corey Huinker
On Thu, Sep 1, 2016 at 2:43 PM, Tom Lane wrote: > I wrote: > > Sorry, that probably added no clarity at all, I was confusing > > seconds with milliseconds in the example values :-( > > After a bit of further fooling with sample values, I propose this > progression: > > Time:

Re: [HACKERS] \timing interval

2016-09-01 Thread Tom Lane
Corey Huinker writes: > On Thu, Sep 1, 2016 at 3:01 PM, Tom Lane wrote: >> Well, that code's on the backend side so we're not going to just call it >> in any case. And I think we don't want to be quite so verbose as to go up >> to hh:mm:ss.fff as

Re: [HACKERS] System load consideration before spawning parallel workers

2016-09-01 Thread Gavin Flower
On 02/09/16 04:44, Peter Eisentraut wrote: On 8/1/16 2:17 AM, Gavin Flower wrote: Possibly look how make does it with the '-l' flag? '-l 8' don't start more process when load is 8 or greater, works on Linux at least... The problem with that approach is that it takes about a minute for the

Re: [HACKERS] [PATCH] COPY vs \copy HINT

2016-09-01 Thread Tom Lane
Craig Ringer writes: > On 12 August 2016 at 16:34, Christoph Berg wrote: >> Also, I vaguely get what you wanted to say with "a driver ... >> wrapper", but it's pretty nonsensical if one doesn't know about the >> protocol details. I don't have a better

[HACKERS] [PATCH] OpenSSL 1.1.0 support

2016-09-01 Thread Kurt Roeckx
Hi, Attached is a patch to make it build with OpenSSL 1.1.0. There is probably a minor problem on windows where the name of the dlls got changed. Someone probably should look into that. Kurt >From efd7aa3499b2b4eedd4c4d4164b75175f3c10d2f Mon Sep 17 00:00:00 2001 From: Kurt Roeckx

Re: [HACKERS] [PATCH] OpenSSL 1.1.0 support

2016-09-01 Thread Tom Lane
Kurt Roeckx writes: > Attached is a patch to make it build with OpenSSL 1.1.0. Hi Kurt, There's already been some work on this topic: https://www.postgresql.org/message-id/flat/20160627151604.gd1...@msg.df7cb.de Maybe you should join forces with Andreas to get it finished.

Re: [HACKERS] LOCK TABLE .. DEFERRABLE

2016-09-01 Thread Simon Riggs
On 5 April 2016 at 18:34, Rod Taylor wrote: > > > On Tue, Apr 5, 2016 at 1:10 PM, Simon Riggs wrote: >>> >>> If a lock is successfully obtained on one table, but not on all tables, >>> it releases that lock and will retry to get them as a group in the

Re: [HACKERS] Improve BEGIN tab completion

2016-09-01 Thread Kevin Grittner
On Wed, May 18, 2016 at 8:59 AM, Andreas Karlsson wrote: > I noticed that the tab completion was not aware of that TRANSACTION/WORK is > optional in BEGIN, and that we do not complete [NOT] DEFERRABLE. > > While fixing it I also improved the completion support for SET SESSION

Re: [HACKERS] CommitFest 2016-09

2016-09-01 Thread Fabrízio de Royes Mello
On Thu, Sep 1, 2016 at 5:11 PM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > Please check the "needs reviewer" list > (https://commitfest.postgresql.org/9/?reviewer=-2) for patches to > review. The committers need our help to work. > Just to fix the correct link bellow sent in

Re: [HACKERS] Missing checks when malloc returns NULL...

2016-09-01 Thread Michael Paquier
On Thu, Sep 1, 2016 at 11:16 PM, Tom Lane wrote: > Michael Paquier writes: >> On Thu, Sep 1, 2016 at 10:03 PM, Tom Lane wrote: >>> Don't see the point really ... it's just more API churn without any >>> very compelling reason. >

Re: [HACKERS] Logical Replication WIP

2016-09-01 Thread Erik Rijkers
On 2016-09-01 01:04, Erik Rijkers wrote: On 2016-08-31 22:51, Petr Jelinek wrote: Here are some small changes to logical-replication.sgml ... and other .sgml files. Erik Rijkers--- doc/src/sgml/ref/alter_publication.sgml.orig 2016-09-01 07:20:03.280295807 +0200 +++

Re: [HACKERS] asynchronous and vectorized execution

2016-09-01 Thread Kyotaro HORIGUCHI
This is random thoughts on this patch. At Tue, 30 Aug 2016 12:17:52 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20160830.121752.100817694.horiguchi.kyot...@lab.ntt.co.jp> > > As the result, the attached patchset is functionally the same > > with

[HACKERS] new gcc warning

2016-09-01 Thread Pavel Stehule
Hi I see a new warning in upstream r/include/libxml2 -c -o path.o path.c path.c: In function ‘has_drive_prefix’: path.c:89:26: warning: self-comparison always evaluates to false [-Wtautological-compare] return skip_drive(path) != path; ^~ Regards [pavel@nemesis

Re: [HACKERS] pg_basebackup stream xlog to tar

2016-09-01 Thread Magnus Hagander
On Thu, Sep 1, 2016 at 9:19 AM, Michael Paquier wrote: > On Thu, Sep 1, 2016 at 6:58 AM, Magnus Hagander > wrote: > > Attached patch adds support for -X stream to work with .tar and .tar.gz > file > > formats. > > Nice. > > > If tar mode is

Re: [HACKERS] pgsql: Add putenv support for msvcrt from Visual Studio 2013

2016-09-01 Thread Christian Ullrich
* Michael Paquier wrote: > On Wed, Apr 27, 2016 at 2:39 AM, Christian Ullrich wrote: >> * Christian Ullrich wrote: > And actually, by looking at those patches, isn't it a dangerous > practice to be able to load multiple versions of the same DLL routines > in the same

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-01 Thread Michael Paquier
On Thu, Sep 1, 2016 at 4:01 AM, Abhijit Menon-Sen wrote: > At 2016-08-31 17:15:59 +0100, si...@2ndquadrant.com wrote: >> >> * Recovery parameters would now be part of the main postgresql.conf >> infrastructure >> Any parameters set in $DATADIR/recovery.conf will be read

Re: [HACKERS] pg_basebackup stream xlog to tar

2016-09-01 Thread Michael Paquier
On Thu, Sep 1, 2016 at 6:58 AM, Magnus Hagander wrote: > Attached patch adds support for -X stream to work with .tar and .tar.gz file > formats. Nice. > If tar mode is specified, a separate pg_xlog.tar (or .tar.gz) file is > created and the data is streamed into it. Regular

Re: [HACKERS] Comment on GatherPath.single_copy

2016-09-01 Thread Kyotaro HORIGUCHI
At Wed, 31 Aug 2016 07:26:22 -0400, Tom Lane wrote in <5934.1472642...@sss.pgh.pa.us> > Robert Haas writes: > > On Tue, Aug 30, 2016 at 6:38 PM, Tom Lane wrote: > >> Robert, could you fix the documentation for that field so it's >

Re: [HACKERS] Parallel build with MSVC

2016-09-01 Thread Michael Paquier
On Thu, Apr 28, 2016 at 4:16 PM, Christian Ullrich wrote: > * Michael Paquier wrote: >> On Wed, Apr 27, 2016 at 5:15 PM, Christian Ullrich >> wrote: >>> OK then, hopefully last round attached. >> >> Thanks. Those are fine in my view. It is hard to tell

Re: [HACKERS] pg_basebackup stream xlog to tar

2016-09-01 Thread Magnus Hagander
On Thu, Sep 1, 2016 at 9:43 AM, Michael Paquier wrote: > On Thu, Sep 1, 2016 at 4:41 PM, Magnus Hagander > wrote: > > That's definitely not intended - it's supposed to be 16Mb. And it > actually > > writes 16Mb to the tarfile, it's the extraction

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-01 Thread Andrew Borodin
Hi hackers! Here is the new patch version. With the help of Mikhail Bakhterev I've optimized subroutines of the penalty function. Index build time now is roughly equivalent to build time before patch (test attached to thread start). Time of SELECT statement execution is reduced by 40%. Changes in

[HACKERS] Add support for restrictive RLS policies

2016-09-01 Thread Stephen Frost
Greetings, As outlined in the commit message, this adds support for restrictive RLS policies. We've had this in the backend since 9.5, but they were only available via hooks and therefore extensions. This adds support for them to be configured through regular DDL commands. These policies are,

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-01 Thread Abhijit Menon-Sen
At 2016-09-01 15:51:11 +0900, michael.paqu...@gmail.com wrote: > > - (errmsg("starting point-in-time recovery to XID %u", > - recoveryTargetXid))); > User loses information if those logs are removed. Agreed. I'm revising the patch right now, and I

Re: [HACKERS] pg_basebackup stream xlog to tar

2016-09-01 Thread Michael Paquier
On Thu, Sep 1, 2016 at 4:41 PM, Magnus Hagander wrote: > That's definitely not intended - it's supposed to be 16Mb. And it actually > writes 16Mb to the tarfile, it's the extraction that doesn't see them. That > also means that if you get more than one member of the tarfile

Re: [HACKERS] Declarative partitioning - another take

2016-09-01 Thread Ashutosh Bapat
> > > I don't think you need to do anything in the path creation code for this. > > As is it flattens all AppendPath hierarchies whether for partitioning or > > inheritance or subqueries. We should leave it as it is. > > I thought it would be convenient for pairwise join code to work with the >

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-01 Thread Michael Banck
On Wed, Aug 31, 2016 at 05:15:59PM +0100, Simon Riggs wrote: > Recover mode starts the server as a standby server which > expects to receive changes from a primary/master server using physical > streaming replication or is used for performing a recovery from > backup. I understand where this is

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

2016-09-01 Thread Peter Eisentraut
On 8/11/16 9:12 PM, Michael Paquier wrote: > Note that pg_dump[all] and pg_upgrade already have safeguards against > those things per the same routines putting quotes for execution as > commands into psql and shell. So attached is a patch to implement this > restriction in the backend, How about

Re: [HACKERS] autonomous transactions

2016-09-01 Thread Joel Jacobson
On Thu, Sep 1, 2016 at 12:12 AM, Vik Fearing wrote: > Part of what people want this for is to audit what people *try* to do. > We can already audit what they've actually done. > > With your solution, we still wouldn't know when an unauthorized attempt > to do something

Re: [HACKERS] Hash Indexes

2016-09-01 Thread Jesper Pedersen
On 08/05/2016 07:36 AM, Amit Kapila wrote: On Thu, Aug 4, 2016 at 8:02 PM, Mithun Cy wrote: I did some basic testing of same. In that I found one issue with cursor. Thanks for the testing. The reason for failure was that the patch didn't take into account the

Re: [HACKERS] WAL consistency check facility

2016-09-01 Thread Peter Geoghegan
On Thu, Sep 1, 2016 at 9:23 AM, Robert Haas wrote: > Indeed, it had occurred to me that we might not even want to compile > this code into the server unless WAL_DEBUG is defined; after all, how > does it help a regular user to detect that the server has a bug? Bug > or no

Re: [HACKERS] \timing interval

2016-09-01 Thread Tom Lane
I wrote: > Sorry, that probably added no clarity at all, I was confusing > seconds with milliseconds in the example values :-( After a bit of further fooling with sample values, I propose this progression: Time: 0.100 ms Time: 1.200 ms Time: 1001.200 ms (0:01.001) Time: 12001.200 ms (0:12.001)

[HACKERS] CommitFest 2016-09

2016-09-01 Thread Fabrízio de Royes Mello
Hi all, The 2016-09 commitfest is officially in progress, and I'm the manager. The current status summary is Needs review: 119 Needs *reviewer*: 63 Please check the "needs reviewer" list (https://commitfest.postgresql.org/10/?reviewer=-2 ) for

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-01 Thread Andrew Borodin
Here is new version of the patch. Now function pack_float is commented better. All inline keywords are removed. I haven't found any performance loss for that. Remove of static's showed 1%-7% performance loss in SELECT computation (3 test runs), so I left statics where they are. Actually, to avoid

Re: [HACKERS] \timing interval

2016-09-01 Thread Tom Lane
Corey Huinker writes: > I'm going to hold off a bit to see if anybody else chimes in, and if not > I'm going to deliver a patch. I've already been updating yours, no need for another. regards, tom lane -- Sent via pgsql-hackers mailing list

[HACKERS] incomplete removal of not referenced CTEs

2016-09-01 Thread Tomas Vondra
Hi, While investigating a CTE-related query, I've noticed that we don't really remove all unreachable CTEs. For example, for this query with a as (select 1), b as (select * from a), c as (select * from b) select 2; where none of the CTEs if (directly or indirectly)

Re: [HACKERS] autonomous transactions

2016-09-01 Thread Joel Jacobson
On Wed, Aug 31, 2016 at 6:22 PM, Simon Riggs wrote: > On 31 August 2016 at 14:09, Joel Jacobson wrote: >> My idea on how to deal with this would be to mark the function to be >> "AUTONOMOUS" similar to how a function is marked to be "PARALLEL >> SAFE", >>

Re: [HACKERS] \timing interval

2016-09-01 Thread Tom Lane
Corey Huinker writes: > On Thu, Sep 1, 2016 at 2:43 PM, Tom Lane wrote: >> Note that times from 1 second to 1 hour all get the nn:nn.nnn >> treatment. I experimented with these variants for sub-minute times: >> ... >> but it seems like the first

Re: [HACKERS] \timing interval

2016-09-01 Thread Corey Huinker
On Thu, Sep 1, 2016 at 3:01 PM, Tom Lane wrote: > Corey Huinker writes: > > On Thu, Sep 1, 2016 at 2:43 PM, Tom Lane wrote: > >> Note that times from 1 second to 1 hour all get the nn:nn.nnn > >> treatment. I experimented with

Re: [HACKERS] \timing interval

2016-09-01 Thread Tom Lane
Peter van Hardenberg writes: > Some kind of units on the parenthetical format would be helpful. I was really hoping to not re-open that can of worms :-( regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] incomplete removal of not referenced CTEs

2016-09-01 Thread Andres Freund
On 2016-09-01 21:36:13 +0200, Tomas Vondra wrote: > Of course, it's harmless as none of those CTEs gets actually executed, > but is this intentional, or do we want/need to fix it? I don't see > anything about this in the docs, but it seems a bit awkward and > confusing to remove only some of the

Re: [HACKERS] \timing interval

2016-09-01 Thread Tom Lane
I wrote: > So for clarity's sake: first suitable format among these: > Time: 59.999 ms > Time: 121.999 ms (2:01.999) > Time: 10921.999 ms (3:02:01.999) > Time: 356521.999 ms (4 3:02:01.999) Sorry, that probably added no clarity at all, I was confusing seconds with milliseconds in the example

Re: [HACKERS] \timing interval

2016-09-01 Thread Corey Huinker
On Thu, Sep 1, 2016 at 2:17 PM, Tom Lane wrote: > I wrote: > > So for clarity's sake: first suitable format among these: > > > Time: 59.999 ms > > Time: 121.999 ms (2:01.999) > > Time: 10921.999 ms (3:02:01.999) > > Time: 356521.999 ms (4 3:02:01.999) > > Sorry, that probably

Re: [HACKERS] autonomous transactions

2016-09-01 Thread Andres Freund
On 2016-08-31 06:10:31 +0200, Joel Jacobson wrote: > This is important if you as a caller function want to be sure none of > the work made by anything called down the stack gets committed. > That is, if you as a caller decide to rollback, e.g. by raising an > exception, and you want to be sure

Re: [HACKERS] Speedup twophase transactions

2016-09-01 Thread Simon Riggs
On 13 April 2016 at 15:31, Stas Kelvich wrote: > Fixed patch attached. There already was infrastructure to skip currently > held locks during replay of standby_redo() and I’ve extended that with check > for > prepared xids. Please confirm that everything still works

Re: [HACKERS] System load consideration before spawning parallel workers

2016-09-01 Thread Gavin Flower
On 02/09/16 05:01, Peter Eisentraut wrote: On 8/16/16 3:39 AM, Haribabu Kommi wrote: [...] All of this seems very platform specific, too. You have Windows-specific code, but the rest seems very Linux-specific. The dstat tool I had never heard of before. There is stuff with cgroups, which I

Re: [HACKERS] \timing interval

2016-09-01 Thread Peter van Hardenberg
On Thu, Sep 1, 2016 at 12:14 PM, Tom Lane wrote: > Corey Huinker writes: > > On Thu, Sep 1, 2016 at 3:01 PM, Tom Lane wrote: > >> Well, that code's on the backend side so we're not going to just call it > >> in any case. And I

Re: [HACKERS] incomplete removal of not referenced CTEs

2016-09-01 Thread Tom Lane
Tomas Vondra writes: > While investigating a CTE-related query, I've noticed that we don't > really remove all unreachable CTEs. We expend a grand total of three lines of code on making that happen. I'm pretty much -1 on adding a great deal more code or complexity

Re: [HACKERS] incomplete removal of not referenced CTEs

2016-09-01 Thread Andres Freund
On 2016-09-01 15:46:45 -0400, Tom Lane wrote: > Tomas Vondra writes: > > While investigating a CTE-related query, I've noticed that we don't > > really remove all unreachable CTEs. > > We expend a grand total of three lines of code on making that happen. > I'm

Re: [HACKERS] System load consideration before spawning parallel workers

2016-09-01 Thread Tom Lane
Gavin Flower writes: > On 02/09/16 04:44, Peter Eisentraut wrote: >> You can try this out by building PostgreSQL this way. Please save your >> work first, because you might have to hard-reboot your system. > Hmm... I've built several versions of pg this way,

Re: [HACKERS] Add support for restrictive RLS policies

2016-09-01 Thread Robert Haas
On Thu, Sep 1, 2016 at 12:04 PM, Stephen Frost wrote: > As outlined in the commit message, this adds support for restrictive RLS > policies. We've had this in the backend since 9.5, but they were only > available via hooks and therefore extensions. This adds support for >

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-01 Thread Simon Riggs
On 31 August 2016 at 20:01, Abhijit Menon-Sen wrote: > Unfortunately, some parts conflict with the patch that Simon just posted > (e.g., his patch removes trigger_file altogether, whereas mine converts > it into a GUC along the lines of the original patch). Rather than

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-09-01 Thread Simon Riggs
On 1 September 2016 at 06:34, Michael Paquier wrote: > On Thu, Sep 1, 2016 at 1:15 AM, Simon Riggs wrote: >> This is a summary of proposed changes to the recovery.conf API for >> v10. These are based in part on earlier discussions, and represent

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2016-09-01 Thread Pavan Deolasee
On Thu, Sep 1, 2016 at 1:33 AM, Bruce Momjian wrote: > On Wed, Aug 31, 2016 at 10:15:33PM +0530, Pavan Deolasee wrote: > > Instead, what I would like to propose and the patch currently implements > is to > > restrict WARM update to once per chain. So the first non-HOT update to

Re: [HACKERS] Add support for restrictive RLS policies

2016-09-01 Thread Thom Brown
On 1 September 2016 at 10:02, Robert Haas wrote: > On Thu, Sep 1, 2016 at 12:04 PM, Stephen Frost wrote: >> As outlined in the commit message, this adds support for restrictive RLS >> policies. We've had this in the backend since 9.5, but they were

Re: [HACKERS] PostgreSQL 10 kick-off

2016-09-01 Thread Magnus Hagander
On Aug 4, 2016 2:25 AM, "Michael Paquier" wrote: > > On Thu, Aug 4, 2016 at 12:09 AM, Fabrízio de Royes Mello > wrote: > > If there are no complains about my lack of experience in this field I would > > like do become the next CFM (am I the

Re: [HACKERS] WAL consistency check facility

2016-09-01 Thread Robert Haas
On Wed, Aug 31, 2016 at 7:02 PM, Simon Riggs wrote: > I'd prefer a solution that was not dependent upon RmgrID at all. > > If there are various special cases that we need to cater for, ISTM > they would be flaws in the existing WAL implementation rather than > anything we

Re: [HACKERS] WAL consistency check facility

2016-09-01 Thread Simon Riggs
On 1 September 2016 at 11:16, Robert Haas wrote: > On Wed, Aug 31, 2016 at 7:02 PM, Simon Riggs wrote: >> I'd prefer a solution that was not dependent upon RmgrID at all. >> >> If there are various special cases that we need to cater for, ISTM >>

Re: [HACKERS] PostgreSQL 10 kick-off

2016-09-01 Thread Tom Lane
Magnus Hagander writes: >> On Thu, Aug 4, 2016 at 12:09 AM, Fabrízio de Royes Mello >> wrote: >>> If there are no complains about my lack of experience in this field I >>> would like do become the next CFM (am I the first brazilian??) > Did we make

Re: [HACKERS] CREATE POLICY bug ?

2016-09-01 Thread Dean Rasheed
[Please reply to the list, not just to me, so that others can benefit from and contribute to the discussion] On 31 August 2016 at 11:52, Andrea Adami wrote: > Thnaks Dean, i did further investigations: > i set the owner of the view to: "mana...@scuola247.it" with: > ALTER TABLE

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-01 Thread Rahila Syed
Ok. Please find attached a patch which introduces psql error when autocommit is turned on inside a transaction. It also adds relevant documentation in psql-ref.sgml. Following is the output. bash-4.2$ psql -d postgres psql (10devel) Type "help" for help. postgres=# \set AUTOCOMMIT OFF postgres=#

Re: [HACKERS] pg_basebackup wish list

2016-09-01 Thread Simon Riggs
On 19 August 2016 at 08:46, Michael Paquier wrote: > On Fri, Aug 19, 2016 at 2:04 PM, Masahiko Sawada > wrote: >> I agree with adding this as an option and removing directory by default. >> And it looks good to me except for missing new line in

Re: [HACKERS] Exclude schema during pg_restore

2016-09-01 Thread Peter Eisentraut
On 8/31/16 4:10 AM, Michael Banck wrote: > attached is a small patch that adds an -N option to pg_restore, in order > to exclude a schema, in addition to -n for the restriction to a schema. I think this is a good idea, and the approach looks sound. However, something doesn't work right. If I

Re: [HACKERS] [PATCH] COPY vs \copy HINT

2016-09-01 Thread Craig Ringer
On 2 September 2016 at 04:28, Tom Lane wrote: > Craig Ringer writes: >> On 12 August 2016 at 16:34, Christoph Berg wrote: >>> Also, I vaguely get what you wanted to say with "a driver ... >>> wrapper", but it's pretty nonsensical if

Re: [HACKERS] less expensive pg_buffercache on big shmem

2016-09-01 Thread Andres Freund
On 2016-09-02 08:31:42 +0530, Robert Haas wrote: > I wonder whether we ought to just switch from the consistent method to > the semiconsistent method and call it good. +1. I think, before long, we're going to have to switch away from having locks & partitions in the first place. So I don't see a

Re: [HACKERS] Confusing docs about GetForeignUpperPaths in fdwhandler.sgml

2016-09-01 Thread Robert Haas
On Mon, Aug 1, 2016 at 5:44 PM, Etsuro Fujita wrote: > I noticed that the following note about direct modification via > GetForeignUpperPaths in fdwhandler.sgml is a bit confusing. We have another > approach using PlanDirectModify, so that should be reflected in the

Re: [HACKERS] restoration after crash slowness, any way to improve?

2016-09-01 Thread Jeff Janes
On Wed, Aug 31, 2016 at 6:26 PM, Joshua D. Drake wrote: > -hackers, > > So this is more of a spit balling thread than anything. As I understand > it, if we have a long running transaction or a large number of wal logs and > we crash, we then have to restore those logs on

Re: [HACKERS] less expensive pg_buffercache on big shmem

2016-09-01 Thread Robert Haas
On Wed, Aug 31, 2016 at 8:27 PM, Ivan Kartyshov wrote: > Recently I have finished my work on a patch for pg_buffercache contrib, I > think it's time to share my results. Thanks for sharing your results. > V1.2 implementation contains flexible loop which can collect

Re: [HACKERS] pgbench - allow to store select results into variables

2016-09-01 Thread Amit Langote
Hi Fabien, On 2016/07/16 1:33, Fabien COELHO wrote: > Here is a v2 with more or less this approach, although \into does not end > the query, but applies to the current or last sql command. A query is > still terminated with a ";". This patch needs to be rebased because of commit 64710452 (on

Re: [HACKERS] Missing checks when malloc returns NULL...

2016-09-01 Thread Tom Lane
Michael Paquier writes: > Also, we could take one extra step forward then, and just introduce > ShmemAllocExtended that holds two flags as per the attached: > - SHMEM_ALLOC_ZERO that zeros all the fields > - SHMEM_ALLOC_NO_OOM that does not fail Don't see the point

Re: [HACKERS] GIN logging GIN_SEGMENT_UNMODIFIED actions?

2016-09-01 Thread Fujii Masao
On Wed, Aug 31, 2016 at 8:32 PM, Tom Lane wrote: > Fujii Masao writes: >> On Wed, Aug 31, 2016 at 12:10 AM, Tom Lane wrote: >>> Hmm, comparing gin_desc() to ginRedoInsert() makes me think there are more >>> problems there than that

Re: [HACKERS] Missing checks when malloc returns NULL...

2016-09-01 Thread Michael Paquier
On Thu, Sep 1, 2016 at 10:03 PM, Tom Lane wrote: > Michael Paquier writes: >> Also, we could take one extra step forward then, and just introduce >> ShmemAllocExtended that holds two flags as per the attached: >> - SHMEM_ALLOC_ZERO that zeros all

Re: [HACKERS] pg_basebackup stream xlog to tar

2016-09-01 Thread Michael Paquier
On Thu, Sep 1, 2016 at 5:13 PM, Magnus Hagander wrote: > We don't seem to check for similar issues as the one just found in the > existing tests though, do we? As in, we don't actually verify that the xlog > files being streamed are 16Mb? (Or for that matter that the tarfile

Re: [HACKERS] [WIP] Patches to enable extraction state of query execution from external session

2016-09-01 Thread Oleksandr Shulgin
On Tue, Aug 30, 2016 at 9:34 AM, Maksim Milyutin wrote: > On Mon, Aug 29, 2016 at 5:22 PM, maksim > > wrote: >> >> Hi, hackers! >> >> Now I complete extension that provides facility to see the

Re: [HACKERS] new gcc warning

2016-09-01 Thread Tom Lane
Pavel Stehule writes: > I see a new warning in upstream > r/include/libxml2 -c -o path.o path.c > path.c: In function ‘has_drive_prefix’: > path.c:89:26: warning: self-comparison always evaluates to false > [-Wtautological-compare] > return skip_drive(path) !=

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-01 Thread Tom Lane
Andrew Borodin writes: > Does every supported Postgres platform conforms to IEEE 754 floating > point specification? Possibly, but it is against project policy to allow code to assume so. That pack_float function is absolutely not acceptable IMV, and would still be if it

Re: [HACKERS] GiST penalty functions [PoC]

2016-09-01 Thread Andrew Borodin
Thank you for your coments, Tom. > Modern compilers are generally able to make their own decisions about it, and > trying to put your thumb on the scales this heavily is not likely to improve > the code. Well, I have tested that combination of "static inline" affets performance of index build

Re: [HACKERS] autonomous transactions

2016-09-01 Thread Constantin S. Pan
On Wed, 31 Aug 2016 14:46:30 +0100 Greg Stark wrote: > On Wed, Aug 31, 2016 at 2:50 AM, Peter Eisentraut > wrote: > > - A API interface to open a "connection" to a background worker, run > > queries, get results: AutonomousSessionStart(), > >

Re: [HACKERS] Logical decoding slots can go backwards when used from SQL, docs are wrong

2016-09-01 Thread Simon Riggs
On 20 August 2016 at 15:04, Petr Jelinek wrote: > On 20/08/16 16:01, Craig Ringer wrote: >> >> On 5 June 2016 at 09:54, David G. Johnston > > wrote: >> >> On Thursday, March 17, 2016, Craig Ringer

Re: [HACKERS] GIN logging GIN_SEGMENT_UNMODIFIED actions?

2016-09-01 Thread Tom Lane
Fujii Masao writes: > I applied your suggested changes into the patch. Patch attached. That looks pretty sane to me (but I just eyeballed it, didn't test). One further minor improvement would be to rearrange the XLOG_GIN_VACUUM_DATA_LEAF_PAGE case so that we don't bother

Re: [HACKERS] Choosing parallel_degree

2016-09-01 Thread Simon Riggs
On 12 April 2016 at 14:11, Simon Riggs wrote: > On 12 April 2016 at 13:53, Robert Haas wrote: >> >> On Mon, Apr 11, 2016 at 5:45 PM, Simon Riggs >> wrote: >> > On 8 April 2016 at 17:49, Robert Haas

Re: [HACKERS] Postgres abort found in 9.3.11

2016-09-01 Thread Tom Lane
"K S, Sandhya (Nokia - IN/Bangalore)" writes: > Our setup is a hot-standby architecture. This crash is occurring only on > stand-by node. Postgres continues to run without any issues on active node. > Postmaster is waiting for a start and is throwing this message. > Aug

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

2016-09-01 Thread Peter Eisentraut
On 8/15/16 3:39 PM, David Steele wrote: > That patch got me thinking about what else could be excluded and after > some investigation I found the following: pg_notify, pg_serial, > pg_snapshots, pg_subtrans. These directories are all cleaned, zeroed, > or rebuilt on server start. > > The

Re: [HACKERS] [WIP] Patches to enable extraction state of query execution from external session

2016-09-01 Thread Tom Lane
Maksim Milyutin writes: >> On Tue, Aug 30, 2016 at 9:34 AM, Maksim Milyutin >> > wrote: >> Yes, but the problem is that nothing gives you the guarantee that at the >> moment you decide to handle the interrupt,

Re: [HACKERS] PostgreSQL 10 kick-off

2016-09-01 Thread Magnus Hagander
On Sep 1, 2016 17:44, "Fabrízio de Royes Mello" wrote: > > > > On Thu, Sep 1, 2016 at 8:35 AM, Tom Lane wrote: > > > > Magnus Hagander writes: > > >> On Thu, Aug 4, 2016 at 12:09 AM, Fabrízio de Royes Mello > > >>

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2016-09-01 Thread Bruce Momjian
On Thu, Sep 1, 2016 at 02:37:40PM +0530, Pavan Deolasee wrote: > I like the simplified approach, as long as it doesn't block further > improvements. > > > > Yes, the proposed approach is simple yet does not stop us from improving > things > further. Moreover it has shown good

Re: [HACKERS] Declarative partitioning - another take

2016-09-01 Thread Ashutosh Bapat
Here's something I observed with your set of patches posted in June. I have not checked the latest set of patches. So, if it's something fixed, please ignore the mail and sorry for me being lazy. prt1 is partitioned table and it shows following information with \d+ regression=# \d+ prt1

Re: [HACKERS] [Patch] RBTree iteration interface improvement

2016-09-01 Thread Heikki Linnakangas
On 08/26/2016 04:07 PM, Aleksander Alekseev wrote: Another unrelated change in this patch is the addition of rb_rightmost(). It's not used for anything, so I'm not sure what the point is. Then again, there don't seem to be any callers of rb_leftmost() either. It's just something I needed in

Re: [HACKERS] less expensive pg_buffercache on big shmem

2016-09-01 Thread Peter Geoghegan
On Thu, Sep 1, 2016 at 8:19 PM, Andres Freund wrote: > On 2016-09-02 08:31:42 +0530, Robert Haas wrote: >> I wonder whether we ought to just switch from the consistent method to >> the semiconsistent method and call it good. > > +1. I think, before long, we're going to have to

Re: [HACKERS] Hash Indexes

2016-09-01 Thread Amit Kapila
On Thu, Sep 1, 2016 at 11:33 PM, Jesper Pedersen wrote: > On 08/05/2016 07:36 AM, Amit Kapila wrote: > > Needs a rebase. > Done. > > + if (blkno == P_NEW) > + elog(ERROR, "hash AM does not use P_NEW"); > > Left over ? > No. We need this check

Re: [HACKERS] Missing checks when malloc returns NULL...

2016-09-01 Thread Tom Lane
Michael Paquier writes: > On Thu, Sep 1, 2016 at 10:03 PM, Tom Lane wrote: >> Don't see the point really ... it's just more API churn without any >> very compelling reason. > OK, then no objection to your approach. At least I tried. OK, pushed my

Re: [HACKERS] PostgreSQL 10 kick-off

2016-09-01 Thread Fabrízio de Royes Mello
On Thu, Sep 1, 2016 at 8:35 AM, Tom Lane wrote: > > Magnus Hagander writes: > >> On Thu, Aug 4, 2016 at 12:09 AM, Fabrízio de Royes Mello > >> wrote: > >>> If there are no complains about my lack of experience in this field I >

Re: [HACKERS] new gcc warning

2016-09-01 Thread Pavel Stehule
2016-09-01 18:40 GMT+02:00 Tom Lane : > Pavel Stehule writes: > > 2016-09-01 14:31 GMT+02:00 Tom Lane : > >> That should have gone away in commit a2fd62dd5. What version are > >> you looking at? > > > I am checking 9.5 branch and

Re: [HACKERS] WAL consistency check facility

2016-09-01 Thread Robert Haas
On Thu, Sep 1, 2016 at 4:12 PM, Simon Riggs wrote: > So in my current understanding, a hinted change has by definition no > WAL record, so we just ship a FPW. Hmm. An FPW would have to be contained in a WAL record, so it can't be right to say that we ship an FPW for lack

Re: [HACKERS] new gcc warning

2016-09-01 Thread Pavel Stehule
2016-09-01 14:31 GMT+02:00 Tom Lane : > Pavel Stehule writes: > > I see a new warning in upstream > > > r/include/libxml2 -c -o path.o path.c > > path.c: In function ‘has_drive_prefix’: > > path.c:89:26: warning: self-comparison always evaluates to

Re: [HACKERS] new gcc warning

2016-09-01 Thread Tom Lane
Pavel Stehule writes: > 2016-09-01 14:31 GMT+02:00 Tom Lane : >> That should have gone away in commit a2fd62dd5. What version are >> you looking at? > I am checking 9.5 branch and I cannot to find this commit there Hmm ... it wasn't back-patched,

Re: [HACKERS] System load consideration before spawning parallel workers

2016-09-01 Thread Peter Eisentraut
On 8/1/16 2:17 AM, Gavin Flower wrote: > Possibly look how make does it with the '-l' flag? > > '-l 8' don't start more process when load is 8 or greater, works on > Linux at least... The problem with that approach is that it takes about a minute for the load averages figures to be updated, by

Re: [HACKERS] System load consideration before spawning parallel workers

2016-09-01 Thread Peter Eisentraut
On 8/16/16 3:39 AM, Haribabu Kommi wrote: > Yes, we need to consider many parameters as a system load, not just only > the CPU. Here I attached a POC patch that implements the CPU load > calculation and decide the number of workers based on the available CPU > load. The load calculation code is

Re: [HACKERS] [WIP] Patches to enable extraction state of query execution from external session

2016-09-01 Thread Maksim Milyutin
On Tue, Aug 30, 2016 at 9:34 AM, Maksim Milyutin > wrote: On Mon, Aug 29, 2016 at 5:22 PM, maksim

  1   2   >