Re: [HACKERS] Page Scan Mode in Hash Index

Hi, On 03/22/2017 09:32 AM, Ashutosh Sharma wrote: Done. Please refer to the attached v2 version of patch. Thanks. 1) 0001-Rewrite-hash-index-scans-to-work-a-page-at-a-time.patch: this patch rewrites the hash index scan module to work in page-at-a-time mode. It basically introduces two new

Re: [HACKERS] Partitioned tables and relfilenode

Hi, On Thu, Mar 23, 2017 at 11:27 PM, Maksim Milyutin wrote: > Hi! > > I have noticed that there is scheduled unlinking of nonexistent physical > storage under partitioned table when we execute DROP TABLE statement on this > partitioned table. Though this action

[HACKERS] Schedule and Release Management Team for PG10

The Postgres release team has decided that last year's appointment of a Release Management Team worked pretty well, so we're going to run the version-10 release cycle the same way. The members of the RMT for this year will be Peter Eisentraut, Robert Haas, and Noah Misch. As previously agreed at

Re: [HACKERS] Partitioned tables and relfilenode

Hi! I have noticed that there is scheduled unlinking of nonexistent physical storage under partitioned table when we execute DROP TABLE statement on this partitioned table. Though this action doesn't generate any error under typical behavior of postgres because the error of storage's lack is

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

> On 23 Mar 2017, at 15:53, Craig Ringer wrote: > > On 23 March 2017 at 19:33, Alexey Kondratov > wrote: > >> (1) Add errors handling to COPY as a minimum program > > Huge +1 if you can do it in an efficient way. > > I think the main

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 3:44 PM, Pavan Deolasee wrote: > On Wed, Mar 22, 2017 at 4:06 PM, Amit Kapila > wrote: >> > >> 3. >> + /* >> + * HASH indexes compute a hash value of the key and store that in the >> + * index. So >> we must first obtain

Re: [HACKERS] Monitoring roles patch

Hi On Thu, Mar 23, 2017 at 12:23 PM, Stephen Frost wrote: > >> diff --git a/contrib/pg_stat_statements/pg_stat_statements.c >> b/contrib/pg_stat_statements/pg_stat_statements.c >> index 221ac98d4a..cec94d5896 100644 >> --- a/contrib/pg_stat_statements/pg_stat_statements.c >>

Re: [HACKERS] pageinspect and hash indexes

Hi Amit, >>> + >>> + /* Check if it is an unused hash page. */ >>> + if (pageopaque->hasho_flag == LH_UNUSED_PAGE) >>> + return page; >>> >>> >>> I don't see we need to have a separate check for unused page, why >>> can't it be checked with other page types in below check. >>> >>> if (pagetype !=

Re: [HACKERS] Logical replication existing data copy

On 3/21/17 21:38, Peter Eisentraut wrote: > This patch is looking pretty good to me, modulo the failing pg_dump tests. > > Attached is a fixup patch. I have mainly updated some comments and > variable naming for (my) clarity. No functional changes. Committed all that. -- Peter Eisentraut

Re: [HACKERS] postgres_fdw: support parameterized foreign joins

On 2017/03/21 18:38, Etsuro Fujita wrote: On 2017/03/16 22:23, Arthur Zakirov wrote: Can you rebase the patch? It is not applied now. Ok, will do. Thanks for the report! Done. Also, I added regression tests and revised code and comments a bit. (As for create_foreignscan_path(), I

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

On 23 March 2017 at 19:33, Alexey Kondratov wrote: > (1) Add errors handling to COPY as a minimum program Huge +1 if you can do it in an efficient way. I think the main barrier to doing so is that the naïve approach creates a subtransaction for every row, which is

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

>> 1) Is there anyone out of PG comunity who will be interested in such >> project and can be a menthor? >> 2) These two points have a general idea – to simplify work with a large >> amount of data from a different sources, but mybe it would be better to >> focus on the single task? >> > > I spent

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

On Thursday, March 23, 2017 4:50:20 AM EDT Magnus Hagander wrote: > We should probably consider if there is some way we can implement > these two things the same way. If we're inventing a new variable that > gets pushed on each connection, perhaps we can use that one and avoid > the SHOW command?

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

Hi 2017-03-23 12:33 GMT+01:00 Alexey Kondratov : > Hi pgsql-hackers, > > I'm planning to apply to GSOC'17 and my proposal consists currently of two > parts: > > (1) Add errors handling to COPY as a minimum program > > Motivation: Using PG on the daily basis for years

Re: [HACKERS] postgres_fdw bug in 9.6

On 2017/03/21 18:40, Etsuro Fujita wrote: Ok, I'll update the patch. One thing I'd like to revise in addition to that is (1) add to JoinPathExtraData a flag member to indicate whether to give the FDW a chance to consider a remote join, which will be set to true if the joinrel's fdwroutine is

Re: [HACKERS] [GSoC] Push-based query executor discussion

Oleg Bartunov writes: I don't know exactly if the results of GSoC project should be committed, Technically, they are not required: https://developers.google.com/open-source/gsoc/faq Are mentoring organizations required to use the code produced by students? No. While

Re: [HACKERS] Monitoring roles patch

Dave, * Dave Page (dp...@pgadmin.org) wrote: > On Wed, Mar 22, 2017 at 3:46 PM, Dave Page wrote: > > On Wed, Mar 22, 2017 at 1:15 PM, Stephen Frost wrote: > >> I did specifically ask for explicit roles to be made to enable such > >> capability and that the

Re: [HACKERS] Create replication slot in pg_basebackup if requested and not yet present

On Tue, Mar 21, 2017 at 8:34 PM, Robert Haas wrote: > On Sun, Mar 19, 2017 at 12:01 PM, Magnus Hagander > wrote: > > I think maybe we should output a message when the slot is created, at > least > > in verbose mode, to make sure people realize that

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 4:08 PM, Pavan Deolasee wrote: > > > On Thu, Mar 23, 2017 at 3:02 PM, Amit Kapila > wrote: > >> >> >> That sounds like you are dodging the actual problem. I mean you can >> put that same PageIsFull() check in master

[HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

Hi pgsql-hackers, I'm planning to apply to GSOC'17 and my proposal consists currently of two parts: (1) Add errors handling to COPY as a minimum program Motivation: Using PG on the daily basis for years I found that there are some cases when you need to load (e.g. for a further analytics) a

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

On Thu, Mar 23, 2017 at 8:19 PM, Kuntal Ghosh wrote: > Hence, to be consistent with others, bgworker processes can be > initialized from BackgroundWorkerInitializeConnectionBy[Oid]. Yeah, I am fine with that. Thanks for the updated versions. -- Michael -- Sent via

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

On Wed, Mar 22, 2017 at 9:54 PM, Robert Haas wrote: > On Wed, Mar 22, 2017 at 12:20 PM, Robert Haas wrote: >> On Wed, Mar 22, 2017 at 1:31 AM, Michael Paquier >> wrote: >>> Okay, switched as ready for committer. One note

Re: [HACKERS] Partition-wise aggregation/grouping

On Tue, Mar 21, 2017 at 1:47 PM, Antonin Houska wrote: > Jeevan Chalke wrote: > > > Declarative partitioning is supported in PostgreSQL 10 and work is > already in > > progress to support partition-wise joins. Here is a proposal for >

Re: [HACKERS] pgbench - allow to store select results into variables

On Thu, Mar 16, 2017 at 12:45 AM, Fabien COELHO wrote: > > Hello Rafia, > >> I was reviewing v7 of this patch, to start with I found following white >> space errors when applying with git apply, >> /home/edb/Desktop/patches/others/pgbench-into-7.patch:66: trailing >>

Re: [HACKERS] Parallel Append implementation

On 23 March 2017 at 05:55, Robert Haas wrote: > On Wed, Mar 22, 2017 at 4:49 AM, Amit Khandekar > wrote: >> Attached is the updated patch that handles the changes for all the >> comments except the cost changes part. Details about the specific >>

Re: [HACKERS] segfault in hot standby for hash indexes

On Thu, Mar 23, 2017 at 9:17 AM, Amit Kapila wrote: > > On Thu, Mar 23, 2017 at 8:43 AM, Amit Kapila wrote: > > On Wed, Mar 22, 2017 at 3:39 PM, Ashutosh Sharma > > wrote: > >> Hi, > >> > >> On Wed, Mar 22, 2017 at 8:41

Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall

On 03/21/2017 08:02 AM, Haribabu Kommi wrote: Solution -1) Just ignore dumping these CREATE DATABASE commands and provide the user information in the documentation to create "postgres" and "template1" database in the target in case if they don't exist. If this kind of cases are very rare.

Re: [HACKERS] createlang/droplang deprecated

> On 20 Mar 2017, at 01:37, Peter Eisentraut > wrote: > > On 3/18/17 09:00, Peter Eisentraut wrote: >> I just noticed that createlang and droplang have been listed as >> deprecated since PG 9.1. >> >> Do we dare remove them? > > Patch LGTM, +1 cheers

Re: [HACKERS] Monitoring roles patch

On Wed, Mar 22, 2017 at 3:46 PM, Dave Page wrote: > On Wed, Mar 22, 2017 at 1:15 PM, Stephen Frost wrote: >> >> I did specifically ask for explicit roles to be made to enable such >> capability and that the pg_monitor role be GRANT'd those roles instead >>

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 3:02 PM, Amit Kapila wrote: > > > That sounds like you are dodging the actual problem. I mean you can > put that same PageIsFull() check in master code as well and then you > will most probably again see the same regression. Well I don't see it

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

Thanks Amit. v19 addresses some of the comments below. On Thu, Mar 23, 2017 at 10:28 AM, Amit Kapila wrote: > On Wed, Mar 22, 2017 at 4:06 PM, Amit Kapila > wrote: > > On Tue, Mar 21, 2017 at 6:47 PM, Pavan Deolasee > >

Re: [HACKERS] Measuring replay lag

On Thu, Mar 23, 2017 at 10:50 PM, Simon Riggs wrote: >> Second thoughts... I'll just make LagTrackerWrite externally >> available, so a plugin can send anything it wants to the tracker. >> Which means I'm explicitly removing the "logical replication support" >> from this

Re: [HACKERS] identity columns

On 3/22/17, Peter Eisentraut wrote: > On 3/22/17 03:59, Vitaly Burovoy wrote: >> Column's IDENTITY behavior is very similar to a DEFAULT one. We write >> "SET DEFAULT" and don't care whether it was set before or not, because >> we can't have many of them for a

Re: [HACKERS] Measuring replay lag

> Second thoughts... I'll just make LagTrackerWrite externally > available, so a plugin can send anything it wants to the tracker. > Which means I'm explicitly removing the "logical replication support" > from this patch. Done. Here's the patch I'm looking to commit, with some docs and minor

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 16:07, Craig Ringer wrote: > If preferred I can instead add > > proc.h: > > #define PROC_RESERVED 0x20 > > procarray.h: > > #define PROCARRAY_REPLICATION_SLOTS 0x20 > > and then test for (flags & PROCARRAY_REPLICATION_SLOTS) Attached done that way. --

Re: [HACKERS] ICU integration

I am fine with this version of the patch. The issues I have with it, which I mentioned earlier in this thread, seem to be issues with ICU rather than with this patch. For example there seems to be no way for ICU to validate the syntax of the BCP 47 locales (or ICU's old format). But I think we

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 12:19 AM, Pavan Deolasee wrote: > > Ok, no problem. I did some tests on AWS i2.xlarge instance (4 vCPU, 30GB > RAM, attached SSD) and results are shown below. But I think it is important > to get independent validation from your side too, just to

Re: [HACKERS] createlang/droplang deprecated

On Mon, Mar 20, 2017 at 1:37 AM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 3/18/17 09:00, Peter Eisentraut wrote: > > I just noticed that createlang and droplang have been listed as > > deprecated since PG 9.1. > > > > Do we dare remove them? > > Patch > > +1 -- Magnus

Re: [HACKERS] Fix for grammatical error in code-comment

On Wed, Mar 22, 2017 at 2:47 PM, Emil Iggland wrote: > I found a grammatical error in one of the code comments. > > The comment is made regarding the run-time of the algorithm, and > references big-O performance. > However the comment text uses "effective", instead of the

Re: [HACKERS] bug/oversight in TestLib.pm and PostgresNode.pm

On 2017-03-23 03:28, Michael Paquier wrote: On Thu, Mar 23, 2017 at 12:51 AM, Erik Rijkers wrote: While trying to test pgbench's stderr (looking for 'creating tables' in output of the initialisation step) I ran into these two bugs (or perhaps better 'oversights'). + if

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

On Wed, Mar 22, 2017 at 9:00 PM, Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 3/22/17 14:09, Robert Haas wrote: > >> The opposite means primary. I can flip the GUC name to "is_primary", if > >> that's clearer. > > Hmm, I don't find that clearer. "hot standby" has a very

Re: [HACKERS] Measuring replay lag

On 23 March 2017 at 06:42, Simon Riggs wrote: > On 23 March 2017 at 01:02, Thomas Munro wrote: > >> Thanks! Please find attached v7, which includes a note we can point >> at when someone asks why it doesn't show 00:00:00, as requested. > >

Re: [HACKERS] Multiple false-positive warnings from Valgrind

On Tue, Mar 21, 2017 at 10:57 PM, Aleksander Alekseev wrote: > Recently I've decided to run PostgreSQL under Valgrind according to wiki > description [1]. Lots of warnings are generated [2] but it is my > understanding that all of them are false-positive. For instance

Re: [HACKERS] Logical decoding on standby

On 23 March 2017 at 00:13, Simon Riggs wrote: > On 22 March 2017 at 08:53, Craig Ringer wrote: > >> I'm splitting up the rest of the decoding on standby patch set with >> the goal of getting minimal functionality for creating and managing >>

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

Hi All, I have tried to test 'group_update_clog_v11.1.patch' shared upthread by Amit on a high end machine. I have tested the patch with various savepoints in my test script. The machine details along with test scripts and the test results are shown below, Machine details: 24

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

Hi, Here is a new patch series responding to feedback from Peter and Andres: 1. Support pgstat_report_tempfile and log_temp_files, which I had overlooked as Peter pointed out. 2. Use a patch format that is acceptable to git am, per complaint off-list from Andres. (Not actually made with git

Re: [HACKERS] pageinspect and hash indexes

On Thu, Mar 23, 2017 at 10:02 AM, Ashutosh Sharma wrote: >>> >> >> 1. >> @@ -70,6 +70,17 @@ verify_hash_page(bytea *raw_page, int flags) >> pageopaque = (HashPageOpaque) PageGetSpecialPointer(page); >> + >> + /* Check if it is an unused hash page. */ >> + if

Re: [HACKERS] Measuring replay lag

On 23 March 2017 at 01:02, Thomas Munro wrote: > Thanks! Please find attached v7, which includes a note we can point > at when someone asks why it doesn't show 00:00:00, as requested. Thanks. Now I look harder the handling for logical lag seems like it would be

Re: [HACKERS] ANALYZE command progress checker

On Wed, Mar 22, 2017 at 8:11 PM, vinayak wrote: > > On 2017/03/21 21:25, Haribabu Kommi wrote: > > > > On Tue, Mar 21, 2017 at 3:41 PM, vinayak > wrote: > >> Thank you for testing the patch on Windows platform. >> >> > Thanks for

<    1   2