Re: [HACKERS] Checksums by default?

2017-01-22 Thread Tomas Vondra
On 01/23/2017 08:30 AM, Amit Kapila wrote: On Sun, Jan 22, 2017 at 3:43 PM, Tomas Vondra wrote: That being said, I'm ready to do some benchmarking on this, so that we have at least some numbers to argue about. Can we agree on a set of workloads that we want to

Re: [HACKERS] Checksums by default?

2017-01-22 Thread Amit Kapila
On Sun, Jan 22, 2017 at 3:43 PM, Tomas Vondra wrote: > > That being said, I'm ready to do some benchmarking on this, so that we have > at least some numbers to argue about. Can we agree on a set of workloads > that we want to benchmark in the first round? > I think

Re: [HACKERS] macaddr 64 bit (EUI-64) datatype support

2017-01-22 Thread Kuntal Ghosh
On Thu, Jan 19, 2017 at 7:46 AM, Haribabu Kommi wrote: > > Updated patch attached. > I've looked at the updated patch. There are still some changes that needs to be done. It includes: 1. Adding macaddr8 support for btree_gist and testcases. 2. Implementation of macaddr8

Re: [HACKERS] Checksums by default?

2017-01-22 Thread Vladimir Borodin
> 21 янв. 2017 г., в 18:18, Petr Jelinek > написал(а): > > On 21/01/17 11:39, Magnus Hagander wrote: >> Is it time to enable checksums by default, and give initdb a switch to >> turn it off instead? +1 > > I'd like to see benchmark first, both in terms of CPU

Re: [HACKERS] Assignment of valid collation for SET operations on queries with UNKNOWN types.

2017-01-22 Thread Michael Paquier
On Mon, Jan 23, 2017 at 7:53 AM, Tom Lane wrote: >> 2. I didn't do anything about docs, either, though maybe no change >> is needed. One user-visible change from this is that queries should >> never return any "unknown"-type columns to clients anymore. But I >> think that is

Re: [HACKERS] [PATCH] Fix minor race in commit_ts SLRU truncation vs lookups

2017-01-22 Thread Craig Ringer
On 23 January 2017 at 12:34, Craig Ringer wrote: > On 20 January 2017 at 21:40, Alvaro Herrera wrote: > >> One option would be to add another limit Xid which advances before the >> truncation but which is not used for other decisions other than

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-22 Thread Jeff Davis
On Sat, Jan 21, 2017 at 4:25 AM, Andrew Borodin wrote: > Hello Jeff! > >>Review of the code itself: > How do you think, is there anything else to improve in that patch or > we can mark it as 'Ready for committer'? > > I've checked the concurrency algorithm with original work

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-01-22 Thread Craig Ringer
On 28 December 2016 at 18:00, Craig Ringer wrote: > On 23 December 2016 at 18:00, Craig Ringer wrote: > >> I'll have to follow up with a patch soon, as it's Toddler o'Clock. > > Here we go. > > This patch advances oldestXid, under XidGenLock,

Re: [HACKERS] Parallel Index Scans

2017-01-22 Thread Amit Kapila
On Fri, Jan 20, 2017 at 7:29 AM, Haribabu Kommi wrote: > > On Thu, Jan 19, 2017 at 1:18 AM, Amit Kapila > wrote: >> > >> > +extern BlockNumber _bt_parallel_seize(IndexScanDesc scan, bool >> > *status); >> > +extern void

Re: [HACKERS] increasing the default WAL segment size

2017-01-22 Thread Beena Emerson
Hello, Please find attached an updated WIP patch. I have incorporated almost all comments. This is to be applied over Robert's patches. I will post performance results later on. 1. shift (>>) and AND (&) operations: The assign hook of wal_segment_size sets the WalModMask and WalShiftBit. All the

Re: [HACKERS] Parallel Index Scans

2017-01-22 Thread Amit Kapila
On Sat, Jan 21, 2017 at 12:23 PM, Amit Kapila wrote: > On Sat, Jan 21, 2017 at 1:15 AM, Robert Haas wrote: >> >> I think it's a good idea to put all three of those functions together >> in the listing, similar to what we did in >>

Re: [HACKERS] Declarative partitioning - another take

2017-01-22 Thread Amit Langote
On 2017/01/21 6:29, Robert Haas wrote: > On Fri, Jan 20, 2017 at 1:15 AM, Andres Freund wrote: >> On 2017-01-19 14:18:23 -0500, Robert Haas wrote: >>> Committed. >> >> One of the patches around this topic committed recently seems to cause >> valgrind failures like >>

Re: [HACKERS] Valgrind-detected bug in partitioning code

2017-01-22 Thread Amit Langote
On 2017/01/21 9:01, Tom Lane wrote: > Robert Haas writes: >> The difference is that those other equalBLAH functions call a >> carefully limited amount of code whereas, in looking over the >> backtrace you sent, I realized that equalPartitionDescs is calling >>

Re: [HACKERS] Logical replication launcher's bgworker enabled by default, and max_logical_replication_workers

2017-01-22 Thread Craig Ringer
On 23 January 2017 at 13:19, Jaime Casanova wrote: > On 22 January 2017 at 23:37, Michael Paquier > wrote: >> Hi all, >> >> When spawning a new instance, I found the following thing, which is >> surprising at first sight: >> postgres:

Re: [HACKERS] Logical replication launcher's bgworker enabled by default, and max_logical_replication_workers

2017-01-22 Thread Jaime Casanova
On 22 January 2017 at 23:37, Michael Paquier wrote: > Hi all, > > When spawning a new instance, I found the following thing, which is > surprising at first sight: > postgres: bgworker: logical replication launcher > This is because the downstream needs it

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-01-22 Thread Michael Paquier
On Mon, Jan 23, 2017 at 2:06 PM, Venkata B Nagothi wrote: > > On Sat, Jan 14, 2017 at 5:10 AM, Vladimir Rusinov > wrote: >> >> Attached are two new version of the patch: one keeps aliases, one don't. > > > Both the patches (with and without aliases) are

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-01-22 Thread Venkata B Nagothi
On Sat, Jan 14, 2017 at 5:10 AM, Vladimir Rusinov wrote: > Attached are two new version of the patch: one keeps aliases, one don't. > Both the patches (with and without aliases) are not getting applied to the latest master. Below is the error - Perhaps you used the wrong

Re: [HACKERS] Commit fest 2017-01 will begin soon!

2017-01-22 Thread Stephen Frost
Michael, all, * Michael Paquier (michael.paqu...@gmail.com) wrote: > There is one week left for this commit fest, and now is the time to > push for things that have a chance to get in. As of now, there is a > total of 25 patches waiting for some love from a committer. Many thanks for your work

Re: [HACKERS] Parallel bitmap heap scan

2017-01-22 Thread Dilip Kumar
On Thu, Jan 19, 2017 at 12:26 AM, Robert Haas wrote: >> Patch 0001 and 0003 required to rebase on the latest head. 0002 is >> still the same. > > I've committed the first half of 0001. Thanks. 0001 and 0003 required rebasing after this commit. -- Regards, Dilip Kumar

[HACKERS] Logical replication launcher's bgworker enabled by default, and max_logical_replication_workers

2017-01-22 Thread Michael Paquier
Hi all, When spawning a new instance, I found the following thing, which is surprising at first sight: postgres: bgworker: logical replication launcher There is perhaps no problem to keep that enabled by default until the release 10 wraps to give it some buildfarm coverage similarly to what has

Re: [HACKERS] [PATCH] Fix minor race in commit_ts SLRU truncation vs lookups

2017-01-22 Thread Craig Ringer
On 20 January 2017 at 21:40, Alvaro Herrera wrote: > One option would be to add another limit Xid which advances before the > truncation but which is not used for other decisions other than limiting > what can users consult. This could be useful for other things, but

Re: [HACKERS] Commit fest 2017-01 will begin soon!

2017-01-22 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Michael Paquier > - Cascading standby cannot catch up and get stuck emitting the same message > repeatedly, a 9.3-only bug fix. Thank you for your hard work as a CFM. For a trivial note, this is

Re: [HACKERS] [PATCH] Fix minor race in commit_ts SLRU truncation vs lookups

2017-01-22 Thread Craig Ringer
On 20 January 2017 at 05:32, Peter Eisentraut wrote: > On 1/19/17 10:06 AM, Alvaro Herrera wrote: >>> Also, I wonder whether we should not in vacuum.c change the order of the >>> calls of SetTransactionIdLimit() and SetMultiXactIdLimit() as well, just >>> to keep

Re: [HACKERS] Fix a comment in feelist.c

2017-01-22 Thread Tatsuo Ishii
> I've found a mistake in a comment of StrategyNotifyBgWriter > in freelist.c. bgwriterLatch was replaced by bgwprocno in > the following commit, but this is remained in the comment. > > commit d72731a70450b5e7084991b9caa15cb58a2820df > Author: Andres Freund > Date: Thu Dec

Re: [HACKERS] Commit fest 2017-01 will begin soon!

2017-01-22 Thread Michael Paquier
On Wed, Jan 4, 2017 at 4:02 PM, Michael Paquier wrote: > Here are now the current stats of the patches: > - Needs review: 82. > - Waiting on Author: 18. > - Ready for Committer: 18. > Meaning that some effort is required from committers and reviewers to > get into a

Re: [HACKERS] pg_dump, pg_dumpall and data durability

2017-01-22 Thread Michael Paquier
On Tue, Nov 29, 2016 at 1:30 PM, Michael Paquier wrote: > On Mon, Nov 14, 2016 at 6:07 PM, Michael Paquier > wrote: >> On Mon, Nov 14, 2016 at 6:04 PM, Albe Laurenz >> wrote: >>> Michael Paquier wrote: Meh. I

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2017-01-22 Thread Michael Paquier
On Wed, Jan 18, 2017 at 2:46 PM, Michael Paquier wrote: > FWIW, this patch is on a "waiting on author" state and that's right. > As the discussion on SASLprepare() and the decisions regarding the way > to implement it, or at least have it, are still pending, I am not >

[HACKERS] Enabling replication connections by default in pg_hba.conf

2017-01-22 Thread Michael Paquier
Hi all, As now wal_level = replica has become the default for Postgres 10, could we consider as well making replication connections enabled by default in pg_hba.conf? This requires just uncommenting a couple of lines in pg_hba.conf.sample. Thanks, -- Michael pghba_rep_default.patch

Re: [HACKERS] Checksums by default?

2017-01-22 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Magnus Hagander > Is it time to enable checksums by default, and give initdb a switch to turn > it off instead? > > I keep running into situations where people haven't enabled it, because > (a)

Re: [HACKERS] Logical Replication WIP

2017-01-22 Thread Petr Jelinek
On 20/01/17 22:30, Petr Jelinek wrote: > Since it's not exactly straight forward to find when these need to be > initialized based on commands, I decided to move the initialization code > to exec_replication_command() since that's always called before anything > so that makes it much less error

\if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-01-22 Thread Corey Huinker
> > > Fabien is pressed for time, so I've been speaking with him out-of-thread > about how I should go about implementing it. > > The v1 patch will be \if , \elseif , \else, \endif, where > will be naively evaluated via ParseVariableBool(). > > \ifs and \endifs must be in the same "file" (each

Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)

2017-01-22 Thread Jim Nasby
On 1/16/17 10:09 PM, Haribabu Kommi wrote: Yes, that' correct. Currently with this approach, it is not possible to ditch the heap completely. This approach is useful for the cases, where the user wants to store only some columns as part of clustered index. Ahh, that's unfortunate. Billion row+

Re: [HACKERS] Assignment of valid collation for SET operations on queries with UNKNOWN types.

2017-01-22 Thread Tom Lane
I wrote: > I spent some time fooling with this today and came up with the attached > patch. I think this is basically the direction we should go in, but > there are various loose ends: Here's an updated patch that's rebased against today's HEAD and addresses most of the loose ends: > 1. I

Re: [HACKERS] Logical replication failing when foreign key present

2017-01-22 Thread Petr Jelinek
On 22/01/17 18:50, Thom Brown wrote: > Hi, > > There's an issue which I haven't seen documented as expected > behaviour, where replicating data to a table which has a foreign key > results in a replication failure. This produces the following log > entries: > > LOG: starting logical

Re: [HACKERS] Online enabling of page level checksums

2017-01-22 Thread Jim Nasby
On 1/22/17 5:13 AM, Magnus Hagander wrote: If the system is interrupted before the background worker is done, it starts over from the beginning. Previously touched blocks will be read and verified, but not written (because their checksum is already correct). This will take time, but not

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-22 Thread Tom Lane
Jim Nasby writes: >> Ahh, I hadn't considered that. So one idea would be to only track >> negative entries on caches where we know they're actually useful. That >> might make the performance hit of some of the other ideas more >> tolerable. Presumably you're much less

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-22 Thread Jim Nasby
On 1/22/17 4:41 PM, Jim Nasby wrote: On 1/21/17 8:54 PM, Tom Lane wrote: Jim Nasby writes: The other (possibly naive) question I have is how useful negative entries really are? Will Postgres regularly incur negative lookups, or will these only happen due to user

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-22 Thread Jim Nasby
On 1/21/17 8:54 PM, Tom Lane wrote: Jim Nasby writes: The other (possibly naive) question I have is how useful negative entries really are? Will Postgres regularly incur negative lookups, or will these only happen due to user activity? It varies depending on the

Re: [HACKERS] Too many autovacuum workers spawned during forced auto-vacuum

2017-01-22 Thread Jim Nasby
On 1/20/17 12:40 AM, Amit Khandekar wrote: My impression was that postmaster is supposed to just do a minimal work of starting auto-vacuum launcher if not already. And, the work of ensuring all the things keep going is the job of auto-vacuum launcher. There's already a ton of logic in the

Re: [HACKERS] PATCH: recursive json_populate_record()

2017-01-22 Thread Tom Lane
Nikita Glukhov writes: > On 01/08/2017 09:52 PM, Tom Lane wrote: >> ... attndims is really syntactic sugar only and doesn't affect anything >> meaningful semantically. > I have fixed the first patch: when the number of dimensionsis unknown > we determine it simply by the

[HACKERS] Logical replication failing when foreign key present

2017-01-22 Thread Thom Brown
Hi, There's an issue which I haven't seen documented as expected behaviour, where replicating data to a table which has a foreign key results in a replication failure. This produces the following log entries: LOG: starting logical replication worker for subscription "contacts_sub" LOG:

Re: [HACKERS] new autovacuum criterion for visible pages

2017-01-22 Thread Stephen Frost
Amit, * Amit Kapila (amit.kapil...@gmail.com) wrote: > On Sun, Jan 22, 2017 at 3:27 AM, Stephen Frost wrote: > > * Simon Riggs (si...@2ndquadrant.com) wrote: > >> On 12 August 2016 at 01:01, Tom Lane wrote: > >> > Michael Paquier

[HACKERS] Online enabling of page level checksums

2017-01-22 Thread Magnus Hagander
So, that post I made about checksums certainly spurred a lot of discussion :) One summary is that life would be a lot easier if we could turn checksums on (and off) without re-initdbing. I'm breaking out this question into this thread to talk about it separately. I've been toying with a branch

Re: [HACKERS] Checksums by default?

2017-01-22 Thread Magnus Hagander
On Sat, Jan 21, 2017 at 8:16 PM, Ants Aasma wrote: > On Sat, Jan 21, 2017 at 7:39 PM, Petr Jelinek > wrote: > > So in summary "postgresql.conf options are easy to change" while "initdb > > options are hard to change", I can see this argument

Re: [HACKERS] Checksums by default?

2017-01-22 Thread Tomas Vondra
On 01/21/2017 04:18 PM, Petr Jelinek wrote: On 21/01/17 11:39, Magnus Hagander wrote: Is it time to enable checksums by default, and give initdb a switch to turn it off instead? I'd like to see benchmark first, both in terms of CPU and in terms of produced WAL (=network traffic) given that it

Re: [HACKERS] Checksums by default?

2017-01-22 Thread Tomas Vondra
On 01/21/2017 05:53 PM, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost writes: * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: The change of wal_level was supported by benchmark, I think it's reasonable to ask for this to be as well. No,

Re: [HACKERS] Checksums by default?

2017-01-22 Thread Tomas Vondra
On 01/21/2017 05:51 PM, Stephen Frost wrote: * Petr Jelinek (petr.jeli...@2ndquadrant.com) wrote: On 21/01/17 17:31, Stephen Frost wrote: This is just changing the *default*, not requiring checksums to always be enabled. We do not hold the same standards for our defaults as we do for

Re: [HACKERS] Checksums by default?

2017-01-22 Thread Tomas Vondra
On 01/21/2017 05:35 PM, Tom Lane wrote: Stephen Frost writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: Have we seen *even one* report of checksums catching problems in auseful way? This isn't the right question. I disagree. If they aren't doing something useful for

Re: [HACKERS] [COMMITTERS] pgsql: Add function to import operating system collations

2017-01-22 Thread Michael Paquier
On Sun, Jan 22, 2017 at 5:28 AM, Tom Lane wrote: > Peter Eisentraut writes: >> On 1/21/17 12:50 PM, Tom Lane wrote: >>> I have to question the decision to make "no locales" a hard error. >>> What's the point of that? In fact, should we even