Re: Online enabling of checksums

2018-03-01 Thread Andres Freund
On 2018-03-01 12:56:35 +0500, Andrey Borodin wrote: > I've tried to rebase this patch to 10 and, despite minor rebase issues (oids, > bgw_type, changes to specscanner), patch works fine. > Do we provide backporting for such features? Definitely not. With very rare exceptions (OS compatibility

Re: 2018-03 Commitfest starts tomorrow

2018-03-01 Thread Michael Paquier
On Thu, Mar 01, 2018 at 12:42:25AM -0500, Tom Lane wrote: > I think that we should summarily bounce to the September 'fest anything > submitted in the last two days; certainly anything that's nontrivial. I don't have any issues with simple patches posted at the last minute. There is for example a

Re: Cast jsonb to numeric, int, float, bool

2018-03-01 Thread Komяpa
> Attached new version of the patch in which I removed duplicated code using new subroutine JsonbExtractScalar(). I am not sure what is better to do when a JSON item has an unexpected type: to throw an error or to return SQL NULL. Also JSON nulls could be converted to SQL NULLs. I would expect

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2018-03-01 Thread Andres Freund
On 2018-01-28 23:02:57 +, Rady, Doug wrote: > diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c > index 31ea6ca06e..689f15a772 100644 > --- a/src/bin/pgbench/pgbench.c > +++ b/src/bin/pgbench/pgbench.c > @@ -44,7 +44,13 @@ > #include > #include > #include > -#ifdef

Re: [PATCH] Find additional connection service files in pg_service.conf.d directory

2018-03-01 Thread Andres Freund
Hi, On 2018-01-12 09:53:51 -0500, Curt Tilmes wrote: > I love centralizing connection service definitions in > /pg_service.conf, but for a > large enterprise, sometimes we have multiple sets of connection > service definitions > independently managed. > > The convention widely adopted for this

Re: pgbench - test whether a variable exists

2018-03-01 Thread Andres Freund
Hi, On 2018-02-19 19:23:04 +0100, Fabien COELHO wrote: > Note that it is not really that useful for benchmarking, although it does > not harm. That seems plenty reason to plainly reject this patch? If we end up unifying it'll be added via that, no? - Andres

Re: [PoC PATCH] Parallel dump to /dev/null

2018-03-01 Thread Andres Freund
Hi, On 2018-02-28 14:28:41 +0100, Michael Banck wrote: > I have added this patch to the next commitfest: > > https://commitfest.postgresql.org/17/1576/ Given this is a new patch, submitted for the last commitfest, and not completely trivial, I'd argue this is too late for v11. Does anybody

[GSOC 18] Performance Farm Project

2018-03-01 Thread 马来酸
Dear Sir, Hi, my name is Hongyuan Ma. I am a junior student in software engineering in Shanghai.I am interested in the idea "Develop Performance Farm Database and Website".I started developing web applications since my sophomore year and have experience in developing web sites. I took a look

Re: TAP test module - PostgresClient

2018-03-01 Thread Andres Freund
Hi, On 2018-01-11 16:54:10 +0900, Kyotaro HORIGUCHI wrote: > Thank you for the discussion. If I understand correctly there's been no progress on this since, and there'd definitely need to be major work to get something we can agree upon. Doesn't seem v11 material. I think we should mark this as

Re: PATCH: pgbench - break out timing data for initialization phases

2018-03-01 Thread Andres Freund
Hi, On 2018-02-21 17:58:49 +, Rady, Doug wrote: > - move the time measure in the initialization loop, instead of doing it > in each function, so that it is done just in one place. > > I will do this. Given the last v11 CF is just about to start, there's no new version yet, the

Re: Missing comment edit

2018-03-01 Thread Andres Freund
On 2018-02-28 19:03:01 -0500, Tom Lane wrote: > Kyotaro HORIGUCHI writes: > > I happend to find that the comment on formdesc is missing > > pg_subscription. Please find the attached patch (I'm sure:) to > > fix that . > > Hmm ... certainly, that comment is now

Re: Why chain of snapshots is used in ReorderBufferCommit?

2018-03-01 Thread Arseny Sher
Andres Freund writes: > I don't think that's right. For one there's plenty types of DDL where no > such locking exists, consider e.g. composite datums where additional > columns can be added even after such a type is already used by another > table. Oh, indeed. Never mind

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-01 Thread Jeevan Chalke
On Mon, Feb 26, 2018 at 8:03 PM, Robert Haas wrote: > > Committed after incorporating your other fixes and updating the > optimizer README. > Attached new patchset after rebasing my changes over these changes and on latest HEAD. > > -- > Robert Haas > EnterpriseDB:

Re: Failed to request an autovacuum work-item in silence

2018-03-01 Thread Andres Freund
Hi, On 2018-01-23 14:35:42 +0900, Masahiko Sawada wrote: > While reading the code, I realized that the requesting an autovacuum > work-item could fail in silence if work-item array is full. So the > users cannot realize that work-item is never performed. > AutoVacuumRequestWork() seems to behave

Re: [bug fix] pg_rewind creates corrupt WAL files, and the standby cannot catch up the primary

2018-03-01 Thread Michael Paquier
On Thu, Mar 01, 2018 at 07:49:06AM +, Tsunakawa, Takayuki wrote: > It's a regret that Chen's patch, which limits the WAL to be copied, is > not committed yet. It looks good to be ready for committer. The message I sent provides reasons about why it should not be integrated. Particularly

Re: Documenting PROVE_TESTS in section of TAP tests

2018-03-01 Thread Andres Freund
Hi, On 2018-02-17 23:03:05 +0900, Michael Paquier wrote: > The section of the documentation dedicated to TAP tests mentions > PROVE_FLAGS: > https://www.postgresql.org/docs/devel/static/regress-tap.html > > I think that it would be a good idea to mention PROVE_TESTS as well. I > personally use

Re: 2018-03 Commitfest starts tomorrow

2018-03-01 Thread Andres Freund
Hi, On 2018-03-01 10:52:12 +0100, Magnus Hagander wrote: > 2018-02-28 17:21:13.384903+00 | JIT compiling expressions & tuple deforming Note this is an old thread, just a new CF entry. > > There is no way that we can possibly handle 200+ CF entries in a month. > > A large fraction of these are

Re: pgbench - test whether a variable exists

2018-03-01 Thread Fabien COELHO
Note that it is not really that useful for benchmarking, although it does not harm. That seems plenty reason to plainly reject this patch? If we end up unifying it'll be added via that, no? On the contrary, my point is to add the feature beforehand so that it is not intermixed in the

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-03-01 Thread Heikki Linnakangas
On 16/01/18 15:00, Yoshimi Ichiyanagi wrote: Hi. These patches enable to use Persistent Memory Development Kit(PMDK)[1] for reading/writing WAL logs on persistent memory(PMEM). PMEM is next generation storage and it has a number of nice features: fast, byte-addressable and non-volatile.

Re: Online enabling of checksums

2018-03-01 Thread Michael Paquier
On Thu, Mar 01, 2018 at 12:56:35PM +0500, Andrey Borodin wrote: > I've tried to rebase this patch to 10 and, despite minor rebase issues > (oids, bgw_type, changes to specscanner), patch works fine. Do we > provide backporting for such features? New features are not backported in upstream.

Re: [PATCH] pgbench - refactor some connection finish/null into common function

2018-03-01 Thread Andres Freund
On 2018-02-21 19:11:15 +, Rady, Doug wrote: > > On 1/30/18, 03:41, "Fabien COELHO" wrote: > Hello Doug, > Hi Fabien, > > > This patch refactors all of the connection state PQfinish() and > NULL’ing into a single function. > > Excludes PQfinish() in

Re: [HACKERS] [PATCH v1] Add and report the new "in_hot_standby" GUC pseudo-variable.

2018-03-01 Thread Andres Freund
On 2017-11-28 10:55:34 +0900, Michael Paquier wrote: > On Mon, Oct 2, 2017 at 4:16 PM, Daniel Gustafsson wrote: > > Based on the unaddressed questions raised in this thread, I’m marking this > > patch Returned with Feedback. Please re-submit a new version of the patch > > to a

Re: Minor fix for pgbench documentation

2018-03-01 Thread Andres Freund
On 2018-01-15 12:30:51 +0300, Ildar Musin wrote: > Hello Fabien, > > > 13/01/2018 19:30, Fabien COELHO пишет: > > > >> Here is a patch that adds missing random_zipfian func to the paragraph > >> in pgbench documentation about random functions parameterization. > > > > Indeed. > > > > Patch

Re: [doc fix] Correct calculation of vm.nr_hugepages

2018-03-01 Thread Andres Freund
On 2018-02-20 02:43:32 +, Tsunakawa, Takayuki wrote: > diff --git a/doc/src/sgml/runtime.sgml b/doc/src/sgml/runtime.sgml > index d162acb..1aed070 100644 > --- a/doc/src/sgml/runtime.sgml > +++ b/doc/src/sgml/runtime.sgml > @@ -1472,14 +1472,14 @@ export PG_OOM_ADJUST_VALUE=0 > the kernel

Re: to_typemod(type_name) information function

2018-03-01 Thread Andres Freund
On 2018-01-06 18:45:11 -0500, Tom Lane wrote: > Sophie Herold writes: > > the following patch allows to retrieve the typemod. Without this patch, > > it does not seem to be possible to generate the first column. > > I thought about this a bit, and I now follow the problem you

change in behaviour for format_type() call

2018-03-01 Thread Rushabh Lathia
Commit a26116c6cbf4117e8efaa7cfc5bacc887f01517f changed the behaviour for format_type. Prior to commit, format_type() used to set typemod_given to false for typemode = NULL. format_type() .. if (PG_ARGISNULL(1)) result = format_type_internal(type_oid, -1, false, true, false); else { typemod =

Re: Cast jsonb to numeric, int, float, bool

2018-03-01 Thread Nikita Glukhov
On 01.03.2018 11:19, Darafei "Komяpa" Praliaskouski wrote: > Attached new version of the patch in which I removed duplicated code using new subroutine JsonbExtractScalar(). I am not sure what is better to do when a JSON item has an unexpected type: to throw an  error or to return SQL NULL.

Re: Synchronous replay take III

2018-03-01 Thread Michael Paquier
On Thu, Mar 01, 2018 at 06:55:18PM +1300, Thomas Munro wrote: > On Thu, Mar 1, 2018 at 2:39 PM, Thomas Munro > wrote: >> I was pinged off-list by a fellow -hackers denizen interested in the >> synchronous replay feature and wanting a rebased patch to test. Here >>

Re: pgbench - add \if support

2018-03-01 Thread Andres Freund
Hi, On 2018-01-15 18:28:09 +0100, Fabien COELHO wrote: > Here is a very minor rebase. Your recent patches all seem to have windows line-endings. It'd be good to fix that for the future... - Andres

Re: pgbench - add \if support

2018-03-01 Thread Fabien COELHO
Hello Andres, Here is a very minor rebase. Your recent patches all seem to have windows line-endings. There does not seem to be any on my side: sh> hexdump pgbench-if-6.patch | grep '0[ad]' | head -3 040 6562 636e 2e68 6773 6c6d 690a 646e 7865 060 2031 3031 3630 3434

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2018-03-01 Thread Fabien COELHO
-#ifdef HAVE_SYS_SELECT_H +#ifdef PGBENCH_USE_SELECT /* force use of select(2)? */ +#undef HAVE_PPOLL +#endif +#ifdef HAVE_PPOLL +#include +#elif defined(HAVE_SYS_SELECT_H) +#define POLL_USING_SELECT (random thing noticed while going through patches) It strikes me as a

Re: no partition pruning when partitioning using array type

2018-03-01 Thread Amit Langote
On 2018/02/02 0:20, Robert Haas wrote: > On Thu, Feb 1, 2018 at 4:42 AM, Amit Langote > wrote: >>> I hope someone else chimes in as well. :) >> >> Bug #15042 [1] seems to be caused by this same problem. There, a >> RelabelType node is being slapped (by the

Re: 2018-03 Commitfest starts tomorrow

2018-03-01 Thread Magnus Hagander
On Thu, Mar 1, 2018 at 6:42 AM, Tom Lane wrote: > David Steele writes: > > I'll be starting the Commitfest at midnight AoE (07:00 ET, 13:00 CET) so > > please get your patches in before then. > > Please remember that if you drop a new and large (or

Re: [HACKERS] taking stdbool.h into use

2018-03-01 Thread Andres Freund
On 2018-02-01 09:04:57 -0500, Peter Eisentraut wrote: > I've been testing this a bit further and during a test setup with 4-byte > bools I still got regression test failures related to GIN, so it doesn't > seem quite ready. I'll keep working on it. Are you planning to get this into v11? The CF

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-03-01 Thread Andres Freund
On 2018-02-05 09:59:25 +0900, Yoshimi Ichiyanagi wrote: > I added my patches to the CommitFest 2018-3. > https://commitfest.postgresql.org/17/1485/ Unfortunately this is the last CF for the v11 development cycle. This is a major project submitted late for v11, there's been no code level review,

Re: 2018-03 Commitfest starts tomorrow

2018-03-01 Thread David Steele
On 3/1/18 4:52 AM, Magnus Hagander wrote: > On Thu, Mar 1, 2018 at 6:42 AM, Tom Lane > wrote: > > I think that we should summarily bounce to the September 'fest anything > submitted in the last two days; certainly anything that's

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-01 Thread Alexander Kuzmenkov
On 01.03.2018 06:23, David Gould wrote: In theory the sample pages analyze uses should represent the whole table fairly well. We rely on this to generate pg_statistic and it is a key input to the planner. Why should we not believe in it as much only for reltuples? If the analyze sampling does

Comments on old bug report in light of CVE-2018-1058

2018-03-01 Thread David G. Johnston
Hackers, By happen-stance I recently came across an old bug report that I responded to, #13651 (circa 2015-09), and reading the commentary for CVE-2018-1058 made me think about it in a different light. While no one added to my responses back then I'm thinking it would be worthwhile if one or

Re: Isolation tester result formatting

2018-03-01 Thread Andres Freund
On 2018-02-23 23:33:15 +1300, Thomas Munro wrote: > Hi hackers, > > Due to recent naming inflation, the isolation tester output has become > jagged (when viewed in a monospace font): > > test lock-update-delete ... ok > test lock-update-traversal... ok > test insert-conflict-do-nothing

Re: [HACKERS] path toward faster partition pruning

2018-03-01 Thread Robert Haas
On Tue, Feb 27, 2018 at 4:33 AM, Amit Langote wrote: > Attached an updated version in which I incorporated some of the revisions > that David Rowley suggested to OR clauses handling (in partprune.c) that > he posted as a separate patch on the run-time pruning thread

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-01 Thread Fabien COELHO
Hello Andres, Thanks for managing this CF. - pgbench - allow to store query results into variables RFC, has been worked on for a while. Patch doesn't look 100% there, but not that far away from it either. What would be missing to look 100% there? A bit concerned that we're turning

Re: [PoC PATCH] Parallel dump to /dev/null

2018-03-01 Thread Michael Banck
Hi, Am Donnerstag, den 01.03.2018, 01:28 -0800 schrieb Andres Freund: > Hi, > > On 2018-02-28 14:28:41 +0100, Michael Banck wrote: > > I have added this patch to the next commitfest: > > > > https://commitfest.postgresql.org/17/1576/ > > Given this is a new patch, submitted for the last

Re: parallel append vs. simple UNION ALL

2018-03-01 Thread Ashutosh Bapat
On Sat, Feb 24, 2018 at 2:55 AM, Robert Haas wrote: > > Here's an extended series of patches that now handles both the simple > UNION ALL case (where we flatten it) and the unflattened case: > The patches look clean. I particularly looked at 0003. patch 0001 +/* +

Re: to_typemod(type_name) information function

2018-03-01 Thread Sophie Herold
Hi, On 01/03/18 11:16, Andres Freund wrote: > Sophie, are you planning to > implement something along these lines? I don't have time right now, but yes. Best, Sophie

zheap: a new storage format for PostgreSQL

2018-03-01 Thread Amit Kapila
Sometime back Robert has proposed a solution to reduce the bloat in PostgreSQL [1] which has some other advantages of its own as well. To recap, in the existing heap, we always create a new version of a tuple on an update which must eventually be removed by periodic vacuuming or by HOT-pruning,

Re: Re: Cast jsonb to numeric, int, float, bool

2018-03-01 Thread David Steele
On 2/28/18 7:12 PM, Nikita Glukhov wrote: > On 01.03.2018 00:43, Darafei Praliaskouski wrote: >> >> The new status of this patch is: Ready for Committer > > Attached new version of the patch in which I removed duplicated code > using new subroutine JsonbExtractScalar(). I am not sure what is

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

2018-03-01 Thread David Steele
Hi Anastasia, On 2/28/18 11:03 AM, Anastasia Lubennikova wrote: > I want to propose a bunch of patches which allow to reduce WAL traffic > generated by CREATE INDEX for GiST, GIN and SP-GiST. Similarly to b-tree > and RUM, we can now log index pages of other access methods only once > in the end

2018-03 Commitfest Summary (Andres #1)

2018-03-01 Thread Andres Freund
Hi, Going through all non bugfix CF entries. Here's the summary for the entries I could stomach tonight: RFC: ready for committer NR: needs review WOA: waiting on author. - pgbench - allow to store query results into variables RFC, has been worked on for a while. Patch doesn't look 100%

Re: Incorrect comments in partition.c

2018-03-01 Thread Etsuro Fujita
(2018/03/01 0:12), Robert Haas wrote: On Wed, Feb 28, 2018 at 3:24 AM, Etsuro Fujita wrote: I'll add this to the upcoming commitfest. Committed. Sorry that I didn't notice this thread sooner (and that the original commits didn't take care of it). Thanks!

Re: inserts into partitioned table may cause crash

2018-03-01 Thread Etsuro Fujita
(2018/02/28 17:36), Amit Langote wrote: I've run into what seems to be a bug in ExecInsert() that causes a crash when inserting multiple rows into a partitioned table that each go into different partitions with different tuple descriptors. Crash occurs if ExecInsert() returns without resetting

Re: [HACKERS] MERGE SQL Statement for PG11

2018-03-01 Thread Pavan Deolasee
On Thu, Mar 1, 2018 at 3:50 AM, Peter Geoghegan wrote: > On Fri, Feb 9, 2018 at 6:36 AM, Robert Haas wrote: > > Here's my $0.02: I think that new concurrency errors thrown by the > > merge code itself deserve strict scrutiny and can survive only if they > >

Re: parallel append vs. simple UNION ALL

2018-03-01 Thread Rajkumar Raghuwanshi
On Sat, Feb 24, 2018 at 2:55 AM, Robert Haas wrote: > 0004 causes generate_union_path() to consider both the traditional > method and also Gather -> Parallel Append -> [partial path for each > subquery]. This is still a bit rough around the edges and there's a > lot more

Re: Online enabling of checksums

2018-03-01 Thread Magnus Hagander
On Thu, Mar 1, 2018 at 9:04 AM, Andres Freund wrote: > On 2018-03-01 12:56:35 +0500, Andrey Borodin wrote: > > I've tried to rebase this patch to 10 and, despite minor rebase issues > (oids, bgw_type, changes to specscanner), patch works fine. > > Do we provide backporting

Re: change in behaviour for format_type() call

2018-03-01 Thread Tom Lane
Rushabh Lathia writes: > Commit a26116c6cbf4117e8efaa7cfc5bacc887f01517f changed the behaviour > for format_type. > ... > Is this expected behaviour? attaching patch to get back the older > behaviour. I don't see anything in the commit message or linked discussion to

Re: Re: [PATCH] Opclass parameters

2018-03-01 Thread David Steele
Hi Nikita, On 2/28/18 9:46 AM, Nikolay Shaplov wrote: > В письме от 28 февраля 2018 00:46:36 пользователь Nikita Glukhov написал: > >> I would like to present patch set implementing opclass parameters. >> >> This feature was recently presented at pgconf.ru: >>

Re: row filtering for logical replication

2018-03-01 Thread David Fetter
On Thu, Mar 01, 2018 at 12:41:04PM -0300, Euler Taveira wrote: > 2018-02-28 21:47 GMT-03:00 David Fetter : > > I noticed that the WHERE clause applies to all tables in the > > publication. Is that actually the right thing? I'm thinking of a > > case where we have foo(id, ...)

Re: csv format for psql

2018-03-01 Thread Daniel Verite
Fabien COELHO wrote: > Maybe some \csv command could set the format to csv, fieldsep to ",", > tuples_only to on, recordsep to '\n'? Not sure whether it would be > acceptable, though, and how to turn it off once turned on... Probably an > average (aka not good) idea:-) Thanks for

Re: [PATCH][PROPOSAL] Add enum releation option type

2018-03-01 Thread Nikita Glukhov
Hi. I have refactored patch by introducing new struct relop_enum_element to make it possible to use existing C-enum values in option's definition. So, additional enum GIST_OPTION_BUFFERING_XXX was removed. Also default option value should be placed now in the first element of allowed_values[].

Re: csv format for psql

2018-03-01 Thread Pavel Stehule
2018-03-01 17:10 GMT+01:00 Daniel Verite : > Fabien COELHO wrote: > > > Maybe some \csv command could set the format to csv, fieldsep to ",", > > tuples_only to on, recordsep to '\n'? Not sure whether it would be > > acceptable, though, and how to turn it off once

Re: [PATCH] Find additional connection service files in pg_service.conf.d directory

2018-03-01 Thread Curt Tilmes
On Thu, Mar 1, 2018 at 3:53 AM, Andres Freund wrote: > > On 2018-01-12 09:53:51 -0500, Curt Tilmes wrote: > > The convention widely adopted for this type of thing is to allow > > multiple config files to be in a directory, usually the '.d' version of the > > config filename.

Re: zheap: a new storage format for PostgreSQL

2018-03-01 Thread Amit Kapila
On Thu, Mar 1, 2018 at 7:39 PM, Amit Kapila wrote: > > Preliminary performance results > --- > I have not used plain text mode in my previous email due to which performance results might not be clear in some email clients, so

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-01 Thread Tom Lane
Alexander Kuzmenkov writes: > On 01.03.2018 06:23, David Gould wrote: >> In theory the sample pages analyze uses should represent the whole table >> fairly well. We rely on this to generate pg_statistic and it is a key >> input to the planner. Why should we not believe

SHOW ALL does not honor pg_read_all_settings membership

2018-03-01 Thread Laurenz Albe
I noticed that commit 25fff40798fc4ac11a241bfd9ab0c45c085e2212 forgot to teach SHOW ALL to show all GUCs when the user belongs to pg_read_all_settings. Patch attached; I think this should be backpatched. Yours, Laurenz AlbeFrom f3a92a6ecc8dc116a6843c1bd76aa5e2a97785ff Mon Sep 17 00:00:00 2001

Re: 2018-03 Commitfest starts tomorrow

2018-03-01 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Mar 01, 2018 at 12:42:25AM -0500, Tom Lane wrote: > > that are not Johnny-come-latelies. > > Reference? :) This is a relatively recent term, dating back only two centuries ago according to M-W: https://www.merriam-webster.com/dictionary/Johnny-come-lately

In reference to gsoc

2018-03-01 Thread Tanvi Dadu
Hello ! I am a sophomore pursuing Computer Engineering in NSIT affiliated with Delhi University. I came across your organisation through gsoc portal and am very interested to work on project PostgreSQL sorting algorithm benchmark and implementation. My skill set include android, python , java ,

Re: [HACKERS] Fix performance degradation of contended LWLock on NUMA

2018-03-01 Thread Jesper Pedersen
Hi, On 01/02/2018 11:09 AM, Jesper Pedersen wrote: Oh... there were stupid error in previos file. Attached fixed version. As the patch still applies, make check-world passes and I believe that Yura has provided feedback for Andres' comments I'll leave this entry in "Ready for Committer".

Re: row filtering for logical replication

2018-03-01 Thread Euler Taveira
2018-02-28 21:47 GMT-03:00 David Fetter : > I noticed that the WHERE clause applies to all tables in the > publication. Is that actually the right thing? I'm thinking of a > case where we have foo(id, ...) and bar(foo_id, ). To slice that > correctly, we'd want to do the

Re: row filtering for logical replication

2018-03-01 Thread Erik Rijkers
On 2018-03-01 00:03, Euler Taveira wrote: The attached patches add support for filtering rows in the publisher. 001-Refactor-function-create_estate_for_relation.patch 0002-Rename-a-WHERE-node.patch 0003-Row-filtering-for-logical-replication.patch Comments? Very, very useful. I really do

Re: zheap: a new storage format for PostgreSQL

2018-03-01 Thread Satyanarayana Narlapuram
>> Cons >> --- >> 1. Deletes can be somewhat expensive. >> 2. Transaction aborts will be expensive. >> 3. Updates that update most of the indexed columns can be somewhat expensive. Given transaction aborts are expensive, is there any impact on the crash recovery? Did you perform any

Re: CALL optional in PL/pgSQL

2018-03-01 Thread Tom Lane
Pavel Stehule writes: > 2018-03-01 5:51 GMT+01:00 Peter Eisentraut > This seems to be a popular issue when porting from PL/SQL, so I'll throw >> it out here for discussion. Apparently, in PL/SQL you can call another >> procedure without

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2018-03-01 Thread Andres Freund
On 2018-03-01 15:37:17 -0500, Robert Haas wrote: > On Thu, Mar 1, 2018 at 2:17 PM, Andres Freund wrote: > >> However, if we take the position that no hash collision probability is > >> low enough and that we must eliminate all chance of false collisions, > >> except perhaps

Re: [PATCH] Opclass parameters

2018-03-01 Thread David Steele
On 3/1/18 3:50 PM, Oleg Bartunov wrote: > On Thu, Mar 1, 2018 at 7:02 PM, David Steele wrote: >> >> Any objections to marking this Returned with Feedback? Or, I can move it >> to the next CF as is. > > I think that Returned with Feedback would be good. We will continue >

Re: MCV lists for highly skewed distributions

2018-03-01 Thread Andres Freund
Hi Dean, On 2018-02-07 15:58:14 +, Dean Rasheed wrote: > On 7 February 2018 at 15:25, Robert Haas wrote: > > Do you plan to press forward with this, then, or what's > > the next step? > > > > Yes, I think the results are pretty good so far, especially for the > more

pgstat_report_activity() and parallel CREATE INDEX (was: Parallel index creation & pg_stat_activity)

2018-03-01 Thread Peter Geoghegan
On Wed, Feb 28, 2018 at 9:44 AM, Andres Freund wrote: > Looks like we're not doing a pgstat_report_activity() in the workers? > Any argument for not doing so? No. Just an oversight. Looks like _bt_parallel_build_main() should call pgstat_report_activity(), just like

Re: Challenges preventing us moving to 64 bit transaction id (XID)?

2018-03-01 Thread Andres Freund
On 2018-03-02 01:56:00 +0300, Alexander Korotkov wrote: > On Fri, Mar 2, 2018 at 1:51 AM, Andres Freund wrote: > > > On 2018-03-02 01:48:03 +0300, Alexander Korotkov wrote: > > > Also, the last commitfest is already too late for such big changes. > > > So, I'm marking this

Re: [HACKERS] Removing useless DISTINCT clauses

2018-03-01 Thread David Rowley
On 2 March 2018 at 09:51, Andres Freund wrote: > This patch has been waiting on author since 2018-01-09, the next & last > CF has started. I'm inclined to mark this as returned with feedback. I'm planning on making the required changes at the weekend. -- David Rowley

Re: "failed to find parent tuple for heap-only tuple" error as an ERRCODE_DATA_CORRUPTION ereport()

2018-03-01 Thread Tom Lane
Peter Geoghegan writes: > On Thu, Mar 1, 2018 at 11:23 AM, David Steele wrote: >> Yes, I agree. My thrust was more to discover if there is any testing >> for these conditions being done in core. It sounds like no, but I don't >> think it's the responsibility

Re: zheap: a new storage format for PostgreSQL

2018-03-01 Thread Alexander Korotkov
On Thu, Mar 1, 2018 at 5:09 PM, Amit Kapila wrote: > Preliminary performance results > --- > > *We’ve shown the performance improvement of zheap over heap in a few > different pgbench scenarios. All of these tests were run with

Re: jsonpath

2018-03-01 Thread Alexander Korotkov
On Fri, Mar 2, 2018 at 12:40 AM, Nikita Glukhov wrote: > On 28.02.2018 06:55, Robert Haas wrote: > > On Mon, Feb 26, 2018 at 10:34 AM, Nikita Glukhov >> wrote: >> >>> Attached 10th version of the jsonpath patches. >>> >>> 1. Fixed error handling

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2018-03-01 Thread Andres Freund
On 2018-01-23 17:08:56 +0100, Pavel Stehule wrote: > 2018-01-22 23:15 GMT+01:00 Stephen Frost : > > This really could use a new thread, imv. This thread is a year old and > > about a completely different feature than what you've implemented here. > > > > true, but now it is

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-01 Thread David Gould
On Thu, 1 Mar 2018 17:25:09 +0300 Alexander Kuzmenkov wrote: > Well, that sounds reasonable. But the problem with the moving average > calculation remains. Suppose you run vacuum and not analyze. If the > updates are random enough, vacuum won't be able to reclaim

Re: [HACKERS] Removing LEFT JOINs in more cases

2018-03-01 Thread David Rowley
On 2 March 2018 at 09:49, Andres Freund wrote: > On 2018-01-10 11:14:27 +1300, David Rowley wrote: >> On 10 January 2018 at 08:44, Tom Lane wrote: >> > I'll set the patch back to Waiting on Author. >> >> Many thanks for looking at this. I'll try to resolve

Re: [HACKERS] Removing LEFT JOINs in more cases

2018-03-01 Thread Andres Freund
On 2018-03-02 09:57:35 +1300, David Rowley wrote: > On 2 March 2018 at 09:56, Andres Freund wrote: > > On 2018-03-02 09:53:38 +1300, David Rowley wrote: > >> I'm planning on making the required changes at the weekend. > > > > Sorry if I'm grumpy, but why shouldn't this just

Re: MCV lists for highly skewed distributions

2018-03-01 Thread Dean Rasheed
On 1 March 2018 at 21:01, Andres Freund wrote: > This sounds like the patch's status of "waiting on author" isn't right, > and it should more be ready for committer? > Yes, I'll take a look at it this weekend. Regards, Dean

Re: 2018-03 Commitfest Summary (Andres #2)

2018-03-01 Thread Andres Freund
On 2018-03-01 03:03:44 -0800, Andres Freund wrote: > Going through all non bugfix CF entries. Here's the summary for the > entries I could stomach tonight: > > RFC: ready for committer > NR: needs review > WOA: waiting on author. Second round. - Sample values for pg_stat_statements NR.

Re: row filtering for logical replication

2018-03-01 Thread Euler Taveira
2018-03-01 18:27 GMT-03:00 Andres Freund : > FWIW, I don't think it'd be fair or prudent. There's definitely some > issues (see e.g. Craig's reply), and I don't see why this patch'd > deserve an exemption from the "nontrivial patches shouldn't be submitted > to the last CF"

Re: Faster inserts with mostly-monotonically increasing values

2018-03-01 Thread Claudio Freire
On Sun, Dec 31, 2017 at 8:06 AM, Peter Geoghegan wrote: > I also have my > doubts about unique index enforcement remaining correct with the patch > when there are many physical duplicates, to the extent that more than > a single leaf page is needed for a single value. given... +

Re: [HACKERS] Removing LEFT JOINs in more cases

2018-03-01 Thread Andres Freund
On 2018-01-10 11:14:27 +1300, David Rowley wrote: > On 10 January 2018 at 08:44, Tom Lane wrote: > > I'll set the patch back to Waiting on Author. > > Many thanks for looking at this. I'll try to resolve the things you've > mentioned this coming weekend. This hasn't happened

Re: [HACKERS] Removing LEFT JOINs in more cases

2018-03-01 Thread David Rowley
On 2 March 2018 at 09:56, Andres Freund wrote: > On 2018-03-02 09:53:38 +1300, David Rowley wrote: >> On 2 March 2018 at 09:49, Andres Freund wrote: >> > On 2018-01-10 11:14:27 +1300, David Rowley wrote: >> >> On 10 January 2018 at 08:44, Tom Lane

Re: [HACKERS] Removing LEFT JOINs in more cases

2018-03-01 Thread David Rowley
On 2 March 2018 at 09:59, Andres Freund wrote: > If a patch hasn't been updated after moved waiting-on-author from the > last CF, and the next CF started, that seems too late. Particularly in > the last CF. I understand that I should have resolved these issues before the

Re: Challenges preventing us moving to 64 bit transaction id (XID)?

2018-03-01 Thread Andres Freund
Hi, On 2018-01-11 01:02:52 +0300, Alexander Korotkov wrote: > As I get from cputube, patchset doesn't compiles again. Please find > revised version attached. It'd be good if you could maintain the patches as commits with some description of why you're doing these changes. It's a bit hard to

Re: Challenges preventing us moving to 64 bit transaction id (XID)?

2018-03-01 Thread Alexander Korotkov
Hi! On Fri, Mar 2, 2018 at 1:41 AM, Andres Freund wrote: > On 2018-01-11 01:02:52 +0300, Alexander Korotkov wrote: > > As I get from cputube, patchset doesn't compiles again. Please find > > revised version attached. > > It'd be good if you could maintain the patches as

Re: Challenges preventing us moving to 64 bit transaction id (XID)?

2018-03-01 Thread Andres Freund
On 2018-03-02 01:48:03 +0300, Alexander Korotkov wrote: > Also, the last commitfest is already too late for such big changes. > So, I'm marking this RWF. Agreed. Perhaps extract the 64bit GUC patch and track that separately? Seems like something we should just do... Greetings, Andres Freund

Re: row filtering for logical replication

2018-03-01 Thread Euler Taveira
2018-03-01 18:25 GMT-03:00 Erik Rijkers : > Attached is 'logrep_rowfilter.sh', a demonstration of above-described bug. > Thanks for testing. I will figure out what is happening. There are some leaks around. I'll post another version when I fix some of those bugs. -- Euler

Re: row filtering for logical replication

2018-03-01 Thread Euler Taveira
2018-02-28 21:54 GMT-03:00 Craig Ringer : > Good idea. I haven't read this yet, but one thing to make sure you've > handled is limiting the clause to referencing only the current tuple and the > catalogs. user-catalog tables are OK, too, anything that is >

Re: [PoC PATCH] Parallel dump to /dev/null

2018-03-01 Thread Andres Freund
On 2018-03-01 14:21:24 +0100, Michael Banck wrote: > Hi, > > Am Donnerstag, den 01.03.2018, 01:28 -0800 schrieb Andres Freund: > > Hi, > > > > On 2018-02-28 14:28:41 +0100, Michael Banck wrote: > > > I have added this patch to the next commitfest: > > > > > >

Re: PATCH: Unlogged tables re-initialization tests

2018-03-01 Thread David Steele
Hi Thomas, [Also pulling in Michael for Windows knowledge] On 3/1/18 12:27 AM, Thomas Munro wrote: > On Thu, Mar 1, 2018 at 9:24 AM, David Steele wrote: >> These tests were originally included in the exclude unlogged tables >> patch [1] to provide coverage for the

Re: Sample values for pg_stat_statements

2018-03-01 Thread Andres Freund
Hi, On 2017-12-31 12:34:17 +0100, Vik Fearing wrote: > Often when looking through pg_stat_statements, it would be nice to have > some sample values for the constants and parameters. This patch > implements that by taking the values from the first execution of the > normalized query. > > To keep

Re: pgbench - test whether a variable exists

2018-03-01 Thread Alvaro Herrera
re. pg_strndup() the following places could use it (skim of a very quick grep for pg_strdup): src/bin/pg_waldump/pg_waldump.c:*dir = pg_strdup(path); src/bin/pg_waldump/pg_waldump.c-(*dir)[(sep - path) + 1] = '\0';/* no strndup */ src/bin/psql/prompt.c:

Re: [HACKERS] Support to COMMENT ON DATABASE CURRENT_DATABASE

2018-03-01 Thread Tom Lane
Jing Wang writes: > [ support_CURRENT_DATABASE_keyword_v4.7.patch ] TBH, I think we should reject this patch. While it's not huge, it's not trivial either, and I find the grammar changes rather ugly. The argument for using the feature to fix pg_dump issues has evaporated,

  1   2   3   >