Re: [HACKERS] Early locking option to parallel backup

2018-03-02 Thread Robert Haas
On Fri, Mar 2, 2018 at 2:29 AM, Andres Freund wrote: > There seems to to be consensus in this thread that the approach Lucas > proposed isn't what we want, and that instead some shared lock based > approach is desirable. As that has been the case for ~1.5 months, I > propose

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-03-02 Thread Julian Markwort
Andres Freund wrote on 2018-03-01: > I think the patch probably doesn't apply anymore, due to other changes > to pg_stat_statements since its posting. Could you refresh? pgss_plans_v02.patch applies cleanly to master, there were no changes to pg_stat_statements since the copyright updates at the

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-02 Thread Tomas Vondra
On 03/02/2018 04:29 AM, Andres Freund wrote: > Hi, > > On 2018-02-24 23:01:59 +0100, Tomas Vondra wrote: >> Sadly, this patch series does not seem to move forward very much, and >> I'm not sure how to change that :-/ > > What's your estimate about the patchset's maturity? > It's dying of old

Re: WIP: a way forward on bootstrap data

2018-03-02 Thread Tom Lane
John Naylor writes: > Version 8, rebased against 76b6aa41f41d. I took a preliminary look through this, without yet attempting to execute the script against HEAD. I have a few thoughts: * I'm inclined not to commit the conversion scripts to the repo. I doubt there are third

Re: [HACKERS] path toward faster partition pruning

2018-03-02 Thread David Rowley
On 3 March 2018 at 04:47, Robert Haas wrote: > On Fri, Mar 2, 2018 at 6:21 AM, Amit Langote > wrote: >> Looking at the rough interface sketch in your message, it seems that the >> product of whatever steps we end up grouping into phase 1

Re: zheap: a new storage format for PostgreSQL

2018-03-02 Thread Robert Haas
On Fri, Mar 2, 2018 at 5:35 AM, Alexander Korotkov wrote: > I would propose "zero-bloat heap" disambiguation of zheap. Seems like fair > enough explanation for me without need to rename :) It will be possible to bloat a zheap table in certain usage patterns. For

Re: [GSOC 18] Performance Farm Project

2018-03-02 Thread Mark Wong
Hi Hongyuan, On Thu, Mar 01, 2018 at 05:56:07PM +0800, 马来酸 wrote: > Dear Sir, > Hi, my name is Hongyuan Ma. I am a junior student in software engineering in > Shanghai.I am interested in the idea "Develop Performance Farm Database and > Website".I started developing web applications since my

Re: Rangejoin rebased

2018-03-02 Thread Alexander Kuzmenkov
On 16.01.2018 10:49, Jeff Davis wrote: My proposed fix is to make an internal opfamily identical to the external one, such that it's not recognized as part of the same EC, and the planner won't try to eliminate it. It loses out on potential optimizations, but those are mostly theoretical since

Re: Online enabling of checksums

2018-03-02 Thread Robert Haas
On Fri, Mar 2, 2018 at 8:35 AM, Magnus Hagander wrote: > Do we ever make hintbit changes on the standby for example? If so, it would > definitely cause problems. I didn't realize we did, actually... We do not. > I guess we could get there even if we don't by: > * All

All Taxi Services need Index Clustered Heap Append

2018-03-02 Thread Komяpa
Hi, I work at a ride sharing company and we found a simple scenario that Postgres has a lot to improve at. After my talk at pgconf.ru Alexander Korotkov encouraged me to share my story and thoughts in -hackers. Story setting: - Amazon RDS (thus vanilla unpatchable postgres),

Re: psql tab completion for ALTER INDEX SET

2018-03-02 Thread Fujii Masao
On Fri, Mar 2, 2018 at 3:52 PM, Masahiko Sawada wrote: > Hi, > > I found that tab completion for ALTER INDEX SET [tab] doesn't support > the reloptions of brin and gist. Attached patch adds "buffering", > "pages_per_range" and "autosummarize" options. Thanks for the patch!

Re: [Patch] Checksums for SLRU files

2018-03-02 Thread Robert Haas
On Thu, Mar 1, 2018 at 8:25 PM, Andres Freund wrote: > On 2018-02-02 11:37:34 +1300, Thomas Munro wrote: >> > 3. pg_upgrade isn't considered. This patch should provide upgrading SLRUs >> > to adopt changed useful size of page. That seems to be hardest patch of >> > this

Re: [PROPOSAL] Nepali Snowball dictionary

2018-03-02 Thread Arthur Zakirov
On Thu, Mar 01, 2018 at 10:23:11PM -0800, Andres Freund wrote: > What is that entry for, if I may ask? We need to wait for them to merge > it, then sync the snowball code, including the nepali dictionary. This > doesn't realistically seem doable for this commitfest. Therefore I think > this

Re: jsonlog logging only some messages?

2018-03-02 Thread Greg Stark
On 27 February 2018 at 16:50, Greg Stark wrote: > On 27 February 2018 at 02:04, Michael Paquier wrote: >> On Mon, Feb 26, 2018 at 05:38:56PM +, Greg Stark wrote: >> >> Hm. I have just loaded jsonlog on a 9.6 and 10 instance where >> log_checkpoints is

Re: WIP: BRIN multi-range indexes

2018-03-02 Thread Tomas Vondra
On 03/02/2018 05:08 AM, Tom Lane wrote: > Andres Freund writes: >> On 2018-02-25 01:30:47 +0100, Tomas Vondra wrote: >>> Note: Currently, this only works with float8-based data types. >>> Supporting additional data types is not a big issue, but will >>> require extending the

Re: [PATCH] get rid of StdRdOptions, use individual binary reloptions representation for each relation kind instead

2018-03-02 Thread Nikolay Shaplov
В письме от 1 марта 2018 16:15:32 пользователь Andres Freund написал: > > This is part or my bigger patch > > https://www.postgresql.org/message-id/flat/2146419.veIEZdk4E4@x200m#21464 > > 19.veIEZdk4E4@x200m we've decided to commit by smaller parts. > > I've not read that thread. Is this

Re: [PATCH] Verify Checksums during Basebackups

2018-03-02 Thread Robert Haas
On Fri, Mar 2, 2018 at 6:23 AM, Magnus Hagander wrote: > Another quick note -- we need to assert that the size of the buffer is > actually divisible by BLCKSZ. I don't think it's a common scenario, but it > could break badly if somebody changes BLCKSZ. Either that or perhaps

Re: [HACKERS] Creating backup history files for backups taken from standbys

2018-03-02 Thread Fujii Masao
On Fri, Mar 2, 2018 at 12:26 PM, Andres Freund wrote: > Hi! > > On 2018-03-02 02:29:13 +0900, Fujii Masao wrote: >> On Fri, Feb 2, 2018 at 2:06 PM, Michael Paquier >> wrote: >> > On Fri, Feb 02, 2018 at 12:47:26AM +0900, Fujii Masao wrote: >> >> The

Re: [PATCH] Opclass parameters

2018-03-02 Thread Nikolay Shaplov
В письме от 1 марта 2018 23:02:20 пользователь Oleg Bartunov написал: > > 2. Your patch does not provide any example of your new tool usage. In my > > prototype patch I've shown the implementation of opclass options for > > intarray. May be you should do the same. (Use my example it will be more

Re: Re: reorganizing partitioning code

2018-03-02 Thread Amit Langote
Hi David. On Fri, Mar 2, 2018 at 11:53 PM, David Steele wrote: > Hi Amit, > > On 2/16/18 3:36 AM, Amit Langote wrote: >> >> Attached updated version. > > This patch no longer applies and the conflicts do not appear to be trivial. > > I'm a bit confused about your comment in

Re: New gist vacuum.

2018-03-02 Thread Andrey Borodin
> 2 марта 2018 г., в 21:25, Tom Lane написал(а): > > Andrey Borodin writes: >> So, I agree, unconditional counting is a good idea. Here's the v3 patch. > > Pushed with trivial cosmetic adjustments. > > I've marked the CF entry as committed; please

Re: postgres_fdw: perform UPDATE/DELETE .. RETURNING on a join directly

2018-03-02 Thread Robert Haas
On Fri, Mar 2, 2018 at 5:35 AM, Etsuro Fujita wrote: > Agreed. Better safe than sorry, so I disabled autovacuum for all the tables > created in the postgres_fdw regression test, except the ones with no data > modification created in the sections "test handling of

Re: [GSOC 18] Performance Farm Project

2018-03-02 Thread Mark Wong
On Thu, Mar 01, 2018 at 02:31:08PM -0500, Robert Haas wrote: > On Thu, Mar 1, 2018 at 4:56 AM, 马来酸 wrote: > > Hi, my name is Hongyuan Ma. I am a junior student in software engineering in > > Shanghai.I am interested in the idea "Develop Performance Farm Database and > >

Re: [HACKERS] path toward faster partition pruning

2018-03-02 Thread Robert Haas
On Fri, Mar 2, 2018 at 10:54 AM, David Rowley wrote: > I don't think building it is going to cost a huge amount. Presumably, > there are not many partitioned tables with 10 rows, so probably having > the get_partitions_from_clauses work as quickly as possible is

Re: Online enabling of checksums

2018-03-02 Thread Tomas Vondra
On 03/02/2018 03:22 PM, Magnus Hagander wrote: > > > On Fri, Mar 2, 2018 at 3:17 PM, Alvaro Herrera > wrote: > > Magnus Hagander wrote: > > On Wed, Feb 28, 2018 at 10:07 PM, Tomas Vondra >

Re: [HACKERS] Creating backup history files for backups taken from standbys

2018-03-02 Thread Fujii Masao
On Fri, Mar 2, 2018 at 1:07 PM, Michael Paquier wrote: > On Fri, Mar 02, 2018 at 02:29:13AM +0900, Fujii Masao wrote: >> + * write a backup history file with the same name. >> >> So more than one backup history files with the same name >> but the diffferent content can be

Re: [HACKERS] path toward faster partition pruning

2018-03-02 Thread David Rowley
On 2 March 2018 at 08:13, Robert Haas wrote: > 2. All processing of clauses should happen at plan time, not run time, > so that we're not repeating work. If, for example, a prepared query > is executed repeatedly, the clauses should get fully processed when > it's planned,

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-03-02 Thread Andres Freund
Hi, On 2018-03-02 09:42:22 +, Yuqi Gu wrote: > > Could you show whether it actually improves performance? Usually bulk > > loading data with parallel COPYs is a good way to hit this code path. > > The mini benchmark code: I'd be more interested in a benchmark using postgres itself... > >

Re: Online enabling of checksums

2018-03-02 Thread Robert Haas
On Fri, Mar 2, 2018 at 2:44 AM, Andres Freund wrote: > And even more so, I'm not even sure it makes sense to try to get this > into v11. This is a medium-large complicated feature, submitted to the > last CF for v11. That's pretty late. Now, Magnus is a committer, but >

Re: [HACKERS] [POC] Faster processing at Gather node

2018-03-02 Thread Robert Haas
On Wed, Feb 28, 2018 at 10:06 AM, Robert Haas wrote: > [ latest patches ] Committed. Thanks for the review. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [HACKERS] log_destination=file

2018-03-02 Thread Tom Lane
Robert Haas writes: > On Mon, Jan 22, 2018 at 4:25 PM, Tomas Vondra > wrote: >> Sorry for the naive question, but which of these bottlenecks are we >> actually hitting? I don't recall dealing with an actual production >> system where the log

Re: Rangejoin rebased

2018-03-02 Thread Robert Haas
On Fri, Mar 2, 2018 at 11:12 AM, Alexander Kuzmenkov wrote: > On 16.01.2018 10:49, Jeff Davis wrote: >> My proposed fix is to make an internal opfamily identical to the >> external one, such that it's not recognized as part of the same EC, >> and the planner won't try

Re: Online enabling of checksums

2018-03-02 Thread Tomas Vondra
On 03/02/2018 02:35 PM, Magnus Hagander wrote: > > > On Wed, Feb 28, 2018 at 6:06 PM, Robert Haas > wrote: > > On Sun, Feb 25, 2018 at 9:54 AM, Magnus Hagander > > wrote: > >

Re: Rewrite of pg_dump TAP tests

2018-03-02 Thread Stephen Frost
Andres, * Andres Freund (and...@anarazel.de) wrote: > On 2018-02-26 13:15:04 -0500, Stephen Frost wrote: > > Attached is a patch (which applies cleaning against a2a2205, but not so > > much anymore, obviously, but I will fix after the releases) which > > greatly improves the big pg_dump TAP

Re: heap_lock_updated_tuple_rec can leak a buffer refcount

2018-03-02 Thread Alexander Kuzmenkov
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: not tested Spec compliant: not tested Documentation:not tested Looks like a leak indeed, the fix seems right.

Re: [HACKERS] log_destination=file

2018-03-02 Thread Robert Haas
On Mon, Jan 22, 2018 at 4:25 PM, Tomas Vondra wrote: > Sorry for the naive question, but which of these bottlenecks are we > actually hitting? I don't recall dealing with an actual production > system where the log collector would be an issue, so I don't have a very

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-02 Thread Robert Haas
On Thu, Mar 1, 2018 at 4:52 PM, Robert Haas wrote: > This is not a full review, but I'm out of time for right now. Another thing I see here now is that create_grouping_paths() and create_child_grouping_paths() are extremely similar. Isn't there some way we can refactor

Re: IndexJoin memory problem using spgist and boxes

2018-03-02 Thread Alexander Kuzmenkov
Hi Anton, I can reproduce the high memory consumption with your queries. Looking at the patch, I see that you changed the lifetime of the temporary context from per-tuple to per-index-scan. It is not obvious that this change is correct. Could you explain, what memory context are involved in

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-03-02 Thread Tom Lane
Julian Markwort writes: > Andres Freund wrote on 2018-03-02: >> and I'd checked that 1.5 already exists. But you just renamed the file, >> presumably because it's essentially rewriting the whole file? I'm not >> sure I'm a big fan of doing so, because that makes

Re: Kerberos test suite

2018-03-02 Thread Robbie Harwood
Michael Paquier writes: > On Wed, Feb 14, 2018 at 09:27:04AM -0500, Peter Eisentraut wrote: > >> (If it appears to hang for you in the "setting up Kerberos" step, you >> might need more entropy/wait a while. That problem appears to be >> limited to some virtual machine

Re: [HACKERS] Early locking option to parallel backup

2018-03-02 Thread Andres Freund
On 2018-03-02 11:42:06 -0500, Robert Haas wrote: > On Fri, Mar 2, 2018 at 2:29 AM, Andres Freund wrote: > > There seems to to be consensus in this thread that the approach Lucas > > proposed isn't what we want, and that instead some shared lock based > > approach is desirable.

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-02 Thread Andres Freund
Hi, On 2018-03-02 11:06:12 +0100, Fabien COELHO wrote: > A lot of patches do not even get a review: no immediate interest or more > often no ressources currently available, patch stays put, I'm fine with > that. Well, even if that's the case that's not free of cost to transport them from fest to

Re: [HACKERS] pgbench randomness initialization

2018-03-02 Thread Tom Lane
Fabien COELHO writes: >> Hm ... so I tried to replicate this problem, and failed to: the log files >> get made under the VPATH build directory, as desired, even without this >> patch. Am I doing something wrong, or is this platform-dependent somehow? > As I recall, it

Re: Testing "workers launched" in expected output? Really?

2018-03-02 Thread Robert Haas
On Fri, Mar 2, 2018 at 2:28 PM, Tom Lane wrote: > So buildfarm member piculet just fell over like this: > > == pgsql.build/src/test/regress/regression.diffs > == > *** >

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-02 Thread Robert Haas
On Thu, Mar 1, 2018 at 9:33 PM, Tomas Vondra wrote: > Ah, apologies - that's due to moving the patch from the last CF (it was > marked as RWF so I had to reopen it before moving it). I'll submit a new > version of the patch shortly, please mark it as WOA until then.

Re: Cached/global query plans, autopreparation

2018-03-02 Thread Tom Lane
Andres Freund writes: > On 2018-03-02 15:29:09 -0500, Bruce Momjian wrote: >> While I have heard people complain about how other databases cache >> prepare plans, I have heard few complaints about the Postgres approach, >> and I haven't even heard of people asking to control

Re: Cached/global query plans, autopreparation

2018-03-02 Thread Pavel Stehule
2018-03-02 21:51 GMT+01:00 Tom Lane : > Andres Freund writes: > > On 2018-03-02 15:29:09 -0500, Bruce Momjian wrote: > >> While I have heard people complain about how other databases cache > >> prepare plans, I have heard few complaints about the Postgres

Re: 2018-03 Commitfest Summary (Andres #3)

2018-03-02 Thread Andres Freund
On 2018-03-02 12:22:15 +0300, Ildus Kurbangaliev wrote: > On Thu, 1 Mar 2018 20:34:11 -0800 > Andres Freund wrote: > > - Custom compression methods > > > > RFC. While marked as ready for committer, I think this is mostly > > because some high level design input is needed.

Re: WIP Patch: Precalculate stable functions, infrastructure v1

2018-03-02 Thread Andres Freund
Hi, On 2018-03-02 11:22:01 +0300, Marina Polyakova wrote: > I fixed the failure that Thomas pointed out to me, and I'm finishing work on > it, but it took me a while to study this part of the executor.. I unfortunately think that makes this too late for v11, and we should mark this as returned

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-02 Thread Tomas Vondra
On 03/02/2018 09:21 PM, David Steele wrote: > On 3/2/18 3:06 PM, Robert Haas wrote: >> On Thu, Mar 1, 2018 at 9:33 PM, Tomas Vondra >> wrote: >>> Ah, apologies - that's due to moving the patch from the last CF (it was >>> marked as RWF so I had to reopen it before

Re: Kerberos test suite

2018-03-02 Thread Robbie Harwood
Thomas Munro writes: > Peter Eisentraut wrote: >> On 2/27/18 00:56, Thomas Munro wrote: >>> FWIW it passes for me if I add this: >>> >>> +elsif ($^O eq 'freebsd') >>> +{ >>> + $krb5_bin_dir = '/usr/local/bin'; >>> +

Testing "workers launched" in expected output? Really?

2018-03-02 Thread Tom Lane
So buildfarm member piculet just fell over like this: == pgsql.build/src/test/regress/regression.diffs == *** /home/andres/build/buildfarm-piculet/HEAD/pgsql.build/../pgsql/src/test/regress/expected/select_parallel.out 2018-02-28 16:10:01.986941733 +

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-03-02 Thread Andres Freund
On 2018-03-02 18:07:32 +0100, Julian Markwort wrote: > Andres Freund wrote on 2018-03-01: > > I think the patch probably doesn't apply anymore, due to other changes > > to pg_stat_statements since its posting. Could you refresh? > > pgss_plans_v02.patch applies cleanly to master, there were no

Re: [Patch] Checksums for SLRU files

2018-03-02 Thread Andres Freund
On 2018-03-02 11:49:05 -0500, Robert Haas wrote: > On Thu, Mar 1, 2018 at 8:25 PM, Andres Freund wrote: > > On 2018-02-02 11:37:34 +1300, Thomas Munro wrote: > >> > 3. pg_upgrade isn't considered. This patch should provide upgrading > >> > SLRUs > >> > to adopt changed

Re: Testing "workers launched" in expected output? Really?

2018-03-02 Thread Tom Lane
Robert Haas writes: > On Fri, Mar 2, 2018 at 2:28 PM, Tom Lane wrote: >> ... now I am on the warpath. I have no idea whether or not the diff >> here is significant --- maybe it is --- but I am desperately unhappy >> that we have expected-output files

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-02 Thread Peter Geoghegan
On Fri, Mar 2, 2018 at 1:47 AM, Fabien COELHO wrote: > On the "adequate return" point, my opinion is that currently pgbench is just > below the feature set needed to be generally usable, so not improving it is > a self-fullfilling ensurance that it will not be used further.

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-02 Thread David Steele
On 3/2/18 3:06 PM, Robert Haas wrote: > On Thu, Mar 1, 2018 at 9:33 PM, Tomas Vondra > wrote: >> Ah, apologies - that's due to moving the patch from the last CF (it was >> marked as RWF so I had to reopen it before moving it). I'll submit a new >> version of the

Re: 2018-03 Commitfest Summary (Andres #1)

2018-03-02 Thread Andres Freund
Bcc: Reply-To: Hi, On 2018-03-02 10:47:01 +0100, Fabien COELHO wrote: > For instance, I used extensively tps throttling, latencies and timeouts > measures when developping and testing the checkpointer sorting & throttling > patch. That doesn't say that much about proposed feature additions,

Re: postgres_fdw: perform UPDATE/DELETE .. RETURNING on a join directly

2018-03-02 Thread Robert Haas
On Fri, Mar 2, 2018 at 1:20 PM, Robert Haas wrote: > On Fri, Mar 2, 2018 at 5:35 AM, Etsuro Fujita > wrote: >> Agreed. Better safe than sorry, so I disabled autovacuum for all the tables >> created in the postgres_fdw regression test, except

Re: Cached/global query plans, autopreparation

2018-03-02 Thread Bruce Momjian
On Thu, Feb 15, 2018 at 03:00:17PM +0100, Shay Rojansky wrote: > Just wanted to say that I've seen more than 10% improvement in some real-world > application when preparation was done properly. Also, I'm assuming that > implementing this wouldn't involve "rewriting substantial part of Postgres >

Re: Testing "workers launched" in expected output? Really?

2018-03-02 Thread Robert Haas
On Fri, Mar 2, 2018 at 3:13 PM, Tom Lane wrote: > My point is that just because it isn't falling over on > relatively-lightly-loaded buildfarm machines doesn't mean that it won't > fall over in other environments. It doesn't mean it will, either. > I should think that your

Re: Cached/global query plans, autopreparation

2018-03-02 Thread Andres Freund
On 2018-03-02 15:29:09 -0500, Bruce Momjian wrote: > Postgres uses a conservative method for reusing plans with previous > constants, as described in the PREPARE manual page: > > https://www.postgresql.org/docs/10/static/sql-prepare.html > Prepared statements can use generic plans

Re: [PATCH] get rid of StdRdOptions, use individual binary reloptions representation for each relation kind instead

2018-03-02 Thread Andres Freund
Hi, On 2018-03-02 20:22:21 +0300, Nikolay Shaplov wrote: > Since I get a really big patch as a result, it was decided to commit it in > parts. I get that, but I strongly suggest not creating 10 loosely related threads, but keeping it as a patch series in one thread. It's really hard to follow

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-03-02 Thread Julian Markwort
Andres Freund wrote on 2018-03-02: > Yea, I misread the diff to think you added a conflicting version. Due > to: > -DATA =3D pg_stat_statements--1.4.sql pg_stat_statements--1.4--1.5.sql \ > +DATA =3D pg_stat_statements--1.5.sql pg_stat_statements--1.4--1.5.sql \ > and I'd checked that 1.5 already

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-03-02 Thread Tom Lane
Andres Freund writes: > Yea, I misread the diff to think you added a conflicting version. Due > to: > -DATA =3D pg_stat_statements--1.4.sql pg_stat_statements--1.4--1.5.sql \ > +DATA =3D pg_stat_statements--1.5.sql pg_stat_statements--1.4--1.5.sql \ > and I'd checked that 1.5

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-02 Thread Andres Freund
Hi, On 2018-03-01 21:39:36 -0500, David Steele wrote: > On 3/1/18 9:33 PM, Tomas Vondra wrote: > > On 03/02/2018 02:12 AM, Andres Freund wrote: > > > Hm, this CF entry is marked as needs review as of 2018-03-01 12:54:48, > > > but I don't see a newer version posted? > > > > > > > Ah, apologies

Re: [HACKERS] Creating backup history files for backups taken from standbys

2018-03-02 Thread David Steele
Hi, On 3/2/18 1:03 PM, Fujii Masao wrote: > On Fri, Mar 2, 2018 at 1:07 PM, Michael Paquier wrote: > >> We would talk about two backups running >> simultaneously on a standby, which would overlap with each other to >> generate a file aimed only at being helpful for

Re: JIT compiling with LLVM v11

2018-03-02 Thread Andres Freund
Hi, On 2018-03-02 19:13:01 -0500, Peter Eisentraut wrote: > On 3/1/18 03:02, Andres Freund wrote: > > I've pushed a revised version of my JIT patchset. > > The git tree is at > > https://git.postgresql.org/git/users/andresfreund/postgres.git > > in the jit branch > > > >

Re: [HACKERS] path toward faster partition pruning

2018-03-02 Thread David Rowley
On 3 March 2018 at 04:47, Robert Haas wrote: > On Fri, Mar 2, 2018 at 6:21 AM, Amit Langote > wrote: >> 2. Removing redundant clauses from each list, that is, >>remove_redundant_clauses() to produce lists with just one member per >>

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-02 Thread Tomas Vondra
On 03/02/2018 09:05 PM, Andres Freund wrote: > Hi, > > On 2018-03-01 21:39:36 -0500, David Steele wrote: >> On 3/1/18 9:33 PM, Tomas Vondra wrote: >>> On 03/02/2018 02:12 AM, Andres Freund wrote: Hm, this CF entry is marked as needs review as of 2018-03-01 12:54:48, but I don't see a

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-02 Thread Andres Freund
On 2018-03-03 02:00:46 +0100, Tomas Vondra wrote: > That is somewhat misleading, I think. You're right the last version was > submitted on 2018-01-19, but the next review arrived on 2018-01-31, i.e. > right at the end of the CF. So it's not like the patch was sitting there > with unresolved

Re: [COMMITTERS] pgsql: Add much-more-extensive TAP tests for pgbench.

2018-03-02 Thread Tom Lane
Peter Eisentraut writes: > On 9/8/17 09:32, Tom Lane wrote: >> Add much-more-extensive TAP tests for pgbench. > The use of done_testing() raises the requirement of Test::More from 0.82 > to 0.87. That seems OK, but we should update the version requirement in >

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-02 Thread Andres Freund
On 2018-03-03 02:34:06 +0100, Tomas Vondra wrote: > On 03/03/2018 02:01 AM, Andres Freund wrote: > > On 2018-03-03 02:00:46 +0100, Tomas Vondra wrote: > >> That is somewhat misleading, I think. You're right the last version > >> was submitted on 2018-01-19, but the next review arrived on > >>

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-02 Thread David Steele
On 3/2/18 8:01 PM, Andres Freund wrote: > On 2018-03-03 02:00:46 +0100, Tomas Vondra wrote: >> That is somewhat misleading, I think. You're right the last version was >> submitted on 2018-01-19, but the next review arrived on 2018-01-31, i.e. >> right at the end of the CF. So it's not like the

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-02 Thread Tomas Vondra
On 03/03/2018 02:37 AM, David Steele wrote: > On 3/2/18 8:01 PM, Andres Freund wrote: >> On 2018-03-03 02:00:46 +0100, Tomas Vondra wrote: >>> That is somewhat misleading, I think. You're right the last version was >>> submitted on 2018-01-19, but the next review arrived on 2018-01-31, i.e. >>>

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-02 Thread David Steele
On 3/2/18 8:54 PM, Tomas Vondra wrote: > On 03/03/2018 02:37 AM, David Steele wrote: >> On 3/2/18 8:01 PM, Andres Freund wrote: >>> On 2018-03-03 02:00:46 +0100, Tomas Vondra wrote: That is somewhat misleading, I think. You're right the last version was submitted on 2018-01-19, but the

Re: non-bulk inserts and tuple routing

2018-03-02 Thread Andres Freund
Hi, On 2018-02-22 11:10:57 -0500, Robert Haas wrote: > On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote > wrote: > >> Attached is an updated version for that. > > > > Thanks for updating the patch. > > Committed with a few changes. The big one was that I got rid of

Re: non-bulk inserts and tuple routing

2018-03-02 Thread Andres Freund
On 2018-02-22 11:10:57 -0500, Robert Haas wrote: > On Tue, Feb 20, 2018 at 8:06 PM, Amit Langote > wrote: > >> Attached is an updated version for that. > > > > Thanks for updating the patch. > > Committed with a few changes. The big one was that I got rid of the >

Re: GSOC 2018 ideas

2018-03-02 Thread Charles Cui
Got it, Aleksander! Will study these documents carefully! 2018-02-26 4:21 GMT-08:00 Aleksander Alekseev : > Hello Charles, > > > I saw PostgreSQL is selected in GSOC 2018 and pretty interested in the > > ideas of thrift data types support that proposed by you. So, I

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2018-03-02 Thread Craig Ringer
On 3 March 2018 at 13:08, Peter Eisentraut wrote: > On 1/22/18 21:33, Craig Ringer wrote: > > We don't have much in the way of rules about what input functions can or > > cannot do, so you can't assume much about their behaviour and what must > > / must not be

Re: heap_lock_updated_tuple_rec can leak a buffer refcount

2018-03-02 Thread Amit Kapila
On Sat, Mar 3, 2018 at 3:26 AM, Tom Lane wrote: > Alexander Kuzmenkov writes: >> Looks like a leak indeed, the fix seems right. > > > Hence, I propose the attached patch. The test lobotomization > (the "if (1) //" change) isn't meant for commit

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2018-03-02 Thread Peter Eisentraut
On 1/22/18 21:33, Craig Ringer wrote: > We don't have much in the way of rules about what input functions can or > cannot do, so you can't assume much about their behaviour and what must > / must not be cleaned up. Nor can you just reset the state in a heavy > handed manner like (say) plpgsql

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2018-03-02 Thread Erik Rijkers
On 2018-03-03 01:55, Tomas Vondra wrote: Hi there, attached is an updated patch fixing all the reported issues (a bit more about those below). Hi, 0007-Track-statistics-for-streaming-spilling.patch won't apply. All the other patches apply ok. patch complaints with: patching file

Re: psql tab completion for ALTER INDEX SET

2018-03-02 Thread Masahiko Sawada
On Sat, Mar 3, 2018 at 1:43 AM, Fujii Masao wrote: > On Fri, Mar 2, 2018 at 3:52 PM, Masahiko Sawada wrote: >> Hi, >> >> I found that tab completion for ALTER INDEX SET [tab] doesn't support >> the reloptions of brin and gist. Attached patch adds

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2018-03-02 Thread Tom Lane
Craig Ringer writes: > On 3 March 2018 at 13:08, Peter Eisentraut wrote: >> I think one thing to try would to define a special kind of exception >> that can safely be caught and ignored. Then, input functions can >> communicate benign

Re: WIP Patch: Precalculate stable functions, infrastructure v1

2018-03-02 Thread Marina Polyakova
Ok! On 02-03-2018 22:56, Andres Freund wrote: Hi, On 2018-03-02 11:22:01 +0300, Marina Polyakova wrote: I fixed the failure that Thomas pointed out to me, and I'm finishing work on it, but it took me a while to study this part of the executor.. I unfortunately think that makes this too

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-02 Thread David Gould
On Fri, 2 Mar 2018 18:47:44 +0300 Alexander Kuzmenkov wrote: > The calculation I made for the first step applies to the next steps too, > with minor differences. So, the estimate increases at each step. Just > out of interest, I plotted the reltuples for 60 steps,

Re: Documenting commitfest Rules

2018-03-02 Thread Tom Lane
Andres Freund writes: > Thus I'm trying to summarize my understanding in this email. It probably > doesn't 100% match other people's understanding. After we've resolved > those differences, we should update the wiki page, and delete outdated > content. > Commitfests primarily

Re: Online enabling of checksums

2018-03-02 Thread Tomas Vondra
On 03/02/2018 11:01 PM, Magnus Hagander wrote: > On Fri, Mar 2, 2018 at 5:50 PM, Tomas Vondra > > wrote: > > > > On 03/02/2018 02:35 PM, Magnus Hagander wrote: > > > > > > On Wed, Feb 28, 2018 at 6:06 PM, Robert

Re: prokind column (was Re: [HACKERS] SQL procedures)

2018-03-02 Thread Peter Eisentraut
On 2/28/18 17:37, Peter Eisentraut wrote: > On 2/28/18 15:45, Tom Lane wrote: >> I have reviewed this patch and attach an updated version below. >> I've rebased it up to today, fixed a few minor errors, and adopted >> most of Michael's suggestions. Also, since I remain desperately >> unhappy with

Re: [HACKERS] [POC] Faster processing at Gather node

2018-03-02 Thread Tels
Hello Robert, On Fri, March 2, 2018 12:22 pm, Robert Haas wrote: > On Wed, Feb 28, 2018 at 10:06 AM, Robert Haas > wrote: >> [ latest patches ] > > Committed. Thanks for the review. Cool :) There is a typo, tho: + /* +* If the counterpary is known to have

Documenting commitfest Rules

2018-03-02 Thread Andres Freund
Hi Tomas, Everyone. This seems like a good thing to discuss separately from the original thread. On 2018-03-02 22:06:32 +0100, Tomas Vondra wrote: > Can you guys please point me to the CF rules that say this? Because my > understanding (and not just mine, AFAICS) was obviously different. >

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-02 Thread David Gould
On Thu, 01 Mar 2018 18:49:20 -0500 Tom Lane wrote: > The sticking point in my mind right now is, if we do that, what to do with > VACUUM's estimates. If you believe the argument in the PDF that we'll > necessarily overshoot reltuples in the face of declining true density, >

Re: Re: [HACKERS] plpgsql - additional extra checks

2018-03-02 Thread Pavel Stehule
Hi 2018-03-01 21:14 GMT+01:00 David Steele : > Hi Pavel, > > On 1/7/18 3:31 AM, Pavel Stehule wrote: > > > > There, now it's in the correct Waiting for Author state. :) > > > > thank you for comments. All should be fixed in attached patch > > This patch no longer applies

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2018-03-02 Thread Pavel Stehule
2018-03-02 3:43 GMT+01:00 Pavel Stehule : > > > 2018-03-02 3:38 GMT+01:00 Andres Freund : > >> On 2018-03-02 03:13:04 +0100, Pavel Stehule wrote: >> > 2018-03-01 23:10 GMT+01:00 Andres Freund : >> > >> > > On 2018-01-23 17:08:56

Re: JIT compiling with LLVM v11

2018-03-02 Thread Peter Eisentraut
On 3/1/18 03:02, Andres Freund wrote: > I've pushed a revised version of my JIT patchset. > The git tree is at > https://git.postgresql.org/git/users/andresfreund/postgres.git > in the jit branch > >

Re: [patch] BUG #15005: ANALYZE can make pg_class.reltuples inaccurate.

2018-03-02 Thread Tom Lane
David Gould writes: > I'm confused at this point, I provided a patch that addresses this and a > test case. We seem to be discussing everything as if we first noticed the > issue. Have you reviewed the patch and and attached analysis and tested it? > Please commment on that?

Re: [HACKERS] log_destination=file

2018-03-02 Thread Magnus Hagander
On Fri, Mar 2, 2018 at 7:29 PM, Tom Lane wrote: > Robert Haas writes: > > On Mon, Jan 22, 2018 at 4:25 PM, Tomas Vondra > > wrote: > >> Sorry for the naive question, but which of these bottlenecks are we > >> actually

Re: heap_lock_updated_tuple_rec can leak a buffer refcount

2018-03-02 Thread Tom Lane
Alexander Kuzmenkov writes: > Looks like a leak indeed, the fix seems right. Yup, it's a leak. It's hard to hit because you need to be starting with an update of a tuple in an all-visible page; otherwise we never pin the vm page so there's nothing to leak. But if

Re: Online enabling of checksums

2018-03-02 Thread Magnus Hagander
On Fri, Mar 2, 2018 at 5:50 PM, Tomas Vondra wrote: > > > On 03/02/2018 02:35 PM, Magnus Hagander wrote: > > > > > > On Wed, Feb 28, 2018 at 6:06 PM, Robert Haas > > wrote: > > > > On Sun, Feb 25, 2018 at

  1   2   >