Re: Planning counters in pg_stat_statements (using pgss_store)

2019-02-14 Thread Sergei Kornilov
Hi >>  +#define PG_STAT_STATEMENTS_COLS_V1_4 25 > > I thought it was needed when adding new columns, isn't it ? Yes, this is needed. I mean it should be PG_STAT_STATEMENTS_COLS_V1_8: because such change was made for 1.8 pg_stat_statements version. Same thing for other version-specific places.

Re: ToDo: show size of partitioned table

2019-02-14 Thread Amit Langote
Hi Pavel, Thanks for updating the patch. On 2019/02/08 17:26, Pavel Stehule wrote: > I renamed originally calculated column "size" to "direct partitions size" > .. see Alvaro's comment. Then I introduced new column "total partitions > size" that is calculated like you propose. > > Now the result

Re: Delay locking partitions during INSERT and UPDATE

2019-02-14 Thread John Naylor
On 1/25/19, Tomas Vondra wrote: > Yes, I don't see why would the patch change that and I've been looking > for such cases. I think David was looking at that this week too, and I > assume he'll send an update if he finds anything. If not, I plan to get > it committed soon-ish (possibly next week af

Re: libpq host/hostaddr/conninfo inconsistencies

2019-02-14 Thread Kyotaro HORIGUCHI
Hello. At Thu, 14 Feb 2019 22:51:40 +0100 (CET), Fabien COELHO wrote in > > > On 2018-10-26 09:21:51 +0200, Fabien COELHO wrote: > >> (1) you are somehow against changing the current implementation, eg > >> erroring > >> out on possibly misleading configurations, because you do not think it >

Re: Early WIP/PoC for inlining CTEs

2019-02-14 Thread Andreas Karlsson
On 14/02/2019 16.11, Tom Lane wrote: ... so, have we beaten this topic to death yet? Can we make a decision? Personally, I'd be happy with either of the last two patch versions I posted (that is, either AS [[NOT] MATERIALIZED] or AS [MATERIALIZE [ON|OFF]] syntax). But we gotta pick something.

Re: pg11.1: dsa_area could not attach to segment

2019-02-14 Thread Thomas Munro
On Fri, Feb 15, 2019 at 2:31 AM Sergei Kornilov wrote: > I can not reproduce bug after 30min test long. (without patch bug was after > minute-two) Thank you Justin and Sergei for all your help reproducing and testing this. Fix pushed to all supported releases. It's lightly refactored from the

Re: Using POPCNT and other advanced bit manipulation instructions

2019-02-14 Thread Kyotaro HORIGUCHI
At Thu, 14 Feb 2019 16:45:38 -0500, Tom Lane wrote in <822.1550180...@sss.pgh.pa.us> > Andres Freund writes: > > On 2019-02-14 15:47:13 -0300, Alvaro Herrera wrote: > >> Hah, I just realized you have to add -mlzcnt in order for these builtins > >> to use the lzcnt instructions. It goes from som

Re: proposal: pg_restore --convert-to-text

2019-02-14 Thread Andreas Karlsson
On 14/02/2019 01.31, Euler Taveira wrote: Em qua, 13 de fev de 2019 às 19:56, Andrew Gierth escreveu: I propose we add a new option: --convert-to-text or some such name, and then make pg_restore throw a usage error if neither -d nor the new option is given. However, I agree that pg_restore to

RE: idle-in-transaction timeout error does not give a hint

2019-02-14 Thread Jamison, Kirk
Hi, On Monday, February 4, 2019 2:15 AM +, Michael Paquier wrote: > On Tue, Dec 04, 2018 at 04:07:34AM +, Ideriha, Takeshi wrote: > > Sure. I didn't have a strong opinion about it, so it's ok. > From what I can see this is waiting input from a native English speaker, so > for now I have

Re: Early WIP/PoC for inlining CTEs

2019-02-14 Thread Bruce Momjian
On Thu, Feb 14, 2019 at 04:25:27PM +0100, Peter Eisentraut wrote: > On 06/02/2019 10:00, Bruce Momjian wrote: > > I think "materialize" is the right word since "materialized" would be > > past tense. > > It's an adjective. The sentence is, "with foo as the materialized > $query, do the $main_quer

Re: pg_basebackup ignores the existing data directory permissions

2019-02-14 Thread Kyotaro HORIGUCHI
At Fri, 15 Feb 2019 08:15:24 +0900, Michael Paquier wrote in <20190214231524.gc2...@paquier.xyz> > On Thu, Feb 14, 2019 at 11:21:19PM +1100, Haribabu Kommi wrote: > > On Thu, Feb 14, 2019 at 8:57 PM Magnus Hagander wrote: > >> I think it could be argued that neither initdb *or* pg_basebackup sho

Re: Using POPCNT and other advanced bit manipulation instructions

2019-02-14 Thread Alvaro Herrera
On 2019-Feb-14, Tom Lane wrote: > I think we need a clean test for __builtin_popcount(), and to be willing > to use it if available, independently of -mpopcnt. Then separately we > should test to see if -mpopcnt works, probably with the same > infrastructure we use for checking for other compiler

Re: Ryu floating point output patch

2019-02-14 Thread Andrew Gierth
> "Andrew" == Andrew Dunstan writes: Andrew> Rather than give you the files, I will just tell you, in both Andrew> cases it is matching float8.out, not the small-is-zero file. OK, thanks. That lets me fix the float4 failures in a reasonably straightforward way. -- Andrew (irc:RhodiumToad

Re: pg_basebackup ignores the existing data directory permissions

2019-02-14 Thread Michael Paquier
On Thu, Feb 14, 2019 at 11:21:19PM +1100, Haribabu Kommi wrote: > On Thu, Feb 14, 2019 at 8:57 PM Magnus Hagander wrote: >> I think it could be argued that neither initdb *or* pg_basebackup should >> change the permissions on an existing directory, because the admin may have >> done that intention

Re: [Suspect SPAM] Better error messages when lacking connection slots for autovacuum workers and bgworkers

2019-02-14 Thread Michael Paquier
On Thu, Feb 14, 2019 at 09:04:37PM +0900, Kyotaro HORIGUCHI wrote: > I agree to the distinctive messages, but the autovaccum and > bgworker cases are in a kind of internal error, and they are not > "connection"s. I feel that elog is more suitable for them. I used ereport() for consistency with the

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-14 Thread Michael Paquier
On Thu, Feb 14, 2019 at 11:20:56AM +0300, Arthur Zakirov wrote: > So we read whole page with size XLOG_BLCKSZ. The full code: > https://github.com/postgrespro/pg_probackup/blob/c052651b8c8864733bcabbc2660c387b792229d8/src/parsexlog.c#L1074 > > Here is the little optimization I made. Mainly I just

Re: Using POPCNT and other advanced bit manipulation instructions

2019-02-14 Thread Tom Lane
Alvaro Herrera writes: > That leads me to the attached patch. It creates a new file > pg_popcount.c which is the only one compiled with -mpopcnt (if > available); if there's no compiler switch to enable POPCNT, we just > don't compile the file. I'm not sure that's kosher -- in particular I'm > n

Re: Using POPCNT and other advanced bit manipulation instructions

2019-02-14 Thread Alvaro Herrera
On 2019-Feb-14, Tom Lane wrote: > static inline int > pg_clz(...) Hmm, I missed this bit. So we put all these functions in the header, as in the attached. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services commit

Re: Using POPCNT and other advanced bit manipulation instructions

2019-02-14 Thread Alvaro Herrera
On 2019-Feb-14, Tom Lane wrote: > I'd bet a fair amount of money that we'd be better off *not* using > lzcnt, even if available, because then we could just expose things > along this line: > > static inline int > pg_clz(...) > { > #ifdef HAVE__BUILTIN_CLZ > return __builtin_clz(x); > #else >

Re: Ryu floating point output patch

2019-02-14 Thread Andrew Dunstan
On 2/14/19 12:42 PM, Andrew Dunstan wrote: > On 2/14/19 12:24 PM, Andrew Gierth wrote: >> Andrew, >> >> Is there any chance you can get me the regress/results/float[48].out >> files from lorikeet and jacana? It would help a lot. >> >> Seeing the diffs isn't enough, because I want to know if the f

Re: libpq host/hostaddr/conninfo inconsistencies

2019-02-14 Thread Fabien COELHO
On 2018-10-26 09:21:51 +0200, Fabien COELHO wrote: (1) you are somehow against changing the current implementation, eg erroring out on possibly misleading configurations, because you do not think it is really useful to help users in those cases. I find this formulation somewhat passive aggre

Re: Using POPCNT and other advanced bit manipulation instructions

2019-02-14 Thread Tom Lane
Andres Freund writes: > On 2019-02-14 15:47:13 -0300, Alvaro Herrera wrote: >> Hah, I just realized you have to add -mlzcnt in order for these builtins >> to use the lzcnt instructions. It goes from something like >> >> bsrq %rax, %rax >> xorq $63, %rax > I'm confused how this is a general coun

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-02-14 Thread legrand legrand
Thank you Sergei for your comments, > Did you register patch in CF app? I did not found entry. created today: https://commitfest.postgresql.org/22/1999/ > Currently we have pg_stat_statements 1.7 version and this patch does not > apply... will rebase and create a 1.8 version > -

Re: pgbench - add pseudo-random permutation function

2019-02-14 Thread Fabien COELHO
Hello Andres, +# PGAC_C_BUILTIN_CLZLL I think this has been partially superceded by commit 711bab1e4d19b5c9967328315a542d93386b1ac5 Author: Alvaro Herrera Date: 2019-02-13 16:10:06 -0300 Indeed, the patch needs a rebase & conflit resolution. I'll do it. Later. +Function pr_pe

Re: libpq debug log

2019-02-14 Thread Jacob Champion
On Thu, Feb 14, 2019 at 10:17 AM Andres Freund wrote: > On 2018-11-28 23:20:03 +0100, Peter Eisentraut wrote: > > This does not excite me. It seems mostly redundant with using tcpdump. > > I think the one counter-argument to this is that using tcpdump in > real-world scenarios has become quite ha

Re: pg11.1: dsa_area could not attach to segment

2019-02-14 Thread Thomas Munro
On Fri, Feb 15, 2019 at 5:36 AM Sergei Kornilov wrote: > > Do you think that plausibly explains and resolves symptoms of bug#15585, > > too? > > I think yes. Bug#15585 raised only after "dsa_area could not attach to > segment" in different parallel worker. Leader stuck because waiting all > par

2019-03 CF Summary / Review - Tranche #1

2019-02-14 Thread Andres Freund
Hi, As last year [1], I'll try to summarize all commitfest items in 2019-03 to see which I think could realistically be put into 12. Going through all non bugfix CF entries. Here's the summary for the entries I could stomach today: RFC: ready for committer NR: needs review WOA: waiting on autho

Re: log bind parameter values on error

2019-02-14 Thread Andres Freund
Hi, tiny scroll-through review. On 2019-01-28 00:15:51 +, Alexey Bashtanov wrote: > diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml > index b6f5822..997e6e8 100644 > --- a/doc/src/sgml/config.sgml > +++ b/doc/src/sgml/config.sgml > @@ -6274,6 +6274,23 @@ log_line_prefix = '%m

Re: log bind parameter values on error

2019-02-14 Thread Andres Freund
Hi, On 2018-12-14 23:04:26 +, Alexey Bashtanov wrote: > Unfortunately, when enabled, the feature comes with some memory and CPU > overhead, > as we cannot convert certain values to text when in aborted transaction. Have you analyzed how invasive it'd be to delay that till we actually can safe

Re: Log a sample of transactions

2019-02-14 Thread Andres Freund
Hi, On 2019-01-26 11:44:58 +0100, Adrien NAYRAT wrote: > + xreflabel="log_transaction_sample_rate"> > + log_transaction_sample_rate > (real) > + > + log_transaction_sample_rate configuration > parameter > + > + > + > + > + Set the fraction

Re: more unconstify use

2019-02-14 Thread Peter Eisentraut
On 13/02/2019 19:51, Mark Dilger wrote: > Peter, so sorry I did not review this patch before you committed. There > are a few places where you unconstify the argument to a function where > changing the function to take const seems better to me. I argued for > something similar in 2016, One can c

Re: shared-memory based stats collector

2019-02-14 Thread Andres Freund
Hi Kyatoro, On 2019-02-07 13:10:08 -0800, Andres Freund wrote: > I don't think this is all that close to being committable :( Are you planning to update this soon? I think this needs to be improved pretty quickly to have any shot at getting into v12. I'm willing to put in some resources towards t

Re: explain plans with information about (modified) gucs

2019-02-14 Thread Andres Freund
Hi, On 2019-01-15 02:39:49 +0100, Tomas Vondra wrote: > > > On 1/14/19 11:13 PM, Alvaro Herrera wrote: > > On 2019-Jan-14, Tomas Vondra wrote: > > > >> The one slightly annoying issue is that currently all the options are > >> formatted as text, including e.g. cpu_tuple_cost. That's because the

Re: Per-tablespace autovacuum settings

2019-02-14 Thread Oleksii Kliukin
Tom Lane wrote: > Oleksii Kliukin writes: >> Is there any interest in making autovacuum parameters available on a >> tablespace level in order to apply those to all vacuumable objects in the >> tablespace? > > I understand what you want to accomplish, and it doesn't seem > unreasonable. But I

Re: make installcheck-world in a clean environment

2019-02-14 Thread Andres Freund
Hi, On 2019-02-04 11:11:03 +0900, Michael Paquier wrote: > On Mon, Dec 03, 2018 at 11:58:13AM +0300, Alexander Lakhin wrote: > > Rebased the patch once more after d3c09b9b. > > The patch is ready for committer, so it has not attracted much > attention. Perhaps Teodor or Alexander could look at i

Re: Using POPCNT and other advanced bit manipulation instructions

2019-02-14 Thread Andres Freund
Hi, On 2019-02-14 15:47:13 -0300, Alvaro Herrera wrote: > On 2019-Feb-14, Tom Lane wrote: > > > Some further thoughts here ... > > > > Does the "lzcnt" runtime probe actually do anything useful? > > On the x86_64 compilers I tried (gcc 8.2.1 and 4.4.7), __builtin_clz > > and __builtin_ctz compil

Re: INSTALL file

2019-02-14 Thread Andres Freund
Hi, On 2019-02-04 11:02:44 +0900, Michael Paquier wrote: > +1. I have just looked at the patch, and my take is that listing all > the ways to build Postgres directly in the README is just a > duplication of what we already have in the documentation. So I would > just rip out all that and keep a

Re: Early WIP/PoC for inlining CTEs

2019-02-14 Thread Merlin Moncure
On Thu, Feb 14, 2019 at 10:02 AM Alvaro Herrera wrote: > > On 2019-Feb-14, Peter Eisentraut wrote: > > > On 14/02/2019 16:11, Tom Lane wrote: > > > ... so, have we beaten this topic to death yet? Can we make a decision? > > > > > > Personally, I'd be happy with either of the last two patch versio

Re: Proposal for Signal Detection Refactoring

2019-02-14 Thread Andres Freund
Hi, On 2019-01-23 11:55:09 +0100, Chris Travers wrote: > +Implementation Notes on Globals and Signal/Event Handling > += > + > +The approch to signal handling in PostgreSQL is designed to strictly conform > +with the C89 standard and designed

Re: Using POPCNT and other advanced bit manipulation instructions

2019-02-14 Thread Tom Lane
Alvaro Herrera writes: > Hah, I just realized you have to add -mlzcnt in order for these builtins > to use the lzcnt instructions. It goes from something like > bsrq%rax, %rax > xorq$63, %rax > to > lzcntq %rax, %rax > Significant? I'd bet a fair amount of money tha

Re: Using POPCNT and other advanced bit manipulation instructions

2019-02-14 Thread Alvaro Herrera
On 2019-Feb-14, Tom Lane wrote: > Some further thoughts here ... > > Does the "lzcnt" runtime probe actually do anything useful? > On the x86_64 compilers I tried (gcc 8.2.1 and 4.4.7), __builtin_clz > and __builtin_ctz compile to sequences involving bsrq and bsfq > regardless of -mpopcnt. It's

Re: \describe*

2019-02-14 Thread Andres Freund
Hi, On 2019-01-24 20:37:48 -0500, Corey Huinker wrote: > Attached is a patch to add verbose \describe commands to compliment our > existing but slightly cryptic family of \d commands. Given that this patch has been added to the last commitfest for v12, I think we should mark it as targeting 13, s

Re: libpq host/hostaddr/conninfo inconsistencies

2019-02-14 Thread Andres Freund
Hi, On 2018-10-26 09:21:51 +0200, Fabien COELHO wrote: > (1) you are somehow against changing the current implementation, eg erroring > out on possibly misleading configurations, because you do not think it is > really useful to help users in those cases. I find this formulation somewhat passive

Re: libpq debug log

2019-02-14 Thread Andres Freund
Hi, On 2018-11-28 23:20:03 +0100, Peter Eisentraut wrote: > This does not excite me. It seems mostly redundant with using tcpdump. I think the one counter-argument to this is that using tcpdump in real-world scenarios has become quite hard, due to encryption. Even with access to the private key

Re: Per-tablespace autovacuum settings

2019-02-14 Thread Oleksii Kliukin
Andres Freund wrote: > Hi, > > On 2019-02-14 17:56:17 +0100, Oleksii Kliukin wrote: >> Is there any interest in making autovacuum parameters available on a >> tablespace level in order to apply those to all vacuumable objects in the >> tablespace? >> >> We have a set of tables running on ZFS, w

Re: pgbench - add pseudo-random permutation function

2019-02-14 Thread Andres Freund
Hi, On 2019-02-10 17:46:15 +, Hironobu SUZUKI wrote: > I updated the patch. And also I added some explanations and simple examples > in the modular_multiply function. It'd be good to update the commitfest entry to say 'needs review' the next time. > +# PGAC_C_BUILTIN_CLZLL > +#

Re: Ryu floating point output patch

2019-02-14 Thread Andrew Dunstan
On 2/14/19 12:24 PM, Andrew Gierth wrote: > Andrew, > > Is there any chance you can get me the regress/results/float[48].out > files from lorikeet and jacana? It would help a lot. > > Seeing the diffs isn't enough, because I want to know if the float8 test > (which passes, so there's no diff) is

Re: libpq compression

2019-02-14 Thread Konstantin Knizhnik
On 14.02.2019 19:45, Dmitry Dolgov wrote: For the records, I'm really afraid of interfering with the conversation at this point, but I believe it's necessary for the sake of a good feature :) On Wed, Feb 13, 2019 at 4:03 PM Konstantin Knizhnik wrote: 1. When decompressor has not enough da

Re: Ryu floating point output patch

2019-02-14 Thread Andrew Gierth
Andrew, Is there any chance you can get me the regress/results/float[48].out files from lorikeet and jacana? It would help a lot. Seeing the diffs isn't enough, because I want to know if the float8 test (which passes, so there's no diff) is matching the standard file or the -small-is-zero file.

Re: Per-tablespace autovacuum settings

2019-02-14 Thread Tom Lane
Oleksii Kliukin writes: > Is there any interest in making autovacuum parameters available on a > tablespace level in order to apply those to all vacuumable objects in the > tablespace? I understand what you want to accomplish, and it doesn't seem unreasonable. But I just want to point out that t

Re: Per-tablespace autovacuum settings

2019-02-14 Thread Andres Freund
Hi, On 2019-02-14 17:56:17 +0100, Oleksii Kliukin wrote: > Is there any interest in making autovacuum parameters available on a > tablespace level in order to apply those to all vacuumable objects in the > tablespace? > > We have a set of tables running on ZFS, where autovacuum does almost no goo

Per-tablespace autovacuum settings

2019-02-14 Thread Oleksii Kliukin
Hello, Is there any interest in making autovacuum parameters available on a tablespace level in order to apply those to all vacuumable objects in the tablespace? We have a set of tables running on ZFS, where autovacuum does almost no good to us (except for preventing anti-wraparound) due to the n

Re: libpq compression

2019-02-14 Thread Dmitry Dolgov
For the records, I'm really afraid of interfering with the conversation at this point, but I believe it's necessary for the sake of a good feature :) > On Wed, Feb 13, 2019 at 4:03 PM Konstantin Knizhnik > wrote: > > 1. When decompressor has not enough data to produce any extra output, it > does

Inconsistencies between dependency.c and objectaddress.c

2019-02-14 Thread Tom Lane
In <26527.1549572...@sss.pgh.pa.us> I speculated about adding a function to objectaddress.c that would probe to see if an object with a given ObjectAddress (still) exists. I started to implement this, but soon noticed that objectaddress.c doesn't cover all the object classes that dependency.c know

Re: pg11.1: dsa_area could not attach to segment

2019-02-14 Thread Sergei Kornilov
Hi > Do you think that plausibly explains and resolves symptoms of bug#15585, too? I think yes. Bug#15585 raised only after "dsa_area could not attach to segment" in different parallel worker. Leader stuck because waiting all parallel workers, but one worker has unexpected recursion in dsm_back

Re: WAL insert delay settings

2019-02-14 Thread Andres Freund
On 2019-02-14 11:02:24 -0500, Stephen Frost wrote: > Greetings, > > On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> wrote: > > > On 14/02/2019 11:03, Tomas Vondra wrote: > > > But if you add extra sleep() calls somewhere (say because there's also > > > limit o

Re: pg11.1: dsa_area could not attach to segment

2019-02-14 Thread Justin Pryzby
On Fri, Feb 15, 2019 at 01:12:35AM +1300, Thomas Munro wrote: > The problem is that a DSM handle (ie a random number) can be reused > for a new segment immediately after the shared memory object has been > destroyed but before the DSM slot has been released. Now two DSM > slots have the same handl

Re: WAL insert delay settings

2019-02-14 Thread Andres Freund
Hi, On 2019-02-14 16:16:05 +0100, Peter Eisentraut wrote: > On 13/02/2019 16:40, Andres Freund wrote: > > On February 13, 2019 4:39:21 PM GMT+01:00, Peter Eisentraut > > wrote: > >> On 13/02/2019 13:18, Andres Freund wrote: > >>> But I don't think the way you did it is acceptable - we can't just

Re: Early WIP/PoC for inlining CTEs

2019-02-14 Thread Alvaro Herrera
On 2019-Feb-14, Peter Eisentraut wrote: > On 14/02/2019 16:11, Tom Lane wrote: > > ... so, have we beaten this topic to death yet? Can we make a decision? > > > > Personally, I'd be happy with either of the last two patch versions > > I posted (that is, either AS [[NOT] MATERIALIZED] or > > AS [

Re: Why don't we have a small reserved OID range for patch revisions?

2019-02-14 Thread John Naylor
I wrote: > On 2/8/19, Tom Lane wrote: >> A script such as you suggest might be a good way to reduce the temptation >> to get lazy at the last minute. Now that the catalog data is pretty >> machine-readable, I suspect it wouldn't be very hard --- though I'm >> not volunteering either. I'm envisi

Re: WAL insert delay settings

2019-02-14 Thread Stephen Frost
Greetings, On Thu, Feb 14, 2019 at 10:15 Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 14/02/2019 11:03, Tomas Vondra wrote: > > But if you add extra sleep() calls somewhere (say because there's also > > limit on WAL throughput), it will affect how fast VACUUM works in > > gene

Re: Protect syscache from bloating with negative cache entries

2019-02-14 Thread 'Bruce Momjian'
On Thu, Feb 14, 2019 at 01:31:49AM +, Tsunakawa, Takayuki wrote: > From: Bruce Momjian [mailto:br...@momjian.us] > > > That being said, having a "minimal size" threshold before starting > > > with the time-based eviction may be a good idea. > > > > Agreed. I see the minimal size as a way to ke

Re: ALTER TABLE on system catalogs

2019-02-14 Thread Peter Eisentraut
On 08/02/2019 04:04, Kyotaro HORIGUCHI wrote: > By the way, I'm confused to see that attributes that don't want > to go external are marked as 'x' in system catalogs. Currently > (putting aside its necessity) the following operation ends with > successful attaching a new TOAST relation, which we re

Re: Early WIP/PoC for inlining CTEs

2019-02-14 Thread Peter Eisentraut
On 06/02/2019 10:00, Bruce Momjian wrote: > I think "materialize" is the right word since "materialized" would be > past tense. It's an adjective. The sentence is, "with foo as the materialized $query, do the $main_query". It's the same as "materialized view". -- Peter Eisentraut

Re: COPY support for parameters

2019-02-14 Thread Tom Lane
Adrian Phinney writes: > Does Postgres support COPY with parameters? No. In general you can only use parameters in DML statements (SELECT/INSERT/UPDATE/DELETE); utility statements don't cope, mainly because most of them lack expression eval capability altogether. Perhaps the special case of COP

Re: Early WIP/PoC for inlining CTEs

2019-02-14 Thread Peter Eisentraut
On 14/02/2019 16:11, Tom Lane wrote: > ... so, have we beaten this topic to death yet? Can we make a decision? > > Personally, I'd be happy with either of the last two patch versions > I posted (that is, either AS [[NOT] MATERIALIZED] or > AS [MATERIALIZE [ON|OFF]] syntax). But we gotta pick som

Re: WAL insert delay settings

2019-02-14 Thread Peter Eisentraut
On 14/02/2019 11:03, Tomas Vondra wrote: > But if you add extra sleep() calls somewhere (say because there's also > limit on WAL throughput), it will affect how fast VACUUM works in > general. Yet it'll continue with the cost-based throttling, but it will > never reach the limits. Say you do anothe

Re: WAL insert delay settings

2019-02-14 Thread Peter Eisentraut
On 13/02/2019 16:40, Andres Freund wrote: > On February 13, 2019 4:39:21 PM GMT+01:00, Peter Eisentraut > wrote: >> On 13/02/2019 13:18, Andres Freund wrote: >>> But I don't think the way you did it is acceptable - we can't just >> delay while holding buffer locks, in critical sections, while not

Re: Ryu floating point output patch

2019-02-14 Thread Andrew Dunstan
On 2/13/19 12:09 PM, Andrew Gierth wrote: >> "Andrew" == Andrew Gierth writes: > Andrew> 2. How far do we need to go to support existing uses of > Andrew>extra_float_digits? For example, do we need a way for clients to > Andrew>request that they get the exact same output as previo

Re: Early WIP/PoC for inlining CTEs

2019-02-14 Thread Tom Lane
... so, have we beaten this topic to death yet? Can we make a decision? Personally, I'd be happy with either of the last two patch versions I posted (that is, either AS [[NOT] MATERIALIZED] or AS [MATERIALIZE [ON|OFF]] syntax). But we gotta pick something. regards, tom l

COPY support for parameters

2019-02-14 Thread Adrian Phinney
Hello, I'm trying to add support for specifying parameters when using a COPY command to Npgsql (.NET's Postgres provider): https://github.com/npgsql/npgsql/pull/2332 I've used the extended query protocol to send the COPY command. When I send a COPY command without parameters, the backend issues t

Re: [WIP] CREATE SUBSCRIPTION with FOR TABLES clause (table filter)

2019-02-14 Thread Evgeniy Efimkin
Hi! Thanks for comments! I fixed build and return lines about subscription apply process in doc Efimkin Evgeny diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml index 3f2f674a1a..5d211646bf 100644 --- a/doc/src/sgml/logical-replication.sgml +++ b/

Re: Protect syscache from bloating with negative cache entries

2019-02-14 Thread Bruce Momjian
On Thu, Feb 14, 2019 at 12:40:10AM -0800, Andres Freund wrote: > Hi, > > On 2019-02-13 15:31:14 +0900, Kyotaro HORIGUCHI wrote: > > Instead, I added an accounting(?) interface function. > > > > | MemoryContextGettConsumption(MemoryContext cxt); > > > > The API returns the current consumption in

[Patch] checksumming-related buglets in pg_verify_checksums/pg_basebackup TAP tests

2019-02-14 Thread Michael Banck
Hi, while hacking on pg_verify_checksums and looking at hexdumps, I noticed that the TAP tests of pg_verify_checksums (and pg_basebackup from which it was copy-pasted) actually write "305c305c[...]" (i.e. literal backslashes and number 0s) instead of "000[...]" into the to-be- corrupted relfilenod

Re: pg11.1: dsa_area could not attach to segment

2019-02-14 Thread Sergei Kornilov
Hi! Great work, thank you! I can not reproduce bug after 30min test long. (without patch bug was after minute-two) regards Sergei

Re: [WIP] CREATE SUBSCRIPTION with FOR TABLES clause (table filter)

2019-02-14 Thread Andrey Borodin
Hi! > 11 февр. 2019 г., в 16:30, Evgeniy Efimkin > написал(а): > The patch seems good to me but cfbot is not happy. Can you please investigate what's wrong with this build? https://travis-ci.org/postgresql-cfbot/postgresql/builds/492912868 Also, I'm not sure we should drop this lines from doc

Re: ALTER TABLE on system catalogs

2019-02-14 Thread Chris Travers
I have a couple of thoughts here. On Fri, Feb 8, 2019 at 4:35 AM Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > At Fri, 08 Feb 2019 12:03:31 +0900 (Tokyo Standard Time), Kyotaro > HORIGUCHI wrote in < > 20190208.120331.167280496.horiguchi.kyot...@lab.ntt.co.jp> > > By the way, I'm

Re: Cache relation sizes?

2019-02-14 Thread Kyotaro HORIGUCHI
2019年2月14日(木) 20:41、Kyotaro HORIGUCHI さん(horiguchi.kyot...@lab.ntt.co.jp )のメッセージ: > At Wed, 13 Feb 2019 05:48:28 +, "Jamison, Kirk" < > k.jami...@jp.fujitsu.com> wrote in > > > On February 6, 2019, 8:57 AM +, Andres Freund wrote: > > > Maybe I'm missing something here, but why is it actua

Re: pg_basebackup ignores the existing data directory permissions

2019-02-14 Thread Haribabu Kommi
On Thu, Feb 14, 2019 at 8:57 PM Magnus Hagander wrote: > On Thu, Feb 14, 2019 at 9:10 AM Michael Paquier > wrote: > >> On Thu, Feb 14, 2019 at 06:34:07PM +1100, Haribabu Kommi wrote: >> > we have an application that is used to create the data directory with >> >> Well, initdb would do that happi

Re: pg11.1: dsa_area could not attach to segment

2019-02-14 Thread Thomas Munro
On Tue, Feb 12, 2019 at 10:15 PM Sergei Kornilov wrote: > I still have error with parallel_leader_participation = off. Justin very kindly set up a virtual machine similar to the one where he'd seen the problem so I could experiment with it. Eventually I also managed to reproduce it locally, and

Re: [Suspect SPAM] Better error messages when lacking connection slots for autovacuum workers and bgworkers

2019-02-14 Thread Kyotaro HORIGUCHI
Hello. At Wed, 13 Feb 2019 14:13:09 +0900, Michael Paquier wrote in <20190213051309.gf5...@paquier.xyz> > Hi all, > > When lacking connection slots, the backend returns a simple "sorry, > too many clients", which is something that has been tweaked by recent > commit ea92368 for WAL senders. Ho

Re: Cache relation sizes?

2019-02-14 Thread Kyotaro HORIGUCHI
At Wed, 13 Feb 2019 05:48:28 +, "Jamison, Kirk" wrote in > On February 6, 2019, 8:57 AM +, Andres Freund wrote: > > Maybe I'm missing something here, but why is it actually necessary to > > have the sizes in shared memory, if we're just talking about caching > > sizes? It's pretty darn

RE: [PROPOSAL]a new data type 'bytea' for ECPG

2019-02-14 Thread Matsumura, Ryo
Meskes-san > Yes, I agree with this. But it does not explain why we cannot just add > a length parameter. And it neither explains why we need so many if > (!bytea) { thisandthat } else { somethingelse } blocks. I would prefer > the integration to be smoother. Hopefully that is possible. I agree t

Re: [HACKERS] Block level parallel vacuum

2019-02-14 Thread Masahiko Sawada
On Wed, Feb 13, 2019 at 9:32 PM Haribabu Kommi wrote: > > > On Sat, Feb 9, 2019 at 11:47 PM Masahiko Sawada wrote: >> >> On Tue, Feb 5, 2019 at 12:14 PM Haribabu Kommi >> wrote: >> > >> > >> > On Fri, Feb 1, 2019 at 8:19 AM Masahiko Sawada >> > wrote: >> >> >> >> >> >> The passing stats = NUL

Re: WAL insert delay settings

2019-02-14 Thread Tomas Vondra
On 2/14/19 10:36 AM, Andres Freund wrote: > > > On February 14, 2019 10:31:57 AM GMT+01:00, Tomas Vondra > wrote: >> >> >> On 2/14/19 10:06 AM, Andres Freund wrote: >>> Hi, >>> >>> On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote: On 2/13/19 4:31 PM, Stephen Frost wrote: > Greetings

Re: pg_basebackup ignores the existing data directory permissions

2019-02-14 Thread Magnus Hagander
On Thu, Feb 14, 2019 at 9:10 AM Michael Paquier wrote: > On Thu, Feb 14, 2019 at 06:34:07PM +1100, Haribabu Kommi wrote: > > we have an application that is used to create the data directory with > > Well, initdb would do that happily, so there is no actual any need to > do that to begin with. An

Regarding participating in GSOC 2019 with PostgreSQL

2019-02-14 Thread Abhishek Agrawal
Hello, My name is Abhishek Agrawal and I am a CSE UG student of IIT Patna. I am interested in doing GSOC with PostgreSQL if you guys are applying for it this year. If anyone could direct me to proper links or some channel to prepare for the same then it will be really helpful. I have some quest

useless argument of ATAddForeignKeyConstraint

2019-02-14 Thread Amit Langote
Hi, While reviewing the foreign keys referencing partitioned tables patch, I noticed that the parentConstr argument of ATAddForeignConstraint is rendered useless by the following commit: commit 0325d7a5957ba39a0dce90835ab54a08ab8bf762 Author: Alvaro Herrera Date: Fri Jan 18 14:49:40 2019 -0300

Re: Problems with plan estimates in postgres_fdw

2019-02-14 Thread Etsuro Fujita
(2019/02/12 20:43), Antonin Houska wrote: Etsuro Fujita wrote: Here are my review comments: root->upper_targets[UPPERREL_FINAL] = final_target; + root->upper_targets[UPPERREL_ORDERED] = final_target; root->upper_targets[UPPERREL_WINDOW] = sort_inpu

Re: WAL insert delay settings

2019-02-14 Thread Andres Freund
On February 14, 2019 10:31:57 AM GMT+01:00, Tomas Vondra wrote: > > >On 2/14/19 10:06 AM, Andres Freund wrote: >> Hi, >> >> On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote: >>> On 2/13/19 4:31 PM, Stephen Frost wrote: Greetings, * Peter Eisentraut (peter.eisentr...@2ndquadrant

Re: WAL insert delay settings

2019-02-14 Thread Tomas Vondra
On 2/14/19 10:06 AM, Andres Freund wrote: > Hi, > > On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote: >> On 2/13/19 4:31 PM, Stephen Frost wrote: >>> Greetings, >>> >>> * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: Bulk operations like CREATE INDEX, ALTER TABLE, or bulk load

Re: WAL insert delay settings

2019-02-14 Thread Andres Freund
Hi, On 2019-02-14 10:00:38 +0100, Tomas Vondra wrote: > On 2/13/19 4:31 PM, Stephen Frost wrote: > > Greetings, > > > > * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > >> Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create > >> a lot of WAL. A lot of WAL at on

Re: WAL insert delay settings

2019-02-14 Thread Peter Geoghegan
On Thu, Feb 14, 2019 at 12:53 AM Peter Geoghegan wrote: > I didn't mention that the utility they used would send SIGSTOP and > SIGCONT in close succession. (Yeah, I know.) Actually, it SIGSTOP'd backends, not the WAL writer or background writer. -- Peter Geoghegan

Re: WAL insert delay settings

2019-02-14 Thread Tomas Vondra
On 2/13/19 4:31 PM, Stephen Frost wrote: > Greetings, > > * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: >> Bulk operations like CREATE INDEX, ALTER TABLE, or bulk loads can create >> a lot of WAL. A lot of WAL at once can cause delays in replication. > > Agreed, though I think

Re: WAL insert delay settings

2019-02-14 Thread Peter Geoghegan
On Thu, Feb 14, 2019 at 12:52 AM Peter Geoghegan wrote: > I imagine that it held the critical locks briefly. I didn't mention that the utility they used would send SIGSTOP and SIGCONT in close succession. (Yeah, I know.) -- Peter Geoghegan

Re: WAL insert delay settings

2019-02-14 Thread Peter Geoghegan
On Thu, Feb 14, 2019 at 12:42 AM Andres Freund wrote: > That can't have been the workaround - either you'd interrupt it while > holding critical locks (in which case nobody could write WAL anymore), > or you'd just move all the writing to backends, no? I imagine that it held the critical locks br

Re: WAL insert delay settings

2019-02-14 Thread Andres Freund
Hi, On 2019-02-13 23:21:39 -0800, Peter Geoghegan wrote: > There would occasionally be cases where ops > would find it useful to throttle WAL writing using their own terrible > kludge (it involved sending SIGSTOP to the WAL writer). That can't have been the workaround - either you'd interrupt it

Re: Protect syscache from bloating with negative cache entries

2019-02-14 Thread Andres Freund
Hi, On 2019-02-13 15:31:14 +0900, Kyotaro HORIGUCHI wrote: > Instead, I added an accounting(?) interface function. > > | MemoryContextGettConsumption(MemoryContext cxt); > > The API returns the current consumption in this memory > context. This allows "real" memory accounting almost without > ov

RE: Protect syscache from bloating with negative cache entries

2019-02-14 Thread Ideriha, Takeshi
>From: Kyotaro HORIGUCHI [mailto:horiguchi.kyot...@lab.ntt.co.jp] > > >(2) Another new patch v15-0005 on top of previous design of > limit-by-number-of-a-cache feature converts it to > limit-by-size-on-all-caches feature, which I think is > Tsunakawa-san wanted. Yeah, size looks better to me. >

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-14 Thread Arthur Zakirov
On 14.02.2019 09:51, Michael Paquier wrote: Now I don't actually agree that this qualifies as a bug fix. As things stand, a page may finish by being more than once if what has been read previously equals what is requested, however this does not prevent the code to work correctly. The performanc

  1   2   >