RE: Protect syscache from bloating with negative cache entries

2019-01-24 Thread Tsunakawa, Takayuki
From: Robert Haas [mailto:robertmh...@gmail.com] > On Thu, Jan 24, 2019 at 10:02 AM Tom Lane wrote: > > I will argue hard that we should not do it at all, ever. > > > > There is already a mechanism for broadcasting global GUC changes: > > apply them to postgresql.conf (or use ALTER SYSTEM) and

Re: PostgreSQL vs SQL/XML Standards

2019-01-24 Thread Pavel Stehule
pá 25. 1. 2019 v 5:46 odesílatel Chapman Flack napsal: > Hi, > > On 01/21/19 01:33, Pavel Stehule wrote: > > ne 20. 1. 2019 23:13 odesílatel Chapman Flack > > napsal: > > >> form (whether or not the word LATERAL is used), and re-executes xmltable > >> whenever the referenced column value

Re: proposal - plpgsql unique statement id

2019-01-24 Thread Pavel Stehule
čt 24. 1. 2019 v 23:08 odesílatel Tom Lane napsal: > Peter Eisentraut writes: > > committed > > Why didn't this patch modify the dumping logic in pl_funcs.c to print > the IDs? I'm not aware of other cases where we intentionally omit > fields from debug-support printouts. > I had not a idea,

Use zero for nullness estimates of system attributes

2019-01-24 Thread Edmund Horner
I added some code to selfuncs.c to estimate the selectivity of CTID, including nullness, in my ongoing attempt to add TID range scans [1]. And as Tom pointed out [2], no system attribute can be null, so we might as well handle them all. That's what the attached patch does. I observed a few

Re: backslash-dot quoting in COPY CSV

2019-01-24 Thread Bruce Momjian
On Wed, Jan 2, 2019 at 04:58:35PM +0100, Daniel Verite wrote: > Hi, > > The doc on COPY CSV says about the backslash-dot sequence: > > To avoid any misinterpretation, a \. data value appearing as a > lone entry on a line is automatically quoted on output, and on > input, if quoted, is

Re: WIP: Avoid creation of the free space map for small tables

2019-01-24 Thread Amit Kapila
On Fri, Jan 25, 2019 at 1:03 AM John Naylor wrote: > > On Wed, Jan 23, 2019 at 11:17 PM Amit Kapila wrote: > > I think what doc means to say is > > that it copies any unlinked files present in primary's new cluster > > (which in your case will be data2). > > In that case, I'm still confused why

RE: pg_upgrade: Pass -j down to vacuumdb

2019-01-24 Thread Jamison, Kirk
Hi, According to CF app, this patch needs review so I took a look at it. Currently, your patch applies and builds cleanly, and all tests are also successful based from the CF bot patch tester. I'm not sure if I have understood the argument raised by Peter correctly. Quoting Peter, "it's not

Re: \describe*

2019-01-24 Thread Corey Huinker
Attached is a patch to add verbose \describe commands to compliment our existing but slightly cryptic family of \d commands. The goals of this are: - aid user discovery of \d-commands via tab completion - make scripts and snippets slightly more self-documenting and understandable - save

RE: libpq debug log

2019-01-24 Thread Nagaura, Ryohei
Hi Iwata-san, I used your patch for my private work, so I write my opinion and four feedback below. On Fri, Jan 18, 2019 at 8:19 AM, Iwata, Aya wrote: > - Setting whether to get log or not by using connection strings or environment > variables. It means that application source code changes is

Re: Synchronous replay take III

2019-01-24 Thread Thomas Munro
On Tue, Jan 15, 2019 at 11:17 PM Masahiko Sawada wrote: > Regarding the current (v10 patch) design I have some questions and > comments. Hi Sawada-san, Thanks for your testing and feedback. > The patch introduces new GUC parameter synchronous_replay. We can set > synchronous_commit = off while

Re: monitoring CREATE INDEX [CONCURRENTLY]

2019-01-24 Thread Alvaro Herrera
On 2019-Jan-09, Pavan Deolasee wrote: > Looks like it's missing the validate_index blocks-scanned report, which is > not AM-specific and your original proposal does mention that. True. That phase is actually 3 phases, which would be reported separately: 5. index_bulk_delete() scan 6.

Re: [PATCH] Fix Proposal - Deadlock Issue in Single User Mode When IO Failure Occurs

2019-01-24 Thread Thomas Munro
On Sun, Jan 20, 2019 at 4:45 PM Amit Kapila wrote: > On Sat, Dec 1, 2018 at 2:30 AM Chengchao Yu wrote: > > Recently, we hit a few occurrences of deadlock when IO failure (including > > disk full, random remote disk IO failures) happens in single user mode. We > > found the issue exists on

Re: Old protocol fastpath calls borked?

2019-01-24 Thread Tom Lane
Andres Freund writes: > Ah, brainfade. Probably triggered by the fact that I forgot that we call > input functions even on NULL (which never quite made sense to me). That's so that domain_in can reject NULLs if the domain constraints say to do so. Mind you, the SQL committee should never have

Re: Old protocol fastpath calls borked?

2019-01-24 Thread Andres Freund
Hi, On 2019-01-24 17:04:32 -0500, Tom Lane wrote: > Andres Freund writes: > > In parse_fcall_arguments_20(): > > we appear to constantly setting argnull to true for all arguments? > > Uh, it looks to me like it does so only if the frontend sends a -1 > length field, which is the protocol's

Re: proposal - plpgsql unique statement id

2019-01-24 Thread Tom Lane
Peter Eisentraut writes: > committed Why didn't this patch modify the dumping logic in pl_funcs.c to print the IDs? I'm not aware of other cases where we intentionally omit fields from debug-support printouts. regards, tom lane

Re: Old protocol fastpath calls borked?

2019-01-24 Thread Tom Lane
Andres Freund writes: > In parse_fcall_arguments_20(): > we appear to constantly setting argnull to true for all arguments? Uh, it looks to me like it does so only if the frontend sends a -1 length field, which is the protocol's convention for indicating a null. regards,

Old protocol fastpath calls borked?

2019-01-24 Thread Andres Freund
Hi, In parse_fcall_arguments_20(): c0a8c3ac13f8 (Tom Lane 2003-05-08 18:16:37 + 579) argsize = pq_getmsgint(msgBuf, 4); 0ac6298bb8ac (Tom Lane 2003-05-09 18:08:48 + 580) if (argsize == -1) 0ac6298bb8ac (Tom Lane 2003-05-09 18:08:48 +

Re: Delay locking partitions during INSERT and UPDATE

2019-01-24 Thread Tomas Vondra
On 1/24/19 10:34 PM, John Naylor wrote: > On Thu, Jan 24, 2019 at 4:17 PM Tomas Vondra > wrote: >> I can still see about the same performance as before (on both clusters). > > Thanks for checking! I think the thing to do now is have a committer > look at it. It's a small patch with obvious

Re: Delay locking partitions during INSERT and UPDATE

2019-01-24 Thread John Naylor
On Thu, Jan 24, 2019 at 4:17 PM Tomas Vondra wrote: > I can still see about the same performance as before (on both clusters). Thanks for checking! I think the thing to do now is have a committer look at it. It's a small patch with obvious performance effects -- there's just the question of

Re: Almost bug in COPY FROM processing of GB18030 encoded input

2019-01-24 Thread Robert Haas
On Wed, Jan 23, 2019 at 6:23 AM Heikki Linnakangas wrote: > I happened to notice that when CopyReadLineText() calls mblen(), it > passes only the first byte of the multi-byte characters. However, > pg_gb18030_mblen() looks at the first and the second byte. > CopyReadLineText() always passes \0 as

Re: proposal - plpgsql unique statement id

2019-01-24 Thread Peter Eisentraut
On 22/01/2019 19:04, Pavel Stehule wrote: > It was missing, fixed, thank you for check committed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Log a sample of transactions

2019-01-24 Thread Robert Haas
On Fri, Jan 18, 2019 at 8:23 AM Adrien NAYRAT wrote: > On 1/18/19 9:03 AM, Peter Eisentraut wrote: > > But if you have trouble with a specific transaction, how will a setting > > help that randomly logs transactions, not necessarily the one you are > > concerned about? > > Yes, it assumes your

Re: Protect syscache from bloating with negative cache entries

2019-01-24 Thread Robert Haas
On Thu, Jan 24, 2019 at 10:02 AM Tom Lane wrote: > Bruce Momjian writes: > > On Thu, Jan 24, 2019 at 06:39:24PM +0900, Kyotaro HORIGUCHI wrote: > >> I also thought that there's some other features that is useful if > >> it could be turned on remotely so the remote GUC feature but it > >> was too

Re: Delay locking partitions during INSERT and UPDATE

2019-01-24 Thread Tomas Vondra
On 1/24/19 9:50 PM, John Naylor wrote: > On Wed, Jan 23, 2019 at 7:56 PM David Rowley > wrote: >> On Thu, 24 Jan 2019 at 13:38, John Naylor >> wrote: >>> Hmm, now instead of an 85x speedup over master in the 10k partition >>> case, I only get 20x. Anyone else see this? >> >> What's it like

Implicit timezone conversion replicating from timestamp to timestamptz?

2019-01-24 Thread Jeremy Finzel
We are working to migrate several large tables from the timestamp to the timestamptz data type by using logical replication (so as to avoid long downtime for type conversions). We are using pglogical but curious if what I share below applies to native logical replication as well. Both source and

Re: Delay locking partitions during INSERT and UPDATE

2019-01-24 Thread John Naylor
On Wed, Jan 23, 2019 at 7:56 PM David Rowley wrote: > On Thu, 24 Jan 2019 at 13:38, John Naylor wrote: > > Hmm, now instead of an 85x speedup over master in the 10k partition > > case, I only get 20x. Anyone else see this? > > What's it like with fsync off? No change. Just in case, I tried git

Re: WIP: Avoid creation of the free space map for small tables

2019-01-24 Thread John Naylor
On Wed, Jan 23, 2019 at 11:17 PM Amit Kapila wrote: > > On Thu, Jan 24, 2019 at 3:39 AM John Naylor > wrote: > > mkdir -p data1 data2 standby > > > > echo 'heap' > data1/foo > > echo 'fsm' > data1/foo_fsm > > > > # simulate streaming replication > > rsync --archive data1 standby > > > > #

Re: problems with foreign keys on partitioned tables

2019-01-24 Thread Alvaro Herrera
On 2019-Jan-24, Amit Langote wrote: > A few hunks of the originally proposed patch are attached here as 0001, > especially the part which fixes ATAddForeignKeyConstraint to pass the > correct value of connoninherit to CreateConstraintEntry (which should be > false for partitioned tables). With

Built-in connection pooler

2019-01-24 Thread Konstantin Knizhnik
Hi hacker, I am working for some time under built-in connection pooler for Postgres: https://www.postgresql.org/message-id/flat/a866346d-5582-e8e8-2492-fd32732b0783%40postgrespro.ru#b1c354d5cf937893a75954e1e77f81f5 Unlike existed external pooler, this solution supports session semantic for

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-24 Thread Alvaro Herrera
On 2019-Jan-24, Tom Lane wrote: > > Also, as > > the pseudo-random state is fully controlled, seeded test results are > > deterministic so the expected value can be fully checked. > > I found that the "expected value" was different in v11 than HEAD, > which surprised me. It looks like the

Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD

2019-01-24 Thread Tom Lane
Fabien COELHO writes: >> I had in mind something more like the attached. > Yep. > I'm not too happy that it mixes API levels, and about the int/double/int > path. > Attached an updated version which relies on pg_jrand48 instead. Hm, I'm not sure that's really an improvement, but I pushed it

Re: problems with foreign keys on partitioned tables

2019-01-24 Thread Alvaro Herrera
On 2019-Jan-25, Amit Langote wrote: > On Fri, Jan 25, 2019 at 12:30 AM Amit Langote wrote: > > Doesn't the following performDeletion() at the start of > > ATExecDropConstraint(), through findDependentObject()'s own recursion, > > take care of deleting *all* constraints, including those of? > >

Re: Thread-unsafe coding in ecpg

2019-01-24 Thread Andrew Dunstan
On 1/23/19 10:53 PM, Tom Lane wrote: > I wrote: >> This suggests that, rather than throwing up our hands if the initial >> _configthreadlocale call returns -1, we should act as though the function >> doesn't exist, and just soldier on the same as before. The code in there >> assumes that -1 is

Re: problems with foreign keys on partitioned tables

2019-01-24 Thread Amit Langote
On Fri, Jan 25, 2019 at 12:30 AM Amit Langote wrote: > > On Fri, Jan 25, 2019 at 12:08 AM Alvaro Herrera > wrote: > > On 2019-Jan-24, Amit Langote wrote: > > > > > Thinking more on this, my proposal to rip dependencies between parent and > > > child constraints altogether to resolve the bug I

Re: problems with foreign keys on partitioned tables

2019-01-24 Thread Amit Langote
On Fri, Jan 25, 2019 at 12:08 AM Alvaro Herrera wrote: > On 2019-Jan-24, Amit Langote wrote: > > > Thinking more on this, my proposal to rip dependencies between parent and > > child constraints altogether to resolve the bug I initially reported is > > starting to sound a bit overambitious

Re: problems with foreign keys on partitioned tables

2019-01-24 Thread Alvaro Herrera
Hello On 2019-Jan-24, Amit Langote wrote: > Thinking more on this, my proposal to rip dependencies between parent and > child constraints altogether to resolve the bug I initially reported is > starting to sound a bit overambitious especially considering that we'd > need to back-patch it (the

Re: Protect syscache from bloating with negative cache entries

2019-01-24 Thread Tom Lane
Bruce Momjian writes: > On Thu, Jan 24, 2019 at 06:39:24PM +0900, Kyotaro HORIGUCHI wrote: >> I also thought that there's some other features that is useful if >> it could be turned on remotely so the remote GUC feature but it >> was too complex... > Well, I am thinking if we want to do

Re: Protect syscache from bloating with negative cache entries

2019-01-24 Thread Bruce Momjian
On Thu, Jan 24, 2019 at 06:39:24PM +0900, Kyotaro HORIGUCHI wrote: > > Second, when would you use syscache_memory_target != 0? > > It is a suggestion upthread, we sometimes want to keep some known > amount of caches despite that expration should be activated. > > > If you had > >

Re: Use an enum for RELKIND_*?

2019-01-24 Thread Tom Lane
Kyotaro HORIGUCHI writes: > I might misunderstand something, but my compiler (gcc 7.3.1) > won't be quiet about omitted value even with default:. > ... I would call that a compiler bug, TBH. The code is 100% correct, if you intended to allow the default case to handle some enum values, which is

Re: pgsql: Detach constraints when partitions are detached

2019-01-24 Thread Alvaro Herrera
CC'ing -hackers. On 2019-Jan-24, Amit Langote wrote: > On 2019/01/24 13:58, Tom Lane wrote: > > Alvaro Herrera writes: > >> Detach constraints when partitions are detached > > > > Hm ... it looks like this fails under -DRELCACHE_FORCE_RELEASE: > > Oops, sorry. This is why: > >

Re: using expression syntax for partition bounds

2019-01-24 Thread Amit Langote
Hi, Thanks for looking. On 2019/01/24 21:00, Alvaro Herrera wrote: > Why did you lose the parser_errposition in parse_utilcmd.c line 3854? > >> -/* Fail if we don't have a constant (i.e., non-immutable coercion) */ >> -if (!IsA(value, Const)) >> +/* Make sure the expression does not

Re: Improve selectivity estimate for range queries

2019-01-24 Thread Kyotaro HORIGUCHI
Hi. At Fri, 11 Jan 2019 11:36:47 +0900, "Yuzuko Hosoya" wrote in <01d4a956$806a2ab0$813e8010$@lab.ntt.co.jp> > Hi, > > Thanks for the comments, and I'm sorry for the late reply. > > > From: Tom Lane [mailto:t...@sss.pgh.pa.us] > > Sent: Friday, January 11, 2019 7:04 AM > > > Robert Haas

Re: problems with foreign keys on partitioned tables

2019-01-24 Thread Amit Langote
Hi, On 2019/01/24 6:13, Alvaro Herrera wrote: > On 2019-Jan-22, Amit Langote wrote: >> Done. See the attached patches for HEAD and PG 11. > > I'm not quite sure I understand why the one in DefineIndex needs the > deps but nothing else does. I fear that you added that one just to > appease the

Re: ArchiveEntry optional arguments refactoring

2019-01-24 Thread Dmitry Dolgov
Here is another version, where I accumulated all the suggestions: * Use NULL as a default value where it was an empty string before (this required few minor changes for some part of the code outside ArchiveEntry) * Rename defn, descr to createStmt, description * Use a macro to avoid pgindent

Re: using expression syntax for partition bounds

2019-01-24 Thread Alvaro Herrera
Why did you lose the parser_errposition in parse_utilcmd.c line 3854? > - /* Fail if we don't have a constant (i.e., non-immutable coercion) */ > - if (!IsA(value, Const)) > + /* Make sure the expression does not refer to any vars. */ > + if (contain_var_clause(value)) >

Re: Simplify set of flags used by MyXactFlags

2019-01-24 Thread Alvaro Herrera
On 2019-Jan-22, Michael Paquier wrote: > On Mon, Jan 21, 2019 at 02:58:39PM -0300, Alvaro Herrera wrote: > > "... operated on temp namespace" doesn't look good; seems to me to be > > missing an article, for one thing, but really I'm not sure that > > 'namespace' is the term to be using here. I'd

Re: Connection slots reserved for replication

2019-01-24 Thread Kyotaro HORIGUCHI
Hello. Documentation looks fine for me. By the way, a comment for the caller-site of CheckRequreParameterValues() in xlog.c looks somewhat stale. > /* Check to see if any changes to max_connections give problems */ > CheckRequiredParameterValues(); may be better something like this?: > /*

Re: Use an enum for RELKIND_*?

2019-01-24 Thread Greg Stark
On Wed 16 Jan 2019, 18:10 Alvaro Herrera > Or we could do nothing, since there are no complaints about this > problem. > Unless there are objections my patch is to some add it to the xid case. The tag is pretty useless for users and it's incredibly hard to even see these rows anyways I'll take

Re: Use an enum for RELKIND_*?

2019-01-24 Thread Kyotaro HORIGUCHI
At Wed, 19 Dec 2018 09:55:22 -0500, Tom Lane wrote in <10268.1545231...@sss.pgh.pa.us> > Andres Freund writes: > > On 2018-12-18 23:17:54 -0500, Tom Lane wrote: > >> Yeah, that would sure make things better. I was considering > >> something like > >> > >>#ifndef USE_ASSERT_CHECKING > >>

Re: using expression syntax for partition bounds

2019-01-24 Thread Amit Langote
Horiguchi-san, Thanks for checking. On 2019/01/24 19:03, Kyotaro HORIGUCHI wrote: > At Fri, 18 Jan 2019 17:50:56 +0900, Amit Langote wrote: >> On 2019/01/18 16:48, Peter Eisentraut wrote: How about the following note in the documentation: + Although volatile expressions such

Re: WIP: Avoid creation of the free space map for small tables

2019-01-24 Thread Amit Kapila
On Thu, Jan 24, 2019 at 9:46 AM Amit Kapila wrote: > > On Thu, Jan 24, 2019 at 3:39 AM John Naylor > wrote: > > Few comments related to pg_upgrade patch: 1. + if ((maps[mapnum].relkind != RELKIND_RELATION && + maps[mapnum].relkind != RELKIND_TOASTVALUE) || + first_seg_size >

Re: using expression syntax for partition bounds

2019-01-24 Thread Kyotaro HORIGUCHI
Hello. At Fri, 18 Jan 2019 17:50:56 +0900, Amit Langote wrote in > Thanks for the comments. > > On 2019/01/18 16:48, Peter Eisentraut wrote: > >> How about the following note in the documentation: > >> > >> + Although volatile expressions such as > >> + CURRENT_TIMESTAMP can be >

Re: FETCH FIRST clause PERCENT option

2019-01-24 Thread Surafel Temesgen
On Wed, Jan 9, 2019 at 8:18 PM Tomas Vondra wrote: > > See the attached patch, which recomputes the count regularly. I don't > claim the patch is committable or that it has no other bugs, but > hopefully it shows what I meant. > > I got only one issue it is not work well with cursor

Re: Protect syscache from bloating with negative cache entries

2019-01-24 Thread Kyotaro HORIGUCHI
Thank you for the comments. At Wed, 23 Jan 2019 18:21:45 -0500, Bruce Momjian wrote in <20190123232145.ga8...@momjian.us> > On Wed, Jan 23, 2019 at 05:35:02PM +0900, Kyotaro HORIGUCHI wrote: > > At Mon, 21 Jan 2019 17:22:55 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI > > wrote in > >

Re: RTLD_GLOBAL (& JIT inlining)

2019-01-24 Thread Kyotaro HORIGUCHI
At Thu, 24 Jan 2019 01:02:14 -0500, Tom Lane wrote in <24744.1548309...@sss.pgh.pa.us> > Kyotaro HORIGUCHI writes: > > With the attached patch, external modules are loaded RTLD_LOCAL > > by default. PG_MODULE_EXPORT_SYMBOL in a module source files let > > it loaded RTLD_GLOBAL. I suppose this