Re: proposal: plpgsql pragma statement

2018-12-07 Thread Pavel Stehule
pá 7. 12. 2018 v 21:28 odesílatel Robert Haas napsal: > On Thu, Dec 6, 2018 at 12:28 PM Pavel Stehule > wrote: > > please, can you propose, some what you like? > > > > For my purpose I can imagine PRAGMA on function level with same syntax > like PL/SQL - I need to push somewhere some information

Re: pg_partition_tree crashes for a non-defined relation

2018-12-07 Thread Michael Paquier
On Fri, Dec 07, 2018 at 11:33:32PM -0500, Tom Lane wrote: > How about cases where the relation OID exists but it's the wrong > kind of relation? Such cases already return an error: =# create sequence popo; CREATE SEQUENCE =# select pg_partition_tree('popo'); ERROR: 42809: "popo" is not a table, a

Re: make install getting slower

2018-12-07 Thread Peter Eisentraut
On 07/12/2018 18:34, Andres Freund wrote: >> But I think that we realistically could generate ninja files using some >> Perl code not unlike the one that does the catalog .dat to .bki >> transformation. Just a thought at this point. > That seems not great, because we would lose the ability to gene

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2018-12-07 Thread Peter Eisentraut
On 07/12/2018 17:40, Sergei Kornilov wrote: > I perform some tests and think behavior with partition tables is slightly > inconsistent. > > postgres=# reindex table measurement; > WARNING: REINDEX of partitioned tables is not yet implemented, skipping > "measurement" > NOTICE: table "measureme

Re: pg_partition_tree crashes for a non-defined relation

2018-12-07 Thread Tom Lane
Michael Paquier writes: > On Fri, Dec 07, 2018 at 10:04:06AM +0900, Michael Paquier wrote: >> I think that we should make the function return NULL if the relation >> defined does not exist, as we usually do for system-facing functions. >> It is also easier for the caller to know that the relation

Re: Statement-level rollback

2018-12-07 Thread Robert Haas
On Fri, Dec 7, 2018 at 3:50 PM Alvaro Herrera wrote: > On 2018-Dec-07, Robert Haas wrote: > > More generally, whether or not we should "keep something away from our > > users" really depends on how likely the upsides are to occur relative > > to the downsides. We don't try to keep users from runn

Re: extended query protcol violation?

2018-12-07 Thread Tom Lane
Tatsuo Ishii writes: > While looking into an issue of Pgpool-II, I found an interesting > behavior of a PostgreSQL client. > Below is a trace from pgproto to reproduce the client's behavior. > It starts a transacton. > FE=> Parse(stmt="S1", query="BEGIN") > FE=> Bind(stmt="S1", portal="") > FE=>

Re: Statement-level rollback

2018-12-07 Thread Robert Haas
On Fri, Dec 7, 2018 at 7:25 PM Alexander Korotkov wrote: > The first thing, which comes to the mind is undo log. When you have > undo log, then on new subtransaction you basically memorize the > current undo log position. If subtransaction rollbacks, you have to > just play back undo log until r

Re: [HACKERS] Bug when dumping "empty" operator classes

2018-12-07 Thread Michael Paquier
On Fri, Dec 07, 2018 at 08:11:42PM -0500, Stephen Frost wrote: > Another blast from the past. I've updated this to the new pg_dump > test-suite system but otherwise it's basically the same test. > > Barring concerns, I'll plan to push this later this weekend. Thanks Stephen for including this on

Re: pg_partition_tree crashes for a non-defined relation

2018-12-07 Thread Michael Paquier
On Sat, Dec 08, 2018 at 12:28:53PM +0900, Amit Langote wrote: > Thanks for noticing it and creating the patch. The fix makes sense. Thanks a lot for looking at it! -- Michael signature.asc Description: PGP signature

Re: pg_partition_tree crashes for a non-defined relation

2018-12-07 Thread Amit Langote
Hi, Sorry for not replying sooner. On Sat, Dec 8, 2018 at 8:06 Michael Paquier wrote: > On Fri, Dec 07, 2018 at 10:04:06AM +0900, Michael Paquier wrote: > > While testing another patch, I have bumped into the issue of > > $subject... I should have put some more negative testing from the start

Re: Strange OSX make check-world failure

2018-12-07 Thread Samuel Cochran
On Fri, Dec 7, 2018, at 5:26 PM, Tom Lane wrote: > Interesting proposal, but I think it needs work. Absolutely! I only hacked it together to the point that it worked on my laptop and illustrated the approach. :-) > * As coded, this only fixes the problem for references to libpq, not > any of our

Re: Fixing findDependentObjects()'s dependency on scan order (regressions in DROP diagnostic messages)

2018-12-07 Thread Peter Geoghegan
On Thu, Dec 6, 2018 at 8:52 PM Andrey Lepikhov wrote: > I want to say that we need to localize the rules for the order of the > diagnostic messages as much as possible in dependency.c. But the issue *isn't* confined to dependency.c, anyway. It bleeds into a couple of other modules, like extension

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-12-07 Thread Alexander Korotkov
On Fri, Dec 7, 2018 at 12:50 AM Peter Geoghegan wrote: > On Thu, Dec 6, 2018 at 12:51 PM Alexander Korotkov > wrote: > > So, algorithm introduced by 218f51584d5 is broken. It tries to > > guarantee that there are no inserters in the subtree by placing > > cleanup lock to subtree root, assuming

extended query protcol violation?

2018-12-07 Thread Tatsuo Ishii
While looking into an issue of Pgpool-II, I found an interesting behavior of a PostgreSQL client. Below is a trace from pgproto to reproduce the client's behavior. It starts a transacton. FE=> Parse(stmt="S1", query="BEGIN") FE=> Bind(stmt="S1", portal="") FE=> Execute(portal="") : Commit the tra

Re: [HACKERS] Bug when dumping "empty" operator classes

2018-12-07 Thread Stephen Frost
Greetings, * Michael Paquier (michael.paqu...@gmail.com) wrote: > On Fri, May 26, 2017 at 8:14 AM, Daniel Gustafsson wrote: > >> On 26 May 2017, at 17:08, Tom Lane wrote: > >> I'll commit and back-patch this without a test case. Possibly Frost will > >> be excited enough about it to add somethi

Re: Statement-level rollback

2018-12-07 Thread Alexander Korotkov
On Fri, Dec 7, 2018 at 11:34 PM Alvaro Herrera wrote: > On 2018-Dec-07, Robert Haas wrote: > > Full disclosure: EDB has a feature like this and has for years, but it > > uses a subtransaction per statement, not a subtransaction per row. > > Well, this implementation only uses one subtransaction pe

Re: pg_dump emits ALTER TABLE ONLY partitioned_table

2018-12-07 Thread Stephen Frost
Greetings, * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > On 2017/05/08 12:42, Stephen Frost wrote: > > * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > >> Thanks for committing the patch after improving it quite a bit, and sorry > >> that I couldn't reply promptly during the last

Dead code in toast_fetch_datum_slice?

2018-12-07 Thread Stephen Frost
Greetings, Perhaps I'm missing something, but in toast_fetch_datum_slice() there's: Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer)); Followed, not long after, by: if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer)) SET_VARSIZE_COMPRESSED(result, length + VARHDRSZ); e

Re: Thinking about EXPLAIN ALTER TABLE

2018-12-07 Thread John Naylor
On 12/7/18, Greg Stark wrote: > I've been poking around with a feature I've wanted a number of times > in the past, "EXPLAIN ALTER TABLE". I believe I've seen your messages to that effect in the archives, so I've had it in the back of my mind as well. I think it could be very useful. > 3. Whethe

Re: pg_partition_tree crashes for a non-defined relation

2018-12-07 Thread Michael Paquier
On Fri, Dec 07, 2018 at 10:04:06AM +0900, Michael Paquier wrote: > While testing another patch, I have bumped into the issue of > $subject... I should have put some more negative testing from the start > on this stuff, here is a culprit query when passing directly an OID: > select pg_partition_tre

Re: slow queries over information schema.tables

2018-12-07 Thread Tom Lane
Robert Haas writes: > On Thu, Dec 6, 2018 at 12:50 PM Tom Lane wrote: >> No. You need to do AIM *after* obtaining the lock, else you still >> have the race condition that you can execute a query on a table >> without being aware of recent DDL on it. > Huh? The call in relation_openrv_extended()

Re: Thinking about EXPLAIN ALTER TABLE

2018-12-07 Thread Peter Geoghegan
On Fri, Dec 7, 2018 at 1:18 PM Greg Stark wrote: > I've been poking around with a feature I've wanted a number of times > in the past, "EXPLAIN ALTER TABLE". The idea is that there are a bunch > of optimizations in ALTER TABLE to minimize the amount of work and > lock levels but it's really hard f

Thinking about EXPLAIN ALTER TABLE

2018-12-07 Thread Greg Stark
I've been poking around with a feature I've wanted a number of times in the past, "EXPLAIN ALTER TABLE". The idea is that there are a bunch of optimizations in ALTER TABLE to minimize the amount of work and lock levels but it's really hard for users to tell whether they've written their ALTER TABLE

Re: Statement-level rollback

2018-12-07 Thread Andres Freund
Hi, On 2018-12-07 16:02:53 -0500, Tom Lane wrote: > Alvaro Herrera writes: > > Well, look at this from this point of view: EnterpriseDB implemented > > this because of customer demand (presumably). Fujitsu also implemented > > this for customers. The pgjdbc driver implemented this for its users

Re: Statement-level rollback

2018-12-07 Thread Tom Lane
Alvaro Herrera writes: > Well, look at this from this point of view: EnterpriseDB implemented > this because of customer demand (presumably). Fujitsu also implemented > this for customers. The pgjdbc driver implemented this for its users. > Now 2ndQuadrant also implemented this, and not out of t

Re: Statement-level rollback

2018-12-07 Thread Tom Lane
Robert Haas writes: > I have a hard time arguing against that given that EDB has this thing > in our bag of tricks, but if it weren't for that I'd be fighting > against this tooth and nail. Behavior-changing GUCs sck. Uh, we're not seriously considering a GUC that changes transactional behav

Re: Statement-level rollback

2018-12-07 Thread Joshua D. Drake
On 12/7/18 12:50 PM, Alvaro Herrera wrote: On 2018-Dec-07, Robert Haas wrote: More generally, whether or not we should "keep something away from our users" really depends on how likely the upsides are to occur relative to the downsides. We don't try to keep users from running DELETE because th

Re: Statement-level rollback

2018-12-07 Thread Alvaro Herrera
On 2018-Dec-07, Robert Haas wrote: > More generally, whether or not we should "keep something away from our > users" really depends on how likely the upsides are to occur relative > to the downsides. We don't try to keep users from running DELETE > because they might delete data they want; that w

Re: Statement-level rollback

2018-12-07 Thread Robert Haas
On Fri, Dec 7, 2018 at 3:34 PM Alvaro Herrera wrote: > Yeah, I agree that this downside is real. I think our only protection > against that is to say "don't do that". Like any other tool, it has > upsides and downsides; we shouldn't keep it away from users only because > they might misuse it. I

Re: Statement-level rollback

2018-12-07 Thread Alvaro Herrera
On 2018-Dec-07, Robert Haas wrote: > Full disclosure: EDB has a feature like this and has for years, but it > uses a subtransaction per statement, not a subtransaction per row. Well, this implementation only uses one subtransaction per statement; Andres says per-row referring to the case of one I

Re: rewrite ExecPartitionCheckEmitError

2018-12-07 Thread Robert Haas
On Thu, Dec 6, 2018 at 5:22 PM Alvaro Herrera wrote: > Just on cleanliness grounds, I propose to rewrite the function in > $SUBJECT. I came across this while reviewing some already-committed > patch for partition pruning, and it's been sitting in my laptop ever > since. > > I think the current co

Re: proposal: plpgsql pragma statement

2018-12-07 Thread Robert Haas
On Thu, Dec 6, 2018 at 12:28 PM Pavel Stehule wrote: > please, can you propose, some what you like? > > For my purpose I can imagine PRAGMA on function level with same syntax like > PL/SQL - I need to push somewhere some information that I can use for > plpgsql_check to protect users against fal

Re: slow queries over information schema.tables

2018-12-07 Thread Robert Haas
On Thu, Dec 6, 2018 at 12:50 PM Tom Lane wrote: > > If we called it at the start of every query, couldn't we dispense with > > the call in relation_openrv_extended()? > > No. You need to do AIM *after* obtaining the lock, else you still > have the race condition that you can execute a query on a

Re: Statement-level rollback

2018-12-07 Thread Robert Haas
On Fri, Dec 7, 2018 at 2:57 PM Alvaro Herrera wrote: > The way I envision this to be used in practice is that it's enabled > globally when the migration effort starts, then gradually disabled as > parts of applications affected by these downsides are taught how to deal > with the other behavior.

Re: Statement-level rollback

2018-12-07 Thread Alvaro Herrera
On 2018-Dec-07, Andres Freund wrote: > On December 7, 2018 11:56:55 AM PST, Alvaro Herrera > wrote: > >BTW, a couple of months ago I measured the performance implications for > >a single update under pgbench and it represented a decrease of about > >3%-5%. Side-effects such as xid consumption h

Re: Statement-level rollback

2018-12-07 Thread Andres Freund
On December 7, 2018 11:56:55 AM PST, Alvaro Herrera wrote: >BTW, a couple of months ago I measured the performance implications for >a single update under pgbench and it represented a decrease of about >3%-5%. Side-effects such as xid consumption have worse implications, >but as far as perfor

Re: Statement-level rollback

2018-12-07 Thread Alvaro Herrera
Hi On 2018-Dec-07, Andres Freund wrote: > Isn't the realistic scenario for migrations that people will turn this > feature on on a more global basis? If they need to do per transaction > choices, that makes this much less useful for easy migrations. The way I envision this to be used in practice

Re: Statement-level rollback

2018-12-07 Thread Andres Freund
Hi, On December 7, 2018 11:44:17 AM PST, Alvaro Herrera wrote: >On 2018-Dec-07, Andres Freund wrote: > >> Hi, >> >> On 2018-12-07 16:20:06 -0300, Alvaro Herrera wrote: >> >case TBLOCK_BEGIN: >> > + s->rollbackScope = XactRollbackScope; >> >s->blo

Re: Statement-level rollback

2018-12-07 Thread Alvaro Herrera
On 2018-Dec-07, Andres Freund wrote: > Hi, > > On 2018-12-07 16:20:06 -0300, Alvaro Herrera wrote: > > case TBLOCK_BEGIN: > > + s->rollbackScope = XactRollbackScope; > > s->blockState = TBLOCK_INPROGRESS; > > + if (s->rollbackSco

Re: Statement-level rollback

2018-12-07 Thread Andres Freund
Hi, On 2018-12-07 16:20:06 -0300, Alvaro Herrera wrote: > case TBLOCK_BEGIN: > + s->rollbackScope = XactRollbackScope; > s->blockState = TBLOCK_INPROGRESS; > + if (s->rollbackScope == XACT_ROLLBACK_SCOPE_STMT) > +

Statement-level rollback

2018-12-07 Thread Alvaro Herrera
I would like to bring up again the topic of statement-level rollback. This was discussed in some depth at [1]. This patch is not based on Tsunakawa-san's patch submitted in that thread; although I started from it, I eventually removed almost everything and replaced with a completely different impl

Re: Alter table documentation page (again)

2018-12-07 Thread Lætitia Avrot
Hi all, Here's the patch. The patch should apply to MASTER. I built and tested it successfully on my laptop. I'll add it to January's commitfest. Cheers, Lætitia diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index be1647937d..c49a20fb93 100644 --- a/doc/src/

Re: BUG #15541: Use after release in PQprint

2018-12-07 Thread Tom Lane
Alvaro Herrera writes: > I wonder why don't we just remove PQprint altogether. Yeah, I was wondering that too. AFAICS the whole of fe-print.c is dead code so far as our own usage is concerned. Is there any evidence that outside code is using PQprint, PQprintTuples, or PQdisplayTuples? (The lat

Re: make install getting slower

2018-12-07 Thread Andres Freund
On December 7, 2018 8:04:10 AM PST, Peter Eisentraut wrote: >On 04/12/2018 21:56, Andres Freund wrote: >> As a comparison, I'd a tree that had the cmake patchset applied >around >> (~1.5 yo tree). Using the ninja generator gets a clean build to >> 0m0.073s, a first install to 0m0.201s and a re

Re: Adding support for a fully qualified column-name in UPDATE ... SET

2018-12-07 Thread Tom Lane
Jim Finnerty writes: > The accepted syntax for UPDATE ... SET does not currently permit the column > name to be qualified by schema.table or table or correlation-name, as is > permitted in other systems. This is apparently due to the syntax that > PostgreSQL accepts for composite columns, which w

Re: additional foreign key test coverage

2018-12-07 Thread Alvaro Herrera
On 2018-Dec-07, Peter Eisentraut wrote: > On 04/12/2018 14:23, Alvaro Herrera wrote: > > Hmm. One of the things I did for FKs on partitioned tables was remove > > all the cases involving only unpartitioned tables, then run just the > > foreign_key test and see what the coverage looked like -- in

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2018-12-07 Thread Sergei Kornilov
Hello Thank you for working on this patch! I perform some tests and think behavior with partition tables is slightly inconsistent. postgres=# reindex table measurement; WARNING: REINDEX of partitioned tables is not yet implemented, skipping "measurement" NOTICE: table "measurement" has no in

nice doc of PostgreSQL hooks

2018-12-07 Thread Pavel Stehule
Hi this docs looks well https://github.com/AmatanHead/psql-hooks Regards Pavel

Re: make install getting slower

2018-12-07 Thread Peter Eisentraut
On 04/12/2018 21:56, Andres Freund wrote: > As a comparison, I'd a tree that had the cmake patchset applied around > (~1.5 yo tree). Using the ninja generator gets a clean build to > 0m0.073s, a first install to 0m0.201s and a repeat install to 0m0.170s. Yeah, I've been playing with meson+ninja, a

Re: make install getting slower

2018-12-07 Thread Peter Eisentraut
On 04/12/2018 21:31, Andres Freund wrote: > Obviously this doesn't terribly matter for actual installations, but I > do find it fairly annoying when running regression tests... One possible workaround: https://www.postgresql.org/message-id/5d14d20c-b1f6-69c8-7465-df8adb424...@2ndquadrant.com --

Re: additional foreign key test coverage

2018-12-07 Thread Peter Eisentraut
On 04/12/2018 14:23, Alvaro Herrera wrote: > On 2018-Dec-04, Peter Eisentraut wrote: > >> During the development of my recent patch "unused/redundant foreign key >> code" [0], I had developed a few additional test cases to increase the >> coverage in ri_triggers.c. They are in the attached patche

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2018-12-07 Thread Peter Eisentraut
Here is a revival of this patch. This is Andreas Karlsson's v4 patch (2017-11-01) with some updates for conflicts and changed APIs. AFAICT from the discussions, there were no more conceptual concerns with this approach. Recall that with this patch REINDEX CONCURRENTLY creates a new index (with a

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-12-07 Thread Alexander Korotkov
On Fri, Dec 7, 2018 at 12:14 PM Andrey Borodin wrote: > > 7 дек. 2018 г., в 2:50, Peter Geoghegan написал(а): > > On Thu, Dec 6, 2018 at 12:51 PM Alexander Korotkov > > wrote: > >> However, I'd like to note that 218f51584d5 introduces two changes: > >> 1) Cleanup locking only if there pages to

Re: WIP: Avoid creation of the free space map for small tables

2018-12-07 Thread John Naylor
On 12/6/18, Amit Kapila wrote: > On Thu, Dec 6, 2018 at 10:53 PM John Naylor wrote: >> >> I've added an additional regression test for finding the right block >> and removed a test I thought was redundant. I've kept the test file in >> its own schedule. >> > > +# -- > +# fsm does a vacuum

Re: Log level of logical decoding

2018-12-07 Thread Francesco Nidito
Hi All, One week ago I posted the following question in the general list but probably the developers' list is a better place for such question. Thanks in advance (again)! Francesco From: Francesco Nidito Sent: 30 November 2018 15:10 To: pgsql-gene...@lists.post

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2018-12-07 Thread Ashutosh Bapat
On Fri, Dec 7, 2018 at 11:13 AM Ashutosh Bapat wrote: > > > >> >> >> Robert, Ashutosh, any comments on this? I'm unfamiliar with the >> partitionwise join code. >> > > As the comment says it has to do with the equivalence classes being used > during merge append. EC's are used to create pathkeys

RE: [PROPOSAL]a new data type 'bytea' for ECPG

2018-12-07 Thread Matsumura, Ryo
Meskes-san Tnak you for your comment. > I do think, though, we should change the debug output for > ecpg_free_params(). I try to change about it. Next patch will print binary in hex-format. > > > The patch does not support ECPG.bytea in sqltype of "struct > > > sqlvar_struct" > > > because of

Re: Too many logs are written on Windows (LOG: could not reserve shared memory region (addr=%p) for child %p:)

2018-12-07 Thread Magnus Hagander
On Thu, Dec 6, 2018 at 7:04 AM Takahashi, Ryohei < r.takahash...@jp.fujitsu.com> wrote: > Hi, > > > I found the reason of the message. > > My customer uses "F-secure" antivirus software. > There are several pages that indicate F-secure causes this message such as > [1]. > I told my customer to sto

Re: Connections hang indefinitely while taking a gin index's LWLock buffer_content lock

2018-12-07 Thread Andrey Borodin
Hi! > 7 дек. 2018 г., в 2:50, Peter Geoghegan написал(а): > > On Thu, Dec 6, 2018 at 12:51 PM Alexander Korotkov > wrote: >> >> However, I'd like to note that 218f51584d5 introduces two changes: >> 1) Cleanup locking only if there pages to delete >> 2) Cleanup locking only subtree root >> The