Re: ToDo: show size of partitioned table

2019-04-02 Thread Amit Langote
Hi Alvaro, On 2019/04/03 4:42, Alvaro Herrera wrote: > On 2019-Apr-02, Alvaro Herrera wrote: > >> Something is not right: > > Another thing that was not right is that translated_columns was being > marked static, and modified in the function; so if you called \dP twice > where one called for tra

Re: Problem with default partition pruning

2019-04-02 Thread Amit Langote
Hosoya-san, On 2019/04/02 14:02, Yuzuko Hosoya wrote: > Hi, > >> Maybe we should have two patches as we seem to be improving two things: >> >> 1. Patch to fix problems with default partition pruning originally reported >> by Hosoya-san >> >> 2. Patch to determine if a given clause contradicts a

Re: ToDo: show size of partitioned table

2019-04-02 Thread Amit Langote
Horiguchi-san, Thanks for taking a look. On 2019/04/03 12:02, Kyotaro HORIGUCHI wrote: > \dPn doesn't show children because the are of 'r' relkind. And > \dPn * doesn't show type for children. > > create table pa (a int, b int) partition by range (a); > create table c1 partition of pa for values

Re: [HACKERS] generated columns

2019-04-02 Thread Amit Langote
On 2019/04/03 3:54, Erik Rijkers wrote: > create schema if not exists "tmp"; > CREATE SCHEMA > create server if not exists "tmpserver" foreign data wrapper file_fdw; > CREATE SERVER > drop   foreign table if exists tmp.pg_head cascade; > DROP FOREIGN TABLE > create foreign table   tmp.pg_he

Re: partitioned tables referenced by FKs

2019-04-03 Thread Amit Langote
On 2019/04/04 4:45, Jesper Pedersen wrote: > On 4/3/19 1:52 PM, Alvaro Herrera wrote: >> Pushed, many thanks Amit and Jesper for reviewing. >> > > Thank you for working on this feature. +1, thank you. Regards, Amit

Re: Inadequate executor locking of indexes

2019-04-03 Thread Amit Langote
Hi David, Thanks for updating the patch. On 2019/04/04 9:14, David Rowley wrote: > I also looked over other index_open() calls in the planner and found a > bunch of places in selfuncs.c that we open an index to grab some > information then close it again releasing the lock. At this stage > get_r

Re: Inadequate executor locking of indexes

2019-04-03 Thread Amit Langote
On 2019/04/04 11:13, David Rowley wrote: > On Thu, 4 Apr 2019 at 15:01, Amit Langote > wrote: >> Sorry, I didn't understand why it wouldn't be OK to pass NoLock to >> index_open, for example, here: >> >> @@ -5191,7 +5191,14 @@ get_actual_variable_rang

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-03 Thread Amit Langote
Hi, On 2019/04/04 13:37, Tsunakawa, Takayuki wrote: > Hi Peter, > > From: Peter Eisentraut [mailto:peter.eisentr...@2ndquadrant.com] >> I did a bit of performance testing, both a plain pgbench and the >> suggested test case with 4096 partitions. I can't detect any >> performance improvements. I

Re: [HACKERS] generated columns

2019-04-04 Thread Amit Langote
On 2019/04/04 16:52, Peter Eisentraut wrote: > On 2019-04-02 20:54, Erik Rijkers wrote: >> attached is run_ft.sh which creates a text file: /tmp/pg_head.txt >> then sets it up as a foreign table, and adds a generated column. >> >> Then selects a succesful select, followed by a error-producing sel

Re: ToDo: show size of partitioned table

2019-04-04 Thread Amit Langote
Hi Alvaro, On 2019/04/03 21:01, Alvaro Herrera wrote: > On 2019-Apr-03, Kyotaro HORIGUCHI wrote: > >> Hello. >> >> At Wed, 3 Apr 2019 12:55:06 +0900, Amit Langote >> wrote in >> >>> On 2019/04/03 12:02, Kyotaro HORIGUCHI wrote: >>>>

Re: [HACKERS] generated columns

2019-04-04 Thread Amit Langote
On Thu, Apr 4, 2019 at 8:01 PM Peter Eisentraut wrote: > > On 2019-04-04 11:42, Amit Langote wrote: > > Hmm, I'm afraid we might get bug reports if we go with this. Why is it OK > > to get null in this case when a user explicitly asked for 'foo'? > > T

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2019-04-04 Thread Amit Langote
On 2019/04/05 5:42, Peter Eisentraut wrote: > On 2019-04-04 06:58, Amit Langote wrote: >> Also, since the "speed up partition planning" patch went in (428b260f8), >> it might be possible to see the performance boost even with the >> partitioning example you cited up

Re: speeding up planning with partitions

2019-04-04 Thread Amit Langote
On 2019/04/05 6:59, David Rowley wrote: > On Fri, 5 Apr 2019 at 07:33, Floris Van Nee wrote: >> I had a question about the performance of pruning of functions like now() >> and current_date. I know these are handled differently, as they cannot be >> excluded during the first phases of planning.

Re: speeding up planning with partitions

2019-04-04 Thread Amit Langote
On 2019/04/05 12:18, David Rowley wrote: > On Fri, 5 Apr 2019 at 16:09, Amit Langote > wrote: >> Although, we still have ways >> to go in terms of scaling generic plan execution to larger partition >> counts, solution(s) for which have been proposed by David but haven&

Re: Why does "toast_tuple_target" allow values below TOAST_TUPLE_TARGET?

2019-04-04 Thread Amit Langote
On 2019/04/05 14:09, David Rowley wrote: > Or rather, why does the reloption allow values below the compile-time > constant? Maybe there is already a discussion in progress on the topic? * Caveats from reloption toast_tuple_target * https://www.postgresql.org/message-id/flat/CABOikdMt%3DmOtzW_ax

Re: speeding up planning with partitions

2019-04-04 Thread Amit Langote
On 2019/04/02 14:50, Amit Langote wrote: > Attached patch is only for HEAD this time. I'll post one for PG 11 (if > you'd like) once we reach consensus on the best thing to do here is. While we're on the topic of the relation between constraint exclusion and partition pru

Re: Problem with default partition pruning

2019-04-05 Thread Amit Langote
Hosoya-san, On 2019/04/04 13:00, Yuzuko Hosoya wrote: > I added some test cases to each patch according to tests > discussed in this thread. Thanks a lot. > However, I found another problem as follows. This query should > output "One-Time Filter: false" because rlp3's constraints > contradic

Re: speeding up planning with partitions

2019-04-05 Thread Amit Langote
Hi, On 2019/04/05 18:13, Floris Van Nee wrote: > One unrelated thing I noticed (but I'm not sure if it's worth a separate > email thread) is that the changed default of jit=on in v12 doesn't work very > well with a large number of partitions at run-time, for which a large number > get excluded

Re: Back-branch bugs with fully-prunable UPDATEs

2019-04-07 Thread Amit Langote
y contain the root partitioned table in some cases, which breaks that assumption. I've attached fixes for PG 10 and 11, modifying ExecInitModifyTable() and inheritance_planner(), respectively. > This seems to be related to what Amit Langote complained of in > https://www.postgresql.org/m

Re: Back-branch bugs with fully-prunable UPDATEs

2019-04-07 Thread Amit Langote
On 2019/04/08 1:57, Tom Lane wrote: > Amit Langote writes: >> On Sun, Apr 7, 2019 at 5:28 AM Tom Lane wrote: >>> This test script works fine in HEAD: >>> In v11, it suffers an assertion failure in ExecSetupPartitionTupleRouting. >>> In v10, it doesn't cr

Re: selecting from partitions and constraint exclusion

2019-04-07 Thread Amit Langote
Hi Thibaut, On 2019/04/06 1:12, Thibaut wrote: > Le 25/03/2019 à 01:31, Amit Langote a écrit : >> On 2019/03/22 17:17, Amit Langote wrote: >>> I'll add this to July fest to avoid forgetting about this. >> I'd forgotten to do this, but done today. :) >

Re: Ordered Partitioned Table Scans

2019-04-07 Thread Amit Langote
On 2019/04/06 18:06, Julien Rouhaud wrote: > On Sat, Apr 6, 2019 at 2:45 AM David Rowley > wrote: >> >> On Sat, 6 Apr 2019 at 12:26, Tom Lane wrote: >>> Pushed with some hacking, mostly trying to improve the comments. >> >> Great! Many thanks for improving those and pushing it. >> >> Many thanks

Re: ToDo: show size of partitioned table

2019-04-07 Thread Amit Langote
Hi, On 2019/04/08 12:59, Pavel Stehule wrote: > ne 7. 4. 2019 v 21:13 odesílatel Alvaro Herrera > napsal: >> Added to the docs, and pushed. >> > > Thank you very much Thank you Alvaro for pushing this to completion. Also, thank you Justin and Pavel for reviewing it till the last minute. Looks

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-08 Thread Amit Langote
On 2019/03/14 10:40, Amit Langote wrote: > On 2019/03/14 5:18, Tom Lane wrote: >> Robert Haas writes: >>> On Wed, Mar 13, 2019 at 3:14 PM Tom Lane wrote: >>>> Meanwhile, who's going to take point on cleaning up rd_partcheck? >>>> I don't rea

Re: [Sender Address Forgery]Re: [HACKERS] generated columns

2019-04-08 Thread Amit Langote
On 2019/04/08 20:50, Peter Eisentraut wrote: > On 2019-04-04 16:37, Amit Langote wrote: >> OK, thanks for explaining. We do allow DEFAULT to be specified on >> foreign tables, although locally-defined defaults have little meaning >> if the FDW doesn't allow inserts. M

Re: COLLATE: Hash partition vs UPDATE

2019-04-08 Thread Amit Langote
Hi Jesper, On 2019/04/09 1:33, Jesper Pedersen wrote: > Hi, > > The following case > > -- test.sql -- > CREATE TABLE test (a text PRIMARY KEY, b text) PARTITION BY HASH (a); > CREATE TABLE test_p0 PARTITION OF test FOR VALUES WITH (MODULUS 2, > REMAINDER 0); > CREATE TABLE test_p1 PARTITION OF t

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-08 Thread Amit Langote
On 2019/04/08 22:59, Tom Lane wrote: > Amit Langote writes: >> Should I add this patch to Older Bugs [1]? > > Yeah, it's an open issue IMO. I think we've been focusing on getting > as many feature patches done as we could during the CF, but now it's > time to

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-08 Thread Amit Langote
On 2019/03/16 6:41, Robert Haas wrote: > On Fri, Mar 15, 2019 at 3:45 PM Tom Lane wrote: >> I agree that copying data isn't great. What I don't agree is that this >> solution is better. In particular, copying data out of the relcache >> rather than expecting the relcache to hold still over long

Re: ToDo: show size of partitioned table

2019-04-08 Thread Amit Langote
On 2019/04/09 2:37, Alvaro Herrera wrote: > On 2019-Apr-08, Amit Langote wrote: > >> I noticed that the size shown in the output of both \dP+ and \dPt+ does >> include toast relation sizes of (leaf) partitions, because >> pg_table_size(), which is used

Re: Problem with default partition pruning

2019-04-09 Thread Amit Langote
Horiguchi-san, On 2019/04/09 17:51, Kyotaro HORIGUCHI wrote: > At Tue, 09 Apr 2019 17:37:25 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI > wrote in > <20190409.173725.31175835.horiguchi.kyot...@lab.ntt.co.jp> >>> I'm attaching the latest version. Could you please check it again? > > By the

more isolation tests for update tuple routing

2019-04-09 Thread Amit Langote
Continuing the discussion at: https://www.postgresql.org/message-id/26571.1554741097%40sss.pgh.pa.us Tom wrote: > It struck me just as I was pushing it that this test doesn't exercise > EPQ with any of the interesting cases for partition routing (ie where > the update causes a move to a different

Re: COLLATE: Hash partition vs UPDATE

2019-04-09 Thread Amit Langote
On Tue, Apr 9, 2019 at 9:44 PM Jesper Pedersen wrote: > > Hi Amit, > > On 4/8/19 11:18 PM, Amit Langote wrote: > > As of this commit, hashing functions hashtext() and hashtextextended() > > require a valid collation to be passed in. ISTM, > > satisfies_hash_par

Re: more isolation tests for update tuple routing

2019-04-09 Thread Amit Langote
On 2019/04/10 0:45, Tom Lane wrote: > Amit Langote writes: >> Per what Andres mentioned in his reply on the original thread [1], in >> scenarios 1 and 2 where the 1st session's update causes a row to move, >> session 2 produces the following error when trying to upd

Re: Problem with default partition pruning

2019-04-09 Thread Amit Langote
Horiguchi-san, On 2019/04/09 18:49, Kyotaro HORIGUCHI wrote: > Hi, Amit. Thank you for the explanation. > > At Tue, 9 Apr 2019 18:09:20 +0900, Amit Langote > wrote in > <4c1074cc-bf60-1610-c728-9a5b12f5b...@lab.ntt.co.jp> >>> The partition constraint is equival

Re: Problem with default partition pruning

2019-04-09 Thread Amit Langote
On 2019/04/09 17:37, Kyotaro HORIGUCHI wrote: > At Tue, 9 Apr 2019 16:41:47 +0900, "Yuzuko Hosoya" > wrote >>> So still it is wrong that the new code is added at the beginning of the >>> loop on clauses in >>> gen_partprune_steps_internal. >>> If partqual resul

Re: Problem with default partition pruning

2019-04-09 Thread Amit Langote
On 2019/04/10 12:53, Kyotaro HORIGUCHI wrote: > At Wed, 10 Apr 2019 11:17:53 +0900, Amit Langote > wrote: >> Yeah, I think we should move the "if (partconstr)" block to the "if >> (is_orclause(clause))" block as I originally proposed in: >> >> h

Re: Problem with default partition pruning

2019-04-09 Thread Amit Langote
On 2019/04/10 14:55, Amit Langote wrote: > 2. Hosoya-san reported on 2019/03/22 that a contradictory WHERE clause > applied to a *partition* doesn't return an empty plan: > >> I understood Amit's proposal. But I think the issue Thibaut reported >> would occur reg

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-10 Thread Amit Langote
Hi, On 2019/04/10 15:42, Michael Paquier wrote: > On Mon, Apr 08, 2019 at 10:40:41AM -0400, Robert Haas wrote: >> On Mon, Apr 8, 2019 at 9:59 AM Tom Lane wrote: >>> Amit Langote writes: >>> Yeah, it's an open issue IMO. I think we've been focusing on gettin

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-04-10 Thread Amit Langote
On 2019/04/11 12:34, David Rowley wrote: > On Thu, 21 Mar 2019 at 00:51, David Rowley > wrote: >> Just so I don't forget about this, I've added it to the July 'fest. >> >> https://commitfest.postgresql.org/23/2065/ > > Now that we have 428b260f8, I think the version of this that goes into > mast

Re: Should we add GUCs to allow partition pruning to be disabled?

2019-04-10 Thread Amit Langote
On 2019/04/11 13:50, David Rowley wrote: > On Thu, 11 Apr 2019 at 16:06, Amit Langote > wrote: >> I've posted a patch last week on the "speed up partition planning" thread >> [1] which modifies ddl.sgml to remove the text about UPDATE/DELETE using >> constra

Re: speeding up planning with partitions

2019-04-10 Thread Amit Langote
On 2019/04/11 14:03, David Rowley wrote: > On Fri, 5 Apr 2019 at 19:50, Amit Langote > wrote: >> While we're on the topic of the relation between constraint exclusion and >> partition pruning, I'd like to (re-) propose this documentation update >> patch. Th

Re: bug in update tuple routing with foreign partitions

2019-04-10 Thread Amit Langote
Fujita-san, Thanks for the review. On 2019/04/10 17:38, Etsuro Fujita wrote: > (2019/03/06 18:33), Amit Langote wrote: >> I noticed a bug with how UPDATE tuple routing initializes ResultRelInfos >> to use for partition routing targets.  Specifically, the bug occurs when >> U

Re: BUG #15672: PostgreSQL 11.1/11.2 crashed after dropping a partition table

2019-04-10 Thread Amit Langote
On 2019/04/11 15:57, Michael Paquier wrote: > On Wed, Mar 27, 2019 at 11:56:20AM +0900, Michael Paquier wrote: >> Adding it to the section for older bugs sounds fine to me. Thanks for >> doing so. > > I have begun looking at this issue. Hopefully I'll be able to provide > an update soon. Great,

Re: Issue in ExecCleanupTupleRouting()

2019-04-11 Thread Amit Langote
On 2019/04/11 22:28, David Rowley wrote: > On Fri, 12 Apr 2019 at 01:06, Etsuro Fujita > wrote: >> + /* >> +* Check if this result rel is one belonging to the node's subplans, >> +* if so, let ExecEndPlan() clean it up. >> +*/ >> + if (htab) >> + { >> +

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-13 Thread Amit Langote
Thanks for the updated patch. On Sat, Apr 13, 2019 at 4:47 AM Tom Lane wrote: > Robert Haas writes: > > As a parenthetical note, I observe that relcache.c seems to know > > almost nothing about rd_partcheck. rd_partkey and rd_partdesc both > > have handling in RelationClearRelation(), but rd_pa

Re: COLLATE: Hash partition vs UPDATE

2019-04-14 Thread Amit Langote
Thanks for the review. On 2019/04/15 5:50, Tom Lane wrote: > Jesper Pedersen writes: >> Yeah, that works here - apart from an issue with the test case; fixed in >> the attached. > > Couple issues spotted in an eyeball review of that: > > * There is code that supposes that partsupfunc[] is the

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-14 Thread Amit Langote
On 2019/04/14 0:53, Tom Lane wrote: > Amit Langote writes: >> On Sat, Apr 13, 2019 at 4:47 AM Tom Lane wrote: >>> I concluded that that's not parenthetical but pretty relevant... > >> Hmm, do you mean we should grow the same "keep_" logic for >

Re: Mailing list not working

2019-04-15 Thread Amit Langote
On 2019/04/15 15:37, Tatsuo Ishii wrote: >> Hi, >> >> I am not able to access the mailing list archive. Is the mailing list >> server down or something? > > What kind of problem do you have? > > I can see your posting in the archive. > https://www.postgresql.org/message-id/CAKm4Xs5%2BD%2BgB4yCQts

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-15 Thread Amit Langote
On 2019/04/15 2:38, Tom Lane wrote: > Amit Langote writes: >> On 2019/04/10 15:42, Michael Paquier wrote: >>> It seems to me that Tom's argument to push in the way relcache >>> information is handled by copying its contents sounds sensible to me. >>> That

Re: partitioning performance tests after recent patches

2019-04-15 Thread Amit Langote
Hi, Thanks a lot for very exhaustive testing. David already replied to some points, but let me comment on a couple of points. Please be advised that these may not be the numbers (or scalability pattern of these numbers) you'll see when PG 12 is actually released, because we may end up changing s

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-15 Thread Amit Langote
On Mon, Apr 15, 2019 at 5:05 PM Amit Langote wrote: > On 2019/04/15 2:38, Tom Lane wrote: > > So the point here is that that reasoning is faulty. You *cannot* assume, > > no matter how strong a lock or how many pins you hold, that a relcache > > entry will not get rebuilt u

Re: COLLATE: Hash partition vs UPDATE

2019-04-15 Thread Amit Langote
On 2019/04/16 5:47, Tom Lane wrote: > Amit Langote writes: >> Attached updated patch. > > LGTM, pushed. Thank you. Regards, Amit

Re: Runtime pruning problem

2019-04-16 Thread Amit Langote
Hi, On 2019/04/16 21:09, David Rowley wrote: > On Tue, 16 Apr 2019 at 23:55, Yuzuko Hosoya > wrote: >> postgres=# explain analyze select * from t1 where dt = current_date + 400; >> QUERY PLAN >> ---

Re: Runtime pruning problem

2019-04-16 Thread Amit Langote
On 2019/04/17 11:29, David Rowley wrote: > On Wed, 17 Apr 2019 at 13:13, Amit Langote > wrote: >> When you see this: >> >> explain select * from t1 where dt = current_date + 400; >> QUERY PLAN >> ───

Re: Runtime pruning problem

2019-04-16 Thread Amit Langote
On 2019/04/17 12:58, David Rowley wrote: > On Wed, 17 Apr 2019 at 15:54, Tom Lane wrote: >> >> Amit Langote writes: >>> On 2019/04/17 11:29, David Rowley wrote: >>>> Where do you think the output list for EXPLAIN VERBOSE should put the >>>> output

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-17 Thread Amit Langote
On 2019/04/15 4:29, Tom Lane wrote: > I think that what we ought to do for v12 is have PartitionDirectory > copy the data, and then in v13 work on creating real reference-count > infrastructure that would allow eliminating the copy steps with full > safety. The $64 question is whether that really

Re: hyrax vs. RelationBuildPartitionDesc

2019-04-17 Thread Amit Langote
On 2019/04/17 18:58, Amit Langote wrote: > where index on queries > column Oops, I meant "with an index on the queried column". Thanks, Amit

Re: pgsql: Fix plan created for inherited UPDATE/DELETE with all tables exc

2019-04-17 Thread Amit Langote
On 2019/02/23 2:23, Tom Lane wrote: > Fix plan created for inherited UPDATE/DELETE with all tables excluded. > > In the case where inheritance_planner() finds that every table has > been excluded by constraints, it thought it could get away with > making a plan consisting of just a dummy Result no

Re: bug in update tuple routing with foreign partitions

2019-04-17 Thread Amit Langote
Fujita-san, On 2019/04/17 21:49, Etsuro Fujita wrote: > (2019/04/11 20:31), Etsuro Fujita wrote: >> (2019/04/11 14:33), Amit Langote wrote: >>> BTW, have you noticed that the RETURNING clause returns the same row >>> twice? >> >> I noticed this, but I

Re: bug in update tuple routing with foreign partitions

2019-04-17 Thread Amit Langote
On 2019/04/18 14:06, Amit Langote wrote: > Fujita-san, > > On 2019/04/17 21:49, Etsuro Fujita wrote: >> (2019/04/11 20:31), Etsuro Fujita wrote: >>> (2019/04/11 14:33), Amit Langote wrote: >>>> BTW, have you noticed that the RETURNING clause returns the same ro

Re: Runtime pruning problem

2019-04-17 Thread Amit Langote
On 2019/04/17 13:10, Tom Lane wrote: > David Rowley writes: >> On Wed, 17 Apr 2019 at 15:54, Tom Lane wrote: >>> What I'm more worried about is whether this breaks any internal behavior >>> of explain.c, as the comment David quoted upthread seems to think. >>> If we need to have a tlist to refere

Re: bug in update tuple routing with foreign partitions

2019-04-18 Thread Amit Langote
Fujita-san, On 2019/04/18 22:10, Etsuro Fujita wrote: >> On 2019/04/18 14:06, Amit Langote wrote: >>> On 2019/04/17 21:49, Etsuro Fujita wrote: >>>> So what I'm thinking is to throw an error for cases like this.  >>>> (Though, I >>>> t

Re: bug in update tuple routing with foreign partitions

2019-04-18 Thread Amit Langote
On 2019/04/19 13:00, Amit Langote wrote: > BTW, do you think we should update the docs regarding the newly > established limitation of row movement involving foreign tables? Ah, maybe not, because it's a postgres_fdw limitation, not of the core tuple routing feature. OTOH, we don&#

Re: bug in update tuple routing with foreign partitions

2019-04-18 Thread Amit Langote
On 2019/04/19 14:39, Etsuro Fujita wrote: > (2019/04/19 13:00), Amit Langote wrote: >> On 2019/04/18 22:10, Etsuro Fujita wrote: >>> * I kept all the changes in the previous patch, because otherwise >>> postgres_fdw would fail to release resources for a foreign-inse

Re: pgsql: Fix plan created for inherited UPDATE/DELETE with all tables exc

2019-04-19 Thread Amit Langote
On 2019/04/19 4:41, Tom Lane wrote: > Amit Langote writes: >> On 2019/02/23 2:23, Tom Lane wrote: >>> Fix plan created for inherited UPDATE/DELETE with all tables excluded. > >> I noticed that we may have forgotten to fix one more thing in this commit >> -- nom

Re: Runtime pruning problem

2019-04-19 Thread Amit Langote
On 2019/04/19 2:25, Tom Lane wrote: > Amit Langote writes: >> Another idea is to teach explain.c about this special case of run-time >> pruning having pruned all child subplans even though appendplans contains >> one element to cater for targetlist accesses. That is, Append

Re: Runtime pruning problem

2019-04-19 Thread Amit Langote
On 2019/04/19 3:13, Robert Haas wrote: > On Tue, Apr 16, 2019 at 10:49 PM Amit Langote > wrote: >> Maybe, not show them? That may be a bit inconsistent, because the point >> of VERBOSE is to the targetlist among other things, but maybe the users >> wouldn't min

Re: Runtime pruning problem

2019-04-19 Thread Amit Langote
On 2019/04/19 17:00, Amit Langote wrote: > On 2019/04/19 2:25, Tom Lane wrote: >> Amit Langote writes: >>> Another idea is to teach explain.c about this special case of run-time >>> pruning having pruned all child subplans even though appendplans contains >>>

Re: Runtime pruning problem

2019-04-21 Thread Amit Langote
On 2019/04/21 15:25, David Rowley wrote: > On Fri, 19 Apr 2019 at 20:01, Amit Langote > wrote: >> Another approach, as I mentioned above, is to extend the hack that begins >> in nodeAppend.c (and nodeMergeAppend.c) into explain.c, as in the >> attached. Then: >> &g

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-21 Thread Amit Langote
Hi, On 2019/04/20 20:53, Laurenz Albe wrote: > On Fri, 2018-04-06 at 23:24 +, Robert Haas wrote: >> Allow insert and update tuple routing and COPY for foreign tables. >> >> Also enable this for postgres_fdw. >> >> Etsuro Fujita, based on an earlier patch by

display of variables in EXPLAIN VERBOSE

2019-04-22 Thread Amit Langote
Hi, ISTM show_plan_tlist()'s rule of whether to the show range table prefix with displayed variables contradicts the description of the VERBOSE option in EXPLAIN documentation, which is as follows: === VERBOSE Display additional information regarding the plan. Specifically, include the outpu

Re: bug in update tuple routing with foreign partitions

2019-04-22 Thread Amit Langote
Fujita-san, On 2019/04/22 20:00, Etsuro Fujita wrote: > (2019/04/19 14:39), Etsuro Fujita wrote: >> (2019/04/19 13:00), Amit Langote wrote: >>> I agree that we can move to fix the other issue right away as the fix you >>> outlined above seems reasonable, but I wonder

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Amit Langote
On 2019/04/23 7:51, Alvaro Herrera wrote: > On 2019-Mar-06, Tom Lane wrote: >> David Rowley writes: >>> As far as I can see, the biggest fundamental difference with doing >>> things this way will be that the column order of partitions will be >>> preserved, where before it would inherit the order

Re: speeding up planning with partitions

2019-04-22 Thread Amit Langote
On 2019/04/23 7:08, Tom Lane wrote: > Amit Langote writes: >> On 2019/04/02 2:34, Tom Lane wrote: >>> I'm not at all clear on what we think the interaction between >>> enable_partition_pruning and constraint_exclusion ought to be, >>> so I'm no

Re: display of variables in EXPLAIN VERBOSE

2019-04-22 Thread Amit Langote
On 2019/04/23 0:58, Tom Lane wrote: > BTW, now that I look at this, I think the reason why I didn't make > tlist printouts pay attention to VERBOSE for this purpose is that > you don't get them at all if not verbose: > > regression=# explain select * from tt; > QUERY PLAN

Re: pg_dump is broken for partition tablespaces

2019-04-22 Thread Amit Langote
On 2019/04/23 14:45, David Rowley wrote: > On Tue, 23 Apr 2019 at 13:49, Amit Langote > wrote: >> >> On 2019/04/23 7:51, Alvaro Herrera wrote: >>> To me, it sounds >>> unintuitive to accept partitions that don't exactly match the order of >>> t

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-23 Thread Amit Langote
On 2019/04/22 21:45, Etsuro Fujita wrote: > (2019/04/20 20:53), Laurenz Albe wrote: >> I propose that PostgreSQL only allows COPY FROM on a foreign table if >> the FDW >> implements BeginForeignInsert.  The attached patch implements that. > > I don't think that is a good idea, because there might

Re: BUG #15672: PostgreSQL 11.1/11.2 crashed after dropping a partition table

2019-04-23 Thread Amit Langote
Thanks for looking at this. On 2019/04/24 7:03, Tom Lane wrote: > ISTM we could work around the problem with the attached, which I think > is a good change independently of anything else. Agreed. > There is still an issue, which manifests in both 11 and HEAD, namely > that the code also propagat

Re: pg_dump is broken for partition tablespaces

2019-04-23 Thread Amit Langote
On 2019/04/23 20:03, David Rowley wrote: > On Tue, 23 Apr 2019 at 18:18, Amit Langote > wrote: >> >> If partitions needed a >> map in the old database, this patch means that they will *continue* to >> need it in the new database. > > That's incorrect.

Re: pg_dump partitions can lead to inconsistent state after restore

2019-04-23 Thread Amit Langote
On 2019/04/24 10:19, David Rowley wrote: > On Wed, 24 Apr 2019 at 06:50, Alvaro Herrera wrote: >> Per my comment at https://postgr.es/m/2019045129.GA6126@alvherre.pgsql >> I think that pg_dump can possibly cause bogus partition definitions, >> when the users explicitly decide to join tables as

Re: bug in update tuple routing with foreign partitions

2019-04-24 Thread Amit Langote
Fujita-san, On 2019/04/24 18:55, Etsuro Fujita wrote: > (2019/04/23 10:03), Amit Langote wrote: >> Thanks for adding me as an author, but I think the latest >> patch is mostly your work, so I'm happy to be listed as just a reviewer. :) > > You found this bug, analyz

set relispartition when attaching child index

2019-04-24 Thread Amit Langote
Hi, It seems that DefineIndex() is forgetting to update_relispartition() on a partition's index when it's attached to an index being added to the parent. That results in unexpected behavior when adding a foreign key referencing the parent. create table foo (a int) partition by list (a); create t

Re: set relispartition when attaching child index

2019-04-24 Thread Amit Langote
On Thu, Apr 25, 2019 at 12:35 AM Alvaro Herrera wrote: > On 2019-Apr-25, Amit Langote wrote: > > > It seems that DefineIndex() is forgetting to update_relispartition() > > on a partition's index when it's attached to an index being added to > > the parent. T

Re: set relispartition when attaching child index

2019-04-24 Thread Amit Langote
On Thu, Apr 25, 2019 at 12:38 AM Amit Langote wrote: > On Thu, Apr 25, 2019 at 12:35 AM Alvaro Herrera > wrote: > > On 2019-Apr-25, Amit Langote wrote: > > > > > It seems that DefineIndex() is forgetting to update_relispartition() > > > on a partition's i

Re: set relispartition when attaching child index

2019-04-24 Thread Amit Langote
On Thu, Apr 25, 2019 at 12:39 AM Amit Langote wrote: > On Thu, Apr 25, 2019 at 12:38 AM Amit Langote wrote: > > On Thu, Apr 25, 2019 at 12:35 AM Alvaro Herrera > > wrote: > > > On 2019-Apr-25, Amit Langote wrote: > > > > > > > It seems that DefineI

Re: set relispartition when attaching child index

2019-04-24 Thread Amit Langote
On 2019/04/25 0:55, Amit Langote wrote: > On Thu, Apr 25, 2019 at 12:39 AM Amit Langote wrote: >> On Thu, Apr 25, 2019 at 12:38 AM Amit Langote >> wrote: >>> On Thu, Apr 25, 2019 at 12:35 AM Alvaro Herrera >>> wrote: >>>> On 2019-Apr-25, Amit Langot

Re: BUG #15672: PostgreSQL 11.1/11.2 crashed after dropping a partition table

2019-04-24 Thread Amit Langote
On 2019/04/25 8:27, Tom Lane wrote: > Amit Langote writes: >> On 2019/04/24 7:03, Tom Lane wrote: >>> ISTM we could work around the problem with the attached, which I think >>> is a good change independently of anything else. > >> Agreed. > > After t

Re: BUG #15672: PostgreSQL 11.1/11.2 crashed after dropping a partition table

2019-04-25 Thread Amit Langote
On 2019/04/25 11:21, Amit Langote wrote: > On 2019/04/25 8:27, Tom Lane wrote: >> BTW, I hadn't ever looked closely at what the index reuse code >> does, and now that I have, my heart just sinks. I think that >> logic needs to be nuked from orbit. RelationPreserveStorag

Re: BUG #15672: PostgreSQL 11.1/11.2 crashed after dropping a partition table

2019-04-25 Thread Amit Langote
On Thu, Apr 25, 2019 at 10:46 PM Alvaro Herrera wrote: > Haven't read the patch, but I tried applying it on top of my tablespace > fixing patch ... and my first report is that this query in regress fails > (three times): > > select conname, obj_description(oid, 'pg_constraint') from pg_constraint

Re: BUG #15672: PostgreSQL 11.1/11.2 crashed after dropping a partition table

2019-04-25 Thread Amit Langote
On 2019/04/25 19:02, Amit Langote wrote: > On 2019/04/25 11:21, Amit Langote wrote: >> On 2019/04/25 8:27, Tom Lane wrote: >>> BTW, I hadn't ever looked closely at what the index reuse code >>> does, and now that I have, my heart just sinks. I think that >&g

findTargetlistEntrySQL92() and COLLATE clause

2019-04-25 Thread Amit Langote
Hi, I couldn't find old discussions or source code comments about this, but has someone encountered the following error and wondered whether it's working that way for a reason? select a::text, b from foo order by 1, 2 collate "C"; ERROR: collations are not supported by type integer LINE 1: selec

Re: pgsql: Allow insert and update tuple routing and COPY for foreign table

2019-04-25 Thread Amit Langote
Fujita-san, On 2019/04/25 22:17, Etsuro Fujita wrote: > (2019/04/24 22:04), Laurenz Albe wrote: >>    Before PostgreSQL v11, a foreign data wrapper could be certain that >>    BeginForeignModify is always called before ExecForeignInsert. >>    This is no longer true. > > OK, how about something l

Re: speeding up planning with partitions

2019-04-28 Thread Amit Langote
On Sun, Apr 28, 2019 at 8:10 AM Tom Lane wrote: > Amit Langote writes: > > Not sure if you'll like it but maybe we could ignore even regular > > inheritance child targets that are proven to be empty (is_dummy_rel()) for > > a given query during the initial SELECT p

Re: speeding up planning with partitions

2019-05-02 Thread Amit Langote
On Wed, May 1, 2019 at 4:08 AM Tom Lane wrote: > OK, I tweaked that a bit and pushed both versions. Thank you. Regards, Amit

Re: findTargetlistEntrySQL92() and COLLATE clause

2019-05-07 Thread Amit Langote
On 2019/04/27 0:02, Tom Lane wrote: > Amit Langote writes: >> I couldn't find old discussions or source code comments about this, but >> has someone encountered the following error and wondered whether it's >> working that way for a reason? > >> select a

Re: BUG #15672: PostgreSQL 11.1/11.2 crashed after dropping a partition table

2019-05-07 Thread Amit Langote
On 2019/04/27 3:57, Tom Lane wrote: > Alvaro Herrera writes: >> Um, this one doesn't apply because of yesterday's 87259588d0ab. > > Before we spend too much time on minutiae, we should ask ourselves whether > this patch is even going in the right direction. I'm not sure. > > One point is that i

Re: set relispartition when attaching child index

2019-05-07 Thread Amit Langote
On 2019/04/26 23:12, Alvaro Herrera wrote: > On 2019-Apr-25, Amit Langote wrote: > >> BTW, this will need to be back-patched to 11. > > Done, thanks for the patch. I added the test in master, but obviously > it doesn't work in pg11, so I just verified manually th

Re: pg12 release notes

2019-05-09 Thread Amit Langote
On 2019/05/10 12:18, David Rowley wrote: > On Fri, 10 May 2019 at 12:08, Bruce Momjian wrote: >>> 17f206f Set pg_class.relhassubclass for partitioned indexes >> >> I need help with this one. I know the system column existed in previous >> releases, so how is it different now? Do we document syst

Re: pg12 release notes

2019-05-09 Thread Amit Langote
ed table in order when an Append node will do. FWIW, I've asked [1] Bruce to mention this commit in its own release note item. Currently, it's buried under pruning performance improvement item, like this. Improve performance of pruning many partitions (Amit Langote, David Rowley, Tom L

Re: [HACKERS] advanced partition matching algorithm for partition-wise join

2019-05-13 Thread Amit Langote
Hi Amul, Ashutosh, On 2019/04/24 20:26, amul sul wrote: > Attached version is rebase atop of the latest master head(fdc7efcc30), also > incorporates the Ashutosh's suggestion, thanks. Reading the commit message of 0002 and after testing to confirm, I understand that the patch doesn't handle OUTER

<    4   5   6   7   8   9   10   11   12   13   >