Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-11-12 Thread Dilip Kumar
te(), but I think you should move the following >> comment as well: > > I made an adjustment that I hope will address your concern here, made > a few other adjustments, and committed this. > Thanks, Robert. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql

Re: [HACKERS] path toward faster partition pruning

2017-10-30 Thread Dilip Kumar
t to this function. Or we can do something cleaner because if we want to access the ExprContext inside partkey_datum_from_expr then we may need to pass it to "get_partitions_from_clauses" which is a common function for optimizer and executor. -- Regards, Dilip Kumar EnterpriseDB: http:

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-17 Thread Dilip Kumar
nd execution time increased by 30%. I understand that the this is the worst case for PWA where FinalizeAggregate is getting all the tuple. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-17 Thread Dilip Kumar
1) -> Seq Scan on t2 (cost=0.00..1885.00 rows=10 width=8) (actual time=1.407..11.689 rows=10 loops=1) Planning time: 0.146 ms Execution time: 263.678 ms (11 rows) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailin

Re: [HACKERS] Parallel Bitmap Heap Scans segfaults due to (tbm->dsa==NULL) on PostgreSQL 10

2017-10-12 Thread Dilip Kumar
On Thu, Oct 12, 2017 at 6:37 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: > > > On 10/12/2017 02:40 PM, Dilip Kumar wrote: >> On Thu, Oct 12, 2017 at 4:31 PM, Tomas Vondra >> <tomas.von...@2ndquadrant.com> wrote: >>> Hi, >>> >&

Re: [HACKERS] Parallel Bitmap Heap Scans segfaults due to (tbm->dsa==NULL) on PostgreSQL 10

2017-10-12 Thread Dilip Kumar
r node will create the tbm by itself and isshared is set for BitmapOr. Attached patch fixing the issue for me. I will thoroughly test this patch with other scenario as well. Thanks for reporting. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com diff --git a/src/backend/optim

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-11 Thread Dilip Kumar
On Fri, Oct 6, 2017 at 9:21 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Fri, Oct 6, 2017 at 7:24 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: >> On Fri, Oct 6, 2017 at 6:08 PM, Alexander Kuzmenkov >> <a.kuzmen...@postgrespro.ru> wrote: >&

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-06 Thread Dilip Kumar
On Fri, Oct 6, 2017 at 7:24 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Fri, Oct 6, 2017 at 6:08 PM, Alexander Kuzmenkov > <a.kuzmen...@postgrespro.ru> wrote: >> >>> Analysis: The estimated value of the lossy_pages is way higher than >>> its actua

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-06 Thread Dilip Kumar
On Fri, Oct 6, 2017 at 7:04 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Fri, Oct 6, 2017 at 6:36 PM, Robert Haas <robertmh...@gmail.com> wrote: >> On Fri, Oct 6, 2017 at 2:12 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: >>>> The performance r

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-06 Thread Dilip Kumar
56 > 628087227229 > 8197114179172 > 10269227190192 > 14 110108106105 > Thanks for the testing number looks good to me. -- Regards, Dilip Kumar EnterpriseDB:

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-06 Thread Dilip Kumar
On Fri, Oct 6, 2017 at 6:36 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Oct 6, 2017 at 2:12 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: >>> The performance results look good, but that's a slightly different >>> thing from whether the estimate i

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-10-06 Thread Dilip Kumar
On Thu, Oct 5, 2017 at 8:15 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Sun, Sep 17, 2017 at 7:04 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: >> I used lossy_pages = max(0, total_pages - maxentries / 2). as >> suggesed by Alexander. > > Does that formu

Re: [HACKERS] path toward faster partition pruning

2017-09-28 Thread Dilip Kumar
On Thu, Sep 28, 2017 at 1:44 PM, Amit Langote <langote_amit...@lab.ntt.co.jp> wrote: > On 2017/09/28 13:58, Dilip Kumar wrote: >> I think the above logic is common between this patch and the runtime >> pruning. I think we can make >> a reusable function. Here we are

Re: [HACKERS] path toward faster partition pruning

2017-09-27 Thread Dilip Kumar
l of extra processing for this path i.e converting the Expr array to CONST array? [1] https://www.postgresql.org/message-id/CAOG9ApE16ac-_VVZVvv0gePSgkg_BwYEV1NBqZFqDR2bBE0X0A%40mail.gmail.com -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailin

Re: [HACKERS] path toward faster partition pruning

2017-09-26 Thread Dilip Kumar
On Tue, Sep 26, 2017 at 2:45 PM, Amit Langote <langote_amit...@lab.ntt.co.jp> wrote: > On 2017/09/25 20:21, Dilip Kumar wrote: > I see. So, in the run-time pruning case, only the work of extracting > bounding values is deferred to execution time. Matching clauses with the > p

Re: [HACKERS] path toward faster partition pruning

2017-09-25 Thread Dilip Kumar
g what "get_rel_partitions" is doing. But, at optimizer time for PARAM we will not have Datum values for rightop, so we will keep track of the PARAM itself. And, finally at runtime when we get the PARAM value we can prepare minkey and maxkey and call get_partitions_for_keys function. -- Regards

Re: [HACKERS] Effect of changing the value for PARALLEL_TUPLE_QUEUE_SIZE

2017-09-21 Thread Dilip Kumar
On Thu, Sep 21, 2017 at 4:50 PM, Rafia Sabih <rafia.sa...@enterprisedb.com> wrote: > On Sun, Sep 17, 2017 at 9:10 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: >> On Wed, Sep 6, 2017 at 4:14 PM, Rafia Sabih >> <rafia.sa...@enterprisedb.com> wrote: >

Re: [HACKERS] path toward faster partition pruning

2017-09-19 Thread Dilip Kumar
On Sat, Sep 16, 2017 at 3:13 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Fri, Sep 15, 2017 at 2:20 PM, Amit Langote > <langote_amit...@lab.ntt.co.jp> wrote: >> On 2017/09/15 11:16, Amit Langote wrote: > > Thanks for the updated patch. I was going through the

Re: [HACKERS] UPDATE of partition key

2017-09-19 Thread Dilip Kumar
On Tue, Sep 19, 2017 at 1:15 PM, Amit Khandekar <amitdkhan...@gmail.com> wrote: > On 18 September 2017 at 20:45, Dilip Kumar <dilipbal...@gmail.com> wrote: >> Please find few more comments. >> >> + * in which they appear in the PartitionDesc. Also, extract

Re: [HACKERS] UPDATE of partition key

2017-09-18 Thread Dilip Kumar
On Mon, Sep 18, 2017 at 11:29 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Fri, Sep 15, 2017 at 4:55 PM, Amit Khandekar <amitdkhan...@gmail.com> > wrote: >> On 12 September 2017 at 12:39, Amit Khandekar <amitdkhan...@gmail.com> wrote: >>> On

Re: [HACKERS] UPDATE of partition key

2017-09-17 Thread Dilip Kumar
On Fri, Sep 15, 2017 at 4:55 PM, Amit Khandekar <amitdkhan...@gmail.com> wrote: > On 12 September 2017 at 12:39, Amit Khandekar <amitdkhan...@gmail.com> wrote: >> On 12 September 2017 at 11:57, Dilip Kumar <dilipbal...@gmail.com> wrote: >>> On Tue, Sep

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-09-17 Thread Dilip Kumar
On Sun, Sep 17, 2017 at 4:34 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: >> >> I have repeated one of the tests after fixing the problems pointed by >> you but this time results are not that impressive. Seems like below >> check was the problem in the previous pa

Re: [HACKERS] Effect of changing the value for PARALLEL_TUPLE_QUEUE_SIZE

2017-09-17 Thread Dilip Kumar
set but not used [-Wunused-but-set-variable] uint64 bytes_written, nbytes, tuple_size; -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-09-17 Thread Dilip Kumar
On Mon, Sep 4, 2017 at 11:18 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Thu, Aug 31, 2017 at 11:27 PM, Robert Haas <robertmh...@gmail.com> wrote: > > I have repeated one of the tests after fixing the problems pointed by > you but this time results are not that

Re: [HACKERS] path toward faster partition pruning

2017-09-16 Thread Dilip Kumar
m that the get_partitions_for_keys is directly fetching the "rightop->constvalue" whereas, for runtime pruning, we need to store rightop itself and calculate the value at runtime by param evaluation, I haven't yet thought how can we make this last part generic. -- Regards, Dilip Kumar En

Re: [HACKERS] path toward faster partition pruning

2017-09-14 Thread Dilip Kumar
postgres=# explain select * from t where 2>t.a; QUERY PLAN Append (cost=0.00..4.49 rows=2 width=8) -> Seq Scan on t1 (cost=0.00..2.24 rows=1 width=8) Filter: (2 > a) -> Seq Scan on t2 (cost=0.00..2.25 rows=1 width=8) Filter:

Re: [HACKERS] UPDATE of partition key

2017-09-12 Thread Dilip Kumar
ot; and there is only a "statement level delete trigger" with transition table still we are making the entry in transition table of the delete trigger and that will never be used. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing

Re: [HACKERS] UPDATE of partition key

2017-09-11 Thread Dilip Kumar
On Thu, Sep 7, 2017 at 11:41 AM, Amit Khandekar <amitdkhan...@gmail.com> wrote: > On 6 September 2017 at 21:47, Dilip Kumar <dilipbal...@gmail.com> wrote: > Actually, since transition tables came in, the functions like > ExecARUpdateTriggers() or ExecARInsertTriggers()

Re: [HACKERS] UPDATE of partition key

2017-09-06 Thread Dilip Kumar
not fired but below code call ARUpdateTrigger + if (mtstate->operation == CMD_UPDATE && mtstate->mt_transition_capture) + ExecARUpdateTriggers(estate, resultRelInfo, InvalidOid, + NULL, + tuple, + NULL, + mtstate->mt_transition_capture); -- Regards, Dilip Kumar Enterpris

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-09-03 Thread Dilip Kumar
337894 206824 15 995417 798817 20 1654016 1588498 Still, we see a good reduction in lossy pages count. I will perform the test at different work_mem and for different values of TBM_FILFACTOR and share the number soon. -- Regards, Dilip Kumar EnterpriseD

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-09-01 Thread Dilip Kumar
tivity| CheckpointerMain 1073 LWLock | XidGenLock 869 IPC | ClogGroupUpdate -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UPDATE of partition key

2017-08-31 Thread Dilip Kumar
/executor/nodeModifyTable.c -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-08-31 Thread Dilip Kumar
/www.enterprisedb.com > The Enterprise PostgreSQL Company -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-08-28 Thread Dilip Kumar
On Wed, Aug 23, 2017 at 9:45 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: >> >> ... >> if (tbm->nentries <= tbm->maxentries / 2) >> { >> /* >> * We have made enough room. >>

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-08-22 Thread Dilip Kumar
ossy. > > I didn't benchmark the index scan per se with the 0.9 fill factor, but the > reduction of lossy pages was significant. I will try this and produce some performance number. Thanks for the input. > > [1] > https://www.postgresql.org/message-id/flat/251401bb-6f53-b957-4

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-18 Thread Dilip Kumar
On Fri, 18 Aug 2017 at 4:48 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Thu, Aug 17, 2017 at 2:45 PM, Dilip Kumar <dilipbal...@gmail.com> > wrote: > > On Thu, Aug 17, 2017 at 2:09 PM, Dilip Kumar <dilipbal...@gmail.com> > wrote: > >> > &g

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-17 Thread Dilip Kumar
On Thu, Aug 17, 2017 at 2:09 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > > Either we can pass "num_gene" to merge_clump or we can store num_gene > in the root. And inside merge_clump we can check. Do you see some more > complexity? > After putting some more

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-17 Thread Dilip Kumar
enerate_gather_paths(root, joinrel); } Assuming, we find > some way to skip it for top level scan/join node, I don't think that > will be sufficient, we have some special way to push target list below > Gather node in apply_projection_to_path, we need to move that part as > well in g

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-08-16 Thread Dilip Kumar
On Wed, Jul 26, 2017 at 10:35 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Jun 8, 2017 at 10:44 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: >> On Thu, May 18, 2017 at 8:07 PM, Robert Haas <robertmh...@gmail.com> wrote: >> Thanks for the feedback.

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-07-25 Thread Dilip Kumar
: (l_orderkey = orders.o_orderkey) Filter: (l_commitdate < l_receiptdate) Rows Removed by Filter: 1 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] why not parallel seq scan for slow functions

2017-07-12 Thread Dilip Kumar
On Wed, Jul 12, 2017 at 10:55 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: >> On Mon, Jul 10, 2017 at 9:51 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: >>> >>> So bec

Re: [HACKERS] why not parallel seq scan for slow functions

2017-07-10 Thread Dilip Kumar
t least I know how to start looking into that. > > Also, how do you debug stuff like this? Are there some gdb tricks to make > this easier to introspect into the plans? > > Cheers, > > Jeff -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-06 Thread Dilip Kumar
n { int64 i64; uint8 i8[8]; } swap; and reverse complete array if byte order is changed or we can use something like "be64toh" > > > -- > Thanks & Regards, > Kuntal Ghosh > EnterpriseDB: http://www.enterprisedb.com > > > -- > Sen

Re: [HACKERS] [POC] hash partitioning

2017-07-05 Thread Dilip Kumar
datum memory, so better to check ndatum first. 4. + * next larger modulus. For example, if you have a bunch + * of partitions that all have modulus 5, you can add a + * new new partition with modulus 10 or a new partition Typo, "new new partition" -> "new parti

Re: [HACKERS] Default Partition for Range

2017-07-03 Thread Dilip Kumar
On Fri, Jun 30, 2017 at 11:56 AM, Beena Emerson <memissemer...@gmail.com> wrote: > Hello Dilip, > > On Wed, Jun 21, 2017 at 6:27 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: >> On Tue, Jun 20, 2017 at 6:57 PM, Dilip Kumar <dilipbal...@gmail.com> wrot

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2017-06-22 Thread Dilip Kumar
only upto slot->len, seems like it will always be <= buf_len, and if not then we can do min(buf_len, slot->len) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Default Partition for Range

2017-06-21 Thread Dilip Kumar
need to be careful of early exiting wherever we are looping the content array of the DEFAULT bound. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Dilip Kumar
abstraction. When we are using the DSA we should not directly access the DSM which is under DSA. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Dilip Kumar
cuumShmem->av_dsa_handle)) > { > AutoVacuumDSA = dsa_attach(AutoVacuumShmem->av_dsa_handle); > dsa_pin_mapping(AutoVacuumDSA); > > Thoughts? IMHO, It's not a good idea to use DSM call to verify the DSA handle. -- Regards, Dilip Ku

Re: [HACKERS] Default Partition for Range

2017-06-21 Thread Dilip Kumar
On Tue, Jun 20, 2017 at 6:57 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > This is basically crashing in RelationBuildPartitionDesc, so I think > we don't have any test case for testing DEFAULT range partition where > partition key has more than one attribute. So I sug

Re: [HACKERS] Regarding Postgres Dynamic Shared Memory (DSA)

2017-06-20 Thread Dilip Kumar
facing is with linux systems. Mine is Linux. CentOS Linux release 7.2.1511 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Default Partition for Range

2017-06-20 Thread Dilip Kumar
rocessing the request. This is basically crashing in RelationBuildPartitionDesc, so I think we don't have any test case for testing DEFAULT range partition where partition key has more than one attribute. So I suggest we can add such test case. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Regarding Postgres Dynamic Shared Memory (DSA)

2017-06-20 Thread Dilip Kumar
_data_access(1); 2017-06-20 16:49:39.485 IST [9738] LOG: LinkedList Node Data read is ==> Hello world19 2017-06-20 16:49:39.485 IST [9738] STATEMENT: select test_dsa_data_access(1); test_dsa_data_access ------ 0 I don't see any segmentation fault. Is there s

Re: [HACKERS] [BUGS] Postgresql bug report - unexpected behavior of suppress_redundant_updates_trigger

2017-06-19 Thread Dilip Kumar
field = rpad('', 2001, 'a') WHERE id = 1; > UPDATE 1 <--- BUG: expected 0, as we ran the same update twice Seems like in "suppress_redundant_updates_trigger" we are comparing toasted tuple with the new tuple and that is the cause of the bug. -- Regards, Dilip Kumar Ente

Re: [HACKERS] psql's \d and \dt are sending their complaints to different output files

2017-06-19 Thread Dilip Kumar
named "xxx" ". whereas \dt says "No matching relations found". -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-06-08 Thread Dilip Kumar
onsider the effective_cache size. But, actually, multiple tuples can fall into the same page. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] hash partitioning

2017-06-06 Thread Dilip Kumar
t; the tuple-routing. > > I wonder if we should... Since we don't allow BR triggers on partitioned > tables, there should not be any harm in doing it just before calling > ExecFindPartition(). Perhaps, topic for a new thread. Yeah, correct. -- Regards, Dilip Kumar EnterpriseDB: htt

Re: [HACKERS] [POC] hash partitioning

2017-06-06 Thread Dilip Kumar
that a value is legal for the child but illegal for the parent? Isn't it a good idea that if a user is inserting in the top level relation he should know for which partition exactly the constraint got violated? > > Updated patch attached. Thanks. -- Regards, Dilip Kumar

Re: [HACKERS] Default Partition for Range

2017-06-05 Thread Dilip Kumar
TUM_DEFAULT) + { + if (content1[i] == content2[i]) + return 0; + else if (content1[i] == RANGE_DATUM_DEFAULT) + return -1; I don't see any comments why default partition should be considered smallest in the index comparison. For negative infinity, it's pretty obvious by the enum name itself. --

Re: [HACKERS] [POC] hash partitioning

2017-06-03 Thread Dilip Kumar
identifying the leaf partition and it's only failing during ExecInsert while checking the partition constraint. Other than that, patch looks fine to me. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To m

Re: [HACKERS] <> join selectivity estimate question

2017-06-01 Thread Dilip Kumar
On Thu, Jun 1, 2017 at 8:24 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, May 31, 2017 at 1:18 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: >> + if (jointype = JOIN_SEMI) >> + { >> + sjinfo->jointype = JOIN_INNER; >&

Re: [HACKERS] Error while creating subscription when server is running in single user mode

2017-06-01 Thread Dilip Kumar
e case like creation subscription in normal mode and then ALTER OWNER in single user mode but it makes little sense to me. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com subscription_error_v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] Error while creating subscription when server is running in single user mode

2017-05-31 Thread Dilip Kumar
On Wed, May 31, 2017 at 2:20 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > Yeah, see 0e0f43d6 for example. A simple fix is to look at > IsUnderPostmaster when creating, altering or dropping a subscription > in subscriptioncmds.c. Yeah, below patch fixes that. -- Regar

Re: [HACKERS] COPY (query) TO ... doesn't allow parallelism

2017-05-31 Thread Dilip Kumar
cause Citus hit this... IMHO, For copy_to I don't see any problem in parallelizing it. I just tested by changing the cursorOption and it's working in parallel. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com copy_to_parallel.patch Description: Binary data -- Sent via

Re: [HACKERS] POC: Sharing record typmods between backends

2017-05-31 Thread Dilip Kumar
a more appropriate choice. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] <> join selectivity estimate question

2017-05-31 Thread Dilip Kumar
jointype = JOIN_INNER; + } /* * We want 1 - eqjoinsel() where the equality operator is the one * associated with this != operator, that is, its negator. We may need something similar for anti-join as well. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterpris

Re: [HACKERS] POC: Sharing record typmods between backends

2017-05-31 Thread Dilip Kumar
creates the simplehash will be responsible for copying these static members to the shared location so that other processes can access the SH_TYPE. And, the dynamic part (the actual hash entries) can be allocated using DSA by registering SH_ALLOCATE function. -- Regards, Dilip Kumar EnterpriseDB: http:/

Re: [HACKERS] Error while creating subscription when server is running in single user mode

2017-05-31 Thread Dilip Kumar
_ctl() failed: Bad file > descriptor IMHO, In single user mode, it can not support replication (it can not have background WALReciver task). However, I believe there should be a proper error if the above statement is correct. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] POC: Sharing record typmods between backends

2017-05-30 Thread Dilip Kumar
n option to provide your own allocator function to it. So in the allocator function, you can allocate memory from DSA. After it reaches some threshold it expands the size (double) and it will again call the allocator function to allocate the bigger memory. You can refer pagetable_allocate in tidbitma

Re: [HACKERS] HINT message for "ALTER SUBSCRIPTION.. WITH" need to change with SET keyword

2017-05-19 Thread Dilip Kumar
= "HINT: Use ALTER SUBSCRIPTION ... SET (slot_name = NONE) to > disassociate the subscription from the slot." Seems like syntax got changed in b807f59828fbc02fea612e1cbc0066c6dfa3be9b commit but missed to change the hint. Attached patch fixes that. -- Regards, Dilip Kumar EnterpriseDB: http:

Re: [HACKERS] [POC] hash partitioning

2017-05-18 Thread Dilip Kumar
ro for the remainder right? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Proposal: Improve bitmap costing for lossy pages

2017-05-18 Thread Dilip Kumar
7) compared to the actual unique heap pages fetched (293314). Therefore, while costing bitmap scan using 1112817 pages and 4MB of work_mem, we predicted that even after we lossify all the pages it can not fit into work_mem, hence bitmap scan was not selected. -- Regards, Dilip Kuma

Re: [HACKERS] UPDATE of partition key

2017-05-17 Thread Dilip Kumar
e the same problem. But after putting some more thought I realised that only for "Before Update" or the "Before Insert" trigger row can be changed. Correct me if I am assuming something wrong? So now again option3 will make more sense. -- Regards, Dilip Kumar EnterpriseDB: h

Re: [HACKERS] UPDATE of partition key

2017-05-17 Thread Dilip Kumar
lete/Insert triggers also, he can not rely only upon update level triggers. Earlier I thought that option1 is better but later I think that this can complicate the situation as we are firing first BR update then BR delete and can change the row multiple time and defining such behaviour can be compl

Re: [HACKERS] [POC] hash partitioning

2017-05-16 Thread Dilip Kumar
oo with (modulus 2, remainder 1); You can detach foo1, create two new partitions with modulus 4 and remainders 0 and 2, and move the data over from the old partition I think it will be good information for a user to have? or it's already documented and I missed it? -- Regards, Dilip Kumar

Re: [HACKERS] [POC] hash partitioning

2017-05-15 Thread Dilip Kumar
On Mon, May 15, 2017 at 9:06 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > Test2: > postgres=# insert into x2 values(100); -- it should violates > partition constraint > INSERT 0 1 > > Seems like a bug or am I missing something completely? Sorry, my bad. It's

Re: [HACKERS] [POC] hash partitioning

2017-05-15 Thread Dilip Kumar
2 values(100); -- it should violates partition constraint INSERT 0 1 Seems like a bug or am I missing something completely? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscripti

Re: [HACKERS] Increasing parallel workers at runtime

2017-05-15 Thread Dilip Kumar
increase the performance + * of the query as planned. + */ You might want to do similar changes for gather_merge_readnext. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: h

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-05-15 Thread Dilip Kumar
t column down to the child. But, can't we call rewriteTargetListUD for all the inheritors if the inheritor is a foreign table which will intern call the postgresAddForeignUpdateTargets? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-05-15 Thread Dilip Kumar
holerow` attribute for foreign tables. Oh, right! -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-05-14 Thread Dilip Kumar
On Sun, May 14, 2017 at 9:54 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > After your fix, now tupleid is invalid which is expected, but seems > like we need to do something more. As per the comments seems like it > is expected to get the oldtuple from planSlot. But I don't

Re: [HACKERS] Bug in ExecModifyTable function and trigger issues for foreign tables

2017-05-14 Thread Dilip Kumar
ents seems like it is expected to get the oldtuple from planSlot. But I don't see any code for handling that part. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] hash partitioning

2017-05-13 Thread Dilip Kumar
think it's not a good idea to mix pgindent changes with your patch. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [POC] hash partitioning

2017-05-11 Thread Dilip Kumar
+ int modulus; + int remainder; + int index; +} PartitionHashBound; It will good to add some comments to explain the structure members -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-20 Thread Dilip Kumar
oryContext here as well, but for simplicity of memory usage +analysys we spin up different context. Typo /analysys/analysis -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your sub

Re: [HACKERS] AGG_HASHED cost estimate

2017-04-20 Thread Dilip Kumar
Aggregate is getting picked where actually grouping aggregate with GatherMerge was faster during actual execution time and he suspected problems with costing of hashAggregate. Maybe this is one of those? [1]https://www.postgresql.org/message-id/CAGPqQf2164iV6k-_M75qEZWiCfRarA_SKSmHjc0Uh1rEf5RJrA%40mai

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-05 Thread Dilip Kumar
On Tue, Apr 4, 2017 at 5:51 AM, Dilip Kumar <dilipbal...@gmail.com> wrote: > Sure I can do that, In attached patch, I only fixed the problem of not > executing the bitmap test. Now, I will add few cases to cover other > parts especially rescan and prefetching logic. I have added

[HACKERS] Parallel Bitmap Heap Scan - Prefetch pages are not updated properly

2017-04-04 Thread Dilip Kumar
While analyzing the coverage for the prefetching part, I found an issue that prefetch_pages were not updated properly while executing in parallel mode. Attached patch fixes the same. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com parallel_bitmap_prefetch_fix.patch

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-03 Thread Dilip Kumar
d the tests to > cover the prefetching logic - it's quite bad that it's currently not > tested at all :( Sure I can do that, In attached patch, I only fixed the problem of not executing the bitmap test. Now, I will add few cases to cover other parts especially rescan and prefetching logic

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-01 Thread Dilip Kumar
es that. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com parallel_bitmap_test.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-31 Thread Dilip Kumar
he test ideas. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-30 Thread Dilip Kumar
irectory `/home/dilip/work/pg_codes/pbms_final/postgresql/src' make: *** [all-src-recurse] Error 2 I tried doing maintainer-clean, deleting postgres.bki but still the same error. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] Parallel bitmap heap scan

2017-03-27 Thread Dilip Kumar
to > be called from SH_CREATE, which isn't prepared for a NULL return > anyway. > > Am I all wet? Yes you are right that we are not passing DSA_ALLOC_NO_OOM, so dsa_allocate_extended will return error in case if allocation failed. -- Regards, Dilip Kumar EnterpriseDB: http://www.ente

Re: [HACKERS] Parallel bitmap heap scan

2017-03-27 Thread Dilip Kumar
SA. I have also handled the NULL pointer return from dsa_get_address. Please verify with the attached patch. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com bitmap_hugepage_fix.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] Problem in Parallel Bitmap Heap Scan?

2017-03-26 Thread Dilip Kumar
pen Items list so that we >> don't loose track of this issue. > > The result is correct with this patch. I ran make installcheck then > the same steps as above and the query result was correct after > creating the index. Thanks for confirming. -- Regards, Dilip Kumar Enterprise

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-22 Thread Dilip Kumar
select). But here we calling the parameter to ExecutorRun as execute_once so !fcache->returnsSet || !es->lazyEval is the correct one and future proof. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] Problem in Parallel Bitmap Heap Scan?

2017-03-21 Thread Dilip Kumar
=0.00..12.26 rows=17 width=0) (actual time=0.016..0.016 rows=17 loops=1) Index Cond: (i <> '192.168.1.0/24'::inet) Planning time: 0.113 ms Execution time: 5.691 ms (13 rows) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-ha

Re: [HACKERS] Problem in Parallel Bitmap Heap Scan?

2017-03-21 Thread Dilip Kumar
92.168.1.226/24 192.168.1.0/24 | 192.168.1.255/24 192.168.1.0/24 | 192.168.1.0/25 192.168.1.0/24 | 192.168.1.255/25 192.168.1.0/26 | 192.168.1.226 :::1.2.3.4/128 | ::4.3.2.1/24 10:23::f1/128 | 10:23::f1/64 10:23::8000/113 | 10:23:: (16 rows) -- Regards, Di

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-21 Thread Dilip Kumar
>...will trigger three separate calls to ExecutorRun(), which is a >problem if the plan is a parallel plan. And you also need to test this case what Robert have mentioned up thread. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-16 Thread Dilip Kumar
u. Thanks for verifying. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

  1   2   3   4   5   >