Re: [HACKERS] [POC] Faster processing at Gather node

2017-10-16 Thread Rafia Sabih
On Tue, Oct 17, 2017 at 3:22 AM, Andres Freund <and...@anarazel.de> wrote: > Hi Rafia, > > On 2017-05-19 17:25:38 +0530, Rafia Sabih wrote: >> head: >> explain analyse select * from t where i < 3000; >>

Re: [HACKERS] Effect of changing the value for PARALLEL_TUPLE_QUEUE_SIZE

2017-09-25 Thread Rafia Sabih
On Thu, Sep 21, 2017 at 10:34 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Thu, Sep 21, 2017 at 4:50 PM, Rafia Sabih > <rafia.sa...@enterprisedb.com> wrote: >> On Sun, Sep 17, 2017 at 9:10 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: >>> On W

Re: [HACKERS] Effect of changing the value for PARALLEL_TUPLE_QUEUE_SIZE

2017-09-21 Thread Rafia Sabih
On Sun, Sep 17, 2017 at 9:10 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Wed, Sep 6, 2017 at 4:14 PM, Rafia Sabih > <rafia.sa...@enterprisedb.com> wrote: > >> I worked on this idea of using local queue as a temporary buffer to >> write the tuples whe

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-09-20 Thread Rafia Sabih
On Tue, Sep 19, 2017 at 3:50 PM, Alvaro Herrera <alvhe...@alvh.no-ip.org> wrote: > Rafia Sabih wrote: > >> On completing the benchmark for all queries for the above mentioned >> setup, following performance improvement can be seen, >> Query | Patch | Head >

Re: [HACKERS] utility commands benefiting from parallel plan

2017-09-13 Thread Rafia Sabih
On Wed, Sep 13, 2017 at 2:29 PM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > > On Wed, Sep 13, 2017 at 4:17 PM, Rafia Sabih <rafia.sa...@enterprisedb.com> > wrote: >> >> On Fri, Sep 1, 2017 at 12:31 PM, Haribabu Kommi >> <komm

Re: [HACKERS] utility commands benefiting from parallel plan

2017-09-13 Thread Rafia Sabih
e passing - I agree with Dilip that having similar mechanism for 'insert into select...' statements would add more value to the patch, but even then this looks like a good idea to extend parallelism for atleast a few of the write operations -- Regards, Rafia Sabih EnterpriseDB: http://www.ente

Re: [HACKERS] [POC] Faster processing at Gather node

2017-09-11 Thread Rafia Sabih
ments for quite some cases. I'll be glad if you may have a look at this patch and enlighten me with your suggestions. :-) [1] - https://www.postgresql.org/message-id/CAOGQiiNiMhq5Pg3LiYxjfi2B9eAQ_q5YjS%3DfHiBJmbSOF74aBQ%40mail.gmail.com -- Regards, Rafia Sabih EnterpriseDB: http://www.enterpris

Re: [HACKERS] Parallel Append implementation

2017-09-08 Thread Rafia Sabih
10 and Q21, the estimates are similar. However, on a fresh database the selectivity-estimates and plans as reported by you and with the patched version I posted seems to be the correct one. I'll see if I may check performance of these queries once again to verify these. -- Regards, Rafia Sabih

Re: [HACKERS] Parallel Append implementation

2017-09-07 Thread Rafia Sabih
to investigate this a little further. Additionally I want to point that I also applied patch [1], which I forgot to mention before. [1] https://www.postgresql.org/message-id/CAEepm%3D3%3DNHHko3oOzpik%2BggLy17AO%2Bpx3rGYrg3x_x05%2BBr9-A%40mail.gmail.com > On 16 August 2017 at 18:34, Robert H

Re: [HACKERS] Effect of changing the value for PARALLEL_TUPLE_QUEUE_SIZE

2017-09-06 Thread Rafia Sabih
.054 ms (15 rows) This suggests that with such an idea the range of selectivity for using parallelism can be extended for improving the performance of the queries. Credits: Would like to extend thanks to my colleagues Dilip Kumar, Amit Kapila, and Robert Haas for their discussions and words of encourag

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-07-26 Thread Rafia Sabih
rtition-wise join is not possible then. A different partitioning > scheme may be required there. > Good point, will look into this direction as well. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-07-26 Thread Rafia Sabih
er side of NLJ, the query completes quickly since the number of actual returned rows is too low. However, in case we pick seq scan, or lineitem is on the outer side, the query is going to take a really long time. Now, when Hash-Join is picked in the case of partitioned database and no partition-wise join i

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-07-25 Thread Rafia Sabih
On Wed, Jul 26, 2017 at 11:06 AM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Wed, Jul 26, 2017 at 11:00 AM, Rafia Sabih > <rafia.sa...@enterprisedb.com> wrote: > > > > > > On Wed, Jul 26, 2017 at 10:58 AM, Ashutosh Bapat > &g

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-07-25 Thread Rafia Sabih
On Wed, Jul 26, 2017 at 10:58 AM, Ashutosh Bapat < ashutosh.ba...@enterprisedb.com> wrote: > On Tue, Jul 25, 2017 at 11:01 AM, Rafia Sabih > <rafia.sa...@enterprisedb.com> wrote: > > > Query plans for the above mentioned queries is attached. > > > > Can you

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-07-21 Thread Rafia Sabih
On Thu, Jul 20, 2017 at 2:44 PM, Ashutosh Bapat <ashutosh.ba...@enterprisedb.com> wrote: > > On Thu, Jul 20, 2017 at 11:46 AM, Amit Langote > <langote_amit...@lab.ntt.co.jp> wrote: > > On 2017/07/20 15:05, Ashutosh Bapat wrote: > >> On Wed, Jul 19, 2017 at

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-07-21 Thread Rafia Sabih
, this doesn't look like a problem of partition-wise join patch itself. [1] https://www.postgresql.org/message-id/CAEepm%3D3%3DNHHko3oOzpik%2BggLy17AO%2Bpx3rGYrg3x_x05%2BBr9-A%40mail.gmail.com -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ Q21_SE_patch.out Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel Append implementation

2017-06-30 Thread Rafia Sabih
ead i.e. commit -- 08aed6604de2e6a9f4d499818d7c641cbf5eb9f7, looks like need a rebasing. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-06-30 Thread Rafia Sabih
t head-- commit 08aed6604de2e6a9f4d499818d7c641cbf5eb9f7 Might be in need of rebasing. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-06-06 Thread Rafia Sabih
effort. It's not really wrong, it just sounds > weird to me as an American. > Agree, sentence reframed. I am attaching the patch with the modifications I made on a second look. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ cosmetic_autoprewarm_v2.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-06-05 Thread Rafia Sabih
; >> BTW, have you tested how long this takes to run with, say, shared_buffers = >> 8GB? > > I have tried same on my local machine with ssd as a storage. > > settings: shared_buffers = 8GB, loaded data with pg_bench scale_factor=1000. > > Total blocks got dumped > autopre

Re: [HACKERS] Default Partition for Range

2017-06-03 Thread Rafia Sabih
arate patch in a new thread. > > +1. Sure, attached is the version without those changes. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ cosmetic_range_default_partition_v2.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] Default Partition for Range

2017-06-02 Thread Rafia Sabih
n returns bool, but the actually it returns a List. >> + * >> + * If DEFAULT is the only partiton for the table then this returns TRUE. >> + * >> > Updated. > > [1] http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg315573.html > Hi Beena, I had

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-06-02 Thread Rafia Sabih
doesn't suit at some places particularly in pgfdw_subxact_callback, not sure should change comment or variable name /* Disarm abort_cleanup_incomplete if it all worked. */ + entry->changing_xact_state = abort_cleanup_failure; Also, by any chance should we add a test-case for this? -- Regards, R

Re: [HACKERS] Effect of changing the value for PARALLEL_TUPLE_QUEUE_SIZE

2017-06-01 Thread Rafia Sabih
very less experimentation to ascertain something, I'll continue the experiments and would be grateful to have more suggestions on that. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Effect of changing the value for PARALLEL_TUPLE_QUEUE_SIZE

2017-05-30 Thread Rafia Sabih
significantly. One way to do is to give this parameters as another GUC just like min_parallel_table_scan_size, etc. Attached .txt file gives the plan at head and with this patch, additionally patch is attached for setting PARALLEL_TUPLE_QUEUE_SIZE to 6553600 too. Thoughts? -- Regards, Rafia Sabih

Re: [HACKERS] Increasing parallel workers at runtime

2017-05-23 Thread Rafia Sabih
ience alone. Let's see if anybody else shares my gut feeling. :) -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Increasing parallel workers at runtime

2017-05-22 Thread Rafia Sabih
ces only when required. Certainly there are many more things to think about in terms of implementation and the cases where this can help or regress, will be glad to know opinion of more people on this. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ -- Sent via pgsql-hackers ma

[HACKERS] [POC] Faster processing at Gather node

2017-05-19 Thread Rafia Sabih
ueries, in the meantime would appreciate some feedback on the design, etc. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ faster_gather.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

2017-04-24 Thread Rafia Sabih
4b26bac09379a4cbf1fbd847d1cee2293c5e4 is missed? Not sure if this requires a separate entry or could be merged with -- Support parallel btree index scans. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Range Merge Join v1

2017-04-20 Thread Rafia Sabih
tive to the nestloop-with-inner-index we already offer at the > leaf level today. > > Regards, > Jeff Davis > Looks like an interesting idea, however, in an attempt to test this patch I found following error when compiling, selfuncs.c: In function ‘mergejoinscansel’: selfuncs.c:2901:12: error: ‘op_strategy’ undeclared (first use in this function) _strategy, -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/

Re: [HACKERS] parallel explain analyze support not exercised

2017-04-02 Thread Rafia Sabih
┘ > > should be reproducible. I'd suggest additionally adding one tests that > throws the EXPLAIN output away, but actually enables paralellism. > > Greetings, > > Andres Freund > > [1] > https://coverage.postgresql.org/src/backend/executor/e

Re: [HACKERS] TPC-H Q20 from 1 hour to 19 hours!

2017-03-31 Thread Rafia Sabih
((partsupp.ps_partkey = lineitem.l_partkey) AND (partsupp.ps_suppkey = lineitem.l_suppkey)) Join Filter: ((partsupp.ps_availqty)::numeric > ((0.5 * sum(lineitem.l_quantity So, it looks like in the problematic area, it is not improving much. Plea

Re: [HACKERS] Parallel query execution with SPI

2017-03-31 Thread Rafia Sabih
ssage-id/flat/CAFiTN-vxhvvi-rMJFOxkGzNaQpf%2BKS76%2Bsu7-sG_NQZGRPJkQg%40mail.gmail.com#cafitn-vxhvvi-rmjfoxkgznaqpf+ks76+su7-sg_nqzgrpj...@mail.gmail.com -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-03-30 Thread Rafia Sabih
On Tue, Mar 28, 2017 at 11:11 AM, Rafia Sabih <rafia.sa...@enterprisedb.com> wrote: > On Mon, Mar 27, 2017 at 12:20 PM, Thomas Munro > <thomas.mu...@enterprisedb.com> wrote: >> >> On Sun, Mar 26, 2017 at 3:56 PM, Thomas Munro >> <thomas.mu...@enterprised

Re: [HACKERS] TPC-H Q20 from 1 hour to 19 hours!

2017-03-29 Thread Rafia Sabih
> However, it's a bit hard for me to piece through these plans, the > formatting kind of got messed up - things are wrapped. Could you > possibly attach the plans as attachments? > Sure, please find attached file for the plans before and after commit. -- Regards, Raf

Re: [HACKERS] [COMMITTERS] pgsql: Improve access to parallel query from procedural languages.

2017-03-28 Thread Rafia Sabih
ght be required in this direction. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ system_defined_fn_update_v3.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] [COMMITTERS] pgsql: Improve access to parallel query from procedural languages.

2017-03-27 Thread Rafia Sabih
safe ('u')? > Done. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ system_defined_fn_update_v2.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgbench - allow to store select results into variables

2017-03-27 Thread Rafia Sabih
t; Please find attached a v8 which hopefully fixes these two issues. Looks good to me, marking as ready for committer. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel bitmap heap scan

2017-03-27 Thread Rafia Sabih
001b5187a8, accessMtd=0x1033a6c8 , recheckMtd=0x1033bab8 ) at execScan.c:162 #15 0x1033bb88 in ExecBitmapHeapScan (node=0x1001b5187a8) at nodeBitmapHeapscan.c:667 Please feel free to ask if any more information is required for this. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [COMMITTERS] pgsql: Improve access to parallel query from procedural languages.

2017-03-26 Thread Rafia Sabih
r in the regress output and checked regression test with force_parallel_mode = regress and all testcases are passing now. This concerns me that should we be checking all the system defined functions once again if they are actually parallel safe...? -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprised

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-24 Thread Rafia Sabih
function, and if we happen to > pass false, it's not going to matter, because exec_run_select() is > going to find the plan already initialized. > True, fixed. The attached patch is to be applied over [1]. [1] https://www.postgresql.org/message-id/CA%2BTgmoZ_ZuH%2BauEeeWnmtorPsgc_SmP

Re: [HACKERS] pgbench - allow to store select results into variables

2017-03-23 Thread Rafia Sabih
, instead of so many different files for error why don't you combine >> it into one. > > > Because a pgbench scripts stops on the first error, and I wanted to test > what happens with several kind of errors. > if (my_command->argc > 2) + syntax_error(source, lineno, my_command->li

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-22 Thread Rafia Sabih
". Because > SELECT are always marked as es->lazyEval=true(And so far we have > parallelism only for select). But here we calling the parameter to > ExecutorRun as execute_once so !fcache->returnsSet || !es->lazyEval > is the correct one and future proof. > Agree, done

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-22 Thread Rafia Sabih
ths are never set for those > rels, thus segfaulting in functions like sort_inner_and_outer() which > use those. > > Here's patch fixing both the issues. Please let me know if it fixes > the issues you are seeing. I tested the applied patch, it is fixing the reported issue. -- Regards,

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-22 Thread Rafia Sabih
ee (root=0x2df7550, tour=0x2ff2430, num_gene=6) at geqo_eval.c:199 #10 0x00741df5 in geqo_eval (root=0x2df7550, tour=0x2ff2430, num_gene=6) at geqo_eval.c:102 #11 0x0074288a in random_init_pool (root=0x2df7550, pool=0x2ff23d0) at geqo_pool.c:109 #12 0x007422a6 in geqo (root=0x2df7550,

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-22 Thread Rafia Sabih
three separate calls to ExecutorRun(), which is a >>>problem if the plan is a parallel plan. >> >> And you also need to test this case what Robert have mentioned up thread. > > +1 Checked, nope ExecutorRun is called only once in this case and execute_once is true here. -

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-21 Thread Rafia Sabih
1 IST [117823] LOG: duration: 2584.282 ms plan: Query Text: select not_parallel(); Result (cost=0.00..0.26 rows=1 width=8) (actual time=2144.315..2144.316 rows=1 loops=1) not_parallel -- 0 (1 row) Hence, it appears lazyEval is the main reason behind it and it should be

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-20 Thread Rafia Sabih
e can keep an eye on this that it doesn't exceed the computational bounds for a really large number of tables.. Please find the attached .out file to check the output I witnessed and let me know if anymore information is required Schema and data was similar to the preciously shared schema with t

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-18 Thread Rafia Sabih
46442b076a25a6df Please find the attached .sql file for the complete schema and data and .out file for the result of explain analyse with and without patch. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ pwj_regress_test.out Description: Binary data test_case_pwj.sql Description: B

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-15 Thread Rafia Sabih
r && dynamic_shared_memory_type != DSM_IMPL_NONE && parse->commandType == CMD_SELECT && !parse->hasModifyingCTE && max_parallel_workers_per_gather > 0 && !IsParallelWorker() && !IsolationIsSerializable()) > > On Fri, Mar 10, 2017 at 5:38 PM

Re: [HACKERS] pgbench - allow to store select results into variables

2017-03-14 Thread Rafia Sabih
the intention of this script anyway? Also, instead of so many different files for error why don't you combine it into one. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-13 Thread Rafia Sabih
it is the concern of this patch or not. [1] https://www.postgresql.org/message-id/ca+tgmobxehvhbjtwdupzm9bvslitj-kshxqj2um5gpdze9f...@mail.gmail.com -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ pl_parallel_opt_support_v2.patch Description: Binary data -- Sent via pgsql-hackers mailing

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-03-13 Thread Rafia Sabih
amp; > barrier->condition_variable); > > + > > + return last; > > +} > > > > Doesn't this, again, run into danger of leading to an assert failure in > > the loop in BarrierWait? > > I believe this code is correct. The assertion in BarrierWai

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-10 Thread Rafia Sabih
n it to completion. I find these two things contradictory to each other. So, is this point missed or is there some deep reasoning behind that? -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-07 Thread Rafia Sabih
llowing execution of such queries in parallel mode, that involves infrastructural changes along the lines mentioned upthread (pl_parallel_exec_support_v1.patch). -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ pl_parallel_exec_support_v1.patch Description: Binary data pl_p

[HACKERS] TPC-H Q20 from 1 hour to 19 hours!

2017-03-05 Thread Rafia Sabih
) Planning time: 2.440 ms Execution time: 6057329.179 ms I hope there might be some way-out for such a case which includes the benefits of the commit without hurting other cases (like this one) this bad. Thoughts, comments... -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ -- S

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-02-22 Thread Rafia Sabih
gt;readonly_func, > CURSOR_OPT_PARALLEL_OK); > + else > + exec_res = SPI_execute(querystr, estate->readonly_func, 0); > + } > > The last parameter of SPI_execute is tuple count, not cursorOption, > you need to fix this. Also, this is crossing the 80 line boundary

Re: [HACKERS] Parallel Index-only scan

2017-02-21 Thread Rafia Sabih
r for reviewing and testing the patch. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Passing query string to workers

2017-02-21 Thread Rafia Sabih
nd to Kuntal and Amit for reviewing. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-02-21 Thread Rafia Sabih
the parallel plans for the queries in these functions after this patch. This might be helpful in understanding the level of parallelism this patch is relaxing for PL functions. Thanks to my colleagues Amit Kapila and Dilip Kumar for discussions in this regard. -- Regards, Rafia Sabih EnterpriseDB: http

Re: [HACKERS] Passing query string to workers

2017-02-20 Thread Rafia Sabih
olates the rules > constant-correctness. So, either you should change query_data as const > char*, or as Robert suggested, you can directly use > estate->es_sourceText. > Done. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ pass_queryText_to_workers_v7.patch Descriptio

Re: [HACKERS] Passing query string to workers

2017-02-19 Thread Rafia Sabih
On Sun, Feb 19, 2017 at 10:11 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Feb 16, 2017 at 6:41 PM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: > > On Thu, Feb 16, 2017 at 5:47 PM, Rafia Sabih > > <rafia.sa...@enterprisedb.com> wrote:

Re: [HACKERS] Parallel Index-only scan

2017-02-16 Thread Rafia Sabih
dicate that for detailed > reason refer ExecReScanIndexScan. > Done. Please find the attached patch for the revised version. Just an FYI, in my recent tests on TPC-H 300 scale factor, Q16 showed improved execution time from 830 seconds to 730 seconds with this patch when used with parallel merg

Re: [HACKERS] Passing query string to workers

2017-02-16 Thread Rafia Sabih
NG, shm_toc_lookup(toc, > PARALLEL_KEY_QUERY_TEXT)); > Just one lookup is sufficient. > > Fixed. Other that that I updated some comments and other cleanup things. Please find the attached patch for the revised version. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.c

Re: [HACKERS] Parallel Index-only scan

2017-02-16 Thread Rafia Sabih
On Thu, Feb 16, 2017 at 3:40 PM, Rafia Sabih <rafia.sa...@enterprisedb.com> wrote: > > On Thu, Feb 16, 2017 at 1:26 PM, Rahila Syed <rahilasye...@gmail.com> > wrote: > >> I reviewed the patch. Overall it looks fine to me. >> >> One commen

Re: [HACKERS] Parallel Index-only scan

2017-02-16 Thread Rafia Sabih
node->iss_ScanDesc); >> >> And you've got this: >> >> + if (reset_parallel_scan) >> + index_parallelrescan(node->ioss_ScanDesc); >> > Fixed. Please find the attached patch for rebased and cleaner version. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ parallel_index_only_v6.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Passing query string to workers

2017-02-10 Thread Rafia Sabih
also be moved to execParallel.c. > > Agree and fixed. > Another question is don't we need to set debug_query_string in worker? In the updated version I am setting it in ParallelQueryMain. Please find the attached file for the revised version. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterpr

Re: [HACKERS] Passing query string to workers

2017-02-06 Thread Rafia Sabih
T max_parallel_workers_per_gather=4; > SELECT count(*) FROM t1; > > It is showing the following warning. > WARNING: problem in alloc set ExecutorState: detected write past > chunk end in block 0x14f5310, chunk 0x14f6c50 Fixed. Thanks a lot Kuntal for the review, please find attached p

[HACKERS] Redundant comment in ExecutePlan

2017-02-03 Thread Rafia Sabih
nce attached is the patch to remove this second comment. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ redundant_comment.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: h

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-02-01 Thread Rafia Sabih
On Thu, Feb 2, 2017 at 1:19 AM, Thomas Munro <thomas.mu...@enterprisedb.com> wrote: > On Thu, Feb 2, 2017 at 3:34 AM, Rafia Sabih > <rafia.sa...@enterprisedb.com> wrote: >> 9 | 62928.88 | 59077.909 > > Thanks Rafia. At first glance this plan is using the

Re: [HACKERS] Parallel Index-only scan

2017-01-19 Thread Rafia Sabih
On Fri, Jan 13, 2017 at 2:19 PM, Rafia Sabih <rafia.sa...@enterprisedb.com> wrote: > On Thu, Jan 12, 2017 at 5:39 PM, Rahila Syed <rahilasye...@gmail.com> wrote: >> Hello, >> >> On applying the patch on latest master branch and running regression tests >>

Re: [HACKERS] pgbench - allow backslash continuations in \set expressions

2017-01-16 Thread Rafia Sabih
following note to committer, I am getting whitespace errors in v3 of patch, which I corrected in v4, however, Fabien is of the opinion that v3 is clean and is showing whitespace errors because of downloader, etc. issues in my setup. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb

Re: [HACKERS] pgbench - allow backslash continuations in \set expressions

2017-01-13 Thread Rafia Sabih
lated to transformations operated when the file is downloaded > and saved, because it is a text file? > I think this is delaying the patch unnecessarily, I have attached a version, please see if you can apply it successfully, we can proceed with that safely then... -- Regards, Rafia Sabih Ent

Re: [HACKERS] Passing query string to workers

2017-01-13 Thread Rafia Sabih
tart. Later, in ExecInitParallelPlan a token for queryString is created in shared memory which is used in ExecParallelGetQueryDesc and ParallelWorkerMain as before (in version 1 of patch). Please let me know your feedback over the same. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/

Re: [HACKERS] pgbench - allow backslash continuations in \set expressions

2017-01-13 Thread Rafia Sabih
new meaning of backslash. > > > Hmmm. This is not the behavior of backslash continuation in bash or python, > I do not think that this is desirable to have a different behavior. > Okay, seems sensible. >> Otherwise, it should be mentioned in the docs that backslash should not be &

Re: [HACKERS] Parallel Index-only scan

2017-01-13 Thread Rafia Sabih
gt; Parallel Index Only Scan using tenk1_unique1 on tenk1 > ! (5 rows) > > IIUC, parallel operation being performed here is fine as parallel restricted > function occurs above Gather node > and just the expected output needs to be changed. > True, fixed it, please find the

Re: [HACKERS] Parallel bitmap heap scan

2017-01-12 Thread Rafia Sabih
0039a95f50, hjstate=0x10039a96808, hashvalue=0x3fffd4a5639c) at nodeHashjoin.c:936 I was using TPC-H with scale factor 20, please let me know if there is anything more you require in this regard. [1] https://www.postgresql.org/message-id/CAEepm%3D1vGcv6LBrxZeqPb_rPxfraidWAF_8_4z2ZMQ%2B7DOjj9

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-01-12 Thread Rafia Sabih
g to analyse the performance of TPC-H queries with your patch and came across following results, Q9 and Q21 were crashing, both of them had following bt in core dump (I thought it might be helpful), #0 0x10757da4 in pfree (pointer=0x3fff78d11000) at mcxt.c:1012 #1 0x1032c574 in ExecHashIncreaseNumBatches (hashtable=0x1003af6da60) at nodeHash.c:1124 #2 0x1032d518 in ExecHashTableInsert (hashtable=0x1003af6da60, slot=0x1003af695c0, hashvalue=2904801109, preload=1 '\001') at nodeHash.c:1700 #3 0x10330fd4 in ExecHashJoinPreloadNextBatch (hjstate=0x1003af39118) at nodeHashjoin.c:886 #4 0x103301fc in ExecHashJoin (node=0x1003af39118) at nodeHashjoin.c:376 #5 0x10308644 in ExecProcNode (node=0x1003af39118) at execProcnode.c:490 #6 0x1031f530 in fetch_input_tuple (aggstate=0x1003af38910) at nodeAgg.c:587 #7 0x10322b50 in agg_fill_hash_table (aggstate=0x1003af38910) at nodeAgg.c:2304 #8 0x1032239c in ExecAgg (node=0x1003af38910) at nodeAgg.c:1942 #9 0x10308694 in ExecProcNode (node=0x1003af38910) at execProcnode.c:509 #10 0x10302a1c in ExecutePlan (estate=0x1003af37fa0, planstate=0x1003af38910, use_parallel_mode=0 '\000', operation=CMD_SELECT, sendTuples=1 '\001', numberTuples=0, direction=ForwardScanDirection, dest=0x1003af19390) at execMain.c:1587 In case you want to know, I was using TPC-H with 20 scale factor. Please let me know if you want anymore information on this. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/

[HACKERS] Passing query string to workers

2017-01-10 Thread Rafia Sabih
l_shipdate <= date '1998-12-01' - interval '119' day group by l_returnflag, l_linestatus order by l_returnflag, l_linestatus LIMIT 1; Inputs of all sorts are encouraged. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ pass_queryText_to_workers_v1.patch Description: Bin

Re: [HACKERS] pgbench - allow backslash continuations in \set expressions

2017-01-10 Thread Rafia Sabih
d fails. I think it should be something like if after backslash some spaces are there, followed by end-of-line then it should ignore these spaces and read next line, atleast with this new meaning of backslash. Otherwise, it should be mentioned in the docs that backslash should not be followed by space. -- Reg

Re: [HACKERS] parallelize queries containing subplans

2016-12-28 Thread Rafia Sabih
rAvNjmB2cWuUHz%2Bp3ZTTtbD7o2KUw49PC8HK4r1Wg%40mail.gmail.com [2] https://www

Re: [HACKERS] Parallel Index-only scan

2016-12-27 Thread Rafia Sabih
Rebased patch of parallel-index only scan based on the latest version of parallel index scan [1] is attached. [1] https://www.postgresql.org/message-id/CAA4eK1LiNi7_Z1% 2BPCV4y06o_v%3DZdZ1UThE%2BW9JhthX4B8uifnA%40mail.gmail.com On Sat, Dec 24, 2016 at 7:55 PM, Rafia Sabih <rafia

Re: [HACKERS] Parallel Index-only scan

2016-12-24 Thread Rafia Sabih
point, unless > someone else REALLY feels the need to pile on. > > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company > -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/ parallel_index_only_v2.patch Descripti

[HACKERS] Parallel Index-only scan

2016-12-11 Thread Rafia Sabih
es improved significantly with this new operator and considering the changes required after parallel index scan patches is less if evaluated against the improvement in performance it offers. Attached file: -- 1. parallel_index_only_v1.patch This patch is to be applied over parallel i

Re: [HACKERS] pgbench - allow backslash continuations in \set expressions

2016-11-09 Thread Rafia Sabih
On Wed, Nov 9, 2016 at 3:28 PM, Fabien COELHO <coe...@cri.ensmp.fr> wrote: > > +1. My vote is for backslash continuations. >> > > I'm fine with that! > > -- > Fabien. > Looks good to me also. -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/

Re: [HACKERS] pgbench - allow backslash continuations in \set expressions

2016-11-01 Thread Rafia Sabih
anges a little more > the scanner because on each token the next state (continued or not) must be > decided. > > -- > Fabien. > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > > -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/

Re: [HACKERS] pgbench - allow backslash continuations in \set expressions

2016-11-01 Thread Rafia Sabih
e sane solution here, yes. Not adding any \ > chars at the end of the line would also mean cut-and-paste of the RHS > content would work. > > Thanks for the feedback! > > Christoph > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Regards, Rafia Sabih EnterpriseDB: http://www.enterprisedb.com/

[HACKERS] "Re: Question about grant create on database and pg_dump/pg_dumpall

2016-09-25 Thread Rafia Sabih
statements. Only if we consider the popular method of dump-restore mentioned in the thread (https://www.postgresql.org/me ssage-id/E1VYMqi-0001P4-P4%40wrigleys.postgresql.org) with pg_dumpall -g and then individual pg_dump, then it would be helpful to have this patch. Regards, Rafia Sabih EnterpriseDB