Re: [HACKERS] Setting pd_lower in GIN metapage

2017-09-11 Thread Amit Kapila
On Mon, Sep 11, 2017 at 7:18 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Sun, Sep 10, 2017 at 9:56 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >> Amit Kapila <amit.kapil...@gmail.com> writes: >>> On Sun, Sep 10, 2017 at 11:52 AM, Michael Paquier &

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-09-10 Thread Amit Kapila
On Sun, Sep 10, 2017 at 9:56 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> On Sun, Sep 10, 2017 at 11:52 AM, Michael Paquier >> <michael.paqu...@gmail.com> wrote: >>> Coordinating efforts here would be

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-09-10 Thread Amit Kapila
On Sun, Sep 10, 2017 at 11:52 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Sun, Sep 10, 2017 at 3:15 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Sat, Sep 9, 2017 at 9:00 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> Michael Paquie

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-09-10 Thread Amit Kapila
> > Yeah, I was wondering if we ought not clean up btree/hash while at it. > At the very least, their existing comments saying that it's inessential > to set pd_lower could use some more detail about why or why not. > +1. I think we can even use REGBUF_STANDARD in the hash for metapag

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-09-10 Thread Amit Kapila
t know about, which is surely possible). > > BTW, while nbtree correctly initializes pd_lower, it looks to me like it > is not exploiting that, because it seems never to pass REGBUF_STANDARD for > the metapage anyway. > During the creation of the index, it uses log_newpage to log

Re: [HACKERS] UPDATE of partition key

2017-09-09 Thread Amit Kapila
On Fri, Sep 8, 2017 at 4:51 PM, amul sul <sula...@gmail.com> wrote: > On Thu, May 18, 2017 at 9:13 AM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Wed, May 17, 2017 at 5:17 PM, Robert Haas <robertmh...@gmail.com> >> wrote: >> > On We

Re: [HACKERS] [POC] Faster processing at Gather node

2017-09-08 Thread Amit Kapila
both the problems by one patch. [1] - https://www.postgresql.org/message-id/CAOGQiiNiMhq5Pg3LiYxjfi2B9eAQ_q5YjS%3DfHiBJmbSOF74aBQ%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To mak

Re: [HACKERS] Parallel worker error

2017-09-08 Thread Amit Kapila
On Fri, Sep 8, 2017 at 3:13 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Sep 8, 2017 at 1:17 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> You are right. I have changed the ordering and passed OuterUserId via >> FixedParallelState. >

Re: [HACKERS] Parallel Append implementation

2017-09-08 Thread Amit Kapila
On Fri, Sep 8, 2017 at 3:59 PM, Amit Khandekar <amitdkhan...@gmail.com> wrote: > On 7 September 2017 at 11:05, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Thu, Aug 31, 2017 at 12:47 PM, Amit Khandekar <amitdkhan...@gm

Re: [HACKERS] Parallel worker error

2017-09-07 Thread Amit Kapila
On Fri, Sep 8, 2017 at 3:18 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Sep 4, 2017 at 5:46 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> It seems like the consensus is to move forward with this approach. I >> have written a patch implementing the abo

Re: [HACKERS] Parallel Append implementation

2017-09-06 Thread Amit Kapila
llel-aware append. Follow it's own logic of choosing * the next subplan. */ if (!exec_append_seq_next(node)) I think this is the case of non-parallel-aware appends, but the comments are indicating the opposite. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com --

[HACKERS] GatherMerge misses to push target list

2017-09-05 Thread Amit Kapila
on tests to cover above cases in the patch. [1] - https://www.postgresql.org/message-id/CAA4eK1Ji_0pgrjFotAyvvfxGikxJQEKcxD863VQ-xYtfQBy0uQ%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com pushdown_target_gathermerge_v1.patch Description: Binary dat

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2017-09-05 Thread Amit Kapila
Therefore, if even standby gets some extra space > of empty pages, it would be corrected during further vacuum cycles. > I think one deficiency of this solution is that it will keep on generating extra WAL even if standby doesn't need it (as standby has successfully truncated the relation). I don

Re: [HACKERS] why not parallel seq scan for slow functions

2017-09-05 Thread Amit Kapila
On Fri, Aug 25, 2017 at 10:08 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Aug 21, 2017 at 5:08 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> (b) I have changed the costing of gather path for path target in >> generate_gather_paths which I am not su

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2017-09-04 Thread Amit Kapila
eplica right away. We could try to wait a little > and truncate tbl1 on replica again. > Can max_standby_streaming_delay help in this situation (point number - 2)? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel worker error

2017-09-04 Thread Amit Kapila
parallel workers as this is PGC_INTERNAL guc variable. So, I have passed this value via FixedParallelState. After this patch, I think the check of InitializingParallelWorker in check_role function is redundant. I have prepared a separate patch for it, but may be it can be hand

Re: [HACKERS] Fix warnings and typo in dshash

2017-09-03 Thread Amit Kapila
On Sun, Sep 3, 2017 at 2:56 PM, Thomas Munro <thomas.mu...@enterprisedb.com> wrote: > On Sun, Sep 3, 2017 at 6:57 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I am seeing below warnings (on Win7) in dshash.c >> >> 1> dshash.c >> 1>src/backend/li

Re: [HACKERS] UPDATE of partition key

2017-09-03 Thread Amit Kapila
a206f2%40lab.ntt.co.jp >> > > After recent commit 30833ba154, now the partitions are expanded in > depth-first order. It didn't seem worthwhile rebasing my partition > walker changes onto the latest code. So in the attached patch, I have > removed all the partition walk

[HACKERS] Fix warnings and typo in dshash

2017-09-03 Thread Amit Kapila
h a separate patch is attached. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com fix_warnings_dshash_v1.patch Description: Binary data fix_typo_dshash_v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] Parallel worker error

2017-09-02 Thread Amit Kapila
on > after a concurrent ALTER ROLE removes a membership.) > I think that error won't happen during parallel initialization because of 'InitializingParallelWorker' check. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parallel worker error

2017-09-02 Thread Amit Kapila
ue of search_path GUC. I am not sure whether we need to bother about this, but I thought it might help in choosing the approach to fix this problem. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-09-01 Thread Amit Kapila
we have performed earlier. > Thanks for repeating the performance tests. > OK, these tests seem to show that this is still working. Committed, > again. Let's hope this attempt goes better than the last one. > Thanks for committing. -- With Regards, Amit Kapila. EnterpriseDB: http://w

Re: [HACKERS] parallelize queries containing initplans

2017-08-30 Thread Amit Kapila
On Mon, Aug 21, 2017 at 2:40 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Mon, Aug 21, 2017 at 1:44 PM, Haribabu Kommi > <kommi.harib...@gmail.com> wrote: >> >> >> Thanks for adding more details. It is easy to understand. >> >> I marked th

Re: [HACKERS] Parallel worker error

2017-08-30 Thread Amit Kapila
ally interdependent GUCs are bad news. > I am able to reproduce this without involving session authorization guc as well. One needs to drop the newly created role from another session, then also we can see the same error. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterpri

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-30 Thread Amit Kapila
On Tue, Aug 29, 2017 at 10:05 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> On Tue, Aug 29, 2017 at 8:32 AM, Robert Haas <robertmh...@gmail.com> wrote: >>> There's already ExecParallelReinitialize, which could be ma

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-08-30 Thread Amit Kapila
On Wed, Aug 30, 2017 at 2:43 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Jul 4, 2017 at 12:33 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I have updated the patch to support wait events and moved it to upcoming CF. > > This patch doesn't apply an

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-29 Thread Amit Kapila
master backend specific state for all nodes beneath gather. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-28 Thread Amit Kapila
On Mon, Aug 28, 2017 at 6:34 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> On Mon, Aug 28, 2017 at 6:01 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> Um, what's different about that than before? > >> Ea

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-28 Thread Amit Kapila
On Mon, Aug 28, 2017 at 6:01 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> With this change, it is quite possible that during rescans workers >> will not do any work. > > Um, what's different about that than before?

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-28 Thread Amit Kapila
dependency on the parent Gather or GatherMerge's rescan param > --- and the planner will now bitch if a parallel_aware plan node is not > under any such Gather. Is this reasonable? I think so. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Push limit to sort through a subquery

2017-08-25 Thread Amit Kapila
g this is dead code? > I also think so. I think this was required in some initial versions of gather node patch where we were thinking of having a single node (instead of what we have now that Gather node and beneath there will be partial scan node) to perform parallel scans. -- With Regard

Re: [HACKERS] Pluggable storage

2017-08-23 Thread Amit Kapila
On Wed, Aug 23, 2017 at 11:05 AM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > > On Mon, Aug 21, 2017 at 7:25 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Mon, Aug 21, 2017 at 12:58 PM, Haribabu Kommi >> <kommi.harib...@gmail.co

Re: [HACKERS] Page Scan Mode in Hash Index

2017-08-23 Thread Amit Kapila
On Tue, Aug 22, 2017 at 7:24 PM, Ashutosh Sharma <ashu.coe...@gmail.com> wrote: > On Tue, Aug 22, 2017 at 3:55 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > Okay, I got your point now. I think, currently in _hash_kill_items(), > if an overflow page is pinned w

Re: [HACKERS] Page Scan Mode in Hash Index

2017-08-22 Thread Amit Kapila
On Tue, Aug 22, 2017 at 2:28 PM, Ashutosh Sharma <ashu.coe...@gmail.com> wrote: > On Sat, Aug 19, 2017 at 11:37 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Fri, Aug 11, 2017 at 6:51 PM, Ashutosh Sharma <ashu.coe...@gmail.com> >> wrote: >>>>

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-21 Thread Amit Kapila
On Mon, Aug 21, 2017 at 3:15 PM, Simon Riggs <si...@2ndquadrant.com> wrote: > On 21 August 2017 at 10:08, Amit Kapila <amit.kapil...@gmail.com> wrote: > >> Thoughts? > > This seems like a very basic problem for parallel queries. > > The problem seems to

Re: [HACKERS] Pluggable storage

2017-08-21 Thread Amit Kapila
On Mon, Aug 21, 2017 at 12:58 PM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > On Sun, Aug 13, 2017 at 5:17 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> >> Also, it is quite possible that some of the storage Am's don't even >

Re: [HACKERS] parallelize queries containing initplans

2017-08-21 Thread Amit Kapila
On Mon, Aug 21, 2017 at 1:44 PM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > > Thanks for adding more details. It is easy to understand. > > I marked the patch as ready for committer in the commitfest. > Thank you. -- With Regards, Amit Kapila. EnterpriseDB: htt

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-21 Thread Amit Kapila
On Wed, Aug 16, 2017 at 5:04 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Aug 16, 2017 at 7:23 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Tue, Aug 15, 2017 at 7:15 PM, Robert Haas <robertmh...@gmail.com> wrote: >>> On Sat, Aug 12, 2017

Re: [HACKERS] Page Scan Mode in Hash Index

2017-08-19 Thread Amit Kapila
e still use lock chaining, so removing the above comment doesn't seem like a good idea. I think you should copy part of a comment from hasbucketcleanup starting from "There can't be any concurrent .." 3. _hash_freeovflpage() { .. * Concurrency issues are avoided by using lock c

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-18 Thread Amit Kapila
; I think that should not matter because the costing of gather is mainly based on a number of rows and that should be same for both path1 and path2 in this case. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-18 Thread Amit Kapila
an node is a good > model for what needs to happen. > Yeah, that seems like a good idea. I think another way could be to *not* optimize rescanning when we are in parallel mode (IsInParallelMode()), that might be restrictive as compared to what you are suggesting, but will be somewhat simpler. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-17 Thread Amit Kapila
On Thu, Aug 17, 2017 at 8:08 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Thu, Aug 17, 2017 at 7:49 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >> Amit Kapila <amit.kapil...@gmail.com> writes: >>> On Tue, Aug 15, 2017 at 7:16 PM, Tom Lane <t...@ss

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-17 Thread Amit Kapila
On Thu, Aug 17, 2017 at 7:49 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> On Tue, Aug 15, 2017 at 7:16 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> I should think it wouldn't be that expensive to create a test >

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-17 Thread Amit Kapila
On Thu, Aug 17, 2017 at 10:07 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Tue, Aug 15, 2017 at 7:16 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >> >>> I believe that between this commit and the test-coverage commit from >>> Andres, this open item is

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-16 Thread Amit Kapila
On Tue, Aug 15, 2017 at 7:16 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Robert Haas <robertmh...@gmail.com> writes: >> On Tue, Aug 15, 2017 at 7:31 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> Attached patch fixes the issue for me. I have locally ver

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-16 Thread Amit Kapila
On Tue, Aug 15, 2017 at 7:15 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Sat, Aug 12, 2017 at 9:18 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I think skipping a generation of gather paths for scan node or top >> level join node generated via standard

Re: [HACKERS] [BUGS] [postgresql 10 beta3] unrecognized node type: 90

2017-08-15 Thread Amit Kapila
sue for me. I have locally verified that the gather merge gets executed in rescan path. I haven't added a test case for the same as having gather or gather merge on the inner side of join can be time-consuming. However, if you or others feel that it is important to have a test to cover this c

Re: [HACKERS] parallelize queries containing initplans

2017-08-14 Thread Amit Kapila
On Sun, Aug 13, 2017 at 6:49 PM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > On Fri, Aug 11, 2017 at 1:18 AM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> > > Thanks for the updated patch. Patch looks fine. I just have some > minor comments

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-13 Thread Amit Kapila
On Mon, Aug 14, 2017 at 6:10 AM, AP <a...@zip.com.au> wrote: > On Fri, Aug 11, 2017 at 07:33:51AM +0530, Amit Kapila wrote: >> On Thu, Aug 10, 2017 at 4:11 PM, AP <a...@zip.com.au> wrote: >> > mdstash=# select * from pgstathashindex('link_datum_id_idx')

Re: [HACKERS] Pluggable storage

2017-08-13 Thread Amit Kapila
On Sat, Aug 12, 2017 at 10:34 AM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > On Tue, Aug 8, 2017 at 2:21 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> On Tue, Jun 13, 2017 at 7:20 AM, Haribabu Kommi >> <kommi.harib...@gmail.com> wrote: &

Re: [HACKERS] Pluggable storage

2017-08-13 Thread Amit Kapila
ing the hint bits. > Leaving aside the performance concern, I am not convinced that it is a good idea to remove Buffer as a parameter from the API's you have mentioned above. Would you mind thinking once again keeping the suggestions provided above in this email to see if we can avoid rem

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-12 Thread Amit Kapila
On Thu, Aug 10, 2017 at 1:07 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Aug 8, 2017 at 3:50 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Right. >> >> I see two ways to include the cost of the target list for parallel >> paths before r

Re: [HACKERS] Page Scan Mode in Hash Index

2017-08-10 Thread Amit Kapila
On Wed, Aug 9, 2017 at 2:58 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Mon, Aug 7, 2017 at 5:50 PM, Ashutosh Sharma <ashu.coe...@gmail.com> wrote: > > 7. > _hash_kill_items(IndexScanDesc scan) > { > .. > + /* > + * If page LSN differs it me

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-10 Thread Amit Kapila
. >> >> Whoa. Big improvement. > > > As an aside, btree for the above is around 2.5x bigger than hash v4 so > chances are much better that a hash index will fit into ram which has > its own benefits. :) > Yeah, that's exactly one of the benefit hash indexes can provide over

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-10 Thread Amit Kapila
On Thu, Aug 10, 2017 at 4:11 PM, AP <a...@zip.com.au> wrote: > On Sun, Aug 06, 2017 at 04:32:29PM +1000, AP wrote: >> On Sat, Aug 05, 2017 at 04:41:24PM +0530, Amit Kapila wrote: >> > > (On another note, I committed these patches.) >> > >> > Thanks

Re: [HACKERS] parallelize queries containing initplans

2017-08-10 Thread Amit Kapila
On Wed, Aug 9, 2017 at 6:51 PM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > On Wed, Aug 9, 2017 at 9:26 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> > > By the way, I tested the patch with by DML support for parallel patch to > check the retur

Re: [HACKERS] parallelize queries containing initplans

2017-08-09 Thread Amit Kapila
On Wed, Aug 9, 2017 at 10:24 AM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > > On Mon, Jul 17, 2017 at 10:53 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Tue, Mar 28, 2017 at 7:25 AM, Amit Kapila <amit.kapil...@gmail.com> >

Re: [HACKERS] Page Scan Mode in Hash Index

2017-08-09 Thread Amit Kapila
On Mon, Aug 7, 2017 at 5:50 PM, Ashutosh Sharma <ashu.coe...@gmail.com> wrote: > On Fri, Aug 4, 2017 at 7:14 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Sun, Jul 30, 2017 at 2:07 PM, Ashutosh Sharma <ashu.coe...@gmail.com> >> wrote: >>> H

Re: [HACKERS] free space % calculation in pgstathashindex

2017-08-08 Thread Amit Kapila
On Mon, Aug 7, 2017 at 9:38 PM, Ashutosh Sharma <ashu.coe...@gmail.com> wrote: > On Mon, Aug 7, 2017 at 7:19 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Mon, Aug 7, 2017 at 6:07 PM, Ashutosh Sharma <ashu.coe...@gmail.com> >> wrote: >>> Hi

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-08 Thread Amit Kapila
On Wed, Aug 2, 2017 at 11:12 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Wed, Jul 12, 2017 at 7:08 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: >> >

Re: [HACKERS] Pluggable storage

2017-08-07 Thread Amit Kapila
typedef struct StorageAmRoutine +{ In this structure, you have already covered most of the API's that a new storage module needs to provide, but I think there could be more. One such API could be heap_hot_search. This seems specific to current heap where we have the provision of HOT. I think we can pr

Re: [HACKERS] free space % calculation in pgstathashindex

2017-08-07 Thread Amit Kapila
s you need it. /* * Total space available for tuples excludes the metapage and the bitmap * pages. */ - total_space = (nblocks - (stats.bitmap_pages + 1)) * stats.space_per_page; + total_space = (uint64) (nblocks - (stats.bitmap_pages + 1)) * + (uint64) stats.space_per_

Re: [HACKERS] Pluggable storage

2017-08-07 Thread Amit Kapila
On Tue, Aug 1, 2017 at 1:56 PM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > > On Sun, Jul 23, 2017 at 4:10 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> > >> >> > 1. Design an API that returns values/nulls array and convert th

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-05 Thread Amit Kapila
On Sat, Aug 5, 2017 at 7:50 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Aug 4, 2017 at 2:45 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I have not done anything for this comment as it doesn't sound wrong to >> me. I think it is not making much sense

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-04 Thread Amit Kapila
On Fri, Aug 4, 2017 at 10:59 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Aug 4, 2017 at 6:22 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I have increased the number of hash bitmap pages as a separate patch. >> I am not completely sure if it is a goo

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-04 Thread Amit Kapila
On Fri, Aug 4, 2017 at 11:45 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Fri, Aug 4, 2017 at 11:15 PM, Robert Haas <robertmh...@gmail.com> wrote: >> On Fri, Aug 4, 2017 at 6:22 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> I have implemented t

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-04 Thread Amit Kapila
On Fri, Aug 4, 2017 at 11:15 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Aug 4, 2017 at 6:22 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I have implemented the patch with this approach as other approach >> require quite extensive changes which I am

Re: [HACKERS] Page Scan Mode in Hash Index

2017-08-04 Thread Amit Kapila
t. It makes code look better. The type of code exists few line down as well, change that as well. 5. + /* + * We save the LSN of the page as we read it, so that we know whether it + * safe to apply LP_DEAD hints to the page later. + */ "whether it safe"/"whether it is safe"

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-04 Thread Amit Kapila
On Fri, Aug 4, 2017 at 9:19 AM, AP <a...@zip.com.au> wrote: > On Fri, Aug 04, 2017 at 08:21:01AM +0530, Amit Kapila wrote: >> Note - AP has off list shared the data dump and we (Ashutosh Sharma >> and me) are able to reproduce the problem and we could see that if w

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-04 Thread Amit Kapila
On Fri, Aug 4, 2017 at 8:21 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Wed, Aug 2, 2017 at 9:04 PM, Robert Haas <robertmh...@gmail.com> wrote: >> On Wed, Jul 12, 2017 at 1:10 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>>>> Yes, I also t

Re: [HACKERS] pgsql 10: hash indexes testing

2017-08-03 Thread Amit Kapila
On Wed, Aug 2, 2017 at 9:04 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jul 12, 2017 at 1:10 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>>> Yes, I also think the same idea can be used, in fact, I have mentioned >>>> it [1] as soon as you

Re: [HACKERS] Proposal for CSN based snapshots

2017-08-01 Thread Amit Kapila
e after adding some debug printfs. I wanted to try some things with > the clog next, but for now I'm out of time. > What problem exactly you are seeing in the clog, is it the contention around CLOGControlLock or generally accessing CLOG is slower. If former, then we already have a patch [1] to

Re: LP_DEAD hinting and not holding on to a buffer pin on leaf page (Was: [HACKERS] [WIP] Zipfian distribution in pgbench)

2017-07-28 Thread Amit Kapila
not hold on to a pin on leaf > pages, should have considered workloads like this. > Isn't it possible to confirm if the problem is due to commit 2ed5b87f9? Basically, if we have unlogged tables, then it won't release the pin. So if the commit in question is the culprit, then the same wor

Re: [HACKERS] pl/perl extension fails on Windows

2017-07-25 Thread Amit Kapila
with flags that are incompatible with what we use for plperl and it is not clear why perl is using those flags. Do you think we can do something at our end to make it work or someone should check with Perl community about it? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] why not parallel seq scan for slow functions

2017-07-24 Thread Amit Kapila
On Mon, Jul 24, 2017 at 9:21 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Sat, Jul 22, 2017 at 8:53 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Thu, Jul 13, 2017 at 7:38 AM, Amit Kapila <amit.kapil...@gmail.com> >> wrote: >>

Re: [HACKERS] Pluggable storage

2017-07-23 Thread Amit Kapila
and interfaces like heap_insert, heap_update, heap_lock_tuple, SnapshotSatisfiesFunc, EvalPlanQualFetch, etc. Now, it is quite possible that we don't want to change some of these interfaces, but it can help to see how such a usage can be replaced with new kind of Tuple structure. -- With Regards, Ami

Re: [HACKERS] why not parallel seq scan for slow functions

2017-07-22 Thread Amit Kapila
On Thu, Jul 13, 2017 at 7:38 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: >> >> >> >> Setting parallel_workers to 8 changes the threshold for the parallel to even >> be

Re: [HACKERS] bug in locking an update tuple chain

2017-07-18 Thread Amit Kapila
ms logical to me, though I have not tested it till now. However, I wonder why heap_lock_tuple need to restart from the beginning of update-chain in this case? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-17 Thread Amit Kapila
e can claim that we don't recommend hash indexes to be used prior to 10 in production, so such an error is okay even if there is no crash has happened in the system. [1] - https://www.postgresql.org/docs/devel/static/indexam.html [2] - https://www.postgresql.org/docs/devel/static/index-functions.

Re: [HACKERS] parallelize queries containing initplans

2017-07-17 Thread Amit Kapila
On Tue, Mar 28, 2017 at 7:25 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Thu, Mar 16, 2017 at 2:34 AM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> On Tue, Mar 14, 2017 at 3:20 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> Based

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-14 Thread Amit Kapila
On Fri, Jul 14, 2017 at 6:02 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > (catching up finally with this thread) > > On Mon, Jul 10, 2017 at 11:57 AM, Kyotaro HORIGUCHI > <horiguchi.kyot...@lab.ntt.co.jp> wrote: >> At Mon, 10 Jul 2017 14:58:13

Re: [HACKERS] why not parallel seq scan for slow functions

2017-07-12 Thread Amit Kapila
On Wed, Jul 12, 2017 at 11:20 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Tue, Jul 11, 2017 at 10:25 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Wed, Jul 12, 2017 at 1:50 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: >> > On Mon, J

Re: [HACKERS] why not parallel seq scan for slow functions

2017-07-11 Thread Amit Kapila
rs = ..)) > So the attached patch improves things, but doesn't go far enough. > It seems to that we need to adjust the cost based on if the below node is projection capable. See attached. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com subpath_projection_cost.2.pa

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-11 Thread Amit Kapila
On Tue, Jul 11, 2017 at 11:08 PM, Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > Amit Kapila wrote: > >> Yes, I also think the same idea can be used, in fact, I have mentioned >> it [1] as soon as you have committed that patch. Do we want to do >> anything at

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-11 Thread Amit Kapila
On Tue, Jul 11, 2017 at 8:10 AM, Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > Amit Kapila wrote: >> On Tue, Jul 11, 2017 at 6:51 AM, AP <a...@zip.com.au> wrote: >> > On Fri, Jul 07, 2017 at 05:58:25PM +0530, Amit Kapila wrote: > >> >> I can unders

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-10 Thread Amit Kapila
On Tue, Jul 11, 2017 at 6:51 AM, AP <a...@zip.com.au> wrote: > On Fri, Jul 07, 2017 at 05:58:25PM +0530, Amit Kapila wrote: >> On Fri, Jul 7, 2017 at 8:22 AM, AP <a...@zip.com.au> wrote: >> > On Thu, Jul 06, 2017 at 05:19:59PM +0530, Amit Kapila wrote: >> >

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-10 Thread Amit Kapila
On Mon, Jul 10, 2017 at 3:27 PM, Kyotaro HORIGUCHI <horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hi, > > At Mon, 10 Jul 2017 14:58:13 +0530, Amit Kapila <amit.kapil...@gmail.com> > wrote in

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-10 Thread Amit Kapila
On Mon, Jul 10, 2017 at 10:39 AM, Kyotaro HORIGUCHI <horiguchi.kyot...@lab.ntt.co.jp> wrote: > At Sat, 8 Jul 2017 16:41:27 +0530, Amit Kapila <amit.kapil...@gmail.com> > wrote in <caa4ek1+-duto+myendle9p9u8g3fv6n+sojpsqmpsw6ashh...@mail.gmail.com> >> On Sat, Jul 8

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-07-10 Thread Amit Kapila
> Benchmark have been done in master v10. I am attaching image with results: > . > > It will be interesting to see what the profiling data with perf says about this for PostgreSQL. Can you try to get the perf report? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-08 Thread Amit Kapila
On Sat, Jul 8, 2017 at 9:06 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Sat, Jul 8, 2017 at 8:52 AM, Robert Haas <robertmh...@gmail.com> wrote: >> On Thu, Jul 6, 2017 at 5:54 AM, Kyotaro HORIGUCHI >> <horiguchi.kyot...@lab.ntt.co.jp> wrote: >&g

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-07 Thread Amit Kapila
On Sat, Jul 8, 2017 at 9:08 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Jul 7, 2017 at 11:36 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I think we should do that as a separate patch (I can write the same as >> well) because that is not new behavi

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-07 Thread Amit Kapila
t may be needed around here. > > Yeah, it should probably mention that the init fork of an unlogged > relation is also OK. > I think we should do that as a separate patch (I can write the same as well) because that is not new behavior introduced by this patch, but let me know if you thi

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-07 Thread Amit Kapila
On Fri, Jul 7, 2017 at 8:22 AM, AP <a...@zip.com.au> wrote: > On Thu, Jul 06, 2017 at 05:19:59PM +0530, Amit Kapila wrote: >> I think if you are under development, it is always advisable to create >> indexes after initial bulk load. That way it will be faster and will >>

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-06 Thread Amit Kapila
On Thu, Jul 6, 2017 at 5:04 PM, AP <a...@zip.com.au> wrote: > On Thu, Jul 06, 2017 at 12:38:38PM +0530, Amit Kapila wrote: >> On Thu, Jul 6, 2017 at 9:32 AM, AP <a...@zip.com.au> wrote: >> > On Thu, Jul 06, 2017 at 08:52:03AM +0530, Amit Kapila wrote: >> &

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-06 Thread Amit Kapila
On Thu, Jul 6, 2017 at 3:24 PM, Kyotaro HORIGUCHI <horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hello, > > At Tue, 4 Jul 2017 14:49:26 +0530, Amit Kapila <amit.kapil...@gmail.com> > wrote in <CAA4eK1+SYqCmA7ioTBpJHcO-B-rf8A=n9gr1-rp3rhwecb5...@mail.gmail.com> >

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-06 Thread Amit Kapila
On Thu, Jul 6, 2017 at 9:32 AM, AP <a...@zip.com.au> wrote: > On Thu, Jul 06, 2017 at 08:52:03AM +0530, Amit Kapila wrote: >> On Thu, Jul 6, 2017 at 2:40 AM, AP <a...@zip.com.au> wrote: >> > On Wed, Jul 05, 2017 at 05:52:32PM +0530, Amit Kapila wrote: >

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-07-06 Thread Amit Kapila
On Wed, Jul 5, 2017 at 6:25 PM, Mithun Cy <mithun...@enterprisedb.com> wrote: > On Mon, Jul 3, 2017 at 11:58 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Sun, Jul 2, 2017 at 10:32 PM, Mithun Cy <mithun...@enterprisedb.com> >> wrote: >>> O

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-07-05 Thread Amit Kapila
On Wed, Jul 5, 2017 at 6:25 PM, Mithun Cy <mithun...@enterprisedb.com> wrote: > On Mon, Jul 3, 2017 at 3:34 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> Few comments on the latest patch: >> >> 1. >> + LWLockRelease(_state->lock); >>

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-05 Thread Amit Kapila
On Thu, Jul 6, 2017 at 2:40 AM, AP <a...@zip.com.au> wrote: > On Wed, Jul 05, 2017 at 05:52:32PM +0530, Amit Kapila wrote: >> >> > version | bucket_pages | overflow_pages | bitmap_pages | unused_pages >> >> > | l

Re: [HACKERS] Request more documentation for incompatibility of parallelism and plpgsql exec_run_select

2017-07-05 Thread Amit Kapila
On Wed, Jul 5, 2017 at 9:44 AM, Mark Dilger <hornschnor...@gmail.com> wrote: > >> On Jul 3, 2017, at 10:25 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> On Mon, Jul 3, 2017 at 8:57 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >>> O

<    1   2   3   4   5   6   7   8   9   10   >