Re: [HACKERS] UPDATE of partition key

2017-05-18 Thread Amit Kapila
On Wed, May 17, 2017 at 4:05 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Wed, May 17, 2017 at 3:15 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> Earlier I thought that option1 is better but later I think that this >>> can complicate the situation

Re: [HACKERS] UPDATE of partition key

2017-05-17 Thread Amit Kapila
On Wed, May 17, 2017 at 5:17 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, May 17, 2017 at 6:29 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I think we can do this even without using an additional infomask bit. >> As suggested by Greg up thread, we can

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-17 Thread Amit Kapila
fix all transaction control > commands in a symmetric manner. > +1. Why not similar behavior for any other statements executed in this module by do_sql_command? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-

Re: [HACKERS] UPDATE of partition key

2017-05-17 Thread Amit Kapila
On Mon, May 15, 2017 at 5:28 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, May 12, 2017 at 3:07 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I agree with you that it might not be straightforward to make it work, >> but now that earliest it can go is v

Re: [HACKERS] UPDATE of partition key

2017-05-17 Thread Amit Kapila
the situation as we are firing first BR update then BR > delete and can change the row multiple time and defining such > behaviour can be complicated. > If we have to go by this theory, then the option you have preferred will still execute BR triggers for both delete and inse

Re: [HACKERS] Increasing parallel workers at runtime

2017-05-17 Thread Amit Kapila
ew worker so that it can participate in a > running query. I may be wrong, but the execution state initialization > routines are written with the assumption that all the workers start > simultaneously? > No such assumptions, workers started later can also join the execution of the q

Re: [HACKERS] Increasing parallel workers at runtime

2017-05-17 Thread Amit Kapila
ed previously during the development of Parallel Sequential Scan patch and I think some other databases uses some similar technique for this problem. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql

Re: [HACKERS] bumping HASH_VERSION to 3

2017-05-16 Thread Amit Kapila
On Tue, May 16, 2017 at 5:14 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, May 16, 2017 at 7:31 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > >> I will send an updated patch once we agree on above points. > > Sounds good. > Attached patch addres

Re: [HACKERS] NOT NULL constraints on range partition key columns

2017-05-16 Thread Amit Kapila
On Tue, May 16, 2017 at 3:26 PM, Amit Langote <langote_amit...@lab.ntt.co.jp> wrote: > On 2017/05/16 4:29, Robert Haas wrote: >> On Mon, May 15, 2017 at 9:12 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> Can't we allow NULL to get inserted into the partition (l

Re: [HACKERS] bumping HASH_VERSION to 3

2017-05-16 Thread Amit Kapila
se the hash indexes to be invalided in the old cluster > rather than the new one? > oops. copy-paste. It passed in my testing because I have not used any different options (like port number) for old or new server. > This might need a visit from pgindent in one or two places, too. > I

Re: [HACKERS] NOT NULL constraints on range partition key columns

2017-05-15 Thread Amit Kapila
erted into the partition (leaf partition) if the user uses the partition name in Insert statement? For root partitions, I think for now giving an error is okay, but once we have default partitions (Rahila's patch), we can route NULLS to default partition. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH v2] Progress command to monitor progression of long running SQL queries

2017-05-13 Thread Amit Kapila
or parallel worker as it will just end the query execution. Also, even if it is okay, there doesn't seem to be a way by which a parallel worker can communicate the error back to master backend, rather it will just exit silently which is not right. -- With Regards, Amit Kapila. EnterpriseDB: http

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-05-13 Thread Amit Kapila
ught to be noninterruptible anyway. > During parallel bulk load operations, I think we hold it over multiple kernel calls. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-05-13 Thread Amit Kapila
n an e-mail [1]. [1] - https://www.postgresql.org/message-id/CAFiTN-tkX6gs-jL8VrPxg6OG9VUAKnObUq7r7pWQqASzdF5OwA%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hash Functions

2017-05-12 Thread Amit Kapila
er than > giving up. > Can we think of defining separate portable hash functions which can be used for the purpose of hash partitioning? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] UPDATE of partition key

2017-05-12 Thread Amit Kapila
On Fri, May 12, 2017 at 10:49 AM, Amit Khandekar <amitdkhan...@gmail.com> wrote: > On 12 May 2017 at 08:30, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Thu, May 11, 2017 at 5:41 PM, Amit Khandekar <amitdkhan...@gmail.com> >> wrote: > >> If we

Re: [HACKERS] UPDATE of partition key

2017-05-12 Thread Amit Kapila
d to address this point and see if we can come up with something doable without a big architecture change. What is your take on this point now? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Kapila
On Fri, May 12, 2017 at 9:27 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Thu, May 11, 2017 at 5:45 PM, Amit Khandekar <amitdkhan...@gmail.com> > wrote: >> On 11 May 2017 at 17:24, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> Few comments: >

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Kapila
On Thu, May 11, 2017 at 5:45 PM, Amit Khandekar <amitdkhan...@gmail.com> wrote: > On 11 May 2017 at 17:24, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Few comments: >> 1. >> Operating directly on partition doesn't allow update to move row. >> Refer b

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Kapila
On Thu, May 11, 2017 at 5:41 PM, Amit Khandekar <amitdkhan...@gmail.com> wrote: > On 11 May 2017 at 17:23, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Fri, Mar 17, 2017 at 4:07 PM, Amit Khandekar <amitdkhan...@gmail.com> >> wrote: >>> On 4

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Kapila
rtition patch of Rahila or that patch needs to take care this error case? Basically, if there is no matching partition, then move it to default partition. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] UPDATE of partition key

2017-05-11 Thread Amit Kapila
th Insert and Update statement level triggres (as per below note in docs) whereas the documentation in the patch indicates that this patch will only fire Update statement level triggers which is odd. Note in docs about Insert On Conflict "Note that with an INSERT with an ON CONFLICT DO UPDATE clause

Re: [HACKERS] Remove pre-10 compatibility code in hash index

2017-05-10 Thread Amit Kapila
On Wed, May 10, 2017 at 9:17 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, May 9, 2017 at 1:41 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Commit ea69a0dead5128c421140dc53fac165ba4af8520 has bumped the hash >> index version and obviates the need for

Re: [HACKERS] bumping HASH_VERSION to 3

2017-05-10 Thread Amit Kapila
sion_old_8_3.c::old_8_3_invalidate_hash_gin_indexes() > Thanks for the pointer. > I would be happy to restore that code and make it work for PG 10. > Attached patch implements the two points suggested by Tom. I will add this to PG-10 open issues list so that we don't forget about

[HACKERS] Remove pre-10 compatibility code in hash index

2017-05-08 Thread Amit Kapila
compatibility code in hash index. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com remove_pre10_compat_hash_index_v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

2017-05-08 Thread Amit Kapila
on); \ I think we don't need to use TransactionIdLimitedForOldSnapshots() as that is required to override xmin for table vacuum/pruning purposes. > Maybe we need > to use GetOldestXmin()? It is a costly call as it needs ProcArrayLock, so I don't think it makes sense to use it here. -- Wit

Re: [HACKERS] Atomics for heap_parallelscan_nextpage()

2017-05-08 Thread Amit Kapila
h is confusing. * We don't guarantee any specific ordering in general, though. */ if (scan->rs_syncscan) ss_report_location(scan->rs_rd, page); .. } -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-06 Thread Amit Kapila
ard compatibility, changing it in >> back-branches doesn't seem like a good plan. I'll adjust the patch so >> that it continues to ignore errors in that case. > > Updated patch attached. > Patch looks good to me. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprise

Re: [HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

2017-05-06 Thread Amit Kapila
Can you explain me, what value for xmin should be used there? > I think we can use RecentGlobalDataXmin for non-catalog relations and RecentGlobalXmin for catalog relations (probably a check similar to what we have in heap_page_prune_opt). -- With Regards, Amit Kapila. EnterpriseDB: http://www.e

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-05 Thread Amit Kapila
On Sat, May 6, 2017 at 4:41 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Fri, May 5, 2017 at 9:32 PM, Robert Haas <robertmh...@gmail.com> wrote: >> On Fri, May 5, 2017 at 11:15 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >>> I am not say

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-05 Thread Amit Kapila
On Fri, May 5, 2017 at 9:32 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, May 5, 2017 at 11:15 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I am not saying to rip those changes. Your most of the mail stresses >> about the 30-second timeout which

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-05 Thread Amit Kapila
On Fri, May 5, 2017 at 7:44 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, May 5, 2017 at 1:01 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> I have tried to verify above point and it seems to me in such a >> situation the transaction status wil

Re: [HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

2017-05-05 Thread Amit Kapila
+bool +HeapTupleSatisfiesNonVacuumable(HeapTuple htup, Snapshot snapshot, + Buffer buffer) +{ + return HeapTupleSatisfiesVacuum(htup, snapshot->xmin, buffer) + != HEAPTUPLE_DEAD; +} + Add comments on top of this function and for the sake of consistency update the file header as well (Summary o

Re: [HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

2017-05-05 Thread Amit Kapila
On Fri, May 5, 2017 at 11:57 AM, Andres Freund <and...@anarazel.de> wrote: > Hi, > > On 2017-05-05 11:50:12 +0530, Amit Kapila wrote: >> I see that EndPos can be updated in one of the cases after releasing >> the lock (refer below code). Won't that matter? > >

Re: [HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

2017-05-05 Thread Amit Kapila
On Fri, May 5, 2017 at 11:43 AM, Andres Freund <and...@anarazel.de> wrote: > On 2017-05-05 11:04:14 +0530, Amit Kapila wrote: >> On Fri, May 5, 2017 at 6:54 AM, Andres Freund <and...@anarazel.de> wrote: >> > Hi, >> > >> > On 2016-12-22 19:33:30 +0

Re: [HACKERS] modeling parallel contention (was: Parallel Append implementation)

2017-05-05 Thread Amit Kapila
anywhere near to linear scaling by improving the same. [1] - https://www.pgcon.org/2015/schedule/events/785.en.html -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Change GetLastImportantRecPtr's definition? (wasSkip checkpoints, archiving on idle systems.)

2017-05-04 Thread Amit Kapila
ord, then that sounds sensible thing to do. However, note that we remember the position based on lockno and lock is released before we can determine the EndPos. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-04 Thread Amit Kapila
On Thu, May 4, 2017 at 10:40 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, May 4, 2017 at 1:04 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Thu, May 4, 2017 at 10:18 PM, Robert Haas <robertmh...@gmail.com> wrote: >>> On Thu, May 4, 2017

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-04 Thread Amit Kapila
On Thu, May 4, 2017 at 8:08 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, May 4, 2017 at 7:13 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> In pgfdw_xact_callback, if the execution of ABORT TRANSACTION fails >> due to any reason then I think i

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-04 Thread Amit Kapila
On Thu, May 4, 2017 at 10:18 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, May 4, 2017 at 12:18 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> As soon as the first command fails due to timeout, we will set >> 'abort_cleanup_failure' which will

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-04 Thread Amit Kapila
On Thu, May 4, 2017 at 8:08 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, May 4, 2017 at 7:13 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > >>> - For bonus points, give pgfdw_exec_query() an optional timeout >>> argument, and set it to 30

Re: [HACKERS] statement_timeout is not working as expected with postgres_fdw

2017-05-04 Thread Amit Kapila
xec_query() > instead. If we exit pgfdw_exec_query() with an error, we'll re-enter > the abort path, but now in_abort_cleanup will be set, so we'll just > drop the connection (and force any outer transaction levels to abort > as well). > > - For bonus points, give pgfdw_exec_query() an op

Re: [HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

2017-05-01 Thread Amit Kapila
If that is the case, then how would using SnapshotAny solve this problem. We get the value from index first and then check its visibility in heap, so if time is spent in _bt_checkkeys, why would using a different kind of Snapshot solve the problem? -- With Regards, Amit Kapila. EnterpriseDB: http:/

Re: [HACKERS] RFC: ALTER SYSTEM [...] COMMENT

2017-05-01 Thread Amit Kapila
[1] - https://www.postgresql.org/message-id/CA%2BTgmoZBDLhDexHyTJ%3DH0xZt7-6M%3Dh%2Bv2Xi0Myj7Q37QGZQb4g%40mail.gmail.com [2] - https://www.postgresql.org/message-id/CAA4eK1%2B%3DovYQqYGHcX2OBU3mk%2BhSHjFDpvmrHCos1Vgj8_b6vg%40mail.gmail.com -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterpr

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-30 Thread Amit Kapila
On Mon, May 1, 2017 at 10:23 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> Yeah, that's right. Today, I have spent some time to analyze how and >> where retry logic is required. I think there are two places where we &g

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-30 Thread Amit Kapila
s unless we want to add some special logic for handling process exit or alternatively we might want to just try reattach operation in a loop before giving up. Do we want to keep this retry logic for a certain number of times say (10 times) or we want to try indefinitely? -- With Regards, Amit

Re: [HACKERS] PG 10 release notes

2017-04-27 Thread Amit Kapila
On Thu, Apr 27, 2017 at 7:53 PM, Marko Tiikkaja <ma...@joh.to> wrote: > On Thu, Apr 27, 2017 at 4:13 PM, Bruce Momjian <br...@momjian.us> wrote: >> >> On Thu, Apr 27, 2017 at 08:00:28AM +0530, Amit Kapila wrote: >> > > Oh, so non-correlated

Re: [HACKERS] PG 10 release notes

2017-04-26 Thread Amit Kapila
of hash indexes. >> >> Yes, it is unfotunate that the item is in the incompatibility item. I >> wonder if I should split out the need to rebuild the hash indexes and >> keep it there and move this item into the "Index" section. > > Done, items split. >

Re: [HACKERS] PG 10 release notes

2017-04-26 Thread Amit Kapila
On Wed, Apr 26, 2017 at 8:46 PM, Bruce Momjian <br...@momjian.us> wrote: > On Wed, Apr 26, 2017 at 07:38:05PM +0530, Amit Kapila wrote: >> >> I have already mentioned the commit id (5e6d8d2b). Text can be "Allow >> >> queries containing subplans to

Re: [HACKERS] PG 10 release notes

2017-04-26 Thread Amit Kapila
On Tue, Apr 25, 2017 at 7:19 PM, Bruce Momjian <br...@momjian.us> wrote: > On Tue, Apr 25, 2017 at 09:00:45AM +0530, Amit Kapila wrote: >> On Tue, Apr 25, 2017 at 8:35 AM, Bruce Momjian <br...@momjian.us> wrote: >> > On Tue, Apr 25, 2017 at 08:30:50AM +0530, Amit Kapi

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Amit Kapila
Just wondering if the mention of commit >> 0414b26bac09379a4cbf1fbd847d1cee2293c5e4 is missed? Not sure if this >> requires a separate entry or could be merged with -- Support parallel >> btree index scans. > > This item was merged into the general parallel index scan item: &

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2017-04-25 Thread Amit Kapila
tions. > Can we do few modifications like: improve those latencies --> reduce those latencies such slow standby --> a slow standby -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

2017-04-24 Thread Amit Kapila
On Tue, Apr 25, 2017 at 8:35 AM, Bruce Momjian <br...@momjian.us> wrote: > On Tue, Apr 25, 2017 at 08:30:50AM +0530, Amit Kapila wrote: >> On Tue, Apr 25, 2017 at 7:01 AM, Bruce Momjian <br...@momjian.us> wrote: >> > I have committed the first draft of

Re: [HACKERS] PG 10 release notes

2017-04-24 Thread Amit Kapila
On Tue, Apr 25, 2017 at 8:30 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Tue, Apr 25, 2017 at 7:01 AM, Bruce Momjian <br...@momjian.us> wrote: >> I have committed the first draft of the Postgres 10 release notes. They >> are current as of two days ago, a

Re: [HACKERS] PG 10 release notes

2017-04-24 Thread Amit Kapila
the parallel query in many cases. ea69a0dead5128c421140dc53fac165ba4af8520 Expand hash indexes more gradually. I think the above commit needs a separate mention, as this is a really huge step forward to control the size of hash indexes. -- With Regards, Amit Kapila. EnterpriseDB: http://ww

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-24 Thread Amit Kapila
On Fri, Apr 21, 2017 at 12:55 AM, Andres Freund <and...@anarazel.de> wrote: > On 2017-04-20 16:57:03 +0530, Amit Kapila wrote: >> On Wed, Apr 19, 2017 at 9:01 PM, Andres Freund <and...@anarazel.de> wrote: >> > On 2017-04-19 10:15:31 -0400, Tom Lane wrote: >> >

Re: [HACKERS] dtrace probes

2017-04-20 Thread Amit Kapila
there any commit in PG-10 which has caused this behavior? If not, then I don't think it should be added to open items of PG-10. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] OK, so culicidae is *still* broken

2017-04-19 Thread Amit Kapila
logs.msdn.microsoft.com/winsdk/2009/11/30/how-to-disable-address-space-layout-randomization-aslr/ -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Inadequate parallel-safety check for SubPlans

2017-04-18 Thread Amit Kapila
On Wed, Apr 19, 2017 at 1:27 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> I have ended up doing something along the lines suggested by you (or >> at least what I have understood from your e-mail). Basically, pass

Re: [HACKERS] Inadequate parallel-safety check for SubPlans

2017-04-18 Thread Amit Kapila
On Mon, Apr 17, 2017 at 10:54 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> On Sun, Apr 16, 2017 at 9:30 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> FYI, I have this on my to-look-at list, and expect to fix it bef

Re: [HACKERS] Inadequate parallel-safety check for SubPlans

2017-04-16 Thread Amit Kapila
letely sure whether we need to pass any sort of whitelist of params for parallel safety, but if you think that is the better way to go, I can give it a try. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Typo in htup_details.h

2017-04-15 Thread Amit Kapila
Attached patch to fix $SUBJECT. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com typo_htup_details.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] [sqlsmith] ERROR: badly formatted node string "RESTRICTINFO...

2017-04-12 Thread Amit Kapila
On Wed, Apr 12, 2017 at 10:30 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: >> On Wed, Apr 12, 2017 at 12:40 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> Anyone want to draft a patch for this? > >> Please

Re: [HACKERS] Inadequate parallel-safety check for SubPlans

2017-04-12 Thread Amit Kapila
asonable. Do you want to have a go at that? > > Yeah. I'm knocking off for the day a bit early today, but I'll have > a look at it tomorrow, unless anyone in the Far East beats me to it. > Thanks for looking into it. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] TAP tests take a long time

2017-04-12 Thread Amit Kapila
re if Reindex Index hash_split_index is required for code coverage, but we might need it for the sake of testing that operation. Mithun, as you are the original author of these tests, can you please try some of the above optimizations and any others you can think of and see if we can reduce t

Re: [HACKERS] [sqlsmith] ERROR: badly formatted node string "RESTRICTINFO...

2017-04-12 Thread Amit Kapila
On Wed, Apr 12, 2017 at 12:40 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: > >> However, the worker will >> never execute such a plan as we don't generate a plan where unsafe >> sublan/initplan is referenced in t

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-11 Thread Amit Kapila
ddress the situation Amit brought up > where only one of the indexes receive WARM inserts. > 4. Added code to kill wrong index pointers to do online cleanup. > 5. Added code to set a CLEAR pointer to a WARM pointer when we know that the > entire chain is WARM. This should address the

Re: [HACKERS] [sqlsmith] ERROR: badly formatted node string "RESTRICTINFO...

2017-04-11 Thread Amit Kapila
On Wed, Apr 12, 2017 at 12:40 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Amit Kapila <amit.kapil...@gmail.com> writes: > >> I think there is a possibility of doing ExecInitNode in a parallel >> worker for a parallel-unsafe subplan, because we pass a list of all &g

Re: [HACKERS] [sqlsmith] ERROR: badly formatted node string "RESTRICTINFO...

2017-04-11 Thread Amit Kapila
nedStmt or maybe keep a parallel safe flag in Plan so that we can pass only parallel safe plans to workers. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-05 Thread Amit Kapila
dWorkerShmemInit() simply reset the >> values back to 0? Or do we call ForgetBackgroundWorker() after the crash for >> some reason? > AFAICU, during crash recovery, we wait for all non-syslogger children > to exit, then reset shmem(call BackgroundWorkerShmemInit) and perform > Start

Re: [HACKERS] increasing the default WAL segment size

2017-04-04 Thread Amit Kapila
max_wal_size and min_wal_size to mb. > > Committed first part to allow internal representation change (only). > > No commitment yet to increasing wal-segsize in the way this patch has it. > What part of patch you don't like and do you have any suggestions to improve the same? --

Re: [HACKERS] Page Scan Mode in Hash Index

2017-04-04 Thread Amit Kapila
p pin > on a bucket buf. There won't be any case where we will be having pin > on overflow buf at the end of scan. > What if mark the buffer as invalid after releasing the pin? We are already doing it that way in btree code, refer _bt_drop_lock_and_maybe_pin(). I think if we do that way,

Re: [HACKERS] Page Scan Mode in Hash Index

2017-04-04 Thread Amit Kapila
xtPage = (opaque)->hasho_nextblkno; What makes you think it is safe read opaque after releasing the lock? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [sqlsmith] Failed assertion in _hash_kill_items/MarkBufferDirtyHint

2017-04-01 Thread Amit Kapila
gligible chance of getting that patch, then it is good to proceed with this fix. [1] - https://commitfest.postgresql.org/13/999/ -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Page Scan Mode in Hash Index

2017-04-01 Thread Amit Kapila
quiring lock on the next page. */ .. + next_buf = _hash_getbuf_with_strategy(rel, blkno, HASH_WRITE, + LH_OVERFLOW_PAGE, + bstrategy); + After this change, you need to modify comments on top of function hashbucketcleanup() and _hash_squeezebucket(). -- With Regards, Amit Kapila. EnterpriseDB: http://www.ente

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-01 Thread Amit Kapila
additional design to handle it. Do you think it make sense to have a separate thread to discuss and get feedback on same as I am not seeing much input on the knobs you are proposing to handle second pass over index? -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via p

Re: [HACKERS] TPC-H Q20 from 1 hour to 19 hours!

2017-03-31 Thread Amit Kapila
ich I haven't really tested beyond checking that it passes 'make > check'. > Your patch looks good to me. I have verified some join cases as well where the behaviour is sane after patch. I have also done testing with force_parallel_mode=regress (ran make check-world) and everything seems goo

Re: [HACKERS] Supporting huge pages on Windows

2017-03-31 Thread Amit Kapila
On Fri, Mar 31, 2017 at 8:05 AM, Tsunakawa, Takayuki <tsunakawa.ta...@jp.fujitsu.com> wrote: > From: Amit Kapila [mailto:amit.kapil...@gmail.com] >> The latest patch looks good to me apart from one Debug message, so I have >> marked it as Ready For Committer.

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-30 Thread Amit Kapila
On Thu, Mar 30, 2017 at 5:55 PM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > On Thu, Mar 30, 2017 at 5:27 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> >> >> How have you verified that? Have you checked that in >> heap_

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-30 Thread Amit Kapila
On Thu, Mar 30, 2017 at 4:07 PM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > On Wed, Mar 29, 2017 at 4:42 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Wed, Mar 29, 2017 at 1:10 PM, Pavan Deolasee >> <pavan.deola...@gmail.com> w

Re: [HACKERS] Supporting huge pages on Windows

2017-03-30 Thread Amit Kapila
I think this should be similar to what we display in sysv_shmem.c as below: elog(DEBUG1, "mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m", allocsize); -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing l

Re: [HACKERS] inconsistent page found on STANDBY server

2017-03-30 Thread Amit Kapila
other similar problem, but couldn't find any. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-29 Thread Amit Kapila
On Wed, Mar 29, 2017 at 1:10 PM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > On Wed, Mar 29, 2017 at 12:02 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> On Wed, Mar 29, 2017 at 11:52 AM, Amit Kapila <amit.kapil...@gmail.com> >> wr

Re: [HACKERS] [POC] A better way to expand hash indexes.

2017-03-29 Thread Amit Kapila
On Wed, Mar 29, 2017 at 12:51 PM, Mithun Cy <mithun...@enterprisedb.com> wrote: > On Wed, Mar 29, 2017 at 10:12 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Few other comments: >> +/* >> + * This is just a trick to save a division operation. If you look

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-29 Thread Amit Kapila
On Tue, Mar 28, 2017 at 10:31 PM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > On Tue, Mar 28, 2017 at 4:05 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> As asked previously, can you explain me on what basis are you >> considering it robus

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-29 Thread Amit Kapila
On Wed, Mar 29, 2017 at 11:52 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Tue, Mar 28, 2017 at 10:35 PM, Pavan Deolasee > <pavan.deola...@gmail.com> wrote: >> >> On Tue, Mar 28, 2017 at 7:04 PM, Amit Kapila <amit.kapil...@gmail.com> >> wr

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-29 Thread Amit Kapila
On Tue, Mar 28, 2017 at 10:35 PM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > On Tue, Mar 28, 2017 at 7:04 PM, Amit Kapila <amit.kapil...@gmail.com> > wrote: >> >> >> >> For such an heap insert, we will pass >> the actual valu

Re: [HACKERS] [POC] A better way to expand hash indexes.

2017-03-28 Thread Amit Kapila
PHASE. I think you should name this define as SPLITPOINT_PHASE_WITHIN_GROUP as this refers to only one particular phase within group. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Amit Kapila
On Tue, Mar 28, 2017 at 4:05 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Mon, Mar 27, 2017 at 2:19 PM, Pavan Deolasee > <pavan.deola...@gmail.com> wrote: >> On Fri, Mar 24, 2017 at 11:49 PM, Pavan Deolasee <pavan.deola...@gmail.com> >> wrote: >>

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Amit Kapila
On Mon, Mar 27, 2017 at 2:19 PM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > > On Fri, Mar 24, 2017 at 11:49 PM, Pavan Deolasee <pavan.deola...@gmail.com> > wrote: >> >> >> >> On Fri, Mar 24, 2017 at 6:46 PM, Amit Kapila <amit.kapil...

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-28 Thread Amit Kapila
On Mon, Mar 27, 2017 at 2:19 PM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > On Fri, Mar 24, 2017 at 11:49 PM, Pavan Deolasee <pavan.deola...@gmail.com> > wrote: >> >> >> >> On Fri, Mar 24, 2017 at 6:46 PM, Amit Kapila <amit.kapil...@gmail

Re: [HACKERS] [POC] A better way to expand hash indexes.

2017-03-28 Thread Amit Kapila
On Tue, Mar 28, 2017 at 10:43 AM, Mithun Cy <mithun...@enterprisedb.com> wrote: > On Mon, Mar 27, 2017 at 11:21 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > > > As you have said we can solve it if we allocate buckets always in > power-of-2 when we do hash index m

Re: [HACKERS] parallelize queries containing initplans

2017-03-27 Thread Amit Kapila
On Thu, Mar 16, 2017 at 2:34 AM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: > On Tue, Mar 14, 2017 at 3:20 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Based on that idea, I have modified the patch such that it will >> compute the set of initplans Params tha

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-27 Thread Amit Kapila
On Sat, Mar 25, 2017 at 1:24 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Fri, Mar 24, 2017 at 11:49 PM, Pavan Deolasee > <pavan.deola...@gmail.com> wrote: >> >> On Fri, Mar 24, 2017 at 6:46 PM, Amit Kapila <amit.kapil...@gmail.com> >> wro

Re: [HACKERS] [POC] A better way to expand hash indexes.

2017-03-27 Thread Amit Kapila
On Mon, Mar 27, 2017 at 11:21 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Sun, Mar 26, 2017 at 11:26 AM, Mithun Cy <mithun...@enterprisedb.com> > wrote: >> Thanks, Amit for the review. >> On Sat, Mar 25, 2017 at 7:03 PM, Amit Kapila <amit.kapil...@gmai

Re: [HACKERS] [POC] A better way to expand hash indexes.

2017-03-26 Thread Amit Kapila
On Sun, Mar 26, 2017 at 11:26 AM, Mithun Cy <mithun...@enterprisedb.com> wrote: > Thanks, Amit for the review. > On Sat, Mar 25, 2017 at 7:03 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> >> I think one-dimensional patch has fewer places to touch, so that look

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-25 Thread Amit Kapila
On Sat, Mar 25, 2017 at 11:24 PM, Pavan Deolasee <pavan.deola...@gmail.com> wrote: > > On Sat, 25 Mar 2017 at 11:03 PM, Peter Geoghegan <p...@bowt.ie> wrote: >> >> On Sat, Mar 25, 2017 at 12:54 AM, Amit Kapila <amit.kapil...@gmail.com> >> wrote: >> &

Re: [HACKERS] [POC] A better way to expand hash indexes.

2017-03-25 Thread Amit Kapila
On Sat, Mar 25, 2017 at 10:13 AM, Mithun Cy <mithun...@enterprisedb.com> wrote: > On Tue, Mar 21, 2017 at 8:16 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> Sure, I was telling you based on that. If you are implicitly treating >> it as 2-dimensional array, it

Re: [HACKERS] crashes due to setting max_parallel_workers=0

2017-03-25 Thread Amit Kapila
ong > with just using SET max_parallel_workers = 0; > > My vote would be to leave the GUC behaviour as is and add some tests > to select_parallel.sql which exploit setting max_parallel_workers to 0 > for running some tests. > I think force_parallel_mode=regress should test the same thing.

Re: [HACKERS] Add pgstathashindex() to get hash index table statistics.

2017-03-25 Thread Amit Kapila
On Sat, Mar 25, 2017 at 12:33 PM, Ashutosh Sharma <ashu.coe...@gmail.com> wrote: > On Sat, Mar 25, 2017 at 11:02 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Thu, Mar 23, 2017 at 11:24 PM, Ashutosh Sharma <ashu.coe...@gmail.com> >> wrote: >>>

<    1   2   3   4   5   6   7   8   9   10   >