Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

> On 28 Jul 2017, at 16:46, Robert Haas wrote: > > On Fri, Jul 28, 2017 at 12:39 AM, Pavan Deolasee > wrote: >> I see your point. But I would like to think this way: does the technology >> significantly help many common use cases, that are

Re: In-place index updates and HOT (Was: [HACKERS] Patch: Write Amplification Reduction Method (WARM))

Claudio Freire wrote: README.HOT says that that cost is not worth the benefit of preventing a new index write, but I think that it ought to take into account that not all index writes are equal. There is an appreciable difference between inserting a new tuple, and

Re: In-place index updates and HOT (Was: [HACKERS] Patch: Write Amplification Reduction Method (WARM))

On Fri, Jul 28, 2017 at 8:32 PM, Peter Geoghegan wrote: > README.HOT says that that cost is not worth the benefit of > preventing a new index write, but I think that it ought to take into > account that not all index writes are equal. There is an appreciable > difference between

In-place index updates and HOT (Was: [HACKERS] Patch: Write Amplification Reduction Method (WARM))

Pavan Deolasee wrote: One good thing is that the patch is ready and fully functional. So that allows those who are keen to run real performance tests and see the actual impact of the patch. Very true. I see your point. But I would like to think this way: does the

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Jul 28, 2017 at 12:39 AM, Pavan Deolasee wrote: > I see your point. But I would like to think this way: does the technology > significantly help many common use cases, that are currently not addressed > by HOT? It probably won't help all workloads, that's given.

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Jul 28, 2017 at 5:57 AM, Peter Geoghegan wrote: > Pavan Deolasee wrote: > > I'll be happy if someone wants to continue hacking the patch further and > > get it in a committable shape. I can stay actively involved. But TBH the > > amount of time I

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

Pavan Deolasee wrote: > I'll be happy if someone wants to continue hacking the patch further and > get it in a committable shape. I can stay actively involved. But TBH the > amount of time I can invest is far as compared to what I could during the > last cycle. That's

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Jul 26, 2017 at 6:26 PM, Robert Haas wrote: > On Tue, Apr 18, 2017 at 4:25 AM, Pavan Deolasee > wrote: > > I'll include the fix in the next set of patches. > > I haven't see a new set of patches. Are you intending to continue > working

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Apr 18, 2017 at 4:25 AM, Pavan Deolasee wrote: > I'll include the fix in the next set of patches. I haven't see a new set of patches. Are you intending to continue working on this? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Apr 14, 2017 at 9:21 PM, Jaime Casanova wrote: > > > Hi Pavan, > > I run a test on current warm patchset, i used pgbench with a scale of > 20 and a fillfactor of 90 and then start the pgbench run with 6 > clients in parallel i also run sqlsmith on it. > >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On 5 April 2017 at 13:32, Pavan Deolasee wrote: > > Ok. I've extensively updated the README to match the current state of > affairs. Updated patch set attached. Hi Pavan, I run a test on current warm patchset, i used pgbench with a scale of 20 and a fillfactor of 90

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Apr 12, 2017 at 10:42 PM, Robert Haas wrote: > On Tue, Apr 11, 2017 at 1:20 PM, Pavan Deolasee > > > 5. Added code to set a CLEAR pointer to a WARM pointer when we know that > the > > entire chain is WARM. This should address the workload Dilip ran and > found >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Apr 13, 2017 at 2:04 AM, Peter Geoghegan wrote: > On Wed, Apr 12, 2017 at 10:12 AM, Robert Haas > wrote: > >> I may have missed something, but there is no intention to ignore known > >> regressions/reviews. Of course, I don't think that every

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Apr 12, 2017 at 10:12 AM, Robert Haas wrote: >> I may have missed something, but there is no intention to ignore known >> regressions/reviews. Of course, I don't think that every regression will be >> solvable, like if you run a CPU-bound workload, setting it up in

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Apr 11, 2017 at 1:20 PM, Pavan Deolasee wrote: > I don't know why you say that regressions are not addressed. Here are a few > things I did to address the regressions/reviews/concerns, apart from fixing > all the bugs discovered, but please let me know if there

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Apr 12, 2017 at 9:23 AM, Amit Kapila wrote: > On Tue, Apr 11, 2017 at 10:50 PM, Pavan Deolasee > > > > > And I fixed them as quickly as humanly possible. > > > > Yes, you have responded to them quickly, but I didn't get a chance to > re-verify all of those.

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Apr 11, 2017 at 10:50 PM, Pavan Deolasee wrote: > > > On Tue, Apr 11, 2017 at 7:10 PM, Robert Haas wrote: >> >> >> >> Yes, and as Andres says, you don't help with those, and then you're >> upset when your own patch doesn't get attention. >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Mon, Apr 10, 2017 at 04:34:50PM -0700, Andres Freund wrote: > Hi, > > > On 2017-04-08 23:36:13 +0530, Pavan Deolasee wrote: > > On Wed, Apr 5, 2017 at 11:57 PM, Andres Freund wrote: > > > > > On 2017-04-05 09:36:47 -0400, Robert Haas wrote: > > > > By the way, the

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Apr 11, 2017 at 7:10 PM, Robert Haas wrote: > > > Yes, and as Andres says, you don't help with those, and then you're > upset when your own patch doesn't get attention. I am not upset, I was obviously a bit disappointed which I think is a very natural emotion

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Sat, Apr 8, 2017 at 2:06 PM, Pavan Deolasee wrote: > Thank you all for the reviews, feedback, tests, criticism. And apologies > for keep pushing it till the last minute even though it was clear to me > quite some time back the patch is not going to make it. But if

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

Hi, On 2017-04-08 23:36:13 +0530, Pavan Deolasee wrote: > On Wed, Apr 5, 2017 at 11:57 PM, Andres Freund wrote: > > > On 2017-04-05 09:36:47 -0400, Robert Haas wrote: > > > By the way, the "Converting WARM chains back to HOT chains" section of > > > README.WARM seems to be

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Sat, Apr 8, 2017 at 11:36:13PM +0530, Pavan Deolasee wrote: > Thank you all for the  reviews, feedback, tests, criticism. And apologies for > keep pushing it till the last minute even though it was clear to me quite some > time back the patch is not going to make it. But if I'd given up, it

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Apr 5, 2017 at 11:57 PM, Andres Freund wrote: > On 2017-04-05 09:36:47 -0400, Robert Haas wrote: > > By the way, the "Converting WARM chains back to HOT chains" section of > > README.WARM seems to be out of date. Any chance you could update that > > to reflect the

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Apr 6, 2017 at 12:20 AM, Peter Geoghegan wrote: > On Wed, Apr 5, 2017 at 11:27 AM, Andres Freund wrote: > > I propose we move this patch to the next CF. > > I agree. I think it's too late to be working out fine details around > TOAST like this. This is

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Apr 6, 2017 at 1:06 AM, Robert Haas wrote: > On Wed, Apr 5, 2017 at 2:32 PM, Pavan Deolasee > wrote: > > > The other way is to pass old tuple values along with the new tuple > values to > > amwarminsert, build index tuples and then do a

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Apr 5, 2017 at 2:32 PM, Pavan Deolasee wrote: >> The only other idea that I have for a really clean solution here is to >> support this only for index types that are amcanreturn, and actually >> compare the value stored in the index tuple with the one stored in

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Apr 5, 2017 at 11:27 AM, Andres Freund wrote: > I propose we move this patch to the next CF. I agree. I think it's too late to be working out fine details around TOAST like this. This is a patch that touches the storage format in a fairly fundamental way. The idea of

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On 2017-04-05 09:36:47 -0400, Robert Haas wrote: > By the way, the "Converting WARM chains back to HOT chains" section of > README.WARM seems to be out of date. Any chance you could update that > to reflect the current state and thinking of the patch? I propose we move this patch to the next CF.

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Apr 4, 2017 at 11:43 PM, Pavan Deolasee wrote: > Well, better than causing a deadlock ;-) Yep. > Lets see if we want to go down the path of blocking WARM when tuples have > toasted attributes. I submitted a patch yesterday, but having slept over it, > I think I

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Apr 5, 2017 at 8:42 AM, Robert Haas wrote: > On Tue, Apr 4, 2017 at 10:21 PM, Pavan Deolasee > wrote: > > On Thu, Mar 30, 2017 at 7:55 PM, Robert Haas > wrote: > >> but > >> try to access the TOAST table would be

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Apr 4, 2017 at 10:21 PM, Pavan Deolasee wrote: > On Thu, Mar 30, 2017 at 7:55 PM, Robert Haas wrote: >> but >> try to access the TOAST table would be fatal; that probably would have >> deadlock hazards among other problems. > > Hmm. I

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 7:55 PM, Robert Haas wrote: > but > try to access the TOAST table would be fatal; that probably would have > deadlock hazards among other problems. Hmm. I think you're right. We could make a copy of the heap tuple, drop the lock and then access

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 11:17 PM, Bruce Momjian wrote: > On Tue, Mar 21, 2017 at 04:04:58PM -0400, Bruce Momjian wrote: > > On Tue, Mar 21, 2017 at 04:56:16PM -0300, Alvaro Herrera wrote: > > > Bruce Momjian wrote: > > > > On Tue, Mar 21, 2017 at 04:43:58PM -0300, Alvaro

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Mar 31, 2017 at 12:31 PM, Dilip Kumar wrote: > On Thu, Mar 30, 2017 at 5:27 PM, Amit Kapila > wrote: > > I am not sure if we can consider it as completely synthetic because we > > might see some similar cases for json datatypes. Can we

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Mar 31, 2017 at 11:54 PM, Pavan Deolasee wrote: > > On Fri, Mar 31, 2017 at 11:16 PM, Robert Haas wrote: >> >> Now, I understand you to be suggesting a flag at >> table-creation time that would, maybe, be immutable after that, but >>

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 4:13 AM, Pavan Deolasee wrote: > > > On Thu, Mar 30, 2017 at 3:29 PM, Dilip Kumar > wrote: > >> On Wed, Mar 29, 2017 at 11:51 AM, Pavan Deolasee >> wrote: >> > Thanks. I think your patch of

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Mar 31, 2017 at 11:16 PM, Robert Haas wrote: > On Fri, Mar 31, 2017 at 10:24 AM, Pavan Deolasee > wrote: > > Having worked on it for some time now, I can say that WARM uses pretty > much > > the same infrastructure that HOT uses for

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Mar 31, 2017 at 10:24 AM, Pavan Deolasee wrote: > Having worked on it for some time now, I can say that WARM uses pretty much > the same infrastructure that HOT uses for cleanup/pruning tuples from the > heap. So the risk of having a bug which can eat your data

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 10:49 AM, Petr Jelinek wrote: > While reading this thread I am thinking if we could just not do WARM on > TOAST and compressed values if we know there might be regressions there. > I mean I've seen the problem WARM tries to solve mostly on

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Mar 31, 2017 at 6:47 PM, Robert Haas wrote: > > > 2. WARM is a non-optional feature which touches the on-disk format. > There is nothing more dangerous than that. If hash indexes have bugs, > people can avoid those bugs by not using them; there are good reasons >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Mar 31, 2017 at 7:53 AM, Simon Riggs wrote: > So Andres says defer this, but Robert says "more review", which is > more than just deferral. > > We have some risky things in this release such as Hash Indexes, > function changes. I perfectly understand that perception

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On 30 March 2017 at 16:50, Robert Haas wrote: > On Thu, Mar 30, 2017 at 11:41 AM, Andres Freund wrote: >> On 2017-03-30 16:43:41 +0530, Pavan Deolasee wrote: >>> Looks like OID conflict to me.. Please try rebased set. >> >> Pavan, Alvaro, everyone: I

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 5:27 PM, Amit Kapila wrote: > I am not sure if we can consider it as completely synthetic because we > might see some similar cases for json datatypes. Can we once try to > see the impact when the same test runs from multiple clients? For > your

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 21, 2017 at 04:04:58PM -0400, Bruce Momjian wrote: > On Tue, Mar 21, 2017 at 04:56:16PM -0300, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > On Tue, Mar 21, 2017 at 04:43:58PM -0300, Alvaro Herrera wrote: > > > > Bruce Momjian wrote: > > > > > > > > > I don't think it makes

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 11:41 AM, Andres Freund wrote: > On 2017-03-30 16:43:41 +0530, Pavan Deolasee wrote: >> Looks like OID conflict to me.. Please try rebased set. > > Pavan, Alvaro, everyone: I know you guys are working very hard on this, > but I think at this point it's

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

Hi, On 2017-03-30 16:43:41 +0530, Pavan Deolasee wrote: > Looks like OID conflict to me.. Please try rebased set. Pavan, Alvaro, everyone: I know you guys are working very hard on this, but I think at this point it's too late to commit this for v10. This is patch that's affecting the on-disk

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On 30/03/17 16:04, Pavan Deolasee wrote: > > > On Thu, Mar 30, 2017 at 7:27 PM, Robert Haas > wrote: > > On Thu, Mar 30, 2017 at 6:37 AM, Pavan Deolasee > > wrote: > > I

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 10:08 AM, Amit Kapila wrote: > I think we should not consider doing compression and decompression as > free at this point in code, because we hold a buffer lock during > recheck. Buffer locks are meant for short-term locks (it is even > mentioned

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 5:55 PM, Pavan Deolasee wrote: > > On Thu, Mar 30, 2017 at 5:27 PM, Amit Kapila > wrote: >> >> >> >> How have you verified that? Have you checked that in >> heap_prepare_insert it has called toast_insert_or_update() and

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 7:27 PM, Robert Haas wrote: > On Thu, Mar 30, 2017 at 6:37 AM, Pavan Deolasee > wrote: > > I think we can fix that by comparing compressed values. I know you had > > raised concerns, but Robert confirmed that (IIUC) it's

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 6:37 AM, Pavan Deolasee wrote: > I think we can fix that by comparing compressed values. I know you had > raised concerns, but Robert confirmed that (IIUC) it's not a problem today. I'm not sure that's an entirely fair interpretation of what I

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 5:27 PM, Amit Kapila wrote: > > > How have you verified that? Have you checked that in > heap_prepare_insert it has called toast_insert_or_update() and then > returned a tuple different from what the input tup is? Basically, I > am easily able

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 30, 2017 at 4:07 PM, Pavan Deolasee wrote: > > On Wed, Mar 29, 2017 at 4:42 PM, Amit Kapila > wrote: >> >> On Wed, Mar 29, 2017 at 1:10 PM, Pavan Deolasee >> wrote: >> > >> > On Wed, Mar 29, 2017 at 12:02

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Mar 29, 2017 at 4:42 PM, Amit Kapila wrote: > On Wed, Mar 29, 2017 at 1:10 PM, Pavan Deolasee > wrote: > > > > On Wed, Mar 29, 2017 at 12:02 PM, Amit Kapila > > wrote: > >> > >> On Wed, Mar 29, 2017 at 11:52

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Mar 29, 2017 at 11:51 AM, Pavan Deolasee wrote: > Thanks. I think your patch of tracking interesting attributes seems ok too > after the performance issue was addressed. Even though we can still improve > that further, at least Mithun confirmed that there is no

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

Pavan Deolasee wrote: > On Wed, Mar 29, 2017 at 3:42 AM, Alvaro Herrera > wrote: > > > I pushed 0002 after some makeup, since it's just cosmetic and not > > controversial. > > Thanks. I think your patch of tracking interesting attributes seems ok too > after the

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Mar 29, 2017 at 7:12 AM, Amit Kapila wrote: > No as I agreed above, it won't double-compress, but still looks > slightly risky to rely on different set of values passed to > index_form_tuple and then compare them. It assumes that the compressor is completely

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Mar 29, 2017 at 1:10 PM, Pavan Deolasee wrote: > > On Wed, Mar 29, 2017 at 12:02 PM, Amit Kapila > wrote: >> >> On Wed, Mar 29, 2017 at 11:52 AM, Amit Kapila >> wrote: > > Then during recheck, we pass already

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Mar 29, 2017 at 12:02 PM, Amit Kapila wrote: > On Wed, Mar 29, 2017 at 11:52 AM, Amit Kapila > wrote: > > On Tue, Mar 28, 2017 at 10:35 PM, Pavan Deolasee > > wrote: > >> > >> On Tue, Mar 28, 2017 at 7:04 PM,

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 10:31 PM, Pavan Deolasee wrote: > > On Tue, Mar 28, 2017 at 4:05 PM, Amit Kapila > wrote: >> >> As asked previously, can you explain me on what basis are you >> considering it robust? The comments on top of

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Mar 29, 2017 at 11:52 AM, Amit Kapila wrote: > On Tue, Mar 28, 2017 at 10:35 PM, Pavan Deolasee > wrote: >> >> On Tue, Mar 28, 2017 at 7:04 PM, Amit Kapila >> wrote: >>> >>> For such an heap insert, we will

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 10:35 PM, Pavan Deolasee wrote: > > On Tue, Mar 28, 2017 at 7:04 PM, Amit Kapila > wrote: >> >> >> >> For such an heap insert, we will pass >> the actual value of column to index_form_tuple during index insert. >>

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

I pushed 0002 after some makeup, since it's just cosmetic and not controversial. Here's 0003 rebased on top of it. (Also, I took out the gin and gist changes: it would be wrong to change that unconditionally, because the 0x pattern appears in indexes that would be pg_upgraded. We need a

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 8:34 PM, Robert Haas wrote: > On Mon, Mar 27, 2017 at 10:25 PM, Pavan Deolasee > wrote: > > On Tue, Mar 28, 2017 at 1:59 AM, Robert Haas > wrote: > >> On Thu, Mar 23, 2017 at 2:47 PM, Pavan Deolasee

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 7:04 PM, Amit Kapila wrote: > > > For such an heap insert, we will pass > the actual value of column to index_form_tuple during index insert. > However during recheck when we fetch the value of c2 from heap tuple > and pass it index tuple, the

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 4:07 PM, Amit Kapila wrote: > > Noted few cosmetic issues in 0005_warm_updates_v21: > > 1. > pruneheap.c(939): warning C4098: 'heap_get_root_tuples' : 'void' > function returning a value > Thanks. Will fix. > > 2. > + * HCWC_WARM_UPDATED_TUPLE

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 4:05 PM, Amit Kapila wrote: > > > As asked previously, can you explain me on what basis are you > considering it robust? The comments on top of datumIsEqual() clearly > indicates the danger of using it for toasted values (Also, it will > probably

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

Hi Pavan, On 3/28/17 11:04 AM, Robert Haas wrote: On Mon, Mar 27, 2017 at 10:25 PM, Pavan Deolasee wrote: On Tue, Mar 28, 2017 at 1:59 AM, Robert Haas wrote: On Thu, Mar 23, 2017 at 2:47 PM, Pavan Deolasee wrote:

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Mon, Mar 27, 2017 at 10:25 PM, Pavan Deolasee wrote: > On Tue, Mar 28, 2017 at 1:59 AM, Robert Haas wrote: >> On Thu, Mar 23, 2017 at 2:47 PM, Pavan Deolasee >> wrote: >> > It's quite hard to say that until we see

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 4:05 PM, Amit Kapila wrote: > On Mon, Mar 27, 2017 at 2:19 PM, Pavan Deolasee > wrote: >> On Fri, Mar 24, 2017 at 11:49 PM, Pavan Deolasee >> wrote: >>> >>> >>> >>> On Fri, Mar 24, 2017 at 6:46

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Mon, Mar 27, 2017 at 2:19 PM, Pavan Deolasee wrote: > > > On Fri, Mar 24, 2017 at 11:49 PM, Pavan Deolasee > wrote: >> >> >> >> On Fri, Mar 24, 2017 at 6:46 PM, Amit Kapila >> wrote: >>> >>> >>> >>> I was worried

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Mon, Mar 27, 2017 at 2:19 PM, Pavan Deolasee wrote: > On Fri, Mar 24, 2017 at 11:49 PM, Pavan Deolasee > wrote: >> >> >> >> On Fri, Mar 24, 2017 at 6:46 PM, Amit Kapila >> wrote: >>> >>> >>> >>> I was worried for

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 1:32 AM, Alvaro Herrera wrote: > Is the WARM tap test suite supposed to work when applied without all the > other patches? I just tried applied that one and running "make check -C > src/test/modules", and it seems to hang after giving "ok 5" for

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Mon, Mar 27, 2017 at 4:45 PM, Amit Kapila wrote: > On Sat, Mar 25, 2017 at 1:24 PM, Amit Kapila > wrote: > > On Fri, Mar 24, 2017 at 11:49 PM, Pavan Deolasee > > wrote: > >> > >> On Fri, Mar 24, 2017 at 6:46 PM,

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 08:04:34AM +0530, Pavan Deolasee wrote: > And I've answered it so many times by now :-)  LOL > Just to add more to what I just said in another email, note that HOT/WARM > chains are created when a new root line pointer is created in the heap (a line > pointer that has an

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 7:49 AM, Bruce Momjian wrote: > On Mon, Mar 27, 2017 at 04:29:56PM -0400, Robert Haas wrote: > > On Thu, Mar 23, 2017 at 2:47 PM, Pavan Deolasee > > wrote: > > > It's quite hard to say that until we see many more benchmarks. As

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 1:59 AM, Robert Haas wrote: > On Thu, Mar 23, 2017 at 2:47 PM, Pavan Deolasee > wrote: > > It's quite hard to say that until we see many more benchmarks. As author > of > > the patch, I might have got repetitive with my

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Mon, Mar 27, 2017 at 04:29:56PM -0400, Robert Haas wrote: > On Thu, Mar 23, 2017 at 2:47 PM, Pavan Deolasee > wrote: > > It's quite hard to say that until we see many more benchmarks. As author of > > the patch, I might have got repetitive with my benchmarks. But I've

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Tue, Mar 28, 2017 at 1:32 AM, Alvaro Herrera wrote: > Is the WARM tap test suite supposed to work when applied without all the > other patches? I just tried applied that one and running "make check -C > src/test/modules", and it seems to hang after giving "ok 5" for

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 2:47 PM, Pavan Deolasee wrote: > It's quite hard to say that until we see many more benchmarks. As author of > the patch, I might have got repetitive with my benchmarks. But I've seen > over 50% improvement in TPS even without chain conversion (6

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

Is the WARM tap test suite supposed to work when applied without all the other patches? I just tried applied that one and running "make check -C src/test/modules", and it seems to hang after giving "ok 5" for t/002_warm_stress.pl. (I had to add a Makefile too, attached.) -- Álvaro Herrera

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Sat, Mar 25, 2017 at 1:24 PM, Amit Kapila wrote: > On Fri, Mar 24, 2017 at 11:49 PM, Pavan Deolasee > wrote: >> >> On Fri, Mar 24, 2017 at 6:46 PM, Amit Kapila >> wrote: >>> > >> While looking at this problem, it

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Sat, Mar 25, 2017 at 11:24 PM, Pavan Deolasee wrote: > > On Sat, 25 Mar 2017 at 11:03 PM, Peter Geoghegan wrote: >> >> On Sat, Mar 25, 2017 at 12:54 AM, Amit Kapila >> wrote: >> > I am not sure how do you want to binary compare

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Sat, 25 Mar 2017 at 11:03 PM, Peter Geoghegan wrote: > On Sat, Mar 25, 2017 at 12:54 AM, Amit Kapila > wrote: > > I am not sure how do you want to binary compare two datums, if you are > > thinking datumIsEqual(), that won't work. I think you need to

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Sat, Mar 25, 2017 at 12:54 AM, Amit Kapila wrote: > I am not sure how do you want to binary compare two datums, if you are > thinking datumIsEqual(), that won't work. I think you need to use > datatype specific compare function something like what we do in >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Mar 24, 2017 at 11:49 PM, Pavan Deolasee wrote: > > > On Fri, Mar 24, 2017 at 6:46 PM, Amit Kapila > wrote: >> >> >> >> I was worried for the case if the index is created non-default >> collation, will the datumIsEqual() suffice the

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Mar 24, 2017 at 6:46 PM, Amit Kapila wrote: > > > I was worried for the case if the index is created non-default > collation, will the datumIsEqual() suffice the need. Now again > thinking about it, I think it will because in the index tuple we are > storing the

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 3:54 PM, Pavan Deolasee wrote: > Thanks Amit. v19 addresses some of the comments below. > > On Thu, Mar 23, 2017 at 10:28 AM, Amit Kapila > wrote: >> >> On Wed, Mar 22, 2017 at 4:06 PM, Amit Kapila

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Mar 24, 2017 at 4:04 PM, Amit Kapila wrote: > On Fri, Mar 24, 2017 at 12:25 AM, Pavan Deolasee > wrote: > > > > > > On Thu, Mar 23, 2017 at 7:53 PM, Amit Kapila > > > > The general sense I've got > > here is

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Fri, Mar 24, 2017 at 12:25 AM, Pavan Deolasee wrote: > > > On Thu, Mar 23, 2017 at 7:53 PM, Amit Kapila > wrote: >> >> > >> >> I am not sure on what basis user can set such parameters, it will be >> quite difficult to tune those parameters.

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 7:53 PM, Amit Kapila wrote: > On Thu, Mar 23, 2017 at 3:44 PM, Pavan Deolasee > > > > > Yes, this is a very fair point. The way I proposed to address this > upthread > > is by introducing a set of threshold/scale GUCs specific to WARM. So > users

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 11:44 PM, Mithun Cy wrote: > Hi Pavan, > On Thu, Mar 23, 2017 at 12:19 AM, Pavan Deolasee > wrote: > > Ok, no problem. I did some tests on AWS i2.xlarge instance (4 vCPU, 30GB > > RAM, attached SSD) and results are

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

Hi Pavan, On Thu, Mar 23, 2017 at 12:19 AM, Pavan Deolasee wrote: > Ok, no problem. I did some tests on AWS i2.xlarge instance (4 vCPU, 30GB > RAM, attached SSD) and results are shown below. But I think it is important > to get independent validation from your side too,

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 3:44 PM, Pavan Deolasee wrote: > On Wed, Mar 22, 2017 at 4:06 PM, Amit Kapila > wrote: >> > >> 3. >> + /* >> + * HASH indexes compute a hash value of the key and store that in the >> + * index. So >> we must first obtain

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 4:08 PM, Pavan Deolasee wrote: > > > On Thu, Mar 23, 2017 at 3:02 PM, Amit Kapila > wrote: > >> >> >> That sounds like you are dodging the actual problem. I mean you can >> put that same PageIsFull() check in master

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 3:02 PM, Amit Kapila wrote: > > > That sounds like you are dodging the actual problem. I mean you can > put that same PageIsFull() check in master code as well and then you > will most probably again see the same regression. Well I don't see it

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

Thanks Amit. v19 addresses some of the comments below. On Thu, Mar 23, 2017 at 10:28 AM, Amit Kapila wrote: > On Wed, Mar 22, 2017 at 4:06 PM, Amit Kapila > wrote: > > On Tue, Mar 21, 2017 at 6:47 PM, Pavan Deolasee > >

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 12:19 AM, Pavan Deolasee wrote: > > Ok, no problem. I did some tests on AWS i2.xlarge instance (4 vCPU, 30GB > RAM, attached SSD) and results are shown below. But I think it is important > to get independent validation from your side too, just to

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Wed, Mar 22, 2017 at 4:06 PM, Amit Kapila wrote: > On Tue, Mar 21, 2017 at 6:47 PM, Pavan Deolasee > wrote: >> >>> >> >> Please find attached rebased patches. >> > > Few comments on 0005_warm_updates_v18.patch: > Few more comments on

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Thu, Mar 23, 2017 at 12:19 AM, Pavan Deolasee wrote: > > > On Wed, Mar 22, 2017 at 4:53 PM, Mithun Cy > wrote: > Ok, no problem. I did some tests on AWS i2.xlarge instance (4 vCPU, 30GB > RAM, attached SSD) and results are shown below. But

  1   2   3   >