Re: Locking B-tree leafs immediately in exclusive mode

2018-07-27 Thread Alexander Korotkov
On Fri, Jul 27, 2018 at 6:11 PM Alexander Korotkov wrote: > On Fri, Jul 27, 2018 at 12:30 AM Simon Riggs wrote: > > On 26 July 2018 at 20:59, Alexander Korotkov > > wrote: > > > > > Great, thank you! So, I think the regression is demystified. We can > > > now conclude that on our benchmarks t

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-27 Thread Alexander Korotkov
On Fri, Jul 27, 2018 at 12:30 AM Simon Riggs wrote: > On 26 July 2018 at 20:59, Alexander Korotkov > wrote: > > > Great, thank you! So, I think the regression is demystified. We can > > now conclude that on our benchmarks this patch doesn't cause > > performance regression larger than measurem

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-26 Thread Simon Riggs
On 26 July 2018 at 20:59, Alexander Korotkov wrote: > Great, thank you! So, I think the regression is demystified. We can > now conclude that on our benchmarks this patch doesn't cause > performance regression larger than measurement error. But in some > cases it shows huge performance benefit

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-26 Thread Alexander Korotkov
Hi! On Thu, Jul 26, 2018 at 1:31 PM Imai, Yoshikazu wrote: > On Wed, July 25, 2018 at 0:28 AM, Alexander Korotkov wrote:> > On Tue, Jul > 10, 2018 at 4:39 PM 今井 良一 wrote: > > I've reread the thread. And I found that in my initial letter [1] I > > forget to include index definition. > > CREATE

RE: Locking B-tree leafs immediately in exclusive mode

2018-07-26 Thread Imai, Yoshikazu
On Wed, July 25, 2018 at 0:30 AM, Alexander Korotkov wrote: > > Lefted tasks in my review is doing the regression tests. > > Cool, thank you for review! > I did "make check-world" in patched version and all tests were passed successfully! Yoshikazu Imai

RE: Locking B-tree leafs immediately in exclusive mode

2018-07-26 Thread Imai, Yoshikazu
Hi. On Wed, July 25, 2018 at 0:28 AM, Alexander Korotkov wrote: > Hi! > > On Tue, Jul 10, 2018 at 4:39 PM 今井 良一 wrote: > > On 2018/07/10 20:36, Alexander Korotkov wrote: > > > Thank you for the experiments! It seems that there is real regression > > > here... BTW, which script were you using in

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-25 Thread Alexander Korotkov
On Wed, Jul 25, 2018 at 5:54 AM Imai, Yoshikazu wrote: > On Mon, July 9, 2018 at 3:19 AM, Imai, Yoshikazu wrote: > > I'm planning to do code review and send it in the next mail. > > Sorry for delaying the code review. > > I did the code review, and I think there are no logical wrongs > with B-Tree

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-25 Thread Alexander Korotkov
Hi! On Wed, Jul 25, 2018 at 1:19 PM Simon Riggs wrote: > On 13 July 2018 at 03:14, Imai, Yoshikazu > wrote: > > From an attached graph("some_contention_points_on_leaf_nodes.png"), as > > contention points dispersed, we can see that TPS is increased and TPS > > difference between master and pa

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-25 Thread Alexander Korotkov
Hi! On Tue, Jul 10, 2018 at 4:39 PM 今井 良一 wrote: > On 2018/07/10 20:36, Alexander Korotkov wrote: > > Thank you for the experiments! It seems that there is real regression > > here... BTW, which script were you using in this benchmark: > > script_unordered.sql or script_duplicated.sql? > > Sorr

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-25 Thread Simon Riggs
On 13 July 2018 at 03:14, Imai, Yoshikazu wrote: > On Mon, July 9, 2018 at 5:25 PM, Simon Riggs wrote: >> Please can you check insertion with the index on 2 keys >> 1st key has 10,000 values >> 2nd key has monotonically increasing value from last 1st key value >> >> So each session picks one 1st k

RE: Locking B-tree leafs immediately in exclusive mode

2018-07-24 Thread Imai, Yoshikazu
On Mon, July 9, 2018 at 3:19 AM, Imai, Yoshikazu wrote: > I'm planning to do code review and send it in the next mail. Sorry for delaying the code review. I did the code review, and I think there are no logical wrongs with B-Tree. I tested integrity of B-Tree with amcheck just in case. I execut

RE: Locking B-tree leafs immediately in exclusive mode

2018-07-12 Thread Imai, Yoshikazu
On Mon, July 9, 2018 at 5:25 PM, Simon Riggs wrote: > Please can you check insertion with the index on 2 keys > 1st key has 10,000 values > 2nd key has monotonically increasing value from last 1st key value > > So each session picks one 1st key value > Then each new INSERTion is a higher value of

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-10 Thread 今井 良一
On 2018/07/10 20:36, Alexander Korotkov wrote: > On Tue, Jul 10, 2018 at 2:19 PM Imai, Yoshikazu > wrote: >> On Mon, July 9, 2018 at 4:44 PM, Alexander Korotkov wrote: Firstly, I did performance tests on 72-cores machines(AWS c5.18xlarge) same as you did. >>> >>> OK. But not that c5.18

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-10 Thread Alexander Korotkov
On Tue, Jul 10, 2018 at 2:19 PM Imai, Yoshikazu wrote: > On Mon, July 9, 2018 at 4:44 PM, Alexander Korotkov wrote: > > > Firstly, I did performance tests on 72-cores machines(AWS c5.18xlarge) > > > same as you did. > > > > OK. But not that c5.18xlarge is 72-VCPU machine, which AFAIK is > > clos

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-10 Thread Alexander Korotkov
On Mon, Jul 9, 2018 at 8:18 PM Peter Geoghegan wrote: > On Mon, Jul 9, 2018 at 9:43 AM, Alexander Korotkov > wrote: > > In this case it also looks like we observed 1% regression. Despite 1% > > may seem to be very small, I think we should clarify whether it really > > exists. I have at least tw

RE: Locking B-tree leafs immediately in exclusive mode

2018-07-10 Thread Imai, Yoshikazu
On Mon, July 9, 2018 at 4:44 PM, Alexander Korotkov wrote: > > Firstly, I did performance tests on 72-cores machines(AWS c5.18xlarge) same > > as you did. > > OK. But not that c5.18xlarge is 72-VCPU machine, which AFAIK is > close to the performance of 36 physical cores. Thanks for pointing tha

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-09 Thread Peter Geoghegan
On Mon, Jul 9, 2018 at 3:23 PM, Alexander Korotkov wrote: > I'm sorry, but I didn't understand this guess. I agree that moving > right within _bt_findinsertloc() might be worse than moving right > within _bt_moveright(). But why should it happen more often, if both > with and without patch that

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-09 Thread Alexander Korotkov
On Mon, Jul 9, 2018 at 8:18 PM Peter Geoghegan wrote: > On Mon, Jul 9, 2018 at 9:43 AM, Alexander Korotkov > wrote: > > In this case it also looks like we observed 1% regression. Despite 1% > > may seem to be very small, I think we should clarify whether it really > > exists. I have at least tw

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-09 Thread Simon Riggs
On 9 July 2018 at 04:18, Imai, Yoshikazu wrote: >> # script_ordered.sql >> INSERT INTO ordered (value) VALUES ('abcdefghijklmnoprsqtuvwxyz'); >> >> # script_unordered.sql >> \set i random(1, 100) >> INSERT INTO unordered VALUES (:i, 'abcdefghijklmnoprsqtuvwxyz'); >> # results >> ordered, ma

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-09 Thread Peter Geoghegan
On Mon, Jul 9, 2018 at 9:43 AM, Alexander Korotkov wrote: > In this case it also looks like we observed 1% regression. Despite 1% > may seem to be very small, I think we should clarify whether it really > exists. I have at least two hypothesis about this. > > 1) There is no real regression, obse

Re: Locking B-tree leafs immediately in exclusive mode

2018-07-09 Thread Alexander Korotkov
Hi! On Mon, Jul 9, 2018 at 6:19 AM Imai, Yoshikazu wrote: > Hi, I'm reviewing this. Great. Thank you for giving attention to this patch. > Firstly, I did performance tests on 72-cores machines(AWS c5.18xlarge) same > as you did. OK. But not that c5.18xlarge is 72-VCPU machine, which AFAIK i

RE: Locking B-tree leafs immediately in exclusive mode

2018-07-08 Thread Imai, Yoshikazu
On Mon, June 11, 2018 at 4:31 PM, Alexander Korotkov wrote: > On Mon, Jun 11, 2018 at 1:06 PM Simon Riggs wrote: > > On 5 June 2018 at 14:45, Alexander Korotkov > > wrote: > > > Currently _bt_search() always locks leaf buffer in shared mode (aka > > > BT_READ), while caller can relock it later.

Re: Locking B-tree leafs immediately in exclusive mode

2018-06-14 Thread Alexander Korotkov
On Thu, Jun 14, 2018 at 6:32 PM Peter Geoghegan wrote: > On Thu, Jun 14, 2018 at 5:43 AM, Alexander Korotkov > wrote: > > However, that doesn't > > look like inevitable shortcoming, because we could store heap TID in > > t_tid of pivot index tuples. > > But the offset doesn't have enough space fo

Re: Locking B-tree leafs immediately in exclusive mode

2018-06-14 Thread Peter Geoghegan
On Thu, Jun 14, 2018 at 5:43 AM, Alexander Korotkov wrote: > Could you, please, clarify what do you mean by "fan-in"? As I > understood, higher "fan-in" means more children on single non-leaf > page, and in turn "better branching". Is my understanding correct? Yes, your understanding is correct

Re: Locking B-tree leafs immediately in exclusive mode

2018-06-14 Thread Alexander Korotkov
On Thu, Jun 14, 2018 at 4:56 PM Claudio Freire wrote: > Not at all. Insertion cost in unique indexes with lots of duplicates > (happens, dead duplicates) grows quadratically on the number of > duplicates, and that's improved by making the index unique and sorted. Sorry, I've messed up the terms.

Re: Locking B-tree leafs immediately in exclusive mode

2018-06-14 Thread Claudio Freire
On Thu, Jun 14, 2018 at 9:44 AM Alexander Korotkov wrote: > > > Our B-tree is currently maintaining duplicates unordered. So, during > > > insertion > > > we can traverse rightlinks in order to find page, which would fit new > > > index tuple. > > > However, in this case we're traversing pages i

Re: Locking B-tree leafs immediately in exclusive mode

2018-06-14 Thread Alexander Korotkov
On Thu, Jun 14, 2018 at 1:01 AM Peter Geoghegan wrote: > On Mon, Jun 11, 2018 at 9:30 AM, Alexander Korotkov > wrote: > > On Mon, Jun 11, 2018 at 1:06 PM Simon Riggs wrote: > >> It's a good idea. How does it perform with many duplicate entries? > > I agree that this is a good idea. It independen

Re: Locking B-tree leafs immediately in exclusive mode

2018-06-13 Thread Peter Geoghegan
On Mon, Jun 11, 2018 at 9:30 AM, Alexander Korotkov wrote: > On Mon, Jun 11, 2018 at 1:06 PM Simon Riggs wrote: >> It's a good idea. How does it perform with many duplicate entries? I agree that this is a good idea. It independently occurred to me to do this. The L&Y algorithm doesn't take a pos

Re: Locking B-tree leafs immediately in exclusive mode

2018-06-11 Thread Alexander Korotkov
On Mon, Jun 11, 2018 at 1:06 PM Simon Riggs wrote: > On 5 June 2018 at 14:45, Alexander Korotkov wrote: > > Currently _bt_search() always locks leaf buffer in shared mode (aka > > BT_READ), > > while caller can relock it later. However, I don't see what prevents > > _bt_search() > > from lockin

Re: Locking B-tree leafs immediately in exclusive mode

2018-06-11 Thread Simon Riggs
On 5 June 2018 at 14:45, Alexander Korotkov wrote: > Currently _bt_search() always locks leaf buffer in shared mode (aka BT_READ), > while caller can relock it later. However, I don't see what prevents > _bt_search() > from locking leaf immediately in exclusive mode (aka BT_WRITE) when required.

Re: Locking B-tree leafs immediately in exclusive mode

2018-06-11 Thread Rafia Sabih
On Tue, Jun 5, 2018 at 7:15 PM, Alexander Korotkov wrote: > Hi! > > Currently _bt_search() always locks leaf buffer in shared mode (aka BT_READ), > while caller can relock it later. However, I don't see what prevents > _bt_search() > from locking leaf immediately in exclusive mode (aka BT_WRITE)

Locking B-tree leafs immediately in exclusive mode

2018-06-05 Thread Alexander Korotkov
Hi! Currently _bt_search() always locks leaf buffer in shared mode (aka BT_READ), while caller can relock it later. However, I don't see what prevents _bt_search() from locking leaf immediately in exclusive mode (aka BT_WRITE) when required. When traversing downlink from non-leaf page of level 1