Hi Xiaokai, If I understand you correctly, you are proposing to use key->ops hash to apply the write transaction instead of serializing the operations in PREPARE phrase with a lock? Here are some questions I have with this approach,
1) what the time stamp would be if there is no longer a lock during PREPARE phrase, how to ensure the following scenario happens in order? - T1 deletes the row (key1, key2, value1) - T2 upserts the row (key1, key2, value2) 2) what happens if any of the operations happens to update the keys? e.g - T1 updates the row (key1, key2, value1) to (key3, key2, value2) - T1 updates the row (key3, key2, value3) to (key1, key2, value1) Both of the above scenarios don't seem to have a deterministic behavior. Best, Hao On Fri, Sep 21, 2018 at 2:29 AM, Xiaokai Wang <[email protected]> wrote: > Thanks your reply, Mike. > > In our scenario, some tables have a lot of data to be updated at the same > time. The current implementation of KUDU, if TXN(transaction) can not get > all the ops lock it will wait for 1s to block the entire Tablet processing, > which > will greatly reduce the write performance of the Tablet.Reduce the size > of the batch, TXN may still have the same row key, and can not solve this > problem. > > > In my way, I want to change the work way of 'APPLY' phase, instead of > putting TXN to apply_pool_ queue, I will put each key-op(splitting TXN to > ops) to apply_pool_token_ queue. This will guarantee the same key's op > will be putted to the same thread. When ops belonging to the same TXN > execute over, then send rpc response and write CommitMsg to WAL. In this > way, I can abandon the keys locks. > > Working follow chart just like this below: > [image: op-key_concurrent.png] > 1. client send T1, T2, T3 to tserver, tserver will handle each txn in turn. > > 2. leader send replicated msg to follower, sending them together or > independent. T1, T2, T3 will be ordered to execute by follower and send > them back by together or independent in turn. raft_pool_token_ guarantee > T1, T2 T3 be ordered to apply. > > 3. Splitting Tx to ops, each op is hashed to the queue of > apply_pool_token_, key's op is order to execute as TXN turn. > > -------- > > Kudu origin work way below: > [image: txn_concurrent.png] > 1. client send T1, T2, T3 to tserver, tserver will handle each txn > orderly, they will acquire locks in 'PREPARE' phase in turn. > > 2. leader send replicated msg to follower, sending them together or > independent. T1, T2, T3 will be ordered to execute by follower and send > them back by together or independent in turn. raft_pool_token_ guarantee > T1, T2 T3 be ordered to apply. > > 3. T1, T2, T3 are putted to the queue of apply_pool_, order or > concurrented to be executed, releasing locks. > > ----------- > > Contrast to origin, the keys locks are abandon, this can guarantee kudu > throughput more smoothly. What do you think? Hope to get your advice, > kudu users. > > Thanks. > xiaokai. > > > ---------- Forwarded message --------- >> From: Mike Percy <[email protected]> >> Date: 2018年9月19日周三 上午8:29 >> Subject: Re: Locks are acquired to cost much time in transactions >> To: <[email protected]> >> >> >> Why do you think you are spending a lot of time contending on row locks? >> >> Have you tried configuring your clients to send smaller batches? This may >> decrease throughput on a per-client basis but will likely improve latency >> and reduce the likelihood of row lock contention. >> >> If you are really spending most of your time contending on row locks then >> you will likely run into more fundamental performance issues trying to >> scale your writes, since Kudu's MVCC implementation effectively stores a >> linked list of updates to a given cell until compaction occurs. See >> https://github.com/apache/kudu/blob/master/docs/design- >> docs/tablet.md#historical-mvcc-in-diskrowsets for more information about >> the on-disk design. >> >> If you accumulate too many uncompacted mutations against a given row, >> reading the latest value for that row at scan time will be slow because it >> has to do a lot of work at read time. >> >> Mike >> >> On Tue, Sep 18, 2018 at 8:48 AM Xiaokai Wang <[email protected]> >> wrote: >> >>> Moved here from JIRA. >>> >>> Hi guys, I met a problem about the keys locks that almost impacts the >>> service normal writing. >>> >>> >>> As we all know, a transaction which get all row_key locks will go on >>> next step in kudu. Everything looks good, if keys are not concurrent >>> updated. But when keys are updated by more than one client at the same time >>> , locks are acquired to wait much time. The cases are often in my >>> product environment. Does anybody meet the problem? Has any good ideal for >>> this? >>> >>> >>> In my way, I want to try to abandon keys locks, instead using >>> *_pool_token_ 'SERIAL' mode which keeping the key of transaction is serial >>> and ordered. Dose this work? >>> >>> >>> Hope to get your advice. Thanks. >>> >>> >>> ----- >>> Regards, >>> Xiaokai >>> >>
