Re: [Maria-developers] Understanding binlog group commit (MDEV-232, MDEV-532, MDEV-25611, MDEV-18959)

2023-05-22 Thread Vladislav Vaintroub
 Hi Marko and Kristian >> I also had the idea to use fibers/coroutines as is mentined in the MDEV>> description, but if that can be avoided, so much the better. >I too like https://quoteinvestigator.com/2011/05/13/einstein-simple/>or the KISS principle. About MDEV-24341That was not implemented as

Re: [Maria-developers] Understanding binlog group commit (MDEV-232, MDEV-532, MDEV-25611, MDEV-18959)

2023-05-22 Thread Marko Mäkelä
On Sat, May 20, 2023 at 11:07 PM Kristian Nielsen wrote: > I agree that a function parameter seems simpler. We're requesting to be > notified when a specific commit is durable in the engine, better specify > _which_ commit than for InnoDB to guess, as you say :-). Given that this code is only

Re: [Maria-developers] Understanding binlog group commit (MDEV-232, MDEV-532, MDEV-25611, MDEV-18959)

2023-05-20 Thread Kristian Nielsen
Marko Mäkelä writes: >> recent binlog files. We want to know when scanning the old binlog file is no >> longer necessary; then we will log a CHECKPOINT_EVENT recording this fact. >> But we don't want to stall waiting for everything to be flushed to disk >> immediately; we can just log the

Re: [Maria-developers] Understanding binlog group commit (MDEV-232, MDEV-532, MDEV-25611, MDEV-18959)

2023-05-16 Thread Marko Mäkelä
On Tue, May 16, 2023 at 1:37 PM Andrei Elkin wrote: > Can you comment on chances for `flush_lsn` never advance to the value of > `write_lsn` when/because Innodb has nothing to commit (so at such a time > RM (or PURGE or FLUSH LOGS that have been mentioned too) might be run)? If the last write

Re: [Maria-developers] Understanding binlog group commit (MDEV-232, MDEV-532, MDEV-25611, MDEV-18959)

2023-05-16 Thread Marko Mäkelä
On Fri, May 12, 2023 at 8:32 PM Kristian Nielsen wrote: > The transaction of interest is the last one that called commit_ordered() > prior to this call of commit_checkpoint_request(). I guess binlog code could > save it (it's trivial as commit_ordered() calls are serialised), or InnoDB > could do

Re: [Maria-developers] Understanding binlog group commit (MDEV-232, MDEV-532, MDEV-25611, MDEV-18959)

2023-05-12 Thread Kristian Nielsen
Marko Mäkelä writes: > Do you know how MyRocks implements ACID? There is a function Sorry, I do not. Maybe Sergey Petrunya does. > That function does not currently take any parameter (such as THD) to > identify the transaction of interest, and it cannot indicate that the > most recent state

Re: [Maria-developers] Understanding binlog group commit (MDEV-232, MDEV-532, MDEV-25611, MDEV-18959)

2023-05-12 Thread Marko Mäkelä
Hi Kristian, Thank you for your detailed reply. Before I read the linked blog posts, here are a few quick comments. > 1. The APIs (handlerton methods and server calls) are designed to be > storage-engine independent. So this is one reason why we would not see an > InnoDB LSN anywhere in the

Re: [Maria-developers] Understanding binlog group commit (MDEV-232, MDEV-532, MDEV-25611, MDEV-18959)

2023-05-12 Thread Kristian Nielsen
Marko Mäkelä writes: > later. I understand that you were an early contributor to MariaDB and > implemented some durability I/O optimization between the "distributed > transaction" of binlog and the InnoDB write-ahead log (redo log, > ib_logfile0). Indeed, that was the group commit work:

[Maria-developers] Understanding binlog group commit (MDEV-232, MDEV-532, MDEV-25611, MDEV-18959)

2023-05-11 Thread Marko Mäkelä
Hi Kristian, I am happy to see you being active on the maria-developers list. I don’t think we have collaborated in the past. I have been working on the InnoDB storage engine under MySQL 4.0 through 5.7 (and some 8.0) since 2003, and since 2016 under MariaDB Server, mostly 10.1 and later. I