[MariaDB developers] Re: Next step on MDEV-34705, implement binlog in InnoDB

2025-01-29 Thread Marko Mäkelä via developers
On Wed, Jan 29, 2025 at 4:55 PM Kristian Nielsen wrote: > Yes. Truncate is used only on the one that is being written to. It is used > to implement FLUSH BINARY LOGS, which is used to close the currently written > file early and move on to the next binlog file. If the binlog files would normally

[MariaDB developers] Re: Next step on MDEV-34705, implement binlog in InnoDB

2025-01-29 Thread Kristian Nielsen via developers
Marko Mäkelä writes: > On Wed, Jan 29, 2025 at 2:08 PM Kristian Nielsen > wrote: >> binlog_tablespace_truncate(tablespace, new_length_in_pages) >> Truncate a binlog tablespace (like mtr.trim_pages() and >> mtr.commit_shrink()). Can be independent, does not need to be part of a >> logging

[MariaDB developers] Re: Next step on MDEV-34705, implement binlog in InnoDB

2025-01-29 Thread Marko Mäkelä via developers
Hi Kristian, On Wed, Jan 29, 2025 at 2:08 PM Kristian Nielsen wrote: > I was thinking that the binlog layer would need to be informed of upto which > LSN the redo log has been durably written to disk (in case of > --innodb-flush-log-at-trx-commit=0|2). But does "trim the contents" imply > that th

[MariaDB developers] Re: Next step on MDEV-34705, implement binlog in InnoDB

2025-01-29 Thread Kristian Nielsen via developers
Marko Mäkelä writes: > Finally, I got some more time to think about this. I’m trying to > summarize from the InnoDB point of view what we discussed today. Thanks Marko! This is an interesting development. If we can bypass using the buffer pool and associated machinery it could simplify the logic

[MariaDB developers] Re: Next step on MDEV-34705, implement binlog in InnoDB

2025-01-28 Thread Marko Mäkelä via developers
Hi Kristian, Finally, I got some more time to think about this. I’m trying to summarize from the InnoDB point of view what we discussed today. * We do not need any InnoDB buf_block_t or fil_space_t for any binlog files. * The binlog layer can simply append pages to binlog files (or rewrite the la

[MariaDB developers] Re: Next step on MDEV-34705, implement binlog in InnoDB

2025-01-08 Thread Marko Mäkelä via developers
On Wed, Jan 8, 2025 at 6:25 PM Marko Mäkelä wrote: > I think that we could allow the binlog layer to write directly to the > 4096-byte blocks that are allocated from the InnoDB buffer pool. The > binlog page cleaner thread might even be writing the last (incomplete) > block concurrently while we a

[MariaDB developers] Re: Next step on MDEV-34705, implement binlog in InnoDB

2025-01-08 Thread Marko Mäkelä via developers
On Mon, Jan 6, 2025 at 2:15 PM Kristian Nielsen via developers wrote: > That code line was a bit sloppy and not correct. What I had in mind is more > something like this: > > if (page_offset > FIL_PAGE_DATA && block->page.oldest_modification() <= 1) { > // Adding to a page that was already f

[MariaDB developers] Re: Next step on MDEV-34705, implement binlog in InnoDB

2025-01-06 Thread Kristian Nielsen via developers
Kristian Nielsen via developers writes: > What if the binlog code simply keeps track of whenever the current page of > the binlog gets partially written to the file system? And when this happens, > the next mtr_t write to that page will simply re-write all the data from the > start of the page? T

[MariaDB developers] Re: Next step on MDEV-34705, implement binlog in InnoDB

2025-01-05 Thread Kristian Nielsen via developers
Marko Mäkelä via developers writes: > On Fri, Jan 3, 2025 at 10:23 AM Kristian Nielsen > wrote: >> I think now is a good time for you to take a first real look at the InnoDB >> part of the changes, I would really value your input. > > This is great. I will try to find some time for this before

[MariaDB developers] Re: Next step on MDEV-34705, implement binlog in InnoDB

2025-01-03 Thread Marko Mäkelä via developers
Hi Kristian, On Fri, Jan 3, 2025 at 10:23 AM Kristian Nielsen wrote: > But things have progressed, and I have now reached the point where most of > the basic groundwork is implemented. Event groups are binlogged to InnoDB > tablespaces. Binlog dump thread can read the binlog and send to slave, an