Re: [Maria-developers] commit_checkpoint_request() vs. thd_get_durability_property() (in relation to MDEV-11937)

[Kristian Nielsen]

Marko Mäkelä  writes:

> Thank you for your analysis. I suggest that we remove the unused
> declarations and code as soon as possible. Would you do that already under
> MDEV-11937?

I started on this, but then noticed that several other storage engines
reference the APIs also, and thought I should not remove it without
discussion first.

Another example is the added argument binlog_group_flush to the flush_logs
handlerton method. This is the binlog-based group prepare, which is not in
MariaDB. The InnoDB code seems to always pass binlog_group_flush as true,
isn't this wrong? It will skip log sync in srv_flush_log_at_trx_commit==0,
not sure what bad consequences that will have though, if any.

My opinion would be that storage engines should be adapted to the mariadb
APIs before being pushed into any main tree. It really should not take much
effort to do so, apart from taking the time to understand the code properly
first. For InnoDB, it should be already done, so new code just needs to be
adapted to the different APIs.

(BTW, I hope it is clear that I am not blaming any individual bug on any
individual engineer here. I know full well how MariaDB development happens
and what policies lead to these consequences).

 - Kristian.

> I think that future code merges from MySQL should be based on individual
> commits rather than snapshots. For merging the changes from 5.7.14 to
> 5.7.18, I already did this, and I rewrote or omitted some commits; see
> MDEV-11751.
>
> One particular problem with snapshots is that changes or additions to test
> files were often omitted. When merging changes commit by commit ("git
> format-patch" followed by "git am"), some manual work is needed to import
> the affected test files. The Oracle policy should be to only withhold test
> files for security bugs; we should take full advantage of the public tests.
>
> Best regards,
>
> Marko
>
> On Mon, Aug 7, 2017 at 2:25 PM, Jan Lindström 
> wrote:
>
>> Hi,
>>
>> On Mon, Aug 7, 2017 at 2:09 PM, Kristian Nielsen > > wrote:
>>
>>>
>>> The problem is that somehow the thd_get_durability_property() function was
>>> introduced into MariaDB code, but it is completely non-functional. So now
>>> there is code in InnoDB, TokuDB and RocksDB that calls this function and
>>> does not work correctly. This lead to performance regression due to extra
>>> fsync() calls.
>>>
>>
>> This function was introduced while I was doing InnoDB 5.7.9 merge, this
>> was huge
>> merge but in my understanding all server changes i.e. also storage API
>> changes were
>> reviewed. But, as patch was significant there was error. That function
>> should not be introduced.
>>
>>
>>>
>>>
>>> Or was the intention to eventually replace the whole MariaDB binlog group
>>> commit implementation with the MySQL one, to make MariaDB less divergent?
>>> This would require a number of changes to MariaDB binlog and replication.
>>>
>>
>> No this was not the intention.
>>
>> R: Jan
>>
>> ___
>> Mailing list: https://launchpad.net/~maria-developers
>> Post to : maria-developers@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~maria-developers
>> More help   : https://help.launchpad.net/ListHelp
>>
>>

___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] commit_checkpoint_request() vs. thd_get_durability_property() (in relation to MDEV-11937)

[Marko Mäkelä]

Kristian,

Thank you for your analysis. I suggest that we remove the unused
declarations and code as soon as possible. Would you do that already under
MDEV-11937?

I think that future code merges from MySQL should be based on individual
commits rather than snapshots. For merging the changes from 5.7.14 to
5.7.18, I already did this, and I rewrote or omitted some commits; see
MDEV-11751.

One particular problem with snapshots is that changes or additions to test
files were often omitted. When merging changes commit by commit ("git
format-patch" followed by "git am"), some manual work is needed to import
the affected test files. The Oracle policy should be to only withhold test
files for security bugs; we should take full advantage of the public tests.

Best regards,

Marko

On Mon, Aug 7, 2017 at 2:25 PM, Jan Lindström 
wrote:

> Hi,
>
> On Mon, Aug 7, 2017 at 2:09 PM, Kristian Nielsen  > wrote:
>
>>
>> The problem is that somehow the thd_get_durability_property() function was
>> introduced into MariaDB code, but it is completely non-functional. So now
>> there is code in InnoDB, TokuDB and RocksDB that calls this function and
>> does not work correctly. This lead to performance regression due to extra
>> fsync() calls.
>>
>
> This function was introduced while I was doing InnoDB 5.7.9 merge, this
> was huge
> merge but in my understanding all server changes i.e. also storage API
> changes were
> reviewed. But, as patch was significant there was error. That function
> should not be introduced.
>
>
>>
>>
>> Or was the intention to eventually replace the whole MariaDB binlog group
>> commit implementation with the MySQL one, to make MariaDB less divergent?
>> This would require a number of changes to MariaDB binlog and replication.
>>
>
> No this was not the intention.
>
> R: Jan
>
> ___
> Mailing list: https://launchpad.net/~maria-developers
> Post to : maria-developers@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-developers
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Marko Mäkelä, Lead Developer InnoDB
MariaDB Corporation
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-developers] commit_checkpoint_request() vs. thd_get_durability_property() (in relation to MDEV-11937)

[Jan Lindström]

Hi,

On Mon, Aug 7, 2017 at 2:09 PM, Kristian Nielsen 
wrote:

>
> The problem is that somehow the thd_get_durability_property() function was
> introduced into MariaDB code, but it is completely non-functional. So now
> there is code in InnoDB, TokuDB and RocksDB that calls this function and
> does not work correctly. This lead to performance regression due to extra
> fsync() calls.
>

This function was introduced while I was doing InnoDB 5.7.9 merge, this was
huge
merge but in my understanding all server changes i.e. also storage API
changes were
reviewed. But, as patch was significant there was error. That function
should not be introduced.


>
>
> Or was the intention to eventually replace the whole MariaDB binlog group
> commit implementation with the MySQL one, to make MariaDB less divergent?
> This would require a number of changes to MariaDB binlog and replication.
>

No this was not the intention.

R: Jan
___
Mailing list: https://launchpad.net/~maria-developers
Post to : maria-developers@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-developers
More help   : https://help.launchpad.net/ListHelp