Re: [Maria-discuss] innodb_file_per_table default setting in the online documentation
Thanks Pantelis You're right that ON and OFF are switched around. I'll double check the version that this changed - the 5.5 version of the MySQL docs at http://dev.mysql.com/doc/refman/5.5/en/innodb-parameters.html#sysvar_innodb_file_per_table also say that this was changed in 5.5.7 (so it was probably changed in both MySQL 5.5.7 and 5.6.6), but I'll check the MariaDB diffs to be sure. Sending to this list is fine, although there's also a docs-specific list: maria-d...@lists.launchpad.net ian On 08/05/2015 01:50, Pantelis Theodosiou wrote: I was looking for the default setting of this variable and the MariaDB documentation https://mariadb.com/kb/en/mariadb/xtradbinnodb-server-system-variables/#innodb_file_per_table has: *Default Value:* |OFF| (= MariaDB/MySQL 5.5.7), |ON| (MariaDB/MySQL 5.5.0 to 5.5.6) while MySQL documentation http://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html#sysvar_innodb_file_per_table has (I think correctly): *Permitted Values* (= 5.6.5)* Type*|boolean|*Default *|OFF |*Permitted Values* (= 5.6.6) *Type *|boolean|* Default *|ON | |So, I see two issues: - first, the changing version is 5.6.6 (not 5.5.7), at least in MySQL, and - second, the values OFF / ON in the Maria documentation are reversed. | |Apologies if this is not the correct list to raise the issue. | |Pantelis Theodosiou | | | ___ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Re: [Maria-discuss] innodb_file_per_table default setting in the online documentation
Although the first MariaDB 5.5 release was 5.5.20... Anyway, fixed in the docs now, thanks for the report. On 08/05/2015 09:50, Ian Gilfillan wrote: You're right that ON and OFF are switched around. I'll double check the version that this changed - the 5.5 version of the MySQL docs at http://dev.mysql.com/doc/refman/5.5/en/innodb-parameters.html#sysvar_innodb_file_per_table also say that this was changed in 5.5.7 (so it was probably changed in both MySQL 5.5.7 and 5.6.6), but I'll check the MariaDB diffs to be sure. Sending to this list is fine, although there's also a docs-specific list: maria-d...@lists.launchpad.net ian On 08/05/2015 01:50, Pantelis Theodosiou wrote: I was looking for the default setting of this variable and the MariaDB documentation https://mariadb.com/kb/en/mariadb/xtradbinnodb-server-system-variables/#innodb_file_per_table has: *Default Value:* |OFF| (= MariaDB/MySQL 5.5.7), |ON| (MariaDB/MySQL 5.5.0 to 5.5.6) while MySQL documentation http://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html#sysvar_innodb_file_per_table has (I think correctly): *Permitted Values* (= 5.6.5)* Type*|boolean|*Default *|OFF |*Permitted Values* (= 5.6.6) *Type *|boolean|* Default *|ON | |So, I see two issues: - first, the changing version is 5.6.6 (not 5.5.7), at least in MySQL, and - second, the values OFF / ON in the Maria documentation are reversed. | |Apologies if this is not the correct list to raise the issue. | |Pantelis Theodosiou | | | ___ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Re: [Maria-discuss] innodb_file_per_table default setting in the online documentation
Hi Ian, Pantelis, What MySQL docs say is true, however confusing. The value was switched ON in early versions of MySQL 5.5; caused some problems; was switched back to OFF in 5.5.7; stayed OFF in all further 5.5 releases and early releases of 5.6; and was switched ON from 5.6.6. For MariaDB it's a bit simpler for 5.5 and more complicated for 10.0. As Ian said the first release was 5.5.20, so in MariaDB 5.5 it's just OFF by default. In 10.0, it was OFF in 10.0.0-10.0.3; from 10.0.4, where InnoDB 5.6.10 was merged, it was ON for the the InnoDB plugin (which was the default InnoDB then) but still OFF for XtraDB; and later when XtraDB was upgraded, it became ON for it too. All in all, in 10.0.10 which was the first GA release, it was already ON for both flavors, as it is now. Regards, Elena On 08.05.2015 11:04, Ian Gilfillan wrote: Although the first MariaDB 5.5 release was 5.5.20... Anyway, fixed in the docs now, thanks for the report. On 08/05/2015 09:50, Ian Gilfillan wrote: You're right that ON and OFF are switched around. I'll double check the version that this changed - the 5.5 version of the MySQL docs at http://dev.mysql.com/doc/refman/5.5/en/innodb-parameters.html#sysvar_innodb_file_per_table also say that this was changed in 5.5.7 (so it was probably changed in both MySQL 5.5.7 and 5.6.6), but I'll check the MariaDB diffs to be sure. Sending to this list is fine, although there's also a docs-specific list: maria-d...@lists.launchpad.net ian On 08/05/2015 01:50, Pantelis Theodosiou wrote: I was looking for the default setting of this variable and the MariaDB documentation https://mariadb.com/kb/en/mariadb/xtradbinnodb-server-system-variables/#innodb_file_per_table has: *Default Value:* |OFF| (= MariaDB/MySQL 5.5.7), |ON| (MariaDB/MySQL 5.5.0 to 5.5.6) while MySQL documentation http://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html#sysvar_innodb_file_per_table has (I think correctly): *Permitted Values* (= 5.6.5)* Type*|boolean|*Default *|OFF |*Permitted Values* (= 5.6.6) *Type *|boolean|* Default *|ON | |So, I see two issues: - first, the changing version is 5.6.6 (not 5.5.7), at least in MySQL, and - second, the values OFF / ON in the Maria documentation are reversed. | |Apologies if this is not the correct list to raise the issue. | |Pantelis Theodosiou | | | ___ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Re: [Maria-discuss] MariaDB 10.0.18 now available
Hi, Reindl! On May 07, Reindl Harald wrote: No, it affects the server, not mysql_upgrade. But it's a new statement, that mysql_upgrade is using, no existing query can possibly trigger that bug well, in other words anybody could crash the server by write a specific query and so i am not sure what is worser: the security bugs in 10.0.17 or that bug in 10.0.18 Right. We'll release 10.0.19 to fix that. doesn't upstream run mysql_upgrade mandatory independent of changes? No. Depends on what upstream is. Debian/Ubuntu do that, as far as I remember. RedHat/Fedora/CentoS - don't (again, as far as I remember). OpenVAS against 10.0.17 reports CVE-2013-1861 and CVE-2012-5627 while there still was no answer to the mail below and so the state which of the mysql security bugs are also in mariadb is unknown I've updated MariaDB.org CVE overview page about a week ago. (note that email didn't request an answer, it requested the page to be updated) Regards, Sergei Weitergeleitete Nachricht Betreff: [Maria-developers] Oracle April security notices and MariaDB Datum: Sun, 19 Apr 2015 21:55:19 +0300 Von: Otto Kekäläinen o...@seravo.fi An: maria-develop...@lists.launchpad.net maria-develop...@lists.launchpad.net Hello! Debian security team is pressing me on the information about which recent Oracle CVEs affect MariaDB and which not. They default to assuming all affect so we need to prove otherwise. The Debian CVE tracker: https://security-tracker.debian.org/tracker/source-package/mariadb-10.0 None of these recent CVEs are listed at the MariaDB.org tracker: https://mariadb.com/kb/en/mariadb/security/ Could somebody please update the MariaDB.org CVE overview page? ___ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Re: [Maria-discuss] fsync necessary for synchronous page flush?
Hi, The log does not have whole pages. Pages must not be torn for the recovery process to work. A fsync is required when a page is written to disk. During recovery all changes since the last checkpoint are replayed, then transactions that do not have a commit marker are rolled back. This is called roll forward/roll back recovery. --Justin On Fri, May 8, 2015 at 6:09 PM, Xiaofei Du xiaofei.du...@gmail.com wrote: Justin, I was thinking of if fsync is needed each time after a write. The operations are already in the log. So recovery can always be done from the log. The difference is that during recovery, we need to go back further in the log and it will take longer. But in that way, I guess it would be hard to coordinate with the kernel flush thread. Xiaofei On Fri, May 8, 2015 at 2:06 PM, Justin Swanhart greenl...@gmail.com wrote: Hi, InnoDB recovery can not handle torn pages. An fsync is required to ensure that the page is fully written to disk. This is also why the doublewrite buffer is used. Before pages are written down to disk, they are first written sequentially into the doublewrite buffer. This buffer is synced, then async page writing can proceed. If the database crashes, the pages in flight will be rewritten by the doublewrite buffer. The detection mechanism for torn pages comes from an LSN, which is written into the top and the bottom of the page. If the LSN at the top and bottom do not match the page is torn. Regards, --Justin On Fri, May 8, 2015 at 12:43 PM, Xiaofei Du xiaofei.du...@gmail.com wrote: Laurynas, This is exactly what I was looking for. I went through these functions before. I disabled double write buffer, so I didn't pay attention to code under buf_dblwr... The reason I asked this question is because I didn't know how the recovery process works, so I was wondering if it's necessary to fsync after each write. It's a performance concern. Anyway, thank you very much! Jan -- Thank you for your answer too! Xiaofei On Thu, May 7, 2015 at 9:59 PM, Laurynas Biveinis laurynas.bivei...@gmail.com wrote: Xiaofei - fsync is performed for all the flush types (LRU, flush, single page) if it is asked for (innodb_flush_method != O_DIRECT_NO_FSYNC). The apparent difference in sync and async is not because of the sync difference itself, but because of the flush type difference. The single page flush flushes one page, and requests a fsync for its file. Other flushes flush in batches, don't have to fsync for each written page individually but rather sync once at the end. Then doublewrite complicates this further. If it is disabled, fsync will happen in buf_dblwr_sync_datafiles called from buf_dblwr_flush_buffered_writes called from buf_flush_common called at the end of either LRU or flush list flush. If doublewrite is enabled, fsync will happen in buf_dblwr_update called from buf_flush_write_complete. 2015-05-07 9:01 GMT+03:00 Xiaofei Du xiaofei.du...@gmail.com: Hi Laurynas, On Wed, May 6, 2015 at 9:14 PM, Laurynas Biveinis laurynas.bivei...@gmail.com wrote: Xiaofei - Does InnoDB maintain a dirty page table? You must be referring to the buffer pool flush_list. You are right. The flush_list is can be used for recovery and checkpoint. Is fsync called to guarantee the page to be on persistent storage so that the dirty page table can be updated? If this is the case, when is the dirty page table updated for asynchronous IOs? Check buf_flush_write_complete in buf0flu.cc. For async IO it is called from buf_page_io_complete in buf0buf.cc. You are right that this is the place it updates the dirty page information. But I still don't understand why the fsync is needed for synchronous IOs, but not for the AIOs. Jan Lindstrom said fsync is also called for other AIO operations. But I could only it true in one of many AIO operations. Or maybe I am missing something still? -- Laurynas -- Laurynas ___ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Re: [Maria-discuss] fsync necessary for synchronous page flush?
Justin, I was thinking of if fsync is needed each time after a write. The operations are already in the log. So recovery can always be done from the log. The difference is that during recovery, we need to go back further in the log and it will take longer. But in that way, I guess it would be hard to coordinate with the kernel flush thread. Xiaofei On Fri, May 8, 2015 at 2:06 PM, Justin Swanhart greenl...@gmail.com wrote: Hi, InnoDB recovery can not handle torn pages. An fsync is required to ensure that the page is fully written to disk. This is also why the doublewrite buffer is used. Before pages are written down to disk, they are first written sequentially into the doublewrite buffer. This buffer is synced, then async page writing can proceed. If the database crashes, the pages in flight will be rewritten by the doublewrite buffer. The detection mechanism for torn pages comes from an LSN, which is written into the top and the bottom of the page. If the LSN at the top and bottom do not match the page is torn. Regards, --Justin On Fri, May 8, 2015 at 12:43 PM, Xiaofei Du xiaofei.du...@gmail.com wrote: Laurynas, This is exactly what I was looking for. I went through these functions before. I disabled double write buffer, so I didn't pay attention to code under buf_dblwr... The reason I asked this question is because I didn't know how the recovery process works, so I was wondering if it's necessary to fsync after each write. It's a performance concern. Anyway, thank you very much! Jan -- Thank you for your answer too! Xiaofei On Thu, May 7, 2015 at 9:59 PM, Laurynas Biveinis laurynas.bivei...@gmail.com wrote: Xiaofei - fsync is performed for all the flush types (LRU, flush, single page) if it is asked for (innodb_flush_method != O_DIRECT_NO_FSYNC). The apparent difference in sync and async is not because of the sync difference itself, but because of the flush type difference. The single page flush flushes one page, and requests a fsync for its file. Other flushes flush in batches, don't have to fsync for each written page individually but rather sync once at the end. Then doublewrite complicates this further. If it is disabled, fsync will happen in buf_dblwr_sync_datafiles called from buf_dblwr_flush_buffered_writes called from buf_flush_common called at the end of either LRU or flush list flush. If doublewrite is enabled, fsync will happen in buf_dblwr_update called from buf_flush_write_complete. 2015-05-07 9:01 GMT+03:00 Xiaofei Du xiaofei.du...@gmail.com: Hi Laurynas, On Wed, May 6, 2015 at 9:14 PM, Laurynas Biveinis laurynas.bivei...@gmail.com wrote: Xiaofei - Does InnoDB maintain a dirty page table? You must be referring to the buffer pool flush_list. You are right. The flush_list is can be used for recovery and checkpoint. Is fsync called to guarantee the page to be on persistent storage so that the dirty page table can be updated? If this is the case, when is the dirty page table updated for asynchronous IOs? Check buf_flush_write_complete in buf0flu.cc. For async IO it is called from buf_page_io_complete in buf0buf.cc. You are right that this is the place it updates the dirty page information. But I still don't understand why the fsync is needed for synchronous IOs, but not for the AIOs. Jan Lindstrom said fsync is also called for other AIO operations. But I could only it true in one of many AIO operations. Or maybe I am missing something still? -- Laurynas -- Laurynas ___ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
