Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-13 Thread Kristian Nielsen
Jan Lindström jplin...@mariadb.org writes: Hi, On 08/13/2013 09:49 AM, Kristian Nielsen wrote: You can always use the contents of the binlogs to know this. You can search the binlogs for your GTID and determine if it was a) logged in an earlier binlog that was purged, b) found in the binlog

Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-13 Thread Kristian Nielsen
Vilho Raatikka vilho.raati...@skysql.com writes: What exactly does it require for the user to enable strict or 'relaxed' (=disasterous imho) mode? Technically, one needs to SET GLOBAL gtid_strict_mode=1 with root privileges (config file option works as well of course). That's it. The

Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-13 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: BTW, for the normal non-advanced user, I think silly stuff like running random manual transactions on slaves that get into the binlog is a far more common mistake. Gtid strict mode will cause errors on this, so it needs to be explictly enabled

[Maria-developers] Review of patch for MDEV-4820

2013-08-12 Thread Kristian Nielsen
I took a close look at your patch for MDEV-4820. I think there is a fundamental disconnect. In MariaDB GTID, I do not require or rely on monotonically increasing seqeunce numbers (monoticity is requred per-server-id, but not between different servers). Nor do I enforce or rely on absence of holes

Re: [Maria-developers] GTID and removing binary logs

2013-08-08 Thread Kristian Nielsen
Elena Stepanova ele...@montyprogram.com writes: Reading recent Pavel Ivanov's reports (the last one is MDEV-4820, but I think there were some others which you've already closed as fixed), I started wondering if I conceptually misunderstood something before. From our previous discussions I

Re: [Maria-developers] [Commits] Rev 3797: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg in file:///data0/psergey/dev2/10.0-serg/

2013-08-07 Thread Kristian Nielsen
Sergey Petrunya pser...@askmonty.org writes: revision-id: pser...@askmonty.org-20130802141209-4dqfvx2db8acxwbl message: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg Temorary fix for a number of replication tests (rpl.rpl_temp_table_mix_row rpl.rpl_trunc_temp rpl.rpl_current_user

Re: [Maria-developers] Structure of MariaDB 10.0 binary logs

2013-07-20 Thread Kristian Nielsen
Giuseppe Maxia g.ma...@gmail.com writes: Specifically, I would like to know * what are the differences between binary logs in MariaDB 5.5 and 10.0 * what are the differences between binary logs in MariaDB 10.0 and MySQL 5.6. I think the difference is mainly the addition of new events. At

Re: [Maria-developers] a bug of ding-qing parallel replication

2013-07-11 Thread Kristian Nielsen
nanyi607rao nanyi607...@gmail.com writes: I find a bug of ding-qing parallel replication. let's consider a table like this: create table t1( a int(11) NOT NULL DEFAULT '0', b varchar(10), PRIMARY KEY (a) )ENGINE=InnoDB if we do transactions on master(binlog_format=ROW) like

Re: [Maria-developers] Process suggestion for minor issues

2013-06-14 Thread Kristian Nielsen
Rasmus Johansson ras...@mariadb.com writes: This is a suggestion on how to deal with minor or lower priority bugs and tasks in JIRA, http://mariadb.org/jira . Please provide your thoughts. Do you think it's an OK way to deal with the minors or do you prefer some other way? The whole Jira

Re: [Maria-developers] Buildbot question

2013-06-11 Thread Kristian Nielsen
Sergey Vojtovich s...@mariadb.org writes: I'm observing pretty odd buildbot behavior with some of my pushes (probably merges). Bar and Daniel were unable to answer my questions and suggested to ask you instead. The problem seems to be that Buildbot has become confused about revisions. Look at

Re: [Maria-developers] Group commit comparison between mysql and mariadb

2013-06-11 Thread Kristian Nielsen
Sergei Petrunia pser...@askmonty.org writes: Hi Kristian, I'm wondering, do we have any comparison of MariaDB's binlog group commit implementation vs MySQL's binlog's group commit? I recall Mats writing a blog This is the best I have: http://kristiannielsen.livejournal.com/16382.html

Re: [Maria-developers] GTID work priorities

2013-05-28 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: we would really love to see strict gtid mode and related stuff next. I've now pushed strict mode to 10.0-base, and documented it in the knowledgebase. However, one thing was not clear from your description in MDEV-4478: 4) Probably with a separate flag

Re: [Maria-developers] GTID replication and relay_log.info

2013-05-27 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: See logs in attachment. It looks clear to me that relay logs get replayed after crash and don't get deleted. I'm not sure though if the reason of that can be seen in the logs. No, I agree it's not clear from the logs one way or the other, so good that

Re: [Maria-developers] GTID replication and relay_log.info

2013-05-27 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Thanks for the pointers, Kristian. It turned out that it was an incorrect merge of GTID-related changes into 10.0.1 tree. In particular I didn't quite understand the meaning of LINE_FOR_FIRST_MYSQL_5_6 in sql/rpl_mi.cc and thus left it at the value of

Re: [Maria-developers] GTID replication and relay_log.info

2013-05-25 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: yet). But from time to time we see failures of rpl.rpl_gtid_crash test with the output like this: Hm, strange, it does not fail in our Buildbot, which runs hundreds of tests on different hosts... Can you suggest what debugging information should I

Re: [Maria-developers] Proposed changes to GTID syntax

2013-05-23 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: I am planning some changes to the user interface of Global Transaction ID. Testing by Elena has shown usability issues that I want to address with these changes, but I wanted to give everyone a change to voice their opinion on the changes. I

Re: [Maria-developers] MariaDB and MySQL replication

2013-05-21 Thread Kristian Nielsen
Jan Lindström jplin...@mariadb.org writes: My question is how compatible replication between 1) MySQL (master) - MariaDB (slave) 2) MariaDB (master) - MySQL (slave) In general, it should work, though there are some limitations. It also depends on versions. I always try to make MariaDB

Re: [Maria-developers] GTID replication and relay_log.info

2013-05-21 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: There's a well-known problem in MySQL that relay_log.info is not always in sync with the database state, and killing and restarting server at an unfortunate time will cause it to re-execute last statement and potentially break replication. Did you fix

Re: [Maria-developers] GTID replication and relay_log.info

2013-05-21 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Could you please elaborate what kind of bug is that? The slave connects to master with GTID. The binlog dump thread on the master starts scanning the last binlog file to find the start position. It finds the position for one replication domain, but aother

Re: [Maria-developers] Getting community to write missing test cases

2013-05-20 Thread Kristian Nielsen
Elena Stepanova ele...@montyprogram.com writes: If we start doing it, Oracle will soon realize that this activity is based at least as much on commit comments as on the code. As a result, they will probably start making much vaguer comments than they do now, which in turn might make merge and

[Maria-developers] Proposed changes to GTID syntax

2013-05-17 Thread Kristian Nielsen
I am planning some changes to the user interface of Global Transaction ID. Testing by Elena has shown usability issues that I want to address with these changes, but I wanted to give everyone a change to voice their opinion on the changes. The problem: The issue concerns when a slave server

Re: [Maria-developers] Proposed changes to GTID syntax

2013-05-17 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: I am planning some changes to the user interface of Global Transaction ID. Some small simplication/improvement to naming after feedback from Elena: CHANGE MASTER TO master_use_gtid = SLAVE_POS CHANGE MASTER TO master_use_gtid

Re: [Maria-developers] MariaDB GTID first impression

2013-05-15 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: We reaching consistency in the backup by shutting down the server. I So thinking about this again ... So if we shut down the server gracefully, copy over everything to a new server, delete all binlogs (or omit binlogs from the copy), restart the server -

Re: [Maria-developers] MariaDB GTID first impression

2013-05-14 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: My intention for this was to use the existing facilities for such backup/restore, like mysqldump and XtraDB. I didn't check with the 10.0.2, but in 10.0.0 XtraDB didn't compile. Does it compile now? I do not know, I do not think anyone tried. It may

Re: [Maria-developers] MariaDB build logs

2013-05-14 Thread Kristian Nielsen
Marcin Kulisz deb...@kulisz.net writes: as promised I'm attaching build log for maria DB, it's much smaller then anticipated. Thanks! From what I can tell it was just one test which failed and rendered whole build as a failure. My build env is based on cowbuilder on fully patched SID

Re: [Maria-developers] MariaDB GTID first impression

2013-05-14 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: We reaching consistency in the backup by shutting down the server. I would think InnoDB state shouldn't be a problem in such situation. Am I right? Oh, I see, right then there is certainly no issue. - Kristian.

Re: [Maria-developers] MariaDB GTID first impression

2013-05-13 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Sorry, I didn't quite understand. Sorry for being unclear, let me elaborate: You didn't mention log-bin.state here. Is it intentional? Does it try to look into log-bin.state after rpl_slave_state and binlogs? Or e.g. in the situation when binlogs are

Re: [Maria-developers] MariaDB GTID first impression

2013-05-13 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: How does it detect if binlogs were closed properly? This is the standard binlog crash recovery going back to 5.1 (or possibly 5.0): When a new binlog file is created, the first Format_description event has a flag LOG_EVENT_BINLOG_IN_USE_F set. At normal

Re: [Maria-developers] MariaDB GTID first impression

2013-05-12 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: I was thinking about your words above and they make me wonder: if we have a master and it doesn't write its current state into rpl_slave_state, then we shut down the master, and while it's out we failover to another server. Then the first server comes

Re: [Maria-developers] how to use TC_LOG_MMAP in MySQL

2013-05-09 Thread Kristian Nielsen
nanyi607rao nanyi607...@gmail.com writes: In http://bazaar.launchpad.net/~maria-captains/maria/10.0-mdev452/revision/3434 you make some change to the class TC_LOG_MMAP,and I want to debug those codes to see how it works.but i don't understand in what conditions will MySQL use the

Re: [Maria-developers] GSoC projects related on data replication.

2013-05-09 Thread Kristian Nielsen
QIU Shuang qiush.sum...@gmail.com writes: A quick question: are the codes related to replication in the path `sql/rpl_*' and `sql/repl_*'? Much of the replication code is in: sql/log_event.* sql/slave.* sql/log.* sql/sql_repl.* sql/rpl_* sql/repl* Hope this helps, - Kristian.

Re: [Maria-developers] GTID and failovers with multi-domain replication

2013-05-08 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: assume we have server S1 that is master working with domain_id=0, server S2 is master working with domain_id=1, servers S3 and S4 are slaves and replicate from both of these masters, i.e. they have both domains in their databases. Now let's say S1 has

Re: [Maria-developers] Documentation about GTID

2013-05-08 Thread Kristian Nielsen
Roberto Spadim robe...@spadim.com.br writes: this could be put in manual just as an information of why use 64bits on gtid, it is interesting Thanks for the suggestion, I added a note about this to the docs: https://kb.askmonty.org/en/global-transaction-id/#implementation - Kristian.

Re: [Maria-developers] Documentation about GTID

2013-05-08 Thread Kristian Nielsen
Alex Yurchenko alexey.yurche...@codership.com writes: Since both are configured manually, and domain ID can simply default to 0 in simple setups, I'd imagine that the possibility of having server ID configured incorrectly (just missing to configure it) is way more probable than having domain

Re: [Maria-developers] GTID and failovers with multi-domain replication

2013-05-08 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: I love the idea with START SLAVE UNTIL. Looks very clean and reasonable. And I don't like the idea of special treatment of this in strict mode. Yeah. Let's go with START SLAVE UNTIL and keep the error when starting from non-existing GTID and no UNTIL.

Re: [Maria-developers] MariaDB GTID first impression

2013-05-07 Thread Kristian Nielsen
Giuseppe Maxia g.ma...@gmail.com writes: Thanks for the docs. It's a very good start. Compared to MySQL 5.6, GTID in MariaDB has some advantages: it is enabled by default and it is easy to read. There are a few items, though, that should need some more thought: 1) rpl_slave_state is

Re: [Maria-developers] MariaDB GTID first impression

2013-05-07 Thread Kristian Nielsen
Giuseppe Maxia g.ma...@gmail.com writes: If performance is an issue that prevents writing to the gtid table, perhaps a INFORMATIOn_SCHEMA table would do the trick? (Having the state of Yes, that sounds easy to do. Something like information_schema.gtid_pos, which has one row per domain_id,

Re: [Maria-developers] Documentation about GTID

2013-05-07 Thread Kristian Nielsen
Arjen Lentz ar...@openquery.com writes: In the section describing existing classic replication: A slave server keeps track of the position in the master's binlog of the last event applied on the slave. This allows the slave server to re-connect and resume from where it left off after

Re: [Maria-developers] MariaDB allows for slave to connect with non-existent GTID

2013-05-07 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: I'd say if S3 stopped one event earlier then there would have been no error at all. If S3 stopped one event later then sure it wouldn't be possible to detect the error, but it will be detected in strict mode. Ah, that is a good point. But what I'm not

Re: [Maria-developers] Documentation about GTID

2013-05-07 Thread Kristian Nielsen
Roberto Spadim robe...@spadim.com.br writes: hum, lets think about 10 years, maybe less what happen after 2^64 value of the last digit on gtid? 1-1-2^64 it will go back to 0? It will (and things will break), but to do 2^64 transactions in 10 years, you need to do 58 billion transactions per

Re: [Maria-developers] MariaDB allows for slave to connect with non-existent GTID

2013-05-06 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: I've realized that the way slaves are processed now on the master allows them to connect even if they request non-existent GTID. What happens here is that S3 requests GTID 0-2-3 from S1. S1 has in binlog: 0-1-1 0-1-2 0-1-3 0-2-4 So there is a hole

Re: [Maria-developers] Documentation about GTID

2013-05-06 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: I will try to get some initial docs written up by the end of the week. Giuseppe, here (attached) is my first stab at documentation for MariaDB GTID. Thanks for prodding me to get this done, even if it was a very gentle prodding :-) Daniel, do

Re: [Maria-developers] Documentation about GTID

2013-05-06 Thread Kristian Nielsen
Giuseppe Maxia g.ma...@gmail.com writes: I need just a piece of information: is this syntax applicable to the current binaries, or do I need to recompile after a specific revision in the code? The syntax should work in the 10.0.2 binaries. - Kristian.

Re: [Maria-developers] Documentation about GTID

2013-05-03 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Friendly warning: I've discovered a critical bug in GTID implementation https://mariadb.atlassian.net/browse/MDEV-4473. So use Thanks for digging this up, and even supplying a nice test case! I've pushed a fix to 10.0-base, and merged it to 10.0. The

Re: [Maria-developers] Documentation about GTID

2013-05-03 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Well, I'd argue that accumulating lots of cruft does matter. Let's say it accumulated 10,000 different GTIDs. It will significantly slow down the slave connection initialization and it will blow up binlog size. The first effect could be especially

Re: [Maria-developers] Documentation about GTID

2013-05-03 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Now that you mentioned this I'm worried. Tell me please what will happen with MariaDB in the following situation. Let's say we have Ok, here is what the current code will do. slave fully synced with master, the last GTID in binlogs (and I guess the

Re: [Maria-developers] Documentation about GTID

2013-05-03 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: This is exactly my concern currently (wasn't initially). So let's say as you said above extra transaction was executed when slave was connected, or (more probable case if possible) when IO thread was already disconnected but SQL thread didn't catch up

Re: [Maria-developers] Documentation about GTID

2013-05-03 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Thinking about such situation more it still seems to me that it would be beneficial to have an additional flag that will require for seq_no in GTIDs of committed transactions to be strictly increasing for each domain. It will add fault-tolerance to

Re: [Maria-developers] GTIDs, events sequence and alternate futures

2013-05-01 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: As I understand GTID is constructed as domain id-server id-sequence number. Is it guaranteed that sequence numbers are always increasing inside the domain id? So The intention is that the user/DBA should configure things so that there is only ever one

Re: [Maria-developers] GTIDs, events sequence and alternate futures

2013-05-01 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: What will happen with MariaDB if sem-sync master is enabled, some transactions is completed, but before it gets any semi-sync acks all slaves get disconnected? Will this transaction be rolled back? Or it will be left committed basically creating an

Re: [Maria-developers] Documentation about GTID

2013-05-01 Thread Kristian Nielsen
Giuseppe Maxia g.ma...@gmail.com writes: The only documentation I found is the worklog. Yes, docs will come of course, but they are not written yet. As Pavel mentioned, there are some changes compared to original design, that were done as a result of QA to fix some potential issues with the

Re: [Maria-developers] Installing jemalloc on build VMs

2013-04-25 Thread Kristian Nielsen
Daniel Bartholomew db...@mariadb.org writes: I'm getting ready to install jemalloc on the buildslaves for TODO-459 - https://mariadb.atlassian.net/browse/TODO-459 , and wanted to know if there was a preferred way to do it. Since we want to use a specific version of jemalloc (3.1.0), I

[Maria-developers] Missing locking around THD::set_db() ?

2013-04-23 Thread Kristian Nielsen
Hi, I was looking at a Valgrind warning in Buildbot (appended below). What happens seems to be this: Thread 1 is running SHOW PROCESSLIST, it grabs the pointer THD::db to the current database of thread 2. Thread 2 then does THD::set_db(), freeing the old THD::db pointer and allocating a new

Re: [Maria-developers] Missing locking around THD::set_db() ?

2013-04-23 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: What happens seems to be this: Thread 1 is running SHOW PROCESSLIST, it grabs the pointer THD::db to the current database of thread 2. Thread 2 then does THD::set_db(), freeing the old THD::db pointer and allocating a new one with the new

Re: [Maria-developers] GTID and replication of mysql database

2013-04-18 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Could you please tell me, in your implementation of GTIDs did you make something special for the case when mysql database is replicated? I guess mysql.rpl_slave_state cannot be replicated in any circumstances, so did you put any safeguards to not

Re: [Maria-developers] GTID and replication of mysql database

2013-04-18 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Note that the user should not update mysql.rpl_slave_state manually. Except perhaps to ALTER TABLE ... ENGINE=xxx (which can be replicated). BTW, why did you allow to create rpl_slave_state with MyISAM? I mean I Well, otherwise it becomes impossible to

Re: [Maria-developers] GTID and replication of mysql database

2013-04-18 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: I still don't quite understand when this can be necessary except maybe for tests. New slave should be restored from backup and thus it will I agree that it should not be needed in normal production use. - Kristian.

Re: [Maria-developers] I would like to run a buildbot

2013-04-16 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: Dan Demeter d...@apexis.ro writes: The other builder is set up now :) I have installed clang on the 2nd host also, but I don't know how to make the mariadb build process use clang instead of gcc / g++ ? Do I need to uninstall gcc / g

Re: [Maria-developers] I would like to run a buildbot

2013-04-16 Thread Kristian Nielsen
Vladislav Vaintroub w...@montyprogram.com writes: It should probably be just clang . Thanks, that seems to have worked. Dan, now the next step is to install curses development files: apt-get install libncurses5-dev - Kristian. ___ Mailing

Re: [Maria-developers] I would like to run a buildbot

2013-04-15 Thread Kristian Nielsen
Vladislav Vaintroub w...@montyprogram.com writes: You can do either with environment variables, or passing compiler as parameter to cmake Thanks for the info wlad! Dan: note that I will set this up on the Buildbot side. You do not need to do anything on your side except install clang. -

Re: [Maria-developers] I would like to run a buildbot

2013-04-15 Thread Kristian Nielsen
Dan Demeter d...@apexis.ro writes: The other builder is set up now :) I have installed clang on the 2nd host also, but I don't know how to make the mariadb build process use clang instead of gcc / g++ ? Do I need to uninstall gcc / g++ and make a symlink ? I will handle this from my side, as

[Maria-developers] Cassandra disabled due to excessive memory usage causing test failures

2013-04-02 Thread Kristian Nielsen
Hi, When I added my 10.0-mdev26 tree to buildbot, I got some strange errors on the kvm-fulltest2 builder. I finally tracked it down to be caused by the OOM killer killing random processes. The root issue is Cassandra. At some point it seems cassandra was enabled in the VM used by the fulltest2

Re: [Maria-developers] New problem with GTID patch

2013-03-28 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: of 10.0.1 tarball. It works okay in 10.0-mdev26 though, so I'm not sure if you'll want to look at the problem now. Yes, I would like to look. It could easily be a hidden problem in 10.0-mdev26 too, and I'll have to merge to 10.0 soon anyway.

Re: [Maria-developers] I would like to run a buildbot

2013-03-26 Thread Kristian Nielsen
Dan Demeter d...@apexis.ro writes: I have 2 hardware nodes and I would be really happy to run 2 build bots, on some VMs. :) Thanks a lot for the offer! I will create two accounts for the two nodes and get back to you (hopefully in a couple days at most). In the meantime, you can check here

Re: [Maria-developers] Bugs in GTID implementation

2013-03-26 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: way to fix them. Also if you can please look at the attached patch containing fixes to all tests after GTID addition. There are some questionable changes in there, so I would appreciate if you could evaluate its correctness and consider including it

Re: [Maria-developers] Bugs in GTID implementation

2013-03-26 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: As far as I noticed these digests change every time some change in mysql database is made, sometimes after other changes too. And I actually don't understand what these tests are testing if they need such adjustments all the time... Yeah, good question

Re: [Maria-developers] Bugs in GTID implementation

2013-03-25 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: fixes. Meanwhile I've found one more bug in your code. In Gtid_log_event::do_apply_event() you assume that mysql_parse() always succeeds, you don't check thd-is_error() and always return 0. I don't Ah, right, I see. Can you test the appended patch if

Re: [Maria-developers] accelerating CREATE TABLE

2013-03-13 Thread Kristian Nielsen
Toshikuni Fukaya toshikuni-fuk...@cybozu.co.jp writes: Hi Kristian, I am sorry, I really wanted to look more into this, but I have not been able to find the time to do so, being really busy with global transaction ID, Debian packaging, and other stuff. I will try to find someone else to look

Re: [Maria-developers] deadlock, rotate and prepared xids

2013-03-05 Thread Kristian Nielsen
MARK CALLAGHAN mdcal...@gmail.com writes: http://bugs.mysql.com/bug.php?id=68569 This is the classic semi-sync deadlock that you explained so many times already. The 5.6 group commit implements a similar technique to MariaDB, where a single thread does the fast part of commit for all

Re: [Maria-developers] deadlock, rotate and prepared xids

2013-03-05 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: http://bugs.mysql.com/bug.php?id=68569 This is the classic semi-sync deadlock that you explained so many times already. http://bugs.mysql.com/bug.php?id=68251 This second one is different from the first one. One thread is trying to I

Re: [Maria-developers] understanding commit and prepare APIs in MariaDB 5.5

2013-02-21 Thread Kristian Nielsen
Zardosht Kasheff zardo...@gmail.com writes: I am investigating what our storage engine, TokuDB, needs to do to achieve its best performance in MariaDB 5.5 with respect to preparing and committing transactions, and with respect to binary logging. Great! - In MariaDB 5.5, we must fsync our

Re: [Maria-developers] unexpected Duplicate entry error

2013-02-21 Thread Kristian Nielsen
Kazuhiko Shiozaki kazuh...@nexedi.com writes: I tried setting SERIALIZABLE isolation globally and confirmed that (much) more deadlocks happened. But unfortunately Duplicate entry error still happens. @@GLOBAL.tx_isolation: SERIALIZABLE @@tx_isolation:

Re: [Maria-developers] binlog event of COMMIT statement

2013-02-20 Thread Kristian Nielsen
nanyi607rao nanyi607...@gmail.com writes: I am little confused about binlog event of COMMIT statement, In most of time, a COMMIT statement corresponds a XID_event in the binlog, but sometimes is a Query_event. how that happened ? The rule is: If we binlog a transaction (or autocommitted

Re: [Maria-developers] unexpected Duplicate entry error

2013-02-18 Thread Kristian Nielsen
Kazuhiko Shiozaki kazuh...@nexedi.com writes: It's REPEATABLE-READ. And innodb_locks_unsafe_for_binlog is ON, could it be the reason ? Yes. A SELECT @@global.innodb_locks_unsafe_for_binlog; @@global.innodb_locks_unsafe_for_binlog 1 A SET SESSION TRANSACTION ISOLATION

Re: [Maria-developers] Help with running a transaction inside the server

2013-02-14 Thread Kristian Nielsen
Sergei Golubchik s...@askmonty.org writes: What file is it in? I didn't find it in sql_repl.cc Sorry, it is rpl_slave_state::record_gtid() in sql/log_event.cc: http://bazaar.launchpad.net/~maria-captains/maria/10.0-mdev26/view/head:/sql/log_event.cc#L6140 You could do all that in a

Re: [Maria-developers] Help with running a transaction inside the server

2013-02-14 Thread Kristian Nielsen
Sergei Golubchik s...@askmonty.org writes: Right. In fact, I think perhaps I am doing it in a single statement already: mysql_reset_thd_for_next_command(thd, 0); open_and_lock_tables(thd, tlist, FALSE, 0); table-file-ha_write_row(table-record[0]);

Re: [Maria-developers] Help with running a transaction inside the server

2013-02-14 Thread Kristian Nielsen
Michael Widenius mo...@askmonty.org writes: Why doing an insert + delete instead of one update? This is described in more detail in my blog under On crash-safe slave: http://kristiannielsen.livejournal.com/17008.html The basic problem with just one UPDATE is that this causes row lock

Re: [Maria-developers] accelerating CREATE TABLE

2013-02-08 Thread Kristian Nielsen
Toshikuni Fukaya toshikuni-fuk...@cybozu.co.jp writes: I made a patch to accelerate CREATE TABLE on the innodb plugin. To zero table spaces, I used fallocate instead of normal writes and sync. Thanks for your work. Today I took a closer look at the patch and the deeper issues. I think we need

Re: [Maria-developers] mariadb-server not installable on Debian Testing AMD64

2013-02-06 Thread Kristian Nielsen
Thomas Hackert thack...@nexgo.de writes: loLink encap:Lokale Schleife LOOPBACK MTU:16436 Metrik:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 Kollisionen:0

Re: [Maria-developers] mariadb-server not installable on Debian Testing AMD64

2013-02-05 Thread Kristian Nielsen
Thomas Hackert thack...@nexgo.de writes: Should syslog not a dependency to install mariadb-server then? Yes, this is a possibility, but just a guess on my part. Let's be sure of the actual root cause before deciding how to fix. Well, I have straced it to a file, but am not sure, what it is

Re: [Maria-developers] mariadb-server not installable on Debian Testing AMD64

2013-02-05 Thread Kristian Nielsen
Thomas Hackert thack...@nexgo.de writes: I am not sure, how (and if) I can upload to f.a.o., so I will attach it to this mail. Sorry for the inconvenience ... :( No inconvenience at all, mail is perfect. And indeed, the strace was illuminating. I think the issue is right here: [pid 32673]

Re: [Maria-developers] mariadb-server not installable on Debian Testing AMD64

2013-02-05 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: The second question is why you did not get any error log output with messages about this problem. I suppose this might be because you had no syslog installed. Elena gave you suggestions for how to resolve/investigate this further. But let me

Re: [Maria-developers] mariadb-server not installable on Debian Testing AMD64

2013-02-05 Thread Kristian Nielsen
Thomas Hackert thack...@nexgo.de writes: error: 99: Cannot assign requested address - again, this indicates EADDRNOTAVAIL, suggesting that 127.0.0.1 is missing on the system. But does it mean, I have to install bind to use mariadb-server? I would prefer to not install it ... ;) No, it has

Re: [Maria-developers] mariadb-server not installable on Debian Testing AMD64

2013-02-04 Thread Kristian Nielsen
Thomas Hackert thack...@nexgo.de writes: It seems, that it is a problem to try to stop a non-installed server before starting it, or am I wrong ;? It does not look like the problem is stopping the server. The error is about a failure to *start* the server. I had tested it a little bit

Re: [Maria-developers] accelerating CREATE TABLE

2013-02-04 Thread Kristian Nielsen
Toshikuni Fukaya toshikuni-fuk...@cybozu.co.jp writes: I want to know whether this patch is correct, and if so, could I ask you to merge this to Maria? Sorry that we have not had time to get back to you. There is a lot going on at the moment, but the whole point of MariaDB is to be a place

Re: [Maria-developers] CMake vs. autotools on Linux?

2013-02-03 Thread Kristian Nielsen
Christian Convey christian.con...@gmail.com writes: I noticed that MariaDB is built (indirectly) with CMake on Windows, and with autotools on Linux. Why are both systems used? I would have assumed that because CMake is pretty good at generating Makefiles on Linux and OS X, there would be

Re: [Maria-developers] Summary of the MDEV-3882 (.deb upgrade from mysql to mariadb fails due to apt-get considering it a downgrade)

2013-02-01 Thread Kristian Nielsen
[Moving the discussion to maria-developers@, please use the public mailing lists for discussions like this] Alexey Botchkov holyf...@askmonty.org writes: No matter what dependencies or new packages i created inside MariaDB, Debian's first idea was to try to install the 'later' version of

Re: [Maria-developers] reducing fsyncs during handlerton-prepare and handlerton-commit in 5.6

2013-01-31 Thread Kristian Nielsen
Zardosht Kasheff zardo...@gmail.com writes: - in MySQL 5.6 and MariaDB 10.0, what are the fsyncing requirements for storage engines during prepare and commit? - are there new APIs that the storage engine must comply with in order to get these benefits? In MariaDB 10.0, there is a new

Re: [Maria-developers] mariadb-server not installable on Debian Testing AMD64

2013-01-31 Thread Kristian Nielsen
Thomas Hackert thack...@nexgo.de writes: [] Starting MariaDB database server: mysqld . . . . . . . . . . . . . . . . . . . . FAIL . . . . . . . failed! invoke-rc.d: initscript mysql, action start failed. The problem is a failure while starting the server. You need to find the error

Re: [Maria-developers] [Commits] Rev 3653: MDEV-3882 .deb upgrade from mysql to mariadb fails due to apt-get considering it a downgrade. in file:///home/hf/wmar/mdev-3882/

2013-01-29 Thread Kristian Nielsen
holyf...@askmonty.org writes: message: MDEV-3882 .deb upgrade from mysql to mariadb fails due to apt-get considering it a downgrade. The libmysqlclient18 package and all the dependencies on it removed. The libmariadbclient18 set to provide the libmysqlclient's

Re: [Maria-developers] what pros/cons of storing binary log in an InnoDB table?

2013-01-28 Thread Kristian Nielsen
Zardosht Kasheff zardo...@gmail.com writes: In this email, I will focus on (and hope to understand better) just the problems with having the binary log be an InnoDB table. I don't see issues with using an increasing uint64 as the primary key. You ask how we ensure that the order in the table

Re: [Maria-developers] Bug in trx_group_commit_leader()

2013-01-21 Thread Kristian Nielsen
Michael Widenius mo...@askmonty.org writes: I noticed this compiler error on one of the build machines for 10.0-base: /home/buildbot/buildbot/build/mariadb-10.0.0/sql/log.cc:6564:40: warning: ‘last_in_queue’ may be used uninitialized in this function [-Wuninitialized] Thanks. I've pushed

Re: [Maria-developers] LP:1024058 - mysqld XA crash in replication slave

2013-01-18 Thread Kristian Nielsen
Sergei Golubchik (JIRA) j...@mariadb.atlassian.net writes: [ https://mariadb.atlassian.net/browse/MDEV-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] [Taking this to maria-developers@, which is better for discussions like this] The problem here is very simple

Re: [Maria-developers] 答复: in-order commit

2013-01-16 Thread Kristian Nielsen
Hi, I implemented the last part of in-order commit, which pushes the wait into the transaction coordinator, so that group commit can work and performance can be good. On the one hand I am really pleased to get this done, it is something I have been thinking on for 3 years now. On the other hand

Re: [Maria-developers] Structure of Binary Log

2013-01-16 Thread Kristian Nielsen
Nuno Rebocho rebocho@portugalmail.pt writes: The attractive hypothesis, develop a data recovery tool, never left my mind. I've been trying to learn together as a Maria developers team can get information about the structure of the log file from MySql to navigate this and obtain

Re: [Maria-developers] 答复: in-order commit

2013-01-11 Thread Kristian Nielsen
丁奇 dingqi@taobao.com writes: Hi, Kristian Ok. I have got the information from JIRA. I find you control the commit order inside the user thread. Will it be easier to let Trans_worker thread hold this logic? Yes, I think you are right. Of course, the user thread is the one that

Re: [Maria-developers] 答复: 答复: 答复: MDEV-520: consider parallel replication patch from taobao patches

2013-01-11 Thread Kristian Nielsen
Stewart Smith stew...@flamingspork.com writes: What about just run those txns serially rather than attempting to do in parallel? Maybe. Well, first we need to generalise MyISAM and InnoDB to a general way of putting any storage engine into one or the other category. I assume Monty or Serg can

Re: [Maria-developers] 答复: 答复: 答复: MDEV-520: consider parallel replication patch from taobao patches

2013-01-11 Thread Kristian Nielsen
Gordan Bobic gor...@bobich.net writes: But multiple inserts can occur in parallel if rows are fixed-length. This True. Probably there should be an option to allow out-of-order updates to allow MyISAM parallel replication. This will not be crash-safe (eg. for global transaction ID), but then

Re: [Maria-developers] Kubuntu 12.04.1 and MariaDB

2013-01-11 Thread Kristian Nielsen
Giacomo Picchiarelli gpicchiare...@gmail.com writes: I'm having troubles installing mariadb 10.0 in Kubuntu 12.04.1. When I try to install it, dependencies are getting enraged and apt (through synaptic or muon) suggests to remove kde-plasma-desktop. This happens when I try to install

Re: [Maria-developers] 答复: in-order commit

2013-01-11 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: 丁奇 dingqi@taobao.com writes: Hi, Kristian Ok. I have got the information from JIRA. I find you control the commit order inside the user thread. Will it be easier to let Trans_worker thread hold this logic? Yes, I think you

<    1   2   3   4   5   6   7   8   >