Re: [Maria-developers] parallel replication performance

2014-03-26 Thread Kristian Nielsen
Axel Schwenke a...@askmonty.org writes: The poor performance of MySQL with 1 database is expected. However MySQL is much better with 32 databases. Even better than MariaDB :( Ok, so I looked closer at the numbers (of which there were quite a few, thanks for doing this work :-). I have some

Re: [Maria-developers] Patch for MDEV 5873

2014-03-25 Thread Kristian Nielsen
Rohit Kalhans rohit.kalh...@gmail.com writes: As mentioned earlier i have a patch for MDEV 5873 and i am attaching here in the bundle of the patch. I have run tests on this patch and all of them have passed. Kindly review the patch. Also i am not sure which is the suitable mailing list for

Re: [Maria-developers] parallel replication performance

2014-03-25 Thread Kristian Nielsen
Axel Schwenke a...@askmonty.org writes: The poor performance of MySQL with 1 database is expected. However MySQL is much better with 32 databases. Even better than MariaDB :( I haven't had time to go through all the numbers and details yet, but just a quick check: In this test, are you sure

Re: [Maria-developers] parallel replication performance

2014-03-25 Thread Kristian Nielsen
Axel Schwenke a...@askmonty.org writes: In this test, are you sure that you are running MariaDB in GTID mode? CHANGE MASTER TO master_use_gtid=slave_pos Yes. This is how the shell code looks: mysql -S ${SLAVE_SOCKET} -u root -e SET GLOBAL gtid_slave_pos='' mysql -S ${SLAVE_SOCKET} -u

Re: [Maria-developers] MDEV-5914

2014-03-21 Thread Kristian Nielsen
Jan Lindström jplin...@mariadb.org writes: Here is necessary changes. I have run whole mysql-test-run and I did not see any failures, thus is is mostly safe. I say mostly, because we really do not have much test cases that would use multimaster setup and try to get conflicting transactions

Re: [Maria-developers] Transactions behind a failed transaction could commit in parallel replication

2014-03-21 Thread Kristian Nielsen
nanyi607rao nanyi607...@gmail.com writes: if (!rgi-is_error !skip_event_group) err= rpt_handle_event(events, rpt); else err= thd-wait_for_prior_commit(); ... ... finish_event_group(thd, err, event_gtid_sub_id, entry, rgi); if the

Re: [Maria-developers] Transactions behind a failed transaction could commit in parallel replication

2014-03-19 Thread Kristian Nielsen
nanyi607rao nanyi607...@gmail.com writes: if (unlikely(entry-stop_on_error_sub_id = rgi-wait_commit_sub_id)) skip_event_group= true; this codes can tell latter transactions to skip but can't tell them rollback. because if a transaction started commiting before a former

Re: [Maria-developers] MaxScale as a binlog server

2014-03-18 Thread Kristian Nielsen
Mark Riddoch mark.ridd...@skysql.com writes: we have been approached by a user with a suggestion for a project that we might do with them. They have a replication environment with a large fan out (100 slaves per master) and are looking at doing something to cut one the load on the master. the

Re: [Maria-developers] New MariaDB builds: regular trusty and special kvm-deb-sid-pbuilder-amd and (Re: MariaDB)

2014-03-18 Thread Kristian Nielsen
Otto Kekäläinen o...@seravo.fi writes: I assume Github.com changed how they query dates from git, as there is an option for time format output in git. I can't find any time input format options, so that is not anything we can change AFAIK. The thing is, just today I pushed a test commit to my

Re: [Maria-developers] New MariaDB builds: regular trusty and special kvm-deb-sid-pbuilder-amd and (Re: MariaDB)

2014-03-18 Thread Kristian Nielsen
Otto Kekäläinen o...@seravo.fi writes: You're right. This thing seems to be that in this time format there is no + time zone, instead it's called just 'Z'. So all commits done by Danes and Finns will work, but commits by James Page from England will fail. We can live with that and don't

Re: [Maria-developers] New MariaDB builds: regular trusty and special kvm-deb-sid-pbuilder-amd and (Re: MariaDB)

2014-03-17 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: 3902 time git-buildpackage --git-pristine-tar --git-notify=false Ok, I did that, we will see how it goes. Seems to work now, except that it fails with this: gpg: /tmp/debsign.vybBjDWq/mariadb-5.5_5.5.36-1.dsc: clearsign failed: secret key

Re: [Maria-developers] redundant compilation of targets in mariadb 10.0

2014-03-17 Thread Kristian Nielsen
Rohit Kalhans rohit.kalh...@gmail.com writes: Also if it is relevant in some way the bug i was trying to fix is MDEV-5873. i have changed some function signatures in the header and ~5 lines of change in the cc file. Oh, you're looking into that? I do hope you will share any findings :-) I was

Re: [Maria-developers] New MariaDB builds: regular trusty and special kvm-deb-sid-pbuilder-amd and (Re: MariaDB)

2014-03-14 Thread Kristian Nielsen
Hi Otto, Ok, so I setup something in Buildbot. Whenever something is pushed to git://github.com/ottok/mariadb-5.5.git, Buildbot will do a build in two VMs: trusty amd64 and sid amd64. You need to configure your repository to send notifications to Buildbot for this to take effect (I tested with

Re: [Maria-developers] New MariaDB builds: regular trusty and special kvm-deb-sid-pbuilder-amd and (Re: MariaDB)

2014-03-14 Thread Kristian Nielsen
Otto Kekäläinen o...@seravo.fi writes: Change the line 3902 time git-buildpackage --git-pristine-tar --git-notify=false --git-pbuilder --git-dist=trusty --git-arch=amd64 to 3902 time git-buildpackage --git-pristine-tar --git-notify=false Ok, I did that, we will see how it goes. Buildbot is

Re: [Maria-developers] Group commit id in mysqlbinlog output

2014-03-14 Thread Kristian Nielsen
Robert Hodges robert.hod...@continuent.com writes: a.) It seems logical that transactions within a group commit should appear together in the binlog and should be serialized before and after other transactions in the binlog. Is there *any* way this ordering could be violated, for example to

Re: [Maria-developers] two places maybe need adjusted in rpl_parallel.cc

2014-03-07 Thread Kristian Nielsen
nanyi607rao nanyi607...@gmail.com writes: First is in rpl_parallel_thread::get_rgi() adjusted like this ... else { if(!(rgi= new rpl_group_info(rli))) { my_error(ER_OUTOFMEMORY, MYF(0), (int)sizeof(*rgi)); return NULL; } rgi-is_parallel_exec = true; -

Re: [Maria-developers] A problem of DDL apply in parallel replication

2014-03-06 Thread Kristian Nielsen
nanyi607rao nanyi607...@gmail.com writes: Maybe you have encountered this problem and fixed it, that is a DDL statement Query event has a Gtid event ahead, but has no end_event (like xid_event, commit or rollback) behind. so in the latest codes of parallel replication,

[Maria-developers] Parallel replication benchmarks

2014-03-06 Thread Kristian Nielsen
I've done a set of benchmarks for parallel replication on the same machine I used previously for my group commit benchmarks, http://kristiannielsen.livejournal.com/16382.html The code tested is the newest code in the bzr repository and what will be in 10.0.9 (this is significantly improved from

Re: [Maria-developers] MariaDB slave_parallel_threads and MySQL slave_parallel_workers

2014-03-05 Thread Kristian Nielsen
Rasmus Johansson ras...@mariadb.com writes: I received a question that I wanted to share here to get some insight on. MariaDB 10.0 has a global setting named slave_parallel_threads and MySQL 5.7 has the same parameter named slave_parallel_workers. Should slave_parallel_workers be an alias

Re: [Maria-developers] gsoc replication tasks

2014-03-05 Thread Kristian Nielsen
Sergei Golubchik s...@mariadb.org writes: I'd like to discuss what replication tasks we might want to be implemented in this year's google summer of code. I don't really have the bandwidth for handling any GSoC at the moment. But here are a bunch of replication tasks that should be suitable

Re: [Maria-developers] two places maybe need adjusted in rpl_parallel.cc

2014-03-04 Thread Kristian Nielsen
nanyi607rao nanyi607...@gmail.com writes: First is in rpl_parallel_thread::get_rgi() adjusted like this ... else { if(!(rgi= new rpl_group_info(rli))) { my_error(ER_OUTOFMEMORY, MYF(0), (int)sizeof(*rgi)); return NULL; } rgi-is_parallel_exec = true; -

Re: [Maria-developers] MariaDB Master Replication To MySQL

2014-03-03 Thread Kristian Nielsen
In this particular case, we have someone that wants to use MariaDB 10.0 master with a MySQL 5.6 slave. But they are running into a problem with GTID. Generally, the recommended setup is to only replicate from old version on master to new (or same) version on slave. Replicating from new master

[Maria-developers] Problem with debug_sync, THD::enter_cond(), and kill ...

2014-02-26 Thread Kristian Nielsen
I got a crash in my parallel replication test case, and this one turned out to be caused by a fundamental problem in debug_sync. I am not sure how to solve it, so I wanted to explain the issue in case others have some ideas. The crash happens in THD::awake() when a thread is killed:

Re: [Maria-developers] Problem with debug_sync, THD::enter_cond(), and kill ...

2014-02-26 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Looking at the code it looks like most places (including THD::awake) assume that thd-mysys_var-current_mutex and thd-mysys_var-current_cond can be changed only when thd-mysys_var-mutex is locked. debug_sync.cc is breaking this assumption. I've found

Re: [Maria-developers] How handle_rpl_parallel_thread rollback partial transaction when STOP SLAVE In parallel replication

2014-02-26 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: I put my current patch here: http://knielsen-hq.org/parallel_replication_patch_intermediate.diff Hm, that patch is not sufficient, the bug is still there :-( Sorry about that. At least I did say that the patch was incomplete. I

Re: [Maria-developers] How handle_rpl_parallel_thread rollback partial transaction when STOP SLAVE In parallel replication

2014-02-25 Thread Kristian Nielsen
nanyi607rao nanyi607...@gmail.com writes: hi Kristian, Hi nanyi607rao, In handle_rpl_parallel_thread(), a worker thread has got a whole transaction events, but it only apply partial events. this thread wouldl skip left events when do STOP SLAVE(is that rigth?) ,because

Re: [Maria-developers] How handle_rpl_parallel_thread rollback partial transaction when STOP SLAVE In parallel replication

2014-02-25 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: The idea to fix this isas follows: - We record which transactions have started to commit - When we do STOP SLAVE we remember the transaction that last started to commit at the point at which we stopped

Re: [Maria-developers] MariaDB config for Akonadi - suggestions?

2014-02-12 Thread Kristian Nielsen
Sergey Petrunia ser...@mariadb.com writes: * innodb_flush_log_at_trx_commit. Currently, Akonadi uses innodb_flush_log_at_trx_commit=2 which as far as I understand can cause up to 1 second of changes to be lost. Using value of 1 (always flush on commit) should be safer, but maybe it would

Re: [Maria-developers] Next steps in improving single-threaded performance

2014-02-12 Thread Kristian Nielsen
Axel Schwenke a...@askmonty.org writes: I propose the following change to make it work with MariaDB releases before 10.0.4 (and MySQL) that lack the SHUTDOWN statement: Nice, thanks Axel! - Kristian. ___ Mailing list:

Re: [Maria-developers] MariaDB

2014-02-04 Thread Kristian Nielsen
Otto Kekäläinen o...@seravo.fi writes: 2014-01-31 Kristian Nielsen kniel...@knielsen-hq.org: Wouldn't it be better to discard the existing debian/ stuff in the upstream sources, and work from the native Debian packaging as the base? The existing stuff has _so_ much legacy and crap

Re: [Maria-developers] Extra copying in debian/rules causes troubles

2014-02-04 Thread Kristian Nielsen
Elena Stepanova ele...@montyprogram.com writes: I'm ready to do so, but Otto says that he's currently working on the contents of the debian folder and is getting ready to push it, and thus I should probably not change anything in there. On the other hand, it might be nice to have the change

Re: [Maria-developers] MariaDB build dependencies

2014-02-03 Thread Kristian Nielsen
Otto Kekäläinen o...@seravo.fi writes: Can anybody tell me how in particular these are supposed to affect the build process? - libevent-dev I think libevent was at some point used for the thread pool. But that the current thread pool implementation does not use libevent due to performance

Re: [Maria-developers] buildbot for non ~maria-captains branches on launchpad

2014-01-30 Thread Kristian Nielsen
Daniel Bartholomew db...@mariadb.com writes: On Wed, Jan 29, 2014 at 4:01 AM, Alexander Barkov b...@mariadb.org wrote: Would it be possible to have buildbot watch branches which are on Launchpad, but which are outside of the /~maria-captains/ directory? Sure. It can be configured to do so.

Re: [Maria-developers] MariaDB

2014-01-30 Thread Kristian Nielsen
Otto Kekäläinen o...@seravo.fi writes: 2013-10-01 Otto Kekäläinen o...@fsfe.org: 2013/10/1 Ondřej Surý ond...@sury.org: - it would be better to repack the tarball and kill the upstream debian/ directory to not mess with our packaging I hope to merge my work upstream and that is easier if

Re: [Maria-developers] Extra copying in debian/rules causes troubles

2014-01-30 Thread Kristian Nielsen
Elena Stepanova ele...@montyprogram.com writes: ifeq ($(findstring nocheck,$(DEB_BUILD_OPTIONS)),) # Don't know why the following is necessary... cp unittest/unit.pl $(builddir)/unittest/ cp -r mysql-test/* $(builddir)/mysql-test/ cp -r sql/share/*

Re: [Maria-developers] Next steps in improving single-threaded performance

2014-01-28 Thread Kristian Nielsen
Sergey Vojtovich s...@mariadb.org writes: Still a questions mostly to educate myself. According to proc mysqld executable size is something like: VmExe: 12228 kB VmLib:6272 kB I assume the above refers to overall instructions. Level 1 instruction cache size is like

Re: [Maria-developers] Next steps in improving single-threaded performance

2014-01-28 Thread Kristian Nielsen
Axel Schwenke a...@askmonty.org writes: Wow. 25% is a lot. Have you also tried compiling MySQL 5.6 with PGO? No. Because if that gets the same improvement, we haven't won anything in the comparison. On the contrary, if the same works for MySQL 5.6 (and it seems likely it will), then we have

Re: [Maria-developers] Next steps in improving single-threaded performance

2014-01-27 Thread Kristian Nielsen
Sergey Vojtovich s...@mariadb.org writes: just out of curiosity: is it possible to find out which functions cause highest amount of icache misses? Yes, see the second post, the profiles marked Icache misses (ICACHE.MISSES), before PGO and Icache misses (ICACHE.MISSES), after PGO. These are

Re: [Maria-developers] [Commits] Rev 3965: MDEV-5403 - Reduce usage of LOCK_open: tc_count in lp:maria/10.0

2014-01-24 Thread Kristian Nielsen
Sergey Vojtovich s...@mariadb.org writes: +#ifdef __cplusplus +template typename T, int S class Atomic_type_triat; + + +template typename T class Atomic_type_triatT, 4 +template typename T class Atomic_type_triatT, 8 +template typename T, typename S= Atomic_type_triatT, sizeof(T) class

Re: [Maria-developers] [Commits] Rev 3965: MDEV-5403 - Reduce usage of LOCK_open: tc_count in lp:maria/10.0

2014-01-24 Thread Kristian Nielsen
I really do not like hiding the type. If you did not think carefully about the type of the underlying variable, you _really_ should not be using something as complex and subtle as an atomic operation. And I _really_ do not like that we would now have two _different_ ways to do atomic operations,

[Maria-developers] Next steps in improving single-threaded performance

2014-01-24 Thread Kristian Nielsen
I have been analysing CPU bottlenecks in single-threaded sysbench read-only load. I found that icache misses is the main bottleneck, and that profile-guided compiler optimisation (PGO) with GCC gives a large speedup, 25% or more. (More details in my blog posts:

Re: [Maria-developers] uint6korr optimization

2014-01-23 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: Do it like this: static inline ulonglong mi_uint6korr(const void *p) { uint32 a= *(uint32 *)p; uint16 b= *(uint16 *)(4+(char *)p); ulonglong v= ((ulonglong)a | ((ulonglong)b 32)) 16; asm (bswapq %0 : =r (v) : 0 (v)); return v

Re: [Maria-developers] benchmarking next steps

2014-01-20 Thread Kristian Nielsen
MARK CALLAGHAN mdcal...@gmail.com writes: Are there other goals for single-thread performance regressions? There are many that have nothing to do with replication code although almost every single-threaded replication hurts replication. I am planning to do a general in-depth analysis of

Re: [Maria-developers] [Commits] Rev 3964: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg in file:///home/psergey/dev2/10.0-fix1/

2014-01-20 Thread Kristian Nielsen
Sergey Petrunya pser...@askmonty.org writes: revno: 3964 revision-id: pser...@askmonty.org-20140120123657-g1z4g11xmsdiw2dc committer: Sergey Petrunya pser...@askmonty.org message: MDEV-4816: rpl.rpl_trunc_temp fails in 10.0-serg

Re: [Maria-developers] New message from Yoshinori Matsunobu

2014-01-20 Thread Kristian Nielsen
Facebook notification+...@facebookmail.com writes: I read your latest blog post about profile-guided optisation, which is really interesting. I have one question.. What perf options/arguments did you use to get INST_RETIRED.ANY and ICACHE.MISSES ? perf stat -e cache-misses? I'm not sure

[Maria-developers] Need Review: MDEV-5509: Seconds_behind_master incorrect in parallel replication in http://bazaar.launchpad.net/~maria-captains/maria/10.0

2014-01-08 Thread Kristian Nielsen
Could someone review this patch for MDEV-5509? I am in particular worried that this might change some subtle behaviour of Seconds_Behind_Master in unexpected ways. Of course I tried my best to make sure such could not occur, but the semantics is really tricky and poorly documented, and I do not

[Maria-developers] MDEV-5115 review

2013-12-04 Thread Kristian Nielsen
Hi Serg, Here is my review of your MDEV-5115 patch. First, a general comment: Another way to fix this would be a smaller patch. It would not attempt to implement anything related to the new event types. Instead, it would just have a small bit of code that allows to read the binary format of the

Re: [Maria-developers] MariaDB-10.0-beta sysbench results

2013-11-22 Thread Kristian Nielsen
MARK CALLAGHAN mdcal...@gmail.com writes: Resources are finite, but it would be great if MariaDB also did something about single-thread performance regressions. MySQL has not done/said much Do you have some pointers about where to start with this? I am thinking about which types of queries /

Re: [Maria-developers] Accessing rpl_global_gtid_binlog_state is not consistently protected

2013-11-18 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: sql/log.cc has highly inconsistent pattern of accessing rpl_global_gtid_binlog_state. In some places it's protected by mutex rpl_global_gtid_binlog_state.LOCK_binlog_state, in some places it's protected by global mutex LOCK_rpl_gtid_state, and in some

Re: [Maria-developers] MDEV-5277 - Ensure that all MySQL 5.6 options are supported by the MariaDB 10.0 server

2013-11-15 Thread Kristian Nielsen
Sergey Vojtovich s...@mariadb.org writes: * binlog-max-flush-queue-time This is not needed (it is about MySQL 5.6 group commit, MariaDB uses a different implementation. I suppose one could say it is always set to 0). * binlog-row-image This is not merged to 10.0 yet (as far as I know). It

Re: [Maria-developers] MDEV-5277 - Ensure that all MySQL 5.6 options are supported by the MariaDB 10.0 server

2013-11-15 Thread Kristian Nielsen
Sergey Vojtovich s...@mariadb.org writes: * enforce-gtid-consistency * gtid-mode This is for MySQL 5.6 GTID, which will not be in MariaDB (we have a completely different GTID implementation). So they should not be implemented, nor should we have dummy options for compatibility. You

Re: [Maria-developers] Semisync plugin incompatibility

2013-11-14 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: So basically my question is: if I prepare a patch that will restore the original behavior of semi-sync replication (and remove the tests added for Bug#45672) will that be acceptable for MariaDB? I don't have anything against it, as I said I do not have

Re: [Maria-developers] Slave can take a very long time to start replication

2013-11-14 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: start replicating it passes GTID to start from, master finds binlog file where the earliest GTID is located and then scans through that file to find the exact binlog position to start sending binlog events from. If this binlog file is pretty big then

Re: [Maria-developers] Semisync plugin incompatibility

2013-11-11 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: We've noticed recently that semisync_master plugin in MariaDB (which apparently was fully inherited from MySQL) is seriously incompatible with our understanding of the purpose of semi-sync replication. This incompatibility was apparently introduced as a

Re: [Maria-developers] Preparing MDEV-4506, Parallel replication, for 10.0.5 release

2013-11-03 Thread Kristian Nielsen
Michael Widenius mo...@askmonty.org writes: Did you forget to push the latest changes into mdev-4506-base ? Yes, sorry about that. If all tests passes in your 10.0-base, please push the changes into 10.0-base tree. Done. I have now one failing test in suite/rpl in mdev-4506 that I am

Re: [Maria-developers] Comments for parallel replication patch (MDEV-4506)

2013-10-18 Thread Kristian Nielsen
Michael Widenius michael.widen...@gmail.com writes: What about Rotate_log_event::do_update_pos() and Stop_log_event::do_update_pos()? They currently access the flag in rli. I am not sure that is right, as this could be far ahead of the event they are actually executing? Aren't the above

Re: [Maria-developers] Wrong SQL thread position reporting after IO thread restart

2013-10-11 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Here's my approach to fixing both of these problems if you are interested. Thanks. My patch does basically the similar thing, though I changed some details (MDEV-5130). - Kristian. ___ Mailing list:

[Maria-developers] Comments for parallel replication patch (MDEV-4506)

2013-10-07 Thread Kristian Nielsen
Hi Monty, I read through your latest patch for the MDEV-4506 task, parallel replication. Just to re-cap, this is a work-in-progress patch, so these are just some comments along the way that I hope will be useful. -static bool sql_slave_killed(THD* thd, Relay_log_info* rli) +static bool

Re: [Maria-developers] Galera-10.0 still does not work with rollbacks

2013-09-27 Thread Kristian Nielsen
seppo.jaak...@codership.com writes: Did I understand it right that the problem is with ROLLBACK TO SAVEPOINT..., where the designated savepoint does not exist? Did you No. It is a very simple issue. The beginning of binlog_savepoint_set() reads: #ifdef WITH_WSREP if

Re: [Maria-developers] Review of MDEV-4506, parallel replication, part 1

2013-09-26 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: So we should introduce rgi-abort_slave, and set it during STOP SLAVE for all queued rgi entries. Actually, we do not need to introduce another flag. We can just check rgi-rli-abort_slave. - Kristian

Re: [Maria-developers] Galera-10.0 still does not work with rollbacks

2013-09-25 Thread Kristian Nielsen
Jan Lindström jplin...@mariadb.org writes: The point I do not understant is at ha_savepoint we have again: if ((err= ht-savepoint_set(ht, thd, (uchar *)(sv+1)+ht-savepoint_offset))) (gdb) p *sv $3 = {prev = 0xa5a5a5a5a5a5a5a5, name = 0x7f505c007390 A, length = 1, ha_list =

Re: [Maria-developers] Review of MDEV-4506, parallel replication, part 1

2013-09-25 Thread Kristian Nielsen
Hi Monty, So as promised, I took a look at the existing code for STOP SLAVE, and came up with some ideas for how to extend this to handle parallel replication. In existing code, STOP SLAVE ends up in terminate_slave_threads(). The interesting part here is the SQL thread; stopping the IO thread

Re: [Maria-developers] Galera-10.0 still does not work with rollbacks

2013-09-25 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: if ((err= ht-savepoint_set(ht, thd, (uchar *)(sv+1)+ht-savepoint_offset))) Note that it is not sv+1. It is ((uchar *)sv+1) + ht-savepoint_offset. Sorry, I messed up that explanation. The point is: sv points to a SAVEPOINT object. But (uchar

Re: [Maria-developers] Galera-10.0 still does not work with rollbacks

2013-09-24 Thread Kristian Nielsen
Jan Lindström jplin...@mariadb.org writes: Full unedited log using --valgrind and --mysqld=--debug=+d at location: https:/ /www.dropbox.com/s/dcufp7l1cch8dfs/mysql.err.gz Ok, thanks. Hm, this log looks different from what I am used to seeing when I use mysql-test-run.pl --debug. From

Re: [Maria-developers] Galera-10.0 still does not work with rollbacks

2013-09-24 Thread Kristian Nielsen
this helps, - Kristian. Kristian Nielsen kniel...@knielsen-hq.org writes: Jan Lindström jplin...@mariadb.org writes: Full unedited log using --valgrind and --mysqld=--debug=+d at location: https:/ /www.dropbox.com/s/dcufp7l1cch8dfs/mysql.err.gz Ok, thanks. Hm, this log looks different

Re: [Maria-developers] Review of MDEV-4506, parallel replication, part 1

2013-09-23 Thread Kristian Nielsen
Michael Widenius mo...@askmonty.org writes: Kristian We _do_ need a full memory barrier here (memory barrier is implied in taking a Kristian mutex). Otherwise the compiler or CPU could re-order the setting of the Kristian wakeup_subsequent_commits_running flag with the reads and writes

Re: [Maria-developers] Review of MDEV-4506, parallel replication, part 1

2013-09-23 Thread Kristian Nielsen
Michael Widenius mo...@askmonty.org writes: What should happen if you kill a replication thread is that replication should stop for that master. Kristian This needs more thought, I think ... certainly something looks not right. After looking at the full code, I think that the logical way

Re: [Maria-developers] Galera-10.0 still does not work with rollbacks

2013-09-20 Thread Kristian Nielsen
Jan Lindström jplin...@mariadb.org writes: Yes, I did run using valgrind, while it reported some uninitialized variables, not really any memory corruptions.  Attached, the log from node0. Similarly, Well, uninitialised variables are exactly what I would expect could lead to things looking like

Re: [Maria-developers] Galera-10.0 still does not work with rollbacks

2013-09-20 Thread Kristian Nielsen
Jan Lindström jplin...@mariadb.org writes: Correct log file attached. ==00:00:02:33.708 25289== Use of uninitialised value of size 8 ==00:00:02:33.708 25289==at 0x5EF865B: _itoa_word (_itoa.c:179) ==00:00:02:33.708 25289==by 0x5EFCB91: vfprintf (vfprintf.c:1654) ==00:00:02:33.708

Re: [Maria-developers] Galera-10.0 still does not work with rollbacks

2013-09-19 Thread Kristian Nielsen
Jan Lindström jplin...@mariadb.org writes: I need help. After merge galera-10.0 with 10.0, rollback asserts. I seem not to be able to find the actual reason. I added some extra output on log.cc to dump out the cache_log data from both trx_cache and stmt_cache, but to me these values do not

Re: [Maria-developers] Review of MDEV-4506, parallel replication, part 1

2013-09-17 Thread Kristian Nielsen
Michael Widenius mo...@askmonty.org writes: + if (list-wakeup_subsequent_commits_running) + { +mysql_mutex_lock(list-LOCK_wait_commit); +list-wakeup_subsequent_commits_running= false; +mysql_mutex_unlock(list-LOCK_wait_commit); Why do we need the mutex

Re: [Maria-developers] [All] MariaDB's best (download) month ever

2013-09-17 Thread Kristian Nielsen
Colin Charles b...@mariadb.com writes: It's 17.09 today, and we've hit 50,160 total downloads (this number is considered company-internal, we don't share this number externally). company-internal? What have you been smoking? Colin, you are supposed to be the community guy for crying out lout

Re: [Maria-developers] Review of patch for MDEV-4820

2013-09-15 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: 3. Set gtid_binlog_state and gtid_slave_pos to '0-3-10' on S1 and to '0-1-1' on S2. Try to start slave on S2. Now I get error slave has diverged. What gives? It's not diverged, it's just behind. Yes, it is diverged. S1 has binlog state '0-3-10'. This

Re: [Maria-developers] [Commits] Rev 3810: MDEV-4994: Crash in dynamic column API in file:///home/bell/maria/bzr/work-maria-10.0-MDEV-4994/

2013-09-13 Thread Kristian Nielsen
sa...@askmonty.org writes: At file:///home/bell/maria/bzr/work-maria-10.0-MDEV-4994/ revno: 3810 revision-id: sa...@askmonty.org-20130911151436-rjz6pw6fh6rtj83e parent: s...@mariadb.org-20130904072837-u3zj89fq4xvb4cnp committer:

Re: [Maria-developers] [Commits] Rev 3810: MDEV-4994: Crash in dynamic column API in file:///home/bell/maria/bzr/work-maria-10.0-MDEV-4994/

2013-09-13 Thread Kristian Nielsen
Sorry about the empty reply. Slip of my fingers, Emacs provides very nice facilities for working quickly, and just a bit too quickly in this case ;-) - Kristian. ___ Mailing list: https://launchpad.net/~maria-developers Post to :

Re: [Maria-developers] Review of MDEV-4506, parallel replication, part 1

2013-09-13 Thread Kristian Nielsen
Monty, Thanks for the comments so far. I have pushed fixes to 10.0-knielsen, and some specific replies are below. - Kristian. Michael Widenius mo...@askmonty.org writes: + --slave-parallel-threads=# + If non-zero, number of threads to spawn to apply in + parallel events on the slave that

Re: [Maria-developers] Replicating same server_id problem

2013-09-11 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Kristian Nielsen kniel...@knielsen-hq.org writes: So perhaps it is better to just say that --replicate-same-server-id does not apply to GTID mode at all. Instead, in GTID mode, if we receive an event with our own server_id and smaller seq_no than what

Re: [Maria-developers] Rollback causes assertion with Galera 10.0

2013-09-10 Thread Kristian Nielsen
Jan Lindström jplin...@mariadb.org writes: (gdb) p cache_log $1 = {pos_in_file = 11936128518282651045, end_of_file = 18446744073709547520, (gdb) p *cache_log.pos_in_file Cannot access memory at address 0xa5a5a5a5a5a5a5a5 Ouch. That looks like memory corruption :-( Seems something has

Re: [Maria-developers] Rollback causes assertion with Galera 10.0

2013-09-09 Thread Kristian Nielsen
Jan Lindström jplin...@mariadb.org writes: #3 0x7f1ff9877eb2 in __GI___assert_fail (assertion=0x109b4ff empty(), file=0x109b4d8 /home/jan/mysql/galera-10.0/sql/log.cc, line=294, function= 0x10a01e0 binlog_cache_data::reset()::__PRETTY_FUNCTION__ void binlog_cache_data::reset()) at

Re: [Maria-developers] Incorrect format description event skipping in queue_event()

2013-09-04 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Thanks. This patch works for me. Ok, I'll push it then. Note that there's also Query_log_event::peek() that should have the same dependency on common_header_len. It's hard (if at all possible) for me to experience that code path in my tests, but for

Re: [Maria-developers] Replicating same server_id problem

2013-09-03 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: --replicate-same-server-id flag which as I understand (when set to 0) controls two things: 1) It doesn't allow slave to connect to a master with the same server_id. 2) Slave ignores all binlog events in the replication stream that have the same

Re: [Maria-developers] Replicating same server_id problem

2013-09-03 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: The main problem I see is what should be the default? I suppose we cannot safely change the default for --replicate-same-server-id. On the other hand, if users explicitly set --replicate-same-server-id=0, then it really does not seem correct

Re: [Maria-developers] Incorrect format description event skipping in queue_event()

2013-09-03 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Why not change Format_description_log_event::write()? It seems actually that it will be more correct if we change it. I made the following change (line numbers may be off) and it worked: Agree, that sounds like a good idea, thanks! If you are

Re: [Maria-developers] FWD: Status on MDEV-4506, parallel replication

2013-09-02 Thread Kristian Nielsen
Michael Widenius mo...@askmonty.org writes: On the master, I implemented --binlog-commit-wait-count=N and --binlog-commit-wait-usec=T. A transaction will wait at most T microseconds for at least N transactions to queue up and be ready for group commit. This allows to deliberately delay

Re: [Maria-developers] Incorrect format description event skipping in queue_event()

2013-08-30 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: The attached patch doesn't work properly because the format description event it saves is different from what master has sent. It has different number_of_event_types and potentially it has garbage (or incomplete data) in post_header_len. Aha, so it's

Re: [Maria-developers] Incorrect format description event skipping in queue_event()

2013-08-29 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: But this must be the same problem with normal replication? Whenever the slave decides to rotate the relay log, it will write a format description event created by itself with no following format description created on the master. So it seems this must

Re: [Maria-developers] Incorrect format description event skipping in queue_event()

2013-08-28 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: You have the following comment in the queue_event() in sql/slave.cc: /* Do not queue any format description event that we receive after a reconnect where we are skipping over a partial event group received before the reconnect.

Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-24 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: I took 10.0-base r3685. Started new just bootstrapped server with server_id = 1. It has @@global.gtid_binlog_pos, @@global.gtid_slave_pos and @@global.gtid_current_pos empty. Then I execute set global gtid_binlog_state = '0-10-10' After that

Re: [Maria-developers] Buildbot failure in 10.0-base

2013-08-23 Thread Kristian Nielsen
Sergei Petrunia pser...@askmonty.org writes: 10.0-base started to get compile failures on [nearly] all hosts after your yesterday push: Thanks for the notice! I've pushed a fix and it seems to have solved it. - Kristian. ___ Mailing list:

[Maria-developers] Additional documentation for GTID

2013-08-23 Thread Kristian Nielsen
Hi Daniel, Ian, I have implemented a new system variable for global transaction ID and pushed it into 10.0-base (so it should appear in 10.0.5 I suppose). I have written some text for documenting the new variable, appended below. Can one of you help me get this text into the Knowledgebase with

Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-22 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Could you say are you working on these? Is there an ETA? This is blocking us from pushing MariaDB into testing in the near-production environment, and I'm hesitant to implement fixes myself because I'd think you'll do it completely differently. If you

Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-22 Thread Kristian Nielsen
Kristian Nielsen kniel...@knielsen-hq.org writes: If you have time to test this then that would be a nice help. I'll see if I can come up with a quick patch (ie. later today or tomorrow). Please try this patch and let me know if you find any issues. I still need to implement test cases

Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-19 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: I think to fix this bug we should stop using gtid_slave_pos as indication of the current db state. We should make it possible to Agree. change gtid_binlog_pos when there's no events in binlogs. And when Ok. Actually, I think we should expose the real

Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-15 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Have I misunderstood what you said? Yes, totally :-( By that you are basically saying: I need to support those who set up lousy multi-master replication, thus I won't treat domain_id as a single replication stream, I'll treat a pair (domain_id,

Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-15 Thread Kristian Nielsen
Vilho Raatikka vilho.raati...@skysql.com writes: Is this documented profoundly already? Should be because probably we'll see at least some users as ignorant as I am asking the same questions soon. I wrote up some comprehensive documentation in the knowledgebase, and also wrote a series of

Re: [Maria-developers] Fwd: [Commits] Rev 3674: MDEV-4645: Incorrect reads of frozen binlog events; FDE corrupted in relay log in file:///home/bell/maria/bzr/work-maria-10.0-base-MDEV-4645/

2013-08-15 Thread Kristian Nielsen
Oleksandr Byelkin sa...@montyprogram.com writes: Could you review this? (it is https://mariadb.atlassian.net/browse/MDEV-4645 ) Ok, but it will be at least a couple more days before I will have time to start. If urgent, you can also ask Sergei Golubchik, IIRC this is the patch from Jeremy

Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-14 Thread Kristian Nielsen
SHOW VARIABLES or SELECT @@GLOBAL.gtid_strict_mode will show the current value. But there is no way to check if it was enabled eg. two weeks ago, if that is what you are asking (the value is not stored in the binlog). That is really pity. Is it really so useful to give the user opportunity to

Re: [Maria-developers] ALTER ONLINE TABLE syntax

2013-08-14 Thread Kristian Nielsen
Sergei Golubchik s...@mariadb.org writes: Unless we redefine what online mean. If not, there's no choice but stop claming an online alter support and remove ONLINE and LOCK=NONE from the syntax. Opinions? I think ONLINE should mean: no exclusive lock is taken while data is migrated from the

Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-13 Thread Kristian Nielsen
Jan Lindström jplin...@mariadb.org writes: I found this disturbing and not fully follow what kind of holes are possible. These GTIDS can be used by human users to start slaves on For example, if you use some of the many filtering options, like --replicate-ignore-*. Then there will be GTIDs on

Re: [Maria-developers] Review of patch for MDEV-4820

2013-08-13 Thread Kristian Nielsen
Pavel Ivanov piva...@google.com writes: Note that the patch I've attached have test case that should reproduce the problems. Thanks, I've now gone through the testcases also. Let me number the individual tests as follows: 1. Check that gap in seq_no without binlogs doesn't allow to replicate

<    1   2   3   4   5   6   7   8   >