Hi, Presumably, if the server is using too much ram it will a) crash and leave a stack trace (not happening) or b) invoke the OOMPK which leaves a trace in syslog, which also, presumably, is not happening, since she can find no trace of OOMPK activity.
So memory is a good guess (output of free -m would be useful) I doubt it is memory related do to lack of evidence for it. --Justin On Tue, Mar 3, 2015 at 9:58 AM, jocelyn fournier <[email protected] > wrote: > Hi, > > No variables to tweak the tokudb memory allocation ? > By default tokudb_cache_size takes half of the total system memory. > With 6G allocated to InnoDB buffer pool, and 1G to the MyISAM key buffer, > are you sure you have enough memory for TokuDB ? > > Jocelyn > > > Le 03/03/2015 17:37, Gabriel Sosa a écrit : > > @joocelyn Here is the my.cfg file > > @justin I will check that post. Thanks > > [mysqld] > > # > ------------------------------------------------------------------------------- > # ++++ General > # > ------------------------------------------------------------------------------- > datadir = /var/lib/mysql/data > pid-file = /var/run/mysqld/mysqld.pid > socket = /var/lib/mysql/mysql.sock > tmpdir = /var/lib/mysql/tmp > port = 3306 > > # > ------------------------------------------------------------------------------- > # ++++ Logging > # > ------------------------------------------------------------------------------- > log-error = /var/lib/mysql/log/mysqld-error.log > long_query_time = 10 > slow-query-log-file = > /var/lib/mysql/log/mysqld-queries_slow.log > log-slave-updates > log-bin = /var/lib/mysql/log/bin/slave-bin > log-warnings = 1 > > max_relay_log_size = 200M > relay_log_space_limit = 25000M > > # > ------------------------------------------------------------------------------- > # ++++ Network > # > ------------------------------------------------------------------------------- > max_connections = 3000 > max_connect_errors = 1000 > wait_timeout = 120 > connect_timeout = 30 > interactive_timeout = 3600 > slave_net_timeout = 120 > back_log = 50 > max_allowed_packet = 128M > > > # > ------------------------------------------------------------------------------- > # ++++ Misc > # > ------------------------------------------------------------------------------- > # Text Searchs > ft_min_word_len = 2 > plugin-load = ha_tokudb > > # > ------------------------------------------------------------------------------- > # ++++ Threads > # > ------------------------------------------------------------------------------- > thread_concurrency = 8 > thread_cache = 64 > > # > ------------------------------------------------------------------------------- > # ++++ Memory > # > ------------------------------------------------------------------------------- > # Tables > table_cache = 4096 > tmp_table_size = 256M > # Memory per Thread > sort_buffer_size = 8M > read_buffer_size = 4M > read_rnd_buffer_size = 16M > # Query Cache > query_cache_type = 1 > query_cache_limit = 2M > query_cache_size = 64M > > # > ------------------------------------------------------------------------------- > # ++++ MyISAM Parameters > # > ------------------------------------------------------------------------------- > key_buffer_size = 1024M > myisam_sort_buffer_size = 64M > myisam_recover = FORCE,BACKUP > > # > ------------------------------------------------------------------------------- > # ++++ InnoDB Parameters > # > ------------------------------------------------------------------------------- > # General > innodb_data_home_dir = /var/lib/mysql/innodb > innodb_log_group_home_dir = /var/lib/mysql/innodblogs > innodb_file_per_table > innodb_data_file_path = ibdata1:100M:autoextend > innodb_status_file = ib_status > innodb_autoextend_increment = 10M > innodb_support_xa = 0 > innodb_thread_concurrency = 8 > innodb_flush_method = O_DIRECT > innodb_flush_log_at_trx_commit = 2 > # Memory > innodb_buffer_pool_size = 6G > innodb_additional_mem_pool_size = 8M > innodb_open_files = 512 > # Logging > innodb_log_buffer_size = 8M > innodb_log_file_size = 256M > innodb_log_files_in_group = 2 > > > # > ------------------------------------------------------------------------------- > # ++++ Replication : SLAVE Profile > # > ------------------------------------------------------------------------------- > #skip-slave-start > server-id = 124388 > relay-log = /var/lib/mysql/log/replication/slave-bin > relay-log-info-file = /var/lib/mysql/log/replication/ > slave-log.info > master-info-file = /var/lib/mysql/log/replication/ > master-log.info > max_binlog_size = 20971520 > > read-only = 1 > > [mysqldump] > quick > max_allowed_packet = 128M > > [isamchk] > key_buffer = 256M > sort_buffer_size = 256M > read_buffer = 2M > write_buffer = 2M > > [myisamchk] > key_buffer = 256M > sort_buffer_size = 256M > read_buffer = 2M > write_buffer = 2M > > [mysqlhotcopy] > interactive-timeout > > On Tue, Mar 3, 2015 at 12:56 PM, Justin Swanhart <[email protected]> > wrote: > >> Hi, >> >> You might try to find the source of the termination with this: >> >> http://www.percona.com/blog/2014/07/18/systemtap-solves-phantom-mysqld-sigterm-sigkill-issue/ >> >> On Tue, Mar 3, 2015 at 8:45 AM, jocelyn fournier < >> [email protected]> wrote: >> >>> Hi, >>> >>> Could you send your my.cnf ? >>> >>> Thanks, >>> Jocelyn >>> >>> Le 03/03/2015 16:24, Gabriel Sosa a écrit : >>> >>> Hello, >>> >>> I've been a proudly user of mariadb 5.5.x for a long time now but >>> given the nice feature setI decided to give mariadb 10.x a try. >>> >>> I took one of our current slaves running mariadb 5.5.x (on centos 6.5) >>> and followed the upgrade steps using yum and ran *mysql_upgrade*. The >>> upgrade ran without any trouble...then I moved a huge table (about 1B >>> records right now) from InnoDB to TokuDB. >>> >>> Now, every couple of hours I find that the replication is far behind >>> the master and the reason is because this server keeps checking tables >>> marked as crashed.... >>> >>> I can't seems to find any indicator of OOM killer in the system logs >>> NOR anything related to that in the mysql log, the only clue I have is: >>> >>> >>> --------------- >>> *150303 10:08:57 mysqld_safe Number of processes running now: 0* >>> *150303 10:08:57 mysqld_safe mysqld restarted* >>> 150303 10:08:58 [Warning] 'THREAD_CONCURRENCY' is deprecated and will be >>> removed in a future release. >>> 150303 10:08:59 [Warning] option 'innodb-status-file': boolean value >>> 'ib_status' wasn't recognized. Set to OFF. >>> 150303 10:08:59 [Warning] option 'innodb-autoextend-increment': unsigned >>> value 10485760 adjusted to 1000 >>> 150303 10:08:59 [Note] InnoDB: Using mutexes to ref count buffer pool >>> pages >>> 150303 10:08:59 [Note] InnoDB: The InnoDB memory heap is disabled >>> 150303 10:08:59 [Note] InnoDB: Mutexes and rw_locks use GCC atomic >>> builtins >>> 150303 10:08:59 [Note] InnoDB: Memory barrier is not used >>> 150303 10:08:59 [Note] InnoDB: Compressed tables use zlib 1.2.3 >>> 150303 10:08:59 [Note] InnoDB: Using Linux native AIO >>> 150303 10:08:59 [Note] InnoDB: Using CPU crc32 instructions >>> 150303 10:08:59 [Note] InnoDB: Initializing buffer pool, size = 6.0G >>> 150303 10:08:59 [Note] InnoDB: Completed initialization of buffer pool >>> 150303 10:08:59 [Note] InnoDB: Highest supported file format is >>> Barracuda. >>> 150303 10:08:59 [Note] InnoDB: Log scan progressed past the checkpoint >>> lsn 14178608217047 >>> 150303 10:08:59 [Note] InnoDB: Database was not shutdown normally! >>> 150303 10:08:59 [Note] InnoDB: Starting crash recovery. >>> 150303 10:08:59 [Note] InnoDB: Reading tablespace information from the >>> .ibd files... >>> 150303 10:09:05 [Note] InnoDB: Restoring possible half-written data >>> pages >>> 150303 10:09:05 [Note] InnoDB: from the doublewrite buffer... >>> InnoDB: Doing recovery: scanned up to log sequence number 14178613459456 >>> InnoDB: Doing recovery: scanned up to log sequence number 14178618702336 >>> InnoDB: Doing recovery: scanned up to log sequence number 14178623945216 >>> InnoDB: Doing recovery: scanned up to log sequence number 14178629188096 >>> InnoDB: Doing recovery: scanned up to log sequence number 14178634430976 >>> InnoDB: Doing recovery: scanned up to log sequence number 14178639673856 >>> InnoDB: Doing recovery: scanned up to log sequence number 14178644916736 >>> ...... >>> ...... >>> ...... >>> ...... >>> 150303 10:09:14 [Note] InnoDB: Starting an apply batch of log records >>> to the database... >>> InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 >>> 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 >>> 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 >>> 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 >>> 93 94 95 96 97 98 99 >>> InnoDB: Apply batch completed >>> InnoDB: In a MySQL replication slave the last master binlog file >>> InnoDB: position 0 16433109, file name slave-bin.852141 >>> InnoDB: Last MySQL binlog file position 0 16784214, file name >>> /var/lib/mysql/log/bin/slave-bin.005211 >>> 150303 10:09:34 [Note] InnoDB: 128 rollback segment(s) are active. >>> 150303 10:09:34 [Note] InnoDB: Waiting for purge to start >>> 150303 10:09:34 [Note] InnoDB: Percona XtraDB (http://www.percona.com) >>> 5.6.22-72.0 started; log sequence number 14178834821658 >>> Tue Mar 3 10:09:36 2015 TokuFT recovery starting in env >>> /var/lib/mysql/data/ >>> Tue Mar 3 10:09:36 2015 TokuFT recovery scanning backward from 1330297 >>> Tue Mar 3 10:09:36 2015 TokuFT recovery bw_end_checkpoint at 1330297 >>> timestamp 1425395277901416 xid 1330281 (bw_newer) >>> Tue Mar 3 10:09:36 2015 TokuFT recovery bw_begin_checkpoint at 1330281 >>> timestamp 1425395273487081 (bw_between) >>> Tue Mar 3 10:09:36 2015 TokuFT recovery turning around at begin >>> checkpoint 1330281 time 4414335 >>> Tue Mar 3 10:09:36 2015 TokuFT recovery starts scanning forward to >>> 1330297 from 1330281 left 16 (fw_between) >>> Tue Mar 3 10:09:36 2015 TokuFT recovery closing 14 dictionaries >>> Tue Mar 3 10:09:36 2015 TokuFT recovery making a checkpoint >>> Tue Mar 3 10:09:36 2015 TokuFT recovery done >>> 150303 10:09:36 [Note] Recovering after a crash using >>> /var/lib/mysql/log/bin/slave-bin >>> >>> --------------- >>> >>> The odd thing is that the 5.5.x version was working just fine (since >>> about a year) in the same hardware...nothing changed in that front. >>> >>> Any clue? >>> >>> Thank you in advance >>> >>> >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~maria-discuss >>> Post to : [email protected] >>> Unsubscribe : https://launchpad.net/~maria-discuss >>> More help : https://help.launchpad.net/ListHelp >>> >>> >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~maria-discuss >>> Post to : [email protected] >>> Unsubscribe : https://launchpad.net/~maria-discuss >>> More help : https://help.launchpad.net/ListHelp >>> >>> >> > > > -- > Gabriel Sosa > Sometimes the questions are complicated and the answers are simple. -- Dr. > Seuss > > >
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
