Hello,
We're been running a pretty large MySQLd with InnoDB support, last night it
crashed on us in the middle of the night. I have never sent in a bug report
like this before so please give me a little slack. I do not have any clue as
to what actually caused the crash, I only have the logs and confs.
In order: 1. System specs 2. my.cnf directives and 3. MySQL error log
============= 1. System Specs ============
Dell PowerEdge 2450 Dual PIII 850
2G RAM
5-disk RAID5 for a total of 67G in / (27 used, 39 free)
I think swap is also 2GB but I'm not sure (1060258+ blocks).
SuSE Linux 7.3
ccs012:~ # uname -a
Linux ccs012 2.4.10-64GB-SMP #1 SMP Fri Sep 28 17:26:36 GMT 2001 i686 unknown
ccs012:~ # free -m
total used free shared buffers cached
Mem: 2013 2007 5 0 23 695
-/+ buffers/cache: 1288 724
Swap: 1035 0 1035
Not running any major service except MySQL, standalone sshd and inetd (for
telnet)
============= 2. my.cnf ============
ccs012:~ # grep -v "#" /etc/my.cnf
[client]
port = 3306
socket = /var/lib/mysql/mysql.sock
[mysqld]
port = 3306
socket = /var/lib/mysql/mysql.sock
skip-locking
set-variable = key_buffer=384M
set-variable = max_allowed_packet=1M
set-variable = table_cache=512
set-variable = sort_buffer=2M
set-variable = record_buffer=2M
set-variable = thread_cache=8
set-variable = max_connections=150
set-variable = thread_concurrency=4
set-variable = myisam_sort_buffer_size=64M
log-bin
server-id = 1
innodb_data_file_path = ibdata1:2G
innodb_data_home_dir = /var/lib/mysql/
innodb_log_group_home_dir = /var/lib/mysql/
innodb_log_arch_dir = /var/lib/mysql/
set-variable = innodb_log_files_in_group=3
set-variable = innodb_log_file_size=156M
set-variable = innodb_log_buffer_size=12M
innodb_flush_log_at_trx_commit=1
innodb_log_archive=0
set-variable = innodb_buffer_pool_size=1024M
set-variable = innodb_additional_mem_pool_size=8M
set-variable = innodb_file_io_threads=4
set-variable = innodb_lock_wait_timeout=50
[mysqldump]
quick
set-variable = max_allowed_packet=16M
[mysql]
no-auto-rehash
[isamchk]
set-variable = key_buffer=256M
set-variable = sort_buffer=256M
set-variable = read_buffer=2M
set-variable = write_buffer=2M
[myisamchk]
set-variable = key_buffer=256M
set-variable = sort_buffer=256M
set-variable = read_buffer=2M
set-variable = write_buffer=2M
[mysqlhotcopy]
interactive-timeout
[safe_mysqld]
open-files-limit=256
============= 3. MySQLd-Max error log output ============
ccs012:~ # less /var/lib/mysql/ccs012.err
020216 19:46:15 mysqld started
020216 19:46:20 InnoDB: Started
/usr/sbin/mysqld-max: ready for connections
InnoDB: Error: undo->id is 137339008
InnoDB: Assertion failure in thread 27591729 in file trx0undo.c line 1316
InnoDB: We intentionally generate a memory trap.
InnoDB: Send a detailed bug report to [EMAIL PROTECTED]
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked agaist is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail
key_buffer_size=402649088
record_buffer=2093056
sort_buffer=2097144
max_used_connections=150
max_connections=150
threads_connected=68
It is possible that mysqld could use up to
key_buffer_size + (record_buffer + sort_buffer)*max_connections = 1007010 K
bytes of memory
Hope that's ok, if not, decrease some variables in the equation
InnoDB: Thread 27614285 stopped in file btr0pcur.c line 202
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Stack range sanity check OK, backtrace follows:
0x806c9b9
0x8249998
0x81acb51
0x81a27d4
0x81949f3
0x817d565
0x817d949
0x817dc42
0x8170a1d
0x80baff9
0x809b855
0x80749a5
0x8076548
0x80725d4
0x8071ac7
Stack trace seems successful - bottom reached
Please read http://www.mysql.com/doc/U/s/Using_stack_trace.html and follow
instructions on how to resolve the
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x86f4e00 = delete from cpsearchenginedata.customerkeywordbid
where customerkeywordid='5695'
thd->thread_id=1329667
Successfully dumped variables, if you ran with --log, take a look at the
details of what thread 1329667 did to cause the crash. In some cases of
really
bad corruption, the values shown above may be invalid
The manual page at http://www.mysql.com/doc/C/r/Crashing.html contains
information that should help you find out what is causing the crash
Number of processes running now: 0
020226 01:13:09 mysqld restarted
020226 1:13:09 Can't start server: Bind on TCP/IP port: Address already in
use
020226 1:13:09 Do you already have another mysqld server running on port:
3306 ?
020226 1:13:09 Aborting
020226 1:13:09 /usr/sbin/mysqld-max: Shutdown Complete
020226 01:13:09 mysqld ended
020226 07:09:22 mysqld started
InnoDB: Database was not shut down normally.
InnoDB: Starting recovery from log files...
InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 3 653317891
InnoDB: Doing recovery: scanned up to log sequence number 3 653383168
InnoDB: Doing recovery: scanned up to log sequence number 3 653448704
InnoDB: Doing recovery: scanned up to log sequence number 3 653514240
InnoDB: Doing recovery: scanned up to log sequence number 3 653579776
InnoDB: Doing recovery: scanned up to log sequence number 3 653645312
InnoDB: Doing recovery: scanned up to log sequence number 3 653710848
InnoDB: Doing recovery: scanned up to log sequence number 3 653776384
InnoDB: Doing recovery: scanned up to log sequence number 3 653841920
InnoDB: Doing recovery: scanned up to log sequence number 3 653907456
InnoDB: Doing recovery: scanned up to log sequence number 3 653972992
InnoDB: After this prints a line for every 10th scan sweep:
InnoDB: Doing recovery: scanned up to log sequence number 3 654628352
<snip a lot of lines - let me know if you need all of them... last one is: >
InnoDB: Doing recovery: scanned up to log sequence number 3 760141312
InnoDB: Starting an apply batch of log records to the database...
InnoDB: Apply batch completed
020226 7:19:10 InnoDB: Started
/usr/sbin/mysqld-max: ready for connections
This is a mission-critical DB. If anyone can help out here I'd really
appreciate it. If there's any more details needed just let me know what you
need.
Thanks a million...
--
-------------------------------------------------
Jonathan Wilson
System Administrator
Cedar Creek Software http://www.cedarcreeksoftware.com
Central Texas IT http://www.centraltexasit.com
---------------------------------------------------------------------
Before posting, please check:
http://www.mysql.com/manual.php (the manual)
http://lists.mysql.com/ (the list archive)
To request this thread, e-mail <[EMAIL PROTECTED]>
To unsubscribe, e-mail <[EMAIL PROTECTED]>
Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
