> I know top and ps, but what are your other favorite commands to diagnose > sluggish system performance?
gdb and strace. Here is my favorite trick with MySQL (from http://poormansprofiler.org): gdb -ex "set pagination 0" -ex "thread apply all bt" \ --batch -p $(pidof mysqld) and it will tell you exactly where it is stuck. This can solve a lot of MySQL performance mysteries in a matter of seconds, and the same idea can be applied to troubleshooting the performance of just about anything that is either open-source or at least has decent function-naming conventions and is not stripped so that when you see a stack trace you will have a clue as to where it is at. strace is the first thing I use once I know the pid of the top or iotop dubious "winner". It is also good to be a friend of lsof and netstat to figure out who is on what file descriptor or port. gdb can be used to poke around the memory of a trouble process. You can discover some really cool things - e.g. who is holding the mutex that is being waited upon by everybody. You can even fix up the memory state of a running process if you have a good idea of what the result should be. I have been able to make mysqld shut down cleanly in spite of problems by modifying an internal variable (not a config one!) with gdb. -- Sasha Pachev Fast Running Blog. http://fastrunningblog.com Run. Blog. Improve. Repeat. /* PLUG: http://plug.org, #utah on irc.freenode.net Unsubscribe: http://plug.org/mailman/options/plug Don't fear the penguin. */
