Hi, I've a situation whereby my server have intermittent high I/O wait (more than 25%) reported. This is the server info,
OS: RHEL 5.2 (x86_64) Kernel: 2.6.18-92.el5 Hardware: HP ProLiant BL460c G1 Application: MySQL 5.0.45-7.el5 According to sar history, high /I/O wait was recorded from 6.10AM - 8.20AM and 4.00PM - 6.10PM (Please see attachment for sar output details.) MySQLAdmin notified me that his database performance was quite slow during this period. My questions are, 1) Do I have any justification to relate the slowness of MySQL database and high I/O Wait? 2) How I could trace the root cause of high I/O Wait? 3) What are the impact of high I/O Wait to the system?
Linux 2.6.18-92.el5 (kptordb02) 07/20/2009 12:00:01 AM CPU %user %nice %system %iowait %steal %idle 12:10:01 AM all 2.57 0.00 0.70 4.63 0.00 92.10 12:20:01 AM all 1.86 0.00 0.58 2.57 0.00 95.00 12:30:01 AM all 2.31 0.00 0.61 3.41 0.00 93.67 12:40:01 AM all 1.52 0.00 0.44 2.64 0.00 95.40 12:50:01 AM all 1.35 0.00 0.33 0.21 0.00 98.11 01:00:01 AM all 1.65 0.02 0.35 0.24 0.00 97.74 01:10:01 AM all 2.47 0.00 0.62 2.87 0.00 94.04 01:20:01 AM all 1.71 0.00 0.54 2.45 0.00 95.30 01:30:01 AM all 1.92 0.00 0.59 4.68 0.00 92.82 01:40:01 AM all 1.64 0.00 0.35 0.25 0.00 97.77 01:50:02 AM all 1.29 0.00 0.31 0.19 0.00 98.20 02:00:01 AM all 3.74 0.02 5.23 2.82 0.00 88.20 02:10:01 AM all 2.84 0.00 1.81 4.85 0.00 90.49 02:20:01 AM all 1.79 0.00 0.53 2.14 0.00 95.53 02:30:01 AM all 1.87 0.00 0.59 5.35 0.00 92.19 02:40:01 AM all 1.39 0.00 0.31 0.23 0.00 98.07 02:50:01 AM all 1.70 0.00 0.31 0.20 0.00 97.79 03:00:01 AM all 1.53 0.02 0.32 0.21 0.00 97.93 03:10:01 AM all 2.46 0.00 0.66 3.22 0.00 93.66 03:20:01 AM all 1.72 0.00 0.54 2.21 0.00 95.53 03:30:01 AM all 1.94 0.00 0.61 4.39 0.00 93.06 03:40:01 AM all 1.37 0.00 0.33 0.76 0.00 97.54 03:50:01 AM all 1.31 0.00 0.30 0.14 0.00 98.25 04:00:01 AM all 1.90 0.01 0.37 0.49 0.00 97.23 04:10:01 AM all 13.77 0.00 4.03 14.47 0.00 67.73 04:20:01 AM all 2.26 0.00 0.51 5.43 0.00 91.80 04:30:01 AM all 1.78 0.00 0.59 4.00 0.00 93.63 04:40:01 AM all 1.74 0.00 0.58 6.14 0.00 91.54 04:50:01 AM all 1.40 0.00 0.32 0.17 0.00 98.11 05:00:01 AM all 1.54 0.01 0.32 0.20 0.00 97.93 05:10:01 AM all 2.37 0.00 0.61 2.26 0.00 94.76 05:20:01 AM all 1.71 0.00 0.54 2.34 0.00 95.41 05:30:01 AM all 1.91 0.00 0.62 5.52 0.00 91.96 05:40:01 AM all 1.34 0.00 0.33 0.74 0.00 97.59 05:50:01 AM all 1.31 0.00 0.31 0.21 0.00 98.17 06:00:01 AM all 1.52 0.01 0.32 0.23 0.00 97.92 06:10:01 AM all 5.68 0.00 0.98 28.16 0.00 65.18 06:20:01 AM all 5.45 0.00 0.85 34.38 0.00 59.32 06:30:01 AM all 4.77 0.00 0.81 30.54 0.00 63.87 06:40:01 AM all 4.44 0.00 0.73 30.31 0.00 64.51 06:50:01 AM all 3.82 0.00 0.71 31.74 0.00 63.73 07:00:01 AM all 3.81 0.02 0.90 38.11 0.00 57.16 07:10:01 AM all 5.35 0.00 0.97 31.50 0.00 62.18 07:20:01 AM all 5.65 0.00 0.87 32.96 0.00 60.52 07:30:01 AM all 6.00 0.00 0.92 34.52 0.00 58.55 07:40:01 AM all 5.25 0.00 0.87 29.02 0.00 64.86 07:50:01 AM all 6.21 0.00 0.84 30.93 0.00 62.03 08:00:01 AM all 3.98 0.02 0.90 37.27 0.00 57.83 08:10:01 AM all 4.90 0.00 1.06 34.60 0.00 59.43 08:20:01 AM all 4.97 0.00 0.90 34.78 0.00 59.35 08:30:01 AM all 1.86 0.00 0.58 5.44 0.00 92.12 08:40:01 AM all 2.99 0.00 0.88 4.58 0.00 91.55 08:50:01 AM all 1.73 0.00 0.47 4.30 0.00 93.50 09:00:01 AM all 2.30 0.02 0.51 1.34 0.00 95.83 09:10:01 AM all 2.70 0.00 0.73 2.91 0.00 93.67 09:20:01 AM all 2.79 0.00 0.78 2.69 0.00 93.75 09:30:01 AM all 2.32 0.00 0.63 2.22 0.00 94.83 09:40:01 AM all 3.56 0.00 0.92 3.39 0.00 92.13 09:50:01 AM all 2.32 0.00 0.74 2.98 0.00 93.96 10:00:01 AM all 5.01 0.02 1.64 4.80 0.00 88.53 10:10:01 AM all 6.81 0.00 2.32 5.41 0.00 85.45 10:20:01 AM all 6.64 0.00 2.55 6.84 0.00 83.97 10:30:01 AM all 4.61 0.00 1.73 3.94 0.00 89.73 10:40:01 AM all 5.63 0.00 2.20 5.56 0.00 86.60 10:50:01 AM all 5.30 0.00 1.95 4.48 0.00 88.27 11:00:01 AM all 6.62 0.02 2.71 5.14 0.00 85.52 11:10:01 AM all 5.58 0.00 2.35 3.78 0.00 88.29 11:20:01 AM all 5.72 0.00 2.49 3.92 0.00 87.88 11:30:01 AM all 5.73 0.00 2.29 3.05 0.00 88.94 11:40:01 AM all 5.09 0.00 2.11 3.11 0.00 89.68 11:50:01 AM all 5.91 0.00 2.54 3.74 0.00 87.81 12:00:01 PM all 6.33 0.01 2.47 3.83 0.00 87.36 12:10:01 PM all 6.73 0.00 2.81 5.08 0.00 85.38 12:20:01 PM all 6.09 0.00 2.40 3.97 0.00 87.54 12:30:01 PM all 6.50 0.00 2.98 4.39 0.00 86.13 12:40:01 PM all 6.17 0.00 2.78 4.60 0.00 86.45 12:50:01 PM all 5.17 0.00 2.11 2.99 0.00 89.73 01:00:01 PM all 4.65 0.01 1.75 2.72 0.00 90.87 01:10:02 PM all 5.21 0.00 2.19 3.22 0.00 89.38 01:20:01 PM all 5.77 0.00 2.17 5.17 0.00 86.89 01:30:01 PM all 4.47 0.00 1.86 3.38 0.00 90.29 01:40:01 PM all 4.72 0.00 2.02 4.26 0.00 89.00 01:50:01 PM all 4.47 0.00 1.75 3.78 0.00 90.00 02:00:01 PM all 6.54 0.02 2.54 5.01 0.00 85.90 02:10:01 PM all 5.18 0.00 2.14 3.14 0.00 89.54 02:20:01 PM all 4.52 0.00 1.87 2.52 0.00 91.10 02:30:01 PM all 5.49 0.00 1.98 2.63 0.00 89.90 02:40:01 PM all 4.88 0.00 2.36 2.83 0.00 89.93 02:50:01 PM all 5.21 0.00 2.27 3.60 0.00 88.92 03:00:01 PM all 6.83 0.01 2.81 6.43 0.00 83.91 03:10:01 PM all 4.69 0.00 2.18 3.87 0.00 89.25 03:20:01 PM all 4.84 0.00 2.13 4.51 0.00 88.53 03:30:01 PM all 3.96 0.00 1.89 6.55 0.00 87.60 03:40:01 PM all 4.91 0.00 2.19 6.76 0.00 86.14 03:50:01 PM all 4.17 0.00 1.93 7.71 0.00 86.20 04:00:01 PM all 3.83 0.01 1.55 36.28 0.00 58.33 04:10:01 PM all 3.98 0.00 1.26 35.89 0.00 58.87 04:20:01 PM all 4.26 0.00 1.65 37.14 0.00 56.94 04:30:01 PM all 4.78 0.00 1.84 37.46 0.00 55.93 04:40:01 PM all 2.35 0.00 1.06 34.80 0.00 61.79 04:50:01 PM all 2.74 0.00 1.02 35.68 0.00 60.57 05:00:01 PM all 2.78 0.02 1.27 38.35 0.00 57.59 05:10:01 PM all 3.57 0.00 1.23 40.06 0.00 55.14 05:20:01 PM all 3.96 0.00 1.55 40.51 0.00 53.97 05:30:01 PM all 7.02 0.00 3.12 36.78 0.00 53.08 05:40:01 PM all 4.78 0.00 2.35 38.42 0.00 54.46 05:50:01 PM all 6.44 0.00 3.03 37.68 0.00 52.84 06:00:01 PM all 4.15 0.02 1.91 38.14 0.00 55.79 06:10:01 PM all 6.06 0.00 2.68 38.89 0.00 52.37 06:20:01 PM all 11.23 0.00 6.35 9.88 0.00 72.55 06:30:01 PM all 6.10 0.00 2.85 8.73 0.00 82.32 06:40:01 PM all 6.02 0.00 2.36 4.68 0.00 86.93 06:50:01 PM all 5.95 0.00 2.46 4.74 0.00 86.85 07:00:02 PM all 5.80 0.02 2.52 3.25 0.00 88.42 07:10:01 PM all 5.03 0.00 2.11 2.83 0.00 90.03 07:20:01 PM all 4.27 0.00 1.67 2.60 0.00 91.46 07:30:01 PM all 2.75 0.00 0.90 0.93 0.00 95.42 07:40:01 PM all 3.43 0.00 1.26 1.32 0.00 93.99 07:50:01 PM all 6.10 0.00 2.19 2.66 0.00 89.06 08:00:01 PM all 5.91 0.01 2.40 2.94 0.00 88.73 08:10:01 PM all 3.88 0.00 1.40 1.71 0.00 93.01 08:20:01 PM all 2.75 0.00 0.81 1.09 0.00 95.35 08:30:01 PM all 2.78 0.00 0.83 1.05 0.00 95.34 08:40:01 PM all 2.15 0.00 0.69 0.91 0.00 96.24 08:50:01 PM all 2.56 0.00 0.69 0.84 0.00 95.91 09:00:01 PM all 2.83 0.02 0.78 0.79 0.00 95.59 09:10:01 PM all 2.78 0.00 0.83 1.01 0.00 95.38 09:20:01 PM all 2.98 0.00 0.69 0.95 0.00 95.37 09:30:01 PM all 2.98 0.00 0.90 1.01 0.00 95.11 09:40:01 PM all 2.39 0.00 0.76 0.77 0.00 96.08 09:50:01 PM all 2.49 0.00 0.86 0.95 0.00 95.70 10:00:02 PM all 2.47 0.02 0.65 1.22 0.00 95.64 10:10:01 PM all 2.48 0.00 0.58 2.22 0.00 94.72 10:20:01 PM all 2.49 0.00 0.98 2.53 0.00 94.00 10:30:01 PM all 2.24 0.00 0.87 1.47 0.00 95.42 10:40:01 PM all 3.21 0.00 1.16 4.33 0.00 91.30 10:50:01 PM all 2.24 0.00 0.80 1.13 0.00 95.84 11:00:01 PM all 3.11 0.01 1.11 3.69 0.00 92.08 11:10:01 PM all 2.55 0.00 0.94 3.05 0.00 93.47 11:20:01 PM all 3.53 0.00 1.37 6.85 0.00 88.24 Average: all 3.90 0.00 1.40 9.62 0.00 85.09
_______________________________________________ rhelv5-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/rhelv5-list
