Hi all, I know that we're leaving the setup the revolves around fyodor soon, but I wanted to point out some performance issues that we're currently having as they might be able to help us tune the new configuration. Specifically, I think that I pointed out some issues before that seemed to be caused by slow disk I/O. Today I felt some delays while using fyodor that were confirmed by the vmstat 5 command below:
At about 11:07 EST I got: procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 1 0 866336 41224 150228 234948 0 0 2 518 174 182 11 2 87 0 0 0 866336 37544 150364 235084 679 0 704 513 245 313 28 0 71 0 2 0 866336 31636 150528 235132 0 0 20 257 165 204 29 3 68 0 1 11 866336 36428 150552 235176 0 0 9 572 305 220 8 1 91 0 0 0 866336 37072 150652 235196 0 0 2 515 188 245 14 2 85 0 0 0 866336 37008 150704 235208 0 0 0 213 130 151 21 1 78 0 At one point note that there were 11 processes in the "blocked" state, which seems to me pretty high for the machine that we have running. At this time, I didn't see spamassassin or a backup process running. My intuition is that the slowdowns are caused by the slow disk subsystem using software RAID and normal usage patterns. Since we also have software RAID setup currently on mire, I want to make sure that we're not killing our performance before we even move to the new setup. Perhaps if we do have to go with software RAID on the front-end system, we can mitigate the performance penalty by not using it for AFS caching or through some other simple tuning. At any rate, I would just like to suggest that we do some performance tests on the new setup before we migrate most of the current users to the new infrastructure. I added a bullet on performance testing to the to-do list because I don't want to overlook this issue, and I think that some fairly good performance tests can be done to eliminate the most detrimental bottlenecks before we call the new system production-ready. >From testing that I've done in the past, I've found that while it's really hard to mimic real-world traffic on a test system, there are tools that are pre-made, and others that can be written to do a good job of finding out what the breaking point of a system will be. Finally, although I have resigned my formal sysadmin duties, I don't want to just toss this task out there for someone else to handle. If you want to let me know when the systems are ready for this stage before they're tossed into production, I'll try to spend some doing whatever is necessary to help the current admins tune the new configuration. Best, Justin _______________________________________________ HCoop-SysAdmin mailing list [email protected] http://hcoop.net/cgi-bin/mailman/listinfo/hcoop-sysadmin
