On Thu, 26 Dec 2013 07:36:24 -0800 (PST) Anne Salemme <[email protected]> wrote:
> First thing to check: Is the AFS cache in its own, dedicated > filesystem partition. Well, he's using memcache. But anyway: > From: GALAMBOS Daniel <[email protected]> > >> So we got problems, i cloned the machine and started testing. I used >> apache benchmark (ab) against a simple html (w/o pictures or >> anything, just text). If i downloaded it 2000 times with 50 thread i >> got high load (18) and ~20 000 context switching in vmstat. The same >> file with 100 thread generated ~50 load. To some extent, the performance of accessing stuff in the cache is just not going to be that great. Right now, I think the linux cache manager now isn't really architected to serve data from multiple threads simultaneously very efficiently, and there's not much you can do about it. If you reduce the number of threads, I would expect that the response time you get to vastly improve; does it? But regardless, some things to try: Remove .htaccess processing in Apache. This can slow down accesses a lot, especially in AFS. If I recall correctly, you do this with the "AllowOverride None" directive in Apache's config. Of course, if you need to use .htaccess files, that doesn't help. You could try using a disk cache against the local disk instead of memcache. Memcache may sound "faster", but it's not, really, and disk cache is a more tested code path. I don't expect that to do much, but if you're somehow hitting some weird edge case, it's less likely for there to be obscure issues with disk cache, I think. You can also try sniffing the network traffic for port 7001 UDP. The mentioned problems I believe do indicate just bottlenecks in cache processing, but if you're somehow hitting the net for anything, that would also be a problem. If you see anything hitting the net from port 7001 UDP on the client besides rx "version" packets, then something is not getting cached locally. Oh, and if you can 'strace' a process reading something from /afs while the Apache server is loaded, it could say what specific syscalls are being slow. strace'ing Apache itself is probably too much data to look through, but if you run, say, "cat /afs/path/to/file.html > /dev/null" and strace'd that, it would be easy to see where things are slowing down. -- Andrew Deason [email protected] _______________________________________________ OpenAFS-info mailing list [email protected] https://lists.openafs.org/mailman/listinfo/openafs-info
