Re: NFS root + aufs tmpfs + Apache performance problem

2011-01-30 Thread sfjro
dan...@zoltak.com: When load spikes, which only happens when running AUFS the performance suffers i.e. the number of requests per second drops off on the AUFS node and the request per second graphs becomes jittery. The latency also goes up quite a bit on the node running AUFS. Then

Re: NFS root + aufs tmpfs + Apache performance problem

2011-01-29 Thread sfjro
dan...@zoltak.com: I down graded the kernel to 2.6.31-gentoo-r6, applied the AUFS patch and compiled the kernel and module. I then booted a node without AUFS loaded and ran it in an 8 node cluster where the other nodes were running on the pre upgraded image. The rootfs was a tmpfs in

Re: NFS root + aufs tmpfs + Apache performance problem

2011-01-27 Thread Michael S. Zick
On Wed January 26 2011, dan...@zoltak.com wrote: Quoting sf...@users.sourceforge.net: If you don't mind, would you clarify your story? Other aufs users may meet the similar situation and may think there must exist something bad in aufs similarly. For such users, what you have

Re: NFS root + aufs tmpfs + Apache performance problem

2011-01-26 Thread daniel
Thanks for the script and the info. I have a few answers and questions below... Quoting sf...@users.sourceforge.net: dan...@zoltak.com: It patches the sources with your aufs: dynop supports grsec/pax patch. Ok. So the grsec/pax patch is applied to your kernel, and it is applied for both

Re: NFS root + aufs tmpfs + Apache performance problem

2011-01-26 Thread sfjro
dan...@zoltak.com: Due to the AUFS being the RootFS and the raw NFS mounts are mounted on the RootFS could this in fact be causing AUFS to do unnecessary lookups? I don't think so. Because the root dir is always cached and its cache will never be discarded. Every access (to anywhere) may

Re: NFS root + aufs tmpfs + Apache performance problem

2011-01-26 Thread sfjro
dan...@zoltak.com: Is there anyway to validate this? Do you mean that you want to make sure that aufs code is unrelated to the activity out of aufs? How about adding printk in d_revalidate or code review? How much RAM does your system have? 16GB and 8 cores. Memory large enough. CPU

Re: NFS root + aufs tmpfs + Apache performance problem

2011-01-26 Thread daniel
Quoting sf...@users.sourceforge.net: dan...@zoltak.com: How much RAM does your system have? 16GB and 8 cores. Memory large enough. CPU many enough. Let's keep in mind about lock contension. Just to make sure, you can see them all via /proc. Right? Everything is showing up as

Re: NFS root + aufs tmpfs + Apache performance problem

2011-01-26 Thread sfjro
dan...@zoltak.com: Yes my mistake. The latency has gone up a bit but not by much. ::: I've performed more tests and have found that there is something going on with NFS unrelated to AUFS as I have now managed to verify that AUFS is not causing the performance issue. Thanks again

Re: NFS root + aufs tmpfs + Apache performance problem

2011-01-25 Thread sfjro
dan...@zoltak.com: I will try to be concise with my answers. ::: The following translates to: compiled without: debug support, hardening support, inotify support, nfs export support ::: The Apache 2.2 processes appear to be running hotter when AUFS is the root FS. Thak you