Hello list,

I have about a half a dozen nginx servers sitting in front of Gluster (3.4.6, I know it's old) serving a mix videos and images. It's a moderate amount of traffic, each of two geo-repped sites will do 2-4 gigs/second throughout the day.

Here's the problem. Reads from gluster, due to the way nginx buffers the video, can far exceed what's being served out to the internet. Serving 1 gig of video may read 3 gigs from Gluster.

I can fix this by setting the performance cache on the volume to a pretty large size, right now it's at 2 gigs. This works great, gluster uses 1.5 - 2 gigs of RAM and the in/out bandwidth on the nginx machines becomes a healthy 1:1 or better.

For a few days. Over time, as the machines vfs cache fills, gluster starts to use less RAM, and that ratio gets worse. Rebooting the nginx boxes (or I presume, simply dropping their caches) fixes it immediately.

I'm going to try increasing vfs.cache_pressure on the nginx boxes, as this doc recommends:


Does that make sense to tune this on the clients? Is Gluster competing with the kernel cache? That's sort of my understanding but I can't find a clear explanation.

Other recommendations would be welcome, though tweaking the direct-io options is unfortunately not an option in my setup.

Gluster-users mailing list

Reply via email to