When we used glusterfs for websites, we copied the web dir from gluster to local on frontend boots, then served it from there.
Jaco Kroon <j...@uls.co.za> 于 2022年12月14日周三 12:49写道: > Hi All, > > We've got a glusterfs cluster that houses some php web sites. > > This is generally considered a bad idea and we can see why. > > With performance.nl-cache on it actually turns out to be very > reasonable, however, with this turned of performance is roughly 5x > worse. meaning a request that would take sub 500ms now takes 2500ms. > In other cases we see far, far worse cases, eg, with nl-cache takes > ~1500ms, without takes ~30s (20x worse). > > So why not use nl-cache? Well, it results in readdir reporting files > which then fails to open with ENOENT. The cache also never clears even > though the configuration says nl-cache entries should only be cached for > 60s. Even for "ls -lah" in affected folders you'll notice ???? mark > entries for attributes on files. If this recovers in a reasonable time > (say, a few seconds, sure). > > # gluster volume info > Type: Replicate > Volume ID: cbe08331-8b83-41ac-b56d-88ef30c0f5c7 > Status: Started > Snapshot Count: 0 > Number of Bricks: 1 x 2 = 2 > Transport-type: tcp > Options Reconfigured: > performance.nl-cache: on > cluster.readdir-optimize: on > config.client-threads: 2 > config.brick-threads: 4 > config.global-threading: on > performance.iot-pass-through: on > storage.fips-mode-rchecksum: on > cluster.granular-entry-heal: enable > cluster.data-self-heal-algorithm: full > cluster.locking-scheme: granular > client.event-threads: 2 > server.event-threads: 2 > transport.address-family: inet > nfs.disable: on > cluster.metadata-self-heal: off > cluster.entry-self-heal: off > cluster.data-self-heal: off > cluster.self-heal-daemon: on > server.allow-insecure: on > features.ctime: off > performance.io-cache: on > performance.cache-invalidation: on > features.cache-invalidation: on > performance.qr-cache-timeout: 600 > features.cache-invalidation-timeout: 600 > performance.io-cache-size: 128MB > performance.cache-size: 128MB > > Are there any other recommendations short of abandon all hope of > redundancy and to revert to a single-server setup (for the web code at > least). Currently the cost of the redundancy seems to outweigh the > benefit. > > Glusterfs version 10.2. With patch for --inode-table-size, mounts > happen with: > > /usr/sbin/glusterfs --acl --reader-thread-count=2 --lru-limit=524288 > --inode-table-size=524288 --invalidate-limit=16 --background-qlen=32 > --fuse-mountopts=nodev,nosuid,noexec,noatime --process-name fuse > --volfile-server=127.0.0.1 --volfile-id=gv_home > --fuse-mountopts=nodev,nosuid,noexec,noatime /home > > Kind Regards, > Jaco > > ________ > > > > Community Meeting Calendar: > > Schedule - > Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC > Bridge: https://meet.google.com/cpu-eiue-hvk > Gluster-users mailing list > Gluster-users@gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users >
________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users