----- Original Message ----- > From: "Ron Trompert" <[email protected]> > To: [email protected] > Sent: Wednesday, April 29, 2015 1:25:59 PM > Subject: [Gluster-users] Poor performance with small files > > Hi, > > We run gluster as storage solution for our Owncloud-based sync and share > service. At the moment we have about 30 million files in the system > which addup to a little more than 30TB. Most of these files are as you > may expect very small, i.e. in the 100KB ball park. For about a year > everything ran perfectly fine. We run 3.6.2 by the way.
Upgrade to 3.6.3 and set client.event-threads and server.event-threads to at least 4: "Previously, epoll thread did socket even-handling and the same thread was used for serving the client or processing the response received from the server. Due to this, other requests were in a queue untill the current epoll thread completed its operation. With multi-threaded epoll, events are distributed that improves the performance due the parallel processing of requests/responses received." Here are the guidelines for tuning them: https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3/html/Administration_Guide/Small_File_Performance_Enhancements.html In my testing with epoll threads at 4 I saw a between a 15% and 50% increase depending on the workload. There are several smallfile perf enhancements in the works: *http://www.gluster.org/community/documentation/index.php/Features/Feature_Smallfile_Perf *Lookup unhashed is the next feature and should be ready with 3.7(correct me if I am wrong). *If you are using RAID 6 you may want to do some testing with RAID 10 or JBOD, but the benefits here only come into play with alot of concurrent access(30+ processes / threads working with different files). *Tiering may help here if you want to add some SSDs, this is also a 3.7 feature. HTH! -b > > Now we are trying to commission new hardware. We have done this by > adding the new nodes to our cluster and using the add-brick and > remove-brick procedure to get the data to the new nodes. In a week we > have migrated only 8.5TB this way. What are we doing wrong here? Is > there a way to improve the gluster performance on small files? > > I have another question. If you want to setup a gluster that will > contain lots of very small files. What would be a good practice to set > things up in terms configuration, sizes of bricks related tot memory and > number of cores, number of brick per node etc.? > > > > Best regards and thanks in advance, > > Ron > > > _______________________________________________ > Gluster-users mailing list > [email protected] > http://www.gluster.org/mailman/listinfo/gluster-users > _______________________________________________ Gluster-users mailing list [email protected] http://www.gluster.org/mailman/listinfo/gluster-users
