It is not only ls; even du or tar are extremely slow. Example with tar (from node B): # time tar c /storage/test10/installed-tests/pts/pgbench-1.5.1/>/dev/null real 1m45.291s user 0m0.023s sys 0m0.143s
While on the node that originally wrote the dir (node A): # time tar c /storage/test10/installed-tests/pts/pgbench-1.5.1/>/dev/null real 0m0.095s user 0m0.013s sys 0m0.068s Strangely enough, tar is slighly faster on another node (let's call it node C) that is mouting the FS with the FUSE driver (this node is a OSD node too and I have read the kernel driver cannot be used on OSD nodes else conflicts may arise): # time tar c /storage/test10/installed-tests/pts/pgbench-1.5.1/>/dev/null real 0m26.934s user 0m0.067s sys 0m1.336s On Tue, Jun 16, 2015 at 1:42 PM, John Spray <[email protected]> wrote: > > > On 16/06/2015 12:11, negillen negillen wrote: > >> This is quite a problem because we have several applications that need to >> access a large number of files and when we set them to work on CephFS >> latency skyrockets. >> > > The question of what "access" means here is key. Especially, whether you > need the "-l" part of "ls -l". A plain "ls" is one readdir operation, > adding "-l" turns it into N stat operations. > > This is something that distributed filesystems can be optimised to cope > with (e.g. by batching the stats and associated locking, or Lustre's > statahead), but frequently applications can simply avoid "-l", if all they > really needed to do was list the files. > > John >
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
