On Thu, Sep 27, 2018 at 7:37 AM Chad W Seys <[email protected]> wrote:
> Hi all, > I am exporting cephfs using samba. It is much slower over samba than > direct. Anyone know how to speed it up? > Benchmarked using bonnie++ 5 times either directly to cephfs mounted > by kernel (v4.18.6) module: > bonnie++ -> kcephfs > or through a cifs kernel-module-mounted (protocol version 3.02) Samba > (v4.8.5) share on the same machine. > bonnie++ -> Samba -> kcephfs > > Abbreviated results for 5 runs: > kcephfs: min max > file created 555 619 files/sec > sequential block input: 106.44 108.13 MB/sec > sequential block output: 102.82 110.61 MB/sec > > (There is a gigabit network between the client and the ceph cluster, so > the block in/out is pleasing.) > > samba -> kcephfs: min max > file created 4 5 files/sec > sequential block input: 22.85 29.5 MB/sec > sequential block output: 27.95 30.01 MB/sec > > The block input/output is okay fast, but the files created per second is > low. Anyone know how to tweak Samba to speed it up? > That seems to be a samba tuning issue, which this isn't the right list for. :/ > Would Samba vfs_ceph speed up access? At the moment vfs_ceph in > Debian depends on libceph1 10.2.5, so not too modern. > Well, good news and bad news here. Good news is that since vfs_ceph uses the native cephfs client library, it ought to be faster (though I don't have any data on how much going through samba itself costs against our client). Bad news is that samba's connection model doesn't play very nicely with Ceph's — if you use vfs_ceph, every samba connection will turn into a brand new Ceph connection — smb literally runs a fork on every incoming connection — so you will see RAM/network connections/etc scale on a per-smb-client basis. This works fine for a small number of clients but not so great if you're planning to attach a bunch of them. :( -Greg > > Current Samba settings: > [global] > dns proxy = No > hostname lookups = Yes > kerberos method = secrets and keytab > logging = syslog@1 /var/log/samba/log.%m > max log size = 100000 > panic action = /usr/share/samba/panic-action %d > realm = PHYSICS.WISC.EDU > security = USER > server signing = required > server string = %h server > workgroup = PHYSICS > fruit:nfs_aces = no > idmap config * : backend = tdb > [smb] > ea support = Yes > inherit acls = Yes > inherit permissions = Yes > msdfs root = Yes > path = /srv/smb > read only = No > smb encrypt = desired > vfs objects = catia fruit streams_xattr > fruit:encoding = native > > Thanks! > Chad. > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
