On Wed, Mar 7, 2018 at 8:37 PM, Alex Gorbachev <a...@iss-integration.com> wrote:
> On Wed, Mar 7, 2018 at 9:43 AM, Cassiano Pilipavicius
> <cassi...@tips.com.br> wrote:
>> Hi all, this issue already have been discussed in older threads and I've
>> already tried most of the solutions proposed in older threads.
>> I have a small and  old ceph cluster (slarted in hammer and upgraded until
>> luminous 12.2.2) , connected thru single 1gbe link shared (I know this is
>> not optimal but for my workload it is handling the load reasonably well). I
>> use for RBD for small VMs in libvirtu/qemu.
>> My problem is... If i need to copy a large file (cp, dd, tar), the read
>> speed is very low (15MB/s). I've tested the write speed of a single job with
>> dd zero (direct) > file and the speed is good enought for my environment
>> (80MB/s)
>> If I run paralell jobs, I can saturate the network connection, the speed
>> scales with the number of jobs. I've tried setting read ahead on ceph.conf
>> and in the guest O.S
>> I've never heard any report of a cluster using single 1gbe, maybe this speed
>> is what should I expect? The next week I will be upgrading the network for 2
>> x 10gbe (private and public) but I would like to know if I have any issue
>> that I need to address before, as the problem can be masked by the network
>> upgrade.
>> If anyone can throw some light or point me in any direction or tell me....
>> this is what you should expect.... I really apreciate. If anyone need more
>> info please let me know.
> Workarounds I have heard of or used:
> 1. Use fancy striping and parallelize that way
> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2017-April/017744.html
> 2. Use lvm and set up a striped volume over multiple RBDs
> 3. Weird but we had seen improvement in sequential speeds with larger
> object size (16 MB) in the past
> 4. Caching solutions may help smooth out peaks and valleys of IO -
> bcache, flashcache and we have successfully used EnhanceIO with
> writethrough mode
> 5. Better SSD journals help if using filestore
> 6. Caching controllers, e.g. Areca
> --
> Alex Gorbachev
> Storcium
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
ceph-users mailing list

Reply via email to