Re: [ceph-users] rsync kernel client cepfs mkstemp no space left on device

2016-12-12 Thread Mike Miller
John, thanks for emphasizing this, before this workaround we tried many different kernel versions including 4.5.x, all the same. The problem might be particular to our environment as most of the client machines (compute servers) have large RAM, so plenty of cache space for inodes/dentries.

Re: [ceph-users] rsync kernel client cepfs mkstemp no space left on device

2016-12-11 Thread John Spray
On Sun, Dec 11, 2016 at 4:38 PM, Mike Miller wrote: > Hi, > > you have given up too early. rsync is not a nice workload for cephfs, in > particular, most linux kernel clients cephfs will end up caching all > inodes/dentries. The result is that mds servers crash due to memory > limitations. And rsy

Re: [ceph-users] rsync kernel client cepfs mkstemp no space left on device

2016-12-11 Thread Mike Miller
Hi, you have given up too early. rsync is not a nice workload for cephfs, in particular, most linux kernel clients cephfs will end up caching all inodes/dentries. The result is that mds servers crash due to memory limitations. And rsync basically screens all inodes/dentries so it is the perfe

Re: [ceph-users] rsync kernel client cepfs mkstemp no space left on device

2016-12-06 Thread Hauke Homburg
Am 10.10.2016 um 10:05 schrieb Hauke Homburg: > Am 07.10.2016 um 17:37 schrieb Gregory Farnum: >> On Fri, Oct 7, 2016 at 7:15 AM, Hauke Homburg >> wrote: >>> Hello, >>> >>> I have a Ceph Cluster with 5 Server, and 40 OSD. Aktual on this Cluster >>> are 85GB Free Space, and the rsync dir has lots

Re: [ceph-users] rsync kernel client cepfs mkstemp no space left on device

2016-10-12 Thread Hauke Homburg
Am 10.10.2016 um 11:33 schrieb John Spray: > On Mon, Oct 10, 2016 at 9:05 AM, Hauke Homburg > wrote: >> Am 07.10.2016 um 17:37 schrieb Gregory Farnum: >>> On Fri, Oct 7, 2016 at 7:15 AM, Hauke Homburg >>> wrote: Hello, I have a Ceph Cluster with 5 Server, and 40 OSD. Aktual on th

Re: [ceph-users] rsync kernel client cepfs mkstemp no space left on device

2016-10-10 Thread John Spray
On Mon, Oct 10, 2016 at 9:05 AM, Hauke Homburg wrote: > Am 07.10.2016 um 17:37 schrieb Gregory Farnum: >> On Fri, Oct 7, 2016 at 7:15 AM, Hauke Homburg >> wrote: >>> Hello, >>> >>> I have a Ceph Cluster with 5 Server, and 40 OSD. Aktual on this Cluster >>> are 85GB Free Space, and the rsync dir

Re: [ceph-users] rsync kernel client cepfs mkstemp no space left on device

2016-10-10 Thread Hauke Homburg
Am 07.10.2016 um 17:37 schrieb Gregory Farnum: > On Fri, Oct 7, 2016 at 7:15 AM, Hauke Homburg wrote: >> Hello, >> >> I have a Ceph Cluster with 5 Server, and 40 OSD. Aktual on this Cluster >> are 85GB Free Space, and the rsync dir has lots of Pictures and a Data >> Volume of 40GB. >> >> The Linux

Re: [ceph-users] rsync kernel client cepfs mkstemp no space left on device

2016-10-07 Thread Gregory Farnum
On Fri, Oct 7, 2016 at 7:15 AM, Hauke Homburg wrote: > Hello, > > I have a Ceph Cluster with 5 Server, and 40 OSD. Aktual on this Cluster > are 85GB Free Space, and the rsync dir has lots of Pictures and a Data > Volume of 40GB. > > The Linux is a Centos 7 and the Last stable Ceph. The Client is a