Re: [ceph-users] Giant upgrade - stability issues

2014-11-28 Thread Andrei Mikhailovsky
:30 PM Subject: Re: [ceph-users] Giant upgrade - stability issues Sam, I've done more network testing, this time over 2 days and I believe I have enough evidence to conclude that the osd disconnects are not caused by the network. I have ran about 140 million TCP connects on each osd and host

Re: [ceph-users] Giant upgrade - stability issues

2014-11-27 Thread Andrei Mikhailovsky
not. Not sure what to think now. Andrei - Original Message - From: Andrei Mikhailovsky and...@arhont.com To: sj...@redhat.com Cc: ceph-users@lists.ceph.com Sent: Thursday, 20 November, 2014 4:50:21 PM Subject: Re: [ceph-users] Giant upgrade - stability issues Thanks, I will try

Re: [ceph-users] Giant upgrade - stability issues

2014-11-20 Thread Andrei Mikhailovsky
and...@arhont.com Cc: ceph-users@lists.ceph.com Sent: Wednesday, 19 November, 2014 9:45:40 PM Subject: Re: [ceph-users] Giant upgrade - stability issues Well, the heartbeats are failing due to networking errors preventing the heartbeats from arriving. That is causing osds to go down

Re: [ceph-users] Giant upgrade - stability issues

2014-11-20 Thread Andrei Mikhailovsky
Thanks, I will try that. Andrei - Original Message - From: Samuel Just sam.j...@inktank.com To: Andrei Mikhailovsky and...@arhont.com Cc: ceph-users@lists.ceph.com Sent: Thursday, 20 November, 2014 4:26:00 PM Subject: Re: [ceph-users] Giant upgrade - stability issues You can try

Re: [ceph-users] Giant upgrade - stability issues

2014-11-19 Thread Samuel Just
Mikhailovsky and...@arhont.com Cc: ceph-users@lists.ceph.com Sent: Tuesday, 18 November, 2014 7:54:56 PM Subject: Re: [ceph-users] Giant upgrade - stability issues Ok, why is ceph marking osds down? Post your ceph.log from one of the problematic periods. -Sam On Tue, Nov 18, 2014 at 1:35 AM

Re: [ceph-users] Giant upgrade - stability issues

2014-11-19 Thread Andrei Mikhailovsky
: Tuesday, 18 November, 2014 8:53:47 PM Subject: Re: [ceph-users] Giant upgrade - stability issues pastebin or something, probably. -Sam On Tue, Nov 18, 2014 at 12:34 PM, Andrei Mikhailovsky and...@arhont.com wrote: Sam, the logs are rather large in size. Where should I post

Re: [ceph-users] Giant upgrade - stability issues

2014-11-19 Thread Samuel Just
: Samuel Just sam.j...@inktank.com To: Andrei Mikhailovsky and...@arhont.com Cc: ceph-users@lists.ceph.com Sent: Tuesday, 18 November, 2014 8:53:47 PM Subject: Re: [ceph-users] Giant upgrade - stability issues pastebin or something, probably. -Sam On Tue, Nov 18, 2014 at 12:34 PM, Andrei

Re: [ceph-users] Giant upgrade - stability issues

2014-11-19 Thread Andrei Mikhailovsky
...@inktank.com To: Andrei Mikhailovsky and...@arhont.com Cc: ceph-users@lists.ceph.com Sent: Tuesday, 18 November, 2014 8:53:47 PM Subject: Re: [ceph-users] Giant upgrade - stability issues pastebin or something, probably. -Sam On Tue, Nov 18, 2014 at 12:34 PM, Andrei Mikhailovsky

[ceph-users] Giant upgrade - stability issues

2014-11-18 Thread Andrei Mikhailovsky
Hello cephers, I need your help and suggestion on what is going on with my cluster. A few weeks ago i've upgraded from Firefly to Giant. I've previously written about having issues with Giant where in two weeks period the cluster's IO froze three times after ceph down-ed two osds. I have in

Re: [ceph-users] Giant upgrade - stability issues

2014-11-18 Thread Samuel Just
Ok, why is ceph marking osds down? Post your ceph.log from one of the problematic periods. -Sam On Tue, Nov 18, 2014 at 1:35 AM, Andrei Mikhailovsky and...@arhont.com wrote: Hello cephers, I need your help and suggestion on what is going on with my cluster. A few weeks ago i've upgraded from

Re: [ceph-users] Giant upgrade - stability issues

2014-11-18 Thread Andrei Mikhailovsky
Sam, the logs are rather large in size. Where should I post it to? Thanks - Original Message - From: Samuel Just sam.j...@inktank.com To: Andrei Mikhailovsky and...@arhont.com Cc: ceph-users@lists.ceph.com Sent: Tuesday, 18 November, 2014 7:54:56 PM Subject: Re: [ceph-users] Giant

Re: [ceph-users] Giant upgrade - stability issues

2014-11-18 Thread Samuel Just
and...@arhont.com Cc: ceph-users@lists.ceph.com Sent: Tuesday, 18 November, 2014 7:54:56 PM Subject: Re: [ceph-users] Giant upgrade - stability issues Ok, why is ceph marking osds down? Post your ceph.log from one of the problematic periods. -Sam On Tue, Nov 18, 2014 at 1:35 AM, Andrei

Re: [ceph-users] Giant upgrade - stability issues

2014-11-18 Thread Andrei Mikhailovsky
Message - From: Samuel Just sam.j...@inktank.com To: Andrei Mikhailovsky and...@arhont.com Cc: ceph-users@lists.ceph.com Sent: Tuesday, 18 November, 2014 8:53:47 PM Subject: Re: [ceph-users] Giant upgrade - stability issues pastebin or something, probably. -Sam On Tue, Nov 18, 2014 at 12:34