Re: [ceph-users] problem with RGW

2015-07-31 Thread Brad Hubbard
- Original Message - From: Butkeev Stas staer...@ya.ru To: ceph-us...@ceph.com, ceph-commun...@lists.ceph.com, supp...@ceph.com Sent: Friday, 31 July, 2015 9:10:40 PM Subject: [ceph-users] problem with RGW Hello everybody We have ceph cluster that consist of 8 host with 12 osd per each

Re: [ceph-users] Check networking first?

2015-07-31 Thread Ben Hines
I encountered a similar problem. Incoming firewall ports were blocked on one host. So the other OSDs kept marking that OSD as down. But, it could talk out, so it kept saying 'hey, i'm up, mark me up' so then the other OSDs started trying to send it data again, causing backed up requests.. Which

Re: [ceph-users] dropping old distros: el6, precise 12.04, debian wheezy?

2015-07-31 Thread Sage Weil
On Fri, 31 Jul 2015, Alexandre DERUMIER wrote: As I still haven't heard or seen about any upstream distros for Debian Jessie (see also [1]), Gitbuilder is already done for jessie http://gitbuilder.ceph.com/ceph-deb-jessie-x86_64-basic/ @Sage : Don't known if something is blocking to

Re: [ceph-users] update docs? just mounted a format2 rbd image with client 0.80.8 server 0.87.2

2015-07-31 Thread Shinobu Kinjo
Thanks for your quick action!! - Shinobu On Fri, Jul 31, 2015 at 11:01 PM, Ilya Dryomov idryo...@gmail.com wrote: On Fri, Jul 31, 2015 at 2:21 PM, pixelfairy pixelfa...@gmail.com wrote: according to http://ceph.com/docs/master/rbd/rbd-snapshot/#layering, you have two choices, format

Re: [ceph-users] OSD startup causing slow requests - one tip from me

2015-07-31 Thread Georgios Dimitrakakis
Jan, this is very handy to know! Thanks for sharing with us! People, do you believe that it would be nice to have a place where we can gather either good practices or problem resolutions or tips from the community? We could have a voting system and those with the most votes (or above a

Re: [ceph-users] dropping old distros: el6, precise 12.04, debian wheezy?

2015-07-31 Thread Brian Kroth
That's good to hear. Thanks for the heads up. We're going to be getting another pile of hardware in the next couple of weeks and I'd prefer to not have to start with Wheezy just to have to move to Jessie a little bit later on. As someone said earlier, OS rollouts take some care to do in

Re: [ceph-users] update docs? just mounted a format2 rbd image with client 0.80.8 server 0.87.2

2015-07-31 Thread Ilya Dryomov
On Fri, Jul 31, 2015 at 2:21 PM, pixelfairy pixelfa...@gmail.com wrote: according to http://ceph.com/docs/master/rbd/rbd-snapshot/#layering, you have two choices, format 1: you can mount with rbd kernel module format 2: you can clone just mapped and mounted a this image, rbd image

[ceph-users] some basic concept questions

2015-07-31 Thread Charley Guan
dear Ceph experts; I am pretty new in ceph project and we are working on a management infrastructure and using Ceph / Calamari as our storage resource. I have some basic questions: 1) what is the purpose of installing and configuring salt-master and salt-minion in Ceph environment? is this

Re: [ceph-users] OSD startup causing slow requests - one tip from me

2015-07-31 Thread Haomai Wang
On Fri, Jul 31, 2015 at 5:47 PM, Jan Schermer j...@schermer.cz wrote: I know a few other people here were battling with the occasional issue of OSD being extremely slow when starting. I personally run OSDs mixed with KVM guests on the same nodes, and was baffled by this issue occuring

[ceph-users] rados bench multiple clients error

2015-07-31 Thread Kenneth Waegeman
Hi, I was trying rados bench, and first wrote 250 objects from 14 hosts with --no-cleanup. Then I ran the read tests from the same 14 hosts and ran into this: [root@osd007 test]# /usr/bin/rados -p ectest bench 100 seq 2015-07-31 17:52:51.027872 7f6c40de17c0 -1 WARNING: the following

Re: [ceph-users] OSD startup causing slow requests - one tip from me

2015-07-31 Thread Jan Schermer
On 31 Jul 2015, at 17:28, Haomai Wang haomaiw...@gmail.com wrote: On Fri, Jul 31, 2015 at 5:47 PM, Jan Schermer j...@schermer.cz wrote: I know a few other people here were battling with the occasional issue of OSD being extremely slow when starting. I personally run OSDs mixed with KVM

Re: [ceph-users] Check networking first?

2015-07-31 Thread Jan Schermer
I remember reading that ScaleIO (I think?) does something like this by regularly sending reports to a multicast group, thus any node with issues (or just overload) is reweighted or avoided automatically on the client. OSD map is the Ceph equivalent I guess. It makes sense to gather metrics and

Re: [ceph-users] Check networking first?

2015-07-31 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Even just a ping at max MTU set with nodefrag could tell a lot about connectivity issues and latency without a lot of traffic. Using Ceph messenger would be even better to check firewall ports. I like the idea of incorporating simple network checks

Re: [ceph-users] OSD removal is not cleaning entry from osd listing

2015-07-31 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 I usually do the crush rm step second to last. I don't know if your modifying the osd after removing it from the CRUSH is putting it back in. 1. Stop OSD process 2. ceph osd rm 3. ceph osd crush rm osd. 4. ceph auth del osd. Can you try the crush

[ceph-users] Happy SysAdmin Day!

2015-07-31 Thread Mark Nelson
Most folks have either probably already left or are on their way out the door late on a friday, but I just wanted to say Happy SysAdmin day to all of the excellent System Administrators out there running Ceph clusters. :) Mark ___ ceph-users mailing

Re: [ceph-users] Happy SysAdmin Day!

2015-07-31 Thread Michael Kuriger
Thanks Mark you too Michael Kuriger Sr. Unix Systems Engineer * mk7...@yp.com |( 818-649-7235 On 7/31/15, 3:02 PM, ceph-users on behalf of Mark Nelson ceph-users-boun...@lists.ceph.com on behalf of mnel...@redhat.com wrote: Most folks have either probably already left or are on their

[ceph-users] OSD removal is not cleaning entry from osd listing

2015-07-31 Thread Mallikarjun Biradar
Hi, I had 27 OSD's in my cluster. I removed two of the OSD from (osd.20) host-3 (osd.22) host-6. user@host-1:~$ sudo ceph osd tree ID WEIGHTTYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY -1 184.67990 root default -7 82.07996 chassis chassis2 -4 41.03998 host

Re: [ceph-users] dropping old distros: el6, precise 12.04, debian wheezy?

2015-07-31 Thread Alexandre DERUMIER
As I still haven't heard or seen about any upstream distros for Debian Jessie (see also [1]), Gitbuilder is already done for jessie http://gitbuilder.ceph.com/ceph-deb-jessie-x86_64-basic/ @Sage : Don't known if something is blocking to release package officially ? - Mail original -

Re: [ceph-users] Happy SysAdmin Day!

2015-07-31 Thread Jan Schermer
May your bytes stay with you :) Happy bofhday! Jan On 01 Aug 2015, at 00:10, Michael Kuriger mk7...@yp.com wrote: Thanks Mark you too Michael Kuriger Sr. Unix Systems Engineer * mk7...@yp.com |( 818-649-7235 On 7/31/15, 3:02 PM, ceph-users on behalf of Mark Nelson

Re: [ceph-users] OSD removal is not cleaning entry from osd listing

2015-07-31 Thread Mallikarjun Biradar
For a moment it de-list removed OSD's and after sometime it again comes up in ceph osd tree listing. On Fri, Jul 31, 2015 at 12:45 PM, Mallikarjun Biradar mallikarjuna.bira...@gmail.com wrote: Hi, I had 27 OSD's in my cluster. I removed two of the OSD from (osd.20) host-3 (osd.22) host-6.

Re: [ceph-users] dropping old distros: el6, precise 12.04, debian wheezy?

2015-07-31 Thread Mariusz Gronczewski
On Thu, 30 Jul 2015 06:54:13 -0700 (PDT), Sage Weil sw...@redhat.com wrote: So... given that, I'd like to gauge user interest in these old distros. Specifically, CentOS6 / RHEL6 Ubuntu precise 12.04 Debian wheezy Would anyone miss them? Well, Centos 6 will be supported to 2020,

Re: [ceph-users] RGW + civetweb + SSL

2015-07-31 Thread Bernhard Duebi
Hi,I had the same problem. Aparently civetweb can talk https when run standalone. But I didn't find out how to pass the necessary options to civetweb through ceph. So, I put haproxy in front of civerweb. haproxy terminates the https connection and forwards the requests in plain text to

[ceph-users] OSD startup causing slow requests - one tip from me

2015-07-31 Thread Jan Schermer
I know a few other people here were battling with the occasional issue of OSD being extremely slow when starting. I personally run OSDs mixed with KVM guests on the same nodes, and was baffled by this issue occuring mostly on the most idle (empty) machines. Thought it was some kind of race

Re: [ceph-users] Elastic-sized RBD planned?

2015-07-31 Thread pixelfairy
also, you probably want to reclaim unused space when you delete files. http://ceph.com/docs/master/rbd/qemu-rbd/#enabling-discard-trim On Fri, Jul 31, 2015 at 3:54 AM pixelfairy pixelfa...@gmail.com wrote: rbd is already thin provisioned. when you set its size, your setting the maximum size.

[ceph-users] problem with RGW

2015-07-31 Thread Butkeev Stas
Hello everybody We have ceph cluster that consist of 8 host with 12 osd per each host. It's 2T SATA disks. [13:23]:[root@se087 ~]# ceph osd tree ID WEIGHTTYPE NAMEUP/DOWN REWEIGHT PRIMARY-AFFINITY -1 182.99203 root default

[ceph-users] update docs? just mounted a format2 rbd image with client 0.80.8 server 0.87.2

2015-07-31 Thread pixelfairy
according to http://ceph.com/docs/master/rbd/rbd-snapshot/#layering, you have two choices, format 1: you can mount with rbd kernel module format 2: you can clone just mapped and mounted a this image, rbd image 'vm-101-disk-2': size 5120 MB in 1280 objects order 22 (4096 kB objects)

Re: [ceph-users] Check networking first?

2015-07-31 Thread John Spray
On 31/07/15 06:27, Stijn De Weirdt wrote: wouldn't it be nice that ceph does something like this in background (some sort of network-scrub). debugging network like this is not that easy (can't expect admins to install e.g. perfsonar on all nodes and/or clients) something like: every X min,

Re: [ceph-users] OSD removal is not cleaning entry from osd listing

2015-07-31 Thread John Spray
On 31/07/15 09:47, Mallikarjun Biradar wrote: For a moment it de-list removed OSD's and after sometime it again comes up in ceph osd tree listing. Is the OSD service itself definitely stopped? Are you using any orchestration systems (puppet, chef) that might be re-creating its auth key

Re: [ceph-users] OSD removal is not cleaning entry from osd listing

2015-07-31 Thread Mallikarjun Biradar
Yeah. OSD service stopped. Nope, I am not using any orchestration system. user@host-1:~$ ps -ef | grep ceph root 2305 1 7 Jul27 ?06:52:36 /usr/bin/ceph-osd --cluster=ceph -i 3 -f root 2522 1 6 Jul27 ?06:19:42 /usr/bin/ceph-osd --cluster=ceph -i 0 -f root

Re: [ceph-users] Check networking first?

2015-07-31 Thread Mark Nelson
On 07/31/2015 05:21 AM, John Spray wrote: On 31/07/15 06:27, Stijn De Weirdt wrote: wouldn't it be nice that ceph does something like this in background (some sort of network-scrub). debugging network like this is not that easy (can't expect admins to install e.g. perfsonar on all nodes and/or

Re: [ceph-users] OSD removal is not cleaning entry from osd listing

2015-07-31 Thread Mallikarjun Biradar
I am using hammer 0.94 On Fri, Jul 31, 2015 at 4:01 PM, Mallikarjun Biradar mallikarjuna.bira...@gmail.com wrote: Yeah. OSD service stopped. Nope, I am not using any orchestration system. user@host-1:~$ ps -ef | grep ceph root 2305 1 7 Jul27 ?06:52:36 /usr/bin/ceph-osd

Re: [ceph-users] Elastic-sized RBD planned?

2015-07-31 Thread pixelfairy
rbd is already thin provisioned. when you set its size, your setting the maximum size. its explained here, http://ceph.com/docs/master/rbd/rados-rbd-cmds/ On Thu, Jul 30, 2015 at 12:04 PM Robert LeBlanc rob...@leblancnet.us wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 I'll take a