Re: [ceph-users] Get Unique object id in cephfs

2014-05-27 Thread Shashank Puntamkar
@min chen Thanks for your reply. I am using 0.72.2 and avoiding cephfs as it is not termed as a production ready in this version. I plan to use only Rados gateway for object storage.I checked the documentation of Ceph and observerd that the concept of OID (as an Integer, or to establishes

[ceph-users] When the removed monitor will dissappear in ceph -s?

2014-05-27 Thread Cao, Buddy
Hi, I removed one of the monitor in the cluster (having 4 monitors previously), after 20 mins, the result of # ceph -s still contains ...a mons down Is it a common behavior? Does Ceph need so long to remove the monitor thoroughly? # ceph -s cluster 564a10da-5a0e-4a5f-ae64-a769995217c9

[ceph-users] WSGI file for ceph-rest-api

2014-05-27 Thread Wido den Hollander
Hi, I'm trying to run the ceph_rest_api module as a WSGI application behind Apache with mod_wsgi but I'm struggling a bit. Not having used WSGI that much I'm stuck on the .wsgi file. Has anybody done this before? I've been reading the Flask [0] and Ceph [1] docs, but it's not clear to me

Re: [ceph-users] ceph deploy on rhel6.5 installs ceph from el6 and fails

2014-05-27 Thread Alfredo Deza
Unfortunately, we didn't start building packages for RHEL6 until recently, and we would (historically) point CentOS and RHEL (and Scientific Linux too) to the `el6` repo. There is nothing regardless of what the admin host has in its repo files. The way around this problem is to specify directly

[ceph-users] Usage of Step choose

2014-05-27 Thread Sahana
HI all, Recently I was playing with crushmaps. I would like to know the difference between step choose and step chooseleaf. when I used : ruleset rule1{ ... step take default step choose firstn 0 host step emit } And I tested above using crushtool, it returned ids of bucket host. But when

[ceph-users] Can ceph object storage distinguish the upload domain and the download domain?

2014-05-27 Thread wsnote
There are two different modes of accessing the buckets.The upload domain and the download domain are the same. GET /mybucket HTTP/1.1 Host: cname.domain.com GET / HTTP/1.1 Host: mybucket.cname.domain.com Now I want to distinguish them. The reason that I want to do this is: I have two ceph

Re: [ceph-users] Ceph-deploy to deploy osds simultaneously

2014-05-27 Thread Alfredo Deza
There is no simultaneous support in ceph-deploy as this is something that is not needed for users setting up a cluster to try out ceph (the main objective of ceph-deploy) However, there have been users that used scripts that can call ceph-deploy in parallel so it might be possible to do so. For

Re: [ceph-users] Can ceph object storage distinguish the upload domain and the download domain?

2014-05-27 Thread wsnote
Thanks for reply! At 2014-05-27 22:13:19,Wido den Hollander w...@42on.com wrote: On 05/27/2014 04:10 PM, wsnote wrote: There are two different modes of accessing the buckets.The upload domain and the download domain are the same. GET /mybucket HTTP/1.1 Host: cname.domain.com GET / HTTP/1.1

Re: [ceph-users] How to backup mon-data?

2014-05-27 Thread Craig Lewis
On 5/23/14 16:20 , Cédric Lemarchand wrote: Out of curiosity, what's the current beef with zfs? I know what problems are cited for btrfs, but I haven't heard much about zfs lately. The Linux implementation (ZoL) is actually stable for production, but is quiet memory hungry because of a

Re: [ceph-users] ceph-disk: Error: No cluster conf found in /etc/ceph with fsid

2014-05-27 Thread Alfredo Deza
On Mon, May 26, 2014 at 5:22 AM, JinHwan Hwang calanc...@gmail.com wrote: I'm trying to install ceph 0.80.1 on ubuntu 14.04. All other things goes well except 'activate osd' phase. It tells me they can't find proper fsid when i do 'activate osd'. This is not my first time of installing ceph,

[ceph-users] ceph-deploy or manual?

2014-05-27 Thread Don Talton (dotalton)
I'd love to know how people are deploying their production clouds now. I've heard mixed answers about whether or not the right way is with ceph-deploy, or manual deployment. Are people using automation tools like puppet or ansible? Donald Talton Cloud Systems Development Cisco Systems

Re: [ceph-users] ceph-deploy or manual?

2014-05-27 Thread Alexandre DERUMIER
Are people using automation tools like puppet or ansible? http://www.sebastien-han.fr/blog/2014/05/01/vagrant-up-install-ceph-in-one-command/ enjoy ;) - Mail original - De: Don Talton (dotalton) dotal...@cisco.com À: ceph-users@lists.ceph.com Envoyé: Mardi 27 Mai 2014 18:19:00

[ceph-users] ceph.conf public network

2014-05-27 Thread Ignazio Cassano
Hi all, I read a lot of emails messages and I am confused because in some public network in /etc/ceph/ceph.com is reported like : public_network = a.b.c.d/netmask in others like : public network = a.b.c.d/netmask Does it depend by ceph version ? Ragards Ignazio

[ceph-users] rbd map hangs on Ceph Cluster

2014-05-27 Thread Sharmila Govind
Hi, I am setting up a ceph cluster for some experimentation. The cluster is setup successfully. But, When I try running rbd map on the host, the kernel crashes(system hangs) and I need to do a hard reset for it to recover. Below is my setup. ​ All my nodes have Linux kernel 3.5 with

Re: [ceph-users] ceph-disk: Error: No cluster conf found in /etc/ceph with fsid

2014-05-27 Thread Sharmila Govind
Hi, I have faced a similar issue. This happens if the ceph disks aren't purged/cleaned completely. Clear of the contents in the /dev/sdb1 device. There is a file named ceph_fsid in the disk which would have the old cluster's fsid. This needs to be deleted for it to work. Hope it helps. Sharmila

Re: [ceph-users] ceph-deploy or manual?

2014-05-27 Thread Craig Lewis
On 5/27/14 09:19 , Don Talton (dotalton) wrote: I'd love to know how people are deploying their production clouds now. I've heard mixed answers about whether or not the right way is with ceph-deploy, or manual deployment. Are people using automation tools like puppet or ansible? Donald

Re: [ceph-users] pgs incomplete; pgs stuck inactive; pgs stuck unclean

2014-05-27 Thread Sudarsan, Rajesh
I am seeing the same error message with ceph health command. I am using Ubuntu 14.04 with ceph 0.79. I am using the ceph distribution that comes with the Ubuntu release. My configuration is 1 x mon 1x OSD Both the OSD and mon are on the same host. rsudarsa@rsudarsa-ce1:~/mycluster$ ceph -s

Re: [ceph-users] How to backup mon-data?

2014-05-27 Thread Dimitri Maziuk
On 05/27/2014 10:30 AM, Craig Lewis wrote: A ZFS snapshot is atomic, but it doesn't tell the daemons to flush their logs to disk. Reverting to a snapshot looks the same as if you turned off the machine by yanking the power cord at the instant the snapshot was taken. That sounds more

Re: [ceph-users] How to get Unique object ID of object in ceph

2014-05-27 Thread minchen
Hi Shashank, I think you should use rados -p pool_name ls to list all objects in pool_name, if there is a image in pool_name , like this : root@ceph0:~# rados -p testpool ls rb.0.1017.6b8b4567.1806 rb.0.1017.6b8b4567.0f32 rb.0.1017.6b8b4567.1b5b

[ceph-users] Designing a cluster with ceph and benchmark (ceph vs ext4)

2014-05-27 Thread Listas@Adminlinux
Hi ! I have failover clusters for some aplications. Generally with 2 members configured with Ubuntu + Drbd + Ext4. For example, my IMAP cluster works fine with ~ 50k email accounts and my HTTP cluster hosts ~2k sites. See design here: http://adminlinux.com.br/cluster_design.txt I would like

[ceph-users] (no subject)

2014-05-27 Thread minchen
subscribe ceph-us...@ceph.com___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rbd map hangs on Ceph Cluster

2014-05-27 Thread Ilya Dryomov
On Tue, May 27, 2014 at 9:04 PM, Sharmila Govind sharmilagov...@gmail.com wrote: Hi, I am setting up a ceph cluster for some experimentation. The cluster is setup successfully. But, When I try running rbd map on the host, the kernel crashes(system hangs) and I need to do a hard reset for it

Re: [ceph-users] ceph.conf public network

2014-05-27 Thread Gregory Farnum
On Tue, May 27, 2014 at 9:55 AM, Ignazio Cassano ignaziocass...@gmail.com wrote: Hi all, I read a lot of emails messages and I am confused because in some public network in /etc/ceph/ceph.com is reported like : public_network = a.b.c.d/netmask in others like : public network =

Re: [ceph-users] Expanding pg's of an erasure coded pool

2014-05-27 Thread Gregory Farnum
On Sun, May 25, 2014 at 6:24 PM, Guang Yang yguan...@yahoo.com wrote: On May 21, 2014, at 1:33 AM, Gregory Farnum g...@inktank.com wrote: This failure means the messenger subsystem is trying to create a thread and is getting an error code back — probably due to a process or system thread

[ceph-users] Is there a way to repair placement groups?

2014-05-27 Thread phowell
Hi First apologies if this is the wrong place to ask this question. We are running a small Ceph (0.79) cluster will about 12 osd's which are on top of a zfs raid 1+0 (for another discussion)... which were created on this version. Below I have included the ceph health detail which has rather

Re: [ceph-users] Is there a way to repair placement groups?

2014-05-27 Thread Michael
Hi Peter, Please use ceph pg repair XX.xx. It might take a few seconds to kick in after being instructed. -Michael On 27/05/2014 21:40, phowell wrote: Hi First apologies if this is the wrong place to ask this question. We are running a small Ceph (0.79) cluster will about 12 osd's which

[ceph-users] Fwd: Re: pgs incomplete; pgs stuck inactive; pgs stuck unclean

2014-05-27 Thread Michael
Hi, Under normal conditions one OSD on one host is not enough to get a cluster healthy. You'd need a minimum of one OSD on three hosts to get clean. Your OSD dump shows *replicated size 3 min_size 2* so that's healthy at 3 copies of data, not healthy at two copies but still usable, cluster

Re: [ceph-users] Is there a way to repair placement groups? [Offtopic - ZFS]

2014-05-27 Thread Craig Lewis
On 5/27/14 13:40 , phowell wrote: Hi First apologies if this is the wrong place to ask this question. We are running a small Ceph (0.79) cluster will about 12 osd's which are on top of a zfs raid 1+0 (for another discussion)... which were created on this version. Just a reminder to

Re: [ceph-users] Is there a way to repair placement groups?

2014-05-27 Thread Gregory Farnum
Note that while the repair command *will* return your cluster to consistency, it is not guaranteed to restore the data you want to see there — in general, it will simply put the primary OSD's view of the world on the replicas. If you have a massive inconsistency like that, you probably want to

Re: [ceph-users] Is there a way to repair placement groups?

2014-05-27 Thread Michael
Would it be feasible to try for an odd one out policy by default when repairing from a pool of 3 or more disks? Or is the most common cause of inconsistency most likely to not effect the primary? -Michael On 27/05/2014 23:55, Gregory Farnum wrote: Note that while the repair command *will*

Re: [ceph-users] Ceph-deploy to deploy osds simultaneously

2014-05-27 Thread Craig Lewis
In practice, it's not a big deal. Just deploy your disks sequentially, and Ceph will sort it out. Sure, you'll waste a bit of time watching data copy to a new disk, only to see it get remapped it to a newer disk. It's a small period of time relative to how long it's going to take to remap

Re: [ceph-users] Is there a way to repair placement groups?

2014-05-27 Thread Gregory Farnum
Yeah, there are a lot of smarts like that which could be added in. -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Tue, May 27, 2014 at 4:17 PM, Michael mich...@onlinefusion.co.uk wrote: Would it be feasible to try for an odd one out policy by default when repairing from

Re: [ceph-users] full osd ssd cluster advise : replication 2x or 3x ?

2014-05-27 Thread Łukasz Jagiełło
Hi, I've got 16 nodes cluster ssd only. Each node is 6x600GB, 10Gbit uplink. We're using Intel 320 series. Cluster is running now half year as production and no problems with ssds. Replication x3 (main DC) and x2 in backup DC (10 nodes cluster there = less space). From what I've notice it's

Re: [ceph-users] 70+ OSD are DOWN and not coming up

2014-05-27 Thread Craig Lewis
On 5/22/14 00:26 , Craig Lewis wrote: On 5/21/14 21:15 , Sage Weil wrote: On Wed, 21 May 2014, Craig Lewis wrote: If you do this over IRC, can you please post a summary to the mailling list? I believe I'm having this issue as well. In the other case, we found that some of the OSDs were

[ceph-users] How to implement a rados plugin to encode/decode data while r/w

2014-05-27 Thread Plato
For certain security issue, I need to make sure the data finally saved to disk is encrypted. So, I'm trying to write a rados class, which would be triggered to reading and writing process. That is, before data is written, encrypting method of the class will be invoked; and then after data is

Re: [ceph-users] rbd map hangs on Ceph Cluster

2014-05-27 Thread Sharmila Govind
Thanks IIya. I upgraded the Kernel and it worked :-) Thanks, Sharmila On Tue, May 27, 2014 at 11:26 PM, Ilya Dryomov ilya.dryo...@inktank.comwrote: On Tue, May 27, 2014 at 9:04 PM, Sharmila Govind sharmilagov...@gmail.com wrote: Hi, I am setting up a ceph cluster for some

[ceph-users] How to use Admin Ops API?

2014-05-27 Thread wsnote
Hi, everyone! I am trying to use Admin Ops API. In the document, there is a statement: An admin API request will be done on a URI that starts with the configurable ‘admin’ resource entry point. My question is: where is the configure of 'admin' and what is the default value?