[ceph-users] ceph-deploy install on remote machine error

2013-09-11 Thread kumar rishabh
I am new to ceph.I am trying to follow the official document to install ceph on the machine .All things work fine but whenever I try to install ceph using ceph-deploy install ceph-server I get the following error http://pastebin.com/HxzP5Npi . I am behind a proxy server .Initially I thought the

Re: [ceph-users] ceph-deploy install on remote machine error

2013-09-11 Thread Wolfgang Hennerbichler
Hi, I believe you need to tell apt about your proxy server: cat /etc/apt/apt.conf Acquire::http::Proxy http://my.proxy.server:3142;; wogri On 09/11/2013 08:28 AM, kumar rishabh wrote: I am new to ceph.I am trying to follow the official document to install ceph on the machine .All things

Re: [ceph-users] errors after kernel-upgrade

2013-09-11 Thread Markus Goldberg
Does noone have an idea ? I can't mount the cluster anymore. Thank you, Markus Am 10.09.2013 09:43, schrieb Markus Goldberg: Hi, i made a 'stop ceph-all' on my ceph-admin-host and then a kernel-upgrade from 3.9 to 3.11 on all of my 3 nodes. Ubuntu 13.04, ceph 0,68 The kernel-upgrade

Re: [ceph-users] Ceph space problem, garbage collector ?

2013-09-11 Thread Olivier Bonvalet
Hi, do you need more information about that ? thanks, Olivier Le mardi 10 septembre 2013 à 11:19 -0700, Samuel Just a écrit : Can you post the rest of you crush map? -Sam On Tue, Sep 10, 2013 at 5:52 AM, Olivier Bonvalet ceph.l...@daevel.fr wrote: I also checked that all files in that PG

[ceph-users] orphans rbd_data

2013-09-11 Thread Laurent Barbe
Hello, I deleted rbd images in format 2 but it seems to remain rbd_data objects : $ rados -p datashare ls | grep rbd_data | wc -l 1479669 $ rados -p datashare ls | grep rbd_header | wc -l 0 $ rados -p datashare ls | grep rbd_id | wc -l 0 Does anyone know when rbd_data object must be

[ceph-users] Number of Monitors per OSDs

2013-09-11 Thread Ian_M_Porter
Dell - Internal Use - Confidential Hi, What's a good rule of thumb to work out the number of monitors per OSDs in a cluster Regards Ian Dell Corporation Limited is registered in England and Wales. Company Registration Number: 2081369 Registered address: Dell House, The Boulevard, Cain Road,

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

2013-09-11 Thread Oliver Daudey
Hey Gregory, FYI: I just attempted to upgrade a second cluster where CephFS was in use and got this: -24 2013-09-11 12:00:36.674469 7f0de1438700 1 -- 194.109.43.76:6800/8335 -- 194.109.43.73:6789/0 -- mon_subscribe({mdsmap=525898+,monmap=26+,osdmap=529384}) v2 -- ?+0 0x35b0700 con 0x35a9580

Re: [ceph-users] Number of Monitors per OSDs

2013-09-11 Thread Wolfgang Hennerbichler
by node I mean physical servers. the number of OSD's doesn't really affect the choice of MON's. One Mon will always be the leader. But Mon's (and traffic to them) is very lightweight. So you don't need to fear. And yes, you omit the RAID in your server, you use one OSD per drive, which means you

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

2013-09-11 Thread Yan, Zheng
On Wed, Sep 11, 2013 at 9:12 PM, Yan, Zheng uker...@gmail.com wrote: On Wed, Sep 11, 2013 at 7:51 PM, Oliver Daudey oli...@xs4all.nl wrote: Hey Gregory, I wiped and re-created the MDS-cluster I just mailed about, starting out by making sure CephFS is not mounted anywhere, stopping all MDSs,

Re: [ceph-users] Ceph space problem, garbage collector ?

2013-09-11 Thread Olivier Bonvalet
Very simple test on a new pool ssdtest, with 3 replica full SSD (crushrule 3) : # rbd create ssdtest/test-mysql --size 102400 # rbd map ssdtest/test-mysql # dd if=/dev/zero of=/dev/rbd/ssdtest/test-mysql bs=4M count=500 # ceph df | grep ssdtest ssdtest10 2000M 0 502

[ceph-users] ocfs2 for OSDs?

2013-09-11 Thread Ugis
Hi, I wonder is ocfs2 suitable for hosting OSD data? In ceph documentation only XFS, ext4 and btrfs are discussed, but looking at ocfs2 feature list it theoretically also could host OSDs: Some of the notable features of the file system are: Optimized Allocations (extents, reservations, sparse,

Re: [ceph-users] Number of Monitors per OSDs

2013-09-11 Thread Joao Eduardo Luis
On 09/11/2013 10:55 AM, ian_m_por...@dell.com wrote: *Dell - Internal Use - Confidential * Hi, What’s a good rule of thumb to work out the number of monitors per OSDs in a cluster This question makes little sense. The number of monitors and the number of OSDs (or any other component in

Re: [ceph-users] ocfs2 for OSDs?

2013-09-11 Thread Mark Nelson
On 09/11/2013 08:58 AM, Ugis wrote: Hi, I wonder is ocfs2 suitable for hosting OSD data? In ceph documentation only XFS, ext4 and btrfs are discussed, but looking at ocfs2 feature list it theoretically also could host OSDs: Some of the notable features of the file system are: Optimized

Re: [ceph-users] Number of Monitors per OSDs

2013-09-11 Thread Wolfgang Hennerbichler
On 09/11/2013 11:55 AM, ian_m_por...@dell.com wrote: *Dell - Internal Use - Confidential * if this is dell internal, I probabloy shouldn't answer :) Hi, What’s a good rule of thumb to work out the number of monitors per OSDs in a cluster AFAIK there is no rule of thumb. I would dimension

[ceph-users] Module rbd not found on Ubuntu 13.04

2013-09-11 Thread Prasanna Gholap
Hi, I am new to ceph. I setup a ceph cluster for block storage (I refered : http://www.youtube.com/watch?v=R3gnLrsZSno) Cluster Machines are Centos 6.3 with kernel 2.6.32. Ceph version 0.67.3 I am trying to configure Ubuntu Client 113.04 for the same. I am getting rbd module not found error:

Re: [ceph-users] Ceph space problem, garbage collector ?

2013-09-11 Thread Olivier Bonvalet
I removed some garbage about hosts faude / rurkh / murmillia (they was temporarily added because cluster was full). So the clean CRUSH map : # begin crush map tunable choose_local_tries 0 tunable choose_local_fallback_tries 0 tunable choose_total_tries 50 # devices device 0 device0 device 1

Re: [ceph-users] ocfs2 for OSDs?

2013-09-11 Thread Ugis
No particular use case yet. Ocfs2 could be considered as option if it outperforms xfs/ext4 under osd - therefore question. It just seemed suitable as it can be considered stable(as xfs,ext4) and has unlimited xattrs. Regarding zfs - at times it seems that it is closer to production on linux and

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

2013-09-11 Thread Yan, Zheng
On Wed, Sep 11, 2013 at 10:06 PM, Oliver Daudey oli...@xs4all.nl wrote: Hey Yan, On 11-09-13 15:12, Yan, Zheng wrote: On Wed, Sep 11, 2013 at 7:51 PM, Oliver Daudey oli...@xs4all.nl wrote: Hey Gregory, I wiped and re-created the MDS-cluster I just mailed about, starting out by making sure

Re: [ceph-users] Module rbd not found on Ubuntu 13.04

2013-09-11 Thread Laurent Barbe
Hello, Is it a custom kernel ? You can verify that rbd module is enable in your kernel config : $ grep CONFIG_BLK_DEV_RBD /boot/config-3.8.0-19-generic CONFIG_BLK_DEV_RBD=m Laurent Barbe Le 11/09/2013 10:51, Prasanna Gholap a écrit : Hi, I am new to ceph. I setup a ceph cluster for block

Re: [ceph-users] orphans rbd_data

2013-09-11 Thread Sage Weil
On Wed, 11 Sep 2013, Laurent Barbe wrote: Hello, I deleted rbd images in format 2 but it seems to remain rbd_data objects : $ rados -p datashare ls | grep rbd_data | wc -l 1479669 $ rados -p datashare ls | grep rbd_header | wc -l 0 $ rados -p datashare ls | grep rbd_id | wc -l 0

Re: [ceph-users] orphans rbd_data

2013-09-11 Thread Laurent Barbe
Hello Sage, Thank you, I will do something like this : $ rados -p rbd ls | grep '^rbd_data.24bb2ae8944a.' | xargs -n 100 rados -p rbd rm Laurent Le 11/09/2013 17:37, Sage Weil a écrit : On Wed, 11 Sep 2013, Laurent Barbe wrote: Hello, I deleted rbd images in format 2 but it seems to

Re: [ceph-users] Module rbd not found on Ubuntu 13.04

2013-09-11 Thread Prasanna Gholap
Hi Laurent, Thanks for reply. I checked in my machine: ubuntu@ubuntu:~$ grep CONFIG_BLK_DEV_RBD /boot/config-3.8.0-19-generic CONFIG_BLK_DEV_RBD=m It is not a custom kernel. It is an AWS instance. ~Prasanna On Wed, Sep 11, 2013 at 8:57 PM, Laurent Barbe laur...@ksperis.com wrote: Hello,

Re: [ceph-users] ocfs2 for OSDs?

2013-09-11 Thread Sage Weil
On Wed, 11 Sep 2013, Ugis wrote: Hi, I wonder is ocfs2 suitable for hosting OSD data? In ceph documentation only XFS, ext4 and btrfs are discussed, but looking at ocfs2 feature list it theoretically also could host OSDs: Some of the notable features of the file system are: Optimized

[ceph-users] Ceph at the OpenNebulaConf

2013-09-11 Thread Jaime Melis
Hello everyone, As you may be aware, we are holding the first OpenNebula Conference [1] in Berlin, this 24-26 September. The conference is the perfect place to learn about practical Cloud Computing, aimed at cloud users, developers, executives and IT managers to help them tackle their

Re: [ceph-users] Module rbd not found on Ubuntu 13.04

2013-09-11 Thread Prasanna Gholap
Hi, I don't have rbd module in given location ubuntu@ubuntu:~$ ls /lib/modules/3.8.0-19-generic/kernel/drivers/block cryptoloop.ko floppy.ko nbd.ko ubuntu@ubuntu:~$ modinfo rbd ERROR: Module rbd not found. By the link about aws, rbd.ko isn't included yet in linux aws . I'll try to build the

Re: [ceph-users] Module rbd not found on Ubuntu 13.04

2013-09-11 Thread Laurent Barbe
Remove on AWS images ? For me, the module is here (ubuntu 13.04) : $ ls /lib/modules/$(uname -r)/kernel/drivers/block/ $ modinfo rbd filename: /lib/modules/3.8.0-30-generic/kernel/drivers/block/rbd.ko license:GPL author: Jeff Garzik j...@garzik.org description:rados

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

2013-09-11 Thread Gregory Farnum
On Wed, Sep 11, 2013 at 7:48 AM, Yan, Zheng uker...@gmail.com wrote: On Wed, Sep 11, 2013 at 10:06 PM, Oliver Daudey oli...@xs4all.nl wrote: Hey Yan, On 11-09-13 15:12, Yan, Zheng wrote: On Wed, Sep 11, 2013 at 7:51 PM, Oliver Daudey oli...@xs4all.nl wrote: Hey Gregory, I wiped and

Re: [ceph-users] [ceph-deploy] problem creating mds after a full cluster wipe

2013-09-11 Thread Alphe Salas Michels
Title: signature Alph Salas Ingeniero T.I Kepler Data Recovery Asturias 97, Las Condes Santiago- Chile (56 2) 2362 7504 asa...@kepler.cl

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

2013-09-11 Thread Oliver Daudey
Hey Yan, Just confirming that creating fresh pools and doing the newfs on those fixed the problem, while restarting the OSDs didn't, thanks again! If you come up with a permanent fix, let me know and I'll test it for you. Regards, Oliver On wo, 2013-09-11 at 22:48 +0800, Yan, Zheng

Re: [ceph-users] ocfs2 for OSDs?

2013-09-11 Thread Gregory Farnum
On Wed, Sep 11, 2013 at 12:55 PM, David Disseldorp dd...@suse.de wrote: Hi Sage, On Wed, 11 Sep 2013 09:18:13 -0700 (PDT) Sage Weil s...@inktank.com wrote: REFLINKs (inode-based writeable snapshots) This is the one item on this list I see that the ceph-osds could take real advantage of;

Re: [ceph-users] xfsprogs not found in RHEL

2013-09-11 Thread sriram
Thank you. That worked. I am trying to install the object storage based on the steps here - http://ceph.com/docs/master/install/rpm/#installing-ceph-packages Where can I get these RPMs? rpm -ivh fcgi-2.4.0-10.el6.x86_64.rpm rpm -ivh mod_fastcgi-2.4.6-2.el6.rf.x86_64.rpm On Tue, Sep 10, 2013

Re: [ceph-users] librados vs libcephfs performance for database broker

2013-09-11 Thread Gregory Farnum
On Fri, Sep 6, 2013 at 2:08 AM, Serge Slipchenko serge.slipche...@gmail.com wrote: Hi, I am setting up a cluster that is using Hypertable as one of the key components. This had required some fixes of CephBroker, which I hope would be integrated to the main Hypertable branch soon. However, it

Re: [ceph-users] ocfs2 for OSDs?

2013-09-11 Thread David Disseldorp
Hi Sage, On Wed, 11 Sep 2013 09:18:13 -0700 (PDT) Sage Weil s...@inktank.com wrote: REFLINKs (inode-based writeable snapshots) This is the one item on this list I see that the ceph-osds could take real advantage of; it would make object clones triggered by things like RBD snapshots

Re: [ceph-users] xfsprogs not found in RHEL

2013-09-11 Thread Nigel Williams
On Wed, Aug 28, 2013 at 4:46 PM, Stroppa Daniele (strp) s...@zhaw.ch wrote: You might need the RHEL Scalable File System add-on. Exactly. I understand this needs to be purchased from Red Hat in order to get access to it if you are using the Red Hat subscription management system. I expect you

[ceph-users] how to upload file by the ceph S3 php api

2013-09-11 Thread
To whom it may concern: I run ceph radosgw as s3 service, and the AWS SDK for PHP suport upload file for browser. the class is s3BrowserUpload , how the ceph s3 api use s3BrowserUpload class, and the ceph s3 api support upload file; ___

[ceph-users] how to upload file by the ceph S3 php api

2013-09-11 Thread
To whom it may concern: I run ceph radosgw as s3 service, and the AWS SDK for PHP suport upload file for browser. the class is s3BrowserUpload , how the ceph s3 api use s3BrowserUpload class, and the ceph s3 api support upload file; the AWS example ?php require_once 'sdk.class.php';

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

2013-09-11 Thread Yan, Zheng
On Thu, Sep 12, 2013 at 3:26 AM, Oliver Daudey oli...@xs4all.nl wrote: Hey Yan, Just confirming that creating fresh pools and doing the newfs on those fixed the problem, while restarting the OSDs didn't, thanks again! If you come up with a permanent fix, let me know and I'll test it for you.

Re: [ceph-users] [RadosGW] Performance for Concurrency Connections

2013-09-11 Thread Kuo Hugo
Hi Yehuda, Here's my ceph.conf root@p01:/tmp# cat /etc/ceph/ceph.conf [global] fsid = 6e05675c-f545-4d88-9784-ea56ceda750e mon_initial_members = s01, s02, s03 mon_host = 192.168.2.61,192.168.2.62,192.168.2.63 auth_supported = cephx osd_journal_size = 1024 filestore_xattr_use_omap = true

Re: [ceph-users] [RadosGW] Performance for Concurrency Connections

2013-09-11 Thread Yehuda Sadeh
On Wed, Sep 11, 2013 at 9:34 PM, Kuo Hugo tonyt...@gmail.com wrote: Hi Yehuda, Here's my ceph.conf root@p01:/tmp# cat /etc/ceph/ceph.conf [global] fsid = 6e05675c-f545-4d88-9784-ea56ceda750e mon_initial_members = s01, s02, s03 mon_host = 192.168.2.61,192.168.2.62,192.168.2.63

Re: [ceph-users] [RadosGW] Performance for Concurrency Connections

2013-09-11 Thread Kuo Hugo
Hmm Interesting now. I have no admin socket opened around. root@p01:/var/run/ceph# ls /var/run/ceph -al total 0 drwxr-xr-x 2 root root 40 Sep 9 07:47 . drwxr-xr-x 17 root root 600 Sep 11 21:23 .. root@p01:/var/run/ceph# lsof | grep radosgw.asok root@p01:/var/run/ceph# I review the

Re: [ceph-users] [RadosGW] Performance for Concurrency Connections

2013-09-11 Thread Yehuda Sadeh
On Wed, Sep 11, 2013 at 9:57 PM, Kuo Hugo tonyt...@gmail.com wrote: Hmm Interesting now. I have no admin socket opened around. Maybe your radosgw process doesn't have permissions to write into /var/run/ceph? root@p01:/var/run/ceph# ls /var/run/ceph -al total 0 drwxr-xr-x 2 root root

Re: [ceph-users] [RadosGW] Performance for Concurrency Connections

2013-09-11 Thread Kuo Hugo
thanks 1) I'm sure there's no asok socket filer for the radosgw in my RadosGW host. 2) The rgw_thread_pool_size was set to 200 in my ceph.conf. So that the radosgw is using the value now generally. 3) If so, the tweaking of rgw_thread_pool_size value from 100-200 was not help for improve the

Re: [ceph-users] [RadosGW] Performance for Concurrency Connections

2013-09-11 Thread Yehuda Sadeh
On Wed, Sep 11, 2013 at 10:25 PM, Kuo Hugo tonyt...@gmail.com wrote: thanks 1) I'm sure there's no asok socket filer for the radosgw in my RadosGW host. 2) The rgw_thread_pool_size was set to 200 in my ceph.conf. So that the radosgw is using the value now generally. 3) If so, the tweaking

Re: [ceph-users] [RadosGW] Performance for Concurrency Connections

2013-09-11 Thread Kuo Hugo
Yes. I restart it by /etc/init.d/radosgw for times before. :D btw, I check several things here to prevent any permission issue. root@p01:/var/run# /etc/init.d/radosgw start Starting client.radosgw.gateway... root@p01:/var/run# ps aux | grep rados root 25823 1.8 0.0 16436340 7096 ?

Re: [ceph-users] [RadosGW] Performance for Concurrency Connections

2013-09-11 Thread Yehuda Sadeh
On Wed, Sep 11, 2013 at 10:37 PM, Kuo Hugo tonyt...@gmail.com wrote: Yes. I restart it by /etc/init.d/radosgw for times before. :D btw, I check several things here to prevent any permission issue. root@p01:/var/run# /etc/init.d/radosgw start Starting client.radosgw.gateway...