[ceph-users] infernalis osd activation on centos 7

Hi guys, After managing to get the mons up, I am stuck at activating the osds with the error below [ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.28): /usr/bin/ceph-deploy disk activate osd01:sdb1:sdb2

Re: [ceph-users] Ceph Sizing

You've left out an important factorcost. Otherwise I would just say buy enough SSD to cover the capacity. > -Original Message- > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of > Sam Huracan > Sent: 02 December 2015 15:46 > To: ceph-us...@ceph.com > Subject:

Re: [ceph-users] Ceph Sizing

One more factor we need to consider here is IO size(block size) to get required IOPS, based on this we can calculate the bandwidth and design the solution. Thanks Srinivas -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Nick Fisk Sent:

[ceph-users] Ceph Sizing

Hi, I'm building a storage structure for OpenStack cloud System, input: - 700 VM - 150 IOPS per VM - 20 Storage per VM (boot volume) - Some VM run database (SQL or MySQL) I want to ask a sizing plan for Ceph to satisfy the IOPS requirement, I list some factors considered: - Amount of OSD (SAS

[ceph-users] how to mount a bootable VM image file?

I'm using OpenStack to create VMs. They're KVM VMs, and I can see all the authentication information I need on the process tree. I want to mount this bootable image on the hypervizor node to access its filesystem and fix a file I messed up in /etc/ so I can get the VM to boot. [root@ceph mnt]#

Re: [ceph-users] Infernalis for Debian 8 armhf

i get cluster if PI i can use to build arm ver on it who di i need to talk to about uploading it the ceph repos ? On Wed, Dec 2, 2015 at 9:01 AM, Alfredo Deza wrote: > On Tue, Dec 1, 2015 at 11:58 PM, Swapnil Jain wrote: > > > > Hi, > > > > Any plans to

[ceph-users] Ceph Sizing

Hi, I'm building a storage structure for OpenStack cloud System, input: - 700 VM - 150 IOPS per VM - 20 Storage per VM (boot volume) - Some VM run database (SQL or MySQL) I want to ask a sizing plan for Ceph to satisfy the IOPS requirement, I list some factors considered: - Amount of OSD (SAS

Re: [ceph-users] does anyone know what xfsaild and kworker are?they make osd disk busy. produce 100-200iops per osd disk?

Ignore my last reply. I read the thread [Re: XFS Syncd]("http://oss.sgi.com/archives/xfs/2015-06/msg00111.html;), and found that might be okay. The call xfs_ail_push is almost INODE rather than BUF (1579 vs 99). Our ceph is dedicated to S3 service, and the write is small. So, where are so many

Re: [ceph-users] how to mount a bootable VM image file?

There's a pretty cool thing caled libguestfs, and a tool called guestfish http://libguestfs.org I've never used it (just stumbled on it recently) but it should do exactly what you need :-) And it supports RBD. Jan > On 02 Dec 2015, at 18:07, Gregory Farnum

Re: [ceph-users] radosgw in 0.94.5 leaking memory?

On Wed, Dec 2, 2015 at 11:09 AM, Dan van der Ster wrote: > Hi, > > We've had increased user activity on our radosgw boxes the past two > days and are finding that the radosgw is growing quickly in used > memory. Most of our gateways are VMs with 4GB of memory and these are >

Re: [ceph-users] Flapping OSDs, Large meta directories in OSDs

On Tue, Dec 1, 2015 at 10:02 AM, Tom Christensen wrote: > Another thing that we don't quite grasp is that when we see slow requests > now they almost always, probably 95% have the "known_if_redirected" state > set. What does this state mean? Does it indicate we have OSD maps

Re: [ceph-users] Infernalis for Debian 8 armhf

If you can point me to some documention, I can do that. — Swapnil Jain > On 02-Dec-2015, at 7:31 pm, Alfredo Deza wrote: > > On Tue, Dec 1, 2015 at 11:58 PM, Swapnil Jain wrote: >> >> Hi, >> >> Any plans to release Infernalis Debian 8 binary packages for

Re: [ceph-users] OSD crash, unable to restart

Hi, On 12/02/2015 08:12 PM, Gregory Farnum wrote: On Wed, Dec 2, 2015 at 11:11 AM, Major Csaba wrote: Hi, [ sorry, I accidentaly left out the list address ] This is the content of the LOG file in the directory /var/lib/ceph/osd/ceph-7/current/omap:

Re: [ceph-users] Infernalis for Debian 8 armhf

for now the prosses for me was : git clone https://github.com/ceph/ceph.git git checkout infernalis cd ceph apt-get install debhelper autoconf automake autotools-dev libbz2-dev cmake default-jdk gdisk javahelper junit4 libaio-dev libatomic-ops-dev libbabeltrace-ctf-dev libbabeltrace-dev

Re: [ceph-users] OSD crash, unable to restart

On Wed, Dec 2, 2015 at 10:54 AM, Major Csaba wrote: > Hi, > > I have a small cluster(5 nodes, 20OSDs), where an OSD crashed. There is no > any other signal of problems. No kernel message, so the disks seem to be OK. > > I tried to restart the OSD but the process stops

[ceph-users] OSD crash, unable to restart

Hi, I have a small cluster(5 nodes, 20OSDs), where an OSD crashed. There is no any other signal of problems. No kernel message, so the disks seem to be OK. I tried to restart the OSD but the process stops almost immediately with the same logs. Version is 0.94.5

Re: [ceph-users] OSD crash, unable to restart

On Wed, Dec 2, 2015 at 11:11 AM, Major Csaba wrote: > Hi, > [ sorry, I accidentaly left out the list address ] > > This is the content of the LOG file in the directory > /var/lib/ceph/osd/ceph-7/current/omap: > 2015/12/02-18:48:12.241386 7f805fc27900 Recovering log #26281

Re: [ceph-users] does anyone know what xfsaild and kworker are?they make osd disk busy. produce 100-200iops per osd disk?

I think each write will create 2 objects (512 KB head object + rest of the contents) if your object size > 512KB. Also, it is writing some xattrs on top of what OSD is writing. Don't take my word blindly as I am not fully familiar with RGW :-) This will pollute significant number of INODE I

Re: [ceph-users] does anyone know what xfsaild and kworker are?they make osd disk busy. produce 100-200iops per osd disk?

It works. However, I think the root case is due to the xfs_buf missing? trace-cmd record -e xfs\* trace-cmd report > xfs.txt awk '{print $4}' xfs2.txt |sort -n |uniq -c|sort -n|tail -n 20 14468 xfs_file_splice_write: 16562 xfs_buf_find: 19597 xfs_buf_read: 19634 xfs_buf_get: 21943

[ceph-users] New cluster performance analysis

Hi everyone, I am currently testing our new cluster and I would like some feedback on the numbers I am getting. For the hardware : 7 x OSD : 2 x Intel 2640v3 (8x2.6GHz), 64B RAM, 2x10Gbits LACP for public net., 2x10Gbits LACP for cluster net., MTU 9000 1 x MON : 2 x Intel 2630L (6x2GHz),

[ceph-users] systemctl enable ceph-mon fails in ceph-deploy create initial (no such service)

Hey folks. Running RHEL7.1 with stock 3.10.0 kernel and trying to deploy Infernalis. Haven't done this since Firefly but I used to know what I was doing. My problem is "ceph-deploy new" and "ceph-deploy install" seem to go well but "ceph-deploy mon create-initial" reliably fails when

Re: [ceph-users] New cluster performance analysis

> Let's take IOPS, assuming the spinners can do 50 (4k) synced sustained IOPS > (I hope they can do more ^^), we should be around 50x84/3 = 1400 IOPS, which > is far from rados bench (538) and fio (847). And surprisingly fio numbers are > greater than rados. > I think the missing factor here

[ceph-users] Mon quorum fails

Hi, My Mon quorum includes 3 nodes, if 2 nodes fail out incidently. How could I recover system from 1 node left? Thanks and regards. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] How long will the logs be kept?

Hi ,All Is there anyone who knows How long or how many days will the logs.gz (mon/osd/mds)be kept, maybe before flushed? - wukongming ID: 12019 Tel：0571-86760239 Dept：2014 UIS2 OneStor

Re: [ceph-users] ceph-disk activate Permission denied problems

You should check that the owner of your ceph partitions (both journal and data) is 'ceph', otherwise the ceph user won't mount it. You can simply do : chown ceph:disk /dev/sdc3 If this solve your issue you should set the GPT GUID [1] of the partitions with a tool like sgdisk to make this

Re: [ceph-users] How long will the logs be kept?

it will rotate every week by default, you can see the logrotate file /etc/ceph/logrotate.d/ceph 2015-12-03 12:37 GMT+08:00 Wukongming : > Hi ,All > Is there anyone who knows How long or how many days will the logs.gz > (mon/osd/mds)be kept, maybe before flushed? > >

Re: [ceph-users] infernalis osd activation on centos 7

- Original Message - > From: "Dan Nica" > To: ceph-us...@ceph.com > Sent: Thursday, 3 December, 2015 1:39:16 AM > Subject: [ceph-users] infernalis osd activation on centos 7 > Hi guys, > After managing to get the mons up, I am stuck at activating the

Re: [ceph-users] Ceph Sizing

IO size is 4 KB, and I need a Minimum sizing, cost optimized I intend use SuperMicro Devices http://www.supermicro.com/solutions/storage_Ceph.cfm What do you think? 2015-12-02 23:17 GMT+07:00 Srinivasula Maram : > One more factor we need to consider here is IO

[ceph-users] radosgw in 0.94.5 leaking memory?

Hi, We've had increased user activity on our radosgw boxes the past two days and are finding that the radosgw is growing quickly in used memory. Most of our gateways are VMs with 4GB of memory and these are getting OOM-killed after ~30 mins of high user load. We added a few physical gateways with

Re: [ceph-users] Removing OSD - double rebalance?

On 30 November 2015 at 09:34, Burkhard Linke wrote: > On 11/30/2015 10:08 AM, Carsten Schmitt wrote: >> But after entering the last command, the cluster starts rebalancing again. >> >> And that I don't understand: Shouldn't be one rebalancing

Re: [ceph-users] Removing OSD - double rebalance?

1) if you have the original drive that works and just want to replace it then you can just "dd" it over to the new drive and then extend the partition if the new one is larger, this avoids double backfilling in this case 2) if the old drive is dead you should "out" it and at the same time add a

Re: [ceph-users] Removing OSD - double rebalance?

Hi all, On 12/02/2015 12:10 PM, Jan Schermer wrote: 1) if you have the original drive that works and just want to replace it then you can just "dd" it over to the new drive and then extend the partition if the new one is larger, this avoids double backfilling in this case 2) if the old drive

[ceph-users] RBD: Missing 1800000000 when map block device

Hi all, When I map block device in client kernel 3.10.93-1.el6.elrepo.x86_64, i get error: libceph: mon0 ip:6789 feature set mismatch, my 4a042a42 < server's 184a042a42, *missing 18* libceph: mon0 ip:6789 socket error on read I used ceph version 0.87.2 I tried to find the problem in

Re: [ceph-users] Removing OSD - double rebalance?

On 2 December 2015 at 11:10, Jan Schermer wrote: > 1) if you have the original drive that works and just want to replace it then > you can just "dd" it over to the new drive and then extend the partition if > the new one is larger, this avoids double backfilling in this case >

[ceph-users] infernalis on centos 7

Hi guys, I try to setup the cluster but it seems to fail for me on setting up the mons with the error bellow: [ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.25): /usr/bin/ceph-deploy mon create cmon01 cmon02 cmon03

Re: [ceph-users] Removing OSD - double rebalance?

Here's something that I didn't see mentioned in this thread yet: the set of PGs mapped to an OSD is a function of the ID of that OSD. So, if you replace a drive but don't reuse the same OSD ID for the replacement, you'll have more PG movement than if you kept the ID. -- dan On Wed, Dec 2, 2015

Re: [ceph-users] RBD: Missing 1800000000 when map block device

On Wed, Dec 2, 2015 at 12:15 PM, MinhTien MinhTien wrote: > Hi all, > > When I map block device in client kernel 3.10.93-1.el6.elrepo.x86_64, i get > error: > > libceph: mon0 ip:6789 feature set mismatch, my 4a042a42 < server's > 184a042a42, missing 18 >

Re: [ceph-users] infernalis on centos 7

I'm guessing that your ceph deploy version is not up-to-date and you are usinf one that does not support systemd. Be sure to update to the latest ceph-deploy available in the ceph noarch repo (1.5.28) and you should be fine. Adrien On Wed, Dec 2, 2015 at 1:08 PM, Dan Nica

Re: [ceph-users] Infernalis for Debian 8 armhf

On Tue, Dec 1, 2015 at 11:58 PM, Swapnil Jain wrote: > > Hi, > > Any plans to release Infernalis Debian 8 binary packages for armhf. As I only > see it for amd64. This would be pretty simple to do but we don't have any ARM boxes around and nothing is immediately available for

Re: [ceph-users] F21 pkgs for Ceph Hammer release ?

On Tue, Dec 1, 2015 at 4:59 AM, Deepak Shetty wrote: > Hi, > Does anybody how/where I can get the F21 repo for ceph hammer release ? > > In download.ceph.com/rpm-hammer/ I only see F20 dir, not F21 Right, we haven't built FC binaries for a while, the one node we had for FC

[ceph-users] ceph-disk activate Permission denied problems

Dear Cephers... 1./ I am currently deploying infernalis 9.2.0 in centos7. 2./ I am not using ceph-deploy (I prefer not to use it) and I prefer to use the short form osd installation procedure described here: http://docs.ceph.com/docs/infernalis/install/manual-deployment/#short-form 3./