Re: [ceph-users] All pools have size=3 but MB data and MB used ratio is 1 to 5

2015-03-27 Thread Saverio Proto
I will start now to push a lot of data into the cluster to see if the metadata grows a lot or stays costant. There is a way to clean up old metadata ? I pushed a lot of more data to the cluster. Then I lead the cluster sleep for the night. This morning I find this values: 6841 MB data 25814

Re: [ceph-users] Hammer release data and a Design question

2015-03-27 Thread Udo Lembke
Hi, Am 26.03.2015 11:18, schrieb 10 minus: Hi , I 'm just starting on small Ceph implementation and wanted to know the release date for Hammer. Will it coincide with relase of Openstack. My Conf: (using 10G and Jumboframes on Centos 7 / RHEL7 ) 3x Mons (VMs) : CPU - 2 Memory - 4G

[ceph-users] Fwd: ceph-deploy : Certificate Error using wget on Debian

2015-03-27 Thread Vasilis Souleles
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I'm trying to create a 4-node Ceph Storage Cluster using ceph-deploy, following the official guide: http://docs.ceph.com/docs/master/start/quick-ceph-deploy/ I'm using debian wheezy 7 (x86_64) on all nodes and on each node, `uname -a`

Re: [ceph-users] More than 50% osds down, CPUs still busy; will the cluster recover without help?

2015-03-27 Thread Mikaël Cluseau
Hi, On 03/18/2015 03:01 PM, Gregory Farnum wrote: I think it tended to crash rather than hang like this so I'm a bit surprised, but if this op is touching a broken file or something that could explain it. FWIW, the last time I had the issue (on a 3.10.9 kernel), btrfs was freezing, waiting

Re: [ceph-users] Migrating objects from one pool to another?

2015-03-27 Thread Karan Singh
Hi Jean You would probably need this ceph osd pool create glance-images-bkp 128 128 rados cppool glance-images glance-images-bkp ceph osd pool rename glance-images glance-images-old ceph osd pool rename glance-images-bkp glance-images ceph osd pool delete glance-images-old glance-images-old

Re: [ceph-users] Fwd: ceph-deploy : Certificate Error using wget on Debian

2015-03-27 Thread Brian Rak
It looks like ceph.com is having some major issues with their git repository right now.. https://ceph.com/git/ gives a 500 error On 3/27/2015 8:11 AM, Vasilis Souleles wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I'm trying to create a 4-node Ceph Storage Cluster using

Re: [ceph-users] Snapshots and fstrim with cache tiers ?

2015-03-27 Thread Frédéric Nass
Hello, The snapshot with a cache tier part was answered by Greg Farnum (https://www.mail-archive.com/ceph-users@lists.ceph.com/msg18329.html). What about fstrim with a cache tier ? It doesn't seem to work. Also is there a background task that recovers freed blocks ? Best regards,

Re: [ceph-users] ceph -s slow return result

2015-03-27 Thread Kobi Laredo
What's the current health of the cluster? It may help to compact the monitors' LevelDB store if they have grown in size http://www.sebastien-han.fr/blog/2014/10/27/ceph-mon-store-taking-up-a-lot-of-space/ Depends on the size of the mon's store size it may take some time to compact, make sure to do

[ceph-users] monitor 0.87.1 crashes

2015-03-27 Thread samuel
Hi all, In a fully functional ceph installation today we suffer a problem with ceph monitors, that started crashing with following error: include/interval_set.h: 340: FAILED assert(0) Is there any related bug? Thanks a lot in advance, Samuel ___

Re: [ceph-users] CephFS Slow writes with 1MB files

2015-03-27 Thread Mark Nelson
Specifically related to BTRFS, if you have random IO to existing objects it will cause terrible fragmentation due to COW. BTRFS is often faster than XFS initially but after it starts fragmenting can become much slower for sequential reads. You may want to try XFS again and see if you can

Re: [ceph-users] ceph -s slow return result

2015-03-27 Thread Gregory Farnum
Are all your monitors running? Usually a temporary hang means that the Ceph client tries to reach a monitor that isn't up, then times out and contacts a different one. I have also seen it just be slow if the monitors are processing so many updates that they're behind, but that's usually on a very

Re: [ceph-users] ceph -s slow return result

2015-03-27 Thread Chu Duc Minh
All my monitors running. But i deleting pool .rgw.buckets, now having 13 million objects (just test data). The reason that i must delete this pool is my cluster become unstable, and sometimes an OSD down, PG peering, incomplete,... Therefore i must delete this pool to re-stablize my cluster.

Re: [ceph-users] ERROR: missing keyring, cannot use cephx for authentication

2015-03-27 Thread Jesus Chavez (jeschave)
the thing is that the devices are not mounting after reboot… Any ideas? [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/

[ceph-users] ceph -s slow return result

2015-03-27 Thread Chu Duc Minh
On my CEPH cluster, ceph -s return result quite slow. Sometimes it return result immediately, sometimes i hang few seconds before return result. Do you think this problem (ceph -s slow return) only relate to ceph-mon(s) process? or maybe it relate to ceph-osd(s) too? (i deleting a big bucket,

[ceph-users] CephFS Slow writes with 1MB files

2015-03-27 Thread Barclay Jameson
I did a Ceph cluster install 2 weeks ago where I was getting great performance (~= PanFS) where I could write 100,000 1MB files in 61 Mins (Took PanFS 59 Mins). I thought I could increase the performance by adding a better MDS server so I redid the entire build. Now it takes 4 times as long to

Re: [ceph-users] CephFS Slow writes with 1MB files

2015-03-27 Thread Gregory Farnum
So this is exactly the same test you ran previously, but now it's on faster hardware and the test is slower? Do you have more data in the test cluster? One obvious possibility is that previously you were working entirely in the MDS' cache, but now you've got more dentries and so it's kicking data

Re: [ceph-users] Snapshots and fstrim with cache tiers ?

2015-03-27 Thread Gregory Farnum
On Wed, Mar 25, 2015 at 3:14 AM, Frédéric Nass frederic.n...@univ-lorraine.fr wrote: Hello, I have a few questions regarding snapshots and fstrim with cache tiers. In the cache tier and erasure coding FAQ related to ICE 1.2 (based on Firefly), Inktank says Snapshots are not supported in

Re: [ceph-users] monitor 0.87.1 crashes

2015-03-27 Thread Samuel Just
You'll want to at least include the backtrace. -Sam On 03/27/2015 10:55 AM, samuel wrote: Hi all, In a fully functional ceph installation today we suffer a problem with ceph monitors, that started crashing with following error: include/interval_set.h: 340: FAILED assert(0) Is there any

Re: [ceph-users] monitor 0.87.1 crashes

2015-03-27 Thread samuel
Here it it goes (in case further information is needed, just ask and I would gladly offer it): -5 2015-03-27 19:06:01.168361 7f94b4184700 5 mon.mon01@0(leader).osd e37404 send_incremental [37403..37404] to client.1419434 10.10.200.3:0/280 8592243 -4 2015-03-27 19:06:01.168427

Re: [ceph-users] monitor 0.87.1 crashes

2015-03-27 Thread samuel
apologies for the noise. Host 10.10.200.3 had some issues that made monitors to crash. Thanks a lot for your help, Samuel On 27 March 2015 at 19:09, samuel sam...@gmail.com wrote: Here it it goes (in case further information is needed, just ask and I would gladly offer it): -5

Re: [ceph-users] adding a new pool causes old pool warning pool x has too few pgs

2015-03-27 Thread Chad William Seys
Weird: After a few hours, health check comes back OK without changing the number of PGS for any pools ! Hi All, To a Healthy cluster I recently added two pools to ceph, 1 replicated and 1 ecpool. Then I made the replicated pool into a cache for the ecpool. Afterwards ceph

Re: [ceph-users] CephFS Slow writes with 1MB files

2015-03-27 Thread Barclay Jameson
Yes it's the exact same hardware except for the MDS server (although I tried using the MDS on the old node). I have not tried moving the MON back to the old node. My default cache size is mds cache size = 1000 The OSDs (3 of them) have 16 Disks with 4 SSD Journal Disks. I created 2048 for

Re: [ceph-users] CephFS Slow writes with 1MB files

2015-03-27 Thread Gregory Farnum
On Fri, Mar 27, 2015 at 2:46 PM, Barclay Jameson almightybe...@gmail.com wrote: Yes it's the exact same hardware except for the MDS server (although I tried using the MDS on the old node). I have not tried moving the MON back to the old node. My default cache size is mds cache size = 1000

Re: [ceph-users] 0.93 fresh cluster won't create PGs

2015-03-27 Thread Sage Weil
On Fri, 27 Mar 2015, Robert LeBlanc wrote: I've built Ceph clusters a few times now and I'm completely baffled about what we are seeing. We had a majority of the nodes on a new cluster go down yesterday and we got PGs stuck peering. We checked logs, firewalls, file descriptors, etc and nothing

[ceph-users] 0.93 fresh cluster won't create PGs

2015-03-27 Thread Robert LeBlanc
I've built Ceph clusters a few times now and I'm completely baffled about what we are seeing. We had a majority of the nodes on a new cluster go down yesterday and we got PGs stuck peering. We checked logs, firewalls, file descriptors, etc and nothing is pointing to what the problem is. We thought

Re: [ceph-users] ceph -s slow return result

2015-03-27 Thread Chu Duc Minh
@Kobi Laredo: thank you! It's exactly my problem. # du -sh /var/lib/ceph/mon/ *2.6G * /var/lib/ceph/mon/ # ceph tell mon.a compact compacted leveldb in 10.197506 # du -sh /var/lib/ceph/mon/ *461M*/var/lib/ceph/mon/ Now my ceph -s return result immediately. Maybe monitors' LevelDB store grow

Re: [ceph-users] 0.93 fresh cluster won't create PGs

2015-03-27 Thread Robert LeBlanc
Thanks, we'll give the gitbuilder packages a shot and report back. Robert LeBlanc Sent from a mobile device please excuse any typos. On Mar 27, 2015 10:03 PM, Sage Weil s...@newdream.net wrote: On Fri, 27 Mar 2015, Robert LeBlanc wrote: I've built Ceph clusters a few times now and I'm