Re: [ceph-users] Ceph on Solaris / Illumos

2015-04-15 Thread Mark Nelson
On 04/15/2015 08:16 AM, Jake Young wrote: Has anyone compiled ceph (either osd or client) on a Solaris based OS? The thread on ZFS support for osd got me thinking about using solaris as an osd server. It would have much better ZFS performance and I wonder if the osd performance without a

[ceph-users] Ceph on Solaris / Illumos

2015-04-15 Thread Jake Young
Has anyone compiled ceph (either osd or client) on a Solaris based OS? The thread on ZFS support for osd got me thinking about using solaris as an osd server. It would have much better ZFS performance and I wonder if the osd performance without a journal would be 2x better. A second thought I

Re: [ceph-users] Do I have enough pgs?

2015-04-15 Thread Mark Nelson
On 04/15/2015 08:10 AM, Tony Harris wrote: Hi all, I have a cluster of 3 nodes, 18 OSDs. I used the pgcalc to give a suggested number of PGs - here was my list: Group1 3 rep 18 OSDs 30% data 512PGs Group2 3 rep 18 OSDs 30% data 512PGs Group3 3 rep 18 OSDs 30% data 512PGs Group4

[ceph-users] Do I have enough pgs?

2015-04-15 Thread Tony Harris
Hi all, I have a cluster of 3 nodes, 18 OSDs. I used the pgcalc to give a suggested number of PGs - here was my list: Group1 3 rep 18 OSDs 30% data 512PGs Group2 3 rep 18 OSDs 30% data 512PGs Group3 3 rep 18 OSDs 30% data 512PGs Group4 2 rep 18 OSDs 5% data 256PGs Group5 2

Re: [ceph-users] Ceph on Solaris / Illumos

2015-04-15 Thread Jake Young
On Wednesday, April 15, 2015, Mark Nelson mnel...@redhat.com wrote: On 04/15/2015 08:16 AM, Jake Young wrote: Has anyone compiled ceph (either osd or client) on a Solaris based OS? The thread on ZFS support for osd got me thinking about using solaris as an osd server. It would have much

Re: [ceph-users] Ceph site is very slow

2015-04-15 Thread Gregory Farnum
People are working on it but I understand there was/is a DoS attack going on. :/ -Greg On Wed, Apr 15, 2015 at 1:50 AM Ignazio Cassano ignaziocass...@gmail.com wrote: Many thanks 2015-04-15 10:44 GMT+02:00 Wido den Hollander w...@42on.com: On 04/15/2015 10:20 AM, Ignazio Cassano wrote: Hi

Re: [ceph-users] Binding a pool to certain OSDs

2015-04-15 Thread Giuseppe Civitella
So it was a PG problem. I added a couple of OSD per host, reconfigured the CRUSH map and the cluster began to work properly. Thanks Giuseppe 2015-04-14 19:02 GMT+02:00 Saverio Proto ziopr...@gmail.com: No error message. You just finish the RAM memory and you blow up the cluster because of too

Re: [ceph-users] Ceph on Solaris / Illumos

2015-04-15 Thread Jake Young
On Wednesday, April 15, 2015, Alexandre Marangone amara...@redhat.com wrote: The LX branded zones might be a way to run OSDs on Illumos: https://wiki.smartos.org/display/DOC/LX+Branded+Zones For fun, I tried a month or so ago, managed to have a quorum. OSDs wouldn't start, I didn't look

Re: [ceph-users] Ceph on Solaris / Illumos

2015-04-15 Thread Alexandre Marangone
The LX branded zones might be a way to run OSDs on Illumos: https://wiki.smartos.org/display/DOC/LX+Branded+Zones For fun, I tried a month or so ago, managed to have a quorum. OSDs wouldn't start, I didn't look further as far as debugging. I'll give it a go when I have more time. On Wed, Apr 15,

Re: [ceph-users] Rados Gateway and keystone

2015-04-15 Thread ghislain.chevalier
Hi, Despite the creation of ec2 credentials which provides an accesskey and a secretkey for a user, it’s always impossible to connect using S3 (Forbidden/Access denied). All is right using swift (create container, list container, get object, put object, delete object) I use cloudberry client

Re: [ceph-users] Ceph on Solaris / Illumos

2015-04-15 Thread Mark Nelson
On 04/15/2015 10:36 AM, Jake Young wrote: On Wednesday, April 15, 2015, Mark Nelson mnel...@redhat.com mailto:mnel...@redhat.com wrote: On 04/15/2015 08:16 AM, Jake Young wrote: Has anyone compiled ceph (either osd or client) on a Solaris based OS? The thread

[ceph-users] Ceph repo - RSYNC?

2015-04-15 Thread Paul Mansfield
Sorry for starting a new thread, I've only just subscribed to the list and the archive on the mail listserv is far from complete at the moment. on 8th March David Moreau Simard said http://www.spinics.net/lists/ceph-users/msg16334.html that there was a rsync'able mirror of the ceph repo at

[ceph-users] ceph on Debian Jessie stopped working

2015-04-15 Thread Chad William Seys
Hi All, Earlier ceph on Debian Jessie was working. Jessie is running 3.16.7 . Now when I modprobe rbd , no /dev/rbd appear. # dmesg | grep -e rbd -e ceph [ 15.814423] Key type ceph registered [ 15.814461] libceph: loaded (mon/osd proto 15/24) [ 15.831092] rbd: loaded [ 22.084573] rbd:

Re: [ceph-users] Ceph repo - RSYNC?

2015-04-15 Thread Robert LeBlanc
http://eu.ceph.com/ has rsync and Hammer. On Wed, Apr 15, 2015 at 10:17 AM, Paul Mansfield paul.mansfi...@alcatel-lucent.com wrote: Sorry for starting a new thread, I've only just subscribed to the list and the archive on the mail listserv is far from complete at the moment. on 8th March

[ceph-users] mds crashing

2015-04-15 Thread Kyle Hutson
I upgraded to 0.94.1 from 0.94 on Monday, and everything had been going pretty well. Then, about noon today, we had an mds crash. And then the failover mds crashed. And this cascaded through all 4 mds servers we have. If I try to start it ('service ceph start mds' on CentOS 7.1), it appears to

Re: [ceph-users] Ceph repo - RSYNC?

2015-04-15 Thread David Moreau Simard
Hey, you're right. Thanks for bringing that to my attention, it's syncing now :) Should be available soon. David Moreau Simard On 2015-04-15 12:17 PM, Paul Mansfield wrote: Sorry for starting a new thread, I've only just subscribed to the list and the archive on the mail listserv is far from

Re: [ceph-users] Upgrade from Giant 0.87-1 to Hammer 0.94-1

2015-04-15 Thread Steffen W Sørensen
Also our calamari web UI won't authenticate anymore, can’t see any issues in any log under /var/log/calamari, any hints on what to look for are appreciated, TIA! # dpkg -l | egrep -i calamari\|ceph ii calamari-clients 1.2.3.1-2-gc1f14b2all Inktank

[ceph-users] many slow requests on different osds (scrubbing disabled)

2015-04-15 Thread Dominik Mostowiec
Hi, From few days we notice on our cluster many slow request. Cluster: ceph version 0.67.11 3 x mon 36 hosts - 10 osd ( 4T ) + 2 SSD (journals) Scrubbing and deep scrubbing is disabled but count of slow requests is still increasing. Disk utilisation is very small after we have disabled scrubbings.

[ceph-users] Upgrade from Giant 0.87-1 to Hammer 0.94-1

2015-04-15 Thread Steffen W Sørensen
Hi, Successfully upgrade a small development 4x node Giant 0.87-1 cluster to Hammer 0.94-1, each node with 6x OSD - 146GB, 19 pools, mainly 2 in usage. Only minor thing now ceph -s complaining over too may PGs, previously Giant had complain of too few, so various pools were bumped up till

Re: [ceph-users] mds crashing

2015-04-15 Thread Kyle Hutson
Thank you, John! That was exactly the bug we were hitting. My Google-fu didn't lead me to this one. On Wed, Apr 15, 2015 at 4:16 PM, John Spray john.sp...@redhat.com wrote: On 15/04/2015 20:02, Kyle Hutson wrote: I upgraded to 0.94.1 from 0.94 on Monday, and everything had been going pretty

Re: [ceph-users] mds crashing

2015-04-15 Thread Adam Tygart
What is significantly smaller? We have 67 requests in the 16,400,000 range and 250 in the 18,900,000 range. Thanks, Adam On Wed, Apr 15, 2015 at 8:38 PM, Yan, Zheng uker...@gmail.com wrote: On Thu, Apr 16, 2015 at 9:07 AM, Adam Tygart mo...@ksu.edu wrote: We are using 3.18.6-gentoo. Based on

Re: [ceph-users] mds crashing

2015-04-15 Thread Adam Tygart
We are using 3.18.6-gentoo. Based on that, I was hoping that the kernel bug referred to in the bug report would have been fixed. -- Adam On Wed, Apr 15, 2015 at 8:02 PM, Yan, Zheng uker...@gmail.com wrote: On Thu, Apr 16, 2015 at 5:29 AM, Kyle Hutson kylehut...@ksu.edu wrote: Thank you, John!

Re: [ceph-users] mds crashing

2015-04-15 Thread Yan, Zheng
On Thu, Apr 16, 2015 at 9:48 AM, Adam Tygart mo...@ksu.edu wrote: What is significantly smaller? We have 67 requests in the 16,400,000 range and 250 in the 18,900,000 range. that explains the crash. could you help me to debug this issue. send /sys/kernel/debug/ceph/*/mdsc to me. run echo

Re: [ceph-users] Upgrade from Giant 0.87-1 to Hammer 0.94-1

2015-04-15 Thread Christian Balzer
Hello, On Thu, 16 Apr 2015 00:41:29 +0200 Steffen W Sørensen wrote: Hi, Successfully upgrade a small development 4x node Giant 0.87-1 cluster to Hammer 0.94-1, each node with 6x OSD - 146GB, 19 pools, mainly 2 in usage. Only minor thing now ceph -s complaining over too may PGs,

Re: [ceph-users] mds crashing

2015-04-15 Thread Yan, Zheng
On Thu, Apr 16, 2015 at 9:07 AM, Adam Tygart mo...@ksu.edu wrote: We are using 3.18.6-gentoo. Based on that, I was hoping that the kernel bug referred to in the bug report would have been fixed. The bug was supposed to be fixed, but you hit the bug again. could you check if the kernel client

Re: [ceph-users] live migration fails with image on ceph

2015-04-15 Thread Yuming Ma (yumima)
The issue is reproducible in svl-3 with rbd cache set to false. On the 5th ping-pong, the instance experienced ping drops and did not recover for 20+ minutes: (os-clients)[root@fedora21 nimbus-env]# nova live-migration lmtest1 (os-clients)[root@fedora21 nimbus-env]# nova show lmtest1 |grep -E

Re: [ceph-users] mds crashing

2015-04-15 Thread Yan, Zheng
On Thu, Apr 16, 2015 at 5:29 AM, Kyle Hutson kylehut...@ksu.edu wrote: Thank you, John! That was exactly the bug we were hitting. My Google-fu didn't lead me to this one. here is the bug report http://tracker.ceph.com/issues/10449. It's a kernel client bug which causes the session map size

[ceph-users] Managing larger ceph clusters

2015-04-15 Thread Stillwell, Bryan
I'm curious what people managing larger ceph clusters are doing with configuration management and orchestration to simplify their lives? We've been using ceph-deploy to manage our ceph clusters so far, but feel that moving the management of our clusters to standard tools would provide a little

Re: [ceph-users] mds crashing

2015-04-15 Thread John Spray
On 15/04/2015 20:02, Kyle Hutson wrote: I upgraded to 0.94.1 from 0.94 on Monday, and everything had been going pretty well. Then, about noon today, we had an mds crash. And then the failover mds crashed. And this cascaded through all 4 mds servers we have. If I try to start it ('service

[ceph-users] Is ceph.com down?

2015-04-15 Thread Lindsay Mathieson
Can't open at the moment, niever the website or apt. Trying from Brisbane, Australia. -- Lindsay ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] v0.80.8 and librbd performance

2015-04-15 Thread Josh Durgin
On 04/14/2015 08:01 PM, shiva rkreddy wrote: The clusters are in test environment, so its a new deployment of 0.80.9. OS on the cluster nodes is reinstalled as well, so there shouldn't be any fs aging unless the disks are slowing down. The perf measurement is done initiating multiple cinder

Re: [ceph-users] Is ceph.com down?

2015-04-15 Thread Wido den Hollander
On 04/15/2015 09:30 AM, Lindsay Mathieson wrote: Can't open at the moment, niever the website or apt. Yes, it's down here as well. You can try eu.ceph.com if you need the packages. Or this one: http://ceph.mirror.digitalpacific.com.au/ (working on au.ceph.com) Trying from Brisbane,

Re: [ceph-users] Ceph site is very slow

2015-04-15 Thread Wido den Hollander
On 04/15/2015 10:20 AM, Ignazio Cassano wrote: Hi all, why ceph.com is very slow ? Not known right now. But you can try eu.ceph.com for your packages and downloads. It is impossible download files for installing ceph. Regards Ignazio ___

Re: [ceph-users] Ceph site is very slow

2015-04-15 Thread Ignazio Cassano
Many thanks 2015-04-15 10:44 GMT+02:00 Wido den Hollander w...@42on.com: On 04/15/2015 10:20 AM, Ignazio Cassano wrote: Hi all, why ceph.com is very slow ? Not known right now. But you can try eu.ceph.com for your packages and downloads. It is impossible download files for installing

[ceph-users] Ceph site is very slow

2015-04-15 Thread Ignazio Cassano
Hi all, why ceph.com is very slow ? It is impossible download files for installing ceph. Regards Ignazio ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] how to compute Ceph durability?

2015-04-15 Thread ghislain.chevalier
Thanks Mark Loic also gave me this link It would be a good start for sure Best regards -Message d'origine- De : ceph-users [mailto:ceph-users-boun...@lists.ceph.com] De la part de Mark Nelson Envoyé : mardi 14 avril 2015 14:11 À : ceph-users@lists.ceph.com Objet : Re: [ceph-users] how

Re: [ceph-users] Rados Gateway and keystone

2015-04-15 Thread ghislain.chevalier
Thanks a lot That helps. De : Erik McCormick [mailto:emccorm...@cirrusseven.com] Envoyé : lundi 13 avril 2015 18:32 À : CHEVALIER Ghislain IMT/OLPS Cc : ceph-users Objet : Re: [ceph-users] Rados Gateway and keystone I haven't really used the S3 stuff much, but the credentials should be in