Re: [ceph-users] Giant or Firefly for production
On Sun, Dec 7, 2014 at 1:51 PM, René Gallati c...@gallati.net wrote: Hello Antonio, I use aptly to manage my repositories and mix and match (and snapshot / pin) I didn't know aptly, thank you for mentioning. specific versions and non-standard packages, but as far as I know, the kernel from utopic unicorn is already in the main repositories for trusty and is a 3.16 line. apt-cache policy linux-image-generic-lts-utopic should give you the information about availability in your repository. My information is: linux-image-generic-lts-utopic: Installed: (none) Candidate: 3.16.0.25.19 Version table: 3.16.0.25.19 0 500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages Note the lts in the name, these are officially supported although they don't specifically announce those kernels when they become available or I'm not on the correct mailing lists for that. Generally about one to two month after a new non-lts release, they will be there for the LTS version. I didn't know this either! That's useful, since it's a release kernel _and_ lts. Thank you René, cheers Antonio -- antonio.s.mess...@gmail.com antonio.mess...@uzh.ch +41 (0)44 635 42 22 S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
On Sun, Dec 7, 2014 at 1:51 PM, René Gallati c...@gallati.net wrote: Hello Antonio, I use aptly to manage my repositories and mix and match (and snapshot / pin) I didn't know aptly, thank you for mentioning. specific versions and non-standard packages, but as far as I know, the kernel from utopic unicorn is already in the main repositories for trusty and is a 3.16 line. apt-cache policy linux-image-generic-lts-utopic should give you the information about availability in your repository. My information is: linux-image-generic-lts-utopic: Installed: (none) Candidate: 3.16.0.25.19 Version table: 3.16.0.25.19 0 500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages Note the lts in the name, these are officially supported although they don't specifically announce those kernels when they become available or I'm not on the correct mailing lists for that. Generally about one to two month after a new non-lts release, they will be there for the LTS version. I didn't know this either! That's useful, since it's a release kernel _and_ lts. Thank you René, cheers Antonio -- antonio.s.mess...@gmail.com antonio.mess...@uzh.ch +41 (0)44 635 42 22 S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
On Fri, Dec 5, 2014 at 2:24 AM, Anthony Alba ascanio.al...@gmail.com wrote: Hi Cephers, Have anyone of you decided to put Giant into production instead of Firefly? This is very interesting to me too: we are going to deploy a large ceph cluster on Ubuntu 14.04 LTS, and so far what I have found is that the rbd module in Ubuntu Trusty doesn't seem compatible with giant: feature set mismatch, my 4a042a42 server's 2104a042a42, missing 210 I tried with different ceph osd tunables but nothing seems to fix the issue However, this cluster will be mainly used for OpenStack, and qemu is able to access the rbd volume, so this might not be a big problem for me. .a. -- antonio.s.mess...@gmail.com antonio.mess...@uzh.ch +41 (0)44 635 42 22 S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
On Fri, Dec 5, 2014 at 2:24 AM, Anthony Alba ascanio.al...@gmail.com wrote: Hi Cephers, Have anyone of you decided to put Giant into production instead of Firefly? This is very interesting to me too: we are going to deploy a large ceph cluster on Ubuntu 14.04 LTS, and so far what I have found is that the rbd module in Ubuntu Trusty doesn't seem compatible with giant: feature set mismatch, my 4a042a42 server's 2104a042a42, missing 210 I tried with different ceph osd tunables but nothing seems to fix the issue However, this cluster will be mainly used for OpenStack, and qemu is able to access the rbd volume, so this might not be a big problem for me. .a. -- antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22 antonio.s.mess...@gmail.com S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
This is probably due to the Kernel RBD client not being recent enough. Have you tried upgrading your kernel to a newer version? 3.16 should contain all the relevant features required by Giant. -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Antonio Messina Sent: 05 December 2014 09:37 To: Anthony Alba Cc: ceph-users@lists.ceph.com Subject: Re: [ceph-users] Giant or Firefly for production On Fri, Dec 5, 2014 at 2:24 AM, Anthony Alba ascanio.al...@gmail.com wrote: Hi Cephers, Have anyone of you decided to put Giant into production instead of Firefly? This is very interesting to me too: we are going to deploy a large ceph cluster on Ubuntu 14.04 LTS, and so far what I have found is that the rbd module in Ubuntu Trusty doesn't seem compatible with giant: feature set mismatch, my 4a042a42 server's 2104a042a42, missing 210 I tried with different ceph osd tunables but nothing seems to fix the issue However, this cluster will be mainly used for OpenStack, and qemu is able to access the rbd volume, so this might not be a big problem for me. .a. -- antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22 antonio.s.mess...@gmail.com S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
What are the kernel versions involved ? We have Ubuntu precise clients talking to a Ubuntu trusty cluster without issues - with tunables optimal. 0.88 (Giant) and 0.89 has been working well for us as far the client and Openstack are concerned. This link provides some insight as to the possible problems: http://cephnotes.ksperis.com/blog/2014/01/21/feature-set-mismatch-error-on-ceph-kernel-client Things to look for: - Kernel versions - Cache tiering - Tunables - hashpspool -- David Moreau Simard On Dec 5, 2014, at 4:36 AM, Antonio Messina antonio.mess...@s3it.uzh.ch wrote: On Fri, Dec 5, 2014 at 2:24 AM, Anthony Alba ascanio.al...@gmail.com wrote: Hi Cephers, Have anyone of you decided to put Giant into production instead of Firefly? This is very interesting to me too: we are going to deploy a large ceph cluster on Ubuntu 14.04 LTS, and so far what I have found is that the rbd module in Ubuntu Trusty doesn't seem compatible with giant: feature set mismatch, my 4a042a42 server's 2104a042a42, missing 210 I tried with different ceph osd tunables but nothing seems to fix the issue However, this cluster will be mainly used for OpenStack, and qemu is able to access the rbd volume, so this might not be a big problem for me. .a. -- antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22 antonio.s.mess...@gmail.com S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
On Fri, Dec 5, 2014 at 4:25 PM, David Moreau Simard dmsim...@iweb.com wrote: What are the kernel versions involved ? We have Ubuntu precise clients talking to a Ubuntu trusty cluster without issues - with tunables optimal. 0.88 (Giant) and 0.89 has been working well for us as far the client and Openstack are concerned. This link provides some insight as to the possible problems: Both servers and clients are Ubuntu Trusty. Kernel versions are a bit different: client: 3.13.0-39-generic #66 server: 3.13.0-32-generic #57 ceph version on both: 0.87 http://cephnotes.ksperis.com/blog/2014/01/21/feature-set-mismatch-error-on-ceph-kernel-client Things to look for: - Kernel versions - Cache tiering - Tunables - hashpspool I have already read the blogpost, but I don't have much experience with tunables. From what I understood I am missing: * CEPH_FEATURE_CRUSH_TUNABLES3 * CEPH_FEATURE_CRUSH_V2 but I don't know how to disable them, and I can't see them set in the crushmap I get from ceph osd getcrushmap .a. -- antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22 antonio.s.mess...@gmail.com S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
On Fri, Dec 5, 2014 at 4:25 PM, Nick Fisk n...@fisk.me.uk wrote: This is probably due to the Kernel RBD client not being recent enough. Have you tried upgrading your kernel to a newer version? 3.16 should contain all the relevant features required by Giant. I would rather tune the tunables, as upgrading the kernel would require a reboot of the client. Besides, Ubuntu Trusty does not provide a 3.16 kernel, so I would need to recompile... .a. -- antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22 antonio.s.mess...@gmail.com S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
http://kernel.ubuntu.com/~kernel-ppa/mainline/ I'm running 3.17 on my trusty clients without issue On Fri, Dec 5, 2014 at 9:37 AM, Antonio Messina antonio.mess...@s3it.uzh.ch wrote: On Fri, Dec 5, 2014 at 4:25 PM, Nick Fisk n...@fisk.me.uk wrote: This is probably due to the Kernel RBD client not being recent enough. Have you tried upgrading your kernel to a newer version? 3.16 should contain all the relevant features required by Giant. I would rather tune the tunables, as upgrading the kernel would require a reboot of the client. Besides, Ubuntu Trusty does not provide a 3.16 kernel, so I would need to recompile... .a. -- antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22 antonio.s.mess...@gmail.com S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
Ok sorry, I thought you had a need for some of the features in Giant, using tunables is probably easier in that case. However if you do want to upgrade there are debs available:- http://kernel.ubuntu.com/~kernel-ppa/mainline/ and I believe 3.16 should be available in the 14.04.2 release, which should be released early next year. Nick -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Antonio Messina Sent: 05 December 2014 15:38 To: Nick Fisk Cc: ceph-users@lists.ceph.com; Antonio Messina Subject: Re: [ceph-users] Giant or Firefly for production On Fri, Dec 5, 2014 at 4:25 PM, Nick Fisk n...@fisk.me.uk wrote: This is probably due to the Kernel RBD client not being recent enough. Have you tried upgrading your kernel to a newer version? 3.16 should contain all the relevant features required by Giant. I would rather tune the tunables, as upgrading the kernel would require a reboot of the client. Besides, Ubuntu Trusty does not provide a 3.16 kernel, so I would need to recompile... .a. -- antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22 antonio.s.mess...@gmail.com S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
Thank you James and Nick, On Fri, Dec 5, 2014 at 4:46 PM, Nick Fisk n...@fisk.me.uk wrote: Ok sorry, I thought you had a need for some of the features in Giant, using tunables is probably easier in that case. I'm not sure :) I never played with the tunables before (still running a testbed only) I will test it again with 14.04.2 and default kernel beginning of next year, I prefer to use the official kernel for the production cluster, but since it's going to be deployed Q1-Q2 next year I should be safe. .a. However if you do want to upgrade there are debs available:- http://kernel.ubuntu.com/~kernel-ppa/mainline/ and I believe 3.16 should be available in the 14.04.2 release, which should be released early next year. Nick -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Antonio Messina Sent: 05 December 2014 15:38 To: Nick Fisk Cc: ceph-users@lists.ceph.com; Antonio Messina Subject: Re: [ceph-users] Giant or Firefly for production On Fri, Dec 5, 2014 at 4:25 PM, Nick Fisk n...@fisk.me.uk wrote: This is probably due to the Kernel RBD client not being recent enough. Have you tried upgrading your kernel to a newer version? 3.16 should contain all the relevant features required by Giant. I would rather tune the tunables, as upgrading the kernel would require a reboot of the client. Besides, Ubuntu Trusty does not provide a 3.16 kernel, so I would need to recompile... .a. -- antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22 antonio.s.mess...@gmail.com S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22 antonio.s.mess...@gmail.com S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
On Fri, 5 Dec 2014, Antonio Messina wrote: On Fri, Dec 5, 2014 at 2:24 AM, Anthony Alba ascanio.al...@gmail.com wrote: Hi Cephers, Have anyone of you decided to put Giant into production instead of Firefly? This is very interesting to me too: we are going to deploy a large ceph cluster on Ubuntu 14.04 LTS, and so far what I have found is that the rbd module in Ubuntu Trusty doesn't seem compatible with giant: feature set mismatch, my 4a042a42 server's 2104a042a42, missing 210 Can you attach the output of ceph osd crush show-tunables -f json-pretty ceph osd crush dump -f json-pretty Thanks! sage ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
On Fri, Dec 5, 2014 at 4:59 PM, Sage Weil s...@newdream.net wrote:
On Fri, 5 Dec 2014, Antonio Messina wrote:
On Fri, Dec 5, 2014 at 2:24 AM, Anthony Alba ascanio.al...@gmail.com wrote:
Hi Cephers,
Have anyone of you decided to put Giant into production instead of Firefly?
This is very interesting to me too: we are going to deploy a large
ceph cluster on Ubuntu 14.04 LTS, and so far what I have found is that
the rbd module in Ubuntu Trusty doesn't seem compatible with giant:
feature set mismatch, my 4a042a42 server's 2104a042a42, missing
210
Can you attach the output of
I modified the crushmap and set:
tunable chooseleaf_vary_r 0
(it was 1 before)
Now the cluster is rebalancing, and since it's on crappy hardware is
taking some time.
I'm pasting the output of the two commands, but please keep in mind
that this is the output *after* I've updated the chooseleaf_vary_r
tunable.
ceph osd crush show-tunables -f json-pretty
{ choose_local_tries: 0,
choose_local_fallback_tries: 0,
choose_total_tries: 50,
chooseleaf_descend_once: 1,
profile: bobtail,
optimal_tunables: 0,
legacy_tunables: 0,
require_feature_tunables: 1,
require_feature_tunables2: 1,
require_feature_tunables3: 0,
has_v2_rules: 1,
has_v3_rules: 0}
ceph osd crush dump -f json-pretty
I'm attaching it as a text file, as it is quite big and unreadable.
However, from the output I see the following tunables:
tunables: { choose_local_tries: 0,
choose_local_fallback_tries: 0,
choose_total_tries: 50,
chooseleaf_descend_once: 1,
profile: bobtail,
optimal_tunables: 0,
legacy_tunables: 0,
require_feature_tunables: 1,
require_feature_tunables2: 1,
require_feature_tunables3: 0,
has_v2_rules: 1,
has_v3_rules: 0}}
.a.
--
antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22
antonio.s.mess...@gmail.com
S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/
University of Zurich
Winterthurerstrasse 190
CH-8057 Zurich Switzerland
crushmap.json
Description: application/json
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
Hi all, just an update
After setting chooseleaf_vary_r to 0 _and_ removing an pool with
erasure coding, I was able to run rbd map.
Thank you all for the help
.a.
On Fri, Dec 5, 2014 at 5:07 PM, Antonio Messina
antonio.mess...@s3it.uzh.ch wrote:
On Fri, Dec 5, 2014 at 4:59 PM, Sage Weil s...@newdream.net wrote:
On Fri, 5 Dec 2014, Antonio Messina wrote:
On Fri, Dec 5, 2014 at 2:24 AM, Anthony Alba ascanio.al...@gmail.com
wrote:
Hi Cephers,
Have anyone of you decided to put Giant into production instead of
Firefly?
This is very interesting to me too: we are going to deploy a large
ceph cluster on Ubuntu 14.04 LTS, and so far what I have found is that
the rbd module in Ubuntu Trusty doesn't seem compatible with giant:
feature set mismatch, my 4a042a42 server's 2104a042a42, missing
210
Can you attach the output of
I modified the crushmap and set:
tunable chooseleaf_vary_r 0
(it was 1 before)
Now the cluster is rebalancing, and since it's on crappy hardware is
taking some time.
I'm pasting the output of the two commands, but please keep in mind
that this is the output *after* I've updated the chooseleaf_vary_r
tunable.
ceph osd crush show-tunables -f json-pretty
{ choose_local_tries: 0,
choose_local_fallback_tries: 0,
choose_total_tries: 50,
chooseleaf_descend_once: 1,
profile: bobtail,
optimal_tunables: 0,
legacy_tunables: 0,
require_feature_tunables: 1,
require_feature_tunables2: 1,
require_feature_tunables3: 0,
has_v2_rules: 1,
has_v3_rules: 0}
ceph osd crush dump -f json-pretty
I'm attaching it as a text file, as it is quite big and unreadable.
However, from the output I see the following tunables:
tunables: { choose_local_tries: 0,
choose_local_fallback_tries: 0,
choose_total_tries: 50,
chooseleaf_descend_once: 1,
profile: bobtail,
optimal_tunables: 0,
legacy_tunables: 0,
require_feature_tunables: 1,
require_feature_tunables2: 1,
require_feature_tunables3: 0,
has_v2_rules: 1,
has_v3_rules: 0}}
.a.
--
antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22
antonio.s.mess...@gmail.com
S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/
University of Zurich
Winterthurerstrasse 190
CH-8057 Zurich Switzerland
--
antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22
antonio.s.mess...@gmail.com
S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/
University of Zurich
Winterthurerstrasse 190
CH-8057 Zurich Switzerland
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
On Fri, 5 Dec 2014, Antonio Messina wrote:
On Fri, Dec 5, 2014 at 4:59 PM, Sage Weil s...@newdream.net wrote:
On Fri, 5 Dec 2014, Antonio Messina wrote:
On Fri, Dec 5, 2014 at 2:24 AM, Anthony Alba ascanio.al...@gmail.com
wrote:
Hi Cephers,
Have anyone of you decided to put Giant into production instead of
Firefly?
This is very interesting to me too: we are going to deploy a large
ceph cluster on Ubuntu 14.04 LTS, and so far what I have found is that
the rbd module in Ubuntu Trusty doesn't seem compatible with giant:
feature set mismatch, my 4a042a42 server's 2104a042a42, missing
210
Can you attach the output of
I modified the crushmap and set:
tunable chooseleaf_vary_r 0
(it was 1 before)
Now the cluster is rebalancing, and since it's on crappy hardware is
taking some time.
I'm pasting the output of the two commands, but please keep in mind
that this is the output *after* I've updated the chooseleaf_vary_r
tunable.
ceph osd crush show-tunables -f json-pretty
{ choose_local_tries: 0,
choose_local_fallback_tries: 0,
choose_total_tries: 50,
chooseleaf_descend_once: 1,
profile: bobtail,
optimal_tunables: 0,
legacy_tunables: 0,
require_feature_tunables: 1,
require_feature_tunables2: 1,
require_feature_tunables3: 0,
has_v2_rules: 1,
has_v3_rules: 0}
The v2 rule means you have a crush rule for erasure coding. Do you have
an EC pool in your cluster?
The tunables3 feature bit is set because you set the vary_r parameter.
If you want older kernels to talk to the cluster, you need to avoid the
new tunables and features!
sage
ceph osd crush dump -f json-pretty
I'm attaching it as a text file, as it is quite big and unreadable.
However, from the output I see the following tunables:
tunables: { choose_local_tries: 0,
choose_local_fallback_tries: 0,
choose_total_tries: 50,
chooseleaf_descend_once: 1,
profile: bobtail,
optimal_tunables: 0,
legacy_tunables: 0,
require_feature_tunables: 1,
require_feature_tunables2: 1,
require_feature_tunables3: 0,
has_v2_rules: 1,
has_v3_rules: 0}}
.a.
--
antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22
antonio.s.mess...@gmail.com
S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/
University of Zurich
Winterthurerstrasse 190
CH-8057 Zurich Switzerland
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Giant or Firefly for production
On Fri, Dec 5, 2014 at 5:24 PM, Sage Weil s...@newdream.net wrote: The v2 rule means you have a crush rule for erasure coding. Do you have an EC pool in your cluster? Yes indeed. I didn't know EC pool was incompatible with the current kernel, I only tested it with rados bench and VMs, I guess. The tunables3 feature bit is set because you set the vary_r parameter. This I don't really know where it comes from. I think at a certain point I ran ceph osd crush tunables optimal, and it probably added vary_r, but then I run ceph osd crush tunables firefly and it didn't remove it... is it normal? If you want older kernels to talk to the cluster, you need to avoid the new tunables and features! Well, as I said, I'm not a ceph expert, I didn't even know I enabled features the kernel of the distribution did not support. I guess the problem is that I am using packages from the ceph.com repo, while the kernel comes from ubuntu. However, it's at least curious that when I was running firefly from ubuntu repositories I could create an EC pool, but the kernel was not compatible with EC2 pools... .a. -- antonio.mess...@s3it.uzh.ch +41 (0)44 635 42 22 antonio.s.mess...@gmail.com S3IT: Service and Support for Science IT http://www.s3it.uzh.ch/ University of Zurich Winterthurerstrasse 190 CH-8057 Zurich Switzerland ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
