Re: [ceph-users] Kernel version recommendation

[Bogdan SOLGA]

Thank you for the provided information and considerations, David! Our
experience with Ubuntu (so far) adheres to your opinion.

Nevertheless, we will try CentOS, on the medium / long term. Given the
overall reliability of our current setup, it will have to provide
significant benefits in order to switch to it, at least for the Ceph
infrastructure.

Any other thoughts / perspectives are highly welcomed.

My best,
Bogdan


On Sat, Oct 28, 2017 at 8:26 PM, David Turner <drakonst...@gmail.com> wrote:

> All of that said, the most important factor is in regards to your most
> expensive asset... personnel. If you have a dozen sysadmins and you're
> maintaining a few platforms each using their own OS, then your company is
> wasting a lot of money in your admins dealing with that extra intellectual
> cost of needing to know all of them or to retrain them.
>
> If Ubuntu wasn't stable and secure, it wouldn't be popular. It may not be
> the most stable or secure, but it sure does get new features faster.
>
> On Sat, Oct 28, 2017, 1:01 PM David Turner <drakonst...@gmail.com> wrote:
>
>> Saying Ubuntu doesn't have a place on servers negates your assertion that
>> the OS is a tool and you should use the right tool for the right job.
>> Sometimes you need an OS that updates its kernel more often than basically
>> never. Back when VMs were gaining traction and CentOS 6 was running the 2.6
>> kernel. That kernel version had zero features that were designed for a
>> virtual deployment. Yes CentOS is incredibly stable and backports all of
>> the security patches, but for VM performance CentOS 6 was outpaced by
>> Ubuntu as a virtual server by a 4x performance increase as a webserver
>> during our testing.
>>
>> For ceph, it was originally developed solely for Ubuntu and major work
>> into compatibility with CentOS and RHEL didn't start until Redhat purchased
>> ceph. That compatibility took some time and is pretty much fully there now,
>> but back in Firefly and earlier running ceph on centos was a pita.
>>
>> Back to the point of centos vs Ubuntu... Why limit it there? On the
>> previous mentioned testing I had done with virtual webservers way back
>> when, we had centos responding in 50ms, Ubuntu in 10ms, and a properly
>> configured Gentoo server in 5ms. Please don't be an OS bigot while touting
>> that you should learn the proper tool. Redhat is not the only distribution
>> with a large support structure.
>>
>> The OS is a tool, but you should actually figure out and use the proper
>> tool for your job. If stability is by far the most important factor, then
>> centos/rhel is likely a great choice for you.
>>
>> On Sat, Oct 28, 2017, 9:21 AM Bogdan SOLGA <bogdan.so...@gmail.com>
>> wrote:
>>
>>> Thanks for the indirect recommendation; we'll ponder on the switch to
>>> CentOS, on the medium / long run.
>>>
>>> Until then, we're looking forward for best practices and advice from the
>>> Ceph community, on running the cluster on Ubuntu 16.
>>>
>>> Regards,
>>> Bogdan
>>>
>>>
>>> On Sat, Oct 28, 2017 at 2:46 PM, Marc Roos <m.r...@f1-outsourcing.eu>
>>> wrote:
>>>
>>>>
>>>> I hope I can post here a general question/comment regarding
>>>> distributions. Because I see a lot of stability issues passing by here.
>>>>
>>>> Why are people choosing an ubuntu distribution to run in production?
>>>> Mostly I get an answer like they are accustomed to using it. But is the
>>>> OS not just a tool? And you have to choose the correct tool for the job
>>>> (and then learn to use it)?
>>>> When I chose centos it was because it is close related to redhat, and
>>>> for critical situations I would choose or switch to the rhel license.
>>>> There over 10k people working at redhat to produce a stable os. I am
>>>> very pleased with the level of knowledge here and what redhat is doing
>>>> general.
>>>>
>>>> I just have to finish with; You people working on Ceph are doing a great
>>>> job and are working on a great project!
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -Original Message-
>>>> From: Bogdan SOLGA [mailto:bogdan.so...@gmail.com]
>>>> Sent: vrijdag 27 oktober 2017 18:33
>>>> To: ceph-users
>>>> Cc: Stephen Oliver; Ákos Nagy
>>>> Subject: [ceph-users] Kernel version recommendation
>>>>
>>>> Hello, everyone!
>>>>

Re: [ceph-users] Kernel version recommendation

[David Turner]

All of that said, the most important factor is in regards to your most
expensive asset... personnel. If you have a dozen sysadmins and you're
maintaining a few platforms each using their own OS, then your company is
wasting a lot of money in your admins dealing with that extra intellectual
cost of needing to know all of them or to retrain them.

If Ubuntu wasn't stable and secure, it wouldn't be popular. It may not be
the most stable or secure, but it sure does get new features faster.

On Sat, Oct 28, 2017, 1:01 PM David Turner <drakonst...@gmail.com> wrote:

> Saying Ubuntu doesn't have a place on servers negates your assertion that
> the OS is a tool and you should use the right tool for the right job.
> Sometimes you need an OS that updates its kernel more often than basically
> never. Back when VMs were gaining traction and CentOS 6 was running the 2.6
> kernel. That kernel version had zero features that were designed for a
> virtual deployment. Yes CentOS is incredibly stable and backports all of
> the security patches, but for VM performance CentOS 6 was outpaced by
> Ubuntu as a virtual server by a 4x performance increase as a webserver
> during our testing.
>
> For ceph, it was originally developed solely for Ubuntu and major work
> into compatibility with CentOS and RHEL didn't start until Redhat purchased
> ceph. That compatibility took some time and is pretty much fully there now,
> but back in Firefly and earlier running ceph on centos was a pita.
>
> Back to the point of centos vs Ubuntu... Why limit it there? On the
> previous mentioned testing I had done with virtual webservers way back
> when, we had centos responding in 50ms, Ubuntu in 10ms, and a properly
> configured Gentoo server in 5ms. Please don't be an OS bigot while touting
> that you should learn the proper tool. Redhat is not the only distribution
> with a large support structure.
>
> The OS is a tool, but you should actually figure out and use the proper
> tool for your job. If stability is by far the most important factor, then
> centos/rhel is likely a great choice for you.
>
> On Sat, Oct 28, 2017, 9:21 AM Bogdan SOLGA <bogdan.so...@gmail.com> wrote:
>
>> Thanks for the indirect recommendation; we'll ponder on the switch to
>> CentOS, on the medium / long run.
>>
>> Until then, we're looking forward for best practices and advice from the
>> Ceph community, on running the cluster on Ubuntu 16.
>>
>> Regards,
>> Bogdan
>>
>>
>> On Sat, Oct 28, 2017 at 2:46 PM, Marc Roos <m.r...@f1-outsourcing.eu>
>> wrote:
>>
>>>
>>> I hope I can post here a general question/comment regarding
>>> distributions. Because I see a lot of stability issues passing by here.
>>>
>>> Why are people choosing an ubuntu distribution to run in production?
>>> Mostly I get an answer like they are accustomed to using it. But is the
>>> OS not just a tool? And you have to choose the correct tool for the job
>>> (and then learn to use it)?
>>> When I chose centos it was because it is close related to redhat, and
>>> for critical situations I would choose or switch to the rhel license.
>>> There over 10k people working at redhat to produce a stable os. I am
>>> very pleased with the level of knowledge here and what redhat is doing
>>> general.
>>>
>>> I just have to finish with; You people working on Ceph are doing a great
>>> job and are working on a great project!
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> -Original Message-
>>> From: Bogdan SOLGA [mailto:bogdan.so...@gmail.com]
>>> Sent: vrijdag 27 oktober 2017 18:33
>>> To: ceph-users
>>> Cc: Stephen Oliver; Ákos Nagy
>>> Subject: [ceph-users] Kernel version recommendation
>>>
>>> Hello, everyone!
>>>
>>>
>>> We have recently upgraded our Ceph pool to the latest Luminous release.
>>> On one of the servers that we used as Ceph clients we had several freeze
>>> issues, which we empirically linked to the concurrent usage of some I/O
>>> operations - writing in an LXD container (backed by Ceph) while there
>>> was an ongoing PG rebalancing. We searched for the issue's cause through
>>> the logs, but we haven't found anything useful.
>>>
>>>
>>> At that time the server was running Ubuntu 16 with a 4.5 kernel. We
>>> thought an upgrade to the latest HWE kernel (4.10) would help, but we
>>> had the same freezing issues after the kernel upgrade. Of course, we're
>>> aware that we have tried to fix / avoid the issue without understanding

Re: [ceph-users] Kernel version recommendation

[David Turner]

Saying Ubuntu doesn't have a place on servers negates your assertion that
the OS is a tool and you should use the right tool for the right job.
Sometimes you need an OS that updates its kernel more often than basically
never. Back when VMs were gaining traction and CentOS 6 was running the 2.6
kernel. That kernel version had zero features that were designed for a
virtual deployment. Yes CentOS is incredibly stable and backports all of
the security patches, but for VM performance CentOS 6 was outpaced by
Ubuntu as a virtual server by a 4x performance increase as a webserver
during our testing.

For ceph, it was originally developed solely for Ubuntu and major work into
compatibility with CentOS and RHEL didn't start until Redhat purchased
ceph. That compatibility took some time and is pretty much fully there now,
but back in Firefly and earlier running ceph on centos was a pita.

Back to the point of centos vs Ubuntu... Why limit it there? On the
previous mentioned testing I had done with virtual webservers way back
when, we had centos responding in 50ms, Ubuntu in 10ms, and a properly
configured Gentoo server in 5ms. Please don't be an OS bigot while touting
that you should learn the proper tool. Redhat is not the only distribution
with a large support structure.

The OS is a tool, but you should actually figure out and use the proper
tool for your job. If stability is by far the most important factor, then
centos/rhel is likely a great choice for you.

On Sat, Oct 28, 2017, 9:21 AM Bogdan SOLGA <bogdan.so...@gmail.com> wrote:

> Thanks for the indirect recommendation; we'll ponder on the switch to
> CentOS, on the medium / long run.
>
> Until then, we're looking forward for best practices and advice from the
> Ceph community, on running the cluster on Ubuntu 16.
>
> Regards,
> Bogdan
>
>
> On Sat, Oct 28, 2017 at 2:46 PM, Marc Roos <m.r...@f1-outsourcing.eu>
> wrote:
>
>>
>> I hope I can post here a general question/comment regarding
>> distributions. Because I see a lot of stability issues passing by here.
>>
>> Why are people choosing an ubuntu distribution to run in production?
>> Mostly I get an answer like they are accustomed to using it. But is the
>> OS not just a tool? And you have to choose the correct tool for the job
>> (and then learn to use it)?
>> When I chose centos it was because it is close related to redhat, and
>> for critical situations I would choose or switch to the rhel license.
>> There over 10k people working at redhat to produce a stable os. I am
>> very pleased with the level of knowledge here and what redhat is doing
>> general.
>>
>> I just have to finish with; You people working on Ceph are doing a great
>> job and are working on a great project!
>>
>>
>>
>>
>>
>>
>>
>> -Original Message-
>> From: Bogdan SOLGA [mailto:bogdan.so...@gmail.com]
>> Sent: vrijdag 27 oktober 2017 18:33
>> To: ceph-users
>> Cc: Stephen Oliver; Ákos Nagy
>> Subject: [ceph-users] Kernel version recommendation
>>
>> Hello, everyone!
>>
>>
>> We have recently upgraded our Ceph pool to the latest Luminous release.
>> On one of the servers that we used as Ceph clients we had several freeze
>> issues, which we empirically linked to the concurrent usage of some I/O
>> operations - writing in an LXD container (backed by Ceph) while there
>> was an ongoing PG rebalancing. We searched for the issue's cause through
>> the logs, but we haven't found anything useful.
>>
>>
>> At that time the server was running Ubuntu 16 with a 4.5 kernel. We
>> thought an upgrade to the latest HWE kernel (4.10) would help, but we
>> had the same freezing issues after the kernel upgrade. Of course, we're
>> aware that we have tried to fix / avoid the issue without understanding
>> it's cause.
>>
>> After seeing the OS recommendations from the Ceph page
>> <
>> https://github.com/ceph/ceph/blob/master/doc/start/os-recommendations.rst
>> >
>> , we reinstalled the server (and got the 4.4 kernel), we ran into a
>> feature set mismatch issue when mounting a RBD image. We concluded
>> <
>> http://cephnotes.ksperis.com/blog/2014/01/21/feature-set-mismatch-error-on-ceph-kernel-client
>> >
>> that the feature set requires a kernel > 4.5.
>>
>>
>> Our question - how would you recommend us to proceed? Shall we
>> re-upgrade to the HWE kernel (4.10) or to another kernel version? Would
>> you recommend an alternative solution?
>>
>>
>> Thank you very much, we're looking forward for your advice.
>>
>>
>> Kind regards,
>>
>> Bogdan
>>
>>
>>
>>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Kernel version recommendation

[Bogdan SOLGA]

Thanks for the indirect recommendation; we'll ponder on the switch to
CentOS, on the medium / long run.

Until then, we're looking forward for best practices and advice from the
Ceph community, on running the cluster on Ubuntu 16.

Regards,
Bogdan


On Sat, Oct 28, 2017 at 2:46 PM, Marc Roos <m.r...@f1-outsourcing.eu> wrote:

>
> I hope I can post here a general question/comment regarding
> distributions. Because I see a lot of stability issues passing by here.
>
> Why are people choosing an ubuntu distribution to run in production?
> Mostly I get an answer like they are accustomed to using it. But is the
> OS not just a tool? And you have to choose the correct tool for the job
> (and then learn to use it)?
> When I chose centos it was because it is close related to redhat, and
> for critical situations I would choose or switch to the rhel license.
> There over 10k people working at redhat to produce a stable os. I am
> very pleased with the level of knowledge here and what redhat is doing
> general.
>
> I just have to finish with; You people working on Ceph are doing a great
> job and are working on a great project!
>
>
>
>
>
>
>
> -Original Message-
> From: Bogdan SOLGA [mailto:bogdan.so...@gmail.com]
> Sent: vrijdag 27 oktober 2017 18:33
> To: ceph-users
> Cc: Stephen Oliver; Ákos Nagy
> Subject: [ceph-users] Kernel version recommendation
>
> Hello, everyone!
>
>
> We have recently upgraded our Ceph pool to the latest Luminous release.
> On one of the servers that we used as Ceph clients we had several freeze
> issues, which we empirically linked to the concurrent usage of some I/O
> operations - writing in an LXD container (backed by Ceph) while there
> was an ongoing PG rebalancing. We searched for the issue's cause through
> the logs, but we haven't found anything useful.
>
>
> At that time the server was running Ubuntu 16 with a 4.5 kernel. We
> thought an upgrade to the latest HWE kernel (4.10) would help, but we
> had the same freezing issues after the kernel upgrade. Of course, we're
> aware that we have tried to fix / avoid the issue without understanding
> it's cause.
>
> After seeing the OS recommendations from the Ceph page
> <https://github.com/ceph/ceph/blob/master/doc/start/os-recommendations.rst
> >
> , we reinstalled the server (and got the 4.4 kernel), we ran into a
> feature set mismatch issue when mounting a RBD image. We concluded
> <http://cephnotes.ksperis.com/blog/2014/01/21/feature-set-
> mismatch-error-on-ceph-kernel-client>
> that the feature set requires a kernel > 4.5.
>
>
> Our question - how would you recommend us to proceed? Shall we
> re-upgrade to the HWE kernel (4.10) or to another kernel version? Would
> you recommend an alternative solution?
>
>
> Thank you very much, we're looking forward for your advice.
>
>
> Kind regards,
>
> Bogdan
>
>
>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Kernel version recommendation

[Marc Roos]

 
I hope I can post here a general question/comment regarding 
distributions. Because I see a lot of stability issues passing by here. 

Why are people choosing an ubuntu distribution to run in production? 
Mostly I get an answer like they are accustomed to using it. But is the 
OS not just a tool? And you have to choose the correct tool for the job 
(and then learn to use it)? 
When I chose centos it was because it is close related to redhat, and 
for critical situations I would choose or switch to the rhel license. 
There over 10k people working at redhat to produce a stable os. I am 
very pleased with the level of knowledge here and what redhat is doing 
general.

I just have to finish with; You people working on Ceph are doing a great 
job and are working on a great project!







-Original Message-
From: Bogdan SOLGA [mailto:bogdan.so...@gmail.com] 
Sent: vrijdag 27 oktober 2017 18:33
To: ceph-users
Cc: Stephen Oliver; Ákos Nagy
Subject: [ceph-users] Kernel version recommendation

Hello, everyone!


We have recently upgraded our Ceph pool to the latest Luminous release. 
On one of the servers that we used as Ceph clients we had several freeze 
issues, which we empirically linked to the concurrent usage of some I/O 
operations - writing in an LXD container (backed by Ceph) while there 
was an ongoing PG rebalancing. We searched for the issue's cause through 
the logs, but we haven't found anything useful.


At that time the server was running Ubuntu 16 with a 4.5 kernel. We 
thought an upgrade to the latest HWE kernel (4.10) would help, but we 
had the same freezing issues after the kernel upgrade. Of course, we're 
aware that we have tried to fix / avoid the issue without understanding 
it's cause.

After seeing the OS recommendations from the Ceph page 
<https://github.com/ceph/ceph/blob/master/doc/start/os-recommendations.rst> 
, we reinstalled the server (and got the 4.4 kernel), we ran into a 
feature set mismatch issue when mounting a RBD image. We concluded 
<http://cephnotes.ksperis.com/blog/2014/01/21/feature-set-mismatch-error-on-ceph-kernel-client>
  
that the feature set requires a kernel > 4.5.


Our question - how would you recommend us to proceed? Shall we 
re-upgrade to the HWE kernel (4.10) or to another kernel version? Would 
you recommend an alternative solution?


Thank you very much, we're looking forward for your advice.


Kind regards,

Bogdan



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Kernel version recommendation

[David Turner]

If you can do an ssh session to the IPMI console and then do that inside of
a screen, you can save the output of the screen to a file and look at what
was happening on the console when the server locked up.  That's how I track
kernel panics.

On Fri, Oct 27, 2017 at 1:53 PM Bogdan SOLGA  wrote:

> Thank you very much for the reply, Ilya!
>
> The server was completely frozen / hard lockup, we had to restart it via
> IPMI. We grepped the logs trying to find the culprit, but to no avail.
> Any hint on how to troubleshoot the (eventual) freezes is highly
> appreciated.
>
> Understood on the kernel recommendation. We'll continue to use 4.10, then.
>
> Thanks, a lot!
>
> Kind regards,
> Bogdan
>
>
>
>
> On Fri, Oct 27, 2017 at 8:04 PM, Ilya Dryomov  wrote:
>
>> On Fri, Oct 27, 2017 at 6:33 PM, Bogdan SOLGA 
>> wrote:
>> > Hello, everyone!
>> >
>> > We have recently upgraded our Ceph pool to the latest Luminous release.
>> On
>> > one of the servers that we used as Ceph clients we had several freeze
>> > issues, which we empirically linked to the concurrent usage of some I/O
>> > operations - writing in an LXD container (backed by Ceph) while there
>> was an
>> > ongoing PG rebalancing. We searched for the issue's cause through the
>> logs,
>> > but we haven't found anything useful.
>>
>> What kind of freezes -- temporary slowdowns or hard lockups?  Did they
>> resolve on their own or did you have to intervene?
>>
>> >
>> > At that time the server was running Ubuntu 16 with a 4.5 kernel. We
>> thought
>> > an upgrade to the latest HWE kernel (4.10) would help, but we had the
>> same
>> > freezing issues after the kernel upgrade. Of course, we're aware that we
>> > have tried to fix / avoid the issue without understanding it's cause.
>> >
>> > After seeing the OS recommendations from the Ceph page, we reinstalled
>> the
>> > server (and got the 4.4 kernel), we ran into a feature set mismatch
>> issue
>> > when mounting a RBD image. We concluded that the feature set requires a
>> > kernel > 4.5.
>> >
>> > Our question - how would you recommend us to proceed? Shall we
>> re-upgrade to
>> > the HWE kernel (4.10) or to another kernel version? Would you recommend
>> an
>> > alternative solution?
>>
>> The OS recommendations page lists upstream kernels, as a general
>> guidance.  As long as the kernel is fairly recent and maintained
>> (either upstream or by the distributor), it should be fine.  4.10 is
>> certainly better than 4.4-based kernels, at least for the kernel
>> client.
>>
>> Thanks,
>>
>> Ilya
>>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Kernel version recommendation

[Bogdan SOLGA]

Thank you very much for the reply, Ilya!

The server was completely frozen / hard lockup, we had to restart it via
IPMI. We grepped the logs trying to find the culprit, but to no avail.
Any hint on how to troubleshoot the (eventual) freezes is highly
appreciated.

Understood on the kernel recommendation. We'll continue to use 4.10, then.

Thanks, a lot!

Kind regards,
Bogdan



On Fri, Oct 27, 2017 at 8:04 PM, Ilya Dryomov  wrote:

> On Fri, Oct 27, 2017 at 6:33 PM, Bogdan SOLGA 
> wrote:
> > Hello, everyone!
> >
> > We have recently upgraded our Ceph pool to the latest Luminous release.
> On
> > one of the servers that we used as Ceph clients we had several freeze
> > issues, which we empirically linked to the concurrent usage of some I/O
> > operations - writing in an LXD container (backed by Ceph) while there
> was an
> > ongoing PG rebalancing. We searched for the issue's cause through the
> logs,
> > but we haven't found anything useful.
>
> What kind of freezes -- temporary slowdowns or hard lockups?  Did they
> resolve on their own or did you have to intervene?
>
> >
> > At that time the server was running Ubuntu 16 with a 4.5 kernel. We
> thought
> > an upgrade to the latest HWE kernel (4.10) would help, but we had the
> same
> > freezing issues after the kernel upgrade. Of course, we're aware that we
> > have tried to fix / avoid the issue without understanding it's cause.
> >
> > After seeing the OS recommendations from the Ceph page, we reinstalled
> the
> > server (and got the 4.4 kernel), we ran into a feature set mismatch issue
> > when mounting a RBD image. We concluded that the feature set requires a
> > kernel > 4.5.
> >
> > Our question - how would you recommend us to proceed? Shall we
> re-upgrade to
> > the HWE kernel (4.10) or to another kernel version? Would you recommend
> an
> > alternative solution?
>
> The OS recommendations page lists upstream kernels, as a general
> guidance.  As long as the kernel is fairly recent and maintained
> (either upstream or by the distributor), it should be fine.  4.10 is
> certainly better than 4.4-based kernels, at least for the kernel
> client.
>
> Thanks,
>
> Ilya
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Kernel version recommendation

[Ilya Dryomov]

On Fri, Oct 27, 2017 at 6:33 PM, Bogdan SOLGA  wrote:
> Hello, everyone!
>
> We have recently upgraded our Ceph pool to the latest Luminous release. On
> one of the servers that we used as Ceph clients we had several freeze
> issues, which we empirically linked to the concurrent usage of some I/O
> operations - writing in an LXD container (backed by Ceph) while there was an
> ongoing PG rebalancing. We searched for the issue's cause through the logs,
> but we haven't found anything useful.

What kind of freezes -- temporary slowdowns or hard lockups?  Did they
resolve on their own or did you have to intervene?

>
> At that time the server was running Ubuntu 16 with a 4.5 kernel. We thought
> an upgrade to the latest HWE kernel (4.10) would help, but we had the same
> freezing issues after the kernel upgrade. Of course, we're aware that we
> have tried to fix / avoid the issue without understanding it's cause.
>
> After seeing the OS recommendations from the Ceph page, we reinstalled the
> server (and got the 4.4 kernel), we ran into a feature set mismatch issue
> when mounting a RBD image. We concluded that the feature set requires a
> kernel > 4.5.
>
> Our question - how would you recommend us to proceed? Shall we re-upgrade to
> the HWE kernel (4.10) or to another kernel version? Would you recommend an
> alternative solution?

The OS recommendations page lists upstream kernels, as a general
guidance.  As long as the kernel is fairly recent and maintained
(either upstream or by the distributor), it should be fine.  4.10 is
certainly better than 4.4-based kernels, at least for the kernel
client.

Thanks,

Ilya
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Kernel version recommendation

[Bogdan SOLGA]

Thanks a lot for the recommendation, David!

Are you aware of any drawbacks and / or known issues with using rbd-nbd?

On Fri, Oct 27, 2017 at 7:47 PM, David Turner  wrote:

> rbd-nbd is gaining a lot of followers for use as mapping rbds.  The kernel
> driver for RBD's has taken a while to support features of current ceph
> versions.  The nice thing with rbd-nbd is that it has feature parity with
> the version of ceph you are using and can enable all of the rbd features
> you want to.
>
> When I do use the kernel driver, I usually find the kernel I want and then
> disable RBD features until the RBD is compatible to be mapped by that
> kernel.
>
> On Fri, Oct 27, 2017 at 12:34 PM Bogdan SOLGA 
> wrote:
>
>> Hello, everyone!
>>
>> We have recently upgraded our Ceph pool to the latest Luminous release.
>> On one of the servers that we used as Ceph clients we had several freeze
>> issues, which we empirically linked to the concurrent usage of some I/O
>> operations - writing in an LXD container (backed by Ceph) while there was
>> an ongoing PG rebalancing. We searched for the issue's cause through the
>> logs, but we haven't found anything useful.
>>
>> At that time the server was running Ubuntu 16 with a 4.5 kernel. We
>> thought an upgrade to the latest HWE kernel (4.10) would help, but we had
>> the same freezing issues after the kernel upgrade. Of course, we're aware
>> that we have tried to fix / avoid the issue without understanding it's
>> cause.
>>
>> After seeing the OS recommendations from the Ceph page
>> ,
>> we reinstalled the server (and got the 4.4 kernel), we ran into a feature
>> set mismatch issue when mounting a RBD image. We concluded
>> 
>> that the feature set requires a kernel > 4.5.
>>
>> Our question - how would you recommend us to proceed? Shall we re-upgrade
>> to the HWE kernel (4.10) or to another kernel version? Would you recommend
>> an alternative solution?
>>
>> Thank you very much, we're looking forward for your advice.
>>
>> Kind regards,
>> Bogdan
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Kernel version recommendation

[David Turner]

rbd-nbd is gaining a lot of followers for use as mapping rbds.  The kernel
driver for RBD's has taken a while to support features of current ceph
versions.  The nice thing with rbd-nbd is that it has feature parity with
the version of ceph you are using and can enable all of the rbd features
you want to.

When I do use the kernel driver, I usually find the kernel I want and then
disable RBD features until the RBD is compatible to be mapped by that
kernel.

On Fri, Oct 27, 2017 at 12:34 PM Bogdan SOLGA 
wrote:

> Hello, everyone!
>
> We have recently upgraded our Ceph pool to the latest Luminous release. On
> one of the servers that we used as Ceph clients we had several freeze
> issues, which we empirically linked to the concurrent usage of some I/O
> operations - writing in an LXD container (backed by Ceph) while there was
> an ongoing PG rebalancing. We searched for the issue's cause through the
> logs, but we haven't found anything useful.
>
> At that time the server was running Ubuntu 16 with a 4.5 kernel. We
> thought an upgrade to the latest HWE kernel (4.10) would help, but we had
> the same freezing issues after the kernel upgrade. Of course, we're aware
> that we have tried to fix / avoid the issue without understanding it's
> cause.
>
> After seeing the OS recommendations from the Ceph page
> ,
> we reinstalled the server (and got the 4.4 kernel), we ran into a feature
> set mismatch issue when mounting a RBD image. We concluded
> 
> that the feature set requires a kernel > 4.5.
>
> Our question - how would you recommend us to proceed? Shall we re-upgrade
> to the HWE kernel (4.10) or to another kernel version? Would you recommend
> an alternative solution?
>
> Thank you very much, we're looking forward for your advice.
>
> Kind regards,
> Bogdan
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Kernel version recommendation

[Bogdan SOLGA]

Hello, everyone!

We have recently upgraded our Ceph pool to the latest Luminous release. On
one of the servers that we used as Ceph clients we had several freeze
issues, which we empirically linked to the concurrent usage of some I/O
operations - writing in an LXD container (backed by Ceph) while there was
an ongoing PG rebalancing. We searched for the issue's cause through the
logs, but we haven't found anything useful.

At that time the server was running Ubuntu 16 with a 4.5 kernel. We thought
an upgrade to the latest HWE kernel (4.10) would help, but we had the same
freezing issues after the kernel upgrade. Of course, we're aware that we
have tried to fix / avoid the issue without understanding it's cause.

After seeing the OS recommendations from the Ceph page
,
we reinstalled the server (and got the 4.4 kernel), we ran into a feature
set mismatch issue when mounting a RBD image. We concluded

that the feature set requires a kernel > 4.5.

Our question - how would you recommend us to proceed? Shall we re-upgrade
to the HWE kernel (4.10) or to another kernel version? Would you recommend
an alternative solution?

Thank you very much, we're looking forward for your advice.

Kind regards,
Bogdan
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com