Re: ceph from poc to production
Thx! Am 17.01.2013 um 17:24 schrieb Sébastien Han : > Hi Stephan, > >> - Increase the osdmax value > > Well actually this doesn't seem to be necessary because when you issue > a "ceph osd create" the osdmax value is automatically incremented. It > just that a while ago I saw on the ML or IRC that it was a good > practice to do this. And this documentation > http://ceph.com/docs/master/rados/operations/control/ says: > > "Set the max_osd parameter in the osd map. This is necessary when > expanding the storage cluster." > > Maybe the doc is outdated on this option, I don't know. > >> - Set “filestore flusher” option to false, in your [osd] section > > During my benchmarks sessions I noticed that disabling this option > brought way better performance, at least for old hardware. Fo recent > hardware I haven't seen any difference, but I kept the habit to > disable it. There are some discussions about this option on the ML. As > far as I can remember, it's harmless to disable it. > From the doc here > http://ceph.com/docs/master/rados/configuration/filestore-config-ref/: > > "The filestore flusher forces data from large writes to be written out > using sync file range before the sync in order to (hopefully) reduce > the cost of the eventual sync. In practice, disabling ‘filestore > flusher’ seems to improve performance in some cases." > > Cheers, > > -- > Regards, > Sébastien Han. > > > On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe wrote: >> >> Hello Sebastien, >> hello list, >> >> first nice article sebastien ;-) thanks for contributing. >> >> I've two question regarding the tips: >> >> Can somebody explain me why i should do this?: >> - Increase the osdmax value >> - Set “filestore flusher” option to false, in your [osd] section >> >> Greets, >> Stefan >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ceph from poc to production
Great! Any ETA? Am 17.01.2013 um 17:29 schrieb Mark Nelson : > On 01/17/2013 10:24 AM, Sébastien Han wrote: >> Hi Stephan, >> >>> - Increase the osdmax value >> >> Well actually this doesn't seem to be necessary because when you issue >> a "ceph osd create" the osdmax value is automatically incremented. It >> just that a while ago I saw on the ML or IRC that it was a good >> practice to do this. And this documentation >> http://ceph.com/docs/master/rados/operations/control/ says: >> >> "Set the max_osd parameter in the osd map. This is necessary when >> expanding the storage cluster." >> >> Maybe the doc is outdated on this option, I don't know. >> >>> - Set “filestore flusher” option to false, in your [osd] section >> >> During my benchmarks sessions I noticed that disabling this option >> brought way better performance, at least for old hardware. Fo recent >> hardware I haven't seen any difference, but I kept the habit to >> disable it. There are some discussions about this option on the ML. As >> far as I can remember, it's harmless to disable it. >> From the doc here >> http://ceph.com/docs/master/rados/configuration/filestore-config-ref/: > > I actually have been doing parametric sweeps over the various ceph tunables > to try to see what affects performance. I'll have an article coming out > soonish documenting some of the findings. Typically disabling the flusher > seems to be beneficial on XFS and BTRFS, but it seems to be the opposite on > EXT4. > >> >> "The filestore flusher forces data from large writes to be written out >> using sync file range before the sync in order to (hopefully) reduce >> the cost of the eventual sync. In practice, disabling ‘filestore >> flusher’ seems to improve performance in some cases." >> >> Cheers, >> >> -- >> Regards, >> Sébastien Han. >> >> >> On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe wrote: >>> >>> Hello Sebastien, >>> hello list, >>> >>> first nice article sebastien ;-) thanks for contributing. >>> >>> I've two question regarding the tips: >>> >>> Can somebody explain me why i should do this?: >>> - Increase the osdmax value >>> - Set “filestore flusher” option to false, in your [osd] section >>> >>> Greets, >>> Stefan >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majord...@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ceph from poc to production
Cool, I look forward to reading it! -- Regards, Sébastien Han. On Thu, Jan 17, 2013 at 5:29 PM, Mark Nelson wrote: > On 01/17/2013 10:24 AM, Sébastien Han wrote: >> >> Hi Stephan, >> >>> - Increase the osdmax value >> >> >> Well actually this doesn't seem to be necessary because when you issue >> a "ceph osd create" the osdmax value is automatically incremented. It >> just that a while ago I saw on the ML or IRC that it was a good >> practice to do this. And this documentation >> http://ceph.com/docs/master/rados/operations/control/ says: >> >> "Set the max_osd parameter in the osd map. This is necessary when >> expanding the storage cluster." >> >> Maybe the doc is outdated on this option, I don't know. >> >>> - Set “filestore flusher” option to false, in your [osd] section >> >> >> During my benchmarks sessions I noticed that disabling this option >> brought way better performance, at least for old hardware. Fo recent >> hardware I haven't seen any difference, but I kept the habit to >> disable it. There are some discussions about this option on the ML. As >> far as I can remember, it's harmless to disable it. >> From the doc here >> http://ceph.com/docs/master/rados/configuration/filestore-config-ref/: > > > I actually have been doing parametric sweeps over the various ceph tunables > to try to see what affects performance. I'll have an article coming out > soonish documenting some of the findings. Typically disabling the flusher > seems to be beneficial on XFS and BTRFS, but it seems to be the opposite on > EXT4. > > >> >> "The filestore flusher forces data from large writes to be written out >> using sync file range before the sync in order to (hopefully) reduce >> the cost of the eventual sync. In practice, disabling ‘filestore >> flusher’ seems to improve performance in some cases." >> >> Cheers, >> >> -- >> Regards, >> Sébastien Han. >> >> >> On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe >> wrote: >>> >>> >>> Hello Sebastien, >>>hello list, >>> >>> first nice article sebastien ;-) thanks for contributing. >>> >>> I've two question regarding the tips: >>> >>> Can somebody explain me why i should do this?: >>> - Increase the osdmax value >>> - Set “filestore flusher” option to false, in your [osd] section >>> >>> Greets, >>> Stefan >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majord...@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ceph from poc to production
On 01/17/2013 10:24 AM, Sébastien Han wrote: Hi Stephan, - Increase the osdmax value Well actually this doesn't seem to be necessary because when you issue a "ceph osd create" the osdmax value is automatically incremented. It just that a while ago I saw on the ML or IRC that it was a good practice to do this. And this documentation http://ceph.com/docs/master/rados/operations/control/ says: "Set the max_osd parameter in the osd map. This is necessary when expanding the storage cluster." Maybe the doc is outdated on this option, I don't know. - Set “filestore flusher” option to false, in your [osd] section During my benchmarks sessions I noticed that disabling this option brought way better performance, at least for old hardware. Fo recent hardware I haven't seen any difference, but I kept the habit to disable it. There are some discussions about this option on the ML. As far as I can remember, it's harmless to disable it. From the doc here http://ceph.com/docs/master/rados/configuration/filestore-config-ref/: I actually have been doing parametric sweeps over the various ceph tunables to try to see what affects performance. I'll have an article coming out soonish documenting some of the findings. Typically disabling the flusher seems to be beneficial on XFS and BTRFS, but it seems to be the opposite on EXT4. "The filestore flusher forces data from large writes to be written out using sync file range before the sync in order to (hopefully) reduce the cost of the eventual sync. In practice, disabling ‘filestore flusher’ seems to improve performance in some cases." Cheers, -- Regards, Sébastien Han. On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe wrote: Hello Sebastien, hello list, first nice article sebastien ;-) thanks for contributing. I've two question regarding the tips: Can somebody explain me why i should do this?: - Increase the osdmax value - Set “filestore flusher” option to false, in your [osd] section Greets, Stefan -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ceph from poc to production
Hi Stephan, > - Increase the osdmax value Well actually this doesn't seem to be necessary because when you issue a "ceph osd create" the osdmax value is automatically incremented. It just that a while ago I saw on the ML or IRC that it was a good practice to do this. And this documentation http://ceph.com/docs/master/rados/operations/control/ says: "Set the max_osd parameter in the osd map. This is necessary when expanding the storage cluster." Maybe the doc is outdated on this option, I don't know. > - Set “filestore flusher” option to false, in your [osd] section During my benchmarks sessions I noticed that disabling this option brought way better performance, at least for old hardware. Fo recent hardware I haven't seen any difference, but I kept the habit to disable it. There are some discussions about this option on the ML. As far as I can remember, it's harmless to disable it. >From the doc here http://ceph.com/docs/master/rados/configuration/filestore-config-ref/: "The filestore flusher forces data from large writes to be written out using sync file range before the sync in order to (hopefully) reduce the cost of the eventual sync. In practice, disabling ‘filestore flusher’ seems to improve performance in some cases." Cheers, -- Regards, Sébastien Han. On Wed, Jan 16, 2013 at 7:41 PM, Stefan Priebe wrote: > > Hello Sebastien, > hello list, > > first nice article sebastien ;-) thanks for contributing. > > I've two question regarding the tips: > > Can somebody explain me why i should do this?: > - Increase the osdmax value > - Set “filestore flusher” option to false, in your [osd] section > > Greets, > Stefan > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ceph from poc to production
Hello Sebastien, hello list, first nice article sebastien ;-) thanks for contributing. I've two question regarding the tips: Can somebody explain me why i should do this?: - Increase the osdmax value - Set “filestore flusher” option to false, in your [osd] section Greets, Stefan -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
