Re: [ceph-users] NUMA zone_reclaim_mode
On Mon, Jan 12, 2015 at 8:25 AM, Dan Van Der Ster daniel.vanders...@cern.ch wrote: On 12 Jan 2015, at 17:08, Sage Weil s...@newdream.net wrote: On Mon, 12 Jan 2015, Dan Van Der Ster wrote: Moving forward, I think it would be good for Ceph to a least document this behaviour, but better would be to also detect when zone_reclaim_mode != 0 and warn the admin (like MongoDB does). This line from the commit which disables it in the kernel is pretty wise, IMHO: On current machines and workloads it is often the case that zone_reclaim_mode destroys performance but not all users know how to detect this. Favour the common case and disable it by default. Sounds good to me. Do you mind submitting a patch that prints a warning from either FileStore::_detect_fs()? That will appear in the local ceph-osd.NNN.log. Alternatively, we should send something to the cluster log (osd-clog.warning() ...) but if we go that route we need to be careful that the logger it up and running first, which (I think) rules out FileStore::_detect_fs(). It could go in OSD itself although that seems less clean since the recommendation probably doesn't apply when using a backend that doesn't use a file system… Sure, I’ll try to prepare a patch which warns but isn’t too annoying. MongoDB already solved the heuristic: https://github.com/mongodb/mongo/blob/master/src/mongo/db/startup_warnings_mongod.cpp It’s licensed as AGPLv3 -- do you already know if we can borrow such code into Ceph? https://www.gnu.org/licenses/license-list.html#AGPL I've read that and the linked Affero Article 13 and I actually can't tell if Ceph is safe to integrate or not, but I'm thinking no since the servers are under LGPL. :/ Also I'm not sure if storage system users qualify as remote users but I don't think we're going to print an Affero string every time somebody runs a ceph tool. ;) -Greg ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] NUMA zone_reclaim_mode
On Mon, 12 Jan 2015, Dan Van Der Ster wrote: Sure, I?ll try to prepare a patch which warns but isn?t too annoying. MongoDB already solved the heuristic: https://github.com/mongodb/mongo/blob/master/src/mongo/db/startup_warnings_mongod.cpp It?s licensed as AGPLv3 -- do you already know if we can borrow such code into Ceph? I don't think AGPL is compatible, nope! Sorry... sage ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] NUMA zone_reclaim_mode
On 13/01/2015 01:10, Gregory Farnum wrote: On Mon, Jan 12, 2015 at 8:25 AM, Dan Van Der Ster daniel.vanders...@cern.ch wrote: On 12 Jan 2015, at 17:08, Sage Weil s...@newdream.net wrote: On Mon, 12 Jan 2015, Dan Van Der Ster wrote: Moving forward, I think it would be good for Ceph to a least document this behaviour, but better would be to also detect when zone_reclaim_mode != 0 and warn the admin (like MongoDB does). This line from the commit which disables it in the kernel is pretty wise, IMHO: On current machines and workloads it is often the case that zone_reclaim_mode destroys performance but not all users know how to detect this. Favour the common case and disable it by default. Sounds good to me. Do you mind submitting a patch that prints a warning from either FileStore::_detect_fs()? That will appear in the local ceph-osd.NNN.log. Alternatively, we should send something to the cluster log (osd-clog.warning() ...) but if we go that route we need to be careful that the logger it up and running first, which (I think) rules out FileStore::_detect_fs(). It could go in OSD itself although that seems less clean since the recommendation probably doesn't apply when using a backend that doesn't use a file system… Sure, I’ll try to prepare a patch which warns but isn’t too annoying. MongoDB already solved the heuristic: https://github.com/mongodb/mongo/blob/master/src/mongo/db/startup_warnings_mongod.cpp It’s licensed as AGPLv3 -- do you already know if we can borrow such code into Ceph? https://www.gnu.org/licenses/license-list.html#AGPL I've read that and the linked Affero Article 13 and I actually can't tell if Ceph is safe to integrate or not, but I'm thinking no since the servers are under LGPL. :/ Also I'm not sure if storage system users qualify as remote users but I don't think we're going to print an Affero string every time somebody runs a ceph tool. ;) AGPL does not require that, the approach is more practical: if the server provides you with an API / call to retrieve the sources, such a call can't be removed. It would be good thing to be able to implement the following scenario: * I'm connected to a Ceph cluster via RADOS * I'd like to migrate all I have in this cluster to my own cluster * Let's ask the Ceph server for the complete and corresponding sources and recompile / repackage them locally * Deploy my Ceph cluster from the local packages * Migrate pools from the remote cluster to the local cluster Regardless of license requirements, in the long run, there is almost zero chance to migrate successfully from a service provider (Ceph or otherwise) to a local service otherwise. Note that I'm not only referring to the data stored in the cluster but also how you're using the service. The same problem you would experience if migrating from a MySQL server to a PostgresQL server, for instance. Cheers -Greg ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Loïc Dachary, Artisan Logiciel Libre signature.asc Description: OpenPGP digital signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] NUMA zone_reclaim_mode
Hi Dan, On 12/01/2015 17:25, Dan Van Der Ster wrote: On 12 Jan 2015, at 17:08, Sage Weil s...@newdream.net mailto:s...@newdream.net wrote: On Mon, 12 Jan 2015, Dan Van Der Ster wrote: Moving forward, I think it would be good for Ceph to a least document this behaviour, but better would be to also detect when zone_reclaim_mode != 0 and warn the admin (like MongoDB does). This line from the commit which disables it in the kernel is pretty wise, IMHO: On current machines and workloads it is often the case that zone_reclaim_mode destroys performance but not all users know how to detect this. Favour the common case and disable it by default. Sounds good to me. Do you mind submitting a patch that prints a warning from either FileStore::_detect_fs()? That will appear in the local ceph-osd.NNN.log. Alternatively, we should send something to the cluster log (osd-clog.warning() ...) but if we go that route we need to be careful that the logger it up and running first, which (I think) rules out FileStore::_detect_fs(). It could go in OSD itself although that seems less clean since the recommendation probably doesn't apply when using a backend that doesn't use a file system… Sure, I’ll try to prepare a patch which warns but isn’t too annoying. MongoDB already solved the heuristic: https://github.com/mongodb/mongo/blob/master/src/mongo/db/startup_warnings_mongod.cpp It’s licensed as AGPLv3 -- do you already know if we can borrow such code into Ceph? If you're looking at adapting the https://github.com/mongodb/mongo/blob/master/src/mongo/db/startup_warnings_mongod.cpp#L107 code block and a few others into Ceph the licensing terms do not apply. If you were to disregard the mongodb implementation and rewrite it from scratch, you would come up with the same code because there is no way to do it differently. A contrario, if the licensing terms were to be applied, every implementation recommending that /proc/sys/vm/zone_reclaim_mode is set to zero would always be under AGPLv3 because they are all a) reading the file, b) printing a warning. That would not make sense and this is why software implementing a trivial logic can't be copyrighted. In my opinion you can borrow code from startup_warnings_mongod.cpp into Ceph because it's trivial and non copyrightable. Cheers Cheers, Dan Thanks! sage ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Loïc Dachary, Artisan Logiciel Libre signature.asc Description: OpenPGP digital signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] NUMA zone_reclaim_mode
On Mon, 12 Jan 2015, Dan Van Der Ster wrote: Moving forward, I think it would be good for Ceph to a least document this behaviour, but better would be to also detect when zone_reclaim_mode != 0 and warn the admin (like MongoDB does). This line from the commit which disables it in the kernel is pretty wise, IMHO: On current machines and workloads it is often the case that zone_reclaim_mode destroys performance but not all users know how to detect this. Favour the common case and disable it by default. Sounds good to me. Do you mind submitting a patch that prints a warning from either FileStore::_detect_fs()? That will appear in the local ceph-osd.NNN.log. Alternatively, we should send something to the cluster log (osd-clog.warning() ...) but if we go that route we need to be careful that the logger it up and running first, which (I think) rules out FileStore::_detect_fs(). It could go in OSD itself although that seems less clean since the recommendation probably doesn't apply when using a backend that doesn't use a file system... Thanks! sage ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] NUMA zone_reclaim_mode
On 12 Jan 2015, at 17:08, Sage Weil s...@newdream.netmailto:s...@newdream.net wrote: On Mon, 12 Jan 2015, Dan Van Der Ster wrote: Moving forward, I think it would be good for Ceph to a least document this behaviour, but better would be to also detect when zone_reclaim_mode != 0 and warn the admin (like MongoDB does). This line from the commit which disables it in the kernel is pretty wise, IMHO: On current machines and workloads it is often the case that zone_reclaim_mode destroys performance but not all users know how to detect this. Favour the common case and disable it by default. Sounds good to me. Do you mind submitting a patch that prints a warning from either FileStore::_detect_fs()? That will appear in the local ceph-osd.NNN.log. Alternatively, we should send something to the cluster log (osd-clog.warning() ...) but if we go that route we need to be careful that the logger it up and running first, which (I think) rules out FileStore::_detect_fs(). It could go in OSD itself although that seems less clean since the recommendation probably doesn't apply when using a backend that doesn't use a file system… Sure, I’ll try to prepare a patch which warns but isn’t too annoying. MongoDB already solved the heuristic: https://github.com/mongodb/mongo/blob/master/src/mongo/db/startup_warnings_mongod.cpp It’s licensed as AGPLv3 -- do you already know if we can borrow such code into Ceph? Cheers, Dan Thanks! sage ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
