Bug#705262: ceph
Thank you for not Cc-ing me. However there are several problems left: - ceph-common depends on python-flask and python-ceph. - ceph-common is not _arch-all_, why does it exist? - Why ceph-mds? - ceph depends on fdisk, parted and whole lot other crap it does not need. - A lot of Replaces. - python-ceph needs stricter dependencies. - Split between -java and -jni for no apparent reason, it only add a package to the global index. About the debug packages: I can also ask ftp-team to kill it, because it ten packages that can't be used independently fills the package index. Bastian -- Oblivion together does not frighten me, beloved. -- Thalassa (in Anne Mulhall's body), Return to Tomorrow, stardate 4770.3. -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#705262: ceph
On Sun, Sep 15, 2013 at 12:00 PM, Bastian Blank wa...@debian.org wrote: Thank you for not Cc-ing me. Can be my fault, I'd the thought as a bugreport owner, you'll get all mails. However there are several problems left: - ceph-common depends on python-flask and python-ceph. Why this is a problem? What I see is that python-flask should be moved to python-ceph . - ceph-common is not _arch-all_, why does it exist? Tools that used by other packages that can be installed independently. Should it be named like ceph-base or ceph-tools? - Why ceph-mds? Ceph has three independent blocks. Metadata servers (-mds) are one of them. Please see the overview[1]. - ceph depends on fdisk, parted and whole lot other crap it does not need. Agree on this. Don't know how it made there. - A lot of Replaces. There were package renames, users may have packages from upstream or Ubuntu. That's make it complex. - python-ceph needs stricter dependencies. Will check. - Split between -java and -jni for no apparent reason, it only add a package to the global index. James? About the debug packages: I can also ask ftp-team to kill it, because it ten packages that can't be used independently fills the package index. Still not sure they should be integrated. In a rush now, may write more later. Laszlo/GCS [1] http://ceph.com/docs/next/cephfs/ -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#705262: ceph
On Sun, Sep 15, 2013 at 12:00 PM, Bastian Blank wa...@debian.org wrote: - ceph depends on fdisk, parted and whole lot other crap it does not need. This is most likely a dependency of the ceph-disk-prepare or ceph-deploy scripts, which handle preparing partitions and filesystems on new disks intended for Ceph OSDs.
Bug#705262: ceph
On Sun, Sep 15, 2013 at 12:30:22PM +0200, László Böszörményi (GCS) wrote: On Sun, Sep 15, 2013 at 12:00 PM, Bastian Blank wa...@debian.org wrote: However there are several problems left: - ceph-common depends on python-flask and python-ceph. Why this is a problem? What I see is that python-flask should be moved to python-ceph . There is a python script ceph-rest-api. Where is this used? Why does it warrant a dependency on 20 other packages? - ceph-common is not _arch-all_, why does it exist? Tools that used by other packages that can be installed independently. Should it be named like ceph-base or ceph-tools? Not sure right now. It looks like a random dash of tools, stashed together without much thinking. - ceph, rados, rbd: Tools to manage different parts of ceph, remotely. - ceph-authtool: Works on keyring files, so needs to run locally on the monitor. - ceph-rest-api: Where does this belong to? - ceph-dencoder, ceph-syn: This are test or debugging tools. I would - move ceph-authotool and maybe ceph-rest-api into ceph, - move cephfs and mount.ceph into ceph-common - move ceph-dencoder and ceph-syn into ceph-test, - move stuff from ceph-resource-agents into ceph and - drop ceph-fs-common and ceph-resource-agents. - Why ceph-mds? Ceph has three independent blocks. Metadata servers (-mds) are one of them. Please see the overview[1]. Yeah. But why does it need a different package? What does this extra package bring for the user? - ceph depends on fdisk, parted and whole lot other crap it does not need. Agree on this. Don't know how it made there. Because ceph-disk (another incompletely documented indirection) uses it. The important parts (ceph-mon, ceph-osd) works fine without it. - A lot of Replaces. There were package renames, users may have packages from upstream or Ubuntu. That's make it complex. Not a concern for Debian. It was never in a stable release. - python-ceph needs stricter dependencies. Will check. At least the dependencies for librados2 and librdb1 needs to be stricter. The dependency on libcephfs1 is missing. - Symbols files are missing. Bastian -- There's a way out of any cage. -- Captain Christopher Pike, The Menagerie (The Cage), stardate unknown. -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#705262: ceph
On Sun, Sep 15, 2013 at 1:03 PM, Bastian Blank wa...@debian.org wrote: There is a python script ceph-rest-api. Where is this used? Why does it warrant a dependency on 20 other packages? This is the as yet sparsely documented admin REST WSGI app (which can also run as a standalone server), which was a new feature in the 0.67 dumpling release. This should not be confused with the RADOSGW REST stuff. I have not looked at the source of the script yet, but have had a play with it. I suspect it depends on things like Jinja2 because it renders HTML tables for self-documenting purposes if you simply open the API with a web browser. This seems like a bit of an overkill for using a template engine like Jinja2, but oh well. If you make an API request using a non-browser client (e.g. wget or XmlHttpRequest), it returns JSON for the most part. I suspect this could be made a separate package, since it is not essential to running an OSD, MDS or mon. - Why ceph-mds? Ceph has three independent blocks. Metadata servers (-mds) are one of them. Please see the overview[1]. Yeah. But why does it need a different package? What does this extra package bring for the user? Ceph MDS are not needed on every OSD, and are in fact only needed if using CephFS. If you're only using RBD pools and RADOSGW, then a MDS is completely unnecessary. Similarly, a mon is not needed on every Ceph node either. A typical setup consists of two or more OSDs (likely many more), three or more (preferably an odd number) monitors - either on selected OSD nodes, or separate servers, and one or more MDS nodes - but only if using CephFS. I only know of one or two people who've actually tried CephFS, and according to the developers, it should not be considered quite production-ready yet. Bear in mind however that OSDs, mons, and MDSs all use the same /etc/ceph/ceph.conf config file, making it potentially hard to split them into separate packages.
Bug#705262: ceph
On Sun, Sep 15, 2013 at 04:18:05PM +0200, Daniel Swarbrick wrote: On Sun, Sep 15, 2013 at 1:03 PM, Bastian Blank wa...@debian.org wrote: There is a python script ceph-rest-api. Where is this used? Why does it warrant a dependency on 20 other packages? This is the as yet sparsely documented admin REST WSGI app (which can also run as a standalone server), which was a new feature in the 0.67 dumpling release. This should not be confused with the RADOSGW REST stuff. Thats why I asked it this sparsely documented and not enabled by default piece of software warrants the hard inclusion of a lot of dependencies. Yeah. But why does it need a different package? What does this extra package bring for the user? Ceph MDS are not needed on every OSD, and are in fact only needed if using CephFS. If you're only using RBD pools and RADOSGW, then a MDS is completely unnecessary. Similarly, a mon is not needed on every Ceph node either. Neither mon, osd nor mds should run on the same system I would assume, at least in large installations. So my question is just _why_ mds needs this extra package, while mon/osd can live in the same. Bastian -- Vulcans never bluff. -- Spock, The Doomsday Machine, stardate 4202.1 -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#705262: ceph
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Bastian Lazlo On 15/09/13 12:03, Bastian Blank wrote: On Sun, Sep 15, 2013 at 12:30:22PM +0200, László Böszörményi (GCS) wrote: On Sun, Sep 15, 2013 at 12:00 PM, Bastian Blank wa...@debian.org wrote: However there are several problems left: - ceph-common depends on python-flask and python-ceph. Why this is a problem? What I see is that python-flask should be moved to python-ceph . There is a python script ceph-rest-api. Where is this used? Why does it warrant a dependency on 20 other packages? Dumpling (0.67.x) is the first release with basic support for a REST api for administering a ceph cluster - this binary supports this feature - but more of a demo of how to use than anything really usable (dev mode only). However I do agree that it looks like it should be in ceph. - ceph-common is not _arch-all_, why does it exist? Tools that used by other packages that can be installed independently. Should it be named like ceph-base or ceph-tools? Not sure right now. It looks like a random dash of tools, stashed together without much thinking. - ceph, rados, rbd: Tools to manage different parts of ceph, remotely. - ceph-authtool: Works on keyring files, so needs to run locally on the monitor. - ceph-rest-api: Where does this belong to? - ceph-dencoder, ceph-syn: This are test or debugging tools. I would - move ceph-authotool and maybe ceph-rest-api into ceph, - move cephfs and mount.ceph into ceph-common - move ceph-dencoder and ceph-syn into ceph-test, - move stuff from ceph-resource-agents into ceph and - drop ceph-fs-common and ceph-resource-agents. - Why ceph-mds? Ceph has three independent blocks. Metadata servers (-mds) are one of them. Please see the overview[1]. Yeah. But why does it need a different package? What does this extra package bring for the user? For context, the ceph-mds function was split out in the packaging as MDS is not as well tested/production grade as the RADOS and RBD features; In Ubuntu we use this to signal which parts receive full support from the Ubuntu Server and Security teams. I appreciate that this is not required for Debian but its also in the upstream packaging so it would be nice if these splits could stay (see notes below about why keeping packaging consistent across upstream, Debian and Ubuntu is a good idea). - ceph depends on fdisk, parted and whole lot other crap it does not need. Agree on this. Don't know how it made there. Because ceph-disk (another incompletely documented indirection) uses it. The important parts (ceph-mon, ceph-osd) works fine without it. ceph-disk is a key part of the deployment infrastructure for ceph so these are important dependencies for anyone deploying ceph at scale using upstream Chef recipes or ceph-deploy. - A lot of Replaces. There were package renames, users may have packages from upstream or Ubuntu. That's make it complex. Not a concern for Debian. It was never in a stable release. I think it is a concern for Debian; the principle source of Ceph packages for Debian to-date has been from upstream, so ensuring smooth transitions to/from Debian/upstream is important IMHO. - python-ceph needs stricter dependencies. Will check. At least the dependencies for librados2 and librdb1 needs to be stricter. The dependency on libcephfs1 is missing. Agreed - python-ceph only just grew bindings for cephfs so that's just an oversight. Good spot. = on equiv binary package version should do the job for the librbd/rados depends. Laszlo - re -java/-jni split - this is in compliance with the Java packaging policy: http://www.debian.org/doc/packaging-manuals/java-policy/x114.html I really think we need to stick with the packaging structure and naming that is already in place as much as possible; All of the existing deployment tooling (chef, juju, ceph-deploy) is written based on this structure on Debian based systems; diverging is just going to create work in other areas and potentially alienate the Debian packaging as its different to the packaging that the Ceph community has been using for the last two years... at least! FYI once ceph is up-to-date in Debian I'm intending on landing ceph-deploy as a way of deploying ceph on Debian - it makes testing much easier. - -- James Page Ubuntu and Debian Developer james.p...@ubuntu.com jamesp...@debian.org -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQIcBAEBCAAGBQJSNe0LAAoJEL/srsug59jDGLEP/3KDiq6fbzcNUeaw0JuCSK0F QNfMdAQWRa86UX8zOJvtlhjbRJlwPWY4kRj52fxsXZTOcs+9t+ooTZ0ACwYTy0K3 jPscTwoev1rBvbYD1kJXCrd8OVg9I1WC0cK30OiPQ7y6SrTf+84fpHWHAZhcoQa7 TD5j+stSk0Nfko8Wj6K3xEy1J7PgoY16gcAbozEW/35zAz7tFUjhRgwJSJ9tBCdR 2H7rn2Dmxdk5RPNutKM1pKFiOdCTYOz87+ptYYdeKbvRuxU9b+dl2wb5U9D5IfTH mBJm4dcQQr0IDxMdhG7VJxb/wSA8Nog5fxIxelloPg7wI65D4Lf/m0QvYXEpRc56 N5xCWaLKw5aTmh4Cviz7k6SvnXNfWoemF7LueETJFo1IUFS2jBB2tbYOcXFpzvhX
Bug#705262: ceph
On Sun, Sep 15, 2013 at 06:23:23PM +0100, James Page wrote: On 15/09/13 12:03, Bastian Blank wrote: There is a python script ceph-rest-api. Where is this used? Why does it warrant a dependency on 20 other packages? Dumpling (0.67.x) is the first release with basic support for a REST api for administering a ceph cluster - this binary supports this feature - but more of a demo of how to use than anything really usable (dev mode only). However I do agree that it looks like it should be in ceph. How does the authentication work? The whole Python code in Ceph looks pretty bad for me. Because ceph-disk (another incompletely documented indirection) uses it. The important parts (ceph-mon, ceph-osd) works fine without it. ceph-disk is a key part of the deployment infrastructure for ceph so these are important dependencies for anyone deploying ceph at scale using upstream Chef recipes or ceph-deploy. Well. Then it would be one, not three diffrent implementations of fdisk. Not a concern for Debian. It was never in a stable release. I think it is a concern for Debian; the principle source of Ceph packages for Debian to-date has been from upstream, so ensuring smooth transitions to/from Debian/upstream is important IMHO. Nope. Other sources are no concern. At least the dependencies for librados2 and librdb1 needs to be stricter. The dependency on libcephfs1 is missing. Agreed - python-ceph only just grew bindings for cephfs so that's just an oversight. Good spot. = on equiv binary package version should do the job for the librbd/rados depends. Nope. They need to be ==. The (bad written) Python wrappers uses ctypes, which does not implement a full ELF-loader. Laszlo - re -java/-jni split - this is in compliance with the Java packaging policy: http://www.debian.org/doc/packaging-manuals/java-policy/x114.html You forgot the should. Plus this is not part of the Debian policy. I really think we need to stick with the packaging structure and naming that is already in place as much as possible; All of the existing deployment tooling (chef, juju, ceph-deploy) is written based on this structure on Debian based systems; diverging is just going to create work in other areas and potentially alienate the Debian packaging as its different to the packaging that the Ceph community has been using for the last two years... at least! I'm talked to the ftp-masters. They usualy don't like if a package is split too much for no apparent reason. It is nice if they are compatible, but for Debian all problems have to be taken into account. Bastian -- Pain is a thing of the mind. The mind can be controlled. -- Spock, Operation -- Annihilate! stardate 3287.2 -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#705262: ceph
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 15/09/13 20:04, Bastian Blank wrote: On Sun, Sep 15, 2013 at 06:23:23PM +0100, James Page wrote: On 15/09/13 12:03, Bastian Blank wrote: There is a python script ceph-rest-api. Where is this used? Why does it warrant a dependency on 20 other packages? Dumpling (0.67.x) is the first release with basic support for a REST api for administering a ceph cluster - this binary supports this feature - but more of a demo of how to use than anything really usable (dev mode only). However I do agree that it looks like it should be in ceph. How does the authentication work? The whole Python code in Ceph looks pretty bad for me. Ceph itself uses a key based kerberos style authentication system called cephx; the ceph_rest_api library uses the keys/config in /etc/ceph but provides no additional auth for HTTP API access; neither does the wrapper ceph-rest-api which is why right now there is no init script or upstart configuration for this feature. I make no general observations about the python code in ceph other than it's worked well for me for the last 18 months. Because ceph-disk (another incompletely documented indirection) uses it. The important parts (ceph-mon, ceph-osd) works fine without it. ceph-disk is a key part of the deployment infrastructure for ceph so these are important dependencies for anyone deploying ceph at scale using upstream Chef recipes or ceph-deploy. Well. Then it would be one, not three diffrent implementations of fdisk. Technically its just two - gdisk and parted; I don't have any strong objection to upstream using two different tools if it makes massaging block devices easier for them. Not a concern for Debian. It was never in a stable release. I think it is a concern for Debian; the principle source of Ceph packages for Debian to-date has been from upstream, so ensuring smooth transitions to/from Debian/upstream is important IMHO. Nope. Other sources are no concern. I'd like to try and play nice with other methods of distribution especially if one of those has been used extensively by Debian users to-date; makes life easier for potential users of these packages. At least the dependencies for librados2 and librdb1 needs to be stricter. The dependency on libcephfs1 is missing. Agreed - python-ceph only just grew bindings for cephfs so that's just an oversight. Good spot. = on equiv binary package version should do the job for the librbd/rados depends. Nope. They need to be ==. The (bad written) Python wrappers uses ctypes, which does not implement a full ELF-loader. OK; I know they are just a simple wrap over the core libs so that makes sense. Laszlo - re -java/-jni split - this is in compliance with the Java packaging policy: http://www.debian.org/doc/packaging-manuals/java-policy/x114.html You forgot the should. Plus this is not part of the Debian policy. Meh - well it works for me; the -java is arch independent whereas the - -jni is not; saves duping the jar file in binary packages. If the -jni every gets multi-arched then this approach is required anyway. I really think we need to stick with the packaging structure and naming that is already in place as much as possible; All of the existing deployment tooling (chef, juju, ceph-deploy) is written based on this structure on Debian based systems; diverging is just going to create work in other areas and potentially alienate the Debian packaging as its different to the packaging that the Ceph community has been using for the last two years... at least! I'm talked to the ftp-masters. They usualy don't like if a package is split too much for no apparent reason. It is nice if they are compatible, but for Debian all problems have to be taken into account. I don't think the ceph packages are split for no apparent reason; I believe that by adopting the current structure which works with the most popular deployment tooling for ceph we work with upstream, rather than going against the grain. - -- James Page Ubuntu and Debian Developer james.p...@ubuntu.com jamesp...@debian.org -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iQIcBAEBCAAGBQJSNhlBAAoJEL/srsug59jDgQgQAM91nm57XBaWGQoxJ6mC7hy3 n0hsT9ehI+FJDL2nC9jr1vSBCMgQKgfkt4LHrBXUt4t/Dz6YVu1vwZbmrhfP36Bi O1WcZ+StIJZaBxHYq37XheoLqShR39VSPfGmp7dK5u0RBrcqj1PatVfF/oYrLSxL KSkqmccaOeAvbE0G4gGuceEuLnR6ZI1kgeJkFCE0daEtJdDe89ojj3PBRz6H4Ya0 zRSP3Es4YcHgEr6vlFV0+jDNrsuq72g/1kn+ouW0jLQQ+a9+12+v4kkVBNL4fUBj d9+2wwOEuiFOK1/tOGQjwc5XdOtPe+P6oHQ1uWoZzYOYOCw45+LDO6IIMgrr8+Xa QtolHNKPJtEr24myCowOkQ9tmbrx7YRzs/y7EMAo6faYI7ZPDc6gwRZESUtiqAuJ +HgyEGyqUjuJGOjQ0PIhB1KteJdJNoFFeAUAvbDh1UaEgMLdrRw+ZyFoC0NkeGjX 2xoHPIInN8v/qeOc1vGkwzCXg9SFe6VkVi338SU8M4k/nEGwPxr6TCgKs0bK+P9x lbdK17HPr5ZtzmoMyVT83WPTyIad7IfyeLOU2N0EX3zbWaBwp1NplZhKRx+qeAGP
