Re: [Lf_desktop] LSB Package API
On Mon, 2008-06-23 at 15:14 +0100, Scott James Remnant wrote: > My general feeling, having spoken to lots of ISVs, is that they don't > actually _want_ this. > > In order to support their customers, they're well aware that they have > to target particular distributions and versions - and they're quite > happy working and certifying particular vendors individually. I'm not in the position of having the chance to speak with ISVs myself unfortunately. I only relied on what seemed to have been communicated on the LSB face-to-face meeting. It might be good if we had more application vendors (and also open-source project maintainers) speaking up on this issue in the general or the particular proposal. Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 28, 2008, at 4:50 AM, Denis Washington wrote: I completely agree. Did I add v3 somewhere? You've included COPYING which starts with GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 I'd suggest LGPLv2 instead. Where do I find (or generate) the org.linuxbase.Packages.xml file? I forgot to add some files to Makefile.am, so they weren't in the "make dist" tarball. I added them and uploaded the fixed version to the LSB wiki (same place as the previous one). Cool, (although I'm way smarter about dbus-glib xml than I ever wanted to be now ;-) I'll likely have *.rpm packages generated this weekend. 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Thu, 2008-06-26 at 14:55 -0400, Jeff Johnson wrote: > On Jun 25, 2008, at 11:49 AM, Denis Washington wrote: > > > > > I hope what is in the data structures is sufficient and well-defined > > enough. And, what I increasingly tempt to believe, that we don't talk > > past each other. ;) > > > > I'm trying to build from your 4 tarballs. > > For starters, I'd suggest LGLPv2 rather than GPLv3 licensing. The > reason is that the intended target is commercial ISV's, who are > often (and justfiably) uncomfortable with the viral nature of GPL. > The commercial ISV's _MUST_ link to your code. I completely agree. Did I add v3 somewhere? > Second, I'd suggest a different name than "lsb-package" for now. > I see no "official" buy-in from LSB yet, and all package names starting > "lsb-" are reserved for LANANANANANANA ... Yeah. I thought of renaming it to "Burgdorf API" as this is my home town (and happens to be a German town too like Berlin). > Alternatively, get your act togther and register "lsb-package". But you > are almost sure to annoy and confuse many with "lsb-package". > > Personally, I'd suggest "lsb-berlin" until you get more buy-in, but > YMMV. Or that. > Back to building ... > > Build fails because > #include "i18n-defs.h" > from lib/lsb_package.c is nowhere obvious to be found. > > Doing > touch lib/i18n-defs.h > keeps gcc happy, but if you don't need the content, don't include the > file. > > Next failure is this > > creating liblsb_package.la > (cd .libs && rm -f liblsb_package.la && ln -s ../liblsb_package.la > liblsb_package.la) > make[2]: Leaving directory `/X/lsb-package/lsb_package-0.1/lib' > Making all in daemon > make[2]: Entering directory `/X/lsb-package/lsb_package-0.1/daemon' > make[2]: *** No rule to make target `org.linuxbase.Packages.xml', > needed by `dbus-server-bindings.h'. Stop. > make[2]: Leaving directory `/X/lsb-package/lsb_package-0.1/daemon' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/X/lsb-package/lsb_package-0.1' > make: *** [all] Error 2 > error: Bad exit status from /X/tmp/rpm-tmp.86094 (%build) > > Where do I find (or generate) the org.linuxbase.Packages.xml file? I forgot to add some files to Makefile.am, so they weren't in the "make dist" tarball. I added them and uploaded the fixed version to the LSB wiki (same place as the previous one). Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: [Rpm-maint] Re: LSB Package API
On Wed, Jun 25, 2008 at 8:35 PM, Pär Lidén <[EMAIL PROTECTED]> wrote: > I recently installed SPSS (a closed-source statistical program), and the > downloaded .bin file for Linux contained an installation progam using > Installshield (probably the same as they are using on Windows). I have had a similar experience with websphere on linux but i am sure for experience that the problem is common at other . The "graphical and interactive" installation program was Installshield. Result: 1- The installer had installed 6 rpm without no file - empty 2- The installer had installed an true - almost - rpm but it doesn't exist apparently on in installation CD 3 - The installer had installed the websphere sw as is, without rpm. 4- For update the software i have - always interactive - apply some patch: sometime worked fine, sometime no 5- The postinstaller of websphere changed my httpd.conf . The default install is for websphere to run as "root" : nice. 6 - i have to patch manually some file of one of or four (iirc) jvm included and have to create an init script ecc. 7- The installer want X windows and so : ok, exists also a silent install The time necessary generally was 2 or 3 days ( often also more) Problem: Q- it is repetable the installation ? It is simple to install on many system in batch mode, without human intervention? R: No Q- Can i update my system thereafter with safety? R: No as I don't know the dependency Q- Can I update webshere on a live system without loss of services ? R- No So, guess: I had done an RPM (two, one was an rpmrebuild), the patch are %patch, the deps are included ecc. and so i can live happy. What is more i can install it, thanks at rpm, in a multiarch (32/64) system and i can upgrade it online . Ok, it is a huge rpm (28000 files or so) but who cares ? (sure for deb it is possible to do the some thing ) So, at the end, the question is : sure that this installer method is a good reference model ? I prefer a good package management system (as rpm or deb) and not an installer as this. As everyone know there are basic difference between an installer and an package management system. Sorry for the long post. Regards > > I'm pretty sure the SPSS people don't want to invest in switching > installation system, or make any other big changes to it. They also probably > wants to use the same installation system they use on Windows. It would be > relatively easy for the Installshield guys to add support for the Berlin > API, and for SPSS to take advantage of that. That would make it possible for > the files it installed to show up in some way in the dpkg system, and I > could then manage it via synaptics. And that'd sure be nice! > > I'm not an expert in the field, but from what I can understand, none of the > existing solutions out there provide this feature, and in a way that is so > easy to implement (and learn) for the ISVs (and installation program > makers). > > To those who say ISVs anyway targets specific distributions: Yes, they > would still have to test it on all the major distributions, but they could > use a single installation implementation. And they would have to learn only > one system (which, if they are using Installshield-like software, is mostly > the same as on Windows). And I'm quite sure most ISVs sees these two things > as a big advantage. > > Sure it would be much better if they had made a real .deb for me, but them > using something similar to this would sure be much better than what they > have today. And as SPSS and many other companies probably don't want to > spend so much time and money on the Linux installer, many of them won't in > the for-seeable future do something which is a too big change from what they > use (and know) today. Maybe sometimes in the future when Linux has gained a > much larger market-share than today, but until then, the Berling API would > certainly be good. If this Berlin API actually becomes widely adopted, and > the ISVs still won't make the effort to support it, that's bad. But if they > can't even support something as simple as this, then they will not support > any of the other systems (which, as far as I can judge, are more complicated > for them). So, as an end-user sometimes installing programs from ISVs, I'm > definitely in favor of the Berlin API, as I'm hoping it will ease things a > bit for us. > > Regards > Pär Lidén > > 2008/6/24 Denis Washington <[EMAIL PROTECTED]>: > >> On Tue, 2008-06-24 at 12:03 +0100, Richard Hughes wrote: >> > On Sun, 2008-06-22 at 20:02 +0200, Denis Washington wrote: >> > > We shouldn't resignate just because nothing came out of the previous >> > > attempts. Also, the LSB Package API is designed to requ
Re: LSB Package API
On Jun 25, 2008, at 11:49 AM, Denis Washington wrote: I hope what is in the data structures is sufficient and well-defined enough. And, what I increasingly tempt to believe, that we don't talk past each other. ;) I'm trying to build from your 4 tarballs. For starters, I'd suggest LGLPv2 rather than GPLv3 licensing. The reason is that the intended target is commercial ISV's, who are often (and justfiably) uncomfortable with the viral nature of GPL. The commercial ISV's _MUST_ link to your code. Second, I'd suggest a different name than "lsb-package" for now. I see no "official" buy-in from LSB yet, and all package names starting "lsb-" are reserved for LANANANANANANA ... Alternatively, get your act togther and register "lsb-package". But you are almost sure to annoy and confuse many with "lsb-package". Personally, I'd suggest "lsb-berlin" until you get more buy-in, but YMMV. Back to building ... Build fails because #include "i18n-defs.h" from lib/lsb_package.c is nowhere obvious to be found. Doing touch lib/i18n-defs.h keeps gcc happy, but if you don't need the content, don't include the file. Next failure is this creating liblsb_package.la (cd .libs && rm -f liblsb_package.la && ln -s ../liblsb_package.la liblsb_package.la) make[2]: Leaving directory `/X/lsb-package/lsb_package-0.1/lib' Making all in daemon make[2]: Entering directory `/X/lsb-package/lsb_package-0.1/daemon' make[2]: *** No rule to make target `org.linuxbase.Packages.xml', needed by `dbus-server-bindings.h'. Stop. make[2]: Leaving directory `/X/lsb-package/lsb_package-0.1/daemon' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/X/lsb-package/lsb_package-0.1' make: *** [all] Error 2 error: Bad exit status from /X/tmp/rpm-tmp.86094 (%build) Where do I find (or generate) the org.linuxbase.Packages.xml file? My no-brainer goal is to generate *.rpm packages so that I have a "reference" build from which to send patches ... 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Wed, 2008-06-25 at 11:26 -0400, Jeff Johnson wrote: > On Jun 25, 2008, at 10:45 AM, Denis Washington wrote: > > > On Wed, 2008-06-25 at 10:12 -0400, Jeff Johnson wrote: > >> On Jun 24, 2008, at 1:38 PM, Denis Washington wrote: > >> > >>> > >>> > Sound like a plan? My primary goals here are two-fold: > > 1) avoiding disasters if bogus headers start to be added to an > rpmdb. > > 2) exposing rpmdbAdd() (and rpmdbRemove()) methods for use by > LSB/ISV/whatever applications that wish to register/unregister > software > on RPM managed systems. > >>> > >>> Sounds like a good plan, yeah. I'm glad being able to work with > >>> you on > >>> this, as you certainly have a LOT more experience than me concerning > >>> this. Thank you very much! > >>> > >> > >> No problem. > >> > >> Enumerating the necessary data elements that need to be present > >> in a RPM header, and choosing _SOME_ representational markup, > >> would seem to be on the critical path. > >> > >> (aside) dpkg its really the same fundamental problem, but a different > >> target metadata representation. Ditto _your_package_manager_here > >> for all instances of class. > >> > >> There are several existing representations of "package" manifests, > >> both explicit and/or implicit that can be used to enumerate the > >> necessary data elements to be included in the target metadata > >> representation > >> (note I did not say "rpmdb"). > >> > >> Simplest by far is find(1) output of a tree. i.e. an explicit list > >> of paths to files, with stat(2) and digest (and acls/xattrs and > >> selinux > >> file contexts and whatever else is needed) implicitly derived from > >> the tree. > > > > With "implicitly derived", do you mean "read from the installed files > > instead of being explicitly in the manifest file"? > > > > Yes. Basically I mean populating target metadata with stat(2) > info, not with explicitly parsed values. > > The advantage is KISS: it don't come any simpler (for ISV's and other > lusers) > than providing a file manifest. > > The disadvantage of a KISS file manifest is that indeed, the files > must be > 1) actually present (and available) on a file system > 2) correctly installed. Presumably the ISV (or other installer) > is functional, or > the ISV (or other installer) would not be trying to register a > "package", would it? > > >> Other soft "branding" identification information, like vendor, > >> packager, description, > >> build host, etc would need to be added to the list of paths. While > >> all of that > >> information may be vitally important to ISV's and LSB and installer > >> GUI's, > >> all that rpmlib needs is NEVRA (N==name, E==epoch, etc), and > >> mostly for > >> human identification rather than installer functioning purposes. > > > > Not sure if epoch versioning is important for third-party software, > > if I > > understand correctly it is more of a disto tool for changing package > > names etc. It might be safe to set always set the epoch to some > > default > > value. But maybe it also makes sense, you may know a good use case for > > it. > > > > I ignored the revision and set it to 1, but revisions could be quite > > handy for ISVs too. > > > > I speak in RPM NEVRA jargon, apologies. > > Whether LSB "version" contains an Epoch: (or not) simply does not > matter. > > Always having Epoch: 0 (or equivalently, never including RPMTAG_EPOCH) > are all that is needed for identification purposes of "packages" > using RPM target > metadata. I think Epoch:0 for all packages would be OK. I don't see where third-party app vendors would really need epochs. > In fact, "version" and even "name" can be synthesized if/when/where > necessary. Presumably > human lusers need more than "" as an identification tag. The "" > string in RPMTAG_NAME > and RPMTAG_VERSION etc is more than adequate to prevent rpmdb disasters. As already stated, "version" needs some specified format and a guaranteed comparison scheme (as package managers seem to handle this differently in some cases). IMHO the package name should be as defined in the LSB, that is, lsb--. > But clearly better needs to be specified with "version" and "upgrade" > and ... > > >> Is a find(1) path list "gud enuf" as a starting point? Or do you want > >> to establish > >> other, alternative, markup for expressing the necessary data > >> elements. > > > > If you mean what I thought you meant, that would be OK. And another > > question: do you mean to take the _data_ that is in a find(1) path > > list, > > or also its _format_, abadoning the XML representation? The current > > format is already a path list with some metadata added. > > > >> Other obviously complete and unsurprising candidates to describe > >> necessary > >> data elements to be included in target metadata are "tar tvf" and/or > >> "ls -al". > >> Those formats are explicit, no data is implicitly derived from stat > >> (2) of a
Re: LSB Package API
On Jun 25, 2008, at 10:45 AM, Denis Washington wrote: On Wed, 2008-06-25 at 10:12 -0400, Jeff Johnson wrote: On Jun 24, 2008, at 1:38 PM, Denis Washington wrote: Sound like a plan? My primary goals here are two-fold: 1) avoiding disasters if bogus headers start to be added to an rpmdb. 2) exposing rpmdbAdd() (and rpmdbRemove()) methods for use by LSB/ISV/whatever applications that wish to register/unregister software on RPM managed systems. Sounds like a good plan, yeah. I'm glad being able to work with you on this, as you certainly have a LOT more experience than me concerning this. Thank you very much! No problem. Enumerating the necessary data elements that need to be present in a RPM header, and choosing _SOME_ representational markup, would seem to be on the critical path. (aside) dpkg its really the same fundamental problem, but a different target metadata representation. Ditto _your_package_manager_here for all instances of class. There are several existing representations of "package" manifests, both explicit and/or implicit that can be used to enumerate the necessary data elements to be included in the target metadata representation (note I did not say "rpmdb"). Simplest by far is find(1) output of a tree. i.e. an explicit list of paths to files, with stat(2) and digest (and acls/xattrs and selinux file contexts and whatever else is needed) implicitly derived from the tree. With "implicitly derived", do you mean "read from the installed files instead of being explicitly in the manifest file"? Yes. Basically I mean populating target metadata with stat(2) info, not with explicitly parsed values. The advantage is KISS: it don't come any simpler (for ISV's and other lusers) than providing a file manifest. The disadvantage of a KISS file manifest is that indeed, the files must be 1) actually present (and available) on a file system 2) correctly installed. Presumably the ISV (or other installer) is functional, or the ISV (or other installer) would not be trying to register a "package", would it? Other soft "branding" identification information, like vendor, packager, description, build host, etc would need to be added to the list of paths. While all of that information may be vitally important to ISV's and LSB and installer GUI's, all that rpmlib needs is NEVRA (N==name, E==epoch, etc), and mostly for human identification rather than installer functioning purposes. Not sure if epoch versioning is important for third-party software, if I understand correctly it is more of a disto tool for changing package names etc. It might be safe to set always set the epoch to some default value. But maybe it also makes sense, you may know a good use case for it. I ignored the revision and set it to 1, but revisions could be quite handy for ISVs too. I speak in RPM NEVRA jargon, apologies. Whether LSB "version" contains an Epoch: (or not) simply does not matter. Always having Epoch: 0 (or equivalently, never including RPMTAG_EPOCH) are all that is needed for identification purposes of "packages" using RPM target metadata. In fact, "version" and even "name" can be synthesized if/when/where necessary. Presumably human lusers need more than "" as an identification tag. The "" string in RPMTAG_NAME and RPMTAG_VERSION etc is more than adequate to prevent rpmdb disasters. But clearly better needs to be specified with "version" and "upgrade" and ... Is a find(1) path list "gud enuf" as a starting point? Or do you want to establish other, alternative, markup for expressing the necessary data elements. If you mean what I thought you meant, that would be OK. And another question: do you mean to take the _data_ that is in a find(1) path list, or also its _format_, abadoning the XML representation? The current format is already a path list with some metadata added. Other obviously complete and unsurprising candidates to describe necessary data elements to be included in target metadata are "tar tvf" and/or "ls -al". Those formats are explicit, no data is implicitly derived from stat (2) of a file, and the file does not have to exist in order to construct a representation of target metadata. I would go with the simple path list. With explicit stat data etc, we run into the problem that the data in the manifest might run out of sync with the installed files (as the files may change them during install). Implicit stat data also means less changes in existing installers, which most likely already do chmod's etc. I hear "simple path list". Yes there are many issues with implicit file metadata, all well known. No matter what, a simple file list is the bare minimum expression of target metadata. Without file paths, one has only disk blocks for ISV's to sell. I undersand that some disk manufacturer had a monoply selling disk blocks 15 years ago ... (aside) that's a very dry & obscure joke, don't worry if it makes
Re: LSB Package API
On Wed, 2008-06-25 at 10:12 -0400, Jeff Johnson wrote: > On Jun 24, 2008, at 1:38 PM, Denis Washington wrote: > > > > > > >> Sound like a plan? My primary goals here are two-fold: > >> > >> 1) avoiding disasters if bogus headers start to be added to an rpmdb. > >> > >> 2) exposing rpmdbAdd() (and rpmdbRemove()) methods for use by > >> LSB/ISV/whatever applications that wish to register/unregister > >> software > >> on RPM managed systems. > > > > Sounds like a good plan, yeah. I'm glad being able to work with you on > > this, as you certainly have a LOT more experience than me concerning > > this. Thank you very much! > > > > No problem. > > Enumerating the necessary data elements that need to be present > in a RPM header, and choosing _SOME_ representational markup, > would seem to be on the critical path. > > (aside) dpkg its really the same fundamental problem, but a different > target metadata representation. Ditto _your_package_manager_here > for all instances of class. > > There are several existing representations of "package" manifests, > both explicit and/or implicit that can be used to enumerate the > necessary data elements to be included in the target metadata > representation > (note I did not say "rpmdb"). > > Simplest by far is find(1) output of a tree. i.e. an explicit list > of paths to files, with stat(2) and digest (and acls/xattrs and selinux > file contexts and whatever else is needed) implicitly derived from > the tree. With "implicitly derived", do you mean "read from the installed files instead of being explicitly in the manifest file"? > Other soft "branding" identification information, like vendor, > packager, description, > build host, etc would need to be added to the list of paths. While > all of that > information may be vitally important to ISV's and LSB and installer > GUI's, > all that rpmlib needs is NEVRA (N==name, E==epoch, etc), and mostly for > human identification rather than installer functioning purposes. Not sure if epoch versioning is important for third-party software, if I understand correctly it is more of a disto tool for changing package names etc. It might be safe to set always set the epoch to some default value. But maybe it also makes sense, you may know a good use case for it. I ignored the revision and set it to 1, but revisions could be quite handy for ISVs too. > Is a find(1) path list "gud enuf" as a starting point? Or do you want > to establish > other, alternative, markup for expressing the necessary data elements. If you mean what I thought you meant, that would be OK. And another question: do you mean to take the _data_ that is in a find(1) path list, or also its _format_, abadoning the XML representation? The current format is already a path list with some metadata added. > Other obviously complete and unsurprising candidates to describe > necessary > data elements to be included in target metadata are "tar tvf" and/or > "ls -al". > Those formats are explicit, no data is implicitly derived from stat > (2) of a file, > and the file does not have to exist in order to construct a > representation > of target metadata. I would go with the simple path list. With explicit stat data etc, we run into the problem that the data in the manifest might run out of sync with the installed files (as the files may change them during install). Implicit stat data also means less changes in existing installers, which most likely already do chmod's etc. > But there's lots and lots of other markups that could/should be used > instead. > > What representation of target metadata works for you? >From the content, find(1) path lists would be the best IMHO. We could also take its representation (that is, a file with newline-separated files with somehow marked up metadata in front), but I think XML is pretty nice because it is well-defined and relatively easy to parse. Note that the backends don't have to deal with the manifest file format as they already get the parsed binary representation of the manifest. Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 24, 2008, at 1:38 PM, Denis Washington wrote: Sound like a plan? My primary goals here are two-fold: 1) avoiding disasters if bogus headers start to be added to an rpmdb. 2) exposing rpmdbAdd() (and rpmdbRemove()) methods for use by LSB/ISV/whatever applications that wish to register/unregister software on RPM managed systems. Sounds like a good plan, yeah. I'm glad being able to work with you on this, as you certainly have a LOT more experience than me concerning this. Thank you very much! No problem. Enumerating the necessary data elements that need to be present in a RPM header, and choosing _SOME_ representational markup, would seem to be on the critical path. (aside) dpkg its really the same fundamental problem, but a different target metadata representation. Ditto _your_package_manager_here for all instances of class. There are several existing representations of "package" manifests, both explicit and/or implicit that can be used to enumerate the necessary data elements to be included in the target metadata representation (note I did not say "rpmdb"). Simplest by far is find(1) output of a tree. i.e. an explicit list of paths to files, with stat(2) and digest (and acls/xattrs and selinux file contexts and whatever else is needed) implicitly derived from the tree. Other soft "branding" identification information, like vendor, packager, description, build host, etc would need to be added to the list of paths. While all of that information may be vitally important to ISV's and LSB and installer GUI's, all that rpmlib needs is NEVRA (N==name, E==epoch, etc), and mostly for human identification rather than installer functioning purposes. Is a find(1) path list "gud enuf" as a starting point? Or do you want to establish other, alternative, markup for expressing the necessary data elements. Other obviously complete and unsurprising candidates to describe necessary data elements to be included in target metadata are "tar tvf" and/or "ls -al". Those formats are explicit, no data is implicitly derived from stat (2) of a file, and the file does not have to exist in order to construct a representation of target metadata. But there's lots and lots of other markups that could/should be used instead. What representation of target metadata works for you? 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
Hi, First of all, your nicest post so far. Thanks. ;) On Sun, 2008-06-22 at 11:25 -0400, Jeff Johnson wrote: > On Jun 21, 2008, at 12:48 PM, Denis Washington wrote: > > > >> > My current interest in your code is disaster prevention, not > otherwise. > >>> > >>> I welcome any motive if it improves code quality, so thanks > >>> anyway. ;) > >>> > >> > >> NP. My life is hell when rpmdb's get hosed up. Doesn't matter whether > >> its a kernel mmap(2) flaw, an selinux policy-of-the-day typo, or a > >> python > >> script kiddies dain bramage. > >> > >> The "Berlin API" is a recipe for disaster so far. > >> > >> But I'm most definitely deeply and personally interested in not > >> having to do the necessary rpmdb maintenance post-mortem > >> if the implementation problems can be solved. > > > > Thought so. > > > > Hehe, now that your ideas have been thoroughly shredded ... > > Here's what is right about the Berlin API, and your approach: > > 0) A means to register/unregister a "container" (I'm avoiding the > abused term > "package") of content on linux systems that is more lightweight and > more flexible > than existing containers like rpm/dpkg needs to be devised. > > 1) The "Berlin API" -- for all its flaws -- is a reasonable step > towards the > goal of registering/unregistering content on Linux systems. The largest > flaw with the "Berlin API" LSB proposal imho was stating the problem as > API methods without specifying the "container" internals, and how the > internals > should be represented and used. Implementing an API also involves > practical > development choices, like what language to use, that are incidental to > the goal or registering/unregistering content. I hoped I could get around the language choice using D-Bus actually. But as there is just a convienience library for C currently, that's still a problem, somehow. > 2) Your implementation of the "Berlin API" got lots of things right. > First of all, > the API itself in your implementation is in Yet Another Library, > which avoids > he hugely complicated problem of how to retrofit an API onto already > released > software. Second, your implementation exists, a necessary precursor > to identifying > what else needs to be done. Making an implementation of the "Berlin API" > exist in some meaningful form is more than anyone at LSB or on the > packaging > has achieved in ~1.5 years. Using dbus, and splitting the API from > back-end > processors using a domain-specific markup, are also sound architectural > choices imho. > > So Congratulations! are most definitely in order. Thanks. > If you want to proceed, I'll be happy to write the RPM back-end for you. > > Its easier for me to just build a Header that I know will Just Work > (tm) when > passed to rpmdbAdd() than it is to explain to you what needs to be done. > So far, your "Berlin API" implementation is sufficiently complete that > I can see that a well-formed Header can be achieved. OTOH, you > will have to supply the necessary data elements to add to the header, > I can give you a list if you wish to proceed. That would be great. Although there a some things that aren't there on purpose currently. One example is file permissions; an installer should be able to adjust its installed files as it wishes and let ClosePackage() gather the permission information. Such things could also be in the package manifest files though if this is better. > (aside) Note that I'm quite capable of generating digital signatures > on the generated > header from the "Berlin API" any time that the "trust" discussions > get out of hand. Great. I guess the whole "trust" and security issue will need to be discussed thoroughly in the near future. > I can also likely help generate the XML (or YAML or klik or 0install or > repo-md or ...) markup that is going to be needed to express the > "container" contents. I'm actively generating markup of various > persuasions > from Header content @rpm5.org already. One more (or less) markup > implementation > simply doesn't matter, just try to get the markups prioritized please. > > What does matter is that all the widdle markups interconvert easily, and > are sufficiently rich in expressive power that common elements, like > file stat(2) data, or ACL's or xattr's or ... can be represented (and > converted) > without hugely complicated political packaging war battles that are > ultimately > about cellophane "container" wrappers. Makes sense. > Sound like a plan? My primary goals here are two-fold: > > 1) avoiding disasters if bogus headers start to be added to an rpmdb. > > 2) exposing rpmdbAdd() (and rpmdbRemove()) methods for use by > LSB/ISV/whatever applications that wish to register/unregister software > on RPM managed systems. Sounds like a good plan, yeah. I'm glad being able to work with you on this, as you certainly have a LOT more experience than me concerning this. Thank you very much! Regards, Denis _
Re: LSB Package API
On Tue, 2008-06-24 at 12:03 +0100, Richard Hughes wrote: > On Sun, 2008-06-22 at 20:02 +0200, Denis Washington wrote: > > We shouldn't resignate just because nothing came out of the previous > > attempts. Also, the LSB Package API is designed to require as little > > adjustments as possible to installers - it's just to calls and a single > > file, after all. > > Uses a DBUS service: check > Uses pluggable backends: check > Use PolicyKit: check > Use an XML parser: check > System activation: check > Define own linked list implementation: check I don't know where you a heading. The D-Bus service, pluggable backends, the XML parser, and system activation are all things that installers don't have to deal with. They just use the few functions from liblsb_package. > > As already mentioned before in this thread, the focus of PackageKit and the > > LSB Package API are quite different, so there is no big reason for them to > > not exist side-by-side. > > Err, http://www.linuxfoundation.org/en/LSB_Package_API suggests > otherwise. > > You've got calls to PolicyKit, a system activated daemon, pluggable > backends - you might as well call the project LSBPackageKit. You don't > appear to have any defined scope for the project and it seems to be just > be technology-bingo at the moment. Just because it does use the same technologies, that doesn't mean the APIs' scope is the same. You should know enough about your project to realize that the LSB Package API is focused on entirely different needs (providing an interface for third-party app installers) than PackageKit (provide an API for the packaging system, based on distro repositories). > > I don't think this is a corner case at all. For one thing, propietary > > applications might just don't play a role _because_ there is no really > > good distribution method for them - the typical chicken-and-egg problem. > > (I'm not saying this is the only reason, but an important one.) We're > > just not giving them an easy method of cross-distro integration. I think > > providing this is important. > > Have you talked to customers? I have. Lots of them. Customers don't want > DBUS services or PolicyKit, they want one of two things: > > 1. A tested (supported) binary package for something like RHEL and SLED. > 2. An installer that uses something like /bin/sh for the other distros. Again, ISVs don't have to deal with D-BUS etc. Those are _implementation details_. They can just use a simple C API which could also be easily wrapped into simple command-line tools. > If you want them to use a library to install stuff, you better make it > static (else they have to depend on really new versions of distros) and > also make it very lightweight, libc type stuff. Most of this closed > source stuff has to install on distros 5 years old, and continue to work > on distros 2 years in the future. The LSB Package API would only be in newer versions of the LSB, so support of legacy distros is not that high on the list. (On any older distro, no one could rely on the API even being there.) > > Second, this way of distribution will help open-source projects as well. > > It would make it really easy for them to distribute bleeding-edge > > versions of there apps that integrate well into the packaging system > > without having to package for each and every package manager. > > Talk to the distro maintainers. They really don't want random projects > replacing supported packages. Packages are not normally just the > upstream tarball with a spec file - normally the packager includes spec > files to make the package compile, or integrate well with the distro. > Then there's the world of pain that comes from security errata. No packages are going to be replaced. LSB applications are required to install to /opt, so nothing is overridden. Even the package naming won't clash (it's "lsb--" in the implemented RPM and DPKG backends). > I really think you should talk to distro maintainers as well as closed > source vendors before coming up with any more API. A number of ISVs have already been talked to; see the comments from Jeff Licquia. Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 24, 2008, at 7:03 AM, Richard Hughes wrote: I really think you should talk to distro maintainers as well as closed source vendors before coming up with any more API. Are you referring to LSB or to Denis? The "Berlin API" was proposed by Jeff Licquia, not Denis, here https://www.linuxfoundation.org/en/Berlin_Packaging_API And we're talking 3 methods in the API, this is hardly a radical or disturbing API: bool compare_dependency(const char *package_name, relation_t relationship, const char *version) bool register_package(const char *package_name, const char *version, manifest_t manifest) bool unregister_package(const char *package_name)
Re: LSB Package API
On Sun, 2008-06-22 at 20:02 +0200, Denis Washington wrote: > We shouldn't resignate just because nothing came out of the previous > attempts. Also, the LSB Package API is designed to require as little > adjustments as possible to installers - it's just to calls and a single > file, after all. Uses a DBUS service: check Uses pluggable backends: check Use PolicyKit: check Use an XML parser: check System activation: check Define own linked list implementation: check > As already mentioned before in this thread, the focus of PackageKit and the > LSB Package API are quite different, so there is no big reason for them to > not exist side-by-side. Err, http://www.linuxfoundation.org/en/LSB_Package_API suggests otherwise. You've got calls to PolicyKit, a system activated daemon, pluggable backends - you might as well call the project LSBPackageKit. You don't appear to have any defined scope for the project and it seems to be just be technology-bingo at the moment. > I don't think this is a corner case at all. For one thing, propietary > applications might just don't play a role _because_ there is no really > good distribution method for them - the typical chicken-and-egg problem. > (I'm not saying this is the only reason, but an important one.) We're > just not giving them an easy method of cross-distro integration. I think > providing this is important. Have you talked to customers? I have. Lots of them. Customers don't want DBUS services or PolicyKit, they want one of two things: 1. A tested (supported) binary package for something like RHEL and SLED. 2. An installer that uses something like /bin/sh for the other distros. If you want them to use a library to install stuff, you better make it static (else they have to depend on really new versions of distros) and also make it very lightweight, libc type stuff. Most of this closed source stuff has to install on distros 5 years old, and continue to work on distros 2 years in the future. > Second, this way of distribution will help open-source projects as well. > It would make it really easy for them to distribute bleeding-edge > versions of there apps that integrate well into the packaging system > without having to package for each and every package manager. Talk to the distro maintainers. They really don't want random projects replacing supported packages. Packages are not normally just the upstream tarball with a spec file - normally the packager includes spec files to make the package compile, or integrate well with the distro. Then there's the world of pain that comes from security errata. I really think you should talk to distro maintainers as well as closed source vendors before coming up with any more API. Richard. __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
[Commenting on this thread is a disaster; different people strip off different mailing lists, and parts of the conversation are happening everywhere. Can we all agree, at least, to keep the packaging list in followups?] Richard Hughes wrote: Being blunt, no distro is going to support a LSB package API. In 2006, representatives from Red Hat, SuSE/Novell, Debian, and Ubuntu committed in principle to doing just such an API once it was done. Of course, that's not a guarantee, but it holds a little more weight, I think, than the above quote. At that 2006 meeting (December, in Berlin, thus the "Berlin API" name), these representatives from the distros were told by numerous ISVs why distro package systems were not acceptable. The Berlin API was a compromise; the idea was that third-party software installers and package managers would be able to communicate and cooperate. Part of the issue may be that most of the implementations so far have assumed that communication from a third-party installer would result in a pseudo-package being registered in the native package database, which leads people to believe that this is a "new package format" of some kind. The original idea, though, was for a communication protocol only. The native package manager may decide to store the results by creating a pseudo-package, but does not *have* to. I think we're willing to accept that the particular implementations of the Berlin API idea are wrong-headed, and perhaps re-do them. But the general idea--accepting that things such as InstallShield and InstallAnywhere are going to exist, and finding a way for them to cooperate with the underlying system instead of fighting with it--isn't something I see anyone else trying to address. __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: [packaging] LSB Package API
On Jun 23, 2008, at 3:43 PM, Jeff Licquia wrote: Part of the issue may be that most of the implementations so far have assumed that communication from a third-party installer would result in a pseudo-package being registered in the native package database, which leads people to believe that this is a "new package format" of some kind. The original idea, though, was for a communication protocol only. The native package manager may decide to store the results by creating a pseudo-package, but does not *have* to. I think we're willing to accept that the particular implementations of the Berlin API idea are wrong-headed, and perhaps re-do them. But the general idea--accepting that things such as InstallShield and InstallAnywhere are going to exist, and finding a way for them to cooperate with the underlying system instead of fighting with it-- isn't something I see anyone else trying to address. Speaking as the developer of an installer that has to fight with this all the time, all I'm really looking for is a simple command-line utility to interface to the native package manager beneath me. A simple abstraction layer in the style of the xdg-utils of the Portland project. Something that didn't require root would be nice, but I'm not sure how you'd handle this on a multi-user system. As it is now, I have to manipulate the underlying packaging system by-hand and through fake packages built at runtime and the like. It's crap, but it's the only outlet available until something better comes along. I guess I don't understand what is so difficult about the decision except that everyone has a better way than the other guy. Make something simple that gets the job done. Believe me, plenty of people will come along later and glom more crap onto it. It's open source, after all. Damon __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: [packaging] LSB Package API
On Sun, Jun 22, 2008 at 01:34:31PM -0700, Dan Kegel wrote: > On Sun, Jun 22, 2008 at 11:02 AM, Denis Washington <[EMAIL PROTECTED]> wrote: > > I don't think this is a corner case at all. For one thing, propietary > > applications might just don't play a role _because_ there is no really > > good distribution method for them - the typical chicken-and-egg problem. > > (I'm not saying this is the only reason, but an important one.) We're > > just not giving them an easy method of cross-distro integration. I think > > providing this is important. > > Sure, and that's why I support the LSB. > Has everybody else given up on it? I've probably missed part of this discussion, but wanted to inject one anecdote. Stand-alone binary package installers are nothing particular novel or new; I'd gained experience using one, Autopackage, with Inkscape several years ago. Inkscape was virtually unknown at the time, so Autopackage gave us a significant benefit of providing users a way to quickly get Inkscape on distros that didn't yet include Inkscape. Before Autopackage we would maintain our own .deb and .rpm rules and specs, and we hoped Autopackage would obsolete that in favor of having a "Universal Installer". Yet that really never came to be. First, Inkscape became recognized by distros, and they took over handling our packaging work for us. Meanwhile, the Autopackage developers (who had been subsidizing us by maintaining the .autopackage file for us), turned maintenance over to us. So on the one hand we were seeing our team workload *reduce* by relying on packaging-system-specific stuff, and *increase* by using autopackage. You might argue that it's different for proprietary software since distros wouldn't adopt them. Yet look at Xara, Flash, Opera, proprietary Java, and so on to see that they can and do (to the level they're able anyway). Second, while in theory Autopackage promised "100% easy install, anywhere", it was not without problems. The issue always seemed to be with low level dependencies that varied just subtly enough to break on one distro or another. So you ended up doing per-distro testing anyway (and couldn't count on help from the distro once you figured out the bug). I think this is an important point to not dismiss by saying, "The design wasn't as good as ours is", or "it just needed more testing", or whatever. The sad truth is that getting this to work 100% requires invalidating Murphy's Law. It's a broad fronted fight against entropy. I felt like we had tried to change our support from N distros to 1, yet ended up with N+1. Third, as Inkscape grew we had to account for more dependencies. Typically, there'd already be .rpm and .deb packages for them, but we'd be stuck having to do the autopackage packaging work ourselves. Now, you might think such an issue is irrelevant for proprietary software since they'd be packaged with all dependencies already included. Yet consider dependencies beyond just dynamic libraries. Consider if the app wants to interface with external programs or tools (imagemagick, java, sqlite, ...) or to shared data repositories (openclipart, fonts, etc.) Eventually, you find yourself having to do a lot of work that distros already take care of. Anyway, to sum up, as much as I loved the idea of autopackage and helped to advocate it, I really don't think the idea of a "universal installer" is viable. In the end it's a lot less effort to just collaborate with each of the distros and have packaging optimized for each. And I think efforts put into creating yet more universal installer techs maybe better invested in helping bring the existing packaging systems better into consistency with one another, or establishing "best practices" documents. Bryce __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
>> I think you are looking for a solution to a problem that doesn't exist. >> For the corner cases of where this does apply (proprietary software) >> this is not enough of a use case to justify all the work required. > >I don't think this is a corner case at all. For one thing, propietary >applications might just don't play a role _because_ there is no really >good distribution method for them - the typical chicken-and-egg problem. >(I'm not saying this is the only reason, but an important one.) We're >just not giving them an easy method of cross-distro integration. I think >providing this is important. > >Second, this way of distribution will help open-source projects as well. >It would make it really easy for them to distribute bleeding-edge >versions of there apps that integrate well into the packaging system >without having to package for each and every package manager. It will >also help those projects which are not widely used enough to be in >distro repos, as they can more easily release binary distributions >without having to depend on getting packaged by the distros. I really >believe this decentralism would be very nice to have, especially in >something as naturally decentralized as the open source community. > Hi, I'd agree with Richard, because helping ISV won't automatically help Debian community to improve itself or its product : the Debian distro. Helping ISV, in a way that can benefit to all is to provide a very good, and solid-rock, basis for all to package software, respectful of LSB and knowledgeable in UNIX :). ISV may benefits from an improved packaging infrastructure but we would not benefits from them : we do not care about them at all :). So we want to care about invest time for those fishes... Sure, I'd like to have a Debian package (updated monthly) for IBM/Rational ClearCase (though I'd like to use Aegis...), for NVIDIA Gelato or for Autodesk Maya But we don't want to invest time before they take effort in packaging they stuff ;) I'm used to comply with very heterogeneous environment as Linux + HP-UX + Solaris + Window$ ... that's crazy... Common drawbacks in packaging approaches are, usually: - poor asserts of target particularities, - lack of UNIX knowledge, - (very, very) bad scripting (all to often are very bad *csh* scripts for example). IMHO, to improve such things, to help design install scripts, or install schema, in a way that can fit in UNIX / Debian philosophy, we can produce documentation that: - respect LSB hierarchy so that files goes where we want them to go, /etc/init.d [debian]/etc/default/ /usr/bin /usr/bin /var/share/ /var/lib/ - help in admin files leveraging (portability) : init.d scripts, cron scripts, ... - help in libs dependencies (& symbols) : if ISV package depends on libjpeg6. then we must provides ways in packaging system to assert on this dep() : with the new *dpkg-shlibdeps* we can tracks such symbol dependencies... - help document the new *dpkg* feature of "triggers", this might help... - and last but not the least : KISS ; why use DBUS !!! when all Debian packaging tools use either Bash either Perl ? why complexify things ? that's the root of all evil : To conclude: the best way is to document, and eventually to implement helper scripts. Not to imagine API that would become obsolete in a month of two... Figure out where ISV script are not able to cope with Debian system and offer solution for that problems, specifically :). Best regards, Michel -- .''. -our own. Resistance is futile. __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sun, 2008-06-22 at 14:15 +0100, Richard Hughes wrote: > On Sat, 2008-06-21 at 11:51 +0200, Denis Washington wrote: > > Some time ago, it was discussed on an LSB face-to-face meeting that an > > API should be developed that allows ISVs to install sotware packages > > which integrate into the package manager - the "Berlin Packaging API". > > While the idea seemed to be well received, there didn't seem much > > progress since then, except for a wiki page with a rundimentary proposal > > [1]. Considering that third-party software installation is an undeniably > > important weak spot of the Linux infrastructure, I found this was a > > shame. > > Sure, it's been tried before a few times before and fallen on it's face > each time. There's a reason that PackageKit uses the distro supplied > packages, rather than trying to spin it's own thing. We shouldn't resignate just because nothing came out of the previous attempts. Also, the LSB Package API is designed to require as little adjustments as possible to installers - it's just to calls and a single file, after all. > > To reignite the the initiative, I decided to design and develop a > > prototype implementation of the Berlin API, most creatively named the > > "LSB Package API". It is designed as a simple D-Bus interface > > accompanied with an XML-based package description format. A detailed > > description and the source code can be found on this page: > > > > http://www.linuxfoundation.org/en/LSB_Package_API > > Sounds quite like PackageKit, which has been developed for the last year > or so with buy-in from most of the big distros. PackageKit doesn't try > to replace the entire stack, only make some sort of abstraction for GUI > tools where such an abstraction makes sense. > As already mentioned before in this thread, the focus of PackageKit and the LSB Package API are quite different, so there is no big reason for them to not exist side-by-side. > Being blunt, no distro is going to support a LSB package API. To me, a > distro is basically: > > community+trust+infrastructure > > If you take away the trust (letting other people create and sign > packages), the infrastructure (letting other people host packages and > manage security errata) and the community (basically reduced to PR) > you've got nothing left. > > The cost of a distro rolling it's own packages is trivial considering > the advantages of the vendor trust model, with a single infrastructure > and support. The distro model is nice (and arguably better than the LSB Package API) if the packages you like to have are in the repos in sufficiently new versions. But if you need to go past that (bleeding edge versions, not widely spread apps), things get more difficult currently. Especially propietary applications just cannot be distributed by the distros directly. > > The implementation currently supports integration into RPM and dpkg; due > > to its modular nature, support for more package managers could be added > > later on. > > Looks like you've not considered localisation, multi-lib, multiple > versions of packages, or any of the problems solved with solutions like > NEVRAs. I would suggest to read the rpm(+yum), dpkg and conary sources > before you even start to propose APIs. Naturally some things are not considered yet. That's why I called the spec and implementation I released a "starting point", not the completely finished thing ready for production. There are quite a few points that will have to be thought through and added, but I don't think it's impossible to do so on top of the current basic design. > > I hope this implementation will act as a starting point for resurrecting > > the Berlin API process. Let us overcome the "Third-party software > > installation on Linux sucks" problem and strive to a brave new world of > > easily distributable Linux software! ;) > > I think you are looking for a solution to a problem that doesn't exist. > For the corner cases of where this does apply (proprietary software) > this is not enough of a use case to justify all the work required. I don't think this is a corner case at all. For one thing, propietary applications might just don't play a role _because_ there is no really good distribution method for them - the typical chicken-and-egg problem. (I'm not saying this is the only reason, but an important one.) We're just not giving them an easy method of cross-distro integration. I think providing this is important. Second, this way of distribution will help open-source projects as well. It would make it really easy for them to distribute bleeding-edge versions of ther
Re: LSB Package API
On Jun 21, 2008, at 12:48 PM, Denis Washington wrote: My current interest in your code is disaster prevention, not otherwise. I welcome any motive if it improves code quality, so thanks anyway. ;) NP. My life is hell when rpmdb's get hosed up. Doesn't matter whether its a kernel mmap(2) flaw, an selinux policy-of-the-day typo, or a python script kiddies dain bramage. The "Berlin API" is a recipe for disaster so far. But I'm most definitely deeply and personally interested in not having to do the necessary rpmdb maintenance post-mortem if the implementation problems can be solved. Thought so. Hehe, now that your ideas have been thoroughly shredded ... Here's what is right about the Berlin API, and your approach: 0) A means to register/unregister a "container" (I'm avoiding the abused term "package") of content on linux systems that is more lightweight and more flexible than existing containers like rpm/dpkg needs to be devised. 1) The "Berlin API" -- for all its flaws -- is a reasonable step towards the goal of registering/unregistering content on Linux systems. The largest flaw with the "Berlin API" LSB proposal imho was stating the problem as API methods without specifying the "container" internals, and how the internals should be represented and used. Implementing an API also involves practical development choices, like what language to use, that are incidental to the goal or registering/unregistering content. 2) Your implementation of the "Berlin API" got lots of things right. First of all, the API itself in your implementation is in Yet Another Library, which avoids he hugely complicated problem of how to retrofit an API onto already released software. Second, your implementation exists, a necessary precursor to identifying what else needs to be done. Making an implementation of the "Berlin API" exist in some meaningful form is more than anyone at LSB or on the packaging has achieved in ~1.5 years. Using dbus, and splitting the API from back-end processors using a domain-specific markup, are also sound architectural choices imho. So Congratulations! are most definitely in order. If you want to proceed, I'll be happy to write the RPM back-end for you. Its easier for me to just build a Header that I know will Just Work (tm) when passed to rpmdbAdd() than it is to explain to you what needs to be done. So far, your "Berlin API" implementation is sufficiently complete that I can see that a well-formed Header can be achieved. OTOH, you will have to supply the necessary data elements to add to the header, I can give you a list if you wish to proceed. (aside) Note that I'm quite capable of generating digital signatures on the generated header from the "Berlin API" any time that the "trust" discussions get out of hand. I can also likely help generate the XML (or YAML or klik or 0install or repo-md or ...) markup that is going to be needed to express the "container" contents. I'm actively generating markup of various persuasions from Header content @rpm5.org already. One more (or less) markup implementation simply doesn't matter, just try to get the markups prioritized please. What does matter is that all the widdle markups interconvert easily, and are sufficiently rich in expressive power that common elements, like file stat(2) data, or ACL's or xattr's or ... can be represented (and converted) without hugely complicated political packaging war battles that are ultimately about cellophane "container" wrappers. Sound like a plan? My primary goals here are two-fold: 1) avoiding disasters if bogus headers start to be added to an rpmdb. 2) exposing rpmdbAdd() (and rpmdbRemove()) methods for use by LSB/ISV/whatever applications that wish to register/unregister software on RPM managed systems. 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, 2008-06-21 at 11:51 +0200, Denis Washington wrote: > Some time ago, it was discussed on an LSB face-to-face meeting that an > API should be developed that allows ISVs to install sotware packages > which integrate into the package manager - the "Berlin Packaging API". > While the idea seemed to be well received, there didn't seem much > progress since then, except for a wiki page with a rundimentary proposal > [1]. Considering that third-party software installation is an undeniably > important weak spot of the Linux infrastructure, I found this was a > shame. Sure, it's been tried before a few times before and fallen on it's face each time. There's a reason that PackageKit uses the distro supplied packages, rather than trying to spin it's own thing. > To reignite the the initiative, I decided to design and develop a > prototype implementation of the Berlin API, most creatively named the > "LSB Package API". It is designed as a simple D-Bus interface > accompanied with an XML-based package description format. A detailed > description and the source code can be found on this page: > > http://www.linuxfoundation.org/en/LSB_Package_API Sounds quite like PackageKit, which has been developed for the last year or so with buy-in from most of the big distros. PackageKit doesn't try to replace the entire stack, only make some sort of abstraction for GUI tools where such an abstraction makes sense. Being blunt, no distro is going to support a LSB package API. To me, a distro is basically: community+trust+infrastructure If you take away the trust (letting other people create and sign packages), the infrastructure (letting other people host packages and manage security errata) and the community (basically reduced to PR) you've got nothing left. The cost of a distro rolling it's own packages is trivial considering the advantages of the vendor trust model, with a single infrastructure and support. > The implementation currently supports integration into RPM and dpkg; due > to its modular nature, support for more package managers could be added > later on. Looks like you've not considered localisation, multi-lib, multiple versions of packages, or any of the problems solved with solutions like NEVRAs. I would suggest to read the rpm(+yum), dpkg and conary sources before you even start to propose APIs. > I hope this implementation will act as a starting point for resurrecting > the Berlin API process. Let us overcome the "Third-party software > installation on Linux sucks" problem and strive to a brave new world of > easily distributable Linux software! ;) I think you are looking for a solution to a problem that doesn't exist. For the corner cases of where this does apply (proprietary software) this is not enough of a use case to justify all the work required. From somebody that's spent the best part of a year researching the packaging formats and distribution of metadata, I don't see such a LSB package API as a viable project. Richard Hughes (PackageKit maintainer) __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 21, 2008, at 9:50 PM, Wichmann, Mats D wrote: LSB has chosen to leave "upgrade" UNSPECIFIED, and has also chose in the "Berlin API" to ignore the fact that both dpkg/rpm versions are a triple of Epoch/Version/Release. Pretending that a "version" string can be anything, opaquely handled, including E:V-R, or something else, misses the issue that "upgrade" based on "version" is undecidable until "version" is well formed, and a collate sequence is defined for "upgrade" comparison. the absence of any description of what "version" means is a bug in LSB, whether or not that issue is picked up by the Berlin proposal. upgrade is a little dicier in the LSB sense, as it seems different packaging systems may do quite different things here. Responding to that by pretending upgrades don't exist is the cowardly approach, I know... Leaving "version" as an unspecified opaque string, but pointing out that there are two obvious example usage cases, dpkg & rpm, that have chosen to use E:V-R, would seem to take perhaps 4 paragraphs (maybe 2 hours of work) in a document. Similarly, "upgrade" can be left to the installer, with the obvious candidates of dpkg & rpm used as examples, and suggestions about ISO major.minor.micro versioning Suggested/Recommended as sensible. For extra credit, add the glibc vercmp routine as an LSB API and a 3rd alternative collation sequence for "upgrade", so that _BOTH_ rpm & dpkg fan boys have something to rant about. And the above is tho no-cojones wussy wriggle room solution. Any movement whatsoever is preferred to the frozen LSB deer staring at the approaching vendor distro headlights ... 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
RE: LSB Package API
> LSB has chosen to leave "upgrade" UNSPECIFIED, > and has also chose in the "Berlin API" to ignore the > fact that both dpkg/rpm versions are a triple of > Epoch/Version/Release. > > Pretending that a "version" string can be anything, opaquely handled, > including E:V-R, or something else, misses the > issue that "upgrade" based on "version" is undecidable > until "version" is well formed, and a collate sequence > is defined for "upgrade" comparison. the absence of any description of what "version" means is a bug in LSB, whether or not that issue is picked up by the Berlin proposal. upgrade is a little dicier in the LSB sense, as it seems different packaging systems may do quite different things here. Responding to that by pretending upgrades don't exist is the cowardly approach, I know... __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, Jun 21, 2008 at 9:46 PM, Jeff Johnson <[EMAIL PROTECTED]> wrote: > > On Jun 21, 2008, at 2:45 PM, devzero2000 wrote: > >> >> Ok. I already know this and also agreed on the motivation. In the meantime >> could be useful >> to have more docu on the rpm4 packaging format, almost for the tags. There >> is some dubt about the semantic of some of these (RPMTAG_SIZE for example >> and %ghost and the like discussed recently) >> >> > There is rpm --xml, true WYSIWIG. > > There is also rpm --yaml, much easier on the eyes. > > And if one looks carefully, one can also see that RPMTAG_FILENAMES > MUST be sorted, and that dependencies SHOULD be sorted (excwpt > when vendors/packagers choose to do something different). > > Without any "standard", more doco just adds to the cacophony of > packaging wars imho. > > A true semantic interpretation of how tags should be used/interpreted is > largely > out of rpm development scope these days. > > Which is also the basis for my opinion that the opportunity > for a "LSB Packaging Standard" to be useful closed several years ago. > > There are way too many RPM differences these days for documentation to > clarify much of anything. > > But YMMV, everyone has their own opinion, easily and obviously understood. > No. I am wrong and you are right: I am finally aware. What is important it is the rpm5 development no other thing. Best Regards Elia > > 73 de Jeff > __ > RPM Package Managerhttp://rpm5.org > LSB Communication Listrpm-lsb@rpm5.org >
Re: LSB Package API
On Jun 21, 2008, at 2:45 PM, devzero2000 wrote: Ok. I already know this and also agreed on the motivation. In the meantime could be useful to have more docu on the rpm4 packaging format, almost for the tags. There is some dubt about the semantic of some of these (RPMTAG_SIZE for example and %ghost and the like discussed recently) There is rpm --xml, true WYSIWIG. There is also rpm --yaml, much easier on the eyes. And if one looks carefully, one can also see that RPMTAG_FILENAMES MUST be sorted, and that dependencies SHOULD be sorted (excwpt when vendors/packagers choose to do something different). Without any "standard", more doco just adds to the cacophony of packaging wars imho. A true semantic interpretation of how tags should be used/interpreted is largely out of rpm development scope these days. Which is also the basis for my opinion that the opportunity for a "LSB Packaging Standard" to be useful closed several years ago. There are way too many RPM differences these days for documentation to clarify much of anything. But YMMV, everyone has their own opinion, easily and obviously understood. 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, Jun 21, 2008 at 8:19 PM, Jeff Johnson <[EMAIL PROTECTED]> wrote: > > On Jun 21, 2008, at 1:52 PM, devzero2000 wrote: > > >> (aside) It is time for LSB RPM SPEC to move to RPM4 packaging format >> >> > Indeed. That is the raison d'etre for <[EMAIL PROTECTED]>. I have not > pursued > because of zero (yes zero!) interest from vendor's or LSB. So it is likely also for Berlin API zero interest because it is based on LSB RPM specs. > Not my problem. I will do a IETF RFC when I get around to it, my forward > looking > develoment goal is XAR, not RPMv4/LSB, format for packaging. Ok. I already know this and also agreed on the motivation. In the meantime could be useful to have more docu on the rpm4 packaging format, almost for the tags. There is some dubt about the semantic of some of these (RPMTAG_SIZE for example and %ghost and the like discussed recently) Best regards > > 73 de Jeff > > __ > RPM Package Managerhttp://rpm5.org > LSB Communication Listrpm-lsb@rpm5.org >
Re: LSB Package API
On Jun 21, 2008, at 1:52 PM, devzero2000 wrote: (aside) It is time for LSB RPM SPEC to move to RPM4 packaging format Indeed. That is the raison d'etre for <[EMAIL PROTECTED]>. I have not pursued because of zero (yes zero!) interest from vendor's or LSB. Not my problem. I will do a IETF RFC when I get around to it, my forward looking develoment goal is XAR, not RPMv4/LSB, format for packaging. 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, 2008-06-21 at 19:52 +0200, devzero2000 wrote: > This was your word in this thread > > "Despite thinking that opinions can hardly be measured in terms of > "correctness", there are enough people that keep flawed opinions for > their entire life without reflecting on after some time. Maybe my > comparatively little experience just gives my the flexibility of mind > that you might be missing after more than ten years. But I was > actually > not planning to start a whose-right flame war." Ah, ok. Excuse my arrogance. I do think, though, that the most experience isn't equal to the "best" opinion. Not that mine is necessarily "better". Arrgh, I hate assigning quality to opinions. Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, Jun 21, 2008 at 7:52 PM, Denis Washington <[EMAIL PROTECTED]> wrote: > On Sat, 2008-06-21 at 13:31 -0400, Jeff Johnson wrote: > > On Jun 21, 2008, at 1:17 PM, Denis Washington wrote: > > > > > On Sat, 2008-06-21 at 13:01 -0400, Jeff Johnson wrote: > > >> > > >> > > >> OTOH, RPMTAG_FILESTATES is gonna matter a _LOT_. So > > >> will leaving stale locks, and forgetting to attach stderr when > > >> your widdle daemon forks. > > > > > > Could you explain what should go in RPM_FILESTATES? It's not listed in > > > the LSB specification. > > > > > > > Zeros are same as RPMFILE_STATE_NORMAL and will suffice. > > > > What is primarily important is that the tag exists (so the pointer > > does not go NULL), > > and that the memory is sized correctly (array of unsigned character > > #files is the dimension). > > RPMFILE_STATE_NORMAL is what most files have attached. > > Ok, thanks. > > > Until one starts to get into multilib, another UNSPECIFIED area > > that will affect ISV's that the LSB "Berlin API" is worfully silent on. > > Unfortunately I don't know about multilibs. > Just for begin http://rpm5.org/community/rpm-lsb/0004.html
Re: LSB Package API
On Sat, Jun 21, 2008 at 7:45 PM, Denis Washington <[EMAIL PROTECTED]> wrote: > On Sat, 2008-06-21 at 19:35 +0200, devzero2000 wrote: > > > > > > On Sat, Jun 21, 2008 at 7:17 PM, Denis Washington > > <[EMAIL PROTECTED]> wrote: > > > > On Sat, 2008-06-21 at 13:01 -0400, Jeff Johnson wrote: > > > On Jun 21, 2008, at 12:48 PM, Denis Washington wrote: > > > > > > > On Sat, 2008-06-21 at 12:27 -0400, Jeff Johnson wrote: > > > >> On Jun 21, 2008, at 12:05 PM, Denis Washington wrote: > > > >> > > > >>> > > > >>> What if the transaction fails? register_package() would > > have > > > >>> returned > > > >>> without error although the registration was unsuccessful > > then, > > > >>> and all > > > >>> files would already be installed. > > > >>> > > > >> > > > >> What if you've added a header, but your daemon exits > > before > > > >> successfully computing and adding RPMTAG_SIZE withthe > > > >> _close_package() method? > > > > > > > > Got me. Although, if a dummy value (e.g. 0) was added in > > > > _register_package(), an unsuccessful _close_package() > > wouldn't be a > > > > harm > > > > at all. The header would be complete anyway. > > > > > > > > > > Hint: RPMTAG_SIZE simply does not matter. Nor do Vendor: > > Packager: > > > Description: Summary: and all the other goopiness carried in > > > markup (because its easy to add) and rpmdb Headers. > > > > > > OTOH, RPMTAG_FILESTATES is gonna matter a _LOT_. So > > > will leaving stale locks, and forgetting to attach stderr > > when > > > your widdle daemon forks. > > > > > > Could you explain what should go in RPM_FILESTATES? It's not > > listed in > > the LSB specification. > > > > > > Sorry, but who care on LSB RPM specification aka RPM v3 (other for > > some useful docu) ? RPM 4.4.2 could not produce it, do you know ? > > > > Also , do you know that the LSB RPM spec was bourne only because > > "someone" suggest to write some referral on the LSB on "MAXIMUN RPM" ? > > > > Also again do you know that in "REDHAT RPM GUIDE" "someone" suggest > > the author to describe in appendices the RPMV3 package format only > > for the better docu ? > > > > And guess who it is this "someone" ? > > > > R : Jeff Johnson > > > > So think more carefully before expressing silly opinions on Jeff > > Johnson, which authority in the filed is beyond discussion. > > > > I didn't want to express an opinion, I just want to know what > RPMTAG_FILESTATES implements so I can fill it in properly. I cannot see > where I attacked him with this question, but I apologize when I did. This was your word in this thread "Despite thinking that opinions can hardly be measured in terms of "correctness", there are enough people that keep flawed opinions for their entire life without reflecting on after some time. Maybe my comparatively little experience just gives my the flexibility of mind that you might be missing after more than ten years. But I was actually not planning to start a whose-right flame war." (aside) It is time for LSB RPM SPEC to move to RPM4 packaging format > > > Regards, > Denis > > __ > RPM Package Managerhttp://rpm5.org > LSB Communication Listrpm-lsb@rpm5.org >
Re: LSB Package API
On Sat, 2008-06-21 at 13:31 -0400, Jeff Johnson wrote: > On Jun 21, 2008, at 1:17 PM, Denis Washington wrote: > > > On Sat, 2008-06-21 at 13:01 -0400, Jeff Johnson wrote: > >> > >> > >> OTOH, RPMTAG_FILESTATES is gonna matter a _LOT_. So > >> will leaving stale locks, and forgetting to attach stderr when > >> your widdle daemon forks. > > > > Could you explain what should go in RPM_FILESTATES? It's not listed in > > the LSB specification. > > > > Zeros are same as RPMFILE_STATE_NORMAL and will suffice. > > What is primarily important is that the tag exists (so the pointer > does not go NULL), > and that the memory is sized correctly (array of unsigned character > #files is the dimension). > RPMFILE_STATE_NORMAL is what most files have attached. Ok, thanks. > Until one starts to get into multilib, another UNSPECIFIED area > that will affect ISV's that the LSB "Berlin API" is worfully silent on. Unfortunately I don't know about multilibs. > Hmmm, you have not included any scrtlets in you _register_package() > methods. AFAIK from listening to Ted T'so, the ability for an > ISV package to run scriptlets is an important need. Yeah. Post-install scripts are not needed as there is an ISV installer running which can do whatever it wishes. Pre-removal scripts should be there though. > Of courrse the "Berlin API" is woefully silent on how to include > scriptlet actions, and how/when those scriptlets should be run. That would have to be specified, naturally. The API is far from complete, but you have to start somewhere. Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, Jun 21, 2008 at 7:35 PM, devzero2000 <[EMAIL PROTECTED]> wrote: > > > On Sat, Jun 21, 2008 at 7:17 PM, Denis Washington <[EMAIL PROTECTED]> > wrote: > >> On Sat, 2008-06-21 at 13:01 -0400, Jeff Johnson wrote: >> > On Jun 21, 2008, at 12:48 PM, Denis Washington wrote: >> > >> > > On Sat, 2008-06-21 at 12:27 -0400, Jeff Johnson wrote: >> > >> On Jun 21, 2008, at 12:05 PM, Denis Washington wrote: >> > >> >> > >>> >> > >>> What if the transaction fails? register_package() would have >> > >>> returned >> > >>> without error although the registration was unsuccessful then, >> > >>> and all >> > >>> files would already be installed. >> > >>> >> > >> >> > >> What if you've added a header, but your daemon exits before >> > >> successfully computing and adding RPMTAG_SIZE withthe >> > >> _close_package() method? >> > > >> > > Got me. Although, if a dummy value (e.g. 0) was added in >> > > _register_package(), an unsuccessful _close_package() wouldn't be a >> > > harm >> > > at all. The header would be complete anyway. >> > > >> > >> > Hint: RPMTAG_SIZE simply does not matter. Nor do Vendor: Packager: >> > Description: Summary: and all the other goopiness carried in >> > markup (because its easy to add) and rpmdb Headers. >> > >> > OTOH, RPMTAG_FILESTATES is gonna matter a _LOT_. So >> > will leaving stale locks, and forgetting to attach stderr when >> > your widdle daemon forks. >> >> Could you explain what should go in RPM_FILESTATES? It's not listed in >> the LSB specification. >> > > Sorry, but who care on LSB RPM specification aka RPM v3 (other for some > useful docu) ? RPM 4.4.2 could not produce it, do you know ? > > Also , do you know that the LSB RPM spec was bourne only because "someone" > suggest to write some referral on the LSB on "MAXIMUN RPM" ? > > Also again do you know that in "REDHAT RPM GUIDE" "someone" suggest the > author to describe in appendices the RPMV3 package format only > for the better docu ? > > And guess who it is this "someone" ? > > R : Jeff Johnson > > So think more carefully before expressing silly opinions on Jeff Johnson, > which authority in the filed is beyond discussion. > > >
Re: LSB Package API
On Sat, 2008-06-21 at 19:35 +0200, devzero2000 wrote: > > > On Sat, Jun 21, 2008 at 7:17 PM, Denis Washington > <[EMAIL PROTECTED]> wrote: > > On Sat, 2008-06-21 at 13:01 -0400, Jeff Johnson wrote: > > On Jun 21, 2008, at 12:48 PM, Denis Washington wrote: > > > > > On Sat, 2008-06-21 at 12:27 -0400, Jeff Johnson wrote: > > >> On Jun 21, 2008, at 12:05 PM, Denis Washington wrote: > > >> > > >>> > > >>> What if the transaction fails? register_package() would > have > > >>> returned > > >>> without error although the registration was unsuccessful > then, > > >>> and all > > >>> files would already be installed. > > >>> > > >> > > >> What if you've added a header, but your daemon exits > before > > >> successfully computing and adding RPMTAG_SIZE withthe > > >> _close_package() method? > > > > > > Got me. Although, if a dummy value (e.g. 0) was added in > > > _register_package(), an unsuccessful _close_package() > wouldn't be a > > > harm > > > at all. The header would be complete anyway. > > > > > > > Hint: RPMTAG_SIZE simply does not matter. Nor do Vendor: > Packager: > > Description: Summary: and all the other goopiness carried in > > markup (because its easy to add) and rpmdb Headers. > > > > OTOH, RPMTAG_FILESTATES is gonna matter a _LOT_. So > > will leaving stale locks, and forgetting to attach stderr > when > > your widdle daemon forks. > > > Could you explain what should go in RPM_FILESTATES? It's not > listed in > the LSB specification. > > > Sorry, but who care on LSB RPM specification aka RPM v3 (other for > some useful docu) ? RPM 4.4.2 could not produce it, do you know ? > > Also , do you know that the LSB RPM spec was bourne only because > "someone" suggest to write some referral on the LSB on "MAXIMUN RPM" ? > > Also again do you know that in "REDHAT RPM GUIDE" "someone" suggest > the author to describe in appendices the RPMV3 package format only > for the better docu ? > > And guess who it is this "someone" ? > > R : Jeff Johnson > > So think more carefully before expressing silly opinions on Jeff > Johnson, which authority in the filed is beyond discussion. > I didn't want to express an opinion, I just want to know what RPMTAG_FILESTATES implements so I can fill it in properly. I cannot see where I attacked him with this question, but I apologize when I did. Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, Jun 21, 2008 at 7:17 PM, Denis Washington <[EMAIL PROTECTED]> wrote: > On Sat, 2008-06-21 at 13:01 -0400, Jeff Johnson wrote: > > On Jun 21, 2008, at 12:48 PM, Denis Washington wrote: > > > > > On Sat, 2008-06-21 at 12:27 -0400, Jeff Johnson wrote: > > >> On Jun 21, 2008, at 12:05 PM, Denis Washington wrote: > > >> > > >>> > > >>> What if the transaction fails? register_package() would have > > >>> returned > > >>> without error although the registration was unsuccessful then, > > >>> and all > > >>> files would already be installed. > > >>> > > >> > > >> What if you've added a header, but your daemon exits before > > >> successfully computing and adding RPMTAG_SIZE withthe > > >> _close_package() method? > > > > > > Got me. Although, if a dummy value (e.g. 0) was added in > > > _register_package(), an unsuccessful _close_package() wouldn't be a > > > harm > > > at all. The header would be complete anyway. > > > > > > > Hint: RPMTAG_SIZE simply does not matter. Nor do Vendor: Packager: > > Description: Summary: and all the other goopiness carried in > > markup (because its easy to add) and rpmdb Headers. > > > > OTOH, RPMTAG_FILESTATES is gonna matter a _LOT_. So > > will leaving stale locks, and forgetting to attach stderr when > > your widdle daemon forks. > > Could you explain what should go in RPM_FILESTATES? It's not listed in > the LSB specification. > Sorry, but who care on LSB RPM specification aka RPM v3 (other for some useful docu) ? RPM 4.4.2 could not produce it, do you know ? Also , do you know that the LSB RPM spec was bourne only because "someone" suggest to write some referral on the LSB on "MAXIMUN RPM" ? Also again do you know that in "REDHAT RPM GUIDE" "someone" suggest the author to describe in appendices the RPMV3 package format only for the better docu ? And guess who it is this "someone" ? R : Jeff Johnson So think more carefully before expressing silly opinions on Jeff Johnson, which authority in the filed is beyond discussion.
Re: LSB Package API
On Jun 21, 2008, at 1:17 PM, Denis Washington wrote: On Sat, 2008-06-21 at 13:01 -0400, Jeff Johnson wrote: OTOH, RPMTAG_FILESTATES is gonna matter a _LOT_. So will leaving stale locks, and forgetting to attach stderr when your widdle daemon forks. Could you explain what should go in RPM_FILESTATES? It's not listed in the LSB specification. Zeros are same as RPMFILE_STATE_NORMAL and will suffice. What is primarily important is that the tag exists (so the pointer does not go NULL), and that the memory is sized correctly (array of unsigned character #files is the dimension). RPMFILE_STATE_NORMAL is what most files have attached. Until one starts to get into multilib, another UNSPECIFIED area that will affect ISV's that the LSB "Berlin API" is worfully silent on. Ditto SELinux, but most of thiose issues are outside of RPM packaging these days. That can/will change if "modular" rather than "monolithic" SELinux policy ever succeeds. Hmmm, you have not included any scrtlets in you _register_package() methods. AFAIK from listening to Ted T'so, the ability for an ISV package to run scriptlets is an important need. Of courrse the "Berlin API" is woefully silent on how to include scriptlet actions, and how/when those scriptlets should be run. Again, LSB has no concept of file states before/during/after install, and has never cared to specify any RPM tag contents that were not of interest to the "LSB package format" that they have chosen to adopt from RPM and doggedly persists in continuing into the future of Linux. 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, 2008-06-21 at 13:01 -0400, Jeff Johnson wrote: > On Jun 21, 2008, at 12:48 PM, Denis Washington wrote: > > > On Sat, 2008-06-21 at 12:27 -0400, Jeff Johnson wrote: > >> On Jun 21, 2008, at 12:05 PM, Denis Washington wrote: > >> > >>> > >>> What if the transaction fails? register_package() would have > >>> returned > >>> without error although the registration was unsuccessful then, > >>> and all > >>> files would already be installed. > >>> > >> > >> What if you've added a header, but your daemon exits before > >> successfully computing and adding RPMTAG_SIZE withthe > >> _close_package() method? > > > > Got me. Although, if a dummy value (e.g. 0) was added in > > _register_package(), an unsuccessful _close_package() wouldn't be a > > harm > > at all. The header would be complete anyway. > > > > Hint: RPMTAG_SIZE simply does not matter. Nor do Vendor: Packager: > Description: Summary: and all the other goopiness carried in > markup (because its easy to add) and rpmdb Headers. > > OTOH, RPMTAG_FILESTATES is gonna matter a _LOT_. So > will leaving stale locks, and forgetting to attach stderr when > your widdle daemon forks. Could you explain what should go in RPM_FILESTATES? It's not listed in the LSB specification. Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 21, 2008, at 12:48 PM, Denis Washington wrote: On Sat, 2008-06-21 at 12:27 -0400, Jeff Johnson wrote: On Jun 21, 2008, at 12:05 PM, Denis Washington wrote: What if the transaction fails? register_package() would have returned without error although the registration was unsuccessful then, and all files would already be installed. What if you've added a header, but your daemon exits before successfully computing and adding RPMTAG_SIZE withthe _close_package() method? Got me. Although, if a dummy value (e.g. 0) was added in _register_package(), an unsuccessful _close_package() wouldn't be a harm at all. The header would be complete anyway. Hint: RPMTAG_SIZE simply does not matter. Nor do Vendor: Packager: Description: Summary: and all the other goopiness carried in markup (because its easy to add) and rpmdb Headers. OTOH, RPMTAG_FILESTATES is gonna matter a _LOT_. So will leaving stale locks, and forgetting to attach stderr when your widdle daemon forks. (aside) You won't be the first to write stderr directly into an rpmdb. At least 2 major linux vendors have forgotten to do the right thing with stderr when forking a daemon, and error messages got written to customer databases because of the astonishing lack of QA associated with Linux vendors. Guess who got to fox the problem? Same issue, sh*t happens. I'm just trying to minimize the window where disasters occur because I _WILL_ have to do the support even if your "Berlin API" code is flawed. Which motivates me to do the best I can to avoid the "desaster window". Really. Good. Too little: currently yes. Too late: no. Just my opinion. We both know that our thoughts on this differ quite much. I've been at RPM packaging for over a decade, you mebbe a month. Wanna bet on whose opinion is correct? ;-) Despite thinking that opinions can hardly be measured in terms of "correctness", there are enough people that keep flawed opinions for their entire life without reflecting on after some time. Maybe my comparatively little experience just gives my the flexibility of mind that you might be missing after more than ten years. But I was actually not planning to start a whose-right flame war. My interest in opinions is directly related to the amount of money I've wagered, so far none on the LSB "Berlin API". And I've heard all the "trust" foolishness on the packaging list for years. No SHA1 == no trust is the engineering answer. 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, 2008-06-21 at 12:27 -0400, Jeff Johnson wrote: > On Jun 21, 2008, at 12:05 PM, Denis Washington wrote: > > > > > What if the transaction fails? register_package() would have returned > > without error although the registration was unsuccessful then, and all > > files would already be installed. > > > > What if you've added a header, but your daemon exits before > successfully computing and adding RPMTAG_SIZE withthe > _close_package() method? Got me. Although, if a dummy value (e.g. 0) was added in _register_package(), an unsuccessful _close_package() wouldn't be a harm at all. The header would be complete anyway. > Same issue, sh*t happens. I'm just trying to minimize the window > where disasters occur because I _WILL_ have to do the support > even if your "Berlin API" code is flawed. Which motivates me to do the best I can to avoid the "desaster window". Really. > > Too little: currently yes. Too late: no. Just my opinion. We both know > > that our thoughts on this differ quite much. > > > > I've been at RPM packaging for over a decade, you mebbe a month. > > Wanna bet on whose opinion is correct? ;-) Despite thinking that opinions can hardly be measured in terms of "correctness", there are enough people that keep flawed opinions for their entire life without reflecting on after some time. Maybe my comparatively little experience just gives my the flexibility of mind that you might be missing after more than ten years. But I was actually not planning to start a whose-right flame war. > > >> My current interest in your code is disaster prevention, not > >> otherwise. > > > > I welcome any motive if it improves code quality, so thanks anyway. ;) > > > > NP. My life is hell when rpmdb's get hosed up. Doesn't matter whether > its a kernel mmap(2) flaw, an selinux policy-of-the-day typo, or a > python > script kiddies dain bramage. > > The "Berlin API" is a recipe for disaster so far. > > But I'm most definitely deeply and personally interested in not > having to do the necessary rpmdb maintenance post-mortem > if the implementation problems can be solved. Thought so. Regard, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 21, 2008, at 12:05 PM, Denis Washington wrote: What if the transaction fails? register_package() would have returned without error although the registration was unsuccessful then, and all files would already be installed. What if you've added a header, but your daemon exits before successfully computing and adding RPMTAG_SIZE withthe _close_package() method? Same issue, sh*t happens. I'm just trying to minimize the window where disasters occur because I _WILL_ have to do the support even if your "Berlin API" code is flawed. Note that there are no commit/apply transactions, and no D == Durability as in ACID present with an rpmdb. Wishing an rpmdb had ACID properties is not going to change the status quo. Too little: currently yes. Too late: no. Just my opinion. We both know that our thoughts on this differ quite much. I've been at RPM packaging for over a decade, you mebbe a month. Wanna bet on whose opinion is correct? ;-) My current interest in your code is disaster prevention, not otherwise. I welcome any motive if it improves code quality, so thanks anyway. ;) NP. My life is hell when rpmdb's get hosed up. Doesn't matter whether its a kernel mmap(2) flaw, an selinux policy-of-the-day typo, or a python script kiddies dain bramage. The "Berlin API" is a recipe for disaster so far. But I'm most definitely deeply and personally interested in not having to do the necessary rpmdb maintenance post-mortem if the implementation problems can be solved. 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 21, 2008, at 11:41 AM, Denis Washington wrote: The "upgrade issue" must certainly be dealt with. That needs to be discussed. About package erasure: the package manager is good enough to handle that on his own actually. We might need a feature to add a post-remove script, but basic file removal we get pretty much for free by registring in the package database. LSB has chosen to leave "upgrade" UNSPECIFIED, and has also chose in the "Berlin API" to ignore the fact that both dpkg/rpm versions are a triple of Epoch/Version/Release. Pretending that a "version" string can be anything, opaquely handled, including E:V-R, or something else, misses the issue that "upgrade" based on "version" is undecidable until "version" is well formed, and a collate sequence is defined for "upgrade" comparison. Leaving it to rpm to clean up _register_package()'s mess is _EXACTLY_ what I mean by disaster avoidance. E.g., files have a RPMTAG_FILESTATES array that is computed and attached to package headers before rpmdbAdd() is called. If you do not also add RPMTAG_FILESTATES, you _WILL_ cause rpm to segfault under certain conditions. Note this comment in lib/transaction.c, been there for years and years, that you _WILL_ end up traversing if you insist on registering "LSB Format" headers: /* XXX there's an obscure segfault here w/o NULL check ... */ if (otherStates != NULL) Note also that there's nothing other than avoiding the segfault that has ever been attempted in RPM because a header in a rpmdb without RPMTAG_FILESTATES is a "should never ever happen" condition. Guess what Newer! Better! Besttest! rule your "Berlin API" _register_package() method is about to impose on already released legacy versions of RPM? Along these lines, you should attempt to add a SHA1 on you rregistered header. Almost all headers (LSB and Sun java being the Luddites) in an rpmdb carry a SHA1 to guarantee package metadata integrity many years now. No such guarantee can be attempted unless you also compute and supply the necessary Header SHA1. The code that is needed to add a header SHA1 is in build/pack.c. Essentially (this is rpm5 code, there are minor API differences) /* Reallocate the header into one contiguous region. */ Header h = headerReload(h, RPMTAG_HEADERIMMUTABLE); ... size_t uhlen = 0; void * uh = headerUnload(h, &uhlen); DIGEST_CTX ctx = rpmDigestInit(PGPHASHALGO, 0); const char * SHA1 = NULL; (void) rpmDigestUpdate(ctx, uh, uhlen) (void) rpmDigestFinal(ctx, &SHA1, NULL, 1) headerAddEntry(h, RPMTAG_SHA1HEADER, RPM_STRING_TYPE, SHA1, 1); Note that the ordering of tag additions is also significant. E.g. RPMTAG_FILESTATES should be added _AFTER_ headerReload() and the SHA1 computation I've just described, because that's what is done during installation with RPM. Ditto adding RPMTAG_INSTALLTIME. FYI, the complete set of additions is in lib/psm.c near the case PSM_RPMDB_ADD: code. But I've likely digressed, sigh ... hth 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, 2008-06-21 at 11:44 -0400, Jeff Johnson wrote: > On Jun 21, 2008, at 11:19 AM, Denis Washington wrote: > > > > The LSB specification says RPMTAG_BASENAMES and friends should be > > used, > > so the choice is pretty clear. > > > > Good. It took bloody years to get that "choice" into the "LSB > Packaging Standard". > > The transaction set (and the underlying configuration/rpmdb > handling) > cannot be scoped within the individual methods if you are going > to compute and add RPMTAG_SIZE lazily in _close_package(). > >>> > >>> I don't know what you mean here. I use a separate transaction set > >>> for > >>> _register_package() and _close_package(), closing after each > >>> function. > >>> > >> > >> What I mean is that the _register_package() method constructs > >> what you are calling a Header, and calls rpmdbAdd(). > >> > >> Then the _close_package() method comes along later, reopen's > >> an rpmdb, adds RPMTAG_SIZE, and rewrites the header. > >> > >> There's a racy window between your 2 method calls that will lead > >> to very hard to diagnose issues. Adding RPMTAG_SIZE in > >> the _register_package() method turns _close_package() into > >> a no-operation needed stub, and otherwise avoids statefulness. > > > > True. But the problem is that the complete header cannot be created in > > _register_package() as the package files are still missing. And > > doing it > > in _close_package() kills the guarantee that a successful run of > > _register_package() means a successful package registration - we only > > know if we were successful after rpmdbAdd(). > > > > So keep the header around until "complete", then do rpmdbAdd(), > which will need a transaction and more. What if the transaction fails? register_package() would have returned without error although the registration was unsuccessful then, and all files would already be installed. > The raciness I've pointed out has everything to do with database <-> > daemon > interactions, nothing at all to do with rpmdb specifically. > > >>> P.S.: For follow-ups of general interest, I recommend you to reply > >>> on [EMAIL PROTECTED] You can subscribe here: > >>> > >>> https://lists.linux-foundation.org/mailman/listinfo/packaging > >>> > >> > >> I will not reply on the packaging list, having received hostile > >> threats there > >> in the past. > > > > Unfortunate. I hope you'll at least follow the discussion over there. > > > > I follow, yes. Nothing that I haven't heard for years and years sadly. > The window where LSB Packaging might have made a difference > for Linux closed years ago, the "Berlin API" proposal was already too > little and too late. JMHO ... Too little: currently yes. Too late: no. Just my opinion. We both know that our thoughts on this differ quite much. > My current interest in your code is disaster prevention, not otherwise. I welcome any motive if it improves code quality, so thanks anyway. ;) Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 21, 2008, at 11:19 AM, Denis Washington wrote: The LSB specification says RPMTAG_BASENAMES and friends should be used, so the choice is pretty clear. Good. It took bloody years to get that "choice" into the "LSB Packaging Standard". The transaction set (and the underlying configuration/rpmdb handling) cannot be scoped within the individual methods if you are going to compute and add RPMTAG_SIZE lazily in _close_package(). I don't know what you mean here. I use a separate transaction set for _register_package() and _close_package(), closing after each function. What I mean is that the _register_package() method constructs what you are calling a Header, and calls rpmdbAdd(). Then the _close_package() method comes along later, reopen's an rpmdb, adds RPMTAG_SIZE, and rewrites the header. There's a racy window between your 2 method calls that will lead to very hard to diagnose issues. Adding RPMTAG_SIZE in the _register_package() method turns _close_package() into a no-operation needed stub, and otherwise avoids statefulness. True. But the problem is that the complete header cannot be created in _register_package() as the package files are still missing. And doing it in _close_package() kills the guarantee that a successful run of _register_package() means a successful package registration - we only know if we were successful after rpmdbAdd(). So keep the header around until "complete", then do rpmdbAdd(), which will need a transaction and more. The raciness I've pointed out has everything to do with database <-> daemon interactions, nothing at all to do with rpmdb specifically. P.S.: For follow-ups of general interest, I recommend you to reply on [EMAIL PROTECTED] You can subscribe here: https://lists.linux-foundation.org/mailman/listinfo/packaging I will not reply on the packaging list, having received hostile threats there in the past. Unfortunate. I hope you'll at least follow the discussion over there. I follow, yes. Nothing that I haven't heard for years and years sadly. The window where LSB Packaging might have made a difference for Linux closed years ago, the "Berlin API" proposal was already too little and too late. JMHO ... My current interest in your code is disaster prevention, not otherwise. 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, 2008-06-21 at 11:25 -0400, Jeff Johnson wrote: > On Jun 21, 2008, at 10:55 AM, Jeff Johnson wrote: > > > > > On Jun 21, 2008, at 10:32 AM, Denis Washington wrote: > > > >> > >> Thanks for taking a look. I am really new to programming RPM, so any > >> help is greatly appreciated. > >> > > > > This code snippet could be simplified: > > ts = rpmtsCreate(); > tid = time(NULL); > rpmtsSetTid(ts, tid); > > rpmtsCreate() already initializes the transaction ID. > > In lib/rpmts.c rpmtsCreate() > ts->tid = (int_32) time(NULL); Ok. I really appreciate that you take such a thorough look at the implementation, thanks. I hope I'll have the chance to implement a better version in the next few days. > The issue will become important if/when multiple > packages need registration with an identical transaction > identifier is attempted. They should have the same > transaction identifier. > > But the "Berlin API" is woefully silent on issues > of "upgrade" and "erasure" or multiple package > registration. Oh well ... Yeah, many corner cases are not handled yet. The RPM backend is currently more a less a see-it-works-in-basic implementation right now. The "upgrade issue" must certainly be dealt with. That needs to be discussed. About package erasure: the package manager is good enough to handle that on his own actually. We might need a feature to add a post-remove script, but basic file removal we get pretty much for free by registring in the package database. Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 21, 2008, at 10:55 AM, Jeff Johnson wrote: On Jun 21, 2008, at 10:32 AM, Denis Washington wrote: Thanks for taking a look. I am really new to programming RPM, so any help is greatly appreciated. This code snippet could be simplified: ts = rpmtsCreate(); tid = time(NULL); rpmtsSetTid(ts, tid); rpmtsCreate() already initializes the transaction ID. In lib/rpmts.c rpmtsCreate() ts->tid = (int_32) time(NULL); The issue will become important if/when multiple packages need registration with an identical transaction identifier is attempted. They should have the same transaction identifier. But the "Berlin API" is woefully silent on issues of "upgrade" and "erasure" or multiple package registration. Oh well ... 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, 2008-06-21 at 10:55 -0400, Jeff Johnson wrote:
> On Jun 21, 2008, at 10:32 AM, Denis Washington wrote:
>
> >
> > Thanks for taking a look. I am really new to programming RPM, so any
> > help is greatly appreciated.
> >
>
>
> >> In general, what is being called a Header does not include sufficient
> >> metadata to be casually installed in an rpmdb. You can/will create
> >> cause existing rpm deployments to segfault if you proceed with
> >> the header as constructed in this implementation.
> >
> > I already figured that I might not have added all mandatory
> > metadata to
> > the header. Unfortunately, I didn't find any documentation on which
> > data
> > must be present. Can you point me at the right direction?
> >
>
> The tag descriptions in the LSB packaging standard are included
> as an appendix to the "RedHat RPM Guide" which is available
> somewhere in Fedorable land. The text is also maintained somewhere
> within LSB as well.
Ok, I'll have a look at those, that will most certainly help. A quick
look already tells me that there is still a lot to do..
> >> Look closely at the "LSB Packaging" documentation that describes
> >> tag content, paying particular attention to the content that LSB has
> >> chosen to call "MANDATORY". The fact that the header is in a rpmdb
> >> rather than a *.rpm package does not permit MANDATORY to be ignored.
> >>
> >> Adding both RPMTAG_FILENAMES and RPMTAG_
> >> {DIRNAMES,BASENAMES,DIRINDEXES}
> >> is just wrong. One or the other should be done, not both.
> >
> > OK. Again, I didn't find any documentation on this, so I just filled
> > both. Which of them is preferred?
> >
>
> RPMTAG_FILENAMES was abandoned by RPM in RHL 7.0 ~2000, but is mired in
> what is now known as the "LSB format".
>
> You will have to choose which of the conflicting representations for
> file paths you wish to use. Including both is clearly a flaw, and worse
> than either.
The LSB specification says RPMTAG_BASENAMES and friends should be used,
so the choice is pretty clear.
> >> The transaction set (and the underlying configuration/rpmdb handling)
> >> cannot be scoped within the individual methods if you are going
> >> to compute and add RPMTAG_SIZE lazily in _close_package().
> >
> > I don't know what you mean here. I use a separate transaction set for
> > _register_package() and _close_package(), closing after each function.
> >
>
> What I mean is that the _register_package() method constructs
> what you are calling a Header, and calls rpmdbAdd().
>
> Then the _close_package() method comes along later, reopen's
> an rpmdb, adds RPMTAG_SIZE, and rewrites the header.
>
> There's a racy window between your 2 method calls that will lead
> to very hard to diagnose issues. Adding RPMTAG_SIZE in
> the _register_package() method turns _close_package() into
> a no-operation needed stub, and otherwise avoids statefulness.
True. But the problem is that the complete header cannot be created in
_register_package() as the package files are still missing. And doing it
in _close_package() kills the guarantee that a successful run of
_register_package() means a successful package registration - we only
know if we were successful after rpmdbAdd().
> > P.S.: For follow-ups of general interest, I recommend you to reply
> > on [EMAIL PROTECTED] You can subscribe here:
> >
> > https://lists.linux-foundation.org/mailman/listinfo/packaging
> >
>
> I will not reply on the packaging list, having received hostile
> threats there
> in the past.
Unfortunate. I hope you'll at least follow the discussion over there.
Regards,
Denis
__
RPM Package Managerhttp://rpm5.org
LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 21, 2008, at 10:32 AM, Denis Washington wrote:
Thanks for taking a look. I am really new to programming RPM, so any
help is greatly appreciated.
In general, what is being called a Header does not include sufficient
metadata to be casually installed in an rpmdb. You can/will create
cause existing rpm deployments to segfault if you proceed with
the header as constructed in this implementation.
I already figured that I might not have added all mandatory
metadata to
the header. Unfortunately, I didn't find any documentation on which
data
must be present. Can you point me at the right direction?
The tag descriptions in the LSB packaging standard are included
as an appendix to the "RedHat RPM Guide" which is available
somewhere in Fedorable land. The text is also maintained somewhere
within LSB as well.
Look closely at the "LSB Packaging" documentation that describes
tag content, paying particular attention to the content that LSB has
chosen to call "MANDATORY". The fact that the header is in a rpmdb
rather than a *.rpm package does not permit MANDATORY to be ignored.
Adding both RPMTAG_FILENAMES and RPMTAG_
{DIRNAMES,BASENAMES,DIRINDEXES}
is just wrong. One or the other should be done, not both.
OK. Again, I didn't find any documentation on this, so I just filled
both. Which of them is preferred?
RPMTAG_FILENAMES was abandoned by RPM in RHL 7.0 ~2000, but is mired in
what is now known as the "LSB format".
You will have to choose which of the conflicting representations for
file paths you wish to use. Including both is clearly a flaw, and worse
than either.
The transaction set (and the underlying configuration/rpmdb handling)
cannot be scoped within the individual methods if you are going
to compute and add RPMTAG_SIZE lazily in _close_package().
I don't know what you mean here. I use a separate transaction set for
_register_package() and _close_package(), closing after each function.
What I mean is that the _register_package() method constructs
what you are calling a Header, and calls rpmdbAdd().
Then the _close_package() method comes along later, reopen's
an rpmdb, adds RPMTAG_SIZE, and rewrites the header.
There's a racy window between your 2 method calls that will lead
to very hard to diagnose issues. Adding RPMTAG_SIZE in
the _register_package() method turns _close_package() into
a no-operation needed stub, and otherwise avoids statefulness.
A package header can change in an rpmdb between method
calls, and keeping the rpmdb open (or alternatively, verifying that
the header retrieved is the one that should be modified). I see
no reason why RPMTAG_SIZE needs to be added in _close_package().
Yeah, I should probably verify that I got the right package in
_close_package(). Note that when _register_package() is finished,
there
are no files installed yet, only stub files; that's why we can only
add
the files' sizes in _close_package(), after the installer has
copied the
"real" files over. RPMTAG_SIZE could be set to 0 in
_register_package(),
though.
Why bother with verify? Just avoid the issue, calculate RPMTAG_SIZE
when you
have complete information, and add the header when the Header
is complete. Incremental additions to binary Header blobs will
lead to all sorts of issues, including the raciness I pointed out.
(aside) There will be other DoS issues that you will encounter if you
try
to keep an rpmdb (or any database) open persistently in a daemon.
That's why both _register_package() and _close_package() close the DB
when there finished. The database is just open for while those two
functions are running.
There are two calls to
rpmdbSetIteratorModified(iter, 1);
one of the calls is unnecessary.
Oh, right. Oops.
I also strongly encourage you _NOT_ to incrementally add tags to a
header with a lazy rewrite into an rpmdb using
rpmdbSetIteratorModified().
What is the better way?
Build a complete header, call rpmdbAdd() exactly once, removes
an immense amount of statefulness.
Whether the one rpmdbAdd() call is done in _register_package()
or _close_package() matters little. Limiting the time window where
you need exclusive access to an rpmdb is what is needed.
That's generally true for all database applications, not just an rpmdb,
and even more important if/when you are attempting a daemon back end
to a database.
Regards,
Denis
P.S.: For follow-ups of general interest, I recommend you to reply
on [EMAIL PROTECTED] You can subscribe here:
https://lists.linux-foundation.org/mailman/listinfo/packaging
I will not reply on the packaging list, having received hostile
threats there
in the past.
73 de Jeff
__
RPM Package Managerhttp://rpm5.org
LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
Ah, didn't know that. Thanks! Regards, Denis On Sat, 2008-06-21 at 10:24 -0400, Jeff Johnson wrote: > On Jun 21, 2008, at 9:55 AM, Jeff Johnson wrote: > > > > > On Jun 21, 2008, at 5:15 AM, Denis Washington wrote: > >> > > As promised, here's certain issues I see in the current > > implementation. > > > > > The data type for Summary:/Description: is RPM_I18NSTRING_TYPE, > not RPM_STRING_TYPE. Basically that means that yo should not > use headerAddEntry(), but rather > > /** \ingroup header > * Add locale specific tag to header. > * A NULL lang is interpreted as the C locale. Here are the rules: > * \verbatim > * - If the tag isn't in the header, it's added with the passed > string > * as new value. > * - If the tag occurs multiple times in entry, which tag is > affected > * by the operation is undefined. > * - If the tag is in the header w/ this language, the entry is > * *replaced* (like headerModifyEntry()). > * \endverbatim > * This function is intended to just "do the right thing". If you need > * more fine grained control use headerAddEntry() and > headerModifyEntry(). > * > * @param h header > * @param tag tag > * @param stringtag value > * @param lang locale > * @return 1 on success, 0 on failure > */ > /[EMAIL PROTECTED]@*/ static inline > int headerAddI18NString(Header h, int_32 tag, const char * string, > const char * lang) > /[EMAIL PROTECTED] h @*/ > > In this snippet: > > if (mf->pkgdisplayedname) > headerAddEntry(header, RPMTAG_SUMMARY, RPM_STRING_TYPE, mf- > >pkgdisplayedname, 1); > if (mf->pkgdescription) > headerAddEntry(header, RPMTAG_DESCRIPTION, RPM_STRING_TYPE, > mf->pkgdescription, 1); > > Nite that RPMTAG_GROUP is also RPM_I18NSTRING_TYPE if/when you get > around > to including. > > And also note that there are much deeper issues with I18N in *.rpm > packages. > > hth > > 73 de Jeff > __ > RPM Package Managerhttp://rpm5.org > LSB Communication Listrpm-lsb@rpm5.org __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 21, 2008, at 9:55 AM, Jeff Johnson wrote: On Jun 21, 2008, at 5:15 AM, Denis Washington wrote: As promised, here's certain issues I see in the current implementation. IMHO, your markup needs to be thought through more carefully, if the "Exmples and attributes" section at http://www.linuxfoundation.org/en/LSB_Package_API is any indication. Sure markup is easy, and you can always consider your markup to be domain specific to the LSB "Berlin API". Meanwhile, there are any number of types of pre-existing markup that could/should be borrowed instead Re-inventing a Newer! Better! Bestest! markup that requires Yet Another Set Of Parsers to be written in python/perl/ruby/whatever will determine the rate of adoption of the application, not otherwise, imho. FWIW your "Berlin API" markup is no better or worse than most, just different. hth 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, 2008-06-21 at 09:55 -0400, Jeff Johnson wrote:
> On Jun 21, 2008, at 5:15 AM, Denis Washington wrote:
>
> > Hi,
> >
> > Some time ago, it was discussed on an LSB face-to-face meeting that an
> > API should be developed that allows ISVs to install sotware packages
> > which integrate into the package manager - the "Berlin Packaging API".
> > While the idea seemed to be well received, there didn't seem much
> > progress since then, except for a wiki page with a rundimentary
> > proposal
> > [1]. Considering that third-party software installation is an
> > undeniably
> > important weak spot of the Linux infrastructure, I found this was a
> > shame.
> >
> > To reignite the the initiative, I decided to design and develop a
> > prototype implementation of the Berlin API, most creatively named the
> > "LSB Package API". It is designed as a simple D-Bus interface
> > accompanied with an XML-based package description format. A detailed
> > description and the source code can be found on this page:
> >
> > http://www.linuxfoundation.org/en/LSB_Package_API
> >
> > The implementation currently supports integration into RPM and
> > dpkg; due
> > to its modular nature, support for more package managers could be
> > added
> > later on.
> >
> > I hope this implementation will act as a starting point for
> > resurrecting
> > the Berlin API process. Let us overcome the "Third-party software
> > installation on Linux sucks" problem and strive to a brave new
> > world of
> > easily distributable Linux software! ;)
> >
>
> As promised, here's certain issues I see in the current implementation.
Thanks for taking a look. I am really new to programming RPM, so any
help is greatly appreciated.
> In general, what is being called a Header does not include sufficient
> metadata to be casually installed in an rpmdb. You can/will create
> cause existing rpm deployments to segfault if you proceed with
> the header as constructed in this implementation.
I already figured that I might not have added all mandatory metadata to
the header. Unfortunately, I didn't find any documentation on which data
must be present. Can you point me at the right direction?
> Look closely at the "LSB Packaging" documentation that describes
> tag content, paying particular attention to the content that LSB has
> chosen to call "MANDATORY". The fact that the header is in a rpmdb
> rather than a *.rpm package does not permit MANDATORY to be ignored.
>
> Adding both RPMTAG_FILENAMES and RPMTAG_{DIRNAMES,BASENAMES,DIRINDEXES}
> is just wrong. One or the other should be done, not both.
OK. Again, I didn't find any documentation on this, so I just filled
both. Which of them is preferred?
> The transaction set (and the underlying configuration/rpmdb handling)
> cannot be scoped within the individual methods if you are going
> to compute and add RPMTAG_SIZE lazily in _close_package().
I don't know what you mean here. I use a separate transaction set for
_register_package() and _close_package(), closing after each function.
> A package header can change in an rpmdb between method
> calls, and keeping the rpmdb open (or alternatively, verifying that
> the header retrieved is the one that should be modified). I see
> no reason why RPMTAG_SIZE needs to be added in _close_package().
Yeah, I should probably verify that I got the right package in
_close_package(). Note that when _register_package() is finished, there
are no files installed yet, only stub files; that's why we can only add
the files' sizes in _close_package(), after the installer has copied the
"real" files over. RPMTAG_SIZE could be set to 0 in _register_package(),
though.
> (aside) There will be other DoS issues that you will encounter if you
> try
> to keep an rpmdb (or any database) open persistently in a daemon.
That's why both _register_package() and _close_package() close the DB
when there finished. The database is just open for while those two
functions are running.
> There are two calls to
> rpmdbSetIteratorModified(iter, 1);
> one of the calls is unnecessary.
Oh, right. Oops.
> I also strongly encourage you _NOT_ to incrementally add tags to a
> header with a lazy rewrite into an rpmdb using
> rpmdbSetIteratorModified().
What is the better way?
Regards,
Denis
P.S.: For follow-ups of general interest, I recommend you to reply on [EMAIL
PROTECTED] You can subscribe here:
https://lists.linux-foundation.org/mailman/listinfo/packaging
__
RPM Package Managerhttp://rpm5.org
LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 21, 2008, at 9:55 AM, Jeff Johnson wrote: On Jun 21, 2008, at 5:15 AM, Denis Washington wrote: As promised, here's certain issues I see in the current implementation. The data type for Summary:/Description: is RPM_I18NSTRING_TYPE, not RPM_STRING_TYPE. Basically that means that yo should not use headerAddEntry(), but rather /** \ingroup header * Add locale specific tag to header. * A NULL lang is interpreted as the C locale. Here are the rules: * \verbatim * - If the tag isn't in the header, it's added with the passed string * as new value. * - If the tag occurs multiple times in entry, which tag is affected * by the operation is undefined. * - If the tag is in the header w/ this language, the entry is * *replaced* (like headerModifyEntry()). * \endverbatim * This function is intended to just "do the right thing". If you need * more fine grained control use headerAddEntry() and headerModifyEntry(). * * @param h header * @param tag tag * @param stringtag value * @param lang locale * @return 1 on success, 0 on failure */ /[EMAIL PROTECTED]@*/ static inline int headerAddI18NString(Header h, int_32 tag, const char * string, const char * lang) /[EMAIL PROTECTED] h @*/ In this snippet: if (mf->pkgdisplayedname) headerAddEntry(header, RPMTAG_SUMMARY, RPM_STRING_TYPE, mf- >pkgdisplayedname, 1); if (mf->pkgdescription) headerAddEntry(header, RPMTAG_DESCRIPTION, RPM_STRING_TYPE, mf->pkgdescription, 1); Nite that RPMTAG_GROUP is also RPM_I18NSTRING_TYPE if/when you get around to including. And also note that there are much deeper issues with I18N in *.rpm packages. hth 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 21, 2008, at 5:15 AM, Denis Washington wrote:
Hi,
Some time ago, it was discussed on an LSB face-to-face meeting that an
API should be developed that allows ISVs to install sotware packages
which integrate into the package manager - the "Berlin Packaging API".
While the idea seemed to be well received, there didn't seem much
progress since then, except for a wiki page with a rundimentary
proposal
[1]. Considering that third-party software installation is an
undeniably
important weak spot of the Linux infrastructure, I found this was a
shame.
To reignite the the initiative, I decided to design and develop a
prototype implementation of the Berlin API, most creatively named the
"LSB Package API". It is designed as a simple D-Bus interface
accompanied with an XML-based package description format. A detailed
description and the source code can be found on this page:
http://www.linuxfoundation.org/en/LSB_Package_API
The implementation currently supports integration into RPM and
dpkg; due
to its modular nature, support for more package managers could be
added
later on.
I hope this implementation will act as a starting point for
resurrecting
the Berlin API process. Let us overcome the "Third-party software
installation on Linux sucks" problem and strive to a brave new
world of
easily distributable Linux software! ;)
As promised, here's certain issues I see in the current implementation.
In general, what is being called a Header does not include sufficient
metadata to be casually installed in an rpmdb. You can/will create
cause existing rpm deployments to segfault if you proceed with
the header as constructed in this implementation.
Look closely at the "LSB Packaging" documentation that describes
tag content, paying particular attention to the content that LSB has
chosen to call "MANDATORY". The fact that the header is in a rpmdb
rather than a *.rpm package does not permit MANDATORY to be ignored.
Adding both RPMTAG_FILENAMES and RPMTAG_{DIRNAMES,BASENAMES,DIRINDEXES}
is just wrong. One or the other should be done, not both.
The transaction set (and the underlying configuration/rpmdb handling)
cannot be scoped within the individual methods if you are going
to compute and add RPMTAG_SIZE lazily in _close_package().
A package header can change in an rpmdb between method
calls, and keeping the rpmdb open (or alternatively, verifying that
the header retrieved is the one that should be modified). I see
no reason why RPMTAG_SIZE needs to be added in _close_package().
(aside) There will be other DoS issues that you will encounter if you
try
to keep an rpmdb (or any database) open persistently in a daemon.
There are two calls to
rpmdbSetIteratorModified(iter, 1);
one of the calls is unnecessary.
I also strongly encourage you _NOT_ to incrementally add tags to a
header with a lazy rewrite into an rpmdb using
rpmdbSetIteratorModified().
hth
73 de Jeff
__
RPM Package Managerhttp://rpm5.org
LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
Le samedi 21 juin 2008 à 13:20 +0200, Yaakov Nemoy a écrit : > How is this different than PackageKit? It would make possible for ISVs to create packages in a non-native packaging format, so they don't have to care about the format each distro uses, or about understanding each distro dependency checks, or generally speaking wasting time and money on integration and QA. Of course that's supposing you can actually do good packaging in non-native formats and the distros won't be left to collect the pieces afterwards. -- Nicolas Mailhot signature.asc Description: Ceci est une partie de message numériquement signée
Re: LSB Package API
On Sat, 2008-06-21 at 13:20 +0200, Yaakov Nemoy wrote: > How is this different than PackageKit? PackageKit seems to cover the > use case of presenting a comprehensive API and userspace tools to > manage packages consistently across distros. What can the Berlin API > do that PackageKit doesn't do, and doesn't make sense for PackageKit > to do? > > -Yaakov While the use cases of PackageKit are related to the Berlin API, they are pretty different. PackageKit is focused on providing a frontend for managing repository-based package systems, like apt and and yum. It is mainly thought to abstract installation and upgrades from package repositories, like when an application likes to install a package with a particular name from the distro's repos. However, it does not address the problem of software distribution itself - the repositories and package files are still specific to the packagaing system. The Berlin API, on the other side, does exlclusively deal with providing a package-manager-neutral software distribution method. So the Berlin API is not a replacement for PackageKit, but a complement. In fact, as the software installed with the Berlin API is added to the package system's database, it can be managed (e.g. uninstalled) with PackageKit afterwards - a dream team! ;) Regards, Denis Washington __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
LSB Package API
(Resending because I previously forgot to fill in Reply-To.) Hi, Some time ago, it was discussed on an LSB face-to-face meeting that an API should be developed that allows ISVs to install sotware packages which integrate into the package manager - the "Berlin Packaging API". While the idea seemed to be well received, there didn't seem much progress since then, except for a wiki page with a rundimentary proposal [1]. Considering that third-party software installation is an undeniably important weak spot of the Linux infrastructure, I found this was a shame. To reignite the the initiative, I decided to design and develop a prototype implementation of the Berlin API, most creatively named the "LSB Package API". It is designed as a simple D-Bus interface accompanied with an XML-based package description format. A detailed description and the source code can be found on this page: http://www.linuxfoundation.org/en/LSB_Package_API The implementation currently supports integration into RPM and dpkg; due to its modular nature, support for more package managers could be added later on. I hope this implementation will act as a starting point for resurrecting the Berlin API process. Let us overcome the "Third-party software installation on Linux sucks" problem and strive to a brave new world of easily distributable Linux software! ;) Best regards, Denis Washington [1] http://www.linuxfoundation.org/en/Berlin_Packaging_API -- fedora-devel-list mailing list [EMAIL PROTECTED] https://www.redhat.com/mailman/listinfo/fedora-devel-list __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
LSB Package API
Hi, Some time ago, it was discussed on an LSB face-to-face meeting that an API should be developed that allows ISVs to install sotware packages which integrate into the package manager - the "Berlin Packaging API". While the idea seemed to be well received, there didn't seem much progress since then, except for a wiki page with a rundimentary proposal [1]. Considering that third-party software installation is an undeniably important weak spot of the Linux infrastructure, I found this was a shame. To reignite the the initiative, I decided to design and develop a prototype implementation of the Berlin API, most creatively named the "LSB Package API". It is designed as a simple D-Bus interface accompanied with an XML-based package description format. A detailed description and the source code can be found on this page: http://www.linuxfoundation.org/en/LSB_Package_API The implementation currently supports integration into RPM and dpkg; due to its modular nature, support for more package managers could be added later on. I hope this implementation will act as a starting point for resurrecting the Berlin API process. Let us overcome the "Third-party software installation on Linux sucks" problem and strive to a brave new world of easily distributable Linux software! ;) Best regards, Denis Washington [1] http://www.linuxfoundation.org/en/Berlin_Packaging_API __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
