Re: [Lf_desktop] LSB Package API
On Mon, 2008-06-23 at 15:14 +0100, Scott James Remnant wrote: My general feeling, having spoken to lots of ISVs, is that they don't actually _want_ this. In order to support their customers, they're well aware that they have to target particular distributions and versions - and they're quite happy working and certifying particular vendors individually. I'm not in the position of having the chance to speak with ISVs myself unfortunately. I only relied on what seemed to have been communicated on the LSB face-to-face meeting. It might be good if we had more application vendors (and also open-source project maintainers) speaking up on this issue in the general or the particular proposal. Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Thu, 2008-06-26 at 14:55 -0400, Jeff Johnson wrote: On Jun 25, 2008, at 11:49 AM, Denis Washington wrote: I hope what is in the data structures is sufficient and well-defined enough. And, what I increasingly tempt to believe, that we don't talk past each other. ;) I'm trying to build from your 4 tarballs. For starters, I'd suggest LGLPv2 rather than GPLv3 licensing. The reason is that the intended target is commercial ISV's, who are often (and justfiably) uncomfortable with the viral nature of GPL. The commercial ISV's _MUST_ link to your code. I completely agree. Did I add v3 somewhere? Second, I'd suggest a different name than lsb-package for now. I see no official buy-in from LSB yet, and all package names starting lsb- are reserved for LANANANANANANA ... Yeah. I thought of renaming it to Burgdorf API as this is my home town (and happens to be a German town too like Berlin). Alternatively, get your act togther and register lsb-package. But you are almost sure to annoy and confuse many with lsb-package. Personally, I'd suggest lsb-berlin until you get more buy-in, but YMMV. Or that. Back to building ... Build fails because #include i18n-defs.h from lib/lsb_package.c is nowhere obvious to be found. Doing touch lib/i18n-defs.h keeps gcc happy, but if you don't need the content, don't include the file. Next failure is this creating liblsb_package.la (cd .libs rm -f liblsb_package.la ln -s ../liblsb_package.la liblsb_package.la) make[2]: Leaving directory `/X/lsb-package/lsb_package-0.1/lib' Making all in daemon make[2]: Entering directory `/X/lsb-package/lsb_package-0.1/daemon' make[2]: *** No rule to make target `org.linuxbase.Packages.xml', needed by `dbus-server-bindings.h'. Stop. make[2]: Leaving directory `/X/lsb-package/lsb_package-0.1/daemon' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/X/lsb-package/lsb_package-0.1' make: *** [all] Error 2 error: Bad exit status from /X/tmp/rpm-tmp.86094 (%build) Where do I find (or generate) the org.linuxbase.Packages.xml file? I forgot to add some files to Makefile.am, so they weren't in the make dist tarball. I added them and uploaded the fixed version to the LSB wiki (same place as the previous one). Regards, Denis __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 24, 2008, at 1:38 PM, Denis Washington wrote: Sound like a plan? My primary goals here are two-fold: 1) avoiding disasters if bogus headers start to be added to an rpmdb. 2) exposing rpmdbAdd() (and rpmdbRemove()) methods for use by LSB/ISV/whatever applications that wish to register/unregister software on RPM managed systems. Sounds like a good plan, yeah. I'm glad being able to work with you on this, as you certainly have a LOT more experience than me concerning this. Thank you very much! No problem. Enumerating the necessary data elements that need to be present in a RPM header, and choosing _SOME_ representational markup, would seem to be on the critical path. (aside) dpkg its really the same fundamental problem, but a different target metadata representation. Ditto _your_package_manager_here for all instances of class. There are several existing representations of package manifests, both explicit and/or implicit that can be used to enumerate the necessary data elements to be included in the target metadata representation (note I did not say rpmdb). Simplest by far is find(1) output of a tree. i.e. an explicit list of paths to files, with stat(2) and digest (and acls/xattrs and selinux file contexts and whatever else is needed) implicitly derived from the tree. Other soft branding identification information, like vendor, packager, description, build host, etc would need to be added to the list of paths. While all of that information may be vitally important to ISV's and LSB and installer GUI's, all that rpmlib needs is NEVRA (N==name, E==epoch, etc), and mostly for human identification rather than installer functioning purposes. Is a find(1) path list gud enuf as a starting point? Or do you want to establish other, alternative, markup for expressing the necessary data elements. Other obviously complete and unsurprising candidates to describe necessary data elements to be included in target metadata are tar tvf and/or ls -al. Those formats are explicit, no data is implicitly derived from stat (2) of a file, and the file does not have to exist in order to construct a representation of target metadata. But there's lots and lots of other markups that could/should be used instead. What representation of target metadata works for you? 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: [packaging] LSB Package API
On Sun, Jun 22, 2008 at 01:34:31PM -0700, Dan Kegel wrote: On Sun, Jun 22, 2008 at 11:02 AM, Denis Washington [EMAIL PROTECTED] wrote: I don't think this is a corner case at all. For one thing, propietary applications might just don't play a role _because_ there is no really good distribution method for them - the typical chicken-and-egg problem. (I'm not saying this is the only reason, but an important one.) We're just not giving them an easy method of cross-distro integration. I think providing this is important. Sure, and that's why I support the LSB. Has everybody else given up on it? I've probably missed part of this discussion, but wanted to inject one anecdote. Stand-alone binary package installers are nothing particular novel or new; I'd gained experience using one, Autopackage, with Inkscape several years ago. Inkscape was virtually unknown at the time, so Autopackage gave us a significant benefit of providing users a way to quickly get Inkscape on distros that didn't yet include Inkscape. Before Autopackage we would maintain our own .deb and .rpm rules and specs, and we hoped Autopackage would obsolete that in favor of having a Universal Installer. Yet that really never came to be. First, Inkscape became recognized by distros, and they took over handling our packaging work for us. Meanwhile, the Autopackage developers (who had been subsidizing us by maintaining the .autopackage file for us), turned maintenance over to us. So on the one hand we were seeing our team workload *reduce* by relying on packaging-system-specific stuff, and *increase* by using autopackage. You might argue that it's different for proprietary software since distros wouldn't adopt them. Yet look at Xara, Flash, Opera, proprietary Java, and so on to see that they can and do (to the level they're able anyway). Second, while in theory Autopackage promised 100% easy install, anywhere, it was not without problems. The issue always seemed to be with low level dependencies that varied just subtly enough to break on one distro or another. So you ended up doing per-distro testing anyway (and couldn't count on help from the distro once you figured out the bug). I think this is an important point to not dismiss by saying, The design wasn't as good as ours is, or it just needed more testing, or whatever. The sad truth is that getting this to work 100% requires invalidating Murphy's Law. It's a broad fronted fight against entropy. I felt like we had tried to change our support from N distros to 1, yet ended up with N+1. Third, as Inkscape grew we had to account for more dependencies. Typically, there'd already be .rpm and .deb packages for them, but we'd be stuck having to do the autopackage packaging work ourselves. Now, you might think such an issue is irrelevant for proprietary software since they'd be packaged with all dependencies already included. Yet consider dependencies beyond just dynamic libraries. Consider if the app wants to interface with external programs or tools (imagemagick, java, sqlite, ...) or to shared data repositories (openclipart, fonts, etc.) Eventually, you find yourself having to do a lot of work that distros already take care of. Anyway, to sum up, as much as I loved the idea of autopackage and helped to advocate it, I really don't think the idea of a universal installer is viable. In the end it's a lot less effort to just collaborate with each of the distros and have packaging optimized for each. And I think efforts put into creating yet more universal installer techs maybe better invested in helping bring the existing packaging systems better into consistency with one another, or establishing best practices documents. Bryce __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: [packaging] LSB Package API
On Jun 23, 2008, at 3:43 PM, Jeff Licquia wrote: Part of the issue may be that most of the implementations so far have assumed that communication from a third-party installer would result in a pseudo-package being registered in the native package database, which leads people to believe that this is a new package format of some kind. The original idea, though, was for a communication protocol only. The native package manager may decide to store the results by creating a pseudo-package, but does not *have* to. I think we're willing to accept that the particular implementations of the Berlin API idea are wrong-headed, and perhaps re-do them. But the general idea--accepting that things such as InstallShield and InstallAnywhere are going to exist, and finding a way for them to cooperate with the underlying system instead of fighting with it-- isn't something I see anyone else trying to address. Speaking as the developer of an installer that has to fight with this all the time, all I'm really looking for is a simple command-line utility to interface to the native package manager beneath me. A simple abstraction layer in the style of the xdg-utils of the Portland project. Something that didn't require root would be nice, but I'm not sure how you'd handle this on a multi-user system. As it is now, I have to manipulate the underlying packaging system by-hand and through fake packages built at runtime and the like. It's crap, but it's the only outlet available until something better comes along. I guess I don't understand what is so difficult about the decision except that everyone has a better way than the other guy. Make something simple that gets the job done. Believe me, plenty of people will come along later and glom more crap onto it. It's open source, after all. Damon __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Jun 21, 2008, at 12:48 PM, Denis Washington wrote: My current interest in your code is disaster prevention, not otherwise. I welcome any motive if it improves code quality, so thanks anyway. ;) NP. My life is hell when rpmdb's get hosed up. Doesn't matter whether its a kernel mmap(2) flaw, an selinux policy-of-the-day typo, or a python script kiddies dain bramage. The Berlin API is a recipe for disaster so far. But I'm most definitely deeply and personally interested in not having to do the necessary rpmdb maintenance post-mortem if the implementation problems can be solved. Thought so. Hehe, now that your ideas have been thoroughly shredded ... Here's what is right about the Berlin API, and your approach: 0) A means to register/unregister a container (I'm avoiding the abused term package) of content on linux systems that is more lightweight and more flexible than existing containers like rpm/dpkg needs to be devised. 1) The Berlin API -- for all its flaws -- is a reasonable step towards the goal of registering/unregistering content on Linux systems. The largest flaw with the Berlin API LSB proposal imho was stating the problem as API methods without specifying the container internals, and how the internals should be represented and used. Implementing an API also involves practical development choices, like what language to use, that are incidental to the goal or registering/unregistering content. 2) Your implementation of the Berlin API got lots of things right. First of all, the API itself in your implementation is in Yet Another Library, which avoids he hugely complicated problem of how to retrofit an API onto already released software. Second, your implementation exists, a necessary precursor to identifying what else needs to be done. Making an implementation of the Berlin API exist in some meaningful form is more than anyone at LSB or on the packaging has achieved in ~1.5 years. Using dbus, and splitting the API from back-end processors using a domain-specific markup, are also sound architectural choices imho. So Congratulations! are most definitely in order. If you want to proceed, I'll be happy to write the RPM back-end for you. Its easier for me to just build a Header that I know will Just Work (tm) when passed to rpmdbAdd() than it is to explain to you what needs to be done. So far, your Berlin API implementation is sufficiently complete that I can see that a well-formed Header can be achieved. OTOH, you will have to supply the necessary data elements to add to the header, I can give you a list if you wish to proceed. (aside) Note that I'm quite capable of generating digital signatures on the generated header from the Berlin API any time that the trust discussions get out of hand. I can also likely help generate the XML (or YAML or klik or 0install or repo-md or ...) markup that is going to be needed to express the container contents. I'm actively generating markup of various persuasions from Header content @rpm5.org already. One more (or less) markup implementation simply doesn't matter, just try to get the markups prioritized please. What does matter is that all the widdle markups interconvert easily, and are sufficiently rich in expressive power that common elements, like file stat(2) data, or ACL's or xattr's or ... can be represented (and converted) without hugely complicated political packaging war battles that are ultimately about cellophane container wrappers. Sound like a plan? My primary goals here are two-fold: 1) avoiding disasters if bogus headers start to be added to an rpmdb. 2) exposing rpmdbAdd() (and rpmdbRemove()) methods for use by LSB/ISV/whatever applications that wish to register/unregister software on RPM managed systems. 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
I think you are looking for a solution to a problem that doesn't exist. For the corner cases of where this does apply (proprietary software) this is not enough of a use case to justify all the work required. I don't think this is a corner case at all. For one thing, propietary applications might just don't play a role _because_ there is no really good distribution method for them - the typical chicken-and-egg problem. (I'm not saying this is the only reason, but an important one.) We're just not giving them an easy method of cross-distro integration. I think providing this is important. Second, this way of distribution will help open-source projects as well. It would make it really easy for them to distribute bleeding-edge versions of there apps that integrate well into the packaging system without having to package for each and every package manager. It will also help those projects which are not widely used enough to be in distro repos, as they can more easily release binary distributions without having to depend on getting packaged by the distros. I really believe this decentralism would be very nice to have, especially in something as naturally decentralized as the open source community. Hi, I'd agree with Richard, because helping ISV won't automatically help Debian community to improve itself or its product : the Debian distro. Helping ISV, in a way that can benefit to all is to provide a very good, and solid-rock, basis for all to package software, respectful of LSB and knowledgeable in UNIX :). ISV may benefits from an improved packaging infrastructure but we would not benefits from them : we do not care about them at all :). So we want to care about invest time for those fishes... Sure, I'd like to have a Debian package (updated monthly) for IBM/Rational ClearCase (though I'd like to use Aegis...), for NVIDIA Gelato or for Autodesk Maya But we don't want to invest time before they take effort in packaging they stuff ;) I'm used to comply with very heterogeneous environment as Linux + HP-UX + Solaris + Window$ ... that's crazy... Common drawbacks in packaging approaches are, usually: - poor asserts of target particularities, - lack of UNIX knowledge, - (very, very) bad scripting (all to often are very bad *csh* scripts for example). IMHO, to improve such things, to help design install scripts, or install schema, in a way that can fit in UNIX / Debian philosophy, we can produce documentation that: - respect LSB hierarchy so that files goes where we want them to go, /etc/init.d [debian]/etc/default/ /usr/bin /usr/bin /var/share/ /var/lib/ - help in admin files leveraging (portability) : init.d scripts, cron scripts, ... - help in libs dependencies ( symbols) : if ISV package depends on libjpeg6. then we must provides ways in packaging system to assert on this dep() : with the new *dpkg-shlibdeps* we can tracks such symbol dependencies... - help document the new *dpkg* feature of triggers, this might help... - and last but not the least : KISS ; why use DBUS !!! when all Debian packaging tools use either Bash either Perl ? why complexify things ? that's the root of all evil : To conclude: the best way is to document, and eventually to implement helper scripts. Not to imagine API that would become obsolete in a month of two... Figure out where ISV script are not able to cope with Debian system and offer solution for that problems, specifically :). Best regards, Michel -- .''. -our own. Resistance is futile. __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, Jun 21, 2008 at 7:17 PM, Denis Washington [EMAIL PROTECTED] wrote: On Sat, 2008-06-21 at 13:01 -0400, Jeff Johnson wrote: On Jun 21, 2008, at 12:48 PM, Denis Washington wrote: On Sat, 2008-06-21 at 12:27 -0400, Jeff Johnson wrote: On Jun 21, 2008, at 12:05 PM, Denis Washington wrote: What if the transaction fails? register_package() would have returned without error although the registration was unsuccessful then, and all files would already be installed. What if you've added a header, but your daemon exits before successfully computing and adding RPMTAG_SIZE withthe _close_package() method? Got me. Although, if a dummy value (e.g. 0) was added in _register_package(), an unsuccessful _close_package() wouldn't be a harm at all. The header would be complete anyway. Hint: RPMTAG_SIZE simply does not matter. Nor do Vendor: Packager: Description: Summary: and all the other goopiness carried in markup (because its easy to add) and rpmdb Headers. OTOH, RPMTAG_FILESTATES is gonna matter a _LOT_. So will leaving stale locks, and forgetting to attach stderr when your widdle daemon forks. Could you explain what should go in RPM_FILESTATES? It's not listed in the LSB specification. Sorry, but who care on LSB RPM specification aka RPM v3 (other for some useful docu) ? RPM 4.4.2 could not produce it, do you know ? Also , do you know that the LSB RPM spec was bourne only because someone suggest to write some referral on the LSB on MAXIMUN RPM ? Also again do you know that in REDHAT RPM GUIDE someone suggest the author to describe in appendices the RPMV3 package format only for the better docu ? And guess who it is this someone ? R : Jeff Johnson So think more carefully before expressing silly opinions on Jeff Johnson, which authority in the filed is beyond discussion.
Re: LSB Package API
On Sat, Jun 21, 2008 at 7:35 PM, devzero2000 [EMAIL PROTECTED] wrote: On Sat, Jun 21, 2008 at 7:17 PM, Denis Washington [EMAIL PROTECTED] wrote: On Sat, 2008-06-21 at 13:01 -0400, Jeff Johnson wrote: On Jun 21, 2008, at 12:48 PM, Denis Washington wrote: On Sat, 2008-06-21 at 12:27 -0400, Jeff Johnson wrote: On Jun 21, 2008, at 12:05 PM, Denis Washington wrote: What if the transaction fails? register_package() would have returned without error although the registration was unsuccessful then, and all files would already be installed. What if you've added a header, but your daemon exits before successfully computing and adding RPMTAG_SIZE withthe _close_package() method? Got me. Although, if a dummy value (e.g. 0) was added in _register_package(), an unsuccessful _close_package() wouldn't be a harm at all. The header would be complete anyway. Hint: RPMTAG_SIZE simply does not matter. Nor do Vendor: Packager: Description: Summary: and all the other goopiness carried in markup (because its easy to add) and rpmdb Headers. OTOH, RPMTAG_FILESTATES is gonna matter a _LOT_. So will leaving stale locks, and forgetting to attach stderr when your widdle daemon forks. Could you explain what should go in RPM_FILESTATES? It's not listed in the LSB specification. Sorry, but who care on LSB RPM specification aka RPM v3 (other for some useful docu) ? RPM 4.4.2 could not produce it, do you know ? Also , do you know that the LSB RPM spec was bourne only because someone suggest to write some referral on the LSB on MAXIMUN RPM ? Also again do you know that in REDHAT RPM GUIDE someone suggest the author to describe in appendices the RPMV3 package format only for the better docu ? And guess who it is this someone ? R : Jeff Johnson So think more carefully before expressing silly opinions on Jeff Johnson, which authority in the filed is beyond discussion.
Re: LSB Package API
On Jun 21, 2008, at 1:52 PM, devzero2000 wrote: (aside) It is time for LSB RPM SPEC to move to RPM4 packaging format Indeed. That is the raison d'etre for [EMAIL PROTECTED]. I have not pursued because of zero (yes zero!) interest from vendor's or LSB. Not my problem. I will do a IETF RFC when I get around to it, my forward looking develoment goal is XAR, not RPMv4/LSB, format for packaging. 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, Jun 21, 2008 at 8:19 PM, Jeff Johnson [EMAIL PROTECTED] wrote: On Jun 21, 2008, at 1:52 PM, devzero2000 wrote: (aside) It is time for LSB RPM SPEC to move to RPM4 packaging format Indeed. That is the raison d'etre for [EMAIL PROTECTED]. I have not pursued because of zero (yes zero!) interest from vendor's or LSB. So it is likely also for Berlin API zero interest because it is based on LSB RPM specs. Not my problem. I will do a IETF RFC when I get around to it, my forward looking develoment goal is XAR, not RPMv4/LSB, format for packaging. Ok. I already know this and also agreed on the motivation. In the meantime could be useful to have more docu on the rpm4 packaging format, almost for the tags. There is some dubt about the semantic of some of these (RPMTAG_SIZE for example and %ghost and the like discussed recently) Best regards 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
Re: LSB Package API
On Sat, Jun 21, 2008 at 9:46 PM, Jeff Johnson [EMAIL PROTECTED] wrote: On Jun 21, 2008, at 2:45 PM, devzero2000 wrote: Ok. I already know this and also agreed on the motivation. In the meantime could be useful to have more docu on the rpm4 packaging format, almost for the tags. There is some dubt about the semantic of some of these (RPMTAG_SIZE for example and %ghost and the like discussed recently) There is rpm --xml, true WYSIWIG. There is also rpm --yaml, much easier on the eyes. And if one looks carefully, one can also see that RPMTAG_FILENAMES MUST be sorted, and that dependencies SHOULD be sorted (excwpt when vendors/packagers choose to do something different). Without any standard, more doco just adds to the cacophony of packaging wars imho. A true semantic interpretation of how tags should be used/interpreted is largely out of rpm development scope these days. Which is also the basis for my opinion that the opportunity for a LSB Packaging Standard to be useful closed several years ago. There are way too many RPM differences these days for documentation to clarify much of anything. But YMMV, everyone has their own opinion, easily and obviously understood. No. I am wrong and you are right: I am finally aware. What is important it is the rpm5 development no other thing. Best Regards Elia 73 de Jeff __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
RE: LSB Package API
LSB has chosen to leave upgrade UNSPECIFIED, and has also chose in the Berlin API to ignore the fact that both dpkg/rpm versions are a triple of Epoch/Version/Release. Pretending that a version string can be anything, opaquely handled, including E:V-R, or something else, misses the issue that upgrade based on version is undecidable until version is well formed, and a collate sequence is defined for upgrade comparison. the absence of any description of what version means is a bug in LSB, whether or not that issue is picked up by the Berlin proposal. upgrade is a little dicier in the LSB sense, as it seems different packaging systems may do quite different things here. Responding to that by pretending upgrades don't exist is the cowardly approach, I know... __ RPM Package Managerhttp://rpm5.org LSB Communication Listrpm-lsb@rpm5.org
