Re: [gentoo-dev] Re: maintainer-needed@ packages need you!
Hello, On Mon, 01 Sep 2014 07:15:53 +0800 Patrick Lauer wrote: > On Sunday 31 August 2014 11:39:22 hasufell wrote: > > Martin Vaeth: > > > hasufell wrote: > > >> On 08/30/2014 02:35 PM, J. Roeleveld wrote: > > >>> For net-im/skype, > > >> > > >> Screw skype. > > > > > > Please don't. Not all communication partners are linux users. > > > > Tox is multiplatform. > > > > >> We have tox [1] on the way. > > > > > > This is far from being ready, especially for non-specialists. > > > It is not even foreseeable whether the Android client will ever > > > be able to do at least audio. (So far, I was not even able to > > > exchange any messages at all. It may be my mistakes, but > > > if I am not able to do it, how could I expect this from casual > > > computer users?) > > > > This has nothing to do with specialists. Tox is configuration-free. > > > > And sure, it's pre-alpha as indicated in my previous mail. > > So it doesn't work, but you feel the need to feel superior by telling > everyone > else that they are doing it wrong. Can't agree with you here. I just tried tox (utox client) from tox-overlay. Works like charm from the box, the only "unusual" thing I did is keyword added to package.keywords in order to install live ebuild. I tested text messages, audio and video from double-nat environment (where SIP clients can work only using stune and only some of them). It should be noted that at least in Linux skype is much harder to install and use since it requires pulseaudio and I don't use that sh^W stuff. So skype reqires its own LXC container set up which is doable, but costed me a day (with all tight isolation stuff). And I even had not mentione that installation of skype equals to trojan injection into the system (that's why I used all that LXC and separate X server precautions). Best regards, Andrew Savchenko pgpuKJtMSKbzZ.pgp Description: PGP signature
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
On 09/06/14 14:15, Ciaran McCreesh wrote: On Sat, 06 Sep 2014 14:10:50 -0400 "Anthony G. Basile" wrote: It is incorrect to say "I'm going about it all wrong" when I'm not doing any designing. Not doing any designing *is* going about it all wrong... Getting this right is hard work. Someone needs to do it. I plan to help the portage team with this. In fact, I really like mgorny's query-installed|| thingy. I am in the process of familiarizing myself with paludis and can help there too. Brace yourself for patches! At this point I've gotten enough feedback for the next re-write. Basically I'm going to change the emphasis on ELF and make it clearer that we're looking for whatever dynamic linking information is appropriate for the executable format in question and relegate ELF to an example. That's all very well, but it's completely useless until someone designs the whole thing. We're back to specifying the colours of the shelves before we've designed the bikeshed. This is the beginings of design. We can talk about something which does not exist and then make it exist. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
On Sat, 06 Sep 2014 14:10:50 -0400 "Anthony G. Basile" wrote: > It is incorrect to say "I'm going about it all wrong" when I'm not > doing any designing. Not doing any designing *is* going about it all wrong... Getting this right is hard work. Someone needs to do it. > At this point I've gotten enough feedback for the next re-write. > Basically I'm going to change the emphasis on ELF and make it clearer > that we're looking for whatever dynamic linking information is > appropriate for the executable format in question and relegate ELF to > an example. That's all very well, but it's completely useless until someone designs the whole thing. We're back to specifying the colours of the shelves before we've designed the bikeshed. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
On 09/06/14 13:09, Ciaran McCreesh wrote: I'm not disagreeing with an API for interoperability, in principle. I just think the way you're going about it is all wrong, and we're all better not having an API at all than having a bad one that we'll have to support for all eternity. You and the portage team can get together to discuss how you wish to design this API. The GLEP is clear on this point: "It is not the purpose of this GLEP to specify the details of a common API for exporting the above information. Even less so is it our purpose to delineate the implemenatation details for each PM. However, a common API for exporting the above information should be developed and specified by the PM teams and include in future PMS documentation." It is incorrect to say "I'm going about it all wrong" when I'm not doing any designing. The GLEP specifies the information to be exported, without implementaton details --- it does give mgorny's suggested API only as a suggestion. I've removed all references to VDB as the caching mechanism. Each PM can choose whatever mechanism it wants as long as it caches and exports the information specified in the GLEP. Once the teams decide, they document their common API in PMS. Here's a quick lesson, starting with bad design: Now this is still fragile and makes all kinds of assumptions, and we could argue eternally about the naming, but it's a heck of a lot better than what we started off with. In particular, it's *very* high level, and relies upon the package mangler for everything involving parsing. I trust you and the portage team to do this right. I've simply pointed out instances of utilities which do not belong in a PM, and which can make use of information generated by a PM at build time, but which is expensive to recalculate later. This argues for caching this information and exporting it via a common API so tools can work as well with paludis as with portage as with xyz from the future. At this point I've gotten enough feedback for the next re-write. Basically I'm going to change the emphasis on ELF and make it clearer that we're looking for whatever dynamic linking information is appropriate for the executable format in question and relegate ELF to an example. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
On Sat, 06 Sep 2014 13:00:03 -0400 "Anthony G. Basile" wrote: > On 09/06/14 12:18, Ciaran McCreesh wrote: > > Well eix is buggy, PMS-violating and doesn't support EAPIs properly, > > and the utilities in gentoolkit and portage-utils are better > > implemented natively in a package manager. I don't know what elfix > > does, but if it's anything like the other three, I'd rather > > reimplement it in a PMS compliant manner for Paludis than to > > provide flaky external APIs that encourage people to write broken > > code... > > elfix contains revdep-pax which does what revdep-rebuild does, except > that it migrates PaX flags from executables to libraries and vice > versa. There's a reason we reimplemented revdep-rebuild: doing it *properly* requires passing special information to the dependency resolver telling it not to reuse certain packages for breaking circular dependencies. > These sorts of utilites pop up over and over again. A lot of them are because Portage doesn't provide a decent API for users... > You cannot put every utility that needs package information into a > package manager. No, just the ones that do anything fiddly, and that certainly includes eix and most of gentoolkit. > An API for interoperability between PM's and other > tools is meaningful. Refusing to do so leads to the sort of comment > you see in https://bugs.gentoo.org/show_bug.cgi?id=506276#c43. > > Ironically, the very standards I seek in GLEP 64 would benefit the > paludis world most. I'm not disagreeing with an API for interoperability, in principle. I just think the way you're going about it is all wrong, and we're all better not having an API at all than having a bad one that we'll have to support for all eternity. Here's a quick lesson, starting with bad design: for x in /var/db/pkg/*/* ; do cat $x/DESCRIPTION # etc But what if VDB isn't there? for x in $(pmtool get_vdb_path)/*/* ; do cat $x/DESCRIPTION But */* makes some strong assumptions about the filesystem layout. In particular, it's going to be wrong if we ever do multi-slot. So: for x in $(pmtool get_vdb_package_directories) ; do cat $x/DESCRIPTION But this is still very fragile. What if we switch to sqlite? for x in $(pmtool get_unique_ids_for_vdb_packages) ; do pmtool get_metadata_variable $x DESCRIPTION But hang on... We don't really want VDB. We want installed packages. (This is important, and it's not just a naming thing.) for x in $(pmtool get_unique_ids_for_installed_packages) ; do pmtool get_metadata_variable $x DESCRIPTION What if descriptions move to metadata.xml? for x in $(pmtool get_unique_ids_for_installed_packages) ; do pmtool get_description $x And finally, what if EAPI 7 changes things? It's probably best to error out. export PMTOOL_BARF_ON_UNKNOWN_EAPIS="1 2 3 4 5 6" for x in $(pmtool get_unique_ids_for_installed_packages) ; do pmtool get_description $x Now this is still fragile and makes all kinds of assumptions, and we could argue eternally about the naming, but it's a heck of a lot better than what we started off with. In particular, it's *very* high level, and relies upon the package mangler for everything involving parsing. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
On 09/06/14 12:18, Ciaran McCreesh wrote: Well eix is buggy, PMS-violating and doesn't support EAPIs properly, and the utilities in gentoolkit and portage-utils are better implemented natively in a package manager. I don't know what elfix does, but if it's anything like the other three, I'd rather reimplement it in a PMS compliant manner for Paludis than to provide flaky external APIs that encourage people to write broken code... elfix contains revdep-pax which does what revdep-rebuild does, except that it migrates PaX flags from executables to libraries and vice versa. There exists a category of tools that can make use of PM's cached information which should not be part of PM. elfix is one such example. Let me give you another: https://bugs.gentoo.org/show_bug.cgi?id=506276#c42. That bug is about removing SYMLINK_LIB=yes. To do so properly and migrate systems that use symlinks for /libXX, vapier wrote a migration script which at one point "walk[s] the vdb looking for files that installed into /lib32 and /lib." These sorts of utilites pop up over and over again. You cannot put every utility that needs package information into a package manager. An API for interoperability between PM's and other tools is meaningful. Refusing to do so leads to the sort of comment you see in https://bugs.gentoo.org/show_bug.cgi?id=506276#c43. Ironically, the very standards I seek in GLEP 64 would benefit the paludis world most. When the package is installed, that data should have been cached. But package.provided packages *aren't* installed. They are merely treated as if they were installed, without actually being installed, so that data isn't available. Right. hasufell's email made it clear. This is pretty hacky so we don't want to go there. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
Anthony G. Basile: > On 09/06/14 12:12, hasufell wrote: >> Anthony G. Basile: And when you do ask, is a package that's "provided" installed, and if so, what's its metadata? >>> When the package is installed, that data should have been cached. >>> >> Afaik there is nothing "cached" if you put stuff in package.provided. >> It's a terrible hack, unless I missed something. >> > > I wasn't sure what Ciaran was talking about there. If its hacky, then > we certainly don't want to standardize it in the GLEP. > Well, you have to to define what tools can expect from provided/installed packages. That means either say "you cannot expect anything, because there might or might not be metadata" or say "you can expect metadata for any provided/installed package" in which case package.provided feature has to be removed from portage.
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
On Sat, 06 Sep 2014 12:07:10 -0400 "Anthony G. Basile" wrote: > No, but the resolution of the conditionals may change over time and > it would be nice to have the information as it was at the time of the > build. You have USE available... > Neither do we want to hamper what developers might want to do. The > authors of sys-apps/elfix, app-portage/gentoolkit, > app-portage/portage-utils, app-portage/eix are not "utterly > inflexible" utilities makeing "strange assumptions". They are useful > utilities as anyone following this thread would agree. Their > assumption would go from being non-standard to standard with the > acceptance of this GLEP. They would then operate with both portage > and paludis. Well eix is buggy, PMS-violating and doesn't support EAPIs properly, and the utilities in gentoolkit and portage-utils are better implemented natively in a package manager. I don't know what elfix does, but if it's anything like the other three, I'd rather reimplement it in a PMS compliant manner for Paludis than to provide flaky external APIs that encourage people to write broken code... > You can do this, but it would be terribly inefficient to regenerate > expensive information already generated by PM. For example, `equery` > from gentoolkit allows the user to gather all sorts of information > about installed packages, eg. "What package does this file belongs > to?" A very natural question. Without this information cached in > portage's VDB, how would equery do this? It would have to rebuild > package by package until it finds one that gives the file we're > looking for?! This is absurd. Rather gentoolkit opens > up /var/db/pkg and read the information out of there. It would use a package-manager provided interface to do it, which would remove the need to implement direct VDB reading externally. This is good, because reading VDB *correctly* is difficult, and we don't want lots of third party tools doing it badly. For example, with Paludis, you would do 'cave print-owners' to get this information, and your code works correctly even if VDB switches formats and even if it isn't located at /var/db/pkg. > For gentoolkit to work "in proper generality" we would need PM's to > export this information by some common API, so there is a > generality. That's what GLEP 64 is about. You point argues in favor > of the GLEP. Well gentoolkit is Portage-specific, since we've reimplemented all the useful stuff properly in Paludis... To reimplement gentoolkit in proper generality, externally, you'd need an awful lot more than what you're asking for. > > And when you do ask, is a package that's "provided" installed, and > > if so, what's its metadata? > > When the package is installed, that data should have been cached. But package.provided packages *aren't* installed. They are merely treated as if they were installed, without actually being installed, so that data isn't available. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
On 09/06/14 12:12, hasufell wrote: Anthony G. Basile: And when you do ask, is a package that's "provided" installed, and if so, what's its metadata? When the package is installed, that data should have been cached. Afaik there is nothing "cached" if you put stuff in package.provided. It's a terrible hack, unless I missed something. I wasn't sure what Ciaran was talking about there. If its hacky, then we certainly don't want to standardize it in the GLEP. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
Anthony G. Basile: >> >> And when you do ask, is a package that's "provided" installed, and if >> so, what's its metadata? >> > > When the package is installed, that data should have been cached. > Afaik there is nothing "cached" if you put stuff in package.provided. It's a terrible hack, unless I missed something.
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
On 09/06/14 10:04, Ciaran McCreesh wrote: On Sat, 06 Sep 2014 09:51:58 -0400 "Anthony G. Basile" wrote: Paludis doesn't do this (and historically, Portage didn't either). We store USE etc. This is useful because it allows us to detect when people have been mucking around with DEPEND and the like. This was a suggestion from mgorny and I trust his opinion on the matter. It does make sense to have metadata catche which is conditionally evaluated be exported. Well what are you planning to use it for? Are you really suggesting that people are going to implement EAPI-aware, compliant dependency parsers that can't figure out conditionals? No, but the resolution of the conditionals may change over time and it would be nice to have the information as it was at the time of the build. However, this point is minor. I'm certain something can be worked out among the PMS people as to what is best here. After all, the request did come from the portage camp. 1) This cannot be utterly arbitrary because there are utilities and portions of portages code which does make use of precisely this information. What Portage does internally is up to Portage. What we don't want is to encourage external utilities that are utterly inflexible and that make strange assumptions. Neither do we want to hamper what developers might want to do. The authors of sys-apps/elfix, app-portage/gentoolkit, app-portage/portage-utils, app-portage/eix are not "utterly inflexible" utilities makeing "strange assumptions". They are useful utilities as anyone following this thread would agree. Their assumption would go from being non-standard to standard with the acceptance of this GLEP. They would then operate with both portage and paludis. 2) With the exception of some embedded systems where everything is statically linked, all modern systems have dynamic linking. And all dynamic linking has information in its objects which associates the executables with the library they link against. This is the essential information to be stored. Which isn't what you're asking for... You're asking for it in a particular format. I will incorporate better language which goes to the heart of what's needed and makes the ELF quantities an example rather than the demanded format. In other words, I will remove the ELF bias. Since dynamic linking information is generated at build time, is expensive to regenerate and is useful to other utilities, it falls under the scope of the GLEP. Of course we wish to extend this to any executable format. We focus on ELF because of its familiarity, but not at the exclusion of others. 6) Without a standard here, we have utilites which make use of this cached information in the tree which only work with portage but not paludis. This problem can be solved by removing those utilities, which is undesireable, by standardizing what needs to be exported from the PM, or by living with the status quo which is having useful packages in the tree which don't work with paludis. The solution is to replace those utilities with something that works in proper generality. You can do this, but it would be terribly inefficient to regenerate expensive information already generated by PM. For example, `equery` from gentoolkit allows the user to gather all sorts of information about installed packages, eg. "What package does this file belongs to?" A very natural question. Without this information cached in portage's VDB, how would equery do this? It would have to rebuild package by package until it finds one that gives the file we're looking for?! This is absurd. Rather gentoolkit opens up /var/db/pkg and read the information out of there. For gentoolkit to work "in proper generality" we would need PM's to export this information by some common API, so there is a generality. That's what GLEP 64 is about. You point argues in favor of the GLEP. You've also not discussed how this interacts with Portage's package.provided misfeature. Finally, you don't have any way of using this information, since you don't have a way of knowing what packages are installed. I don't understand your reasoning behind these points, can you please explain. Well you can't ask for information about packages unless you know what's installed, and you haven't asked for a general API for that. Ah, okay. That should be added explicitly since its only implied right now. Thanks. And when you do ask, is a package that's "provided" installed, and if so, what's its metadata? When the package is installed, that data should have been cached. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
On Sat, 06 Sep 2014 09:51:58 -0400 "Anthony G. Basile" wrote: > > Paludis doesn't do this (and historically, Portage didn't either). > > We store USE etc. This is useful because it allows us to detect when > > people have been mucking around with DEPEND and the like. > > This was a suggestion from mgorny and I trust his opinion on the > matter. It does make sense to have metadata catche which is > conditionally evaluated be exported. Well what are you planning to use it for? Are you really suggesting that people are going to implement EAPI-aware, compliant dependency parsers that can't figure out conditionals? > 1) This cannot be utterly arbitrary because there are utilities and > portions of portages code which does make use of precisely this > information. What Portage does internally is up to Portage. What we don't want is to encourage external utilities that are utterly inflexible and that make strange assumptions. > 2) With the exception of some embedded systems where everything is > statically linked, all modern systems have dynamic linking. And all > dynamic linking has information in its objects which associates the > executables with the library they link against. This is the > essential information to be stored. Which isn't what you're asking for... You're asking for it in a particular format. > 6) Without a standard here, we have utilites which make use of this > cached information in the tree which only work with portage but not > paludis. This problem can be solved by removing those utilities, > which is undesireable, by standardizing what needs to be exported > from the PM, or by living with the status quo which is having useful > packages in the tree which don't work with paludis. The solution is to replace those utilities with something that works in proper generality. > > You've also not discussed how this interacts with Portage's > > package.provided misfeature. > > > > Finally, you don't have any way of using this information, since you > > don't have a way of knowing what packages are installed. > > I don't understand your reasoning behind these points, can you please > explain. Well you can't ask for information about packages unless you know what's installed, and you haven't asked for a general API for that. And when you do ask, is a package that's "provided" installed, and if so, what's its metadata? -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
Anthony G. Basile: >> >>> A list of all files belonging to the package, along with a designation >>> of the file type (regular, directory, symlink, pipe, etc), MD5SUM or >>> other checksum, and mtime time. >> Packages aren't allowed to install pipes. > > Okay I will remove pipes. Do you have a reference btw about what types > of files can be installed. > http://dev.gentoo.org/~ulm/pms/head/pms.html#x1-15800012.6 > 12.6 Other Files > > Ebuilds must not attempt to install any other type of file (FIFOs, device > nodes etc). The explicit list of allowed ones is in the chapters above.
Re: [gentoo-dev] why does net-misc/tor enables "Log notice syslog" ?
On 09/06/2014 01:57 PM, Anthony G. Basile wrote: > On 09/06/14 06:44, Toralf Förster wrote: >> After a recent discussion at #tor an #gentoo-dev /me wonders if the >> syslog level of net-misc/tor should at least be changed from "notice" >> to "warn" -or better - be fully unset like upstream does it ? >> >> > I can entertain that, but this should be in a bug report: 1) While > anyone is welcome to discuss the issue, I don't think most gentoo devs > care about this. 2) A bug report leaves behind a record of what's going > on which can later be easily searched for using bugizlla's magic. > yep: https://bugs.gentoo.org/show_bug.cgi?id=522256 -- Toralf pgp key: 0076 E94E
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
On 09/06/14 09:10, Ciaran McCreesh wrote: On Sat, 06 Sep 2014 09:03:13 -0400 "Anthony G. Basile" wrote: Note that, as with the Metadata Cache, these variable should be stored with all the conditionals evaluated. Paludis doesn't do this (and historically, Portage didn't either). We store USE etc. This is useful because it allows us to detect when people have been mucking around with DEPEND and the like. This was a suggestion from mgorny and I trust his opinion on the matter. It does make sense to have metadata catche which is conditionally evaluated be exported. If PM's also want to the conditionals in there, there is nothing in the GLEP which prevents it. A list of all files belonging to the package, along with a designation of the file type (regular, directory, symlink, pipe, etc), MD5SUM or other checksum, and mtime time. Packages aren't allowed to install pipes. Okay I will remove pipes. Do you have a reference btw about what types of files can be installed. A list of all executable or shared objects for each package and the corresponding linking information, including full path to the object, its architecture and ABI, SONAME, RPATH and any NEEDED objects they link against, as reported by `readelf` on ELF systems, or similar tools for other executable formats. Currently this information is being cached by Portage in NEEDED.ELF.2, NEEDED.MACHO.3, NEEDED.XCOFF, NEEDED.PECOFF, etc. This is utterly arbitrary, and introduces a dependency on particular non-standard package whose behaviour we don't control. Not everything is C and ELF, and we shouldn't be encouraging "solutions" that make the assumption that they are... 1) This cannot be utterly arbitrary because there are utilities and portions of portages code which does make use of precisely this information. 2) With the exception of some embedded systems where everything is statically linked, all modern systems have dynamic linking. And all dynamic linking has information in its objects which associates the executables with the library they link against. This is the essential information to be stored. 3) You are correct that the bias there is towards ELF. So I will add language indicating similar information for the other executable formats. 4) Of course not everything is C. (BTW the better example to make your case, is FFLAGS for fortran which I omitted --- I will add language here too). The point is to record dynamical linking information where it is generated. Such information is obtained at build time, is expensive to regenerate, is useful at a later time for both the PM and other utilities, and so should be cached. We would record this even for an executable generated from fortran source, say, that links against some libraries. 5) For packages which don't have any dynamical linking, we just leave those things blank. ie. asking for the SONAME of some file /usr/share/pkgconfig/ returns null with some error. 6) Without a standard here, we have utilites which make use of this cached information in the tree which only work with portage but not paludis. This problem can be solved by removing those utilities, which is undesireable, by standardizing what needs to be exported from the PM, or by living with the status quo which is having useful packages in the tree which don't work with paludis. You've also not discussed how this interacts with Portage's package.provided misfeature. Finally, you don't have any way of using this information, since you don't have a way of knowing what packages are installed. I don't understand your reasoning behind these points, can you please explain. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] rfc: adding sys-apps/iproute2 to the @system set
On Sat, Sep 6, 2014 at 9:23 AM, Anthony G. Basile wrote: > > I'm not sure we can get rid of python2 and have only python3, but if that's > possible, absolutely punt it! The bloat I'm talking about includes size, > but more importantly, I'm concerned about cpu time. When building on a minor > arch where your CPU speed is 600 MHz and you only have 256MB of ram (and > lots of slow swap to help for monsters like gcc-4.8), you feel the bloat in > days of waiting. The other issue is the parallel build issue. @system and its deps can't be built in parallel. That means a penalty for every update to any of those packages for life. Any kind of actual end-user application that goes into @system greatly compounds this problem. Applications tend to have lots of dependencies. There isn't much question that stuff like rsync and nano (via the editor virtual) should be in the stage3 just so that we're not ripping our hair out during installation. However, they really don't need to be part of the system set. How many packages really need to depend on an editor (and I'm talking linking and other technical issues that affect builds - not practical use)? Of course, people probably don't want to unmerge the last text editor or rsync from their system which is why it doesn't hurt to have some kind of mix-in that defines minimally-useful stuff like this all the same, but which separates it from the practice of not declaring dependencies. I'm sure all of us have our favorite utilities that we put on every Gentoo install we do (tmux/screen, atop, vim, etc). The problem is that once you go down that road we end up in endless debates. If we instead ask questions like "what are all the packages which >30% of the tree would otherwise have to depend on if not in @system?" or "what is the minimum set of packages that need to be preinstalled to build anything else in the tree?" we have unambiguous questions that have unambiguous answers. -- Rich
Re: [gentoo-dev] rfc: adding sys-apps/iproute2 to the @system set
On Sat, Sep 6, 2014 at 8:41 AM, Patrick Lauer wrote: > > And by the same reasoning of "bloat" we should remove openssh ( and maybe even > rsync ;) ) because it's not strictly needed - so maybe we want a "minimal" and > a "useful" stage3 ? I could care less what is in the stage3, which only affects the content of a gentoo system for its first 5 minutes of existence. I care more about what is in the system set. Right now they're forced to be the same thing. bit there is no reason that this has to be so. If the stage3 bundles a bunch of stuff that either goes away at the first --depclean or is just part of the initial world and can be trivially removed (and there are no issues with parallel builds), then I don't have a huge problem with it, though I still think that openssh in the stage3 is overkill. Minimal vs useful is certainly a good distinction, but just as with the whole server profile debate the definition of useful varies considerably. I think that what would make the most sense is to implement mix-ins so that everybody and their uncle can maintain their own personal idea of a useful layer, and then strip the stage3 down to what you really need to bootstrap a system, and limit the system set to the stuff we really don't want to stick in every *DEPEND (libc, baselayout, etc). Trying to get everybody to agree on what is "useful" just leads to endless bikeshedding - better to just let everybody or every project have their own way and let everybody decide which way works for them. -- Rich
Re: [gentoo-dev] rfc: adding sys-apps/iproute2 to the @system set
On 09/06/14 08:41, Patrick Lauer wrote: On Friday 05 September 2014 12:34:11 William Hubbs wrote: All, there is a bug open requesting that we add sys-apps/iproute2 to the system set [1]. Originally the request was to drop net-tools, but it has become just adding iproute2. I wouldn't mind either option - net-tools has been deprecated for a decade, but if we still ship it as default it will be used. Some people seem to think that stage3 is bloated - last time I looked at it there was lots of really-not-needed stuff like two python interpreters. If you want to de-bloat work on that, the extra 150kB or whatever of iproute2 are so small that it's barely noticeable. I'm not sure we can get rid of python2 and have only python3, but if that's possible, absolutely punt it! The bloat I'm talking about includes size, but more importantly, I'm concerned about cpu time. When building on a minor arch where your CPU speed is 600 MHz and you only have 256MB of ram (and lots of slow swap to help for monsters like gcc-4.8), you feel the bloat in days of waiting. And by the same reasoning of "bloat" we should remove openssh ( and maybe even rsync ;) ) because it's not strictly needed - so maybe we want a "minimal" and a "useful" stage3 ? This begs the question "useful for what"? We have competing criteria. So one criterion is "a final stage in a catalyst run which can seed the next round" and the other is "a stage from which any gentoo system can be built." Both depend on the environment in which you are building, eg. if you unpack a stage3 onto a partition, reboot, and then expect to be able to rsync portage, then you need rsync in there, and maybe some other stuff like wget or curl. Alternatively, I could build up my system in a chroot in which I bind mount /usr/portage from the host, the way catalyst does -- its a bit more complicated than that but you get the idea. Then I don't need rsync. So yeah, there is a slippery slope here, but having two packages that achieve the same purpose is overstepping. The better analogy would be having openssh and dropbear and then saying that the latter is only 150kB, so let's just add it. Have fun, Patrick -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
On Sat, 06 Sep 2014 09:03:13 -0400 "Anthony G. Basile" wrote: > Note that, as with the Metadata Cache, these variable should be stored > with all the conditionals evaluated. Paludis doesn't do this (and historically, Portage didn't either). We store USE etc. This is useful because it allows us to detect when people have been mucking around with DEPEND and the like. > A list of all files belonging to the package, along with a designation > of the file type (regular, directory, symlink, pipe, etc), MD5SUM or > other checksum, and mtime time. Packages aren't allowed to install pipes. > A list of all executable or shared objects for each package and the > corresponding linking information, including full path to the object, > its architecture and ABI, SONAME, RPATH and any NEEDED objects they > link against, as reported by `readelf` on ELF systems, or similar > tools for other executable formats. Currently this information is > being cached by Portage in NEEDED.ELF.2, NEEDED.MACHO.3, NEEDED.XCOFF, > NEEDED.PECOFF, etc. This is utterly arbitrary, and introduces a dependency on particular non-standard package whose behaviour we don't control. Not everything is C and ELF, and we shouldn't be encouraging "solutions" that make the assumption that they are... You've also not discussed how this interacts with Portage's package.provided misfeature. Finally, you don't have any way of using this information, since you don't have a way of knowing what packages are installed. -- Ciaran McCreesh signature.asc Description: PGP signature
[gentoo-dev] RFC: GLEP 64: Export Package Manager Cached Information. (Was: RFC: GLEP 64: Standardize contents of VDB and establish and API for exporting this information.)
Hi everyone, I've incorporated suggestions from the previous round of discussions in the GLEP. In particular, note that I also changed the title to avoid referring to VDB as the means for caching. Each package manager can decide how to cache the information internally. The working copy is at https://wiki.gentoo.org/wiki/User:Blueness/GLEP64 The essential point I'm trying to make is: 1) there is information generated when a PM builds+installs a package. 2) some of this information is expensive to regenerate. 3) there are utilites that would like to make use of this information. 4) to spare these utilites the expense of regenerating this information, all package managers should cache this information and then export it via a common API. Thanks in advance for your time! -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] rfc: adding sys-apps/iproute2 to the @system set
On Friday 05 September 2014 12:34:11 William Hubbs wrote: > All, > > there is a bug open requesting that we add sys-apps/iproute2 to the > system set [1]. Originally the request was to drop net-tools, but it has > become just adding iproute2. I wouldn't mind either option - net-tools has been deprecated for a decade, but if we still ship it as default it will be used. Some people seem to think that stage3 is bloated - last time I looked at it there was lots of really-not-needed stuff like two python interpreters. If you want to de-bloat work on that, the extra 150kB or whatever of iproute2 are so small that it's barely noticeable. And by the same reasoning of "bloat" we should remove openssh ( and maybe even rsync ;) ) because it's not strictly needed - so maybe we want a "minimal" and a "useful" stage3 ? Have fun, Patrick
Re: [gentoo-dev] Re: rfc: adding sys-apps/iproute2 to the @system set
On 09/06/14 07:05, Rich Freeman wrote: On Sat, Sep 6, 2014 at 2:44 AM, Duncan <1i5t5.dun...@cox.net> wrote: Rich Freeman posted on Fri, 05 Sep 2014 20:10:02 -0400 as excerpted: The purpose of the system set is to deal with circular deps and the need to bootstrap. We shouldn't have stuff in there if it is possible to run without it. There are loads of things I can't live without which aren't in the system set. I have a default world file that I always start with anytime I do an install. Does portage still force serial builds of anything in the system-set and all deps thereof?[1] If so, given a situation where even most phones are multi-core these days, does /anything/ other than circular deps and bootstrapping really justify forcing /all/ the several @system packages and deps I had before I started pruning, into serial build? @system serves a couple of different purposes, and I think this is part of the problem. 1. One purpose of @system is simply convenience. Devs don't want to stick baselayout, bzip, sed, toolchain, etc in every other ebuild, so it is basically a default dependency for everything. This is what makes parallel builds with @system and its deps unsafe - there is no way for the package manager to know when there are dependency relationships in the packages being built if we intentionally don't specify them. The only safe solution here is to minimize the size of @system, either eliminating packages that aren't really such common dependencies or suffer a bit more inconvenience. The other purposes are all related to catalyst: 2. Another purpose is to break the circular dependency problem when building stage 1/2/3. If you did ditch #1 entirely it would not be straightforward to build a stage1/2/3 without some hints as to what basically just needs to be pre-provided from the outside system as a bootstrap. 3. Yet another of its purposes is to determine what goes into the stage3. I'm actually wondering if something like a default world set might be a better approach to that, or maybe we need a stage3 set or something. This is where packages like openssh fit in. or even man and editor. All true, but returning to the original point, even if packages in stage3 were a superset of @system, I would still argue against including iproute2 because of bloat for the reasons already stated. Don't get me wrong, I really like iproute2, but net-tools is sufficient for a stage3. I like the idea that stage3 is pretty much defined by your points 1 and 2, with the implicit assumption of "a minimal set of packages be able to build any gentoo system" from it. This minimal set idea is important because of its role in building stages via catalyst ... see below. The thing is that #2-3 really only pertain to generating stage3s, and that really only matters for the initial install. After that, everybody lives with it for the rest of their Gentoo lives. If we fell like bikeshedding (and I'm up for a good discussion on the matter :), we can return to the old "stage4 = stage3 + extras" discussion. Not having a clear picture of what a stage4 should and should not have in it, I don't have any a priori objections to adding openssh, man, vi and iproute2. However, there is one big difference I see between stage4 and any of the other stages. A proper catalyst run should be: stage3 -> stage1 -> stage2 -> stage3 -> etc It should NOT include a stage4. A stage4 would just be a spinoff of a stage3, and not be part of the catalyst cycle. You *could* use a stage4 as a stage3 seed, but it should not be necessary. We should NOT have catalyst runs looking like: stage4 -> stage1 -> stage2 -> stage3 -> stage4 -> etc This would unnecessarily increase cpu time, contribute to the total entropy of the universe and speed up its heat death. All bad things. Now that we actually have sets in portage, maybe it would make sense to split up @system into different sets for each of those purposes. Then we can optimize both the stage3 generation and the requirements for installed systems separately. -- Rich -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] why does net-misc/tor enables "Log notice syslog" ?
On 09/06/14 06:44, Toralf Förster wrote: After a recent discussion at #tor an #gentoo-dev /me wonders if the syslog level of net-misc/tor should at least be changed from "notice" to "warn" -or better - be fully unset like upstream does it ? I can entertain that, but this should be in a bug report: 1) While anyone is welcome to discuss the issue, I don't think most gentoo devs care about this. 2) A bug report leaves behind a record of what's going on which can later be easily searched for using bugizlla's magic. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] Re: rfc: adding sys-apps/iproute2 to the @system set
On Sat, Sep 6, 2014 at 2:44 AM, Duncan <1i5t5.dun...@cox.net> wrote: > Rich Freeman posted on Fri, 05 Sep 2014 20:10:02 -0400 as excerpted: > >> The purpose of the system set is to deal with circular deps and the need >> to bootstrap. We shouldn't have stuff in there if it is possible to run >> without it. >> >> There are loads of things I can't live without which aren't in the >> system set. I have a default world file that I always start with >> anytime I do an install. > > Does portage still force serial builds of anything in the system-set and > all deps thereof?[1] If so, given a situation where even most phones are > multi-core these days, does /anything/ other than circular deps and > bootstrapping really justify forcing /all/ the several @system packages > and deps I had before I started pruning, into serial build? @system serves a couple of different purposes, and I think this is part of the problem. 1. One purpose of @system is simply convenience. Devs don't want to stick baselayout, bzip, sed, toolchain, etc in every other ebuild, so it is basically a default dependency for everything. This is what makes parallel builds with @system and its deps unsafe - there is no way for the package manager to know when there are dependency relationships in the packages being built if we intentionally don't specify them. The only safe solution here is to minimize the size of @system, either eliminating packages that aren't really such common dependencies or suffer a bit more inconvenience. The other purposes are all related to catalyst: 2. Another purpose is to break the circular dependency problem when building stage 1/2/3. If you did ditch #1 entirely it would not be straightforward to build a stage1/2/3 without some hints as to what basically just needs to be pre-provided from the outside system as a bootstrap. 3. Yet another of its purposes is to determine what goes into the stage3. I'm actually wondering if something like a default world set might be a better approach to that, or maybe we need a stage3 set or something. This is where packages like openssh fit in. or even man and editor. The thing is that #2-3 really only pertain to generating stage3s, and that really only matters for the initial install. After that, everybody lives with it for the rest of their Gentoo lives. Now that we actually have sets in portage, maybe it would make sense to split up @system into different sets for each of those purposes. Then we can optimize both the stage3 generation and the requirements for installed systems separately. -- Rich
Re: [gentoo-dev] Automated Package Removal and Addition Tracker, for the week ending 2014-08-17 23h59 UTC
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 09/06/2014 09:16 AM, Markos Chandras wrote: > On 09/05/2014 11:42 PM, Robin H. Johnson wrote: >> Oldschool python broke on the new CVS box, fixed now. > > > Hi Robin, > > Ok thanks. Is there a way to generate some stats for August so we > can put them on the GMN? otherwise we will skip that section. > > apologies I just noticed you have already sent these emails. - -- Regards, Markos Chandras -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQF8BAEBCgBmBQJUCuhyXxSAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGRDlGMzA4MUI2MzBDODQ4RDBGOEYxMjQx RjEwRUQ0QjgxREVCRjE5AAoJEB8Q7UuB3r8ZkekH/RNxkWC2hbozov75rLcEvT2+ 602BaurICC1TBv2utPNz73B0TGMz3X4txeMCfBSV4+vL091ZzcXdRT8WbyEJWVEC TZPNb2Tb8g99j7r1EW8ELF2yd3FgmqRljWyJVJYx94TC1z4oBZb3YgWWhM6t+5Gj URYvOSYIb8xrJOQR+aMWJDFVqx3FMNufsXLtSF27+rxt9igUvKhhBaO3lJFi+2g3 hoqpP6nZxp/w+daw/fAwvB5HUQwYG/pcSCrfwoG7F0iS2rS4mO6Hh785ZmVu3LTz EkskcroEDUSwQ96K6yoB9bAZelnXoodet4EjWbDexk3WMeG2XWUOuF01K00RSQQ= =g1xK -END PGP SIGNATURE-
[gentoo-dev] why does net-misc/tor enables "Log notice syslog" ?
After a recent discussion at #tor an #gentoo-dev /me wonders if the syslog level of net-misc/tor should at least be changed from "notice" to "warn" -or better - be fully unset like upstream does it ? -- Toralf pgp key: 0076 E94E
[gentoo-dev] Last rites: python3_2 target
# Michał Górny (06 Sep 2014) # (on behalf of Python team) # Python 3.2 is no longer supported upstream and there are no new # releases planned. Packages are removing support for it in favor # of 3.3 and 3.4. The support for implementations will be disabled # in 30 days. python_targets_python3_2 python_single_target_python3_2 In other words, please don't add 3.2 support to any new packages, rebuild existing if you like and expect the flag to magically disappear on Oct 6. I think we'll keep python:3.2 ebuild though so that people can still use it locally. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Automated Package Removal and Addition Tracker, for the week ending 2014-08-17 23h59 UTC
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 09/05/2014 11:42 PM, Robin H. Johnson wrote: > Oldschool python broke on the new CVS box, fixed now. > Hi Robin, Ok thanks. Is there a way to generate some stats for August so we can put them on the GMN? otherwise we will skip that section. - -- Regards, Markos Chandras -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQF8BAEBCgBmBQJUCsLeXxSAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGRDlGMzA4MUI2MzBDODQ4RDBGOEYxMjQx RjEwRUQ0QjgxREVCRjE5AAoJEB8Q7UuB3r8ZQR8H/22hdfZfk1TZiR2bxR5pPEwW 1zViksO3cuw43zXpfsu81LPA90N1hrlI5n/qPokDUz6EE/Hg08NcVL6DSZ96PIO9 YUUgoIpKVq/LVfIuMs8RzdZn02hGzGerJquqLb6wUKFOnCjSEwZzZaq0YbO2kPzz K1ocxK+5R5J7q/KWAJ2fafbkPd5vSTNZPhtcmyjsa3QkaHhFDHVWqHyI+xAXxka0 VCCbwG+h6yNcORwGuAfkW2nlf3A1yfsRJCaCOOZbzNTHPQ7yNTavYph4o6EdDOoX /LQjJO78J0CXwo9apQBsnixEqdFTZ0eBylTzt3ctGhhKVx0Ubzm2NR38iUqLs7Q= =ewYk -END PGP SIGNATURE-
[gentoo-dev] Re: rfc: adding sys-apps/iproute2 to the @system set
Duncan posted on Sat, 06 Sep 2014 06:44:21 + as excerpted: > several @system packages and deps I had before I started pruning Oops. Several /hundred/ @system packages and deps... including kdelibs, due to USE=kde on some other dep. IIRC, with only openrc in the @system set itself, there were still ~125 @system deps. =:^( -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
