Re: [gentoo-portage-dev] [RFC] Adding extra vars to md5-cache, for QA purposes
On 7/25/19 4:29 PM, Michał Górny wrote: >> >> * In the md5-cache entry, maybe use a common prefix like EXT_ for the >> extra keys in order to distinguish them from normal keys. > > Yeah, I was thinking of something like '__ext_foo', or '__ext[foo]'. > What are the pros/cons of this? The names refer to global variables, so they should already be safely namespaced, right?. There is a possibility that an eclass variable name (e.g. PATCHES) could become standardized at a later date. If that happens, we could wind up with both FOO and __ext_FOO in the cache, and tools would have to figure out what to do with zero, one, or both present. (This has happened in email/web protocols when an X-Foo header was standardized.) It's not the end of the world, but someone would have to stop and think about it. Finally, just having the name be predictable so that I can grep '^FOO=' without having to care where it came from is nice. OTOH for testing, and for figuring out why these weird variables are showing up in my cache, the prefix would help.
Re: [gentoo-portage-dev] [RFC] Adding extra vars to md5-cache, for QA purposes
On Thu, 2019-07-25 at 12:57 -0700, Zac Medico wrote: > On 7/25/19 5:20 AM, Michał Górny wrote: > > Hi, > > > > TL;DR: I'd like to make it possible for ebuilds to define additional > > variables that will be stored in md5-cache. This would be useful for CI > > and other tooling that right now has to parse ebuilds for other data. > > > > > > The idea is to add a new incremental ebuild/eclass variable (technical > > name: QA_EXTRA_CACHE_VARS) that would define additional data to be > > stored in cache. For example, python*-r1 eclasses would define > > 'PYTHON_COMPAT', acct-user would define 'ACCT_USER_ID', etc. > > > > When regenerating cache, the PM would read this variable, and store > > the values of all defined variables into md5-cache. As a result, > > programs needing those variables can get them straight from cache > > without having to attempt to run or parse ebuilds (which is both slow > > and prone to bugs). > > > > This would benefit e.g. gpyutils that right now need to attempt to parse > > PYTHON_COMPAT from ebuilds. It would also benefit writing future > > pkgcheck checks for user/group ID collisions. > > > > > > Notes: > > > > - since md5-cache uses key-value format and allows for future > > extensions, the new values can be added without breaking anything; > > > > - md5-cache is not specified in the PMS, and the whole thing can be > > implemented without need for EAPI bump, > > > > - I would like to have this implemented consistently both in Portage > > and pkgcore, > > > > - we will need to clearly define how to dump arrays. > > > > > > What do you think? > > Sounds good. Some thoughts: > > * Maybe omit QA from the variable name, since it can be could be > generally useful for things that are unrelated to QA. > > * In the md5-cache entry, maybe use a common prefix like EXT_ for the > extra keys in order to distinguish them from normal keys. Yeah, I was thinking of something like '__ext_foo', or '__ext[foo]'. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-portage-dev] [RFC] Adding extra vars to md5-cache, for QA purposes
On 7/25/19 5:20 AM, Michał Górny wrote: > Hi, > > TL;DR: I'd like to make it possible for ebuilds to define additional > variables that will be stored in md5-cache. This would be useful for CI > and other tooling that right now has to parse ebuilds for other data. > > > The idea is to add a new incremental ebuild/eclass variable (technical > name: QA_EXTRA_CACHE_VARS) that would define additional data to be > stored in cache. For example, python*-r1 eclasses would define > 'PYTHON_COMPAT', acct-user would define 'ACCT_USER_ID', etc. > > When regenerating cache, the PM would read this variable, and store > the values of all defined variables into md5-cache. As a result, > programs needing those variables can get them straight from cache > without having to attempt to run or parse ebuilds (which is both slow > and prone to bugs). > > This would benefit e.g. gpyutils that right now need to attempt to parse > PYTHON_COMPAT from ebuilds. It would also benefit writing future > pkgcheck checks for user/group ID collisions. > > > Notes: > > - since md5-cache uses key-value format and allows for future > extensions, the new values can be added without breaking anything; > > - md5-cache is not specified in the PMS, and the whole thing can be > implemented without need for EAPI bump, > > - I would like to have this implemented consistently both in Portage > and pkgcore, > > - we will need to clearly define how to dump arrays. > > > What do you think? Sounds good. Some thoughts: * Maybe omit QA from the variable name, since it can be could be generally useful for things that are unrelated to QA. * In the md5-cache entry, maybe use a common prefix like EXT_ for the extra keys in order to distinguish them from normal keys. -- Thanks, Zac signature.asc Description: OpenPGP digital signature
[gentoo-portage-dev] [RFC] Adding extra vars to md5-cache, for QA purposes
Hi, TL;DR: I'd like to make it possible for ebuilds to define additional variables that will be stored in md5-cache. This would be useful for CI and other tooling that right now has to parse ebuilds for other data. The idea is to add a new incremental ebuild/eclass variable (technical name: QA_EXTRA_CACHE_VARS) that would define additional data to be stored in cache. For example, python*-r1 eclasses would define 'PYTHON_COMPAT', acct-user would define 'ACCT_USER_ID', etc. When regenerating cache, the PM would read this variable, and store the values of all defined variables into md5-cache. As a result, programs needing those variables can get them straight from cache without having to attempt to run or parse ebuilds (which is both slow and prone to bugs). This would benefit e.g. gpyutils that right now need to attempt to parse PYTHON_COMPAT from ebuilds. It would also benefit writing future pkgcheck checks for user/group ID collisions. Notes: - since md5-cache uses key-value format and allows for future extensions, the new values can be added without breaking anything; - md5-cache is not specified in the PMS, and the whole thing can be implemented without need for EAPI bump, - I would like to have this implemented consistently both in Portage and pkgcore, - we will need to clearly define how to dump arrays. What do you think? -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
