On Sunday 17 May 2009 06:43:50 Richard Freeman wrote: > Duncan wrote: > > So I believe the cost to be quite reasonably managed, after all. > > Benchmarks would of course be needed to demonstrate that, but I believe > > it worth pursuing. I thought we had agreed that (1) with GLEP55 you have to source the ebuild anyway (whereas the other proposal allows to just parse it to get at the EAPI value) and (2) you can cache it sanely so that performance isn't the issue?
> Agreed. Perhaps I'm just spoiled by RDBMS's at work or something, but > it seems like we're trying to squeeze every ounce of speed out of a > non-indexed flat file database and do everything we can to avoid > actually putting all that metadata in something that actually is > queryable no matter how lousy the final design ends up being. The performance is really not an issue - the current design is quite limited and would need some interesting tweaks to go a lot faster. In terms of opening-and-looking-at files GLEP55 doesn't really offer a benefit, either you have a metadata cache which includes it (stat for existence of cache, stat for mtime of ebuild, open either ebuild or cache, source ebuild if cache is stale) or you have it in the filename (either the same sequence of operations if you cache it, or you source it because of the current restrictions in glep55) In other words, looking at performance in this case is just a distraction. > Expressing the package database as a set of flat files works nicely - > especially with cvs/git/etc. Actually working with that data directly > on a real system doesn't make sense at all. Index it once and then only > open the flat files on the rare occasion that you actually need to > install one of them. Such an index can be centrally distributed, or it > could be maintained as packages are rsynced (and of course users should > be able to update it on demand as well). That sounds like a funny idea. I propose putting such a cache into /usr/portage/metadata/cache and have it contain pregenerated metadata keys, like DEPEND, HOMEPAGE and EAPI. > When the speed of your package management system depends on the > performance of find vs grep -r, you are doing something wrong. Neither > works all that well. And when the difference is 0.03s out of 0.5s in the hot-cache and 4 seconds out of 75 in the cold-cache case you can't really "optimize" anything without considering more powerful options to increase performance ...
