Re: [gentoo-portage-dev] .53, .54 and beyond...
On Wednesday 07 December 2005 11:57, Marius Mauch wrote: On Wed, 7 Dec 2005 08:41:27 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: On Wednesday 07 December 2005 01:01, Marius Mauch wrote: On Tue, 6 Dec 2005 23:19:38 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: If there's no solid opposition, Saturday I will put current trunk into ~arch as 2.1_beta20051210. Well, I've already stated several times that IMO using a 2.1 branch is wrong (use 2.2 instead), but if I'm overvoted, so it shall be. As Brian stated, 2.2 being a version higher than 2.1 will have all the same expectations placed on it. From what I can see, 1% of users know anything about 2.1 so 99% would be wondering why there was a jump from 2.0 to 2.2. Do you have anything against 2.1 other than fearing that people will expect more from it than it will turn out to be? It isn't about expectations. Ok, I misunderstood your previous posts on this topic then. I just think it's bad engineering to use the same version prefix for two rather different codebases. ... After all, wasn't engineering the reason why we're going to increase the minor? I don't understand where the conflict comes in between the two. Internally, the old 2.1 has been known as HEAD, trunk and now 2.1-experimental. Externally, it's been known as 2.1.0_alpha20050718. The set of new features available in 2.1.0_alpha20050718 are pretty much all available in current trunk as far as I know... You'll need to explain the issue in a little bit more detail. Really, the bottom line is that regardless of what the response was when you asked about portage keywording, if all the arch teams had confidence in what we thought 2.0.53 would have been stable a long time ago. On the surface the only benefit is extra testing (which has already payed off) but it also allows others to take an active hand in the quality of portage as well as strengthens the communication channels. Ok, but it still doesn't really have anything to do with arch teams, just general QA. True, but currently there's no general QA team for coordinating the stability of the tree in general. Instead, this has been left up to the individual arch teams (which is a little strange/disorganized) so Also I didn't mean to criticize you, just stating that this option exists. Isn't it your responsibility to? ;) I can't tell if you followed what I said in my last email so I'll reiterate. Trunk will go into ~arch on Saturday. 2.0.54 will go out (also in ~arch) two weeks after that with the two fixes and include the cache rewrite based on the opinion of a broad range of users (rather than just the noise makers). SHA1 will of course also go in based on how it is voted. Ehm, what's the point of having .54 in ~arch after trunk is in ~arch? You won't get much testing that way as ~arch users would already use trunk and stable users likely won't know about .54 ... (typical visibility problem) The visibility problem is exactly why I'm suggesting it be done that way. Neither ~arch users nor arch users get needless bumps and testing of trunk doesn't get held up at all. .54 would be .53 + selective patches from trunk hence all its parts will have had extensive testing. The whole would only need a minimal amount of testing by those marking it stable before doing so. -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Tuesday 06 December 2005 11:29, Zac Medico wrote: Ned Ludd wrote: On Mon, 2005-12-05 at 23:06 +0900, Jason Stubbs wrote: Okay, new suggestion. Postpone the cache rewrite from above. Have only the minimal mods necessary to fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be 2.0.54 as per the attached patch. Get that out soon and get trunk out masked at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch. However, instead of ~arch meaning regression fixes only we could just limit it to minor changes only (ie. no big refactorings, rewrites or similar high risk changes) until it is time to stable it. I think it would be wise to reconsider the cache fixes. I know you have been away from irc for a while now and have missed the daily events, but most of the people we have interacted with are expecting the cache updates in .54 (alot of people complaining about the hanging at 50%) The code has been pretty well tested and seems safe on the surface. I think ferringb's testing has shown that the cache updates use about 14M of ram where the existing code (as of .52.x) uses about 80M of ram. But still, it's annoying to be stuck with only 2 tiers. Why not put a snapshot of trunk in the tree and package.mask it? Wouldn't that make everyone happy? That's what I'm thinking. Given that the people that are being asked to stable 2.0.53 are complaining that the ldconfig fix and exec/tee fix aren't in it, I'm certain that 2.0.54 would likely go stable *before* 2.0.53 if it is pushed out soon. If the SHA1 stuff is dropped, I can pretty much guarantee it. And as soon as it goes stable, trunk can be pushed into ~arch. Doing it this way, the cache rewrite gets into ~arch at pretty much the same time but all the other bits get added along with it. Sure it will mean that there will be a bit of a delay before it gets into stable but if we can speed up the release cycle the delay won't be by very much. -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Tuesday 06 December 2005 11:17, Ned Ludd wrote: On Mon, 2005-12-05 at 23:06 +0900, Jason Stubbs wrote: Okay, new suggestion. Postpone the cache rewrite from above. Have only the minimal mods necessary to fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be 2.0.54 as per the attached patch. Get that out soon and get trunk out masked at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch. However, instead of ~arch meaning regression fixes only we could just limit it to minor changes only (ie. no big refactorings, rewrites or similar high risk changes) until it is time to stable it. I think it would be wise to reconsider the cache fixes. I know you have been away from irc for a while now and have missed the daily events, but most of the people we have interacted with are expecting the cache updates in .54 (alot of people complaining about the hanging at 50%) Call me wrong, but I'm feeling that the constant pulling and pushing on IRC causes many misjudgements. -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jason Stubbs wrote: On Tuesday 06 December 2005 11:17, Ned Ludd wrote: On Mon, 2005-12-05 at 23:06 +0900, Jason Stubbs wrote: Okay, new suggestion. Postpone the cache rewrite from above. Have only the minimal mods necessary to fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be 2.0.54 as per the attached patch. Get that out soon and get trunk out masked at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch. However, instead of ~arch meaning regression fixes only we could just limit it to minor changes only (ie. no big refactorings, rewrites or similar high risk changes) until it is time to stable it. I think it would be wise to reconsider the cache fixes. I know you have been away from irc for a while now and have missed the daily events, but most of the people we have interacted with are expecting the cache updates in .54 (alot of people complaining about the hanging at 50%) Call me wrong, but I'm feeling that the constant pulling and pushing on IRC causes many misjudgements. -- Jason Stubbs No I agree with you here, I just wanted reasoning because now I have this ML thread to refer people to :p - -Alec Warner (Antarus) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQ5WGIGzglR5RwbyYAQKUyw/8DzLWhxIUX8fwp0UdbGq1nlX7PQw/InBx GZxA9xZOEMdIff72vZCWEbJBkIQVC5eHS/okO9Cs9xZhrBLKEh1xudJZZqFiDwkF C8Au59k8YNtjjL7bHwDd+Z1gXMBfLj70YI1F4GnCrsWbaGp1k+lBOrPBeYLqxK5P GZS1Z2jIeDXGQCHj3gbL98fCe26p3OGby9VCQIrmA0Mm+y9GAftnr/7kaz+EZpxX HRvza1x2oIsNPPkIs7nqDT0RKqOsE3OvoKtcCQGLXpjIEoGHbghxX0FmqxFDSq7K lUk5SSskvaL6AVzqcQ99ietrhVZqCraBx/r8a/9U50czd+rpUs1cACVQM6TEwAu9 A+dv+84QubOQzhe3jPmkag5G7NEsTMjvp55NYnXXqg4ACT4gvvODEAPShlDUmYHu vPDlj4PXkF+jPAmhoUGfHvY0belCeVo9bV00yGBaTHzS40xK/TSP2Zpy0E6zbets pHtg7aU2OoC3DWOUn3tbbc1l3GY6adiRI2v5vYTNhQlzHdKFwLKPaw2GNu81Pq3t p2y5ZmIDLD5k4FAvnTiXTi2T/emqlD8QPPrxG7LbCqofFUd9K0DJlPhZZyub4H2f Eq8iPVKgUNwVg5g5HftYYa9RhAEur4ble9ZtFKp23ulplpaVV6JhXj/3aDcqZvp3 ozJyFy6Xtrc= =85d5 -END PGP SIGNATURE- -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Tue, 6 Dec 2005 23:19:38 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: So I'm going to make a decision and offer until Friday (Saturday in my time) for opposers to solidify and state any opposition. If there's no solid opposition, Saturday I will put current trunk into ~arch as 2.1_beta20051210. I will also post on the 2.0.53 bug that fixes are available for the ldconfig bug and the tee bug stating that we'd like to also add trunk's cache subsystem to 2.0.54 and that dependening on the next council meeting(?) the SHA1 enabling as well. Doing it this way will make the ~arch users happy straight away. If we look at it as our responsibility to get fixes and new functionality into ~arch then our jobs done and we can get back to business. Well, I've already stated several times that IMO using a 2.1 branch is wrong (use 2.2 instead), but if I'm overvoted, so it shall be. As for the cache rewrite in 2.0.54, I don't really prefer one way or the other, from an engineering POV it is 2.2 material, but if it is a major improvement and well tested it can also be in .54 (just in case my previous mail was misunderstood). As for stable users? If arch teams are willing to selectively choose what fixes they want backported to stable (when they're not prepared to move the ~arch version into stable) things will go much smoother. Of course, it would still be our responsibility to get those things backported and assert some confidence that it is working. However, once the requested fixes are backported all that needs to be done is put out the patched stable version with ~arch keywords and then leave it up to the arch teams again. Except for the slight extra burden on (which I believe many would actually find to be a blessing), it should be a win-win situation. Just in case you forgot and also for general reference, when I asked the arch teams about the portage keywording policy a few months ago (wether we should stable even without testing on all archs or to delegate that to arch teams) everyone seemed to be happy with the old policy, at least nobody voted for a change. As portage doesn't really have any arch specific code and a rather short dep list IMO it also doesn't yield any real benefit other than more people testing it (which is of course always a good thing). Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Wednesday 07 December 2005 01:01, Marius Mauch wrote: On Tue, 6 Dec 2005 23:19:38 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: If there's no solid opposition, Saturday I will put current trunk into ~arch as 2.1_beta20051210. Well, I've already stated several times that IMO using a 2.1 branch is wrong (use 2.2 instead), but if I'm overvoted, so it shall be. As Brian stated, 2.2 being a version higher than 2.1 will have all the same expectations placed on it. From what I can see, 1% of users know anything about 2.1 so 99% would be wondering why there was a jump from 2.0 to 2.2. Do you have anything against 2.1 other than fearing that people will expect more from it than it will turn out to be? As for stable users? If arch teams are willing to selectively choose what fixes they want backported to stable (when they're not prepared to move the ~arch version into stable) things will go much smoother. Of course, it would still be our responsibility to get those things backported and assert some confidence that it is working. However, once the requested fixes are backported all that needs to be done is put out the patched stable version with ~arch keywords and then leave it up to the arch teams again. Except for the slight extra burden on (which I believe many would actually find to be a blessing), it should be a win-win situation. Just in case you forgot and also for general reference, when I asked the arch teams about the portage keywording policy a few months ago (wether we should stable even without testing on all archs or to delegate that to arch teams) everyone seemed to be happy with the old policy, at least nobody voted for a change. As portage doesn't really have any arch specific code and a rather short dep list IMO it also doesn't yield any real benefit other than more people testing it (which is of course always a good thing). Really, the bottom line is that regardless of what the response was when you asked about portage keywording, if all the arch teams had confidence in what we thought 2.0.53 would have been stable a long time ago. On the surface the only benefit is extra testing (which has already payed off) but it also allows others to take an active hand in the quality of portage as well as strengthens the communication channels. It's these auxillary benefits as well as the benefit of being able to focus on trunk more (which will yield faster rollout of features) that make me think it is the best way to go. On Wednesday 07 December 2005 01:21, Alec Warner wrote: I spoke with Brian today ( no clue if he will be sending mail or not ) but he stressed that he would like the cache rewrite in ~arch. Any reason why you're speaking on his behalf? I would prefer that it be in .54, the code itself is old, 6+ months. It is a straight backport from the 2.1 branch (the dead one) and the interface code to make it fit with 2.0 is small compared to the patch size ( Brian estimated 100-150 lines ). These are all reasons that I myself have given already. I don't have a problem with releasing 2 ebuilds, one with the patch and one without ( or a use flag ) although the question that raises is will the cache rewrite actually end up in .54 final, or will it be put off :) This, I do have a problem with. You're essentially suggesting that three lines be maintained rather than the current two. I can't tell if you followed what I said in my last email so I'll reiterate. Trunk will go into ~arch on Saturday. 2.0.54 will go out (also in ~arch) two weeks after that with the two fixes and include the cache rewrite based on the opinion of a broad range of users (rather than just the noise makers). SHA1 will of course also go in based on how it is voted. -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
Jason Stubbs wrote: On Wednesday 07 December 2005 01:01, Marius Mauch wrote: On Tue, 6 Dec 2005 23:19:38 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: If there's no solid opposition, Saturday I will put current trunk into ~arch as 2.1_beta20051210. Well, I've already stated several times that IMO using a 2.1 branch is wrong (use 2.2 instead), but if I'm overvoted, so it shall be. As Brian stated, 2.2 being a version higher than 2.1 will have all the same expectations placed on it. From what I can see, 1% of users know anything about 2.1 so 99% would be wondering why there was a jump from 2.0 to 2.2. Do you have anything against 2.1 other than fearing that people will expect more from it than it will turn out to be? As for stable users? If arch teams are willing to selectively choose what fixes they want backported to stable (when they're not prepared to move the ~arch version into stable) things will go much smoother. Of course, it would still be our responsibility to get those things backported and assert some confidence that it is working. However, once the requested fixes are backported all that needs to be done is put out the patched stable version with ~arch keywords and then leave it up to the arch teams again. Except for the slight extra burden on (which I believe many would actually find to be a blessing), it should be a win-win situation. Just in case you forgot and also for general reference, when I asked the arch teams about the portage keywording policy a few months ago (wether we should stable even without testing on all archs or to delegate that to arch teams) everyone seemed to be happy with the old policy, at least nobody voted for a change. As portage doesn't really have any arch specific code and a rather short dep list IMO it also doesn't yield any real benefit other than more people testing it (which is of course always a good thing). Really, the bottom line is that regardless of what the response was when you asked about portage keywording, if all the arch teams had confidence in what we thought 2.0.53 would have been stable a long time ago. On the surface the only benefit is extra testing (which has already payed off) but it also allows others to take an active hand in the quality of portage as well as strengthens the communication channels. It's these auxillary benefits as well as the benefit of being able to focus on trunk more (which will yield faster rollout of features) that make me think it is the best way to go. On Wednesday 07 December 2005 01:21, Alec Warner wrote: I spoke with Brian today ( no clue if he will be sending mail or not ) but he stressed that he would like the cache rewrite in ~arch. Any reason why you're speaking on his behalf? From what I understood he was busy/moving IRL and didn't have time to catch up on his mail, so I told him I would send something expressing what he said when we spoke on IRC; he didn't explicitly tell me to do so, but I told him I was going to. I would prefer that it be in .54, the code itself is old, 6+ months. It is a straight backport from the 2.1 branch (the dead one) and the interface code to make it fit with 2.0 is small compared to the patch size ( Brian estimated 100-150 lines ). These are all reasons that I myself have given already. I don't have a problem with releasing 2 ebuilds, one with the patch and one without ( or a use flag ) although the question that raises is will the cache rewrite actually end up in .54 final, or will it be put off :) This, I do have a problem with. You're essentially suggesting that three lines be maintained rather than the current two. I can't tell if you followed what I said in my last email so I'll reiterate. Trunk will go into ~arch on Saturday. 2.0.54 will go out (also in ~arch) two weeks after that with the two fixes and include the cache rewrite based on the opinion of a broad range of users (rather than just the noise makers). SHA1 will of course also go in based on how it is voted. Bah, for some reason I kept thinking that the rewrite wasn't in trunk, but as I look in svn now I see it there; my apologies :/ The detailed explanation looks good to me ;) -Alec Warner (Antarus) -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Thursday 01 December 2005 22:28, Jason Stubbs wrote: On Monday 28 November 2005 03:49, Marius Mauch wrote: Jason Stubbs wrote: On Sunday 27 November 2005 00:09, Marius Mauch wrote: Jason Stubbs wrote: Well, the vote was more for the SHA1 change actually as that's the one triggering the size increase. The pycrypto stuff itself doesn't do anything really, it would just make the size increase more apparent. Hmm.. I thought it was for hashes supported by pycrypto being added into Manifest before Manifest2 comes along. If it was with regard to SHA1 then I take back my vote to delay. Yeah, I guess the mail could be read both ways. Don't think so given that offhand I don't even know what getlist() does ... getlist() is defined in emerge and is used to access the system and world sets. It wouldn't be too hard to customize it to handle user sets and modify other code to support them but the can't combine sets and atoms rule would get a bit messy. So gutting of emerge ... nope, tried that two times already, but gave up after hitting too many direct references to system and world. Oh, btw, two things that are in trunk but weren't listed in your original mail: - the rewritten versioning code (including the cvs and mult-suffix enhancements) - finally killing of the stupid masked by -* message That makes the current list for .54: * autouse death * cache rewrite * dyn_install cleanup * einfo logging * exec cleanup * flattened vdb *DEPENDs * hash support via pycrypto * ldconfig fix * metascan/auxget * postsync hooks * recursive grab* * RRDEPEND/LDEPEND * sha1 enabling * splitdebug * vdb empty file culling Are we about there yet? Also, what does this mean for 2.1/2.2? Well, if that featurelist is .54 then I really don't see a point for making a 2.1 or 2.2 release line. Before your mail starting this thread I assumed that .54 would just contain the ldconfig fix, the hash stuff and maybe a few other minor things, while trunk would become 2.2. But if things like elog, the new cache subsystem, splitdebug or the *DEPEND changes don't qualify for a minor version bump, then I can't think of anything that would. The ldconfig bug and the exec cleanup are the only urgent ones among them. The exec cleanup could be postponed and the existing code twiddled in a couple of places to fix the logging bug. However, the biggest issue that users are complaining about at present is the caching. The biggest issue for devs is security. Hence my original list: * cache rewrite * exec cleanup * ldconfig fix * sha1 enabling Okay, new suggestion. Postpone the cache rewrite from above. Have only the minimal mods necessary to fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be 2.0.54 as per the attached patch. Get that out soon and get trunk out masked at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch. However, instead of ~arch meaning regression fixes only we could just limit it to minor changes only (ie. no big refactorings, rewrites or similar high risk changes) until it is time to stable it. However, with the trunk's target (2.1?) rather than letting the arch teams decide when we call it stable, I think it would be a better idea to move it to the .0 version when we feel it is ready leaving it in ~arch. As regressions are fixed the .0 can be bumped to .1, .2 or whatever, but the focus can move to what happens beyond that rather than waiting... First paragraph is more important right now. -- Jason Stubbs diff -uNr portage-2.0.53/pym/portage.py portage-2.0.54/pym/portage.py --- portage-2.0.53/pym/portage.py 2005-12-01 22:04:17.0 +0900 +++ portage-2.0.54/pym/portage.py 2005-12-05 22:54:53.0 +0900 @@ -2039,11 +2039,6 @@ myline += +mysum myline += +myarchive myline += +str(mysize) - if sumName != MD5: -# This cannot be used! -# Older portage make very dumb assumptions about the formats. -# We need a lead-in period before we break everything. -continue mylines.append(myline) return mylines @@ -6430,6 +6425,9 @@ writemsg(!!! FAILED postrm: +str(a)+\n) sys.exit(123) + #update environment settings, library paths. Change symlinks. + env_update(makelinks=1) + self.unlockdb() def isowner(self,filename,destroot): @@ -6672,13 +6670,10 @@ writemsg(!!! FAILED postinst: +str(a)+\n) sys.exit(123) - downgrade = False - for v in otherversions: - if pkgcmp(catpkgsplit(self.pkg)[1:], catpkgsplit(v)[1:]) 0: -downgrade = True - #update environment settings, library paths. DO NOT change symlinks. - env_update(makelinks=(not downgrade)) + #only needed if we did not already run unmerge + if not (oldcontents): + env_update(makelinks=0) #dircache may break autoclean because it remembers the -MERGING-pkg file global dircache if
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Tuesday 06 December 2005 00:21, Alec Warner wrote: Jason Stubbs wrote: Okay, new suggestion. Postpone the cache rewrite from above. Have only the minimal mods necessary to fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be 2.0.54 as per the attached patch. Get that out soon and get trunk out masked at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch. However, instead of ~arch meaning regression fixes only we could just limit it to minor changes only (ie. no big refactorings, rewrites or similar high risk changes) until it is time to stable it. Postponing the cache rewrite is going to piss a lot of poeple off, just FYI :) I realize it's a large patch, but it has been tested by plenty of people, and many of them are waiting for this fix to hit stable (don't want to patch portage on a production system). Any particular reason you want it held off (besides it's immensity?) The delay it would cause for other things waiting in the pipeline. -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Mon, 2005-12-05 at 23:06 +0900, Jason Stubbs wrote: Okay, new suggestion. Postpone the cache rewrite from above. Have only the minimal mods necessary to fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be 2.0.54 as per the attached patch. Get that out soon and get trunk out masked at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch. However, instead of ~arch meaning regression fixes only we could just limit it to minor changes only (ie. no big refactorings, rewrites or similar high risk changes) until it is time to stable it. I think it would be wise to reconsider the cache fixes. I know you have been away from irc for a while now and have missed the daily events, but most of the people we have interacted with are expecting the cache updates in .54 (alot of people complaining about the hanging at 50%) The code has been pretty well tested and seems safe on the surface. I think ferringb's testing has shown that the cache updates use about 14M of ram where the existing code (as of .52.x) uses about 80M of ram. -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
Ned Ludd wrote: On Mon, 2005-12-05 at 23:06 +0900, Jason Stubbs wrote: Okay, new suggestion. Postpone the cache rewrite from above. Have only the minimal mods necessary to fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be 2.0.54 as per the attached patch. Get that out soon and get trunk out masked at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch. However, instead of ~arch meaning regression fixes only we could just limit it to minor changes only (ie. no big refactorings, rewrites or similar high risk changes) until it is time to stable it. I think it would be wise to reconsider the cache fixes. I know you have been away from irc for a while now and have missed the daily events, but most of the people we have interacted with are expecting the cache updates in .54 (alot of people complaining about the hanging at 50%) The code has been pretty well tested and seems safe on the surface. I think ferringb's testing has shown that the cache updates use about 14M of ram where the existing code (as of .52.x) uses about 80M of ram. But still, it's annoying to be stuck with only 2 tiers. Why not put a snapshot of trunk in the tree and package.mask it? Wouldn't that make everyone happy? Zac -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Monday 28 November 2005 03:49, Marius Mauch wrote: Jason Stubbs wrote: On Sunday 27 November 2005 00:09, Marius Mauch wrote: Jason Stubbs wrote: Well, the vote was more for the SHA1 change actually as that's the one triggering the size increase. The pycrypto stuff itself doesn't do anything really, it would just make the size increase more apparent. Hmm.. I thought it was for hashes supported by pycrypto being added into Manifest before Manifest2 comes along. If it was with regard to SHA1 then I take back my vote to delay. Yeah, I guess the mail could be read both ways. Don't think so given that offhand I don't even know what getlist() does ... getlist() is defined in emerge and is used to access the system and world sets. It wouldn't be too hard to customize it to handle user sets and modify other code to support them but the can't combine sets and atoms rule would get a bit messy. So gutting of emerge ... nope, tried that two times already, but gave up after hitting too many direct references to system and world. Oh, btw, two things that are in trunk but weren't listed in your original mail: - the rewritten versioning code (including the cvs and mult-suffix enhancements) - finally killing of the stupid masked by -* message That makes the current list for .54: * autouse death * cache rewrite * dyn_install cleanup * einfo logging * exec cleanup * flattened vdb *DEPENDs * hash support via pycrypto * ldconfig fix * metascan/auxget * postsync hooks * recursive grab* * RRDEPEND/LDEPEND * sha1 enabling * splitdebug * vdb empty file culling Are we about there yet? Also, what does this mean for 2.1/2.2? Well, if that featurelist is .54 then I really don't see a point for making a 2.1 or 2.2 release line. Before your mail starting this thread I assumed that .54 would just contain the ldconfig fix, the hash stuff and maybe a few other minor things, while trunk would become 2.2. But if things like elog, the new cache subsystem, splitdebug or the *DEPEND changes don't qualify for a minor version bump, then I can't think of anything that would. The ldconfig bug and the exec cleanup are the only urgent ones among them. The exec cleanup could be postponed and the existing code twiddled in a couple of places to fix the logging bug. However, the biggest issue that users are complaining about at present is the caching. The biggest issue for devs is security. Hence my original list: * cache rewrite * exec cleanup * ldconfig fix * sha1 enabling The cache rewrite has existing for many months and has gone through a fair bit of testing so I personally don't have any issues with it going into stable. As I said, the exec cleanup could be postponed and the existing code twiddled only where necessary... This is more what I was hoping for 2.0.54. It should really follow hot on the heals of 2.0.53 and be in stable hopefully by the end of the year. If everybody's happy with that, the remainder is: * autouse death * dyn_install cleanup * einfo logging * exec cleanup * flattened vdb *DEPENDs * hash support via pycrypto * metascan/auxget * postsync hooks * recursive grab* * RRDEPEND/LDEPEND * splitdebug * vdb empty file culling When I said shortterm goals in the original email, I was actually referring to after 2.0.54. ;) -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Sunday 27 November 2005 02:03, Ned Ludd wrote:
On Sat, 2005-11-26 at 13:15 +0900, Jason Stubbs wrote:
On Saturday 26 November 2005 02:05, Ned Ludd wrote:
On Sat, 2005-11-26 at 00:51 +0900, Jason Stubbs wrote:
On Saturday 26 November 2005 00:31, Ned Ludd wrote:
* post_sync action hook (.53/.54 )
* VDB prevention of single byte NULL entries being created. ( .54 )
Doable for .54.
Yeah and from the sounds of it we may want more than 1 set of postsync
hooks or the addition of a postsync.d/
(dev thread on getting vital info to users)
Heh.. that was my suggestion. ;)
Yeah it seems doable for .53 but if you want to wait till .54 or
a .53-rXX thats fine. I'd prefer to see it in .53 unless
that's going out the door today.
2.0.53 is pretty much closed. After it hits 2.0.53, the only -r releases will
be for major regressions and security fixes. The only reason I think killing
of CDEPEND for 2.0.53 is okay is because nothing uses CDEPEND (I killed off
emerge scanning it already) so there's nothing to regress.
Should generating internal
debug info still be offered?
When FEATURES=nostrip is enabled we should not split off
any debug info or touch the executable.
FEATURES=splitdebug|stripdebug and do nothing if neither is set?
If feature based it seems logical to me to have it as either
splitdebug || nosplitdebug
That would be splitdebug or -splitdebug. I was suggesting stripdebug as a
replacement for -nostrip to get rid of another no* option and so that portage
could still be configured to give the current behaviour. As far as I can
tell, there are three possibilities:
* Do nothing
* Split of the debug info
* Strip any debug info
Have I got that right?
I know some people hate no* functions or rather they hate no* USE
flags. But it would seem fitting if we decided to make it a default vs
sticking in splitdebug in all profiles.
There's no need to touch the profiles. /etc/make.globals defines portage
defaults and is controlled by portage.
* flattened vdb {P,R,}DEPEND (.54 )
I might be wrong but I can't really see this being done cleanly. At
best, only USE flags could be gotten rid of which would still leave
|| () constructs. This leads me to question of what use it would
really be. If it can only do a half-arsed job and in a messy way at
that I'd personally prefer it to be done later on.
It will indeed still leave you with || ( foo bar ) or =cpv which you
can be parsed just fine. Yeah it would be nice if it could be reduced
more but later on or now I don't see how it can be reduced anymore than
to the requirements that the ebuild requested.
Again, if it can be done cleanly code-wise no issues with resolving the
USE flags out.
Should only be a few lines of code. (I hope)
Something like hand off the env varable from dyn_compile() in ebuild.sh
to python and python passes it back to ebuild.sh in the next phase for
dyn_install() which gets recorded flattened
There's no passing back of variables at the moment except through temporary
files. Perhaps USE and *DEPEND could be read in and *DEPEND rewritten out
after dyn_compile() completes?
--
Jason Stubbs
--
gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
Jason Stubbs wrote: On Saturday 26 November 2005 11:07, Marius Mauch wrote: On Sat, 26 Nov 2005 00:01:15 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: The only other new thing in trunk that I know of is logging but there's still a question mark over the ordering of messages... Can that be resolved soon? Anything else missing? Any reasons against any of the above? Resolved how? I'm not really sure I understood the original problem (other than listdir being underdeterministic in theory). TGL suggested that all the messages go into a single file with some sort of prefix that would then be parsed on the python side. The original order of messages could then be maintained. However, seeing as there needs to be no compatibility with the temporary files it could wait for later anyway. a) haven't seen a patch for it, so no clue about how complex it is code wise and b) I generally dislike any markup/parsing in the temporary files. I'd rather get it out as-is now and incorporate any feedback later. As you said this interface doesn't have to be compatible, also I intended to add a general might change-note for elog in the release notes. - the pycrypto hash additions (for .54) This is only useful if the vote goes in favour of adding further hash types to Manfiest, right? If the vote goes that way I've got no issues with it, but if it doesn't it would essentially be dead code. Well, the vote was more for the SHA1 change actually as that's the one triggering the size increase. The pycrypto stuff itself doesn't do anything really, it would just make the size increase more apparent. - Killing off auto-use+USE_ORDER? Yep, I'd really like to see this one gone too. We should probably state the intention on -dev first though as there might be a lot of people against it. Well, Spanky liked the suggestion ;) - the recursive grab* functions I just posted (for .54) Needs a small amount of work (/etc/portage/package.mask/foo/bar would break it) but I like the general idea. How would it break? - integration of set modules, either as emerge targets (requires serious gutting of emerge) or a first-class atoms (semantically tricky, no clue about implementation yet) I'm working on this with my refactoring. Defininately a post-.54 thing unless you want to quickly hack it into getlist()? Don't think so given that offhand I don't even know what getlist() does ... Oh, btw, two things that are in trunk but weren't listed in your original mail: - the rewritten versioning code (including the cvs and mult-suffix enhancements) - finally killing of the stupid masked by -* message Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Sat, 2005-11-26 at 13:15 +0900, Jason Stubbs wrote:
On Saturday 26 November 2005 02:05, Ned Ludd wrote:
On Sat, 2005-11-26 at 00:51 +0900, Jason Stubbs wrote:
On Saturday 26 November 2005 00:31, Ned Ludd wrote:
* post_sync action hook (.53/.54 )
* VDB prevention of single byte NULL entries being created. ( .54 )
Doable for .54.
Yeah and from the sounds of it we may want more than 1 set of postsync
hooks or the addition of a postsync.d/
(dev thread on getting vital info to users)
Heh.. that was my suggestion. ;)
Yeah it seems doable for .53 but if you want to wait till .54 or
a .53-rXX thats fine. I'd prefer to see it in .53 unless
that's going out the door today.
* new prepstrip offering splitdebug ( .53/.54 )
Need to work out exactly what will be offered when on this one, but at
the earliest it will be .54. Perhaps go with your patch for .54 and leave
Olivier's fancy bits for later?
I can only assure you the code I wrote will function properly.
So that's the only thing I'm trying/willing to push myself.
As long as he has those [ -x checks ] his code should be harmless,
but I don't see the advantage in it over building with -g3.
There are a few other questions too... Should
the default be to generate external debug info?
I think the security team would say yes they want it by default and
would be willing (taviso) to write a proper debug-HOWTO.xml for gentoo.
I think ferringb would say make it FEATURES=splitdebug if
it's going in midstream.
It does add some size which would make peoples $ROOT's a little bit
bigger. But from mine and other peoples testing it's pretty damn
minimal. I think Halcy0n @ gentoo said after doing an -e world he ended
up with only 18M of split debug info
I'm also fond of split packaging of debug info also (but I'm not going
to push for that till I find a more elegant way)
[EMAIL PROTECTED] debug $ qsize pax-utils
app-misc/pax-utils-debug-0.1.4-r0: 3 files, 5 non-files, 16.27 KB
app-misc/pax-utils-0.1.4: 6 files, 6 non-files, 102.485 KB
Perhaps we should get input from gentoo-dev ml ?
Sounds good for pretty much all aspects of split debug (other than the
capability itself).
Should generating internal
debug info still be offered?
When FEATURES=nostrip is enabled we should not split off
any debug info or touch the executable.
FEATURES=splitdebug|stripdebug and do nothing if neither is set?
If feature based it seems logical to me to have it as either
splitdebug || nosplitdebug
I know some people hate no* functions or rather they hate no* USE
flags. But it would seem fitting if we decided to make it a default vs
sticking in splitdebug in all profiles.
The feeling I get in general is that some devs want it by default to
make helping users who don't really know how to debug give us the info
we need without much hassles.
stripdebug I'm not sure what you envision with that name.
* flattened vdb {P,R,}DEPEND (.54 )
I might be wrong but I can't really see this being done cleanly. At best,
only USE flags could be gotten rid of which would still leave || ()
constructs. This leads me to question of what use it would really be. If
it can only do a half-arsed job and in a messy way at that I'd personally
prefer it to be done later on.
It will indeed still leave you with || ( foo bar ) or =cpv which you
can be parsed just fine. Yeah it would be nice if it could be reduced
more but later on or now I don't see how it can be reduced anymore than
to the requirements that the ebuild requested.
Again, if it can be done cleanly code-wise no issues with resolving the USE
flags out.
Should only be a few lines of code. (I hope)
Something like hand off the env varable from dyn_compile() in ebuild.sh
to python and python passes it back to ebuild.sh in the next phase for
dyn_install() which gets recorded flattened
If it's not doable then I'll just go for a simple cosmetic patch to
remove newlines and extra spaces.
--
Ned Ludd [EMAIL PROTECTED]
Gentoo Linux
--
gentoo-portage-dev@gentoo.org mailing list
[gentoo-portage-dev] .53, .54 and beyond...
Hi all, I don't think there's really anything else that can be done for 2.0.53 so am thinking that we should probably push _rc7 + docs out and let the arch teams mark it stable when they're ready (or stick with 2.0.51.22-r3 if it pleaseth them). We should put out a 2.0.54_pre1 out soon after that. What I'm wondering in is what will be going in? So far I'm thinking along the lines of: * cache rewrite * exec cleanup * ldconfig fix * sha1 enabling The only other new thing in trunk that I know of is logging but there's still a question mark over the ordering of messages... Can that be resolved soon? Anything else missing? Any reasons against any of the above? What's on the map for after that? There's a few things listed on the new (still unreleased?) project index and I'm looking to get the dependency stuff refactored and moved out of emerge.. What are the shortterm goals? -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Saturday 26 November 2005 00:31, Ned Ludd wrote:
On Sat, 2005-11-26 at 00:01 +0900, Jason Stubbs wrote:
Hi all,
I don't think there's really anything else that can be done for 2.0.53 so
am thinking that we should probably push _rc7 + docs out and let the arch
teams mark it stable when they're ready (or stick with 2.0.51.22-r3 if it
pleaseth them).
[snip]
There's a few things listed on the new
(still unreleased?) project index and I'm looking to get the dependency
stuff refactored and moved out of emerge.. What are the shortterm goals?
For me my short term goals are to see these things happen
* pax-utils depends ( .53 )
* seeing CDEPEND stop being created for the VDB ( .53 )
Definitely doable.
* post_sync action hook (.53/.54 )
* VDB prevention of single byte NULL entries being created. ( .54 )
Doable for .54.
* new prepstrip offering splitdebug ( .53/.54 )
Need to work out exactly what will be offered when on this one, but at the
earliest it will be .54. Perhaps go with your patch for .54 and leave
Olivier's fancy bits for later? There are a few other questions too... Should
the default be to generate external debug info? Should generating internal
debug info still be offered? What platforms is it supported on?..
* misc cleanups of dyn_install (.54 )
Need more info.
* flattened vdb {P,R,}DEPEND (.54 )
I might be wrong but I can't really see this being done cleanly. At best, only
USE flags could be gotten rid of which would still leave || () constructs.
This leads me to question of what use it would really be. If it can only do a
half-arsed job and in a messy way at that I'd personally prefer it to be done
later on.
* introduction of RRDEPEND to the VDB ( .54 )
What is this again?
--
Ned Ludd [EMAIL PROTECTED]
Gentoo Linux
--
gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Sat, 2005-11-26 at 00:51 +0900, Jason Stubbs wrote:
On Saturday 26 November 2005 00:31, Ned Ludd wrote:
On Sat, 2005-11-26 at 00:01 +0900, Jason Stubbs wrote:
Hi all,
I don't think there's really anything else that can be done for 2.0.53 so
am thinking that we should probably push _rc7 + docs out and let the arch
teams mark it stable when they're ready (or stick with 2.0.51.22-r3 if it
pleaseth them).
[snip]
There's a few things listed on the new
(still unreleased?) project index and I'm looking to get the dependency
stuff refactored and moved out of emerge.. What are the shortterm goals?
For me my short term goals are to see these things happen
* pax-utils depends ( .53 )
* seeing CDEPEND stop being created for the VDB ( .53 )
Definitely doable.
* post_sync action hook (.53/.54 )
* VDB prevention of single byte NULL entries being created. ( .54 )
Doable for .54.
Yeah and from the sounds of it we may want more than 1 set of postsync
hooks or the addition of a postsync.d/
(dev thread on getting vital info to users)
* new prepstrip offering splitdebug ( .53/.54 )
Need to work out exactly what will be offered when on this one, but at the
earliest it will be .54. Perhaps go with your patch for .54 and leave
Olivier's fancy bits for later?
I can only assure you the code I wrote will function properly.
So that's the only thing I'm trying/willing to push myself.
As long as he has those [ -x checks ] his code should be harmless,
but I don't see the advantage in it over building with -g3.
There are a few other questions too... Should
the default be to generate external debug info?
I think the security team would say yes they want it by default and
would be willing (taviso) to write a proper debug-HOWTO.xml for gentoo.
I think ferringb would say make it FEATURES=splitdebug if
it's going in midstream.
It does add some size which would make peoples $ROOT's a little bit
bigger. But from mine and other peoples testing it's pretty damn
minimal. I think Halcy0n @ gentoo said after doing an -e world he ended
up with only 18M of split debug info
I'm also fond of split packaging of debug info also (but I'm not going
to push for that till I find a more elegant way)
[EMAIL PROTECTED] debug $ qsize pax-utils
app-misc/pax-utils-debug-0.1.4-r0: 3 files, 5 non-files, 16.27 KB
app-misc/pax-utils-0.1.4: 6 files, 6 non-files, 102.485 KB
Perhaps we should get input from gentoo-dev ml ?
Should generating internal
debug info still be offered?
When FEATURES=nostrip is enabled we should not split off
any debug info or touch the executable.
What platforms is it supported on?..
Everywhere ELF is a standard.
* misc cleanups of dyn_install (.54 )
Need more info.
This is just something I want todo for my own sanity,
ie break parts of our existing dyn_install() out
into /usr/lib/portage/bin/ scripts.
The current function is about 209 lines of code and I
can see it growing even more.
* flattened vdb {P,R,}DEPEND (.54 )
I might be wrong but I can't really see this being done cleanly. At best,
only
USE flags could be gotten rid of which would still leave || () constructs.
This leads me to question of what use it would really be. If it can only do a
half-arsed job and in a messy way at that I'd personally prefer it to be done
later on.
It will indeed still leave you with || ( foo bar ) or =cpv which you
can be parsed just fine. Yeah it would be nice if it could be reduced
more but later on or now I don't see how it can be reduced anymore than
to the requirements that the ebuild requested. One big advantage for me
here is that virtuals would be resolved.
This will probably lead to an overall reduction in size of the VDB.
* introduction of RRDEPEND to the VDB ( .54 )
What is this again?
Ok I talked a little bit about this on this list the other day and a few
months ago with you on #-portage.
man
RRDEPEND
This entry is automatically created by portage. It contains a
list of reverse dependencies that is achieved by resolving the
DT_NEEDED entries of an ELF executable.
/man
Justification
Programs such as revdep-rebuild, verify-rdepend would be able to make
immediate use. A little bit of a longer term goal is to see portage gain
the ability to request to only use RRDEPEND entries to be used for
depgraph creation for use with embedded/mimimal systems.
ROOT=/dev/shm/minimal emerge -KO --deps=RRDEPEND pkgfoo
RRDEPEND will need to exist due to the RDEPEND explosion and lack of a
clear definition when it was first introduced to portage.
The advantage from where I'm sitting is that devs don't really have a
chance to make mistakes with R/DEPEND handling and people who are
attempting to stage $ROOT can get exactly what they are after in the
embedded world.
--
Ned Ludd [EMAIL PROTECTED]
Gentoo Linux
--
gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Fri, 2005-11-25 at 21:00 +, Ciaran McCreesh wrote: On Fri, 25 Nov 2005 12:05:57 -0500 Ned Ludd [EMAIL PROTECTED] wrote: | Programs such as revdep-rebuild, verify-rdepend would be able to make | immediate use. A little bit of a longer term goal is to see portage | gain the ability to request to only use RRDEPEND entries to be used | for depgraph creation for use with embedded/mimimal systems. How will that work for packages that have a runtime dependency upon a text file supplied by a different package? text files which are true runtime deps belong in RDEPEND. Clearly c++ templates are beyond the scope of the what RRDEPEND can provide. I could be wrong but I don't think those c++ templates are anything revdep-rebuild or verify-rdepends handle any differently. -- Ned Ludd [EMAIL PROTECTED] Gentoo Linux -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
[ Apologies if two of these show up. I kinda, uh, broke Exim slightly... ] On Fri, 25 Nov 2005 16:41:19 -0500 Ned Ludd [EMAIL PROTECTED] wrote: | On Fri, 2005-11-25 at 21:00 +, Ciaran McCreesh wrote: | How will that work for packages that have a runtime dependency upon | a text file supplied by a different package? | | text files which are true runtime deps belong in RDEPEND. So an embedded system creating tool thing will end up providing broken installs when ignoring RDEPEND? | Clearly c++ templates are beyond the scope of the what RRDEPEND can | provide. I could be wrong but I don't think those c++ templates are | anything revdep-rebuild or verify-rdepends handle any differently. Separate issue. I was thinking more along the lines of (non-minimal) vim needing vim-core, for example. -- Ciaran McCreesh : Gentoo Developer (Supreme Lord Gerbil Wrangler) Mail: ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm signature.asc Description: PGP signature
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Fri, 2005-11-25 at 22:02 +, Ciaran McCreesh wrote: On Fri, 25 Nov 2005 16:41:19 -0500 Ned Ludd [EMAIL PROTECTED] wrote: | On Fri, 2005-11-25 at 21:00 +, Ciaran McCreesh wrote: | How will that work for packages that have a runtime dependency upon | a text file supplied by a different package? | | text files which are true runtime deps belong in RDEPEND. Ok. So embedded tools which rely upon RRDEPEND for runtime dependencies will end up producing incomplete installs? Yeah that's what we want, We intend to create tools that leave systems broken. You want to be the first tester? Please take your spin of things off of this and look at it for what it is. Your not going to use a feature for something unless it's suited for the job at hand. -- Ned Ludd [EMAIL PROTECTED] Gentoo Linux -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Fri, 25 Nov 2005 17:49:50 -0500 Ned Ludd [EMAIL PROTECTED] wrote: | Yeah that's what we want, We intend to create tools that leave systems | broken. You want to be the first tester? Please take your spin of | things off of this and look at it for what it is. Your not going to | use a feature for something unless it's suited for the job at hand. So why not create a better feature? -- Ciaran McCreesh : Gentoo Developer (Supreme Lord Gerbil Wrangler) Mail: ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm signature.asc Description: PGP signature
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Fri, 2005-11-25 at 23:10 +, Ciaran McCreesh wrote: On Fri, 25 Nov 2005 17:49:50 -0500 Ned Ludd [EMAIL PROTECTED] wrote: | Yeah that's what we want, We intend to create tools that leave systems | broken. You want to be the first tester? Please take your spin of | things off of this and look at it for what it is. Your not going to | use a feature for something unless it's suited for the job at hand. So why not create a better feature? What the hell are you talking about? No tools have even been created yet. Nobody builds tools before the framework is in place. The ability to make use of RRDEPEND as I've pointed out by verify-rdepend and revdep-rebuild could be pretty much immediate. The ability to control what level of depends a user wants in his/her depgraph is up to the user. The way I envision it you could just as easliy do ROOT=/dev/shm/minimal emerge -KO --deps=RDEPEND:DEPEND:PDEPEND pkgfoo To yield the same results as portage by default. In general I'd suggest that you not attempt to make use of features that don't suit your needs. -- Ned Ludd [EMAIL PROTECTED] Gentoo Linux -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Fri, 2005-11-25 at 23:53 +, Ciaran McCreesh wrote: On Fri, 25 Nov 2005 18:48:41 -0500 Ned Ludd [EMAIL PROTECTED] wrote: | What the hell are you talking about? No tools have even been | created yet. Nobody builds tools before the framework is in place. The | ability to make use of RRDEPEND as I've pointed out by verify-rdepend | and revdep-rebuild could be pretty much immediate. The ability to | control what level of depends a user wants in his/her depgraph is up | to the user. The way I envision it you could just as easliy do | ROOT=/dev/shm/minimal emerge -KO --deps=RDEPEND:DEPEND:PDEPEND | pkgfoo To yield the same results as portage by default. In general | I'd suggest that you not attempt to make use of features that don't | suit your needs. Why introduce a feature which is crippled? It would be almost as easy to allow ebuilds to mess with their 'real' runtime dependency value as appropriate rather than forcing an incorrect auto-generated list onto everyone. Please go back to trolling on dev We are trying to get work done here and solve real problems. -- Ned Ludd [EMAIL PROTECTED] Gentoo Linux -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Fri, 25 Nov 2005 19:00:07 -0500 Ned Ludd [EMAIL PROTECTED] wrote: | Why introduce a feature which is crippled? It would be almost as | easy to allow ebuilds to mess with their 'real' runtime dependency | value as appropriate rather than forcing an incorrect | auto-generated list onto everyone. | | Please go back to trolling on dev We are trying to get work done here | and solve real problems. Sure. You're inventing some arbitrary problem which has no reflection upon reality and then solving some other arbitrary problem which has no reflection upon either reality or what you say you're solving. End result is more unnecessary complexity, more unnecessary mess and, once you realise your solution is inadequate, no doubt yet another incomplete hack on top of that. -- Ciaran McCreesh : Gentoo Developer (Supreme Lord Gerbil Wrangler) Mail: ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm signature.asc Description: PGP signature
Re: [gentoo-portage-dev] .53, .54 and beyond...
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ciaran McCreesh wrote: On Fri, 25 Nov 2005 19:00:07 -0500 Ned Ludd [EMAIL PROTECTED] wrote: | Why introduce a feature which is crippled? It would be almost as | easy to allow ebuilds to mess with their 'real' runtime dependency | value as appropriate rather than forcing an incorrect | auto-generated list onto everyone. Talking on solar about this confirmed my suspicions, the ELF data can't be wrong, otherwise things won't link properly. Thus if we were just to use ELF NEEDED entries, how could the list of reverse runtime deps be incorrect as you imply above? The only assumption here is that ELF is supported on that platform/arch. | | Please go back to trolling on dev We are trying to get work done here | and solve real problems. Sure. You're inventing some arbitrary problem which has no reflection upon reality and then solving some other arbitrary problem which has no reflection upon either reality or what you say you're solving. End result is more unnecessary complexity, more unnecessary mess and, once you realise your solution is inadequate, no doubt yet another incomplete hack on top of that. So in regards to reverse dependency tracking, do you have a solution/advice or just useless criticism? Please attempt to be constructive here. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQ4evCWzglR5RwbyYAQKXlw//YqvAsDkbt3PYmCII6LELOs5qEo8iUPI9 IZuauacBt6uqoVY7UyP36Wt2ky9rqnO2fFlON6i39MjdfN3XsyVIRqVwf4agwWFM QuH19h3wQfCsum5vMKZMej8qfskdpozj4VeTeU2f/NxS6g19LW8vzH7MTDY13tr3 bmY1unyQK7rx6bN+qtV/l22Doq4WnFBDWrY68L00wqHBGzn/VNl9Gh6JTVMTO/AL +yEMma4b0+feCcfrSyxgiliSnaZS+ghJyLPyY4P/gVxDlOP577ufzKxPHgaOh9FN hGaiSaS69Xf4XMcawcdmsE/Tp9Kp1uWXfJibaDCSw4xlmRwm7J26s97NaBu6YsWh keJ1nnMl1O9fjuVCiERVJGMQCYJNAP7up+YAwC62FwNqJSOk5PMS8jz/+uPbWwSW FRTZZCxTDe6JSbZ1RAPLY8xzQFtfdeU4T/wEiWj61w8yRqV132bHiay/lsVNq6P9 GWCvU7pphfe7cNDlk1cHT8eQOE91bVfmKdZBZ+eUgPQk6esMuMCh1MIj38s1lJOi XGxIe3pECS7NPinv8n9ujaYoY7y7Uw+AQTbfFJjdRyldfciqbOpjiv4DfwgVIeiN BE3bio08ybIT7Hb1g9GwPIkycbTbpT4JlBgAKrH3BIBs1d2Syae8DOR3P6WlJDZ/ lD1dIhX5JQ8= =GkJo -END PGP SIGNATURE- -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Fri, 25 Nov 2005 19:42:14 -0500 Alec Warner [EMAIL PROTECTED] wrote: | Ciaran McCreesh wrote: | On Fri, 25 Nov 2005 19:00:07 -0500 Ned Ludd [EMAIL PROTECTED] | wrote: | | Why introduce a feature which is crippled? It would be almost as | | easy to allow ebuilds to mess with their 'real' runtime | | dependency value as appropriate rather than forcing an incorrect | | auto-generated list onto everyone. | | Talking on solar about this confirmed my suspicions, the ELF data | can't be wrong, otherwise things won't link properly. Thus if we were | just to use ELF NEEDED entries, how could the list of reverse runtime | deps be incorrect as you imply above? It can be incomplete. Of course, finding the ELF NEEDED entries is not a sufficient solution to the initial problem, nor is it a sufficient solution to the real problem here. | So in regards to reverse dependency tracking, do you have a | solution/advice or just useless criticism? Please attempt to be | constructive here. Sure. My advice is to scrap the current idea and redo it to take into account things which are not just ELF-related. -- Ciaran McCreesh : Gentoo Developer (The one that looks before leaping) Mail: ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm signature.asc Description: PGP signature
Re: [gentoo-portage-dev] .53, .54 and beyond...
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ciaran McCreesh wrote: On Fri, 25 Nov 2005 19:42:14 -0500 Alec Warner [EMAIL PROTECTED] wrote: | Ciaran McCreesh wrote: | On Fri, 25 Nov 2005 19:00:07 -0500 Ned Ludd [EMAIL PROTECTED] | wrote: | | Why introduce a feature which is crippled? It would be almost as | | easy to allow ebuilds to mess with their 'real' runtime | | dependency value as appropriate rather than forcing an incorrect | | auto-generated list onto everyone. | | Talking on solar about this confirmed my suspicions, the ELF data | can't be wrong, otherwise things won't link properly. Thus if we were | just to use ELF NEEDED entries, how could the list of reverse runtime | deps be incorrect as you imply above? It can be incomplete. Of course, finding the ELF NEEDED entries is not a sufficient solution to the initial problem, nor is it a sufficient solution to the real problem here. Well I bought this bug-spray and it only kills 99% of the bugs in my home, so I guess I should scrap it and work on something better that kills 100% of bugs? If this helps stem system breakage by repairing a number of broken library deps, how is that bad? Because it doesn't adhere to Ciaran's ideal feature standards? | So in regards to reverse dependency tracking, do you have a | solution/advice or just useless criticism? Please attempt to be | constructive here. Sure. My advice is to scrap the current idea and redo it to take into account things which are not just ELF-related. Bricklayers build walls, one brick at a time. - -Alec Warner (antarus) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iQIVAwUBQ4e5yGzglR5RwbyYAQJJWQ/+KdAbQKfUg6HWzzrNRlMJWjKau+PLuW6x gvT/vRU6lYz6612lVbrAmXxir0UdkYjZIe5AXXEz+bTg9K78xL0HuvBb2/fyYihy +pAhkpOhI5eE7dc8fn5vV1dIEOtnBco+UZakSfi9Eai4+PqmaLWwOtiyMnNw3veM yby6Pm/H0VXVzJS5aVQXdPI4PDyyRO3dbLrbuMR9BOPn/qDsZIaB+A+Loy3TajKY BVPsPicGG1OsTnEL8YCgQ4Tl03aepLVuQDomV5vIje4rKfk80fi5UjhMCdHRzFrR Ej2cyyTexQqfNK2IXGh/0R0vgeJCfyMLhCK6b9PkVLSPfv38sJPxWOZuyBd5t8Qu jzJIz0Fhqqg1spfo9rOeFyuW/oOe86hGmFqj+QCrnGhKG0kmyzWeC4IFXClk1PJz P5Kt65uOJU8xOUUZKiUrQ+BnmK0KEYW0InBHmSCCGIjbwC9QCII9gLFlzwqzEd+8 xmnisEa2O5qiX0dSgQoUkenZR9ZvMAWYXkwa9REPiI2uappcagADDL6rR4YIRCOX E96sTByTji7fqu1FsOuARIasdp1PvGoxOr2M5dxoPV/ENcZG0X+NHaw6eUqq13dT ed30XQq/nYxTUcxDKAsWEpOMfMIkzlqxnIL6mN5rjoUZLXnQYDhCJlC3QCmS3uGW eH6ypTlY/9s= =gApf -END PGP SIGNATURE- -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Sat, 26 Nov 2005 00:01:15 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: Hi all, I don't think there's really anything else that can be done for 2.0.53 so am thinking that we should probably push _rc7 + docs out and let the arch teams mark it stable when they're ready (or stick with 2.0.51.22-r3 if it pleaseth them). We should put out a 2.0.54_pre1 out soon after that. What I'm wondering in is what will be going in? So far I'm thinking along the lines of: * cache rewrite * exec cleanup * ldconfig fix * sha1 enabling The only other new thing in trunk that I know of is logging but there's still a question mark over the ordering of messages... Can that be resolved soon? Anything else missing? Any reasons against any of the above? Resolved how? I'm not really sure I understood the original problem (other than listdir being underdeterministic in theory). What's on the map for after that? There's a few things listed on the new (still unreleased?) project index and I'm looking to get the dependency stuff refactored and moved out of emerge.. What are the shortterm goals? - the pycrypto hash additions (for .54) - Manifest2 verification support (need the GLEP first so the format is sanctioned). Technically it's complete, just generation is still unfinished. (so a maybe for .54 depending on the timeframe) - Killing off auto-use+USE_ORDER? - the recursive grab* functions I just posted (for .54) - addition of auxget/metascan tools (could be for .54) - integration of set modules, either as emerge targets (requires serious gutting of emerge) or a first-class atoms (semantically tricky, no clue about implementation yet) Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Saturday 26 November 2005 11:07, Marius Mauch wrote: On Sat, 26 Nov 2005 00:01:15 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: The only other new thing in trunk that I know of is logging but there's still a question mark over the ordering of messages... Can that be resolved soon? Anything else missing? Any reasons against any of the above? Resolved how? I'm not really sure I understood the original problem (other than listdir being underdeterministic in theory). TGL suggested that all the messages go into a single file with some sort of prefix that would then be parsed on the python side. The original order of messages could then be maintained. However, seeing as there needs to be no compatibility with the temporary files it could wait for later anyway. What's on the map for after that? There's a few things listed on the new (still unreleased?) project index and I'm looking to get the dependency stuff refactored and moved out of emerge.. What are the shortterm goals? - the pycrypto hash additions (for .54) This is only useful if the vote goes in favour of adding further hash types to Manfiest, right? If the vote goes that way I've got no issues with it, but if it doesn't it would essentially be dead code. - Manifest2 verification support (need the GLEP first so the format is sanctioned). Technically it's complete, just generation is still unfinished. (so a maybe for .54 depending on the timeframe) Again, depends on -dev. - Killing off auto-use+USE_ORDER? Yep, I'd really like to see this one gone too. We should probably state the intention on -dev first though as there might be a lot of people against it. - the recursive grab* functions I just posted (for .54) Needs a small amount of work (/etc/portage/package.mask/foo/bar would break it) but I like the general idea. - addition of auxget/metascan tools (could be for .54) No qualms. - integration of set modules, either as emerge targets (requires serious gutting of emerge) or a first-class atoms (semantically tricky, no clue about implementation yet) I'm working on this with my refactoring. Defininately a post-.54 thing unless you want to quickly hack it into getlist()? -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Saturday 26 November 2005 02:05, Ned Ludd wrote:
On Sat, 2005-11-26 at 00:51 +0900, Jason Stubbs wrote:
On Saturday 26 November 2005 00:31, Ned Ludd wrote:
* post_sync action hook (.53/.54 )
* VDB prevention of single byte NULL entries being created. ( .54 )
Doable for .54.
Yeah and from the sounds of it we may want more than 1 set of postsync
hooks or the addition of a postsync.d/
(dev thread on getting vital info to users)
Heh.. that was my suggestion. ;)
* new prepstrip offering splitdebug ( .53/.54 )
Need to work out exactly what will be offered when on this one, but at
the earliest it will be .54. Perhaps go with your patch for .54 and leave
Olivier's fancy bits for later?
I can only assure you the code I wrote will function properly.
So that's the only thing I'm trying/willing to push myself.
As long as he has those [ -x checks ] his code should be harmless,
but I don't see the advantage in it over building with -g3.
There are a few other questions too... Should
the default be to generate external debug info?
I think the security team would say yes they want it by default and
would be willing (taviso) to write a proper debug-HOWTO.xml for gentoo.
I think ferringb would say make it FEATURES=splitdebug if
it's going in midstream.
It does add some size which would make peoples $ROOT's a little bit
bigger. But from mine and other peoples testing it's pretty damn
minimal. I think Halcy0n @ gentoo said after doing an -e world he ended
up with only 18M of split debug info
I'm also fond of split packaging of debug info also (but I'm not going
to push for that till I find a more elegant way)
[EMAIL PROTECTED] debug $ qsize pax-utils
app-misc/pax-utils-debug-0.1.4-r0: 3 files, 5 non-files, 16.27 KB
app-misc/pax-utils-0.1.4: 6 files, 6 non-files, 102.485 KB
Perhaps we should get input from gentoo-dev ml ?
Sounds good for pretty much all aspects of split debug (other than the
capability itself).
Should generating internal
debug info still be offered?
When FEATURES=nostrip is enabled we should not split off
any debug info or touch the executable.
FEATURES=splitdebug|stripdebug and do nothing if neither is set?
* misc cleanups of dyn_install (.54 )
Need more info.
This is just something I want todo for my own sanity,
ie break parts of our existing dyn_install() out
into /usr/lib/portage/bin/ scripts.
The current function is about 209 lines of code and I
can see it growing even more.
Code cleanups are always good. :)
* flattened vdb {P,R,}DEPEND (.54 )
I might be wrong but I can't really see this being done cleanly. At best,
only USE flags could be gotten rid of which would still leave || ()
constructs. This leads me to question of what use it would really be. If
it can only do a half-arsed job and in a messy way at that I'd personally
prefer it to be done later on.
It will indeed still leave you with || ( foo bar ) or =cpv which you
can be parsed just fine. Yeah it would be nice if it could be reduced
more but later on or now I don't see how it can be reduced anymore than
to the requirements that the ebuild requested.
Again, if it can be done cleanly code-wise no issues with resolving the USE
flags out.
One big advantage for me here is that virtuals would be resolved.
Resolving virtuals can't be done. Virtuals are meant to be binary compatible
with each other which means that they can be replaced by each other
interchangeably. However, once portage starts using info from vdb (soon) and
starts doing proper sanity checking (not so soon) one would need to re-emerge
all reverse dependencies in order to switch installed virtual provider.
* introduction of RRDEPEND to the VDB ( .54 )
What is this again?
Ok I talked a little bit about this on this list the other day and a few
months ago with you on #-portage.
man
RRDEPEND
This entry is automatically created by portage. It contains a
list of reverse dependencies that is achieved by resolving the
DT_NEEDED entries of an ELF executable.
/man
Justification
Programs such as revdep-rebuild, verify-rdepend would be able to make
immediate use.
Yes, but an extension to emaint will be needed to create the files for
existing installed packages. I'm not sure that Brian's plugin architecture is
ready for .54 so it would need to be another hardcoded check/fix into emaint
itself. Not sure I really like that idea but I'm not going to fight if the
majority think that it should go into .54... What do the majority think?
A little bit of a longer term goal is to see portage gain
the ability to request to only use RRDEPEND entries to be used for
depgraph creation for use with embedded/mimimal systems.
ROOT=/dev/shm/minimal emerge -KO --deps=RRDEPEND pkgfoo
RRDEPEND will need to exist due to the RDEPEND explosion and lack of a
clear definition when it was first introduced to portage.
The advantage from where I'm sitting is that
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Sat, 2005-11-26 at 13:15 +0900, Jason Stubbs wrote: [snip stuff] Need to head to bed now. Will respond to other parts tomorrow. A little bit of a longer term goal is to see portage gain the ability to request to only use RRDEPEND entries to be used for depgraph creation for use with embedded/mimimal systems. ROOT=/dev/shm/minimal emerge -KO --deps=RRDEPEND pkgfoo This is definitely not something that could be done in any of 2.x unless it's done as an external tool (which would not be so hard as order doesn't matter with binary packages). I'm not sure that I like the idea of selectively ignoring *DEPEND in 3.x anyway - by that stage sanity checking should be 100% but selective *DEPENDs will either poke huge holes through it or make a huge mess. Either way, it's not a tomorrow thing so discussion should probably wait until it's more viable. That's fine I'm not ready to focus on --deps= right away. As stated it's a longer term goal and I would also prefer to discuss it at a later time. -- Ned Ludd [EMAIL PROTECTED] Gentoo Linux -- gentoo-portage-dev@gentoo.org mailing list
