Re: [gentoo-portage-dev] .53, .54 and beyond...
Ned Ludd wrote: On Mon, 2005-12-05 at 23:06 +0900, Jason Stubbs wrote: Okay, new suggestion. Postpone the cache rewrite from above. Have only the minimal mods necessary to fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be 2.0.54 as per the attached patch. Get that out soon and get trunk out masked at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch. However, instead of ~arch meaning "regression fixes only" we could just limit it to "minor changes only" (ie. no big refactorings, rewrites or similar high risk changes) until it is time to stable it. I think it would be wise to reconsider the cache fixes. I know you have been away from irc for a while now and have missed the daily events, but most of the people we have interacted with are expecting the cache updates in .54 (alot of people complaining about the hanging at 50%) The code has been pretty well tested and seems safe on the surface. I think ferringb's testing has shown that the cache updates use about 14M of ram where the existing code (as of .52.x) uses about 80M of ram. But still, it's annoying to be stuck with only 2 tiers. Why not put a snapshot of trunk in the tree and package.mask it? Wouldn't that make everyone happy? Zac -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Mon, 2005-12-05 at 23:06 +0900, Jason Stubbs wrote: > Okay, new suggestion. > > Postpone the cache rewrite from above. Have only the minimal mods necessary > to > fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be > 2.0.54 as per the attached patch. Get that out soon and get trunk out masked > at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch. > However, instead of ~arch meaning "regression fixes only" we could just limit > it to "minor changes only" (ie. no big refactorings, rewrites or similar high > risk changes) until it is time to stable it. I think it would be wise to reconsider the cache fixes. I know you have been away from irc for a while now and have missed the daily events, but most of the people we have interacted with are expecting the cache updates in .54 (alot of people complaining about the hanging at 50%) The code has been pretty well tested and seems safe on the surface. I think ferringb's testing has shown that the cache updates use about 14M of ram where the existing code (as of .52.x) uses about 80M of ram. -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
Jason Stubbs wrote: Postpone the cache rewrite from above. Have only the minimal mods necessary to fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be 2.0.54 as per the attached patch. Get that out soon and get trunk out masked at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch. However, instead of ~arch meaning "regression fixes only" we could just limit it to "minor changes only" (ie. no big refactorings, rewrites or similar high risk changes) until it is time to stable it. That seems reasonable. Perhaps we could introduce a third tier (beyond stable and ~arch) via -* keywords or package.mask? However, with the trunk's target (2.1?) rather than letting the arch teams decide when we call it stable, I think it would be a better idea to move it to the .0 version when we feel it is ready leaving it in ~arch. As regressions are fixed the .0 can be bumped to .1, .2 or whatever, but the focus can move to what happens beyond that rather than waiting... Yeah, this focus on waiting is rather annoying. :) Zac -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Tuesday 06 December 2005 00:21, Alec Warner wrote: > Jason Stubbs wrote: > > Okay, new suggestion. > > > > Postpone the cache rewrite from above. Have only the minimal mods > > necessary to fix the PORT_LOGDIR/tee bug. Include the other two as is. > > That would be 2.0.54 as per the attached patch. Get that out soon and get > > trunk out masked at around the same time. As soon as 2.0.54 goes stable > > put trunk into ~arch. However, instead of ~arch meaning "regression fixes > > only" we could just limit it to "minor changes only" (ie. no big > > refactorings, rewrites or similar high risk changes) until it is time to > > stable it. > > Postponing the cache rewrite is going to piss a lot of poeple off, just > FYI :) I realize it's a large patch, but it has been tested by plenty > of people, and many of them are waiting for this fix to hit stable > (don't want to patch portage on a production system). Any particular > reason you want it held off (besides it's immensity?) The delay it would cause for other things waiting in the pipeline. -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Jason Stubbs wrote:
>
>
> Okay, new suggestion.
>
> Postpone the cache rewrite from above. Have only the minimal mods necessary
> to
> fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be
> 2.0.54 as per the attached patch. Get that out soon and get trunk out masked
> at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch.
> However, instead of ~arch meaning "regression fixes only" we could just limit
> it to "minor changes only" (ie. no big refactorings, rewrites or similar high
> risk changes) until it is time to stable it.
>
Postponing the cache rewrite is going to piss a lot of poeple off, just
FYI :) I realize it's a large patch, but it has been tested by plenty
of people, and many of them are waiting for this fix to hit stable
(don't want to patch portage on a production system). Any particular
reason you want it held off (besides it's immensity?)
> However, with the trunk's target (2.1?) rather than letting the arch teams
> decide when we call it stable, I think it would be a better idea to move it
> to the .0 version when we feel it is ready leaving it in ~arch. As
> regressions are fixed the .0 can be bumped to .1, .2 or whatever, but the
> focus can move to what happens beyond that rather than waiting...
>
> First paragraph is more important right now.
>
> --
> Jason Stubbs
>
>
>
>
> diff -uNr portage-2.0.53/pym/portage.py portage-2.0.54/pym/portage.py
> --- portage-2.0.53/pym/portage.py 2005-12-01 22:04:17.0 +0900
> +++ portage-2.0.54/pym/portage.py 2005-12-05 22:54:53.0 +0900
> @@ -2039,11 +2039,6 @@
> myline += " "+mysum
> myline += " "+myarchive
> myline += " "+str(mysize)
> - if sumName != "MD5":
> - # This cannot be used!
> - # Older portage make very dumb assumptions
> about the formats.
> - # We need a lead-in period before we break
> everything.
> - continue
> mylines.append(myline)
> return mylines
>
> @@ -6430,6 +6425,9 @@
> writemsg("!!! FAILED postrm: "+str(a)+"\n")
> sys.exit(123)
>
> + #update environment settings, library paths. Change symlinks.
> + env_update(makelinks=1)
> +
> self.unlockdb()
>
> def isowner(self,filename,destroot):
> @@ -6672,13 +6670,10 @@
> writemsg("!!! FAILED postinst: "+str(a)+"\n")
> sys.exit(123)
>
> - downgrade = False
> - for v in otherversions:
> - if pkgcmp(catpkgsplit(self.pkg)[1:],
> catpkgsplit(v)[1:]) < 0:
> - downgrade = True
> -
> #update environment settings, library paths. DO NOT change
> symlinks.
> - env_update(makelinks=(not downgrade))
> + #only needed if we did not already run unmerge
> + if not (oldcontents):
> + env_update(makelinks=0)
> #dircache may break autoclean because it remembers the
> -MERGING-pkg file
> global dircache
> if dircache.has_key(self.dbcatdir):
> diff -uNr portage-2.0.53/pym/portage_exec.py
> portage-2.0.54/pym/portage_exec.py
> --- portage-2.0.53/pym/portage_exec.py2005-12-01 22:04:00.0
> +0900
> +++ portage-2.0.54/pym/portage_exec.py2005-12-05 22:52:30.0
> +0900
> @@ -177,9 +177,9 @@
> spawned_pids.append(mypid[-1])
> return mypid
> while len(mypid):
> - retval=os.waitpid(mypid[-1],0)[1]
> + retval=os.waitpid(mypid[0],0)[1]
> if retval != 0:
> - for x in mypid[0:-1]:
> + for x in mypid[1:]:
> try:
> os.kill(x,signal.SIGTERM)
> if os.waitpid(x,os.WNOHANG)[1] == 0:
> @@ -198,7 +198,7 @@
> else:
> return ((retval & 0xff) << 8) # interrupted by
> signal
> else:
> - mypid.pop(-1)
> + mypid.pop(0)
> return 0
>
> def find_binary(myc):
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iQIVAwUBQ5Ra8mzglR5RwbyYAQJkeQ//UzWHZZc80eZj7q8mHbns4BSYvufFbbyY
2zRP2eiwP+MmzLgdpDDq0FsvE3I+6bHeGFQLB18UybwtUEtpQ6Q2BcR7UwLa48DX
CPrzxTGBJZxsAQ+qiGL/F9eupAgmH0tnXqUKp9al1Y+NCMBrfFam8WkAobgn65eB
mv5oqcJT/vl/FeScNXTeTw3Ud0Sy3a1lw1th5ePAPJiWVzneXHaeGVi+iCtQnY8O
ArzVg8O4SC0nSrn/4FdZKgCFoBvc0rmUnjcNi5X59OgWB
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Thursday 01 December 2005 22:28, Jason Stubbs wrote:
> On Monday 28 November 2005 03:49, Marius Mauch wrote:
> > Jason Stubbs wrote:
> > > On Sunday 27 November 2005 00:09, Marius Mauch wrote:
> > >>Jason Stubbs wrote:
> > >>Well, the vote was more for the SHA1 change actually as that's the one
> > >>triggering the size increase. The pycrypto stuff itself doesn't do
> > >>anything really, it would just make the size increase more apparent.
> > >
> > > Hmm.. I thought it was for hashes supported by pycrypto being added
> > > into Manifest before Manifest2 comes along. If it was with regard to
> > > SHA1 then I take back my vote to delay.
> >
> > Yeah, I guess the mail could be read both ways.
> >
> > >>Don't think so given that offhand I don't even know what getlist() does
> > >> ...
> > >
> > > getlist() is defined in emerge and is used to access the system and
> > > world sets. It wouldn't be too hard to customize it to handle user sets
> > > and modify other code to support them but the "can't combine sets and
> > > atoms" rule would get a bit messy.
> >
> > So gutting of emerge ... nope, tried that two times already, but gave up
> > after hitting too many direct references to system and world.
> >
> > >>Oh, btw, two things that are in trunk but weren't listed in your
> > >>original mail:
> > >>- the rewritten versioning code (including the cvs and mult-suffix
> > >>enhancements)
> > >>- finally killing of the stupid "masked by -*" message
> > >
> > > That makes the current list for .54:
> > >
> > > * autouse death
> > > * cache rewrite
> > > * dyn_install cleanup
> > > * einfo logging
> > > * exec cleanup
> > > * flattened vdb *DEPENDs
> > > * hash support via pycrypto
> > > * ldconfig fix
> > > * metascan/auxget
> > > * postsync hooks
> > > * recursive grab*
> > > * RRDEPEND/LDEPEND
> > > * sha1 enabling
> > > * splitdebug
> > > * vdb empty file culling
> > >
> > > Are we about there yet? Also, what does this mean for 2.1/2.2?
> >
> > Well, if that featurelist is .54 then I really don't see a point for
> > making a 2.1 or 2.2 release line. Before your mail starting this thread
> > I assumed that .54 would just contain the ldconfig fix, the hash stuff
> > and maybe a few other minor things, while trunk would become 2.2.
> > But if things like elog, the new cache subsystem, splitdebug or the
> > *DEPEND changes don't qualify for a "minor" version bump, then I can't
> > think of anything that would.
>
> The ldconfig bug and the exec cleanup are the only urgent ones among them.
> The exec cleanup could be postponed and the existing code twiddled in a
> couple of places to fix the logging bug. However, the biggest issue that
> users are complaining about at present is the caching. The biggest issue
> for devs is security. Hence my original list:
>
> * cache rewrite
> * exec cleanup
> * ldconfig fix
> * sha1 enabling
Okay, new suggestion.
Postpone the cache rewrite from above. Have only the minimal mods necessary to
fix the PORT_LOGDIR/tee bug. Include the other two as is. That would be
2.0.54 as per the attached patch. Get that out soon and get trunk out masked
at around the same time. As soon as 2.0.54 goes stable put trunk into ~arch.
However, instead of ~arch meaning "regression fixes only" we could just limit
it to "minor changes only" (ie. no big refactorings, rewrites or similar high
risk changes) until it is time to stable it.
However, with the trunk's target (2.1?) rather than letting the arch teams
decide when we call it stable, I think it would be a better idea to move it
to the .0 version when we feel it is ready leaving it in ~arch. As
regressions are fixed the .0 can be bumped to .1, .2 or whatever, but the
focus can move to what happens beyond that rather than waiting...
First paragraph is more important right now.
--
Jason Stubbs
diff -uNr portage-2.0.53/pym/portage.py portage-2.0.54/pym/portage.py
--- portage-2.0.53/pym/portage.py 2005-12-01 22:04:17.0 +0900
+++ portage-2.0.54/pym/portage.py 2005-12-05 22:54:53.0 +0900
@@ -2039,11 +2039,6 @@
myline += " "+mysum
myline += " "+myarchive
myline += " "+str(mysize)
- if sumName != "MD5":
-# This cannot be used!
-# Older portage make very dumb assumptions about the formats.
-# We need a lead-in period before we break everything.
-continue
mylines.append(myline)
return mylines
@@ -6430,6 +6425,9 @@
writemsg("!!! FAILED postrm: "+str(a)+"\n")
sys.exit(123)
+ #update environment settings, library paths. Change symlinks.
+ env_update(makelinks=1)
+
self.unlockdb()
def isowner(self,filename,destroot):
@@ -6672,13 +6670,10 @@
writemsg("!!! FAILED postinst: "+str(a)+"\n")
sys.exit(123)
- downgrade = False
- for v in otherversions:
- if pkgcmp(catpkgsplit(self.pkg)[1:], catpkgsplit(v)[1:]) < 0:
-downgrade = True
-
#update environment settings, library paths. DO NOT change symlinks.
- env_update(makelinks=(not downgrade))
+
