Re: [gentoo-portage-dev] Re: [gentoo-dev] digest + manifest = new file format

2005-08-05 Thread Jason Stubbs
On Friday 05 August 2005 11:42, Alec Warner wrote:
> Whoopse, should have gone to gentoo-portage-dev, my bad ;)
>
> Alec Warner wrote:
> > This is basically a resubmission of Genone's file format merger [1].
> >
> > The code to handle digests and manifests is currently being designed
> > and rewritten.  As such we need two things from the developer
> > community. One is a decision of what the new format will entail.  Two
> > is a firm agreement on the migration plan.
> >
> > 1.  The file format is specified in Genone's original mail, but I'll go
> > over it again to save you the extra linkage ;)
> >
> > Inside of the digest-manifest we have:
> > EBUILD Filename SIZE 1234567 MD5 A4FD085FF SHA-256 A7439FDB1
> >   
> > 
> > Where type is the type of file (DIGEST, SRC_URI, EBUILD, PATCH, etc)
> >
> > *Note, that the size-tag was added to make the human parsing easier.

That's incorrect. It was added because there is nothing special about size 
other than it being another way to improve your percentage of chance that 
the file you have matches.

> > 2.  If you didn't notice by now, this format breaks versions of portage
> > that don't support it ( ie 2.0.X ).  This leads us to a few options as
> > users migrate to a version of portage that supports the new file
> > format.
> >
> > A.  Write some code into the final stable version to do both kind of
> > formats and/or use the new format style but fall back to the old style
> > ( since at release time the new style format probably won't be in the
> > tree yet ).  Either way, the last release of 2.0.X should work with the
> > new file format in enough of a fashion to not break.  This means that
> > when the old manifest + digests are combined in the tree, most users
> > should be ok even if they aren't upgraded to 2.1.X.
> >
> > B.  Have a migration time where both formats are in the tree.  For a
> > while the tree will be larger, and many people will have issues with
> > this.  However, old portage will use the old files, and new releases
> > will use the new files.  Then after a period of however long ( 6
> > months? ) announce loudly beforehand that portage-2.0.X will no longer
> > be supported and pull the old digests/manifests out of the tree.
> >
> > C.  The Carpaski way ;)[2]  Add support for the new format in 2.1,
> > while also adding support for current format.  Force everyone to
> > upgrade to 2.1.X, while announcing that 2.0.X will not work in the
> > future.  When we have confidence that most users are upgraded, pull the
> > old format and add the new format to the tree.
> >
> > Problems with all of these include the same problems as the cascaded
> > profiles, some goofball doesn't upgrade for a year, syncs with new
> > digests...how does he get his portage upgraded?  An upgrade path should
> > be provided and documented in this case.
> >
> > The best route is probably some combination of the above. 
> > Pre-emptively add support for the new format to stable, while
> > announcing the death of the old format after 2.1's release.  When most
> > users upgrade normally to the latest 2.0.X series ( perhaps fearing the
> > changes of 2.1.X ) they will gain support for the new file format (or
> > if not support, merely a working portage instead of broken).  We should
> > catch the majority of users who upgrade with either the last 2.0.X
> > release or the new release of 2.1.X.
> >
> > Suggestions, criticisms, etc... welcome.

This stuff is all going into the Manifest rather than per ebuild digests, 
right? Portage already ignores any line in Manifest that does not begin 
with MD5. Manifests can be easily bypassed with FEATURES="-strict" anyway. 
The only real issue is SRC_URI files. An upgrade path can be made available 
by maintaining the digest.* files only for portage.

There are/will be several more issues like these. The idea is already to 
have an ebuild format tag that portage will either be able to handle or 
not. How about just patching stable to not allow merging of any ebuild that 
has the tag defined and requesting the user to upgrade?

In this case, the new portage would need to support both formats but the 
tree wouldn't require them both formats. The new commit tool would generate 
an old style manifest/digest for old ebuilds and new style for new ones.

--
Jason Stubbs


pgpuCvWaWLjfY.pgp
Description: PGP signature


[gentoo-portage-dev] Re: [gentoo-dev] digest + manifest = new file format

2005-08-04 Thread Alec Warner
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Whoopse, should have gone to gentoo-portage-dev, my bad ;)

Alec Warner wrote:
> This is basically a resubmission of Genone's file format merger [1].
> 
> The code to handle digests and manifests is currently being designed and
> rewritten.  As such we need two things from the developer community.
> One is a decision of what the new format will entail.  Two is a firm
> agreement on the migration plan.
> 
> 1.  The file format is specified in Genone's original mail, but I'll go
> over it again to save you the extra linkage ;)
> 
> Inside of the digest-manifest we have:
> EBUILD Filename SIZE 1234567 MD5 A4FD085FF SHA-256 A7439FDB1
>   
> 
> Where type is the type of file (DIGEST, SRC_URI, EBUILD, PATCH, etc)
> 
> *Note, that the size-tag was added to make the human parsing easier.
> 
> 2.  If you didn't notice by now, this format breaks versions of portage
> that don't support it ( ie 2.0.X ).  This leads us to a few options as
> users migrate to a version of portage that supports the new file format.
> 
>   A.  Write some code into the final stable version to do both kind of
> formats and/or use the new format style but fall back to the old style (
> since at release time the new style format probably won't be in the tree
> yet ).  Either way, the last release of 2.0.X should work with the new
> file format in enough of a fashion to not break.  This means that when
> the old manifest + digests are combined in the tree, most users should
> be ok even if they aren't upgraded to 2.1.X.
> 
>   B.  Have a migration time where both formats are in the tree.  For a
> while the tree will be larger, and many people will have issues with
> this.  However, old portage will use the old files, and new releases
> will use the new files.  Then after a period of however long ( 6 months?
> ) announce loudly beforehand that portage-2.0.X will no longer be
> supported and pull the old digests/manifests out of the tree.
> 
>   C.  The Carpaski way ;)[2]  Add support for the new format in 2.1,
> while also adding support for current format.  Force everyone to upgrade
> to 2.1.X, while announcing that 2.0.X will not work in the future.  When
> we have confidence that most users are upgraded, pull the old format and
> add the new format to the tree.
> 
>   Problems with all of these include the same problems as the cascaded
> profiles, some goofball doesn't upgrade for a year, syncs with new
> digests...how does he get his portage upgraded?  An upgrade path should
> be provided and documented in this case.
> 
> The best route is probably some combination of the above.  Pre-emptively
> add support for the new format to stable, while announcing the death of
> the old format after 2.1's release.  When most users upgrade normally to
> the latest 2.0.X series ( perhaps fearing the changes of 2.1.X ) they
> will gain support for the new file format (or if not support, merely a
> working portage instead of broken).  We should catch the majority of
> users who upgrade with either the last 2.0.X release or the new release
> of 2.1.X.
> 
> Suggestions, criticisms, etc... welcome.
> 
> [1] http://marc.theaimsgroup.com/?l=gentoo-dev&m=109725383228494&w=2
> [2] http://marc.theaimsgroup.com/?l=gentoo-dev&m=109725779909405&w=2
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQvLSLmzglR5RwbyYAQJkEQ//aQFAjvEFJ1tRq988HOEyh/XxHlsPnksm
HAQCs8iSkLY6cI9IcRniTgkLuNwLYMqElEMACSWlTWCGmL5lrG/6XDkPaXDBeNLg
jyw1h5yCauQnH8THwzuLQ4RVg3pFsmIthrJyB55V4i/DDiAY6JGfs7xwSys1Poih
rUohcrWiI/gWr2UHzrlS4xtqO4iWw5kpUBAslDk8D+RZEjn1i5GcW/3J8GBS3RUF
TFKkzBf/ruhqN50Cpbo6Q0gEnvKcgtQSrYBlIyCdw+GpGqbalPVbGPVbozApwk6d
DQvLbMgZq7G8oM8vo8IL0cThdocNfWcNDAUrPadxzA6szzxxCiu3DiCVNLRoAhOy
2JCyGfTEY3KStX39NvCCcnkGTcP6XbGw3ummPJwLlNxP6x+KwCbHj5Hats+LDO71
dxOjh8+GeOCmafO89OmSD2W2l/zAdmU6KPUYY7OfUttrmbcTZehdTksQBZqlohRg
Coz4Fe6BxMn3dQ2C3wplALwRIikLsP95UwH7MSlAIFzrEdy7XPQydPKB/kjOk4e3
r7nrM/EOoW54QbhX8ItY+y3dEDJWNKJduax7uYXbz+tMH0J74oxiVswcu/LTgXFv
RkCJ4UqED7zvyQafaL5yTk8UYknI/oeo7x9siiqSJ90J7Y6m2GrX1i8vLoiOnyTf
+2gWdQCckT8=
=TOrq
-END PGP SIGNATURE-
-- 
gentoo-portage-dev@gentoo.org mailing list