On Sun, Dec 18, 2005 at 04:15:45AM +0000, Ciaran McCreesh wrote: > Here we go again... Changes: > > * We're now supporting overlays, multiple repositories and magic flying > chickens. To do this, we're shoving a whole load of new requirements > onto Portage. Said requirements are documented under `Required Portage > Enhancements`_. I now expect to get heavily flamed by a different set > of Portage developers.
You forgot the magic mushroom badgers and snake.
> * Change to yyyy-mm-dd for GLEP 45 compatibility. Rereremove the
> timestamp on the Posted: header.
>
> * Tweak Display-If-{Profile,Installed} to work with multiple
> repositories.
>
> * Use ${repoid} rather than magic-chicken for news-*.* files.
Drop the magic-chicken crap (especially in light of your comments
about 'professional' news inline in the glep).
Killjoy maybe, but it detracts from the point of the glep.
> * Be specific on how news-repoid.skip can work.
Still haven't gotten into specifics, merely a different bit of hand
waving.
> You are encouraged to reply to this thread saying "I agree with ciaranm
> that repository IDs should not be allowed to contain spaces".
>
> TODO: ferringb wants spaces added to the first item on the list. I don't,
> because it makes repo id -> filename mappings nasty.
>
> * Every repository (including overlays) will require a unique identifier. It
> is
> assumed that an identifier will be a string consisting of characters from
> ``a`` to ``z``, ``A`` to ``Z``, ``0`` to ``9``, ``+`` (plus), ``-``
> (hyphen),
> ``:`` (colon) and ``_`` (underscore).
Not really. Just requires your code to be space safe.
You write good code, right? Especially since you need to keep our
path handling safe for osx (/volumes) and for users who do strange
things?
The news handling crap should be able to take spaces- remember the
comments about user aliasing of repostory ID's down the line. I don't
care if the actual metadata/repo_id lacks spaces, but the handling
code must be able to take spaces, as such the requirement that spaces
not be used is arbitrary and uneeded.
> * Portage must provide a way for external programs to obtain a list of all
> repository identifiers for a given system. It is assumed that this will be
> in
> the form of a ``portageq`` command (e.g. ``portageq get_repo_ids``).
>
> * Portage must provide a way for external programs to obtain the base path for
> a repository with a given ID. It is assumed that this will be in the form of
> a ``portageq`` command (e.g. ``portageq get_repo_root gentoo-x86``).
>
> * Portage must extend ``portageq has_version`` to support restrictions to a
> given repository ID.
>
> * Portage must extend ``portageq`` to implement a command which returns
> whether
> or not the profile used for a given repository ID matches a certain base
> path
> (e.g. ``portageq profile_used default-linux/sparc/sparc64/2004.3
> gentoo-x86``).
profile_in_use, maybe.
> These extensions are assumed during the following specification.
>
> News Item Identities
> --------------------
>
> Each news item will have a unique identifier. This identifier will be in the
> form ``yyyy-mm-dd-short-name``, where ``yyyy`` is the year (e.g. ``2005``),
> ``mm`` is the month (``01`` through ``12``) and dd is the day of the month
> (``01`` through ``31``). The ``short-name`` is a very short name describing
> the
> news item (e.g. ``yoursql-updates``), consisting only of the characters
> ``a-z``,
> ``0-9``, ``+`` (plus), ``:`` (colon), ``-`` (hyphen) and ``_`` (underscore).
>
> News Item Directories
> ---------------------
>
> Each news item will be represented by a directory whose name is the same as
> the
> news item's identifier.
>
> The directory will contain a file named ``yyyy-mm-dd-short-name.en.txt``,
> which
> contains the text of the news item, in English, in the format described below.
>
> If a news item is translated, other files named
> ``yyyy-mm-dd-short-name.xx.txt``
> (where ``xx`` is the ISO 639 [#iso-639]_ two letter country code) will also be
> provided. However, only the English version of a news item is authoritative.
> This anglocentricity is justified by precedent [#glep-34]_.
>
> News Item Files
> ---------------
>
> A news item file is a text file, encoded using UTF-8 [#rfc-3629]_ for
> compatibility with and for the same reasons as existing Gentoo documentation
> [#docs-policy]_ and the tree [#glep-31]_.
>
> News items should be signed with a detached GPG signature: ::
why should vs must?
Should leaves open the possibility for a tree compromise and a news
item with Very Bad(tm) instructions stuck into it.
Moving towards getting the whole tree signed, introducing new
components that aren't required signed works against that.
> News Item Headers
> '''''''''''''''''
>
> The following headers describe the purpose and format of the news item:
>
> ``Title:``
> A short (maximum 44 characters) descriptive title. Mandatory.
>
> ``Author:``
> Author's name and email address, in the form ``Real Name <[EMAIL
> PROTECTED]>``.
> Mandatory; multiple author headers may be specified if appropriate.
>
> ``Translator:``
> For translated news items, the translator's name and email address.
> Multiple
> translator headers may be specified if appropriate.
>
> ``Content-Type:``
> Must be ``text/plain``. Mandatory.
>
> ``Posted:``
> Date of posting, in ``yyyy-mm-dd`` format (e.g. 2005-12-18) for
> compatibility with GLEP 45 [#glep-45]_. Mandatory.
>
> ``Revision:``
> Initially 1. Incremented every time a non-trivial change is made. Changes
> which require a re-read of the news item should instead use a new news
> item
> file. Mandatory.
non-trivial changes that don't require a re-read sounds like a
contradiction. Clarify, especially since portage will mark this as
read _once_ and only once.
> The following headers are used for filtering:
>
> ``Display-If-Installed:``
> A dependency atom or simple package name (for example,
Drop the 'simple package name'; it still is an atom.
> ``<dev-lang/php-5_alpha`` or ``net-www/apache``). If the user has the
> package specified installed from the repository from which the news item
> was
> obtained, the news item should be displayed.
This isn't incredibly useful if ranged versions are ever introduced.
Ammending the glep for that seems stupid, looser language might be
wise.
> ``Display-If-Keyword:``
> A keyword [#glep-22]_ name, for example ``mips`` or ``x86-fbsd``. If the
> user is on the keyword in question, the news item should be displayed.
>
> ``Display-If-Profile:``
> A profile path, for example ``default-linux/sparc/sparc64/server``. If the
> user is using the exact profile in question, or a subprofile of this
> profile, the news item should be displayed. This header may be used to
> replace ``deprecated`` files in the future.
>
> .. Note:: When performing package moves, developers must also update any
> relevant ``Display-If-Installed`` headers in news files.
Grounds for someone to get off their ass and do some work, but this
should be automated in some fashion- specifically detection of it
(auto-updating won't work with signing).
> The algorithm used to determine whether a news item is 'relevant' is as
> follows:
>
> * For each ``Display-If-`` header type which occurs at least once:
>
> + The news item is not relevant if none of the headers of this type are
> successfully matched.
>
> * Otherwise the news item is relevant.
>
> In particular, if no ``Display-If-`` header is specified, a news item will be
> relevant for all users.
implementation issue= be aware this must be cached. No caching ==
slowing down portage pissing off users.
All news items *will* need to be cached in some format here- just
because the keyword isn't what the user is running at the time of
sync, doesn't mean that the keyword won't be used for ROOT!="/" cross
compilation.
That's an implementation note, but I'm bringing it up since the time
to do the filter/cache instantiation is during portage initialization
(yes it sucks, but your stuck with stable)... so start thinking about
ways to make it fast, since python -c'import portage' is the slowest
part of portage queries
> News Item Quality Control
> -------------------------
>
> There have been complaints regarding the comprehensibility of some upgrade
> notices and news items in the past. This is understandable ??? not every
> Gentoo
'???' ?
> .. Note:: A previous draft of this GLEP allowed news items to be sent to
> ``gentoo-core`` instead of ``gentoo-dev``. It is possible that a situation
> may arise where this will be necessary (for example, a security update
> which
> must break backwards compatibility and which cannot be revealed to the
> public
> before a given date).
Drop that and just state that -core doesn't fly sans security
consideration; referencing one of the previous 5 versions isn't needed
(yes it's minor, but it's uneeded info).
> Server Side
> '''''''''''
>
> News items are to be made available via the standard rsync tree. This removes
> any need for polling of a remote source.
>
> A new repository will be created for news items. The type (CVS or Subversion),
> location and access controls on this repository are beyond the scope of this
> GLEP.
We generate a tree every 30 minutes. You aiming for every update, or
for less frequent (once an hour fex)?
Matters due to the fact rsync tree generation has a window it must not
overrun- minor but continuing the "lets be explicit" goal of yours.
> This repository will contain directories named ``yyyy/mm/``, where ``yyyy`` is
> the current year and ``mm`` is the current month number (01 for January
> through
> 12 for December). This separation will help keep news items more manageable.
>
> The contents of this repository will automatically be merged with the main
> rsync
> tree, placing the items in a ``metadata/news/`` directory. The method used for
> merging these items is beyond the scope of this GLEP ??? a similar setup is
> already used for merging GLSAs into the rsync tree.
>
> The main rsync tree will **not** use the ``yyyy/mm/`` subdirectory layout.
What shall it use? Again, be explicit.
> Client Side
> '''''''''''
>
> Whenever relevant unread news items are found, the package manager will
> create a
> file named ``/var/lib/gentoo/news/news-repoid.unread`` (if it does not
-file named ``/var/lib/gentoo/news/news-repoid.unread`` (if it does not
+file named ``/var/lib/gentoo/news/news-${repoid}.unread`` (if it does not
Clearer at a first read though imo.
> already exist) and append the news item identifier (eg
> ``2005-11-01-yoursql-updates``) on a new line.
Implementation details, but this handling will need to be integrated
into portage (eg, no external scripts). External scripts will slow
things down (shell overhead/fork/exec), plus portage is going to have
to query this cruft already for the notifications on emerge targets
(no we're not execing everytime for that either ;)
> Checks for new news messages should be displayed:
>
> * After an ``emerge sync``
> * After an ``emerge --pretend``
> * Before an ``emerge <target>`` (which may also include a red warning message)
> * Before an ``emerge --ask <target>`` sequence
ask is superfluous, nuke it.
You haven't stated how the 'package manager' will trigger the user's
reader of choice for these targets. Should also extend this to allow
a way to disable any news notices, lest someone's cronjob get hung
displaying news (feature or not, it's needed).
> The package manager must keep track of news items that have already been added
> to the unread list to avoid repeatedly marking a deleted news item. This could
> be handled via a ``news-repoid.skip`` file containing the IDs of news items
> that
> have already been added to a ``news-repoid.unread`` file, but this method is
> not
> required by this GLEP.
Specifics.
> Once a news item is marked for reading, third party tools (or traditional core
> Unix tools) can be used to display and view the news files.
>
> When a news item is read, its name should be removed from the
> ``news-repoid.unread`` file. If a news client acts as an interactive reader
> rather than a gateway, it should then add the name to a ``news-repoid.read``
> file in the same directory with the same file format.
implementation issue; you need locking on this. If portage has to
farm out to the users reader of choice, then it needs to lock the file
to keep readers/writers from screwing with each other.
Flock preferably, since it's faster then resorting to hardlink
trickery (yes this may be harder for shell crap, but so it goes since
hardlink locking sucks).
> News Item Removal
> -----------------
>
> News items can be removed (by removing the news file from the main tree) when
> they are no longer relevant, if they are made obsolete by a future news item
> or
> after a long period of time. This is the same as the method used for
> ``updates``
> entries.
implementation issue; updating unread/skip crap so it doesn't bloat is
required, but time frame also matters (crap sync resulting in news
getting removed, yet it still being valid, just missing from the local
tree).
> Integration with Existing Systems
> =================================
>
> It would be simple to convert these news items into the format used for news
> items on the Gentoo website or posts for the ``gentoo-announce`` mailing list.
>
> There is an existing automated tool [#forums-glsa]_ for posting GLSAs to the
> forums. A similar tool can be used for these news items.
Pawned it off on someone, or something you'll be doing?
~harring
pgpUdum21hBjx.pgp
Description: PGP signature
