Re: [gentoo-dev] Ruby keywording
On Wed, 07 Mar 2012 08:00:16 +0100 Paweł Hajdan, Jr. phajdan...@gentoo.org wrote: Also the inter-bug dependencies are often not resolved correctly, that is the to be keyworded package depends on non-keyworded stuff not listed in the bug. And this is even worse. Please check things with repoman before filing bugs. You can even write automated scripts at least for the check whether we got all deps right part. As a maintainer I can tell you that when you drop keywords on B because it needs non keyworded A, then drop keywords on C because it needs latest B then drop keywords on D because it needs latest C, you have completely forgotten that some arches need A, which ones, etc. There are scripts for this, and I hope arch teams that like to have a list use them. As occasionally doing fbsd keywording, I almost never read nor use a list that is provided since the above scenario often occurs (or at least used to). Instead of this, I do a depth-first keywording of packages repoman tells are missing. The deepest package is in the latest tab of my terminal emulator :) I'll run repoman anyway, and this approach allows a double checking. Also, since this means I'll start committing from the leaves of the depgraph, this ensures no package has broken deps between commits (with the exception of circular deps of course). A.
Re: [gentoo-dev] Ruby keywording
On 09:25 Wed 07 Mar 2012, Alexis Ballier wrote: On Wed, 07 Mar 2012 08:00:16 +0100 Paweł Hajdan, Jr. phajdan...@gentoo.org wrote: Also the inter-bug dependencies are often not resolved correctly, that is the to be keyworded package depends on non-keyworded stuff not listed in the bug. And this is even worse. Please check things with repoman before filing bugs. You can even write automated scripts at least for the check whether we got all deps right part. As a maintainer I can tell you that when you drop keywords on B because it needs non keyworded A, then drop keywords on C because it needs latest B then drop keywords on D because it needs latest C, you have completely forgotten that some arches need A, which ones, etc. There are scripts for this, and I hope arch teams that like to have a list use them. What scripts are out there? I just do iterated repoman calls without much automation (pretty much as described below). Got anything better? - please post it! Cheers, Thomas As occasionally doing fbsd keywording, I almost never read nor use a list that is provided since the above scenario often occurs (or at least used to). Instead of this, I do a depth-first keywording of packages repoman tells are missing. The deepest package is in the latest tab of my terminal emulator :) I'll run repoman anyway, and this approach allows a double checking. Also, since this means I'll start committing from the leaves of the depgraph, this ensures no package has broken deps between commits (with the exception of circular deps of course). A. -- Thomas Kahle http://dev.gentoo.org/~tomka/ signature.asc Description: Digital signature
Re: [gentoo-dev] Ruby keywording
On Wed, 7 Mar 2012 15:54:49 +0100 Thomas Kahle to...@gentoo.org wrote: On 09:25 Wed 07 Mar 2012, Alexis Ballier wrote: On Wed, 07 Mar 2012 08:00:16 +0100 Paweł Hajdan, Jr. phajdan...@gentoo.org wrote: Also the inter-bug dependencies are often not resolved correctly, that is the to be keyworded package depends on non-keyworded stuff not listed in the bug. And this is even worse. Please check things with repoman before filing bugs. You can even write automated scripts at least for the check whether we got all deps right part. As a maintainer I can tell you that when you drop keywords on B because it needs non keyworded A, then drop keywords on C because it needs latest B then drop keywords on D because it needs latest C, you have completely forgotten that some arches need A, which ones, etc. There are scripts for this, and I hope arch teams that like to have a list use them. What scripts are out there? I just do iterated repoman calls without much automation (pretty much as described below). Got anything better? - please post it! gnome team posts nice lists afaik: http://git.overlays.gentoo.org/gitweb/?p=proj/gnome.git;a=blob;f=scripts/gen_archlist.py
Re: [gentoo-dev] Ruby keywording
On Tue, 2012-03-06 at 23:17 +0100, Thomas Kahle wrote: Ruby is an interpreted language, I don't see any point in having every arch team do the testing for every small package. Could the ruby team add ~x86 themselves after testing on ~amd64, or are there compelling reasons to not do this? This discussion has come up in the past, not just for Ruby. The consensus seems to be that it's better to keep handling everything by the arch teams, since there may be other issues besides compilation that may cause differences on arches. I know of several arch-specific bugs in interpreted Ruby code myself. It's also a matter of where to stop. Each new keyword added in the end is an additional maintenenance burden for that arch. I don't want to make that decision since I won't be doing the work. Hans signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Ruby keywording
On Wed, 2012-03-07 at 08:00 +0100, Paweł Hajdan, Jr. wrote: It's trivial to set up x86 chroot on amd64 box, so I can't imagine what's preventing people from creating such chroot, doing the testing and keywording themselves. In my personal case what is preventing me is sheer and utter disinterest. For me x86 and, say, sh, are similar arches. I'm happy to support both via keywording and stabilization bugs, but I'd rather spent what little time I have these days on actual ruby stuff. Kind regards, Hans signature.asc Description: This is a digitally signed message part
[gentoo-dev] RFD: EAPI specification in ebuilds
Hi all, The way how we currently specify the EAPI in ebuilds has some problems. For example, there is no sane way to allow usage of features of a new bash version in a new EAPI. So we are currently stuck with bash 3.2. Also changes of global scope behaviour, like addition of new global scope functions (similar to inherit) are not possible. These flaws are outlined in GLEP 55 [1]: | In order to get the EAPI the package manager needs to source the | ebuild, which itself needs the EAPI in the first place. Otherwise it | imposes a serious limitation, namely every ebuild, using any of the | future EAPIs, will have to be source'able by old package managers | [...] The council has voted down GLEP 55 more than a year ago, but at the same time requested that a solution for the mentioned issues should be found. [2] However, there was no progress since then. The issue arose again in bug 402167 [3] where several solutions have been discussed. Below, I try to summarise the possible options resulting from that discussion. *** Proposal 1: Parse the EAPI assignment statement *** This first proposal would require that the syntax of the EAPI assignment statement in ebuilds matches a well defined regular expression. A scan of the Portage tree shows that the statement only occurs in the following variations (using EAPI 4 as example): EAPI=4 EAPI=4 EAPI='4' Sometimes this is followed by whitespace or a comment (starting with a # sign). Also, with very few exceptions the EAPI assignment occurs within the first few lines of the ebuild. For the vast majority of ebuilds it is in line 5. Written in a more formal way, appropriate for a specification: - Ebuilds must contain at most one EAPI assignment statement. - It must occur within the first N lines of the ebuild (N=10 and N=30 have been suggested). - The statement must match the following regular expression (extended regexp syntax): ^[ \t]*EAPI=([']?)([A-Za-z0-9._+-]*)\1[ \t]*(#.*)?$ Note: The first and the third point are already fulfilled by all ebuilds in the Portage tree. The second point will require very few ebuilds to be changed (9 packages for N=10, or 2 packages for N=30). The package manager would determine the EAPI by parsing the assignment with above regular expression. A sanity check would be added. Citing Zac Medico in [3]: The fact that we can compare the probed EAPI to the actual EAPI variable after the ebuild is sourced seems like a perfect sanity check. We could easily detect inconsistencies and flag such ebuilds as invalid, providing a reliable feedback mechanism to ebuild developers. This proposal comes in two variants: 1a) The change is applied retroactively for all EAPIs. 1b) It is only applied for EAPI 5 and later (which means that the result of the EAPI parsing would be discarded for earlier EAPIs). *** Proposal 2: EAPI in header comment *** A different approach would be to specify the EAPI in a specially formatted comment in the ebuild's header. No syntax has been suggested yet, but I believe that the following would work as a specification: - The EAPI must be declared in a special comment in the first line of the ebuild's header, as follows: - The first line of the ebuild must contain the word ebuild, followed by whitespace, followed by the EAPI, followed by end-of-line or whitespace. Again, the proposal comes in two variants: 2a) It is combined with a one time change of the file extension, like .ebuild - .eb. 2b) The usual EAPI assignment statement in the ebuild is still required, at least for a transition period. In the 2a case, the EAPI variable could be made read-only in bash before sourcing the ebuild. In the 2b case, a sanity check similar to the one mentioned above would be added. What do you think? (I really hope for a constructive discussion here. So, if you want to comment that all of the above proposals suck and GLEP 55 is much superior, then please open a new thread for it.) Ulrich [1] http://www.gentoo.org/proj/en/glep/glep-0055.html#problem [2] http://www.gentoo.org/proj/en/council/meeting-logs/20100823-summary.txt [3] https://bugs.gentoo.org/show_bug.cgi?id=402167
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Wed, 7 Mar 2012 21:41:02 +0100 Ulrich Mueller u...@gentoo.org wrote: (I really hope for a constructive discussion here. So, if you want to comment that all of the above proposals suck and GLEP 55 is much superior, then please open a new thread for it.) I think if you want a constructive discussion, then you should ignore the previous Council's decision and reconsider all the solutions that have been presented. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Wed, 7 Mar 2012 21:41:02 +0100 Ulrich Mueller u...@gentoo.org wrote: Hi all, The way how we currently specify the EAPI in ebuilds has some problems. For example, there is no sane way to allow usage of features of a new bash version in a new EAPI. So we are currently stuck with bash 3.2. Also changes of global scope behaviour, like addition of new global scope functions (similar to inherit) are not possible. These flaws are outlined in GLEP 55 [1]: | In order to get the EAPI the package manager needs to source the | ebuild, which itself needs the EAPI in the first place. Otherwise it | imposes a serious limitation, namely every ebuild, using any of the | future EAPIs, will have to be source'able by old package managers | [...] The council has voted down GLEP 55 more than a year ago, but at the same time requested that a solution for the mentioned issues should be found. [2] However, there was no progress since then. The issue arose again in bug 402167 [3] where several solutions have been discussed. Below, I try to summarise the possible options resulting from that discussion. *** Proposal 1: Parse the EAPI assignment statement *** This first proposal would require that the syntax of the EAPI assignment statement in ebuilds matches a well defined regular expression. A scan of the Portage tree shows that the statement only occurs in the following variations (using EAPI 4 as example): EAPI=4 EAPI=4 EAPI='4' Sometimes this is followed by whitespace or a comment (starting with a # sign). Also, with very few exceptions the EAPI assignment occurs within the first few lines of the ebuild. For the vast majority of ebuilds it is in line 5. Written in a more formal way, appropriate for a specification: - Ebuilds must contain at most one EAPI assignment statement. - It must occur within the first N lines of the ebuild (N=10 and N=30 have been suggested). - The statement must match the following regular expression (extended regexp syntax): ^[ \t]*EAPI=([']?)([A-Za-z0-9._+-]*)\1[ \t]*(#.*)?$ Note: The first and the third point are already fulfilled by all ebuilds in the Portage tree. The second point will require very few ebuilds to be changed (9 packages for N=10, or 2 packages for N=30). The package manager would determine the EAPI by parsing the assignment with above regular expression. A sanity check would be added. Citing Zac Medico in [3]: The fact that we can compare the probed EAPI to the actual EAPI variable after the ebuild is sourced seems like a perfect sanity check. We could easily detect inconsistencies and flag such ebuilds as invalid, providing a reliable feedback mechanism to ebuild developers. This proposal comes in two variants: 1a) The change is applied retroactively for all EAPIs. 1b) It is only applied for EAPI 5 and later (which means that the result of the EAPI parsing would be discarded for earlier EAPIs). +1 for the whole proposal, this seems to be solving the problem and is the less intrusive change As i understand it, $PM will need to try the regexp tingy on any ebuild anyway, guess the EAPI then source the ebuild with the right sourcer to get the real EAPI and compare it. So, for me, the proposal is retroactive. The check whether guessed and actual EAPI are the same can be made a warning, or just ignored, in EAPI5 and fatal in =5 though. A.
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On 7 March 2012 21:07, Alexis Ballier aball...@gentoo.org wrote: As i understand it, $PM will need to try the regexp tingy on any ebuild anyway, guess the EAPI then source the ebuild with the right sourcer to get the real EAPI and compare it. Not exactly... the idea with proposal 2) is that the header comment becomes the One True Way to set the EAPI, so it wouldn't be guessing as such, and ebuilds wouldn't be allowed to change it during sourcing any more than they can redefine PV or the like. As mentioned, 2b) still risks a mismatch between the header and the assignment, but hopefully that would be temporary and the assignments could be dropped eventually. (And I'd suggest that the header EAPI is still considered authoritative if present - the mismatch check should probably be a hard error, or if not it should generate a warning and use the header value. There should be minimal if any risk of this changing behaviour for any existing ebuild, since existing ebuilds almost certainly won't match the chosen header syntax.) As for my opinion, I would prefer 2a), or 2b) as a second choice. IMHO it's much simpler and cleaner to come up with a strict, well-defined header format than to try and work within (an arbitrarily restricted version of) bash syntax.
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On 03/07/2012 03:41 PM, Ulrich Mueller wrote: *** Proposal 2: EAPI in header comment *** A different approach would be to specify the EAPI in a specially formatted comment in the ebuild's header. No syntax has been suggested yet, but I believe that the following would work as a specification: - The EAPI must be declared in a special comment in the first line of the ebuild's header, as follows: - The first line of the ebuild must contain the word ebuild, followed by whitespace, followed by the EAPI, followed by end-of-line or whitespace. Someone suggested using a standard shebang the last time this came up, and if I remember correctly it was one of the least-disagreeable solutions proposed. We could of course define our own custom format, but I think something like, #!/usr/bin/eapi5 would be perfect if we could hand off the interpretation of the ebuild to that program. That solves the problem with new bash features, too, since you could point that command at a specific version.
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Wed, 2012-03-07 at 21:41 +0100, Ulrich Mueller wrote: .ebuild - .eb FYI, any Russian speaker is *guaranteed* to read the name .eb as a very common obscenity.
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Wed, 07 Mar 2012, Michael Orlitzky wrote: Someone suggested using a standard shebang the last time this came up, and if I remember correctly it was one of the least-disagreeable solutions proposed. We could of course define our own custom format, but I think something like, #!/usr/bin/eapi5 would be perfect if we could hand off the interpretation of the ebuild to that program. That solves the problem with new bash features, too, since you could point that command at a specific version. Please note that I've formulated the specification in a way that accounts for this: - The first line of the ebuild must contain the word ebuild, followed by whitespace, followed by the EAPI, followed by end-of-line or whitespace. This would also match the line #!/usr/bin/ebuild 5 (if the magic word should be ebuild or eapi can be discussed of course). So even if we would introduce the shebang only in a later EAPI, previous portage versions would still recognise the new header. Ulrich
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Wed, 07 Mar 2012 17:36:05 -0500 Alexandre Rostovtsev tetrom...@gentoo.org wrote: FYI, any Russian speaker is *guaranteed* to read the name .eb as a very common obscenity. In Dutch it means the low tide, and as a verb, it means becoming low or decreasing as in the tide or some other fluid. In English it means something very similar as in the Dutch verb, only without the tidal reference. Other Germanic languages will probably have their own variants. In Crimean Tatar it apparently[1] references easiness or simplicity, in Hungarian it's the word for dog, while in Nauruan it means country. But that's not important right now (see also [2]). The important thing to remember is, the bike shed protects our bicycles from the elements. jer [1] http://en.wiktionary.org/wiki/eb [2] http://en.wikipedia.org/wiki/bytesex
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Wed, Mar 7, 2012 at 12:41 PM, Ulrich Mueller u...@gentoo.org wrote:
Hi all,
The way how we currently specify the EAPI in ebuilds has some
problems. For example, there is no sane way to allow usage of features
of a new bash version in a new EAPI. So we are currently stuck with
bash 3.2. Also changes of global scope behaviour, like addition of new
global scope functions (similar to inherit) are not possible.
These flaws are outlined in GLEP 55 [1]:
| In order to get the EAPI the package manager needs to source the
| ebuild, which itself needs the EAPI in the first place. Otherwise it
| imposes a serious limitation, namely every ebuild, using any of the
| future EAPIs, will have to be source'able by old package managers
| [...]
The council has voted down GLEP 55 more than a year ago, but at the
same time requested that a solution for the mentioned issues should be
found. [2] However, there was no progress since then.
The issue arose again in bug 402167 [3] where several solutions have
been discussed. Below, I try to summarise the possible options
resulting from that discussion.
*** Proposal 1: Parse the EAPI assignment statement ***
This first proposal would require that the syntax of the EAPI
assignment statement in ebuilds matches a well defined regular
expression. A scan of the Portage tree shows that the statement only
occurs in the following variations (using EAPI 4 as example):
EAPI=4
EAPI=4
EAPI='4'
Sometimes this is followed by whitespace or a comment (starting with
a # sign). Also, with very few exceptions the EAPI assignment occurs
within the first few lines of the ebuild. For the vast majority of
ebuilds it is in line 5.
Written in a more formal way, appropriate for a specification:
- Ebuilds must contain at most one EAPI assignment statement.
- It must occur within the first N lines of the ebuild (N=10 and N=30
have been suggested).
- The statement must match the following regular expression (extended
regexp syntax):
^[ \t]*EAPI=([']?)([A-Za-z0-9._+-]*)\1[ \t]*(#.*)?$
Note: The first and the third point are already fulfilled by all
ebuilds in the Portage tree. The second point will require very few
ebuilds to be changed (9 packages for N=10, or 2 packages for N=30).
The package manager would determine the EAPI by parsing the assignment
with above regular expression. A sanity check would be added. Citing
Zac Medico in [3]: The fact that we can compare the probed EAPI to
the actual EAPI variable after the ebuild is sourced seems like a
perfect sanity check. We could easily detect inconsistencies and flag
such ebuilds as invalid, providing a reliable feedback mechanism to
ebuild developers.
This proposal comes in two variants:
1a) The change is applied retroactively for all EAPIs.
1b) It is only applied for EAPI 5 and later (which means that the
result of the EAPI parsing would be discarded for earlier EAPIs).
I don't like this idea because the sane way should be easy and straightforward.
Mixing a constant declaration with bash assignment just confuses users
who think the assignment is full bash when in fact it is not.
EAPI=$(somefunc)
EAPI=${SOMEVAR%%-*}
and so forth all don't meet the regex (and would be flagged invalid.)
However a naive author might think they work.
I don't think any naive author would think either would work in a comment
# EAPI=$(somefunc)
# EAPI=${SOEMVAR%%-*}
Bash doesn't parse comments, so confusion is unlikely.
*** Proposal 2: EAPI in header comment ***
A different approach would be to specify the EAPI in a specially
formatted comment in the ebuild's header. No syntax has been suggested
yet, but I believe that the following would work as a specification:
- The EAPI must be declared in a special comment in the first line of
the ebuild's header, as follows:
- The first line of the ebuild must contain the word ebuild,
followed by whitespace, followed by the EAPI, followed by
end-of-line or whitespace.
Again, the proposal comes in two variants:
2a) It is combined with a one time change of the file extension, like
.ebuild - .eb.
2b) The usual EAPI assignment statement in the ebuild is still
required, at least for a transition period.
In the 2a case, the EAPI variable could be made read-only in bash
before sourcing the ebuild. In the 2b case, a sanity check similar to
the one mentioned above would be added.
What do you think?
Overloading is bad.
There is no real difference between:
#!/usr/bin/ebuild --eapi 5
# EAPI=5
The problem is that the former is also the way to specify how to
execute the ebuild; so unless you plan to make ebuilds executable and
having /usr/bin/ebuild provide the ebuild environment, using that
syntax is confusing to users.
(I really hope for a constructive discussion here. So, if you want
to comment that all of the above proposals suck and GLEP 55 is much
superior, then please open a new thread for it.)
You don't mention voting on
[gentoo-dev] Re: [gentoo-dev-announce] Submit project ideas NOW for Google Summer of Code 2012
On Wed, Mar 07, 2012 at 11:38:47PM -0600, Donnie Berkholz wrote: http://wiki.gentoo.org/wiki/Google_Summer_of_Code/2012/Ideas. If you have project ideas, this is the place to put them. It's critical to also include potential mentors and prerequisite skills so students know which ideas make sense and where to learn more about them. For any devs with ideas from the previous years that didn't get chosen, the old page is here: http://www.gentoo.org/proj/en/userrel/soc/ideas.xml I'm not going to be putting my Upstart idea forward again, due to lack of time, and I don't think it's quite suited. Somebody else might want to look at the potential of porting OpenRC's oldnet into systemd however... -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] RFD: EAPI specification in ebuilds
On Wed, 7 Mar 2012, Alec Warner wrote:
*** Proposal 1: Parse the EAPI assignment statement ***
[...]
I don't like this idea because the sane way should be easy and
straightforward. Mixing a constant declaration with bash assignment
just confuses users who think the assignment is full bash when in
fact it is not.
EAPI=$(somefunc)
EAPI=${SOMEVAR%%-*}
and so forth all don't meet the regex (and would be flagged
invalid.) However a naive author might think they work.
Such constructs also cannot be used with any of the other proposed
solutions. And in fact, nobody is using such things in practice.
_All_ ebuilds in the Portage tree can be successfully parsed with the
regexp proposed.
The obvious sanity check, i.e. comparing the EAPI obtained by parsing
and by sourcing, could be added to repoman, which would prevent such
non-conforming ebuilds from being committed to the tree.
*** Proposal 2: EAPI in header comment ***
[...]
Overloading is bad.
There is no real difference between:
#!/usr/bin/ebuild --eapi 5
# EAPI=5
The problem is that the former is also the way to specify how to
execute the ebuild; so unless you plan to make ebuilds executable and
having /usr/bin/ebuild provide the ebuild environment, using that
syntax is confusing to users.
I agree with this point.
Many file formats are identifying themselves with a magic token (as
it is used by sys-apps/file), but it is not necessarily a shebang.
Ulrich
Re: [gentoo-dev] Re: [gentoo-dev-announce] Submit project ideas NOW for Google Summer of Code 2012
I am not a developer yet, but I would like to suggest some idea possibilities: Minix port of Gentoo Illumos port of Gentoo LLVM/Clang System Compiler Support ICC System Compiler Support (probably easier than LLVM/Clang) Port of Gentoo/FreeBSD to amd64 (or other architectures) Gentoo/FreeBSD KVM port (we don't have to let upstream have all of the fun) Gentoo Prefix --as-needed support on one or more architectures that currently lack it Gentoo Prefix Cygwin support Gentoo Prefix Minix support I might be too inexperienced to serve as a mentor this summer, provided that the recruitment process finishes in time. I would appreciate it if anyone interested in being a mentor for any of these ideas adopted them. On Thu, Mar 8, 2012 at 2:03 AM, Robin H. Johnson robb...@gentoo.org wrote: On Wed, Mar 07, 2012 at 11:38:47PM -0600, Donnie Berkholz wrote: http://wiki.gentoo.org/wiki/Google_Summer_of_Code/2012/Ideas. If you have project ideas, this is the place to put them. It's critical to also include potential mentors and prerequisite skills so students know which ideas make sense and where to learn more about them. For any devs with ideas from the previous years that didn't get chosen, the old page is here: http://www.gentoo.org/proj/en/userrel/soc/ideas.xml I'm not going to be putting my Upstart idea forward again, due to lack of time, and I don't think it's quite suited. Somebody else might want to look at the potential of porting OpenRC's oldnet into systemd however... -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
