[gentoo-dev] Re: Last rites: user.eclass
Mike Gilbert wrote: > On Wed, Oct 19, 2022 at 3:08 PM Martin Vaeth wrote: >> >> Mike Gilbert wrote: >> > user.eclass [...] >> >> It is needed for ebuilds in non-gentoo repositories which cannot >> reserve a fixed number for users and groups. > > If you set ACCT_USER_ID=-1 or ACCT_GROUP_ID=-1, they > will pick an available UID/GID upon installation. I was not aware of that. Indeed, it was probably added after I looked at the documentation last time: https://github.com/gentoo-mirror/gentoo/commit/ c871b87b56208684b0a1fedeaf24fe10943748e2 Sorry for the noise.
[gentoo-dev] Re: Last rites: user.eclass
Mike Gilbert wrote: > user.eclass has been deprecated for two years. In the gentoo repo, it > is currently only used by acct-group.eclass and acct-user.eclass It is needed for ebuilds in non-gentoo repositories which cannot reserve a fixed number for users and groups.
[gentoo-dev] Re: Please port your packages to Python 3.8
Martin Vaeth wrote: > >> Even if I believe in a metadata angel and if we pretend that the PMS >> requires the metadata to be there, then rebuilding whenever metadata >> changes is still not 100% correct (as you point out), because it often >> rebuilds pointlessly. But that's getting into a harder problem. Oh, I think I misunderstood you here. If the PM would always "prefer" the repository's metadata (if available) over the installed metadata, it would not be necessary to rebuild packages only because the metadata has changed (or, alternatively, portage could just update the installed metadata in such cases). A "forced" rebuild would then only be necessary in special situations, e.g. if a subslot dependency resolves differently. That's why prefering repository metadata over installed metadata requires some "smartness" of the package manager; there are still several corner cases where it is a political decision whether to rebuild. Currently, portage has this smartness only partially (subslot resolving does not work), and portage has no mechanism to just update installed metadata without recompilation.
[gentoo-dev] Re: Please port your packages to Python 3.8
Michael Orlitzky wrote:
>
> Who generates the metadata when I `git pull`?
For the gentoo repository, it is in general some gentoo server
which then pushes the calculated metadata to the repository which
you pull as a user.
It is *possible* to use the "plain" repository, but you have to
set up quite a bit for that (updating metadata is only one of
several steps which you have to do manually in that case).
A collection of scripts which do the missing steps is
https://github.com/vaeth/portage-postsyncd-mv
though I do not know whether they still work. Indeed (as
probably most users) I am using since years one of the
repositories with generated metadata already.
> Or in overlays?
There are overlays which provide metadata, and there are overlays
where you have to do it on your own with egencache. Overlays which
use eclasses from the gentoo repository usually do the latter,
since otherwise the metadata is soon out-of-date.
However, for most overlays, egencache takes far less than a minute,
so for overlays, this is not really an issue. For the gentoo repository,
the time of a full metadata generation is considerable. As mentioned,
checksums and timestamps are used to minimize the amount, though this
does not work for eclass changes (see below).
> but if you record that wasted time in a different "metadata generation"
> ledger, then it looks like I've saved a bunch of time because the "bash
> sourcing" ledger reads zero.
No. You need correct metadata after every syncing of the repository.
Either the gentoo server or your machine has to do it.
This is independent of whether the PM "prefers" the installed or
the repositories' metadata. Excluding installed packages from metadata
updating would not be sane and would not safe much time (since the vast
majority of ebuilds are not installed, anyway).
> Even if I believe in a metadata angel and if we pretend that the PMS
> requires the metadata to be there, then rebuilding whenever metadata
> changes is still not 100% correct (as you point out), because it often
> rebuilds pointlessly. But that's getting into a harder problem.
Yes, usually the metadata rebuilds due to eclass changes are pointless
(except in the few cases where the eclass change is related with the
metadata).
I remember when I used to sync from the "plain" repository and rebuilt
the metadata on my system, that the syncing (i.e. metadata regeneration)
costed 10-30 minutes whenever one of the basic eclasses (which are
sourced by almost every ebuild) had a trivial change. Probably,
meanwhile machines are slightly faster and there are less such "basic"
eclasses needed in newer EAPIs, but it will still need a considerable
time.
> The recompilation isn't always pointless. In the present scenario, the
> rebuild is what installs the python-3.8 copy of the program.
No. If users use the defaults for PYTHON_{,SINGLE}_TARGET, the
rebuild has absolutely no effect except for changing some metadata
in /var/db/pkg (and some file timestamps).
> I'm not arguing that this is the best way to do things, but that it's
> the only way to do them given what's in the PMS today.
Of course. That was the decision made some years ago.
> maybe it should pay a few people to sit around and think about
> this for a week.
There was such a debate some years ago, before the decision mentioned
above was made.
It was a hefty discussion, but there were strong proponents of pointless
recompilation vs. improvement of the dependency handling (for both
sides, there are much more arguments which I will not repeat now).
The discussion might have turned if I would have found the time to
implement the necessary change in portage, but neither then nor now
I have the time to do so. (To be honest: Maybe I had the time, but I
dislike python too much.) Nevertheless, I do not think that it was a
good decision.
I am not posting this to re-roll the above discussion again.
But your posting shows that apparently not everybody knows the reasons
why things are the way they are now.
[gentoo-dev] Re: Please port your packages to Python 3.8
Michael Orlitzky wrote: > What's happening is that the PM is using the metadata from the installed > version of the package, rather than the ninja-edited metadata in the > repo (how would it know which ebuilds were edited meaningfully?). The question is easy to answer: It is reasonable to assume that the repository metadata is correct and more up-to-date than the metadata of the installed version. But the PM would need to be smart enough to interpret subslot-deps correctly and merge that from the data of installed packages in a clever way. > That's completely legal according to the PMS, and also the > smart thing to do: s/smart/dumb/, but necessary for a dumb PM. > sourcing a few thousand lines of bash *just in case* there was an > important change in some ebuild is a huge waste of users' time. That's wrong: The metadata has to be (re)generated anyway, independent of whether this is a revision bump or just a modification. For most syncing methods the bash sourcing does not happen on the user's machine, and on those machines where the metadata is actually calculated, there are checksums and timestamps used to minimize the effort. From this perspective there is no difference between a modification or a revision bump. > Developers have an easy way to signal that there was an important > change: to bump the "r" number at the end of the ebuild. That's why it was decided to do it this way instead of requiring a smarter PM. Instead of putting some smart logic into the PM (which in case of portage was not fully implemented and in case of other package managers not at all), the decision was to force recompilations for every user and every tiny dependency change. > One developer makes a change, and it saves > thousands of users each a lot of time. ... costs thousands of users a lot of compilation time, although the recompilation is completely pointless for each of them, but done only to keep the PM simpler.
[gentoo-dev] Re: mcrypt status
Andrew Savchenko wrote: > > Actually for local symmetric encryption this is the best tool I > know. What is the advantage over gpg --symmetric?
[gentoo-dev] Re: rfc: why are we still distributing the portage tree via rsync?
Matt Turner wrote: > The ebuild tree is 600MB with rsync and cannot fit on the partition > with git. > > I'd be happy to switch if the space requirements were similar. If one git repacks every few syncs one needs currently about 800 MB. With additionally squashfs (zstd) (+ overlayfs) the full archive size is currently <600 MB. In both cases, the temporary disk space is slightly more, of course. For a 1GB reserved partition I'd use the partition for the temporary mounting and store the archive somewhere else, but I think chances are good that you also come through with only a git repack after every sync. A difficulty might be the very first git sync.
[gentoo-dev] Re: Current status with openssl-1.1
Lars Wendler wrote: > So, basically openssl is the last big showstopper for openssl-1.1 to > get out of p.mask. s/openssl/openssh/ Another showstopper is net-libs/wvstreams, hence net-dialup/wvdial. BTW, this is a Debian bug open without any comment since April 2017: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859791 Debians and Archs workarounds can be called slotting of openssl:1.0 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=828603 While many of the new opaqueness problems are easy to fix, some functions of wvstreams rely so much on internals which are inaccessible with openssl-1.1 that they seem impossible to convert. My guess (I am not an openssl guru) is that the whole wvstreams implementation would need to change, maybe also the provided API with some functionality being dropped or attacked completely differently. The uncommented bugreports and missing PRs suggest that this will never happen by upstream, and there also seems to be no fork with it. Maybe for wvstreams a workaround might be to mask/remove USE=ssl: Perhaps wvdial would then loose some functionality (does anybody know what would break?) but given how useful wvdial is, this is better than dropping it completely.
[gentoo-dev] Re: [RFC] multiversion ebuilds
R0b0t1 wrote: > On Tue, May 15, 2018 at 11:15 PM, Duncan <1i5t5.dun...@cox.net> wrote: >> >> AFAIK symlinks aren't allowed in the gentoo tree [...] >> >> Tho perhaps that can be reevaluated. [...] > > Cygwin and MSYS(2) are currently mostly supported by Prefix [...] For rsync users, the non-symlink policy is not necessary: They could simply pass the option --copy-links to get a working tree (of course, correspondingly longer). For git users (essential for overlays!), I do not know a simple way. It might be possible to set up appropriate git hooks, but that might be tricky: The hooks would need to apply before the files are actually checked out, and it might become even more tricky if one does not want that subsequent git commands do consider the transformation as a local modification. For git perhaps an opposite approach might be easier: One could provide hooks which every git user is supposed to use which manage symlinks and/or copies. More precisely, these hooks should have a symlink and a non-symlink mode (depending on whether the underlying (file)system can generate symlinks which can easily and quickly be detected), and the tree would need to contain a symlinks.dat file with a list of symlinks, and a .gitignore containing the symlinks. During checkout, the hooks should simply generate the corresponding symlinks/files (depending on symlinks.dat) according to the stored list of symlinks. During checkin in symlink mode, these hooks automatically update symlinks.dat and .gitignore so that most developers simply can use/modify symlinks transparently. During checkin in non-symlink mode the hook does nothing: Developers who checkin on filesystems without symlink have to update these 2 files manually, of course, when they want to create/remove/rename a symlink. Maybe one could provide corresponding helper scripts for those developers with non-symlink-aware filesystems if necessary.
[gentoo-dev] Re: Mailing list moderation and community openness
Rich Freeman wrote: > > Certainly. Closing lists won't stop the private abuse, nor is it intended to. > > What it would stop is this particular thread talking endlessly about it. > >> Closing a mailing list >> will not close such a debate; it will then just happen elsewhere. > > And that is the goal. So now we finally get to the point: The whole story has actually *nothing* to with Fred. It is about what I said in the very first posting: It is an attempt to suppress opinions, by taking away people an important channel to raise their voice. The whole Fred example was only a rhetorical trick: An attempt to find at least *one* example where you believe that the developers' opinion is undoubtfully the right one, an attempt to justify the ivory tower. This one example - it plays no role whether it is justified or whether there is another one - is completely suppressing the fact that in almost all cases on dev-ml (trivial "ACK" things aside) *are* clearly discussable (concerning technical topics) and *should* be discussed. In fact, all these *other* discussions are the actual purpose of dev-ml. Closing the channel simply excludes non-developers from these discussions dev-ml is made for. Concering Gentoo's reputation, you can be sure that this step will be only contraproductive: - In Fred's case anyway, because people with the opinion that something strange is going on with this case will see their opinion just confirmed; outsiders anyway. - For people not involved or not interested in Fred's case it is clearly even worse. From the outsider viewpoint as well. This closing harms Gentoo a lot: I am driven away from Gentoo by such an undemocratic step. Certainly I am not the only one: Others also already formulated similar opinions on this and the project mailing list, at least if you are able to read between the lines. > Could you take this debate to the appropriate place then? Do not worry, this is presumably my last post on the topic (soon I would not be able to post, anyway). I am aware that the undemocratic decision has already been made (BTW unsurprisingly in a not very democratic way), so it makes no sense to discuss about it further. My post was just a final attempt at least to mitigate the damage done by this decision by speaking for the only thing which can still be done purely technically: Blacklisting instead of whitelisting. With whitelisting you will only attract that type of non-developers who are willing to beg a gang to be a member of them. Of course, if a secondary aim should be to get only uncritical followers (or pretenders) and to drive away everybody else, whitelisting is the correct choice.
[gentoo-dev] Re: Mailing list moderation and community openness
Rich Freeman wrote: > > Fred is a community member. Fred consistently harasses and trolls new > contributors in private. Sure, it's a problem. But not a problem which can be solved by closing the mailing list, in no step of the issue. First of all, this happens in private, so you cannot prevent it by closing a mailing list. > No mention is made of why Fred as booted out, because everything > happened in private. That's the mistake which is made in this example. Be open in the decisions. If you cannot be open in order to protect other people's privacy, be open at least by saying exactly this. > Now a bunch of community members get upset about Fred being booted out > without reason. Fred claims it is because he disagrees with the > leadership on something. People start arguing endlessly about > openness. Yes, this might happen due to the non-openness. This might happen even if you are open. And nothing will prevent it. Closing a mailing list will not close such a debate; it will then just happen elsewhere. Anyway, such a debate does not belong to dev-ml. The correct solution is to continue to point people to have this debate on the appropriate place, not on the mainly technically oriented dev-ml. Making the posters silent by blacklisting even more is contra-productive and will give the impression that they are actually right. As it is a commonplace: You cannot solve social problems by technical measurements. > Ultimately the leaders just want Fred gone so that new contributors > aren't getting driven away. They can't explain that because then they > create potential civil liability for the project. Why not? Is it against a law to exclude somebody who is hurting a project? If it is (or if there is a danger that it is), then the problem is not that they cannot explain it but that they must not do it in the first place. In any case, this is a different problem and cannot be solved by closing a mailing list. > The problem is that > the debate goes on for over a year despite intervening elections and > now this becomes the issue that is driving new contributors away. > What solution would you propose for this problem? How would closing the mailing list solve the problem? It will give the impression that you want to close the debate by taking away the medium where people can argue. And the impression is correct, because this actually *is* the intention if you are honest. Of course, it will not close said debate. The debate will just happen on another channel. (Which in this example might be appropriate, but pointing to the proper channel is what should have happened and not closing a mailing list and thus excluding random people from posting things about clompletely different topics which *are* on-topic on dev-ml). > Sure, but we can at least force the negative advertising of Gentoo to > go elsewhere, rather than basically paying to run a negative PR > campaign against ourselves. Closing dev-ml will not help here. If people have a strong disagreement with a decision, this will happen on gentoo channels. If you want to prevent it technically, you have to close all channels. > And what about the freedom to endlessly troll and harass you and > others? [...] Closing a mailing list will not prevent this. Somebody who behaves this way (or feels being treated wrong) will not stop this only because one channel is closed for him. What is really happening by closing the mailing list is that you stop innocent contributors. In any case, that's the discussion blacklisting vs. whitelisting: To stop one specific single poster, blacklisting is enough, at least for the beginning. Sure, technically it can be circumvented, but you will not stop this social problem anyway by technical means. > Surely Gentoo's mission isn't to run completely unrestricated forums > for discussion of anything and everything. Our main purpose here is > to maintain a Linux distro, not provide a platform for anybody who has > an opinion on anything. Sure, pointing to the right channel is appropriate. This is something completely else than to prevent posting *by default*. > without being endlessly trolled and harassed. This is unrelated about closing the mailing list. Especially if this happened in private, anyway. BTW, I do not think that contributors are that blue-eyed that they will stop contributing only because one person does not know how to behave. Especially if it is made clear somewhere that this happens in disagreement with gentoo as a whole. *This* might be a way how one might react to such a problem. Anyway, this discussion now is getting off-topic: All these problems have nothing to do with closing a ml and cannot be solved by this.
[gentoo-dev] Re: Mailing list moderation and community openness
Rich Freeman wrote: > On Tue, Mar 27, 2018 at 3:34 AM, Martin Vaeth wrote: >> >> It is about openness vs. isolation. > > I'm pretty sure most developers, myself included, want to welcome > contributions. Closing of the mailing list does not sound like that. > Much of the concern is that the lists have been turning into endless > arguing over things like very topic. Yes. Some people could not be stopped from continuously expressing their opinion. Some developers do not want to hear them. So the list is being closed. > If a newcomer comes along and reads your post, they're going to get > the impression that the developers live in an ivory tower. IMHO this impression is completely right. > Why would somebody want to > contribute to Gentoo in the first place if that is their first > impression? Exactly. This is why closing the list is the absolutely wrong signal. Sticking at least to a blacklist-only mode might mitigate the IMHO severe damage which has already happened by the decision to close the list. > The problem is when it turns into a personal attack or > hyperbole, which IMO the part I quoted falls into. Personal animosities are always a problem. This can and will not be solved by technical measurements. Taking all non-developers as hostages - including those which were not involved at all in the debate and even worse even possible future contributors - is certainly not a solution for that type of problem. And anyway, you can be sure that the problem will appear again, no matter how closed the list will be. > The intent isn't to stifle debate/discussion. But this is exactly what is happening by closing the list to non-developers. > A lot of this comes down to considering that most people in these > debates probably are well-intended. Taking away freedom is never justified by good intention.
[gentoo-dev] Re: Mailing list moderation and community openness
Rich Freeman wrote: > On Tue, Mar 20, 2018 at 7:54 PM, Benda Xu wrote: >> >> It boils down to an attitude of assuming outsiders are good (blacklist >> to ML) or bad (whitelist to ML) by default. ++ This is the most crucial point. It is the general attitude: Does Gentoo welcome contributions or want to make their developers live in an ivory tower? It is about openness vs. isolation. > It is > basically impossible to blacklist somebody on a mailing list, since > all they need to do is roll up a new email address. That may be technically true but it is not a problem the mailing list is currently facing. If it should eventually happen to be the case that the mailing list is filled by tons of spam of anonymous posters or faked identities causing serious problems, one can still think about changing the modus operandi. But without such an absolute need, it is very bad to restrict freedom.
[gentoo-dev] Re: How to deal with git sources?
Ulrich Mueller wrote: > > I think the conclusion is that github generates tarballs on the fly, > and therefore we cannot rely on them being invariant over a long time. Yes, but with emphasis on _long_ time and theory. In practice this was happening now exactly _once_ in a decade (according to all we learnt so far) for the understandable reason of fixing an annoying incompatibility in an exotic case. And the existence of zopfli shows that other backward-compatible improvements _would_ have been possible, but apparently non-changing of the produced tarball was always rated higher than anything else (up to this exception). So I would not worry too much about it: It is not worth the cost of hosting a huge number of tarballs permanently (or to convince upstream to let them be hosted by github for every single version, only because one cannot theoretically exclude that a similar thing won't ever happen again). Yes, for the transition period (until all github servers use a new enough version) a solution for the few involved tarballs has to be found (like temporarily hosting on devspace). But after this period it is only a question of updating the checksum once for the involved packages.
[gentoo-dev] Re: How to deal with git sources?
NP-Hardass wrote: > > IIRC, it was attributed to > https://github.com/git/git/commit/22f0dcd9634a818a0c83f23ea1a48f2d620c0546 Thanks. That explains why I was not able to produce a difference: It involves only the rather exotic case that a path in a git repository is longer than 100 characters.
[gentoo-dev] Re: How to deal with git sources?
Vadim A. Misbakh-Soloviov wrote: > > GH support answered me (in TL;DR version) "that's because we've upgraded git > on *some* of our nodes" (means, some other using older git) That would still require that the "git archive" output would have changed in some recent git versions. And at least between the most current 2.16.2 and comparing with all my git tarballs (some as mentioned rather old), I could not produce any difference. So I still do not understand what should be going on.
[gentoo-dev] Re: How to deal with git sources?
Michael Orlitzky wrote: > On 03/12/2018 04:29 AM, Martin Vaeth wrote: >> This was so many many years ago in the beginning of github. >> This has long been fixed since then. > > Every once in a while they still change. This is from a few weeks ago: > > https://marc.info/?l=openbsd-ports&m=151973450514279&w=2 As mentioned, github uses "git archive" to generate the tarballs. So - theoretically - if a new version of git should have "git archive" implemented differently (which might happen indirectly by new versions of tar/zlib), there might indeed be change. However, this is purely theoretical: I just upgrade to most current git-2.16.2 and tar-1.30, and checked: The generated tarballs still match with that from 2014. So I really do not know which change "a few months ago" the above message refers to: I could not detect any change (and I really checked a lot of packages now). Perhaps they refer to .zip instead of .tar.gz which as mentioned is a less stable format due to the inclusion of the timezone.
[gentoo-dev] Re: How to deal with git sources?
Duncan <1i5t5.dun...@cox.net> wrote: > > If I'm recalling correctly a warning posted on this list, repeated calls > to the github tarballing API for the same commit will result in delivery > of tarballs with differing checksums. This was so many many years ago in the beginning of github. This has long been fixed since then. Actually, github is just using "git archive" which produces always the same tarball. Only possible exception is if you use .zip instead of .tar.gz because the former (due to FAT compatibility in the format) has to store the timezone. So if the timezone of the github server ever changes, the .zip "tar"ball might also change.
[gentoo-dev] Re: SAT-based dependency solver: request for test cases
Michael Lienhardt wrote:
> the criteria list you gave (maybe it's in the PMS)
I doubt that it is in PMS, and IMHO it also does not belong there:
As long as the result configuration is valid (no collisions
or unresolvable loops) all should be equally fine from the
viewpoint of PMS: It is in the responsibility of the package
manager and its configuration to determine correctly what the
_user_ actually wants.
> - in criteria a., there could be an ambiguity between what I call a package
> (e.g., 'app-editors/nano-2.9.3') and a package group (e.g.,
> 'app-editors/nano')
This is what already Michał has oberved: In my list, I had completely
forgotten to refer to versions ('app-editors/nano-2.9.3') and had only
packages ('app-editors/nano') in mind.
(I think "version" vs. "package" is the usual terminology in gentoo;
there are also "slots" which in a sense might be considered as
different packages, although perhaps with a different "penalty")
> is the criteria about package group (i.e., are version changes allowed)?
Version upgrades should even be preferred over remaining with the version.
OTOH, version downgrades are perhaps even worse than using a different package
(opinions might differ for the latter).
> - similarly in criteria b.: is this criteria valid across versions
> (i.e., when changing version, the USE-flag change should be minimal)
Cf. the next point: The USE-flag change compared to user configuration
(package.use etc.) should be minimal.
> - just to be sure, the "less keyword and mask change" criteria is at the
> top of the list?
Yes. Only "illegal" configurations (colliding packages or unresolvable
dependency loops of uninstalled packages) should weight stronger:
If possible at all, the user's choice should always be preferred,
and changes to the configuration should only be suggested if no other
solution exists (and probably the number of suggested changes should
be minimal in a sense).
[gentoo-dev] Re: SAT-based dependency solver: request for test cases
Michał Górny wrote:
>>
>> d. In || ( ... ) clauses the left-most packages should be preserved.
s/preserved/preferred/
> you've missed the most important point: we want to prefer
> the newest version, whenever possible
> ;-).
Yes, you are right: I had thought only about packages, not about versions.
Of course, a version upgrade within the same slot should have a
negative penalty. I am already not sure how upgrades with slot
changes should be treated. And then there are subslots...
The list is probably still not exhaustive and - as already mentioned -
the penalties are certainly a topic of discussion (and even more of
trial and error: which works best in practice).
Anyway, I think that it would be a huge improvement over the current
(portage's) solver if one could formulate such a list explicitly in
some specified format, and then one abstract algorithm takes care to
find the best solution according to the specified penalties:
If I understand it correctly, all these rules are currently hard-coded
into the algorithm by using various hacks in backtracking steps,
finding of a solution is convoluted with determining the order, etc.
One would obtain a solver which not only is "provably" correct,
but also much easier to maintain and to tweak (e.g. if one realizes
that certain penalties were not cleverly chosen).
This would also provide the possibility for much richer user configuration.
An example which immediately come to mind is "weak-masking" of packages
or versions ("use it or upgrade only if no alternative is available").
Theoretically, one could then even endow the entries of package.mask
with a penalty number.
[gentoo-dev] Re: SAT-based dependency solver: request for test cases
Michael Lienhardt wrote: > > ad-hoc fixes and tweaks that can hardly be encoded into SAT constraints. The main difficulty which I see is that one does not want only _some_ solution, but among all solutions one which optimizes certain quantities. So it seems to me that a discrete optimization under constraints is required instead of a pure SAT solver (although formally, of course, such an optimization problem can be reduced to SAT, I do not know whether you went the road): a. The number of packages which change their status (from installed to uninstalled or vice versa) should be minimal. b. Similarly, the number of USE-flag changes necessary to achieve this aim should be minimized. (You didn't tell whether your solver already supports such changes, but when it is finished, it definitely should.) Hopefully in near future, there will be a second class of USE-flags whose change is "cheap" which should be excluded from this minimization restriction: https://bugs.gentoo.org/show_bug.cgi?id=424283 I think the main reason why nobody dared to implement them yet (although almost everybody wants them) are exactly these implications on the current solver in portage which nobody dares to change anymore seriously. c. A solution with dependency loops should be avoided if possible (which is why currently the PDEPEND hacks do exist: To tell the solver which loops are not a problem.) d. In || ( ... ) clauses the left-most packages should be preserved. There are similar (and more difficult) rules for REQUIRED_USE. e. Last but not least: The preferences are ordered a > b > c > d (A dependency loop of uninstalled packages should probably have even higher priority than a). Additionally the change installed -> uninstalled should be less "expensive" than the change uninstalled -> installed. The correct weighting of the quantities should probably be a matter of discussion (or preferrably even user-customizable), and preferrably should not depend only on the number of packages but also on other customizable quantities (e.g. the package size). Perhaps - if one can rely on some solver - in a future EAPI instead of the size heuristic, one could give a hint for how "expensive" it is to recompile a certain package, so that dependency results can be better optimized for that. IIRC, this is already built in rudimentarily in portage's current solver by defining the recompilation costs of virtual packages as 0. > I don't deal with installation order [...] circular dependency problem ++ Once the packages are known, the installation order and breaking of unavoidable circles is a matter of a single graph traversal in the resulting subgraph which needs neglectable linear time. However, if there is a solution without a circle this should have been found instead in the first place (cf. c).
[gentoo-dev] Re: News Item: Portage Dynamic Deps
Ulrich Mueller wrote: > This is an OpenPGP/MIME signed message (RFC 2440 and 3156) > --pgp+signed++Zg8D+I6sgRUw0D > Content-Type: text/plain; charset=us-ascii > Content-Transfer-Encoding: 7bit > >>>>>> On Wed, 24 Jan 2018, Martin Vaeth wrote: > >> Rich Freeman wrote: >>> >>> It would already be broken on any PMS-compliant package manager > >> No, it is solely the interpretation of the package manager. >> PMS does not specify what information is stored in /var/db >> or how that information is used. > > "Runtime dependencies (RDEPEND). These must be installed and usable > before the results of an ebuild merging are treated as usable." > https://projects.gentoo.org/pms/6/pms.html#x1-770008.1 > > IMHO this implies that the dependencies at merge time are the relevant > ones IMHO this specifies what is relevant when an emerge merging happens. Nothing more, nothing less. _If_ one would be willing to interpret it to have a meaning also _after_ the emerge, then of course the RDEPEND in PMS can refer to the only value which is specified in PMS, i.e. that stored in the ebuild (and not in any database which is explicitly not specified by PMS). In other words: _If_ one puts any unsaid interpretation into that sentence, then this can only be dynamic deps.
[gentoo-dev] Re: News Item: Portage Dynamic Deps
Rich Freeman wrote: > > It would already be broken on any PMS-compliant package manager No, it is solely the interpretation of the package manager. PMS does not specify what information is stored in /var/db or how that information is used.
[gentoo-dev] Re: [PATCH] linux-mod.eclass: IUSE default support for MODULES_OPTIONAL_USE
Robin H. Johnson wrote:
> That is counter-intuitive for somebody that puts
> MODULES_OPTIONAL_USE_IUSE_DEFAULT=0
> Or tries to otherwise have it unset.
What I usually do is:
case ${MODULES_OPTIONAL_USE_IUSE_DEFAULT:-n} in
[nNfF]*|[oO][fF]*|0|-) # false case
;;
*) # true case
esac
This covers most of the "intuitive" attempts (0 - Off FALSE no ...)
to switch it off, and everything else (nonempty) means "on".
[gentoo-dev] Re: cmake + ninja vs autotools
Georg Rudoy <0xd34df...@gmail.com> wrote: > 1. Doing a full clean build [..] > the speed of make or ninja is hugely offset by the compilation speed, > and their overhead is negligible. It depends on the definition of negligible. For huge projects like boost or chromium it is several minutes: That's why these projects switched to ninja. Admittedly, for eix the difference is probably only a few seconds on most systems.
[gentoo-dev] Re: cmake + ninja vs autotools
William L. Thomson Jr. wrote:
>
> case ${CMAKE_MAKEFILE_GENERATOR} in
> emake)
> DEPEND="sys-devel/make"
> ;;
> ninja)
> DEPEND="dev-util/ninja"
> ;;
This is broken: Static metadata like DEPEND must not depend
on dynamic data like environment variables which are supposed
to change at emerge time.
> Only 2 thus far does not sound like most things would have issues.
In fact, almost nothing has issues. I am using
CMAKE_MAKEFILE_GENERATOR=ninja in my make.conf
since years, and the total list of packages which had
ever failed here (out of currently ~1500 installed)
is this:
dev-util/cmake
kde-apps/kate
kde-apps/gwenview
media-libs/avidemux-core
media-libs/avidemux-plugins
media-video/avidemux
media-video/kaffeine
media-video/kmplayer
net-vpn/kvpnc
sci-mathematics/reduce
This list might appear long, but note that
1. */avidemux* is essentially only 1 package
2. net-vpn/kvpnc is ancient and masked
3. sci-mathematics/reduce is ancient and masked
and moreover, perhaps some other entries of this list are even
obsolete (I do not have installed KDE 5, so I cannot try quickly;
when I had added the packages to the list, KDE 4 was rather new...)
> Ninja is most of the speed of meson less configure time savings
++
For eix, the main motivation to support meson as an
alternative build system was to be able to use ninja...
[gentoo-dev] Re: An example overlayfs sandbox test
Rich Freeman wrote: >> >> | "simple" | "fine grained" >> -++--- >> Overlay | 1 mount| 1 mount >> -++--- >> Container| 10? bind mounts| 1000? bind mounts > > Except it is more like: > > | "simple" | "fine grained" > -++--- > Overlay | 1 mount| 1 mount + 1000? file deletions in the overlay > -++--- > Container| 1-2 bind mounts| 1000? bind mounts I was not talking about the time to setup the overlay. File deletions involve only the latter. > I left out dev+sys+proc in both cases No, they were not forgotten: They are not necessary for the overlay approach! As I emphasized, you do not even need a single bind for that approach. > And there is really no difference in performance between 1 mount and > 10 in practice. Really? Tested with a few million file creations/deletions/openings etc? Such a number is not unusual for some projects: Already gentoo-sources has ~60k files, all of them being accessed several times in various manner. So even a very small delay multiplies by a huge number. That's also a reason why I mentioned that a slow machine would be good for timing. For instance, gentoo-sources needs several minutes to emerge on a machine with a slow processor and little ram: the harddisk speed is not the reason for the delay. I would not like to see another factor due to a sandbox which is perhaps negligible on a fast system.
[gentoo-dev] Re: An example overlayfs sandbox test
Rich Freeman wrote: >> >> For containers, at least a dozens of binds are minimally required >> (/usr /proc /sys /dev ...). > > I wouldn't be surprised if it works with a single bind mount with > /proc and /dev and so on mounted on top of that. Either you start with a writable tree and bind-mount some directories non-writable or the opposite way. Either way, a dozen or so bind-mounts are minimally necessary. > You say "not even a bind" as if that is a benefit. In case the "non-scaling" argument has not become clear, I try to visualize it by a table: | "simple" | "fine grained" -++--- Overlay | 1 mount| 1 mount -++--- Container| 10? bind mounts| 1000? bind mounts > Honestly, you can't really claim that overlayfs is superior to bind Correct. If the number of bind mounts really has no influence on the file operations in the corresponding part of the tree - e.g. if there is really a clever hashing of bind mounts - the above table does not indicate any scaling problem. We are at a point where some kernel source code inspection (or at the very least serious benchmarking, preferrably with a slow and low-memory machine) is needed before we can continue the discussion in a serious way. I do not have the time for this currently.
[gentoo-dev] Re: An example overlayfs sandbox test
Rich Freeman wrote: > On Sun, Sep 24, 2017 at 4:24 AM, Martin Vaeth wrote: >> Tim Harder wrote: >> >> It is the big advantage of overlay that it is implemented in >> kernel and does not involve any time-consuming checks during >> normal file operations. > > Why would you expect containers to behave any differently? For overlay, there is only one directory to be checked in addition for every file access. For containers, at least a dozens of binds are minimally required (/usr /proc /sys /dev ...). But as you mentioned in your posting, if you want to take more care you easily have thousands of bind mounts. At least implicitly in the kernel, all of these binds must be checked for every file access. I am not sure whether this happens very quickly by hashing (so that essentially really only the creation costs time). As mentioned, I do not have actual timing results. I am just afraid that it might easily cost more than a context-switch which already gives a slowdown for fuse-overlay which is so large that I would not recommend it for a sandbox. > Now, I am concerned about the time to create the container, if we're > going to specify individual files, but the same would be true of an > overlay. [...] > to populate an overlayfs with just that specific list of files. No. For overlay you need only one mount (not even a bind) and only one directory traversal at the end to check for violations. The nice thing is that this is practically independent of the number or structure of directories/files you want to protect, i.e. it scales perfectly well. For the more fine-grained approach, you just delete the files you do not want to have in the beginning. Not sure, how quick this can be done, but once it is done, the slowdown when running the sandbox is independent of the number of deleted files (because here certainly only one hash lookup is required). Of course, as mgorny already observed, overlay alone is not an absolute protection (e.g. against writing to some /dev/...), so perhaps it is a good idea to use containers as an additional protection level. > If you just replicate the current sandbox > functionality then setup time is tiny I am not so much concerned about the setup time but more about the delay caused for file operations once the sandbox is set up. Perhaps even a dozen bind directories already give a considerable slowdown...
[gentoo-dev] Re: An example overlayfs sandbox test
Tim Harder wrote: > On 2017-09-23 19:59, Rich Freeman wrote: >> A read-only container > > I doubt bind mounts will scale > > As has been mentioned before, a different way would be to write some > sort of FUSE fs The problem with both, containers and FUSE, is performance. (For containers with thousands of binds, I haven't tried, but for FUSE I know how unionfs-fuse slows down "normal" operation: only for the reason that the implementation in userspace requires many additional context switches.) Both is fine for testing, but I am afraid not for regular user's emerge operations which usually involve too many file operations, at least for certain packages (e.g. *-sources, texlive-*). It is the big advantage of overlay that it is implemented in kernel and does not involve any time-consuming checks during normal file operations. Indeed, the price you pay is that the actual checking can be done only once only at the very end of the compilation, and so the only information you get is the name and time of violation (paths and filestamps). But concerning performance this "only once" checking is an advantage, of course. Main disadvantages: It requires the user to have overlay support in the kernel and extended attribute support for the file system containing the upper directory.
[gentoo-dev] Re: [PATCH] eapi7-ver.eclass: 'Early adopter' version of EAPI 7 version manip
Michał Górny wrote: > +# 1.2b-alpha4 -> 1 . 2 '' b - alpha '' 4 Is this only to explain the syntax or are there plans to extend the allowed versions for pms? There is a reason why pms currently does not allow "-" as separators within versions (with the exception of -r): With this general syntax, you would have a hard time to split into name and version for e.g. media-fonts/font-bitstream-75dpi-1.0.3 (Currently the version starts at the latest /-[0-9]/ match.) Also the ordering needs a discussion when version strings are allowed which are not covered by PMS. Note that e.g. 02 > 1 while 1.02 < 1.1 Is it still "correct" to have 1-02 < 1-1?
[gentoo-dev] Re: New SYMLINK_LIB=no migration tool for review
Michał Górny wrote: > > I have been running such a layout for over a year. [...] Thanks for clarifying that this already was discussed. Obviously, I was not aware about this discussion, and perhaps I was not the only one. > instead of waking up last-minute to redesign [...] Pointing me to the discussion would have been sufficient: If this was already discussed, it makes no sense to open that bottle again. As mentioned in my first posting: In this case I am sorry for the noise.
[gentoo-dev] Re: New SYMLINK_LIB=no migration tool for review
Mike Gilbert wrote:
> Debian puts 64-bit libs in /lib/(host)
Yes, this is somewhat weird:
They have /lib/i386-linux-gnu/ and /lib/x86_64-linux-gnu/
but anyway they use /lib32 instead of e.g. /lib/i686-linux-gnu/
Their reasons for this are mysterious to me.
> Migrating Gentoo to a "multiarch" config is a larger project.
Migrating to the completely split setting /lib{,32,64,x32}
would be _less_ intrusive than to simultaneously omitting /lib32:
With a slight modification of Michał's tool (in the instructions
and keeping lib32 untouched) not even a re-emerge of anything
would be necessary (as is the case now).
This is one of the reasons why a completely split layout appears
safer to me than to combine it with yet another new merge we
have no experience with.
[gentoo-dev] Re: New SYMLINK_LIB=no migration tool for review
Michał Górny wrote: > > 'No mainstream' as you claim it doesn't mean it's fine to invent yet > another completely incompatible solution. As I understand, the compatibility with Debian might be increased (keeping /lib32), at the cost of slightly decreasing the compatibility with Red Hat (concerning 32bit emulation). So depending on the use case, it might as well be more compatible. >> in _all_ /lib* directories > > This is not true. You are right, I had made a stupid mistake. (I forgot that I still have the symlink lib->lib64 so that of course I should not have wondered why all ld-* existed in both libs...) > the linker uses /lib path independently of how multilib is done > on the system. I am not sure whether I understand what you mean. Do you mean that practically sys-libs/binutils or sys-libs/glibc have hardcoded /lib -> 32bit /lib64 -> 64bit (concerning ld-*) unless one patches it out? In this case you are right that there is an upstream to follow. In this case I wonder why Debian decided to patch out this upstream decision to support /lib32. > The 32-bit proprietary binaries use exactly /lib which is the main > reason for the switch. I thought the main reason for the switch is to avoid the merging of /lib and /lib64 which in bug 506276 led to problems for debugging. For this reason I am afraid that now merging /lib and /lib32 might again lead to a similar type of problems. I am aware that exactly the cited problem is now mitigated, because this time the merging does not happen over a symlink. But anyway, it seems unnatural when "curing" from a merge to decide to merge something else.
[gentoo-dev] Re: New SYMLINK_LIB=no migration tool for review
Mike Gilbert wrote:
> On Wed, Aug 2, 2017 at 1:51 PM, Martin Vaeth wrote:
>> If this already was discussed then sorry for the noise:
>>
>> What is the rationale for merging lib32 with lib?
>> Wouldn't it be somewhat cleaner to have a completely
>> split structure
>>
>> lib64
>> lib32
>> libx32 (possibly)
>> lib
>
> Here are a couple of reasons:
>
> 1. Other distros (notably Red Hat and Fedora) put 32-bit libs in "lib".
According to bug 506276, Debian has instead merged 64-bit to lib.
So it seems to me that there is no "mainstream" to follow.
Perhaps striving for the cleanest solution would be the best?
> 2. The path to the 32-bit runtime linker (/lib/ld-linux.so.2) is
> hard-coded in every x86 binary on your system.
I am afraid that these must stay exceptional in any case:
Also currently, gentoo (and if I understood correctly, also Debian
and Red Hat) has the possible ld-linux{,-x86-64,-x32}.so.2 symlinks
in _all_ /lib* directories; I suppose that this is not intended to
change. (I assume that a change might break some proprietary binaries
which might have hard-coded the "wrong" directory.)
[gentoo-dev] Re: New SYMLINK_LIB=no migration tool for review
Michał Górny wrote: > > What are your thoughts? If this already was discussed then sorry for the noise: What is the rationale for merging lib32 with lib? Wouldn't it be somewhat cleaner to have a completely split structure lib64 lib32 libx32 (possibly) lib
[gentoo-dev] Re: lua upgrade plan
Duncan <1i5t5.dun...@cox.net> wrote: >> >> http://article.gmane.org/gmane.comp.lang.lua.general/18519 > > That reply is from 2005 and is apparently specific to (32-bit) x86's Even more is true! The only argument there concerns pic. But most distributions (hopefully also gentoo in a not-so-distant future, hopefully also on x86) will have PIE as a built-in gcc default, and programs should better all be built with PIE due to security considerations. So the whole argument concerning efficiency is simply invalid, nowadays. Yes, there might be very exceptional situations where it might be appropriate to sacrifice security considerations completely as a tradeoff for slightly more speed (although on amd64 this is probably already hardly measurable). But it should be exactly this: very exceptional situations. Most games should better not be exceptional concerning security, at least in a "standard" distribution.
[gentoo-dev] Re: [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)
Martin Vaeth wrote: Let me be more precise to avoid misunderstandings: > For the "standard" 2SAT case, one first determines the strongly > connected components in the implication graph (linear time). > Then for each such component one either _enables_ or _disables_ > all vertices. > The only difference to this "standard" 2SAT case is that we do > not want a random choice here This choice applies of course only to the _roots_ of the reduced implication graph (which has strongly connected components collapsed to vertices). In other words: Only for those strongly connected components which have no predecessor in the reduced implication graph, we can apply the subsequent algorithm. (For the other components, one has to follow the arrows in the reduced implication graph, of course.) > Look for that implication arrow in the strongly connected component > which occurs first in the specified ENFORCE_USE string; if > the starting vertex of this arrow is enabled, then enable also > the rest, otherwise disable it. By "starting vertex" I meant here that vertex which is written in front of ?. For instance, in a? ( b ) the "starting vertex" is "a" while in !b? ( !a ) the "starting vertex" is "!b". (Recall that every USE-flag "a" occurs as 2 vertices in the implication graph: Once as "a" and once as its negation "!a".) The 2-clause || ( a b ) has to be interpreted as !b? ( a ), of course. For ^^ ( a b ) it does probably not matter whether it is interpreted as a? ( !b ) !b? ( a ) or in the opposite order !b? ( a ) a? ( !b )
[gentoo-dev] Re: [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)
Ciaran McCreesh wrote: >> For 2SAT, there are linear time algorithms. > > "foo? ( bar )" does not encode as "( !foo \/ bar )" It does (see below). More precisely, at first it encodes as an arrow foo -> bar in the implication graph. > Good luck figuring out how to encode grounding in SAT Easy game (in the 2SAT case): For the "standard" 2SAT case, one first determines the strongly connected components in the implication graph (linear time). Then for each such component one either _enables_ or _disables_ all vertices. The only difference to this "standard" 2SAT case is that we do not want a random choice here whether we _enable_ or _disable_: We want a certain algorithm, specified by the order of the entries. If I understood the suggestion correctly, the algorithm would be: Look for that implication arrow in the strongly connected component which occurs first in the specified ENFORCE_USE string; if the starting vertex of this arrow is enabled, then enable also the rest, otherwise disable it. Doing some pre-indexing, when parsing the string, it is easy to implement this checking in linear running time (in the size of the component). Hence, altogether linear running time is needed for the full algorithm (i.e. to process all components).
[gentoo-dev] Re: [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)
Michał Górny wrote: > b? ( a ) c? ( b ) d? ( c ) e? ( d ) ... As written in another posting: This is 2SAT. 2SAT is solvable in linear time. To get a hard example you have to use several 3SAT clauses, i.e. || ( ... ) with 3 (or more) entries (and all of these entries must occur in the other clauses, of course, to get a hard example).
[gentoo-dev] Re: [RFC] Forced/automatic USE flag constraints (codename: ENFORCED_USE)
Alexis Ballier wrote: > > I think we should really try to find a sub-exponential solution Most examples mentioned earlier were actually 2SAT, i.e., they are only of the form foo? ( bar ) (possibly with negations) or can be easily reduced to that. E.g. ^^ ( foo bar ) foo? ( !bar graulty bazola ) can be rewritten as 2 or 3 2SAT-clauses as above, respectively. For 2SAT, there are linear time algorithms. If besides the above there occur not many clauses which are longer than 2 terms of the form ^^ ( foo !bar !graulty bazola ) then each such clause "just" multiplies the time by its size. (Yes, this is exponential, but I doubt that there will be many such clauses with more than 2 terms). The only problem which can really trigger a worst case (if the algorithm always reduces to solving a 2SAT problem) are longer || ( ) clauses. Either one should by definitoin limit the total number of USE-flags involved in those, or simply try only a limited combination of those (perhaps even only 1) and report a failure if this cannot be resolved (although an exponential time algorithm might be able to solve it).
[gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: eclass/
Michał Górny wrote: >> If this is such a big problem, maybe we should be discussing how to >> redesign things to improve it? > > Like, by not using eclasses and instead inlining all the stuff? There are other ways. One way to mitigate the problem might be to require that eclasses contain some # @VERSION: major.minor-revision line and that the metadata cache of an ebuild contains that version number (or at least the major component of it) instead of an md5 sum. Then the commiter of the eclass has full control whether a metadata regeneration will happen or not: For trivial changes (see below), the committer simply does not increase that version number (or increase only minor-revision), and one could agree that a rebuild of the metadata only happens if the major version number of an eclass changes. By "trivial changes" I mean in this connection not only changes in comments but also minor changes in functionality or even in the API: The only strict requirement is that the major version has to increase if the eclass change can induce a change in the metadata of some ebuild using it (e.g. if a value printed by some function changes which might involve IUSE, REQUIRED_USE, DEPEND, ...)
[gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: eclass/
Kent Fredric wrote: > Fabian Groffen wrote: > >> Hardware or more deltas to >> download by users? Just wondering. > > Both. > > - End users using git end up having to do massive metadata-updates. > - Infra needs to have massive hits to metadata regeneration > - End users using rsync have to fetch large deltas - Users using git repositories with pre-generated metadata have to download/store much more history data
[gentoo-dev] Re: [RFC] News item: GCC 6 defaults to USE="pie ssp"
Luis Ressel wrote: > Martin Vaeth wrote: > >> For instance, you cannot even compile the kernel without special >> patches (which disable pie) if you use a gcc which default-enables >> pie. > > Now I'm curious. Wouldn't that also affect the hardened gcc? I would guess so, but I did not try: I didn't use hardened gcc since years, because (a) I had to switch profiles too often because of forced pie which used to break compilation for almost every second package (some years ago). (b) -fstack-protector-all slowed down my system too much, especially since the security improvement over -fstack-protector-strong (or with older gcc versions -fstack-protector) is rather negligible. > I've never had any issues compiling vanilla-sources The experience I had reported was with the first non-beta versions of gcc-6[pie] from the hardened overlay and several (at that time current) versions of hardened-sources. I retried now with gcc-7.1.0-r1[pie] and current gentoo-sources, and it turned out that the issue does no longer exist. I do not know whether the reason is due to the change hardened-sources -> gentoo-sources, due to an upstream kernel fix, or due to a fix in the pie support of gcc (compared to the first gcc-6 versions).
[gentoo-dev] Re: [RFC] News item: GCC 6 defaults to USE="pie ssp"
Hanno Böck wrote: > > I could add my voice that I ran pie by default for a while I can confirm that the situation apparently has changed drastically since my last attempt. My previous assertion is no longer valid: Currently, I recompile world on x86 system with default pie, so far with only one issue caused implicitly by clisp. Afterwards, I will recompile world on amd64 and will report back in case the situation should be very different. > We have a tracker bug for default-pie-problems in bugzilla: I reported the clisp issue and will report if I meet further. At the time when this tracker bug was opened, I had so many issues with default pie that I decided to switch it off since reporting so much would have been too time-consuming for me. I do not know what is the reason for this change. Perhaps the first gcc versions with default pie had another bug.
[gentoo-dev] Re: [RFC] News item: GCC 6 defaults to USE="pie ssp"
Hanno Böck wrote: > I really think it's about time that pie becomes the default in Gentoo. Although I agree from a security perspective, I must warn that this is not realistic, currently: I am using gcc-6 since ages and tried to run a desktop with default pie for quite a while, but soon was forced to give up: There are simply too many package which fail to compile; this cannot even be recommended to early testers yet, not to speak about the wide public. The difficulty is not the static libraries (which except for embedded systems are hardly needed at all), but simply that too many projects are not prepared for this. The main problem is that it is not easy (as it was for the hardened gcc) to switch the compiler profiles if you have a non-working project: If a project fails to emerge and does not honour CFLAGS throughout - there simply are quite a lot of projects which do neither - the user either has to write/get some patch manually or he has to re-emerge gcc just in order to compile/update that single project. For instance, you cannot even compile the kernel without special patches (which disable pie) if you use a gcc which default-enables pie. Thus, unless practically all upstream projects deal with pie or unless gentoo manages to get a huge group of persons which patches all new releases of upstream projects to this purpose very quickly - both is not realistic, of course - using pie by default is a no-go for "normal" systems: Perhaps some very hardened servers with only very few packages and a very active administrator can afford to do this, but not "normal" users who run a desktop.
[gentoo-dev] Re: Requirements for UID/GID management
Christopher Head wrote: > > Are you sure that said utility isn't simply chown --from=? As usual, I just checked the POSIX standard and not the GNU extensions before posting ;) I did now a quick audit of the coreutils-8.25 source: It seems to be safely implemented in the way I mentioned.
[gentoo-dev] Re: Requirements for UID/GID management
Michael Orlitzky wrote: > > The fact that all permission and ownership information is shared is > precisely the problem. When you change ownership of the hardlink (which > you'll never know is a hardlink), you change ownership of /etc/shadow. Why should this be a problem except for a race between reading and changing the ownership? Admittedly, by using "find ... -exec ... +" the time for an exploit of the race is even increased when a "standard" chown command is used. However, it is no rocket science to write a race-free chown command in C: Just open the file and use stat() and fchown() to be sure to change only files from the "correct" user. Since this works on the filehandle and not on the filename, I think that there is no possibility for an exploit when this is used in the above find loop.
[gentoo-dev] Re: tmpfiles virtual
Rich Freeman wrote: > > If systemd-tmpfiles can work when systemd isn't running According to a brief test (not very exhaustive), this seems to work, though I did not investigate whether it requires that e.g. dbus is running. Without entering the discussion _how_ an init-script should be installed, I would welcome if that script would autodetect which variant is installed and if it could be configured which variant is chosen if both are installed: For instance, the systemd-tmpfiles implementation has some features concerning btrfs which are not (yet) supported by opentmpfiles. Some users might want to use that features. On the other hand, some users might prefer to be able to boot properly even if systemd/systemd-tmpfiles blows up for them for some reason...
[gentoo-dev] Re: tmpfiles virtual
Rich Freeman wrote: > On Thu, Nov 17, 2016 at 3:07 PM, Ian Stakenvicius wrote: >> >> Realistically, software should ensure the directories it needs at >> runtime are created through their own code, but upstreams are lazy [...] > > This isn't really being lazy. This is just not re-inventing the > wheel. ++ Conceptionally, it is also a question of efficiency and clean code separation: Why should several independent scripts of e.g. eix first care about directories (and force being run with root permissions...). More important, the latter is also a security topic: > or having > the program run as root, create directories, and then drop privs. eix had such code originally, but this required running quite a lot of code with root permissions, while now running everything with dropped permissions is possible. One could have written an init-script only to create the directories, but instead of providing such support for each init system separately, it is perhaps better to use the more standard "tmpfiles.d" >> In those cases, we'd need that rdepend. > > I tend to agree with this thinking. ++ But the argument of Ian is correct: If the user has both systemd and openrc installed (and thus virtual/tmpfiles.d is satisfied) he would not understand why tmpfiles.d is not processed if he starts the system with openrc.
[gentoo-dev] Re: grub-2 configuration
Mike Gilbert wrote: > > I have added an example grub.cfg to the gentoo repository. I finally made a repository of my grub.cfg "library" with a correpsonding example. This is a more sophisticated setup with a menu where one can choose resolution, init-program, etc. It can be found here: https://github.com/vaeth/grub-cfg-mv/ Library and exmample can also be installed by portage: Package sys-boot/grub-cfg-mv in the mv overlay (over layman)
[gentoo-dev] Re: Building a portage repo?
Michał Górny wrote: >> >>and subsequently egencache should give me the same type of portage tree >>that is currently officially distributed to users? > > Then remanifest, egencache with all options. You will also need to > include additional repositories for news, glsas and fetch extra files > for projects.xml. Not to forget dtd and xml-schema. There is app-portage/portage-postsyncd-mv in the mv overlay which does all this (and some more) for you in various (separately configurable) hooks in /etc/portage/repo.postsync.d/* Well, all except changing the manifest format and remanifest, but you can easily add another hook if you really want that.
[gentoo-dev] Re: chromium-54 needs ffmpeg-3.0.1
Duncan <1i5t5.dun...@cox.net> wrote: > Martin Vaeth posted on Wed, 31 Aug 2016 18:08:17 + as excerpted: > >> Kent Fredric wrote: >>> >>> I really wish there was a way to run ancient firefox with security >>> fixes :( >> >> There's palemoon > > For Linux? Even for gentoo: There's a "palemoon" overlay in layman. For instance, palemoon still provides the old privacy-keeping sync method instead of firefox's new data harvester. The only reason I still want firefox installed is that palemoon still only supports gstreamer (though they recently updated optionally to gstreamer:1.0) but not ffmpeg.
[gentoo-dev] Re: chromium-54 needs ffmpeg-3.0.1
Kent Fredric wrote: > > I really wish there was a way to run ancient firefox with security > fixes :( There's palemoon
[gentoo-dev] Re: chromium-54 needs ffmpeg-3.0.1
Duncan <1i5t5.dun...@cox.net> wrote: > >> Alexis Ballier wrote: >> >> As opposed to "roll out ffmpeg 3 prematurely" which will break *more* >> than just firefox. > > Umm... you mean chromium, not firefox, correct? I also think so. I have unmasked ffmpeg-3 since several months, because I don't use any package from bug 574788, but instead >gcc-5. Everything for which I (and I suppose most people) need ffmpeg compiles (and so far runs) without any problems: libquicktime, xine-lib-1.2.6-r1, mplayer-1.3.0, mpv-0.19.0, handbrake-0.10.5-r1, firefox-48.0, avidemux-2.6.8 (Well, for the apparently maintainer-needed avidemux I had lately to do a version bump to 2.6.12 and some patching in the mv overlay; but this was only for the very latest bump of ffmpeg-1.3.2 -> 1.3.3 plus a simultaneous bump gcc-5.4.0 -> 6.2.0. I didn't check whether the ffmpeg bump was related here, at all, or whether a similar patching could have been done for avidemux-2.6.8)
[gentoo-dev] Re: rfc: /etc/hostname on gentoo
William Hubbs wrote: > > I am planning to change the logic in /etc/init.d/hostname so that if > /etc/hostname exists, the first word out of that file will be used as > the hostname rather than any setting in /etc/conf.d/hostname. You could also make the content of the default /etc/conf.d/hostname: hostname=`cat /etc/hostname` or read hostname
[gentoo-dev] Re: rfc: /etc/init.d/modules loading modules defined in files
Mike Gilbert wrote: >> >> modules=$(sed -n -e '/^[^;#]/p' /etc/modules-load.d/*.conf \ >> /usr/lib/modules-load.d/*.conf 2>/dev/null || : ) > > This simple implementation does not follow the precedence rules > documented in modules-load.d(5). I didn't mean it as the recommended implementation for openrc, but just wanted to illustrate that such pre-implementation are already in use. (Currently, this implementation is sufficient, because currently practically nothing installs into /usr/lib/modules-load.d, so masking is currently unneeded). _Of course_ I would suggest that a proper implementation in openrc should follow the standard completely.
[gentoo-dev] Re: rfc: /etc/init.d/modules loading modules defined in files
William Hubbs wrote: > > but I'm open to making the behaviour compatible > with what systemd does Since openrc already supports tmpfiles.d, support for modules-load.d would be natural. In fact, this is already done for quite a while in https://github.com/vaeth/firewall-mv/blob/master/openrc/conf.d/modules The last line supports modules-load.d: modules=$(sed -n -e '/^[^;#]/p' /etc/modules-load.d/*.conf \ /usr/lib/modules-load.d/*.conf 2>/dev/null || : ) For compatibility with other init systems one should perhaps not extend the format of modules-load.d: If one also wants to e.g. support args for modules, this should perhaps be done in a separate directory. /etc/modprobe.d already exists for this, but one might still source another directory which contains only files added to /etc/conf.d/modules
[gentoo-dev] Re: [RFC] Masterplan for solving LINGUAS problems
Mart Raudsepp wrote:
> Ühel kenal päeval, K, 01.06.2016 kell 20:24, kirjutas Martin Vaeth:
>> Gentoo has chosen this name so that as a side effect of setting
>> USE=linguas_* you also get a correct LINGUAS variable exported
>> (according to the USE-settings and your settings and according
>> to the ebuild's IUSE.)
>> These are the package with LINGUAS options in their USE flags
>> which you mentioned above.
>
> People already set LINGUAS for autotools control; using the same for
> USE_EXPAND probably seemed natural to re-use that, but the effects on
> PM variable mangling wasn't thought about at the time.
At least, it looks like it was well-thought:
It avoided errors like "language xxx not supported",
it gave the user control to set it easily on a per-peckage basis,
and most important, it made the package manager aware of the LINGUAS
used and handle binary packages correctly.
>From the user perspective it had only advantages.
If all these positive side effects were not the original intention,
it was a very good accident that it happened.
> There were no problems, and there still are no problem with implicit
> LINGUAS handling (by means of not listing them in IUSE), as long as the
> package manager doesn't modify the variable set in make.conf.
As long as there is only one setting valid for all packages
and all systems on which one will ever install the binary package.
A quite unrealistic setting for quite some use cases.
> The user has set it explicitly somewhere (make.conf most likely) and
> simply gets honored in upstream build system, just like CFLAGS.
That handling of CFLAGS is bad enough - no reason to repeat this mistake:
This is usable only if you have only one architecture but not in a
hybrid setup with e.g. roughly compatible machines where e.g. you change
CFLAGS by /etc/portage/env or /etc/portage/package.env (or locally
in the environment) to compile for another machine or in a ways
compatible for another machine, etc.
Well, for CFLAGS, CXXFLAGS, LDFLAGS at least the
variables actually used are stored in
/var/db/cat/pkg/{CFLAGS,CXXFLAGS,LDFLAGS}
and similarly stored in the metadata of binary packages (not
hidden in some hard-to-parse compressed environment file),
so that the package manager _could_ check their validity
(and even if it does not - which AFAIK is the case currently -
it is not very hard to write a script which does this).
That's why I suggested that - as the very least - if you really want
to kill all the current advantages for no reason, at the very least
handle LNGUAS not even *worse* than CFLAGS and at least store them
in a new /var/db variable so that - if you already let the user do
the job which the package manager is supposed to do - at least the
user can write his own scripts to do portage's job.
(Of course, this scripts can never work as good as the current
solution if you really kill it, because these scripts are lacking
the information which LINGUAS are actually supported by the package.)
> LINGUAS does not imply any order whatsoever. Packages that assumed so
> were seriously buggy and if any still remain, this needs to be fixed
> ASAP.
So if you consider this as a non-issue, I see even less reason
why to remove the *transpararent* (to the package manager and user)
IUSE=linguas_* and go back to the intransparant LINGUAS=...
(I am not complaining about a renaming of the IUSE, as long as
this IUSE in the end affects LINUGAS)
> Including them all in IUSE for simple translation catalogs is not
> feasible to maintain.
I disagree. The list of packages is in most cases very easy to get
(e.g. ls po/*.po) and inserting it correctly is a usually a
one-time thing. For bumping, one will have to check for dependency
changes anyway, and that a package adds a new language without listing
it in its ChangeLog is also very rare.
> And yes, there are packages that have 196 different language and
> dialects translations. Even core GNOME packages [...]
Nevertheless these packages are very rare. One could also manage
*one* list of all languages globally and allow to use it for
convenience for such cases (if a few of the listed languages are
actually not provided, nothing critical will happen: The
user's output has perhaps a few more entries than necessary and
perhaps an unnecessary rebuild can happen if the user
selects/deselects an exotic language.)
> It also clutters emerge --verbose --pretend or --ask output hard with
> LINGUAS="long list of 196 language codes"
About the situation which we currently have (e.g. k3b): For some
packages the LINGUAS list is not 1 line but 2-4. So what?
If this really is an issue for somebody (I don't consider it some),
then one can discuss to add an option to suppress the LINGUAS output
for those not interested in it.
But it's certainly not a reason to make it *intransparent* for the
pack
[gentoo-dev] Re: [RFC] Masterplan for solving LINGUAS problems
Kent Fredric wrote: > On 2 June 2016 at 07:33, Martin Vaeth wrote: >> >> I prefer to have at least 5% of the ebuilds working and the other >> being fixable (if their maintainers want to spend the effort) >> than to remove a concept which breaks also these 5% and turns >> all ebuilds non-fixable, in principle. > > Changing the status-quo to "broken by default and needs 95% of the > tree to change to not be broken" is a bad precedent. What you describe _is_ the status-quo. The 95% are broken and remain broken (unless a lot of work is spent), no matter which of the suggested solutions is chosen. I am voting to *keep* better this than to change the status-quo by breaking the 5% working percent, too: mgorny's suggestion to kill l10n.eclass (or, more abstractly speaking, to forbid setting LINGUAS based in USE-flags within an ebuild) just influences the 5% of ebuilds which are currently using this mechanism and thus are non-broken (if their IUSE is correctly maintained). Removing l10n.eclass would throw these 5% back to the broken state of the other 95%, and even forbid by policy that any other would be fixed. I really cannot see an advantage in doing this, except for some irrational feeling of "consistency" to have everything equally broken.
[gentoo-dev] Re: [RFC] Masterplan for solving LINGUAS problems
Raymond Jennings wrote: > > I'd honestly as a minor issue like ot know why we called it linguas in the > first place. > [...] > I think though I should also point out...don't we already have existing > ebuilds that expose LINGUAS options in their USE flags? >From this posting, it is obvious that the discussion between mgorny and me was not well understandable, so I try to summarize the situation: The variable name LINGUAS was not chosen by gentoo: autotools based build systems (and some others) support this variable to control, for which languages some support is installed. The effect can be that simply some files are (not) installed, but the effect can also be much more complex: It really depends on the package. Gentoo has chosen this name so that as a side effect of setting USE=linguas_* you also get a correct LINGUAS variable exported (according to the USE-settings and your settings and according to the ebuild's IUSE.) These are the package with LINGUAS options in their USE flags which you mentioned above. This had the advantage that no additional code except a correct IUSE is needed for these ebuilds. (That's why gentoo has chosen this name). It had the disadvantage that: 1. Ebuilds without handling IUSE=linguas_* explicitly (or with wrong values, because maintainers did not care about the values) had problems. 2. Some packages which needed a different handling of LINGUAS (like respecting the order) also had problems. There are at least the following solutions to these disadvantages: a) One _could_ solve problem 1 simply by not touching LINGUAS if IUSE=linguas_* is not in the ebuild. (Main problem with this solution: It is not PMS compatible; one needs e.g. an exception for LINGUAS). In a similar manner, one could solve problem 2 by allowing ebuilds to modify LINGUAS at build time (which is perhaps also not PMS compatible). b) Or one could, for all packages with LINGUAS in their USE-flag simply rename IUSE=linguas_* to IUSE=l10n_* and set export LINGUAS=$L10N in these ebuilds (this would require practically no manual ebuild editing if one does it in the l10n.eclass). I had originally understood mgorny's suggestion such that b) is meant, and I would complain neither against a) nor b). But in his reply, mgorny says that, moreover, he wants to remove the l10n.eclass, more precisely, that he considers it as broken that packages use IUSE=l10n_* for setting LINGUAS. This suggestion means that setting LINGUAS can be done *only* in a hackish way by the user, "hidden" from the package manager, not in the clean way as it is currently done by those ebuilds with LINGUAS in their use-falgs. I agree with him that a hidden setting is a bad idea. Where our opinions strongly differ is whether a way to cleanly set LINGUAS should be provided (e.g. by allowing IUSE=l10n_* to modify LINGUAS appropriately, and make this the "good" way instead calling it a "broken" way which should be avoided.) The INSTALL_MASK is an independent thing which for _many_ packages "by accident" can be used with a similar effect as setting LINGUAS (because many packages use LINGUAS only to determine the files which they install). However, it is a different thing, and for some packages it is not a replacement at all. For instance, the default language of mplayer will depend on the first entry of LINGUAS, independent of INSTALL_MASK. That's why I suggested that INSTALL_MASK should actually not be discussed at all in this context.
[gentoo-dev] Re: [RFC] Masterplan for solving LINGUAS problems
Michał Górny wrote: > > If more than 95% of ebuilds are broken, this proves that a concept is > broken. > > Well, feel free to live in your fairy land. I prefer to have at least 5% of the ebuilds working and the other being fixable (if their maintainers want to spend the effort) than to remove a concept which breaks also these 5% and turns all ebuilds non-fixable, in principle. > I'm not going to continue the discussion if you are so blinded by > LINGUAS that you are even unable to understand what I'm talking about, > and consistently mix the LINGUAS concept and INSTALL_MASK concept, ...says the man who mixes this wildly in the first posting by suggesting to recommend the user to not use LINGUAS and use INSTALL_MASK *instead*; as a reply to my warning to not mix these completely unrelated concepts.
[gentoo-dev] Re: [RFC] Masterplan for solving LINGUAS problems
Michał Górny wrote: > > Do you have any numbers on how many ebuilds were exactly being fixed? > And how many were broken in the process because someone failed to > update linguas_*? Broken ebuilds are a reason to fix the ebuilds, but not a reason to replace a working concept by a hack which only works sometimes. > How do you feel about the lot of Chromium users who > spent a few hours rebuilding it to discover that they did only for > a linguas_* flag change they didn't even care for? You mean a user who emits --newuse when he only wants --changed-use and who sees the output of --ask, showing that the change is in LINGUAS="..." and nevertheless does not --exclude the package? In other words: Users who do not really know how to use portage? Well, I would recommend them to learn about the tools they use... > We're seriously talking about broken fix to a non-existent problem No, we are talking about removing a working solution (l10n.eclass) to return to a broken state which had caught me and probably many other users who manage systems with various linguas needs - until it was finally fixed at least for a certain number of packages by introducing the l10n eclass. > that actually introduces real problems s/introduces/solves/ like seeing that installation of a binary package conflicts with the machine's configuration. > How are you going to fix it? What should land in LINGUAS then? Leave LINGUAS unchanged for those packages which dont have linguas_* in their IUSE. Then still the packages which don't have IUSE=linguas_* are broken (because they have LINGUAS take effect without knowing of the user or the package manager), but at least those which currently use IUSE=linguas_* are working. > How would you determine the correct value? For ebuilds which are not updated to use IUSE=languas_* there is in principle no clean solution. Leave LINGUAS unchanged for these as a temporary hack until hopefully they are fixed some day. (Or change to l10n and modify l10n.eclass) >> > As for LINGUAS, it should be left as a toy for advanced users >> Selecting the languages which a package supports is an option >> only for advanced users? You must be kidding! > > No, implicitly stripping installed files via insecure mechanism that > is outside of package manager control is. Here we have the same opinion. The point is that LINGUAS is a variable which is not controlled by portage. This is the whole point of my argument: There must be a clean way to control this variable. The IUSE=linguas_* is one such clean way. Another way might be to make LINGUAS really a /var/db variable. > Yes, we know that all binary distributions were based on hacks for > years. Only Gentoo did it right [...] I do not understand what you mean here. I was not talking about binary distributions; maybe there is a misunderstanding. Maybe we have the same opinion: That INSTALL_MASK is some sort of hack; a very convenient one, but actually a hack. In any case it is not capable to do the "right" thing concerning linguas for every package; it only does it "right" for certain packages. >> That the package manager (and the user) is well aware of the >> options actually used to compile/install the package. Not hiding >> the user-selected configuration behind the package manager and >> the user in some random variable. > > It's as random as LINGUAS. Your point? Any variable which is not under the package manager's control and which influences the building of the package is a problem. LINGUAS is one, CFLAGS is another one. For CFLAGS there is at least a variable in /var/db. >> Then fix the package not handling LINGUAS in a sane way yet. > > I'm not sure if you are aware of it but most of us is not getting paid > for working on Gentoo. Not fixing the rest of the ebuild is not an excuse to remove a mechanism which made at least those ebuilds work for which the maintainers took the time to fix them (by using l10n.eclass or doing an analogous thing manually). >> For those who have, you can't trust them being up-to-date. >> That's a non-argument. No ebuild you can trust being >> up-to-date as well as all ebuilds. > > Of course. Your arguments are important arguments, mine are > non-arguments. I was not talking about "your arguments" but about your specific argument "we must remove a working mechanism because an ebuild author might do a mistake and e.g. bump without checking the changes of the package". > When people don't get needed locales installed, it's a non-problem. Who said this? This is exactly the issue I am talking about with binary packages: > *Every* binary package is valid because it doesn't have anything > stripped. This is not true. If you compiled the binary package with LINGUAS="en_US de_DE" and install it on a laptop with LINGUAS="en_US de_DE cz_CZ", you will be missing the Czech translations, because they are not in the binary package. (They are not "stripped" but simply not there...) So binary package compiled with LINGUAS="en_US de_DE
[gentoo-dev] Re: [RFC] Masterplan for solving LINGUAS problems
Michał Górny wrote: >> >>Therefore, I suggest to put this line in l10n.eclass >>(perhaps with a mechanism to avoid it for some exceptional packages >>which have to treat LINGUAS differently.) >>This way, none of these ebuilds inheriting l10n needs to be patched. > > This eclass should be killed with fire as ugly nonsense that > makes some packages useless for binary packages. Exactly the opposite is the case: Letting a variabe like LINGUAS "hiddden from the package manager" decide about the actual content of the package makes any handling of binary packages broken. The introduction of linguas_* (and of the l10n eclass to make this handable for the ebuild writer) was finally fixing this long-broken behaviour of ebuilds. You now want to revert this fix and return to the earlier broken state without any need. If the only reason for the "need" is a badly formulated/thought-through EAPI=5/6 specification to clean LINGUAS for packages not having it in its IUSE, then please fix this specification or make at least an exception for LINGUAS, but don't break the working solution. > As for LINGUAS, it should be left as a toy for advanced users Selecting the languages which a package supports is an option only for advanced users? You must be kidding! > and not presented as a recommended solution. The "recommended solution" is to rely on a hack which removs files behind the back of upstreams installation program (which likely installs other random language data in random files for various languages)? You are kidding again! >>1) In contrast to packages with L10N, the user cannot see with >>tools like eix for which linguas a certain package is installed. >>In fact, the user would have to analyze the compressed environment >>file find this out (this is also very package manager specific): > > What's the use of this? That the package manager (and the user) is well aware of the options actually used to compile/install the package. Not hiding the user-selected configuration behind the package manager and the user in some random variable. > Most of the packages don't have the flags anyway. Then fix the package not handling LINGUAS in a sane way yet. > For those who have, you can't trust them being up-to-date. That's a non-argument. No ebuild you can trust being up-to-date as well as all ebuilds. > As for installed package, all files are listed in vdb > (including those masked), so you can easily figure out the > differences. So to find out whether a binary package is valid for the current configuration (i.e. the current LINGUAS), the package manager or user "just" has to recompile it and compare the files and checksums. Great! And because this reinstallation-solution is possible, it is ok to rely on a hack instead of a solution visible to the package manager and user. >>2) Even worse for binary packages. > > Wrong. All localizations are included in the binary package, but not in the metadata. > Of course, as long as maintainer doesn't start playing with LINGUAS. That's the point: As long as all systems and all packages have the same LINGUAS settings. Which is not the case for many users having various systems. E.g. for a laptop, it is very natural to have different LINGUAS than for a desktop, even if you likely compile for the laptop on the desktop. >>3) The package manager cannot see after a change of LINGUAS, >>which packages need a recompilaten. > > Wrong, as already explained above. So? How can you see it? By checking hackishly the existence of random files in */po/* while e.g. the actual effect of LINGUAS is in some error message provided by the package in god-knows-what manner? *If* you want to keep only LINGUAS and not introduce L10N, *then* at least you should make LINGUAS a metadata variable to be stored in /var/db so that the user and package manager can easily access it.
[gentoo-dev] Re: [RFC] Masterplan for solving LINGUAS problems
Michał Górny wrote: > > 1. Get approval on INSTALL_MASK GLEP [1] and finish implementing it > in Portage. As already mentioned by some, INSTALL_MASK is something else than LINGUAS, because LINGUAS involves also files which are generated by the build system. Although I appreciate a more configurable INSTALL_MASK, this should not be mixed with the LINGUAS problem. > 2. Introduce a new USE_EXPAND that can be used to control localizations > whenever this is really required (dependencies, large files, etc.). > Let's use L10N as a draft name for it. Just to be sure that there is no misunderstanding: L10N should take the role which LINGUAS currently has for most packages from the user perspective. In other words, all ebuilds for packages whose build systems treat LINGUAS in a clean way (not relying on order etc.) and which currently have IUSE=linuguas_* will very likely contain corresponding ISUE=l10n_* and the line export LINGUAS=$L10N Therefore, I suggest to put this line in l10n.eclass (perhaps with a mechanism to avoid it for some exceptional packages which have to treat LINGUAS differently.) This way, none of these ebuilds inheriting l10n needs to be patched. > 3. Fix all packages using LINGUAS as USE_EXPAND, either by converting > to L10N or by removing the needless flags. I would strongly discourage doing the latter (unless IUSE=linguas_* was there only by mistake): When users change their L10N (or USE-flags), users should be able to see which packages need a recompilation when using emerge -N. Also for binary packages, it must be easily possible to see changes. See the more lengthy explanation below. > Request changing LINGUAS to L10N in make.conf, + > and make LINGUAS considered an 'advanced variable' for > implicit localization control (i.e. passed through to build systems). With only some rare exceptions (e.g. where the order matters), ebuilds should better set this variable according to IUSE=l10n_*. It would be better if these exceptions would not have to exist at all, i.e. if e.g. the ebuilds for packages for which the order of LINGUAS matters also use IUSE=l10n_* and introduce other USE flags to specify the order somehow (AFAIK only mplayer is involved, and for this only the first item is important, so that one can introduce e.g. IUSE=default_l10n_* variables for which exactly one must be set, or something similar.) Indeed, these exceptions are very inconvenient for the user as well as for the package manager: 1) In contrast to packages with L10N, the user cannot see with tools like eix for which linguas a certain package is installed. In fact, the user would have to analyze the compressed environment file find this out (this is also very package manager specific): 2) Even worse for binary packages. 3) The package manager cannot see after a change of LINGUAS, which packages need a recompilaten. 4) The same for binary packages. > Recommend clean INSTALL_MASK solution instead. I would also not mix these unrelated things in the documentation. One can hint that this takes additional actions for build systems not honouring L10N correctly, but usually it is not a substitute for setting L10N (or LINGUAS for the exceptional packages) but only a supplement.
[gentoo-dev] Re: Bug #565566: Why is it still not fixed?
Patrick Lauer wrote: > > Notice the --whole-file part there. Are there perhaps plans to remove this? Before the reversed ChangeLogs, this option was useful, but perhaps now removing it would really lower the traffic? One would have to make a bunch of tests over 1-2 months, perhaps: a) two with rsyncing every day b) two with rsyncing 1/week, and perhaps even c) two with rsyncing 1/month. (where "two" means one with and onewithout --whole-file) and add up the traffic. I am not sure whether servers can be set up to implicitly assume --whole-file to save possibly some resources needed to calculate the checksums. This would give wrong results, of course. In this case, it would be necessary for the tests to set up a server with full support, locally. In the latter case, one could also use historical webrsync data if they are available somewhere which would mean that the tests could be done rather quickly...
[gentoo-dev] Re: Bug #565566: Why is it still not fixed?
Rich Freeman wrote: > > Clearly it doesn't increase by a factor of 1 every year The yearly increase of the factor is rather precisely 1: According to current data, it is .95, see below. With xz compression for squashfs, it is even 1.4! (Note: increase _of_ the factor, not _by_ the factor, of course; we are speaking about a linear increase, not an exponential one.) More precisely: If in both cases you extremeley optimize for space (details see below) then a change from rsync to git (non-shallow) costs you a) now: the factor 2.6 of needed disk space b) in future for every year this factor is increased by the summand 1.4. For example, in 2.5 years you will need roughly 2.6 + (1.4 * 2.5) = 6.1 times the disk space than for rsync. After 2.5 more years, the factor will be more than 10. For a) I assumed that in both cases the current repository is kept compressed with squashfs (xz). This first factor will be much larger, of course, if you omit squashfs when you switch to git. (You must take measurements to keep the checked-out repository separate: you cannot use standard emerge --sync to get this optimization.) For both numbers, I even optimized the .git compression by executing repeatedly git prune; git repack -a -d; git gc --agressive which for the historical repository took several hours; thus, unless you use a cron-job, this is not realistic. Without this optimization, both numbers would be even larger. Here are the plain data I used for the calculation: 1. RSYNC =84,062,208 (rsync gentoo repository, compressed with squashfs (-comp xz).) 2. GIT = 136,322,616 (Current .git data, without checked-out tree; compression optimized by the time-costly commands above.) 3. FULL = 1,923,685,435 (.git data as in 2, but with history added) 4. YEARS = 15 (length of the historical data: first checkin was June 2000; change to git was IIRC somewhere in middle 2015). So the number from a) is size with git $GIT + $RSYNC --- = - ~ 2.6 size with rysnc $RSYNC The number from b) is size of history increase per year($FULL - $GIT) / $YEARS - = ~ 1.4 size with rsync $RSYNC In the previus postings, I was assuming the much faster squashfs compression -comp lz4 -Xhc instead of -comp xz. In this case, the number from 1 changes to RSYNC =125784064 which leads to the factor .95 ~ 1 for b) which I mentioned in the beginning.
[gentoo-dev] Re: Bug #565566: Why is it still not fixed?
Rich Freeman wrote: >> >> And currently the git history is still almost empty... >> > > If you want pre-migration history you need to fetch that separately. How? Neither on gitweb.gentoo.org nor on github I found an obvious repository with this data. > It is about 1.7G. > Considering that this represents a LOT more than 2-3 years of history If the 1.7G are fully compressed history, this would confirm my estimate rather precisely, if it represents (1700/120 - 1) ~ 13 years. Gentoo exists since 2002, so it seems my estimate was very good. > (including periods where the commit rate was higher than it is today) One of my assumptions for the estimate was that this rate is constant in the average. Also I am not sure whether you right that this rate was really higher, previously: Nowadays, even a rather trivial eclass-update is separated into several commits, increasing the amount of data needed for storage. > I think your estimates of where the migrated repo will be in 2-3 years > is too high. Note that I compared squashfs with a git user who does not even care about git-internal recompression. Of course, you can decrease the factor somewhat if e.g. your checked-out tree is still stored on squashfs. This does not change the fact that the factor will increase every year by about 1 (or probably more, because git uses the uneffective gzip compression, only).
[gentoo-dev] Re: Bug #565566: Why is it still not fixed?
Gordon Pettey wrote: >> >> Already now this means that you need 2 (or already 3?) times the >> disk space as for an rysnc mirror; multiply all numbers by 4 >> if you used squashfs to store the tree. [...] > > Or, in 2-3 years, maybe people will stop with the hyperbole Hyperbole? Really? Let's first look at the current data. Instead of guessing I now fetched the git tree to get the exact number: git on ext2 (8K blocks): 704 M squashfs with lz4: 120 M lz4 is the fastest algorithm, but not the best concerning space. More seriously: The git data is still missing metadata information which will add some more. It seems my estimate of the factor 2*4 = 8 for the current state was rather realistic. Not to forget that this was a fresh checkout where the .git data itself is fully compressed in one file (which is by default not the case when you update frequently - it depends on your git configuration and perhaps whether you use a cron job for recompression). So perhaps for some git users the bracket in my estimate (3*4=12) is already correct. Whether 1 GB of permanent disk space only for the overhead of package management is appropriate, everybody must decide by himself. Compared to other distributions, this is an awful lot. Only for getting ChangeLogs it is IMHO way too much. And currently the git history is still almost empty... Before I turn to the future, some remarks: > The tree is a bunch of text files, of which a whole lot of text is > repeated That's why squashfs is so effective already compared to plain rsync. Of course, a lot of the *current* factor comes from this. > which is great for compression, which git does. You seem to pretend that I ignored this, but I did not: >> (there is possibility of some >> compression of history, but OTOH, many packages are added and >> removed, eclasses keep changing, etc.) Of course, concerning future, one must make some assumptions. Perhaps it is reasonable to assume that roughly a constant amount of new data is added every year, i.e., the quotient (git data/squashfs) increases every year by a constant summand. Compression will not change this "constantness", but at most influence the summand itself. Quite the opposite, in the moment when the history evades a certain size - depending on the memory window size used by the gzip implementation of git, compression will eventually become much less effective: You can see the difference essentially in the gzip vs. xz compresssion size, because the main difference here is the size of the mentioned memory window. And as mentioned above, unless you are regularly recompressing (by a cron job or by git configuration after updating) you hardly profit from the git compression at all. How large the yearly summand is, can only be guessed, currently. I think my assumption that after 1 year the number of new/modified files is roughly the total amount of files in the tree is realistic, perhaps even too low. (Not to forget that also every commit adds data by itself.) So in 2-3 years, the factor (compared to squashfs) might be roughly: 8*2.5 = 20 without recompressing .git 8 + 2.5 = 10.5 with fully compresssed .git (The latter factor is unrealistically low, because git's gzip compression is less effective than lz4 and *much* less effective than xz). And even if I should have overestimated the yearly summand by the factor 2, you only need to double the number of years which you have to wait...
[gentoo-dev] Re: Bug #565566: Why is it still not fixed?
Luis Ressel wrote: > > That would require a local git clone. And that's exactly what those who > still want Changelogs are trying to avoid. You need even a deep git clone with full history. Already now this means that you need 2 (or already 3?) times the disk space as for an rysnc mirror; multiply all numbers by 4 if you used squashfs to store the tree. In the course of the years the factor will continue to increase; I guess at least by 1 for every year (there is possibility of some compression of history, but OTOH, many packages are added and removed, eclasses keep changing, etc.) So in 2-3 years, it can be for some users 20 times the disk storage than what it needs now. I think there must be a very good reason to require that amount of disk space on everybody's machine. ChangeLogs IMHO isn't.
[gentoo-dev] Re: RFC: intel-sdp-r1.eclass
Justin Lecher (jlec) wrote:
>>> [[ -n ${_INTEL_PV4} ]] && _INTEL_PV+="${_INTEL_PV4}-"
>>> [[ -n ${_INTEL_PV1} ]] && _INTEL_PV+="${_INTEL_PV1}"
>>> [[ -n ${_INTEL_PV2} ]] && _INTEL_PV+=".${_INTEL_PV2}"
>>> [[ -n ${_INTEL_PV3} ]] && _INTEL_PV+=".${_INTEL_PV3}"
>>> [[ -n ${_INTEL_PV4} ]] && _INTEL_PV+="-${_INTEL_PV4}"
>>
>> Now, this is crazy ;-). I don't see immediately how to improve that,
>> but I suggest you thought about that.
>
> [...] So there is no sanity other then reshuffling.
You could put the logic into the string:
_INTEL_PV+="${_INTEL_PV4}${_INTEL_PV4:+-}${_INTEL_PV1}"
_INTEL_PV+="${_INTEL_PV2:+.}${_INTEL_PV2}"
_INTEL_PV+="${_INTEL_PV3:+.}${_INTEL_PV3}"
_INTEL_PV+="${_INTEL_PV4:+-}${_INTEL_PV4}"
[gentoo-dev] Re: GLEP 67 is in, please update your metadata.dtd!
Mike Gilbert wrote:
> On Mon, Jan 25, 2016 at 11:31 AM, Luis Ressel wrote:
>>
>> I might be asking this for a second time, but why does repoman download
>> the metadata.dtd at all? If one fetches from
>> git://../gentoo-mirror/gentoo (or via rsync, afaik) it is included
>> in /usr/portage/metadata/dtd/.
>
> repoman is a developer tool. The git repo used by developers does not
> include metadata.dtd.
repoman is useful for everybody who manages a local
or public repository. I guess only a small number
of these people use the developers' git repo.
(For instance, I am one of these...)
It would be nice if repoman would first check whether
the file is in some ${repo}/metadata/dtd/
and download to DISTDIR only as a fallback.
Or if it can be at least configured (e.g. with an option)
to check for this file (maybe the path can be configured, too)
before downloading metadata.dtd.
In fact, I was so annoyed by the forced downloading
(which as mentioned is a permission/security issue)
that I regularly apply a patch to portage which sets
the download interval to (almost) infinity.
Perhaps I am not the only one who did such a hack.
[gentoo-dev] Re: [PATCH] readme.gentoo-r1.eclass: Do not inherit eutils.
Michał Górny wrote:
>> if [[ -n "${DISABLE_AUTOFORMATTING}" ]]; then
>> echo "${DOC_CONTENTS}" > "${T}"/README.gentoo || die
>> else
>> echo -e ${DOC_CONTENTS} | fold -s -w 70 \
>> | sed 's/[[:space:]]*$//' > "${T}"/README.gentoo
(omitted some lines/space for better context):
> Maybe I'm missing something but is there any reason not to quote
> "${DOC_CONTENTS}" instead of working around the results of not quoting
> it?
With DISABLE_AUTOFORMATTING=true, it is correctly quoted,
and there is no "workaround".
Without DISABLE_AUTOFORMATTING, the autowrapping (and ignoring
of "manual" format information by not quoting) is obviously
intentional.
Maybe, it would be more natural to have the opposite as the default,
but this is how the eclass works from its very beginning.
[gentoo-dev] Re: [EAPI 7] The future goals for PORTDIR, ECLASSDIR and... LICENSEDIR?
Ian Stakenvicius wrote: > '$(get_eclasspath [name])' or $(get_licensepath [name]) or the like Since we are at a brainstorm level: How about filling associative arrays instead? This would BTW also provide the names of the master repositories (although one could perhaps do the latter more explicit in an array, too). I suppose that this cannot be done without a new EAPI.
[gentoo-dev] Re: EAPI 6 portage is out!
Michał Górny wrote: > > commit EAPI 6 ebuilds to ~arch. For an overlay maintainer like me, it is not reasonable to bump eclasses locally. So please bump the relevant eclasses timely, most notably (AFAICS these needs just extending the check; perhaps a *negative* check would be simpler for future EAPI bumps?) readme.gentoo bash-completion-r1 systemd perhaps more work (I didn't check) and less urgent being: distutils-r1 python-r1 python-single-r1 perl-module gnome2 gnome2-utils Or should I open a bug for each of it? And what about: latex-package.eclass games.eclass Are these going to be deprecated due to inheriting base unconditionally?
[gentoo-dev] Re: Revise EAPI 6? (was: [RFC] ban use of base-4 casemods in ebuilds due to locale collation instability)
Jason A. Donenfeld wrote: > > I'd be in favor of full-on LC_ALL=C. Setting LC_ALL seems wrong as it is meant as a quick hack and should not be relied on by a "generic" tool like portage. Better define to *unset* LC_ALL (remembering the previous value, see below) and to set (all?) other LC_* to defined values. When we are at it: Maybe it is even sufficient to define only LC_CTYPE=C LC_NUMERIC=C LC_COLLATE=C LC_MESSAGES=C LC_MONETARY=C In any case, the old values should be kept (and for simplicity defined to the previous LC_ALL if the latter was set), so that the ebuild author is able to stick to the user's choice for certain/all values if he needs to: In particular, for LC_CTYPE, this might be necessary, because of correct UTF8-support, as already mentioned (the ebuild author cannot say LC_CTYPE=*.UTF8). But also e.g. for LC_MONETARY, this might be necessary for some strange local banking tools. It is perhaps not necessary to (re)define LANG at all: Setting LC_MESSAGES should be sufficient for most build-time stuff to get readable logs, and LANG=C might be the main reason, why some people might not like the change and decided e.g. to patch it out in paludis, as mentioned in this thread.
[gentoo-dev] Re: ChangeLog
Rich Freeman wrote: > > Keep in mind that "resources" is a vague term [...] > disk io and CPU to sync [...] For syncing, I think these latter resources are less important, because they influence only the *time* of a syncing action, which is normally not so important for the user. Bandwidth and (effective) harddisk space are more crucial IMHO. For the latter, we have a _clear_ winner: squashdelta (which BTW could also be signed). Concerning bandwidth, comparison is harder - it varies for the different methods dramatically on how often you sync - but squashdelta is also not bad here. At least the mentioned problems with connection loss can be solved easily for squashdelta, too, because only one file has to be transferred whose transfer might be resumed.
[gentoo-dev] Re: ChangeLog
Мисбах-Соловьёв Вадим wrote: >> Perhaps there is a better choice of distribution for you if you are. > > Or you can just... use rsync. Or emerge-webrsync, emerge-delta-webrsync or squashdelta (I strongly hope that the latter will be available again in some future - IMHO it is perfect for most users). Please do not break all these possibilities for users who do not have to waste the resources for a full git clone and want to see regularly ChangeLogs nevertheless!
[gentoo-dev] Re: ChangeLog
Мисбах-Соловьёв Вадим wrote: > >> Now how can this user display the ChangeLog for >> a certain package? > ``` > git log -- pkg-category/pkg-name/ You removed the crucial part of my posting: >> git clone --depth=1
[gentoo-dev] Re: ChangeLog
hasufell wrote: > > git clone --depth=1 The only reasonable option for the gentoo user (not for the gentoo developer) if he does not have megabytes to waste on his harddisk (which probably many users don't), if you want to *force* him to use git. Now how can this user display the ChangeLog for a certain package?
[gentoo-dev] Re: ChangeLog
Rich Freeman wrote: > > What discussion or decision is necessary? > What is needed is for those who want changelogs > to fix the bug The bug can only be fixed by somebody who knows the details how the rsync mirrors are set up. As mentioned in the discussion in bug 561454, *essentially* all it needs to fix the issue is to call egencache --repo=gentoo --update-changelog on the main server providing the data for the mirrors, but details (e.g. when to call it) depend on how exactly the server is organized. For instance, it might be necessary to put ChangeLog /.gitignore into a local .gitignore file or something similar, and somehow the Manifests due to modified ChangeLogs might need to be updated, too.
[gentoo-dev] Re: Dynamic dependencies
Rich Freeman wrote: > > Proposal 3a might be: Anytime an RDEPEND in an eclass is changed, the > eclass must be revisioned unless all ebuilds in the gentoo repository > will continue to work correctly with the old RDEPEND. > Proposal 4a might be: Anytime an RDEPEND in an eclass is changed, all > ebuilds that inherit the eclass in the gentoo repository must be > revisioned if they will not continue to work correctly with the old > RDEPEND. Adding an || alternative should be included here: The installed package would continue to work without that alternative, but without a revbump the user is not able to see that he might possibly drop a package.
[gentoo-dev] Re: QA bikeshed: killing USE=dedicated in favor of uniform USE=client+server
Alexander Berntsen wrote: > > - -1. The way "dedicated" is used in games ebuilds is a very established > term that all gamers know and expect to behave in a specific way. > This will mess with our users. How do you know what the users know and expect? When installing a game, I was regularly surprised by the meaning of this USE-flag and had to read its description several times to be sure that I understand what is going on. Remember that not only permanent gamers install games occassionally.
[gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in eclass: ChangeLog kde5-functions.eclass kde5.eclass
Dan Douglas wrote: > for x in *; do > [[ -e $x ]] || continue > ... > done You should also test for $x not being a symlink. Otherwise you can miss the corner case that you have a dead/unresolvable symlink called "*".
[gentoo-dev] Re: RFC: using Ninja in more CMake-based packages
Franz Fellner wrote: > > IMHO a working build system > always is better than a fast but potentially broken one :) Meanwhile, it is not so clear which of the two systems is more likely the "potentially broken" one: According to some of the mentioned bugs, it appears to me that some upstreams consider rather ninja as the "working" (and well-tested) tool, and Makefile as poor man's fallback.
[gentoo-dev] Re: News item review: SquashDelta syncing support
Brian Dolbec wrote: > Martin Vaeth wrote: >> >> However, currently this does not play nicely with squashdelta: >> You have to "undo" the mounting of squashdelta and have to use >> different command (e.g. squashmount) afterwards. > > No, that is not correct [...] > 2) /etc/portage/repo.postsync.d I know about this hook, but this is not what I meant. What I meant is the possibility to *replace* the automatic mounting of portage by a different command (for instance, a possibility to *avoid* that portage mounts/umounts automatically but expects this to happens in this hook). I give reasons for this below. (This discussion belongs actually to portage-devel mailing list or to some bug, but now I feel the necessity to clarify the misunderstanding.) It is not only inefficient and hackish (with possible problems in case of unexpected situations like a SIGINT or other signal at a "bad" time) if two programs/scripts "fight" about mounts and undo each others' mounts. It also causes severe difficulties in connection with overlayfs/aufs/...: With these filesystems, you must create two (in case of overlayfs even three) auxiliary directories, and in this situation, it might be natural to choose these as temporary direcories (e.g. generated in /tmp with unique names); to understand the following explanation note also that two of these directories (the squash filesystem and the overlay filesystem) should "normally" always be mounted/umounted together. Now if portage umounts only the "overlay" directory, the information about the temporary directory names is lost (leaving possibly quite a bit data in /tmp undeleted forever), and the second necessary "umount" does not happen and is hard to do later on: It prevents the space of the old squash file from actually being freed (and this mount is hard to find later on: It is a mount of a deleted file on an unknown temporary directory. Of course, one could try to find this mount by some heuristic, but this is extremely hackish, and the heuristic might find some other squash file which the user has mounted similarly for some other reason.) In case of the mentioned "squashmount" tool, the situation is better, because "squashmount" is rather complex and e.g. stores/manages the names of temporary directories independently. However, users might also want to use less sophisticated tools than "squashmount" (and also "squashmount" has no easy solution for "random" remounts of the mount points it manages, because it is almost impossible to write a "generic" such solution.) In any case, it is rather hackisch to write a lot of additional code to "undo" an actually undesired umounting+mounting: The "clean" solution appears to be to not do the "undesired" (in this situation) umounting+mounting in the first place but to execute *only* the actually desired (u)mount command(s) instead.
[gentoo-dev] Re: News item review: SquashDelta syncing support
Rich Freeman wrote: > Downsides include: > 2. Impossible to tweak ebuilds without setting up an overlay. This > might be annoying for devs/etc. It is still possible to setup a read-writable portage tree (using overlayfs/aufs/unionfs-fuse/... e.g. using the squashmount tool from the mv overlay). However, currently this does not play nicely with squashdelta: You have to "undo" the mounting of squashdelta and have to use different command (e.g. squashmount) afterwards. Although this can probably be done e.g. in eix-sync with hooks, I hope that in the near future there will be a possibility to combine these methods more conveniently. Currently, I made only some remarks in comment #3 of bug 549716, because it seems that the sync module mechanism is currently lacking the infrastructure for adding custom data (like hooks) to a module.
[gentoo-dev] Re: Quick RFC: USE=libav vs FFMPEG_IMPL=libav|ffmpeg
Zac Medico wrote: > > Also, portage-2.2.16 will have support for special USE_EXPAND syntax in > package.use I knew from reading portage-dev ml Actually, I am hoping that the introduction of the feature be taken as an opportunity to document USE_EXPAND better as a whole on some prominent places where USE-flags are introduced, explaining that these are internally really just USE-flags with special names (example) and that the whole syntax with variables (and the new one in package.use) are just convenience shortcuts for global/local settings. It would also not hurt to refer to desc/ Probably some native speaker can formulate this much better.
[gentoo-dev] Re: Quick RFC: USE=libav vs FFMPEG_IMPL=libav|ffmpeg
Markos Chandras wrote: > we bloat the make.conf file with new variables every now and then Although it often makes sense to set USE_EXPAND-variables in make.conf, it is not *necessary* to do it in this way and in this file: It can also happen in USE or in /etc/portage/package.use In fact, this is something which many users apparently do not know (just check the forums): I think the USE_EXPAND mechanism needs to be documented better. Of course, this is somehow independent of the current discussion. > As a result, the time spent to maintain make.conf has been > increased over time leading to more headaches on world updates etc. Whether you edit a USE_EXPAND variable or a USE variable in make.conf or something in /etc/portage/package.use does not really make much of a difference. And some of these three *must* be edited when the || ( ...:= ) trouble should be avoided, no matter which approach is chosen. > imho, in this particular case, the old behavior is fine by me and it's > what has been around for a long time. What has been around for a long time was the fragile || ( ...:= ) The meaning of libav as selection of the implementation is something new which forces about half of the users to edit USE or package.use anyway.
[gentoo-dev] Re: Moving CPU flags into USE_EXPAND
Alexis Ballier wrote: >> >> >> >> More precisely: When changing the names anyway, >> >> IMHO it would be a very good idea to follow the convention of the >> >> "flag" names in /proc/cpuinfo and add all flags supported >> >> there as possible USE-flags, no matter, whether they are currently >> >> used in some package or not. >> > >> > seriously ? >> > >> > $ grep flags /proc/cpuinfo | head -n 1 | wc -w >> > 94 Well, currently, we have at least 10-20 already: At a very quick check (probably missing some), I found: mmx sse sse2 aes-ni ssse3 sse3 sse4 sse4a sse4.1 sse4.2 3des 3dnow 3dnowext and probably there are more to be added anyway in future packages. I am not even sure whether some of these sse4* are dupes or not since they do not follow /proc/cpuinfo convention. Some others do not follow this convention at all, e.g. "sse3" is called "pni" in /proc/cpuinfo or "aes-ni" is called "aes" in /proc/cpuinfo This is mainly what I meant: Currently, users have to look up how sse3 is called in /proc/cpuinfo to understand whether their processor has support for it. This could be automated. If not all of the 94 flags are included, it is perhaps not a drama; my main suggestion is to follow /proc/cpuinfo convention in the existing ones. > but rather about writing the proper .desc file Well, currently the descriptions are: 3dnow: Use the 3DNow! instruction set mmx: Use the mmx instruction set sse: Use the SSE2 instruction set sse4_1: Enable sse4.1 acceleration ... So they are practically useless unless you know anyway what the corresponding processor flags mean. I suggest to start with a "stub" .desc list, e.g.: yyy: Use the yyy CPU feature, cf. /proc/cpuinfo This is enough for the user to check on which of his systems he might want it. A more verbose description can be added for some flags some day as a bonus; such a list can grow over time, and some experts might want to send some patches to fix/extend the descriptions. But why require for the start more than we have currently? If there is no list to start with, then you can also not expect that somebody will suggest patches (I mean: to a non-listed feature...)
[gentoo-dev] Re: Moving CPU flags into USE_EXPAND
Christopher Head wrote: > > All that requires is knowing the names, though; it would be > fine if no package actually uses the feature yet. ++ More precisely: When changing the names anyway, IMHO it would be a very good idea to follow the convention of the "flag" names in /proc/cpuinfo and add all flags supported there as possible USE-flags, no matter, whether they are currently used in some package or not. The list might still be extended when a new processor with new features comes out. If this is done properly, the user could easily write a script to set these flags appropriate for his current machine. One might even give portage a "--use-native" switch (or FEATURE=use-native) which sets the USE defaults corresponding to the current /proc/cpuinfo output. The latter is what typical desktop users probably would prefer: A complete analogue of CFLAGS=-march=native; just generating optimal code for his machine, without requiring the user to understand all the fancy details of his processor. To properly support packages which can choose the appropriate code at runtime, I suggest to add in addition an "all" flag which just compiles in all possible optimized code-paths.
[gentoo-dev] Re: sys-devel/gcc::mgorny up for testing
viv...@gmail.com wrote:
>
> - The project only has 20 commit, last one 8 months ago
> https://github.com/m-click/mcpdf.git
The "project" is just a few lines anyway: merely a wrapper to the library.
All the work happens in the itext library which AFAIK is the same project
(in different versions) for pdftk as for mcpdf.
Of course, the new library version might contain new bugs,
as your example seems to suggest (did pdftk work with this example?)
> find /g/portage/ -name '*.ebuild' -exec egrep 'sys-devel/gcc.*gcj' {} +
Searching also through all layman (remote) overlays with
$ eix -#Rry 'sys-devel/gcc[^ ]*gcj'
I found besides the mentioned
app-text/pdftk
dev-java/ecj-gcj
dev-java/gcj-jdk
one further dependency:
$ eix -Rvle imule
net-p2p/imule [1]
[...] irouter? ( sys-devel/gcc[gcj] ) [...]
P2P sharing software which connects through I2P and Kad network [...]
[1] "zugaina" layman/zugaina
[gentoo-dev] Re: sys-devel/gcc::mgorny up for testing
Michał Górny wrote: > > Have you tried mcpdf? I heard about it now for the first time. If I understand correctly, it uses the same library as pdfTK, only a somewhat later version (e.g. with improved unicode handling). The main difference seems to be that it does not insist on gcj but apparently should work with any java: I had suspected that it is the library which depends on some gcj internals and not just the front-end which consists just of a few lines, anyway. So, indeed, this looks like it should be a drop-in replacement, though I did not check whether the license of the updated library changed. Anyway, it would be nice to have it in the tree. ;) > but testing with your tricky PDFs I do not remember anymore. I just remember that several times, when some deadline approached, I had to resort to pdftk at my institute's Debian (in the lack of a gcj at home), even for simple tasks as merging pdftex output with some scanned PDF pages: neither pdfjam nor poppler did it satisfactory, though I do not remember the reasons.
[gentoo-dev] Re: sys-devel/gcc::mgorny up for testing
Michał Górny wrote: > > 2. No gcj support. [...] Not sure if anybody > actually uses it, and if it is actually useful for anything The most important consumer is app-text/pdftk Unfortunately, there is still no replacement for the latter which works as good, especially if you have tricky pdfs to process. Loosing gcj would therefore be a real loss.
[gentoo-dev] Re: RFC: News item regarding c++98 vs c++11
Mike Gilbert wrote: > > FYI, Chromium currently has a ban on using C++ 11 library features. I do not see how this would help to avoid the problem: If a function with the same name returns a different type in c++98 than in c++11, you could only avoid the problem by not using the function at all (neither in the c++11 code nor in the c++98 code, since each may refer to the "wrong" symbol). Of course, if all C++ dependencies are bundled, as is apparently the case for "original" chromium, the problem does not occur at all, because in this case everything just uses the c++11 library.
[gentoo-dev] Re: RFC: News item regarding c++98 vs c++11
Ian Stakenvicius wrote: > On 20/10/14 06:58 AM, Anthony G. Basile wrote: > >> I don't think we'll ever want to support a mixed abi system. > > Can we, even? Would it be a mixed-abi system or a multi-abi system? I am afraid, we *have* to, in the moment when at least one program adds -std=c++11 or -std=gnu++11 by itself (which AFAIK chromium does; also eix does, if it can). If not all C++ dependencies of that program (e.g. chromium) are compiled with -std=c++11, you "automatically" get ABI-mixture. (For eix, this is not the case, since eix has no dependencies on C++ programs). That's why I am not sure whether it is a good idea to discourage users from -std=gnu++11: It might in fact even increase stability of such programs which require the new language standard, and if all programs do compile with this flag, it is completely "safe".
[gentoo-dev] Re: RFC: News item regarding c++98 vs c++11
Anthony G. Basile wrote:
>> Since gcc-4.7 there is a -std=c++11 option, do not use it {+yet+}
>> since it breaks the ABI, resulting in a non-functional system.
>
> Yes. Eventually we'll have to clear the road for this.
Isn't Diego just starting a gcc-4.9 tinderbox run?
It might be worth to try with -std=gnu++11 immediately:
Most API incompatibilities which I experienced in my projects
were either name clashes (with symbols not existing yet in c++98)
or other trivial things (like some "char *" returns intead of
"const char *" returns etc.); cleanly written code is likely
to be upward compatible without any issues.
[gentoo-dev] Re: Are users forced to use PAM?
Nikos Chantziaras wrote: > In bug 524074 (https://bugs.gentoo.org/show_bug.cgi?id=524074), Joshua > Kinard mentioned that Gentoo cannot support systems where PAM isn't > installed. All my gentoo boxes run without pam from the very beginning (since many years). I hope that it will not be necessary to maintain ebuils in my overlay, should the USE=-pam option be removed from some packages. > (The issue at hand is that sudo links against -lshadow I suppose that this shouldn't be a hard fix, since previous sudo versions compiled well with USE=-pam, and there is no libshadow on my system (except for an xorg-module of that name, but I suppose that this is unrelated).
[gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
Diamond wrote: > > There's no git cp command. > git mv is just git rm + git add. I think there is a misunderstanding about how git works. If you are used to e.g. svn (I suppose with CVS it is similar) then it makes an important difference whether you use "svn cp A B" or "/bin/cp A B", because svn is file-centric: The latter loses history and duplicates data in the repository, while the former de-duplicates. git, OTOH is "blob"-centric: It cares only about content of the file, not where this content comes from; the information that you copy A to B is not stored, and it does not need to be stored: De-duplication happens automatically. In other words, there is no way for git to "see" that B originally was a copy of A, and normally there is also no such need. If you think that this information is important enough to be stored in the history, you have to make a commit where B is completely identical to A before you make further modifications for B (identical files have the same checksum and thus are immediately identified). If you make no such commit, git can guess only by similarities of the content that B originally was as a copy of A; this, of course, needs slightly longer than omitting such comparisons. For this reason, there is absolutely no reason to use git for file operations like cp or mv (as long as you add/remove the corresponding files from the index before you commit, of course).
[gentoo-dev] tox (was: maintainer-needed@ packages need you!)
This discussion has now become rather OT and does not belong to this list. Anyway, since there appear to be some misunderstandings concerning my previous remarks, I contribute one more post. Duncan <1i5t5.dun...@cox.net> wrote: >>> > > > Please don't. Not all communication partners are linux users. > > The (in)?famous network effect. A network grows in value based on the > number of users it has... Not only: You might convince your communication partners to switch. However, they must be able to switch on *their* systems easily (and not have to install linux or solve other technical obstacles). >>> > So it doesn't work [...] >>> >>> Can't agree with you here. I just tried tox (utox client) from >>> tox-overlay. Works like charm from the box [...] Maybe from utox to utox. From utox to antox (the android client) - which is the only case that I can reasonably test - I was only once able to send a friends request from antox to utox, but no message, and nothing in the opposite direction. After an antox upgrade, I cannot even use my login anymore, probably since the ID has changed due to upgrade... Also, you cannot be behind a restrictive NAT. For instance, one of my NAT routers cannot easily be configured to forward the exotic port, and then there is also the problem of UDP hole punching - not sure, whether this is solved in tox at all. I never tried to examine how skype does it, but opening ports in NAT routers was never necessary for skype, probably because it switches ports automatically as required. Moreover, since it uses a central server, there is no UDP punching problem. Such "magic" (like automatic port switching) is required if you really want to suggest it to users who are not computer-affine: It must really be "configuration free" - also in difficult situations - and work out of the box. Currently, it apparently works only on some only-linux clients, and only if the network is configured fine for them. >> As long as the vast majority of my contacts use Skype and Yahoo > > That was the point above about not everyone being a Linux user No, I think these are different points: My point was about whether people are *technically* able to switch (if all agree). The other is whether people are *socially* able to switch. Of course, both must be solved, but the former is the requirement for the latter, and tox still seems to have a long way to reach the former. >>> It should be noted that at least in Linux skype is much harder to >>> install and use since it requires pulseaudio But this concerns only Linux users who probably are computer-affine and can solve such problems (or know somebody who is). >>> So skype reqires its own LXC container set up Not really. There is a forum post how to use skype (*only* skype) with pulseaudio (once bug 519530 gets fixed, you can even do this without patching any ebuild or configuration.) To avoid a misunderstanding: I really oppose the non-privacy policy of skype, and hardly can await until tox is ready. However, the current state is that skype is without alteratives for many people. Moreover, suggesting tox to casual computer users too early - like now when many technical problems are not yet solved - would be rather contraproductive and actually kill tox: People who once had a bad experience will likely refuse to switch when tox *is* ready.
