Re: [gentoo-dev] RFC: Pre-GLEP: Security Project
On 03/15/2017 08:12 AM, Alexis Ballier wrote: > On Tue, 14 Mar 2017 19:55:44 -0400 > > > Agreed, but I was under the impression that sometimes sec. team was > waiting for cleanup to close a bug. If you've just done the analysis > that it is the only thing left, just do it and close the bug, instead > of pinging on the bug and re-do that analysis in a later pass. This > reduces context switches and makes everything more efficient :) > Indeed, although it should be noted that the amount of context switches is reduced by using whiteboards to tag status along with version information in summary, which is why it is important they follow security team policies for security bugs. In particular the whiteboard status allows for effective filtering when creating work-lists to reduce context switching (so if you're a maintainer starting a stablereq, feel free to update whiteboard from ebuild to stable!) -- Kristian Fiskerstrand OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 signature.asc Description: OpenPGP digital signature
[gentoo-portage-dev] [PATCH] depgraph: fix slot operator rebuild for llvm:0 to llvm:4 upgrade (bug 612772)
Fix check_reverse_dependencies to ignore dependencies of parent packages that could be uninstalled in order to solve a blocker conflict. This case is similar to the one from bug 584626, except that the relevant parent package is in an older slot which is blocked by a newer slot. In this case, the _upgrade_available method returns False, because the package in the older slot is the highest version version available for its slot. Therefore, a new _in_blocker_conflict method is needed to detect parent packages that cold be uninstalled. The included unit test fails without this fix. Since the _in_blocker_conflict method requires information that is collected by the _validate_blockers method, the _validate_blockers method now has to be called before the _process_slot_conflict and _slot_operator_trigger_reinstalls methods. X-Gentoo-bug: 612772 X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=612772 --- pym/_emerge/depgraph.py| 59 --- .../resolver/test_slot_operator_exclusive_slots.py | 109 + 2 files changed, 155 insertions(+), 13 deletions(-) create mode 100644 pym/portage/tests/resolver/test_slot_operator_exclusive_slots.py diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py index 1379b05..96b6d5f 100644 --- a/pym/_emerge/depgraph.py +++ b/pym/_emerge/depgraph.py @@ -387,7 +387,10 @@ class _dynamic_depgraph_config(object): # Contains only unsolvable Package -> Blocker edges self._unsolvable_blockers = digraph() # Contains all Blocker -> Blocked Package edges - self._blocked_pkgs = digraph() + # Do not initialized this until the depgraph _validate_blockers + # method is called, so that the _in_blocker_conflict method can + # assert that _validate_blockers has been called first. + self._blocked_pkgs = None # Contains world packages that have been protected from # uninstallation but may not have been added to the graph # if the graph is not complete yet. @@ -1466,9 +1469,22 @@ class depgraph(object): self._solve_non_slot_operator_slot_conflicts() + if not self._validate_blockers(): + # Blockers don't trigger the _skip_restart flag, since + # backtracking may solve blockers when it solves slot + # conflicts (or by blind luck). + raise self._unknown_internal_error() + + # Both _process_slot_conflict and _slot_operator_trigger_reinstalls + # can call _slot_operator_update_probe, which requires that + # self._dynamic_config._blocked_pkgs has been initialized by a + # call to the _validate_blockers method. for conflict in self._dynamic_config._package_tracker.slot_conflicts(): self._process_slot_conflict(conflict) + if self._dynamic_config._allow_backtracking: + self._slot_operator_trigger_reinstalls() + def _process_slot_conflict(self, conflict): """ Process slot conflict data to identify specific atoms which @@ -1829,9 +1845,12 @@ class depgraph(object): not self._frozen_config.excluded_pkgs. findAtomForPackage(parent, modified_use=self._pkg_use_enabled(parent)) and - self._upgrade_available(parent)): + (self._upgrade_available(parent) or + (parent.installed and self._in_blocker_conflict(parent: # This parent may be irrelevant, since an - # update is available (see bug 584626). + # update is available (see bug 584626), or + # it could be uninstalled in order to solve + # a blocker conflict (bug 612772). continue atom_set = InternalPackageSet(initial_atoms=(atom,), @@ -2125,6 +2144,24 @@ class depgraph(object): self._dynamic_config._need_restart = True + def _in_blocker_conflict(self, pkg): + """ + Check if pkg is involved in a blocker conflict. This method + only works after the _validate_blockers method has been called. + """ + + if self._dynamic_config._blocked_pkgs is None: + raise AssertionError( + '_in_blocker_conflict called
Re: [gentoo-portage-dev] [PATCH] depgraph: fix slot operator rebuild for llvm:0 to llvm:4 upgrade (bug 612772)
On Wed, Mar 15, 2017 at 9:39 PM, Brian Dolbecwrote: > On Wed, 15 Mar 2017 20:23:43 -0700 > Zac Medico wrote: > >> Fix check_reverse_dependencies to ignore dependencies of parent >> packages that could be uninstalled in order to solve a blocker >> conflict. This case is similar to the one from bug 584626, except >> that the relevant parent package is in an older slot which is blocked >> by a newer slot. In this case, the _upgrade_available method returns >> False, because the package in the older slot is the highest version >> version available for its slot. Therefore, a new _in_blocker_conflict >> method is needed to detect parent packages that cold be uninstalled. >> The included unit test fails without this fix. >> >> Since the _in_blocker_conflict method requires information that is >> collected by the _validate_blockers method, the _validate_blockers >> method now has to be called before the _process_slot_conflict and >> _slot_operator_trigger_reinstalls methods. >> >> X-Gentoo-bug: 612772 >> X-Gentoo-bug-url: https://bugs.gentoo.org/show_bug.cgi?id=612772 >> --- >> pym/_emerge/depgraph.py| 59 --- >> .../resolver/test_slot_operator_exclusive_slots.py | 109 >> + 2 files changed, 155 insertions(+), 13 >> deletions(-) create mode 100644 >> pym/portage/tests/resolver/test_slot_operator_exclusive_slots.py >> >> diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py >> index 1379b05..96b6d5f 100644 >> --- a/pym/_emerge/depgraph.py >> +++ b/pym/_emerge/depgraph.py >> @@ -387,7 +387,10 @@ class _dynamic_depgraph_config(object): >> # Contains only unsolvable Package -> Blocker edges >> self._unsolvable_blockers = digraph() >> # Contains all Blocker -> Blocked Package edges >> - self._blocked_pkgs = digraph() >> + # Do not initialized this until the depgraph > / > typo -d > > > Otherwise looks good :) Thanks, merged with typo fix: https://gitweb.gentoo.org/proj/portage.git/commit/?id=15e67f5516e0779d2cba37704c15b42193808197 -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH v3] movefile: support in-kernel file copying on Linux (bug 607868)
On Fri, 3 Mar 2017 18:18:50 -0800 Zac Medicowrote: > Perform in-kernel file copying when possible, and also support > reflinks and sparse files. If the optimized implementation > fails at runtime, gracefully fallback to a plain read/write > loop. > > Compile-time and run-time fallbacks are implemented, so that > any incompatiblities will be handled gracefully. For example, > if the code is compiled on a system that supports the > copy_file_range syscall, but at run-time an older kernel that > does not support this syscall is detected, it will be handled > gracefully. There are similar fallbacks for lack of lseek > SEEK_DATA and sendfile support. > > X-Gentoo-Bug: 607868 > X-Gentoo-Bug-Url: https://bugs.gentoo.org/show_bug.cgi?id=607868 > --- > [PATCH v3] Changes: > * Add function documentation comments > * Rename do_lseek function to do_lseek_data > * Fix do_lseek_data to handle sparse blocks at EOF, and >fix _reflink_linux_file_copy to call ftruncate > * Eliminate indirection in python copyfile function > * Cleaned up error variable handling > * Added lseek calls to ensure correct revovery from EINTR > * Added buf != NULL call before free(buf) > > pym/portage/tests/util/file_copy/__init__.py | 0 > pym/portage/tests/util/file_copy/__test__.py | 0 > pym/portage/tests/util/file_copy/test_copyfile.py | 71 > pym/portage/util/file_copy/__init__.py| 36 ++ > pym/portage/util/movefile.py | 4 +- > setup.py | 9 + > src/portage_util_file_copy_reflink_linux.c| 385 > ++ 7 files changed, 504 insertions(+), 1 > deletion(-) create mode 100644 I'd say go for it, merge. There's been no additional feedback or review -- Brian Dolbec
Re: [gentoo-portage-dev] Re: [PATCH v2] Change how the tmp file for the commit msg is made (bug 571546)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On Mon, 23 May 2016 12:43:04 +0200 Alexander Berntsenwrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > I see you have incorporated Zac's feedback by using shutil. You have > also incorporated my feedback on the commit message. I forgot to say > that the message body should also begin in the imperative mood, so it > should be "Change" not "Changes". But that's OK, we can just change > this when we merge it. My bad. > > The patch still looks OK to my eyes. I will leave it to Brian to ack, > as repoman has been his baby these past months. (Brian, I can merge it > if you don't have the time, but please give an ack.) > - -- > Alexander > berna...@gentoo.org > https://secure.plaimi.net/~alexander > -BEGIN PGP SIGNATURE- > Version: GnuPG v2 > > iQIcBAEBCgAGBQJXQt64AAoJENQqWdRUGk8BksQP/iV301h2bP32hHeAh0e+YUHH > NmUvwx1KSAnAjaL28rTEE8LzJcy+QDTdwhW9Y9x7gUD0fWeUuQpMXEnIQ7DIVmm/ > XfNI8M9bey90VLa8VNzIkfpHqeb5asvkHSLlyVVCb/BcE/wR23peRdW4F9LAhrAs > 0Mdz+g09Bjb5SawiTyaNig+YSid8MDWTDI+6oskB8NAhyaDqaUKGk/HfGqAG95FW > lbXIBE4dIQ2jFdzyn+3/rXWXBVCuXL1OufQPSiIFwfsU4j957SRo/cYlDqXyyQXD > mXXju2fU3cEk16Po+7v9wNp77xu2HdML7jgO75HdSDN5eDMJ3gyFlrlLw1Ztw9OA > v4+IXcbgGEGguN4Tz+DuY1+htGRnjBR5C2dj6y6rZU9HbIMTzJSxI/rQQUsQJ53j > Nd6++GJXqpskDhd+Zx1Gmhj5yV/h7Nc/YDUsT6CTsLNOzVJUy6acyhWqJRt46MAs > 9nw1c9xqoGLhCiWvAK5PZOz4/cfEUtEawUrQg4wtoPap05qzMx8fsk3rX9J7ifrq > rQlBVxwGQqjN9ykzDl2IkY8/W9sx8Nhe5/M5rmUCsk0KUzhcrkQNCOHdzSSU2jSM > Yimj4y+gBnizHIVn2+sAZ4vosTXic/zVKi5meYgSM4pBRAwD4GJgQ5vV//cWTTZl > 6QTa+hD/NQZAQd14COhc > =9N4L > -END PGP SIGNATURE- > Sorry, merged finally - -- Brian Dolbec -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQKTBAEBCgB9FiEEpdfHTggcxw20pKr1+70IcnWCDtgFAljJu0JfFIAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEE1 RDdDNzRFMDgxQ0M3MERCNEE0QUFGNUZCQkQwODcyNzU4MjBFRDgACgkQ+70IcnWC DtibPg/+IyqLtruXwdmDMuUyr+CJtnPlF2k6WD9JK5xp8j8yqAnZ3VhmvWQeLzL0 Gs0PkcquVuNKie6vVA+fgoyS73ji1KGX8c5ZO0eE5zU4tGEOAV3zL15crJ3hZvSl W4vlQlPJgGigld4ztb15nUwYn+O0w+K03uabYhSORBCHRvuDsKTB23+KYJaM9SPC upM4xA+ThQCdF4Vc8d1MiKz12Zmyfpkpmq5h2wHPGfkq5JEf5Hct879l278lD9cl TFrH6BXKKF92isr1bQx6u7Fmap6CT5PHmrKyipDPnI5GEqRQY2j4BuG67DFvQdBp /OwfOKZWOC+lRkGUFVV7WT0EBWBdGTsEe4KWE8pv8wWYrZi8xxqzJPhUcbxyS+uB 42Ry1OM0XkPd2pI+S/yVAJR+QVQDspvRrkOxN850Sf1aJeyCh8h5zCpb/oY0Bhvg siKdjoHd7/xHp0CATiMdlfYQsZJNhRs/f2ks4S+LH0MZr4VabclIG9SN19sHWR00 +TC6mvwCFoQpxDS+VNby4iWRL2mnLgCr3FYCVapbA6Mypr/I2nvfBwQwb3OYlz39 VWD/yDN18uKYk12Gm6VXRQaJImPr4H4+gfXp1zd/FdBkIV4oKpVo03Xp2qcIIzR9 fbI+hUbMVGlcthS4hbxQHJqPCp1PXD/dAoFL/f7yNK/AJiM85ic= =mt0p -END PGP SIGNATURE-
Re: [gentoo-portage-dev] [PATCH v3] movefile: support in-kernel file copying on Linux (bug 607868)
On Wed, Mar 15, 2017 at 3:25 PM, Brian Dolbecwrote: > On Fri, 3 Mar 2017 18:18:50 -0800 > Zac Medico wrote: > >> Perform in-kernel file copying when possible, and also support >> reflinks and sparse files. If the optimized implementation >> fails at runtime, gracefully fallback to a plain read/write >> loop. >> >> Compile-time and run-time fallbacks are implemented, so that >> any incompatiblities will be handled gracefully. For example, >> if the code is compiled on a system that supports the >> copy_file_range syscall, but at run-time an older kernel that >> does not support this syscall is detected, it will be handled >> gracefully. There are similar fallbacks for lack of lseek >> SEEK_DATA and sendfile support. >> >> X-Gentoo-Bug: 607868 >> X-Gentoo-Bug-Url: https://bugs.gentoo.org/show_bug.cgi?id=607868 >> --- >> [PATCH v3] Changes: >> * Add function documentation comments >> * Rename do_lseek function to do_lseek_data >> * Fix do_lseek_data to handle sparse blocks at EOF, and >>fix _reflink_linux_file_copy to call ftruncate >> * Eliminate indirection in python copyfile function >> * Cleaned up error variable handling >> * Added lseek calls to ensure correct revovery from EINTR >> * Added buf != NULL call before free(buf) >> >> pym/portage/tests/util/file_copy/__init__.py | 0 >> pym/portage/tests/util/file_copy/__test__.py | 0 >> pym/portage/tests/util/file_copy/test_copyfile.py | 71 >> pym/portage/util/file_copy/__init__.py| 36 ++ >> pym/portage/util/movefile.py | 4 +- >> setup.py | 9 + >> src/portage_util_file_copy_reflink_linux.c| 385 >> ++ 7 files changed, 504 insertions(+), 1 >> deletion(-) create mode 100644 > > > I'd say go for it, merge. There's been no additional feedback or review Thanks, merged: https://gitweb.gentoo.org/proj/portage.git/commit/?id=8ab5c8835931fd9ec098dbf4c5f416eb32e4a3a4 -- Thanks, Zac
Re: [gentoo-portage-dev] [PROPOSAL] Don't split user visible messages across multiple lines
On Mon, 27 Feb 2017 20:33:28 +0200 Alexandru Eliseiwrote: > I was working on emerge --sync and my test repo configuration > generated several warning messages. Finding the exact location where > the warnings was generated was surprisingly difficult because the > message string was split across three lines due to the 80 character > line limit and I had to grep for different patterns until I found one > that matched part of a line. > > I propose that messages that are visible to the user never be split > across multiple lines. This is also the coding style convention used > by the linux kernel: > https://www.kernel.org/doc/html/latest/process/coding-style.html#breaking-long-lines-and-strings > > If accepted, this could go in the DEVELOPING file. > That could be pretty hard to do for all messages. Especially messages with embedded data eg: "%s is missing %s required use flag..." % ('sys-apps/foo', 'bar') I know I don't always enforce the line length for a few characters, also when clarity is more important than line length. We could also increase the max. line length to something like 120 or 130. -- Brian Dolbec
[gentoo-dev] slight behaviour change of clock service in openrc-0.24.1
All, this is a heads-up more than anything; I don't think it is major. I wrote an entry about it in the NEWS.md file of openrc-0.21.4. There is no action for users, so I don't think it is a newsitem, but there might be action for developers. In a nutshell, if you have a service that should be added to the boot runlevel and it needs to start after the clock service, you may need to specify "after clock" in the depend() function. The short version of the reason for this is I had to remove "before *" from the clock service scripts to avoid deadlocks if rc_parallel=yes [1]. William [1] https://bugs.gentoo.org/show_bug.cgi?id=612618 signature.asc Description: Digital signature
Re: [gentoo-dev] RFC: Pre-GLEP: Security Project
On Tue, 14 Mar 2017 19:55:44 -0400 Yury Germanwrote: > > On Mar 12, 2017, at 4:14 AM, Alexis Ballier > > wrote: > > > > > > Also, it'd be nice to have something more formal for sec. cleanup: > > "After 30 days a sec. issue has been fixed, sec. team is free to > > cleanup old vulnerable versions.". I've seen too much pings by sec. > > team members on old bugs for this and they could have spent the same > > amount of time simply doing it instead. > > > Alexis, here is a problem that I have noticed over the years. > Everyone is short on time, but if the developers do not step in (and > only some) and clean up the packages then we might as well make this > another job for Security team as everyone will just leave it to > security. > > Security looks at every security bug, and needs to do a lot of things > behind the scenes. GLSA writing, look for CVE’s if not there, assign > them to bugs in the CVE system used for GLSA’s. If no CVE’s exist > communicate with upstream to see if it was requested, if not > requested request it on their behalf and work with MITRE in getting > it assigned. When you multiply that time over the 100+ security bugs > at any time. Cleanup is not a 5 second thing as for me typing three > characters to have that bug be submitted with that comment. > > The maintainer also knows the package, dependencies, other bugs > filed, etc. Removing things for your packages might be simple, but it > is not the same across all packages and that is the reason we ask the > Maintainers to take an active step in cleaning up. Agreed, but I was under the impression that sometimes sec. team was waiting for cleanup to close a bug. If you've just done the analysis that it is the only thing left, just do it and close the bug, instead of pinging on the bug and re-do that analysis in a later pass. This reduces context switches and makes everything more efficient :)
Re: [gentoo-dev] Unused profiles
W dniu 14.03.2017, wto o godzinie 17∶19 -0700, użytkownik Jack Morgan napisał: > > On 01/19/17 09:08, Michał Górny wrote: > > (CC-ing a lot of potentially interested teams) > > > > Hi, everyone. > > > > I've did a quick sweep of profiles/ directory for unused profiles. > > Unused means: not used as parent of any other profile, and not listed > > in profiles.desc. > > > > Note that the list does not include parents of the listed profiles > > since they are still used by them ;-). > > > > Please let me know which of the profiles are worth keeping, and which > > can be killed entirely. If you prefer to keep them, please consider > > adding them to profiles.desc so that our users can officially use them. > > > > List follows > > > > Just to follow up, for all Sparc profiles > > I use two profiles.. > default/linux/sparc/13.0 * > default/linux/sparc/13.0/desktop > > I'm happy to remove these profiles... > default/linux/sparc/13.0/desktop/gnome > default/linux/sparc/13.0/desktop/gnome/systemd > default/linux/sparc/13.0/developer > > anyone else? Would you do the removal or should I? -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part