Re: [gentoo-portage-dev] [PATCH] lib/*: exempt two core files from ungrouped-imports check
On 8/6/20 5:03 PM, Aaron Bauman wrote: > * These will take some time to refactor/address if at all possible. > * Turn on ungrouped-imports check for entire repo > > Signed-off-by: Aaron Bauman > --- > lib/portage/__init__.py | 1 + > lib/portage/checksum.py | 15 --- > pylintrc| 1 + > 3 files changed, 10 insertions(+), 7 deletions(-) Thanks, merged: https://gitweb.gentoo.org/proj/portage.git/commit/?id=0dffa633987a49c1a1d7e95edfe6afeb6269d1e5 -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH] *: drop all import __future__ statements
On 8/6/20 6:12 PM, Aaron Bauman wrote: > * We only support Py3 now. > > Signed-off-by: Aaron Bauman > --- > bin/archive-conf | 4 +-- > bin/check-implicit-pointer-usage.py | 2 -- > bin/clean_locks | 4 +-- > bin/dispatch-conf | 4 +-- > bin/dohtml.py | 4 +-- > bin/doins.py | 2 -- > bin/ebuild| 4 +-- > bin/egencache | 4 +-- > bin/emaint| 4 +-- > bin/emerge| 4 +-- > bin/env-update| 4 +-- > bin/fixpackages | 4 +-- > bin/glsa-check| 4 +-- > bin/portageq | 4 +-- > bin/quickpkg | 4 +-- > bin/regenworld| 4 +-- > lib/_emerge/Scheduler.py | 2 -- > lib/_emerge/UserQuery.py | 2 -- > lib/_emerge/actions.py| 2 -- > lib/_emerge/chk_updated_cfg_files.py | 2 -- > lib/_emerge/countdown.py | 4 +-- > lib/_emerge/depgraph.py | 2 -- > lib/_emerge/help.py | 4 +-- > lib/_emerge/main.py | 2 -- > lib/_emerge/post_emerge.py| 4 +-- > lib/_emerge/resolver/circular_dependency.py | 8 +++--- > lib/_emerge/resolver/package_tracker.py | 4 +-- > lib/_emerge/resolver/slot_collision.py| 3 +-- > lib/_emerge/unmerge.py| 2 -- > lib/portage/_emirrordist/FetchTask.py | 2 -- > lib/portage/_global_updates.py| 4 +-- > lib/portage/_sets/__init__.py | 2 -- > lib/portage/_sets/dbapi.py| 2 -- > lib/portage/_sets/libs.py | 4 +-- > lib/portage/cache/anydbm.py | 5 ++-- > lib/portage/cache/sqlite.py | 3 +-- > lib/portage/cvstree.py| 2 -- > lib/portage/dbapi/vartree.py | 2 -- > lib/portage/dispatch_conf.py | 5 +--- > lib/portage/elog/mod_echo.py | 4 +-- > lib/portage/emaint/main.py| 4 +-- > lib/portage/glsa.py | 27 +-- > lib/portage/localization.py | 4 +-- > lib/portage/metadata.py | 2 -- > lib/portage/module.py | 5 +--- > lib/portage/news.py | 2 -- > lib/portage/output.py | 2 -- > lib/portage/package/ebuild/fetch.py | 2 -- > lib/portage/sync/controller.py| 3 --- > lib/portage/sync/old_tree_timestamp.py| 4 +-- > lib/portage/tests/__init__.py | 4 +-- > lib/portage/tests/util/test_xattr.py | 4 +-- > .../util/_dyn_libs/display_preserved_libs.py | 6 ++--- > lib/portage/util/_eventloop/EventLoop.py | 5 +--- > .../util/_eventloop/PollSelectAdapter.py | 5 ++-- > lib/portage/util/_xattr.py| 4 +-- > lib/portage/util/locale.py| 4 +-- > lib/portage/util/movefile.py | 2 -- > repoman/bin/repoman | 5 ++-- > repoman/lib/repoman/actions.py| 4 +-- > repoman/lib/repoman/errors.py | 2 -- > repoman/lib/repoman/gpg.py| 2 -- > repoman/lib/repoman/main.py | 5 ++-- > repoman/lib/repoman/metadata.py | 2 -- > .../lib/repoman/modules/commit/repochecks.py | 2 -- > .../lib/repoman/modules/scan/ebuild/ebuild.py | 3 +-- > repoman/lib/repoman/modules/vcs/settings.py | 3 +-- > repoman/lib/repoman/modules/vcs/vcs.py| 3 +-- > repoman/lib/repoman/profile.py| 2 -- > repoman/lib/repoman/scanner.py| 6 ++--- > repoman/lib/repoman/tests/__init__.py | 4 +-- > repoman/lib/repoman/utilities.py | 4 +-- > repoman/runtests | 4 +-- > repoman/setup.py | 4 +-- > runtests | 4 +-- > setup.py | 2 -- > 76 files changed, 74 insertions(+), 212 deletions(-) Thanks, merged: https://gitweb.gentoo.org/proj/portage.git/commit/?id=d695d866e4e7b2ab79264508f1520f89a5585e04 -- Thanks, Zac signature.asc Description: OpenPGP digital signature
[gentoo-portage-dev] [PATCH] *: drop all import __future__ statements
* We only support Py3 now. Signed-off-by: Aaron Bauman --- bin/archive-conf | 4 +-- bin/check-implicit-pointer-usage.py | 2 -- bin/clean_locks | 4 +-- bin/dispatch-conf | 4 +-- bin/dohtml.py | 4 +-- bin/doins.py | 2 -- bin/ebuild| 4 +-- bin/egencache | 4 +-- bin/emaint| 4 +-- bin/emerge| 4 +-- bin/env-update| 4 +-- bin/fixpackages | 4 +-- bin/glsa-check| 4 +-- bin/portageq | 4 +-- bin/quickpkg | 4 +-- bin/regenworld| 4 +-- lib/_emerge/Scheduler.py | 2 -- lib/_emerge/UserQuery.py | 2 -- lib/_emerge/actions.py| 2 -- lib/_emerge/chk_updated_cfg_files.py | 2 -- lib/_emerge/countdown.py | 4 +-- lib/_emerge/depgraph.py | 2 -- lib/_emerge/help.py | 4 +-- lib/_emerge/main.py | 2 -- lib/_emerge/post_emerge.py| 4 +-- lib/_emerge/resolver/circular_dependency.py | 8 +++--- lib/_emerge/resolver/package_tracker.py | 4 +-- lib/_emerge/resolver/slot_collision.py| 3 +-- lib/_emerge/unmerge.py| 2 -- lib/portage/_emirrordist/FetchTask.py | 2 -- lib/portage/_global_updates.py| 4 +-- lib/portage/_sets/__init__.py | 2 -- lib/portage/_sets/dbapi.py| 2 -- lib/portage/_sets/libs.py | 4 +-- lib/portage/cache/anydbm.py | 5 ++-- lib/portage/cache/sqlite.py | 3 +-- lib/portage/cvstree.py| 2 -- lib/portage/dbapi/vartree.py | 2 -- lib/portage/dispatch_conf.py | 5 +--- lib/portage/elog/mod_echo.py | 4 +-- lib/portage/emaint/main.py| 4 +-- lib/portage/glsa.py | 27 +-- lib/portage/localization.py | 4 +-- lib/portage/metadata.py | 2 -- lib/portage/module.py | 5 +--- lib/portage/news.py | 2 -- lib/portage/output.py | 2 -- lib/portage/package/ebuild/fetch.py | 2 -- lib/portage/sync/controller.py| 3 --- lib/portage/sync/old_tree_timestamp.py| 4 +-- lib/portage/tests/__init__.py | 4 +-- lib/portage/tests/util/test_xattr.py | 4 +-- .../util/_dyn_libs/display_preserved_libs.py | 6 ++--- lib/portage/util/_eventloop/EventLoop.py | 5 +--- .../util/_eventloop/PollSelectAdapter.py | 5 ++-- lib/portage/util/_xattr.py| 4 +-- lib/portage/util/locale.py| 4 +-- lib/portage/util/movefile.py | 2 -- repoman/bin/repoman | 5 ++-- repoman/lib/repoman/actions.py| 4 +-- repoman/lib/repoman/errors.py | 2 -- repoman/lib/repoman/gpg.py| 2 -- repoman/lib/repoman/main.py | 5 ++-- repoman/lib/repoman/metadata.py | 2 -- .../lib/repoman/modules/commit/repochecks.py | 2 -- .../lib/repoman/modules/scan/ebuild/ebuild.py | 3 +-- repoman/lib/repoman/modules/vcs/settings.py | 3 +-- repoman/lib/repoman/modules/vcs/vcs.py| 3 +-- repoman/lib/repoman/profile.py| 2 -- repoman/lib/repoman/scanner.py| 6 ++--- repoman/lib/repoman/tests/__init__.py | 4 +-- repoman/lib/repoman/utilities.py | 4 +-- repoman/runtests | 4 +-- repoman/setup.py | 4 +-- runtests | 4 +-- setup.py | 2 -- 76 files changed, 74 insertions(+), 212 deletions(-) diff --git a/bin/archive-conf b/bin/archive-conf index 8341ffe73..bfc54a629 100755 --- a/bin/archive-conf +++ b/bin/archive-conf @@ -1,5 +1,5 @@ #!/usr/bin/python -b -# Copyright 1999-2014 Gentoo Foundation +# Copyright 1999-2020 Gentoo Authors # Distributed under the terms of the GNU General Public License v2 # @@ -9,8 +9,6 @@ # Jeremy Wohl's dispatch-conf script and the portage chkcontents script. # -from __future__ import print_function - import sys from os import path as osp diff --git a/bin/check-implicit-pointer-usage.py b/bin/check-implicit-pointer-usage.py index 868e4b3c8..3914e038d 100755
[gentoo-portage-dev] [PATCH] lib/*: exempt two core files from ungrouped-imports check
* These will take some time to refactor/address if at all possible.
* Turn on ungrouped-imports check for entire repo
Signed-off-by: Aaron Bauman
---
lib/portage/__init__.py | 1 +
lib/portage/checksum.py | 15 ---
pylintrc| 1 +
3 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/lib/portage/__init__.py b/lib/portage/__init__.py
index 73bb1ed0d..916c93510 100644
--- a/lib/portage/__init__.py
+++ b/lib/portage/__init__.py
@@ -1,5 +1,6 @@
# Copyright 1998-2020 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
+# pylint: disable=ungrouped-imports
VERSION = "HEAD"
diff --git a/lib/portage/checksum.py b/lib/portage/checksum.py
index d4a75c058..b5dae9672 100644
--- a/lib/portage/checksum.py
+++ b/lib/portage/checksum.py
@@ -1,20 +1,21 @@
# checksum.py -- core Portage functionality
# Copyright 1998-2020 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2
+# pylint: disable=ungrouped-imports
-import portage
-from portage.const import PRELINK_BINARY, HASHING_BLOCKSIZE
-from portage.localization import _
-from portage import os
-from portage import _encodings
-from portage import _unicode_decode, _unicode_encode
import errno
import functools
import hashlib
+import portage
import stat
import subprocess
import tempfile
+from portage import _encodings, _unicode_decode, _unicode_encode
+from portage import os
+from portage.const import HASHING_BLOCKSIZE, PRELINK_BINARY
+from portage.localization import _
+
# Summary of all available hashes and their implementations,
# most preferred first. Please keep this in sync with logic below.
@@ -32,7 +33,7 @@ import tempfile
# SHA3_512: hashlib (3.6+), pysha3, pycrypto
-#dict of all available hash functions
+# Dict of all available hash functions
hashfunc_map = {}
hashorigin_map = {}
diff --git a/pylintrc b/pylintrc
index 9b0975329..f2aadf14f 100644
--- a/pylintrc
+++ b/pylintrc
@@ -28,6 +28,7 @@ enable=
trailing-newlines,
trailing-whitespace,
unexpected-line-ending-format,
+ungrouped-imports,
unnecessary-semicolon,
unused-import,
useless-import-alias,
--
2.28.0
[gentoo-portage-dev] [PATCH] lib/portage/xml/metadata.py: fix ungrouped-imports w/refactor
* This drops the import for cElementTree as it is deprecated. See:
https://docs.python.org/3/library/xml.etree.elementtree.html
* Drop the error checking as these modules are a part of stdlib.
* Additionally, the pyexpat issue was fixed in 3.2. See:
https://bugs.python.org/issue14988
Signed-off-by: Aaron Bauman
---
lib/portage/xml/metadata.py | 29 -
1 file changed, 8 insertions(+), 21 deletions(-)
diff --git a/lib/portage/xml/metadata.py b/lib/portage/xml/metadata.py
index 646edb5b9..808ba2246 100644
--- a/lib/portage/xml/metadata.py
+++ b/lib/portage/xml/metadata.py
@@ -31,29 +31,16 @@
__all__ = ('MetaDataXML', 'parse_metadata_use')
-try:
- import xml.etree.cElementTree as etree
-except (SystemExit, KeyboardInterrupt):
- raise
-except (ImportError, SystemError, RuntimeError, Exception):
- # broken or missing xml support
- # https://bugs.python.org/issue14988
- import xml.etree.ElementTree as etree
-
-try:
- from xml.parsers.expat import ExpatError
-except (SystemExit, KeyboardInterrupt):
- raise
-except (ImportError, SystemError, RuntimeError, Exception):
- ExpatError = SyntaxError
-
import re
-import xml.etree.ElementTree
+import xml.etree.ElementTree as etree
+
+from xml.parsers.expat import ExpatError
+
from portage import _encodings, _unicode_encode
from portage.util import cmp_sort_key, unique_everseen
-class _MetadataTreeBuilder(xml.etree.ElementTree.TreeBuilder):
+class _MetadataTreeBuilder(etree.TreeBuilder):
"""
Implements doctype() as required to avoid deprecation warnings with
Python >=2.7.
@@ -198,8 +185,8 @@ class MetaDataXML:
try:
self._xml_tree =
etree.parse(_unicode_encode(metadata_xml_path,
- encoding=_encodings['fs'], errors='strict'),
-
parser=etree.XMLParser(target=_MetadataTreeBuilder()))
+ encoding = _encodings['fs'], errors='strict'),
+ parser =
etree.XMLParser(target=_MetadataTreeBuilder()))
except ImportError:
pass
except ExpatError as e:
@@ -238,7 +225,7 @@ class MetaDataXML:
try:
self._herdstree =
etree.parse(_unicode_encode(self._herds_path,
encoding=_encodings['fs'],
errors='strict'),
-
parser=etree.XMLParser(target=_MetadataTreeBuilder()))
+ parser =
etree.XMLParser(target=_MetadataTreeBuilder()))
except (ImportError, IOError, SyntaxError):
return None
--
2.28.0
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On Thu, Aug 6, 2020 at 4:19 PM Michał Górny wrote: > > On Thu, 2020-08-06 at 23:03 +0300, Joonas Niilola wrote: > > On 8/6/20 10:58 PM, Thomas Deutschmann wrote: > > > Well, the purpose of this is to educate and avoid problems for > > > headless/server users. But if so many devs seem to care about pushing > > > maybe unrelated information and believe that avoiding that has much more > > > value than avoid a problem like an unbootable system for just a few > > > people (and for headless/servers this is a major problem in case you > > > cannot trigger remote reboot)... ¯\_(ツ)_/¯ > > > > > Yeah let's break some setups and make people change distributions instead. > > > > I'd support showing it. Weren't we all taught that too much > > communication is better than no communication? > > > > That's actually bullshit. Too much noise leads to people stopping to > read stuff, and losing important info as a result. Compare: our mailing > lists. Well, we could solve that problem and the Foundation funding challenges by switching the mailing list to a paid-superchat system. The only thing that might bother some would be having to give me a slot on the sponsor's page, but then again I would be paying your salary at that point. :) More seriously, I think the simplest compromise is to just display the news item to those with the relevant genkernel versions installed. While the general principle of using UUIDs and such in boot lines is important, this is better placed in the handbook, wiki, and so on. The news should focus on what is actually changing, IMO. -- Rich
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On Thu, 2020-08-06 at 23:03 +0300, Joonas Niilola wrote: > On 8/6/20 10:58 PM, Thomas Deutschmann wrote: > > Well, the purpose of this is to educate and avoid problems for > > headless/server users. But if so many devs seem to care about pushing > > maybe unrelated information and believe that avoiding that has much more > > value than avoid a problem like an unbootable system for just a few > > people (and for headless/servers this is a major problem in case you > > cannot trigger remote reboot)... ¯\_(ツ)_/¯ > > > Yeah let's break some setups and make people change distributions instead. > > I'd support showing it. Weren't we all taught that too much > communication is better than no communication? > That's actually bullshit. Too much noise leads to people stopping to read stuff, and losing important info as a result. Compare: our mailing lists. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On 2020-08-06 21:50, Jaco Kroon wrote: > Can you detect in the runscript that this will trigger and issue a cold > reboot instead if this would trigger? > > Having never used kexec before ... I may well be missing the point. But > I'd rather have the system issue a cold reboot if kexec (which sounds > really cool in principle) stands any chance of failing. It's software. Of course you can do everything you can do in POSIX shell script :) But I doubt that something like this is practicable -- we also should avoid overengineering. Like I was thinking by myself if we should teach kexec runscript to return persistent name instead (utilizing lsblk for example) but this will raises question like what to do if tools aren't available and maybe user's start environment can't even handle root=UUID=... value :/ -- Regards, Thomas Deutschmann / Gentoo Linux Developer C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On 8/5/20 5:02 AM, Thomas Deutschmann wrote: > Hi, > > please review the news item below: > Not all arches support --reuse-cmdline btw. It may be only x86 which supports it. This looks like a candidate wiki page or a word of warning in a handbook, not a news item.
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On 8/6/20 10:58 PM, Thomas Deutschmann wrote: > Well, the purpose of this is to educate and avoid problems for > headless/server users. But if so many devs seem to care about pushing > maybe unrelated information and believe that avoiding that has much more > value than avoid a problem like an unbootable system for just a few > people (and for headless/servers this is a major problem in case you > cannot trigger remote reboot)... ¯\_(ツ)_/¯ > Yeah let's break some setups and make people change distributions instead. I'd support showing it. Weren't we all taught that too much communication is better than no communication? -- juippis signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On 2020-08-06 19:20, Aaron Bauman wrote: > Wait, changes were made to genkernel to switch from mdev to (e)udev > which causes breakage, but it is *not* an issue with genkernel? Exactly. This failure can happen with genkernel version created 15 years ago, with new genkernel-4.1 which switched device manager or even with dracut -- the mistake is using non-permanent device names for things like root. I assume that most user don't do that. At least their default boot entry in /boot/extlinux/extlinux.cfg or via /etc/default/grub will have a permanent name -- but the problem are tools/scripts appending to that existing command-line. They will overwrite a good value... And it's even more a problem because even when you notice "Ah, something is appending root argument" you won't question that because the value you notice matches your expectation from POV of current running system. So you have to realize that this is a non-permanent value which could be different on next boot because you did X which caused and offset in numbering for example... > Aside from this, do we have any evidence or bugs validating that users > experience breakage with randomly named boot devices in kexec? > > It is great that you found an issue, but why try and be agnostic as to > which one caused the issue? It looks worse that we cannot simply say: > > "genkernel changed for the better and things *may* break now... please > read this!" > > Instead, we are pushing a news item to a lot of people simply because we > *assume* it may be an issue for others with no evidence. Well, the purpose of this is to educate and avoid problems for headless/server users. But if so many devs seem to care about pushing maybe unrelated information and believe that avoiding that has much more value than avoid a problem like an unbootable system for just a few people (and for headless/servers this is a major problem in case you cannot trigger remote reboot)... ¯\_(ツ)_/¯ -- Regards, Thomas Deutschmann / Gentoo Linux Developer C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
Hi, On 2020/08/06 21:45, Thomas Deutschmann wrote: > On 2020-08-06 20:56, Rich Freeman wrote: >> Has anything even changed with kexec? Or is this an issue that has >> been an issue for many years in kexec, that will suddenly become an >> issue in genkernel? In that case it is news from a genkernel >> perspective, and something anybody with a correctly-booting system >> fixed a long time ago if they're using kexec. > Well, first it was an annoyance I became aware of myself when I noticed > a system having dozen of root arguments in kernel command-line. I think > we even talked about this in #gentoo-base a while ago: > >> # cat /proc/cmdline >> domdadm dolvm dosshd crypt_root=UUID=a-b-c-d root=UUID=e-f-g-h rootfs=xfs >> scandelay=3 root_trim=yes vga=0x317 gk.log.keep=/var/log/genkernel-boot.log >> root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 >> root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 >> root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 >> root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 >> root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 >> root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 > ...you can count how often the system was rebooted using kexec ;) > > This week I also received a bug report from a user who upgraded to > genkernel-4.1 where first reboot failed but everything was working after > a reset (cold boot). > > During my investigation I was able to trigger this by myself, for > example when I close and re-open LVM volume and trigger new symlink for > re-opened root volume (this sounds like a non-typical use case for some > people but when dealing LVM backups/snapshots it's not that uncommon). > > So this became a bug for me in our kexec runscript which I fixed > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4860fce5434f46d90e913ff10515a9a256fc6c6a > and already warn about > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=61c03ffab76740c0420e3c8a3185d047d461f7a7 Can you detect in the runscript that this will trigger and issue a cold reboot instead if this would trigger? Having never used kexec before ... I may well be missing the point. But I'd rather have the system issue a cold reboot if kexec (which sounds really cool in principle) stands any chance of failing. Kind Regards, Jaco
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On 2020-08-06 20:56, Rich Freeman wrote: > Has anything even changed with kexec? Or is this an issue that has > been an issue for many years in kexec, that will suddenly become an > issue in genkernel? In that case it is news from a genkernel > perspective, and something anybody with a correctly-booting system > fixed a long time ago if they're using kexec. Well, first it was an annoyance I became aware of myself when I noticed a system having dozen of root arguments in kernel command-line. I think we even talked about this in #gentoo-base a while ago: > # cat /proc/cmdline > domdadm dolvm dosshd crypt_root=UUID=a-b-c-d root=UUID=e-f-g-h rootfs=xfs > scandelay=3 root_trim=yes vga=0x317 gk.log.keep=/var/log/genkernel-boot.log > root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 > root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 > root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 > root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 > root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 > root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 root=/dev/dm-2 ...you can count how often the system was rebooted using kexec ;) This week I also received a bug report from a user who upgraded to genkernel-4.1 where first reboot failed but everything was working after a reset (cold boot). During my investigation I was able to trigger this by myself, for example when I close and re-open LVM volume and trigger new symlink for re-opened root volume (this sounds like a non-typical use case for some people but when dealing LVM backups/snapshots it's not that uncommon). So this became a bug for me in our kexec runscript which I fixed https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4860fce5434f46d90e913ff10515a9a256fc6c6a and already warn about https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=61c03ffab76740c0420e3c8a3185d047d461f7a7 But note: This is not even a kexec issue per-se. If you use kexec on your own with your scripts (which is also not that uncommon) you maybe also appending additional root argument which has the potential to cause boot failures in case you are using non-permanent device names and something will be different in start environment. -- Regards, Thomas Deutschmann / Gentoo Linux Developer C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On Thu, Aug 6, 2020 at 1:41 PM Mike Gilbert wrote: > > On Thu, Aug 6, 2020 at 11:59 AM Thomas Deutschmann wrote: > > > > On 2020-08-06 17:44, Michał Górny wrote: > > > I'm not sure if you've noticed but there are people actively working > > > towards removing stale news items and trying not to dump everything > > > on once on a user freshly installing the system. Don't you consider > > > this a worthwhile goal? > > > > I don't see how this is conflicting. > > > > This news item can probably go away after 1-2 years. > > > > But for now, people who were just lucky will probably trigger this when > > upgrading to genkernel-4.1 on their first reboot due to switched device > > manager. > > > > But again: It's not a genkernel issue, so displaying that only for > > people who have genkernel installed would miss a bunch of users. > > I would guess that most users do not utilize kexec at all, and this > news item is irrelevant for them. > > Personally, I agree that this is not worth spamming every Gentoo user. > Has anything even changed with kexec? Or is this an issue that has been an issue for many years in kexec, that will suddenly become an issue in genkernel? In that case it is news from a genkernel perspective, and something anybody with a correctly-booting system fixed a long time ago if they're using kexec. -- Rich
[gentoo-portage-dev] [PATCH] lib/*: Fix ungrouped-imports
* This fixes ungrouped imports for a portion of the repo. The remaining fixes will come in another batch of atomic commits as the imports are buried behind conditionals, are __futures__ that may be able to go away, etc * Per the above, check will not be enabled yet Signed-off-by: Aaron Bauman --- lib/_emerge/FakeVartree.py | 7 +++--- lib/_emerge/FifoIpcDaemon.py| 2 +- lib/_emerge/MetadataRegen.py| 6 +++-- lib/portage/emaint/modules/move/move.py | 6 +++-- lib/portage/metadata.py | 16 +++-- lib/portage/sync/modules/rsync/rsync.py | 31 + lib/portage/util/_async/BuildLogger.py | 4 ++-- 7 files changed, 40 insertions(+), 32 deletions(-) diff --git a/lib/_emerge/FakeVartree.py b/lib/_emerge/FakeVartree.py index 37349bcdd..fa6c17beb 100644 --- a/lib/_emerge/FakeVartree.py +++ b/lib/_emerge/FakeVartree.py @@ -1,12 +1,14 @@ # Copyright 1999-2020 Gentoo Authors # Distributed under the terms of the GNU General Public License v2 +import portage import warnings -import portage -from portage import os from _emerge.Package import Package from _emerge.PackageVirtualDbapi import PackageVirtualDbapi +from _emerge.resolver.DbapiProvidesIndex import PackageDbapiProvidesIndex + +from portage import os from portage.const import VDB_PATH from portage.dbapi.vartree import vartree from portage.dep._slot_operator import find_built_slot_operator_atoms @@ -14,7 +16,6 @@ from portage.eapi import _get_eapi_attrs from portage.exception import InvalidData, InvalidDependString from portage.update import grab_updates, parse_updates, update_dbentries from portage.versions import _pkg_str -from _emerge.resolver.DbapiProvidesIndex import PackageDbapiProvidesIndex class FakeVardbGetPath: diff --git a/lib/_emerge/FifoIpcDaemon.py b/lib/_emerge/FifoIpcDaemon.py index a21e94db6..22aa6f9fc 100644 --- a/lib/_emerge/FifoIpcDaemon.py +++ b/lib/_emerge/FifoIpcDaemon.py @@ -1,9 +1,9 @@ # Copyright 2010-2020 Gentoo Authors # Distributed under the terms of the GNU General Public License v2 +from _emerge.AbstractPollTask import AbstractPollTask from portage import os -from _emerge.AbstractPollTask import AbstractPollTask from portage.cache.mappings import slot_dict_class class FifoIpcDaemon(AbstractPollTask): diff --git a/lib/_emerge/MetadataRegen.py b/lib/_emerge/MetadataRegen.py index 8eb110a46..dc3ef7e57 100644 --- a/lib/_emerge/MetadataRegen.py +++ b/lib/_emerge/MetadataRegen.py @@ -2,10 +2,12 @@ # Distributed under the terms of the GNU General Public License v2 import portage -from portage import os -from portage.dep import _repo_separator + from _emerge.EbuildMetadataPhase import EbuildMetadataPhase + +from portage import os from portage.cache.cache_errors import CacheError +from portage.dep import _repo_separator from portage.util._async.AsyncScheduler import AsyncScheduler class MetadataRegen(AsyncScheduler): diff --git a/lib/portage/emaint/modules/move/move.py b/lib/portage/emaint/modules/move/move.py index e2df53fd8..4d1bb0df1 100644 --- a/lib/portage/emaint/modules/move/move.py +++ b/lib/portage/emaint/modules/move/move.py @@ -1,10 +1,12 @@ -# Copyright 2005-2018 Gentoo Foundation +# Copyright 2005-2020 Gentoo Authors # Distributed under the terms of the GNU General Public License v2 import portage + +from _emerge.Package import Package + from portage import os from portage.exception import InvalidData -from _emerge.Package import Package from portage.versions import _pkg_str class MoveHandler: diff --git a/lib/portage/metadata.py b/lib/portage/metadata.py index 935bcc307..f932c094a 100644 --- a/lib/portage/metadata.py +++ b/lib/portage/metadata.py @@ -1,19 +1,21 @@ +# Copyright 1998-2020 Gentoo Authors +# Distributed under the terms of the GNU General Public License v2 from __future__ import print_function -import sys -import signal import logging import operator - import portage -from portage import os +import sys +import signal + +from _emerge.ProgressHandler import ProgressHandler + from portage import eapi_is_supported -from portage.util import writemsg_level +from portage import os from portage.cache.cache_errors import CacheError -from _emerge.ProgressHandler import ProgressHandler from portage.eclass_cache import hashed_path - +from portage.util import writemsg_level def action_metadata(settings, portdb, myopts, porttrees=None): if porttrees is None: diff --git a/lib/portage/sync/modules/rsync/rsync.py b/lib/portage/sync/modules/rsync/rsync.py index cf574208d..50d086a60 100644 --- a/lib/portage/sync/modules/rsync/rsync.py +++ b/lib/portage/sync/modules/rsync/rsync.py @@ -1,31 +1,32 @@ # Copyright 1999-2020 Gentoo Authors # Distributed under the terms of the GNU General Public License v2 -import sys -import logging -import time -import signal -import socket import datetime import io -import re +import logging +import portage import random +import re
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On Thu, Aug 6, 2020 at 11:59 AM Thomas Deutschmann wrote: > > On 2020-08-06 17:44, Michał Górny wrote: > > I'm not sure if you've noticed but there are people actively working > > towards removing stale news items and trying not to dump everything > > on once on a user freshly installing the system. Don't you consider > > this a worthwhile goal? > > I don't see how this is conflicting. > > This news item can probably go away after 1-2 years. > > But for now, people who were just lucky will probably trigger this when > upgrading to genkernel-4.1 on their first reboot due to switched device > manager. > > But again: It's not a genkernel issue, so displaying that only for > people who have genkernel installed would miss a bunch of users. I would guess that most users do not utilize kexec at all, and this news item is irrelevant for them. Personally, I agree that this is not worth spamming every Gentoo user.
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On Thu, Aug 06, 2020 at 05:59:14PM +0200, Thomas Deutschmann wrote: > On 2020-08-06 17:44, Michał Górny wrote: > > I'm not sure if you've noticed but there are people actively working > > towards removing stale news items and trying not to dump everything > > on once on a user freshly installing the system. Don't you consider > > this a worthwhile goal? > > I don't see how this is conflicting. > > This news item can probably go away after 1-2 years. > > But for now, people who were just lucky will probably trigger this when > upgrading to genkernel-4.1 on their first reboot due to switched device > manager. > > But again: It's not a genkernel issue, so displaying that only for > people who have genkernel installed would miss a bunch of users. > Wait, changes were made to genkernel to switch from mdev to (e)udev which causes breakage, but it is *not* an issue with genkernel? Aside from this, do we have any evidence or bugs validating that users experience breakage with randomly named boot devices in kexec? It is great that you found an issue, but why try and be agnostic as to which one caused the issue? It looks worse that we cannot simply say: "genkernel changed for the better and things *may* break now... please read this!" Instead, we are pushing a news item to a lot of people simply because we *assume* it may be an issue for others with no evidence. -- Cheers, Aaron signature.asc Description: PGP signature
Re: [gentoo-dev] Packages up for grabs due to jlec being MIA
Taking dev-lua/luaposix https://github.com/gentoo/gentoo/pull/17032 ср, 5 авг. 2020 г. в 10:24, Michał Górny : > > Hello, > > The following packages are looking for a new maintainer: > > [vB] app-backup/cachedir > [vB] app-benchmarks/bootchart2 > [ B] app-benchmarks/ramspeed > [ B] app-office/scribus > [vB] dev-lua/luaposix > [ b] net-analyzer/zmap > [ ] net-libs/czmq > [ ] net-vpn/vpncwatch > [vB] sys-block/blocks > [ ] sys-boot/makebootfat > [v ] sys-fs/aufs-headers > [vB] sys-fs/aufs-util > [vB] sys-fs/bcache-tools > [v ] sys-kernel/aufs-sources > [vB] sys-kernel/kergen > > Legend: > > v - needs version bump > b - has trivial (QA) bug reported > B - has non-trivial bug reported > > -- > Best regards, > Michał Górny > -- >From Siberia with Love!
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On 2020-08-06 17:44, Michał Górny wrote: > I'm not sure if you've noticed but there are people actively working > towards removing stale news items and trying not to dump everything > on once on a user freshly installing the system. Don't you consider > this a worthwhile goal? I don't see how this is conflicting. This news item can probably go away after 1-2 years. But for now, people who were just lucky will probably trigger this when upgrading to genkernel-4.1 on their first reboot due to switched device manager. But again: It's not a genkernel issue, so displaying that only for people who have genkernel installed would miss a bunch of users. -- Regards, Thomas Deutschmann / Gentoo Linux Developer C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On Thu, 2020-08-06 at 17:14 +0200, Thomas Deutschmann wrote:
> On 2020-08-06 14:22, Michał Górny wrote:
> > > - I am not 100% happy with the title but the 50 char limit
> > > doesn't allow any more details.
> >
> > Yes, the title doesn't say a thing why would anyone want to read this
> > news item or not.
>
> Maybe
>
> > Be aware of possible reboot problems
>
> instead?
>
>
> > > - No Display-If condition because it is neither a genkernel nor
> > > kexec-tools issue. We maybe even have additional packages
> > > which are appending to kernel command-line I am not aware of.
> >
> > Showing this news on all old and new Gentoo systems makes little sense.
> > Either someone is newly affected, then Display-if should determine it,
> > or someone is *not* newly affected, then you're either telling him
> > something he already knows or something that is of little value to him.
> >
> > News items should be precisely this -- news. Not random pieces of
> > information you've just discovered and want to share with everyone.
> > This is what documentation is about, and it should be in some kernel-
> > related piece of documentation (handbook?) and not scattered around
> > in news items.
>
> Sure, you are basically repeating what I wrote in my prolog.
>
> But the reason why I drafted that news item despite of this is the
> consideration that an unbootable system outweigh the risk to waste
> anyone's time to read something even if they are not affected. Note that
> news items will appear through multiple channels. So if this will help
> someone who didn't read documentation before or just didn't realize the
> obvious risk he/she is taking when using non-persistent names ("It
> worked that way for me past 15 years!") I believe it has served its purpose.
I'm not sure if you've noticed but there are people actively working
towards removing stale news items and trying not to dump everything
on once on a user freshly installing the system. Don't you consider
this a worthwhile goal?
--
Best regards,
Michał Górny
signature.asc
Description: This is a digitally signed message part
[gentoo-dev] Last rites: dev-python/pythonutils
# Michał Górny (2020-08-06) # Py2.7 only. Last release in 2009. No reverse dependencies left. # Removal in 30 days. Bug #735610. dev-python/pythonutils -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
[gentoo-dev] Last rites: dev-python/gntp
# Michał Górny (2020-08-06) # Py3.6 only. Apparently randomly fails to build (bug #515736). # Last release in 2016. No reverse dependencies left. # Removal in 30 days. Bug #718968. dev-python/gntp -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On 2020-08-06 14:22, Michał Górny wrote:
>> - I am not 100% happy with the title but the 50 char limit
>> doesn't allow any more details.
>
> Yes, the title doesn't say a thing why would anyone want to read this
> news item or not.
Maybe
> Be aware of possible reboot problems
instead?
>> - No Display-If condition because it is neither a genkernel nor
>> kexec-tools issue. We maybe even have additional packages
>> which are appending to kernel command-line I am not aware of.
>
> Showing this news on all old and new Gentoo systems makes little sense.
> Either someone is newly affected, then Display-if should determine it,
> or someone is *not* newly affected, then you're either telling him
> something he already knows or something that is of little value to him.
>
> News items should be precisely this -- news. Not random pieces of
> information you've just discovered and want to share with everyone.
> This is what documentation is about, and it should be in some kernel-
> related piece of documentation (handbook?) and not scattered around
> in news items.
Sure, you are basically repeating what I wrote in my prolog.
But the reason why I drafted that news item despite of this is the
consideration that an unbootable system outweigh the risk to waste
anyone's time to read something even if they are not affected. Note that
news items will appear through multiple channels. So if this will help
someone who didn't read documentation before or just didn't realize the
obvious risk he/she is taking when using non-persistent names ("It
worked that way for me past 15 years!") I believe it has served its purpose.
--
Regards,
Thomas Deutschmann / Gentoo Linux Developer
C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] News item: Multiple root kernel command-line arguments
On Wed, 2020-08-05 at 14:02 +0200, Thomas Deutschmann wrote: > Hi, > > please review the news item below: > > - I am not 100% happy with the title but the 50 char limit > doesn't allow any more details. Yes, the title doesn't say a thing why would anyone want to read this news item or not. > > - No Display-If condition because it is neither a genkernel nor > kexec-tools issue. We maybe even have additional packages > which are appending to kernel command-line I am not aware of. Showing this news on all old and new Gentoo systems makes little sense. Either someone is newly affected, then Display-if should determine it, or someone is *not* newly affected, then you're either telling him something he already knows or something that is of little value to him. News items should be precisely this -- news. Not random pieces of information you've just discovered and want to share with everyone. This is what documentation is about, and it should be in some kernel- related piece of documentation (handbook?) and not scattered around in news items. -- Best regards, Michał Górny signature.asc Description: This is a digitally signed message part
[gentoo-dev] News item v2: Multiple root kernel command-line arguments
Hi, here's v2 based on some IRC feedback (grammar- and punctuation-related) I am planning to add for tomorrow. --- Title: Multiple root kernel command-line arguments Author: Thomas Deutschmann Posted: 2020-08-05 Revision: 1 News-Item-Format: 2.0 Due to genkernel-4.1 development which is changing device manager from MDEV to (E)UDEV it was noticed that some tools like kexec append an additional root argument to kernel command-line. If these tools will set root to a non-persistent device name like root=/dev/dm-3, the next boot might fail if there is *no* root device named like that in start environment (i.e. initramfs). While kexec's runscript was changed in >=sys-apps/kexec-tools-2.0.20-r2 to no longer append root kernel command-line argument when an option like "--reuse-cmdline" (default) is used, a cold reboot *without* kexec may be needed to restore kernel command-line. NOTE: This issue is *not* specific to kexec or genkernel usage. Kernel will always use last set root kernel command-line argument. Any tool which might be appending root argument without a persistent device name might cause a boot failure if system cannot find that referenced root device during boot. To avoid boot problems, user should revise their current kernel command-line (/proc/cmdline) to ensure that only *one* root kernel command-line argument is set. The usage of persistent device names like root=UUID=<...> is highly recommended. -- Regards, Thomas Deutschmann / Gentoo Linux Developer C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Last rites: dev-util/coccigrep, dev-util/coccinelle
But these are very useful tools...
[gentoo-dev] Re: [gentoo-dev-announce] Packages up for grabs due to jlec being MIA
i'll take app-office/scribus. miroslav Dne 05. 08. 20 v 9:24 Michał Górny napsal(a): Hello, The following packages are looking for a new maintainer: [vB] app-backup/cachedir [vB] app-benchmarks/bootchart2 [ B] app-benchmarks/ramspeed [ B] app-office/scribus [vB] dev-lua/luaposix [ b] net-analyzer/zmap [ ] net-libs/czmq [ ] net-vpn/vpncwatch [vB] sys-block/blocks [ ] sys-boot/makebootfat [v ] sys-fs/aufs-headers [vB] sys-fs/aufs-util [vB] sys-fs/bcache-tools [v ] sys-kernel/aufs-sources [vB] sys-kernel/kergen Legend: v - needs version bump b - has trivial (QA) bug reported B - has non-trivial bug reported
Re: [gentoo-portage-dev] [PATCH] lib/*: fix superfluous-parens and enable check
On 8/5/20 9:05 PM, Aaron Bauman wrote: > Signed-off-by: Aaron Bauman > --- > lib/_emerge/actions.py | 14 +++--- > lib/_emerge/resolver/slot_collision.py | 4 ++-- > lib/_emerge/stdout_spinner.py| 2 +- > lib/_emerge/unmerge.py | 4 ++-- > lib/portage/_sets/dbapi.py | 6 +++--- > lib/portage/cache/template.py| 2 +- > lib/portage/dep/__init__.py | 10 +- > lib/portage/dep/dep_check.py | 2 +- > lib/portage/getbinpkg.py | 6 +++--- > lib/portage/glsa.py | 2 +- > lib/portage/localization.py | 2 +- > lib/portage/locks.py | 2 +- > lib/portage/manifest.py | 2 +- > lib/portage/news.py | 4 ++-- > lib/portage/package/ebuild/config.py | 2 +- > lib/portage/process.py | 6 +++--- > lib/portage/sync/controller.py | 4 ++-- > lib/portage/sync/modules/rsync/rsync.py | 8 > lib/portage/tests/resolver/ResolverPlayground.py | 2 +- > lib/portage/util/__init__.py | 4 ++-- > lib/portage/util/env_update.py | 2 +- > lib/portage/xpak.py | 10 +- > pylintrc | 1 + > 23 files changed, 51 insertions(+), 50 deletions(-) Thanks, merged: https://gitweb.gentoo.org/proj/portage.git/commit/?id=897be5c2874ed3edda7b3af84e6ddb34c3095b71 -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH 6/6] pylintrc: enable multiple-imports check
On 8/5/20 4:51 PM, Aaron Bauman wrote: > Signed-off-by: Aaron Bauman > --- > pylintrc | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/pylintrc b/pylintrc > index b7091516c..5f4ec3278 100644 > --- a/pylintrc > +++ b/pylintrc > @@ -19,6 +19,7 @@ enable= > misplaced-future, > missing-final-newline, > mixed-line-endings, > +multiple-imports, > redefined-builtin, > reimported, > relative-beyond-top-level, > Thanks, merged the whole series: https://gitweb.gentoo.org/proj/portage.git/commit/?id=bc0e50294a96623fdfcb51d7cea4c3d2a4c5e261 -- Thanks, Zac signature.asc Description: OpenPGP digital signature
