[OE-core] [PATCH] kernel-devsrc: depend on python3-core instead of python3

[bkylerussell]

Avoids pulling in potential GPLv3 packages through python3-misc catch-all.

python3-core is the intended minimal RDEPENDS for packages requiring python3
support.  Other python3 module dependencies should be listed explicitly.

Signed-off-by: Kyle Russell 
---
 meta/recipes-kernel/linux/kernel-devsrc.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/linux/kernel-devsrc.bb 
b/meta/recipes-kernel/linux/kernel-devsrc.bb
index b4ea5f756a..6764598d48 100644
--- a/meta/recipes-kernel/linux/kernel-devsrc.bb
+++ b/meta/recipes-kernel/linux/kernel-devsrc.bb
@@ -377,7 +377,7 @@ do_install[lockfiles] = "${TMPDIR}/kernel-scripts.lock"
 FILES:${PN} = "${KERNEL_BUILD_ROOT} ${KERNEL_SRC_PATH}"
 FILES:${PN}-dbg += "${KERNEL_BUILD_ROOT}*/build/scripts/*/.debug/*"
 
-RDEPENDS:${PN} = "bc python3 flex bison ${TCLIBC}-utils"
+RDEPENDS:${PN} = "bc python3-core flex bison ${TCLIBC}-utils"
 # 4.15+ needs these next two RDEPENDS
 RDEPENDS:${PN} += "openssl-dev util-linux"
 # and x86 needs a bit more for 4.15+
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#179937): 
https://lists.openembedded.org/g/openembedded-core/message/179937
Mute This Topic: https://lists.openembedded.org/mt/98202780/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 1/2] devtool: get hooks directory path from git rev-parse

[bkylerussell]

Ask git rev-parse to provide the path to the repo's hooks directory.

This allows devtool to support git submodules, where only the
superproject contains a .git directory.

git submodules contain .git files pointing to their parent repo,
which confuses devtool when it tries to modify the commit hooks by
a manually constructed path.

Signed-off-by: Kyle Russell 
---
 meta/lib/oe/patch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 950fe723dc..076018271e 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -510,7 +510,7 @@ class GitApplyTree(PatchTree):
 reporoot = (runcmd("git rev-parse --show-toplevel".split(), self.dir) 
or '').strip()
 if not reporoot:
 raise Exception("Cannot get repository root for directory %s" % 
self.dir)
-hooks_dir = os.path.join(reporoot, '.git', 'hooks')
+hooks_dir = runcmd("git rev-parse --git-path hooks".split(), self.dir)
 hooks_dir_backup = hooks_dir + '.devtool-orig'
 if os.path.lexists(hooks_dir_backup):
 raise Exception("Git hooks backup directory already exists: %s" % 
hooks_dir_backup)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161503): 
https://lists.openembedded.org/g/openembedded-core/message/161503
Mute This Topic: https://lists.openembedded.org/mt/88994805/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 2/2] devtool: don't exec patch commit task funcs for git submodules

[bkylerussell]

When applying a patch directly to a submodule, devtool doesn't need to
generate commits for updated submodule revisions, which may differ each
time a patch is applied to a devtool workspace.  (submodule revision
commits would be effectively useless unless the patch is pushed to a
remote repo from devtool, at which time a commit should be generated by
the user anyway.)

Additionally, attempting to commit new submodule revisions in the parent
repo may error with "no changes added to commit" if the patched module
is nested within multiple submodules below the parent repo.  (Devtool
doesn't and shouldn't recusively create commits on each submodule.)

Signed-off-by: Kyle Russell 
---
 scripts/lib/devtool/standard.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 01fb5ad96f..5282253ac4 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -569,7 +569,8 @@ def _extract_source(srctree, keep_temp, devbranch, sync, 
config, basepath, works
 f.write('DEVTOOL_DEVBRANCH = "%s"\n' % devbranch)
 if not is_kernel_yocto:
 f.write('PATCHTOOL = "git"\n')
-f.write('PATCH_COMMIT_FUNCTIONS = "1"\n')
+if not 'gitsm' in d.getVar('SRC_URI'):
+f.write('PATCH_COMMIT_FUNCTIONS = "1"\n')
 if extra_overrides:
 f.write('DEVTOOL_EXTRA_OVERRIDES = "%s"\n' % 
':'.join(extra_overrides))
 f.write('inherit devtool-source\n')
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161504): 
https://lists.openembedded.org/g/openembedded-core/message/161504
Mute This Topic: https://lists.openembedded.org/mt/88994808/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH 1/2] devtool: get hooks directory path from git rev-parse

[bkylerussell]

Ok, I see.  I didn't realize I dropped that from the original line.  Thanks
for the catch.

On Tue, Feb 8, 2022 at 5:30 AM Jose Quaresma 
wrote:

> Hi,
>
>  escreveu no dia terça, 8/02/2022 à(s) 03:50:
>
>> Ask git rev-parse to provide the path to the repo's hooks directory.
>>
>> This allows devtool to support git submodules, where only the
>> superproject contains a .git directory.
>>
>> git submodules contain .git files pointing to their parent repo,
>> which confuses devtool when it tries to modify the commit hooks by
>> a manually constructed path.
>>
>
> +Signed-off-by: your@mail
>
>
>> ---
>>  meta/lib/oe/patch.py | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
>> index 950fe723dc..976a966083 100644
>> --- a/meta/lib/oe/patch.py
>> +++ b/meta/lib/oe/patch.py
>> @@ -510,7 +510,7 @@ class GitApplyTree(PatchTree):
>>  reporoot = (runcmd("git rev-parse --show-toplevel".split(),
>> self.dir) or '').strip()
>>  if not reporoot:
>>  raise Exception("Cannot get repository root for directory
>> %s" % self.dir)
>> -hooks_dir = os.path.join(reporoot, '.git', 'hooks')
>> +hooks_dir = runcmd("git rev-parse --git-path hooks".split())
>>
>
> This assumes that the git will run inside the self.dir but this is not
> guaranteed so adding self.dir to the runcmd is safer.
>
>
>>  hooks_dir_backup = hooks_dir + '.devtool-orig'
>>  if os.path.lexists(hooks_dir_backup):
>>  raise Exception("Git hooks backup directory already exists:
>> %s" % hooks_dir_backup)
>> --
>> 2.25.1
>>
>>
>> 
>>
>>
>
> --
> Best regards,
>
> José Quaresma
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161502): 
https://lists.openembedded.org/g/openembedded-core/message/161502
Mute This Topic: https://lists.openembedded.org/mt/88989924/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [dunfell][PATCH v2] rpm: fix intermittent compression failure in do_package_write_rpm

[bkylerussell]

rpmbuild can start processing random memory when processing the value
provided by XZ_THREADS, and unintentionally disable encoding for a
file descriptor that in fact requires encoding to be enabled in order
for lzwrite() to actually create an rpm.

  *** Fdopen(0x7f2030002b30,w6T16.xzdio)  | fdio 23 fp (nil)
  ==> lzopen_internal("w6T16", 23, 1)
  ==> lzopen_internal set encoding
  ==> lzopen_internal clear encoding
  ==> Fdopen(0x7f2030002b30,"w6T16.xzdio") returns fd 0x7f2030002b30  | 
xzdio 0x7f2030004e30 fp 23 | fdio -1 fp (nil)
  ==> lzwrite(0x7f2030004e30, 0x7f20789d8070, 6) encoding 0
  ==> Fwrite(0x7f2030002b30,0x7f20789d8070,6) rc -1   | xzdio 
0x7f2030004e30 fp 23 | fdio -1 fp (nil)
  error: create archive failed: cpio: write

When the encoding bit gets cleared on the LZFILE* struct, lzwrite() then
rightfully complains when it detects !lzfile->encoding, which then gets
bubbled up as a write failure when we go to create the archive.

This fix is available in the rpm 4.17-release.

Signed-off-by: Kyle Russell 
---
 ..._internal-mode-parsing-when-Tn-is-us.patch | 34 +++
 meta/recipes-devtools/rpm/rpm_4.14.2.1.bb |  1 +
 2 files changed, 35 insertions(+)
 create mode 100644 
meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch

diff --git 
a/meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch
 
b/meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch
new file mode 100644
index 00..9a5ebb9115
--- /dev/null
+++ 
b/meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch
@@ -0,0 +1,34 @@
+From 405fc8998181353bd510864ca251dc233afec276 Mon Sep 17 00:00:00 2001
+From: Vitaly Chikunov 
+Date: Wed, 6 Jan 2021 23:43:41 +0300
+Subject: [PATCH] rpmio: Fix lzopen_internal mode parsing when 'Tn' is used
+
+When there is number after "T" (suggested number of threads or "0" for
+getncpus), lzopen_internal() mode parser would skip one byte, and when
+it's at the end of the string it would then parse undesired garbage from
+the memory, making intermittent compression failures.
+
+Fixes: 7740d1098 ("Add support for multithreaded xz compression")
+Signed-off-by: Vitaly Chikunov 
+
+Upstream-Status: Backport 
[https://github.com/rpm-software-management/rpm/commit/405fc8998181353bd510864ca251dc233afec276]
+
+---
+ rpmio/rpmio.c | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/rpmio/rpmio.c b/rpmio/rpmio.c
+index ed1e25140..9d32ec6d9 100644
+--- a/rpmio/rpmio.c
 b/rpmio/rpmio.c
+@@ -798,6 +798,7 @@ static LZFILE *lzopen_internal(const char *mode, int fd, 
int xz)
+* should've processed
+* */
+   while (isdigit(*++mode));
++  --mode;
+   }
+ #ifdef HAVE_LZMA_MT
+   else
+-- 
+2.25.1
+
diff --git a/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb 
b/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
index c39a5208e5..376021d913 100644
--- a/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
+++ b/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
@@ -44,6 +44,7 @@ SRC_URI = 
"git://github.com/rpm-software-management/rpm;branch=rpm-4.14.x;protoc

file://0001-mono-find-provides-requires-do-not-use-monodis-from-.patch \

file://0001-Rip-out-partial-support-for-unused-MD2-and-RIPEMD160.patch \
file://0001-rpmplugins.c-call-dlerror-prior-to-dlsym.patch \
+   
file://0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch \
file://CVE-2021-3421.patch \
file://CVE-2021-20266.patch \
"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161501): 
https://lists.openembedded.org/g/openembedded-core/message/161501
Mute This Topic: https://lists.openembedded.org/mt/88994663/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [dunfell][PATCH] rpm: fix intermittent compression failure in do_package_write_rpm

[bkylerussell]

Ok, sorry about that.  I can fixup and resend.

On Tue, Feb 8, 2022 at 5:09 AM Jose Quaresma 
wrote:

> Hi,
>
>  escreveu no dia terça, 8/02/2022 à(s) 03:42:
>
>> rpmbuild can start processing random memory when processing the value
>> provided by XZ_THREADS, and unintentionally disable encoding for a
>> file descriptor that in fact requires encoding to be enabled in order
>> for lzwrite() to actually create an rpm.
>>
>>   *** Fdopen(0x7f2030002b30,w6T16.xzdio)  | fdio 23 fp (nil)
>>   ==> lzopen_internal("w6T16", 23, 1)
>>   ==> lzopen_internal set encoding
>>   ==> lzopen_internal clear encoding
>>   ==> Fdopen(0x7f2030002b30,"w6T16.xzdio") returns fd 0x7f2030002b30
>> | xzdio 0x7f2030004e30 fp 23 | fdio -1 fp (nil)
>>   ==> lzwrite(0x7f2030004e30, 0x7f20789d8070, 6) encoding 0
>>   ==> Fwrite(0x7f2030002b30,0x7f20789d8070,6) rc -1   | xzdio
>> 0x7f2030004e30 fp 23 | fdio -1 fp (nil)
>>   error: create archive failed: cpio: write
>>
>> When the encoding bit gets cleared on the LZFILE* struct, lzwrite() then
>> rightfully complains when it detects !lzfile->encoding, which then gets
>> bubbled up as a write failure when we go to create the archive.
>>
>> This fix is available in the rpm 4.17-release.
>>
>> Upstream-Status: Backport [
>> https://github.com/rpm-software-management/rpm/commit/405fc8998181353bd510864ca251dc233afec276
>> ]
>>
>
> -Upstream-Status: Backport [
> https://github.com/rpm-software-management/rpm/commit/405fc8998181353bd510864ca251dc233afec276
> ]
> +Signed-off-by: your@mail
>
> Upstream-Status need to be placed on the patch an not on the commit message
>
> ---
>>  ..._internal-mode-parsing-when-Tn-is-us.patch | 31 +++
>>  meta/recipes-devtools/rpm/rpm_4.14.2.1.bb |  1 +
>>  2 files changed, 32 insertions(+)
>>  create mode 100644
>> meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch
>>
>> diff --git
>> a/meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch
>> b/meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch
>> new file mode 100644
>> index 00..52110fa2b1
>> --- /dev/null
>> +++
>> b/meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch
>> @@ -0,0 +1,31 @@
>> +From 405fc8998181353bd510864ca251dc233afec276 Mon Sep 17 00:00:00 2001
>> +From: Vitaly Chikunov 
>> +Date: Wed, 6 Jan 2021 23:43:41 +0300
>> +Subject: [PATCH] rpmio: Fix lzopen_internal mode parsing when 'Tn' is
>> used
>> +
>> +When there is number after "T" (suggested number of threads or "0" for
>> +getncpus), lzopen_internal() mode parser would skip one byte, and when
>> +it's at the end of the string it would then parse undesired garbage from
>> +the memory, making intermittent compression failures.
>> +
>> +Fixes: 7740d1098 ("Add support for multithreaded xz compression")
>> +Signed-off-by: Vitaly Chikunov 
>>
>
> +Upstream-Status: Backport [
> https://github.com/rpm-software-management/rpm/commit/405fc8998181353bd510864ca251dc233afec276
> ]
>
>
>> +---
>> + rpmio/rpmio.c | 1 +
>> + 1 file changed, 1 insertion(+)
>> +
>> +diff --git a/rpmio/rpmio.c b/rpmio/rpmio.c
>> +index ed1e25140..9d32ec6d9 100644
>> +--- a/rpmio/rpmio.c
>>  b/rpmio/rpmio.c
>> +@@ -798,6 +798,7 @@ static LZFILE *lzopen_internal(const char *mode, int
>> fd, int xz)
>> +* should've processed
>> +* */
>> +   while (isdigit(*++mode));
>> ++  --mode;
>> +   }
>> + #ifdef HAVE_LZMA_MT
>> +   else
>> +--
>> +2.25.1
>> +
>> diff --git a/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
>> b/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
>> index c39a5208e5..376021d913 100644
>> --- a/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
>> +++ b/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
>> @@ -44,6 +44,7 @@ SRC_URI = "git://
>> github.com/rpm-software-management/rpm;branch=rpm-4.14.x;protoc
>>
>> file://0001-mono-find-provides-requires-do-not-use-monodis-from-.patch \
>>
>> file://0001-Rip-out-partial-support-for-unused-MD2-and-RIPEMD160.patch \
>> file://0001-rpmplugins.c-call-dlerror-prior-to-dlsym.patch \
>> +
>>  file://0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch \
>> file://CVE-2021-3421.patch \
>> file://CVE-2021-20266.patch \
>> "
>> --
>> 2.25.1
>>
>>
>> 
>>
>>
>
> --
> Best regards,
>
> José Quaresma
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161500): 
https://lists.openembedded.org/g/openembedded-core/message/161500
Mute This Topic: https://lists.openembedded.org/mt/88989796/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 2/2] devtool: don't exec patch commit task funcs for git submodules

[bkylerussell]

When applying a patch directly to a submodule, devtool doesn't need to
generate commits for updated submodule revisions, which may differ each
time a patch is applied to a devtool workspace.  (submodule revision
commits would be effectively useless unless the patch is pushed to a
remote repo from devtool, at which time a commit should be generated by
the user anyway.)

Additionally, attempting to commit new submodule revisions in the parent
repo may error with "no changes added to commit" if the patched module
is nested within multiple submodules below the parent repo.  (Devtool
doesn't and shouldn't recusively create commits on each submodule.)
---
 scripts/lib/devtool/standard.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 01fb5ad96f..5282253ac4 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -569,7 +569,8 @@ def _extract_source(srctree, keep_temp, devbranch, sync, 
config, basepath, works
 f.write('DEVTOOL_DEVBRANCH = "%s"\n' % devbranch)
 if not is_kernel_yocto:
 f.write('PATCHTOOL = "git"\n')
-f.write('PATCH_COMMIT_FUNCTIONS = "1"\n')
+if not 'gitsm' in d.getVar('SRC_URI'):
+f.write('PATCH_COMMIT_FUNCTIONS = "1"\n')
 if extra_overrides:
 f.write('DEVTOOL_EXTRA_OVERRIDES = "%s"\n' % 
':'.join(extra_overrides))
 f.write('inherit devtool-source\n')
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161474): 
https://lists.openembedded.org/g/openembedded-core/message/161474
Mute This Topic: https://lists.openembedded.org/mt/88989926/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/2] devtool: get hooks directory path from git rev-parse

[bkylerussell]

Ask git rev-parse to provide the path to the repo's hooks directory.

This allows devtool to support git submodules, where only the
superproject contains a .git directory.

git submodules contain .git files pointing to their parent repo,
which confuses devtool when it tries to modify the commit hooks by
a manually constructed path.
---
 meta/lib/oe/patch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 950fe723dc..976a966083 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -510,7 +510,7 @@ class GitApplyTree(PatchTree):
 reporoot = (runcmd("git rev-parse --show-toplevel".split(), self.dir) 
or '').strip()
 if not reporoot:
 raise Exception("Cannot get repository root for directory %s" % 
self.dir)
-hooks_dir = os.path.join(reporoot, '.git', 'hooks')
+hooks_dir = runcmd("git rev-parse --git-path hooks".split())
 hooks_dir_backup = hooks_dir + '.devtool-orig'
 if os.path.lexists(hooks_dir_backup):
 raise Exception("Git hooks backup directory already exists: %s" % 
hooks_dir_backup)
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161473): 
https://lists.openembedded.org/g/openembedded-core/message/161473
Mute This Topic: https://lists.openembedded.org/mt/88989924/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [dunfell][PATCH] rpm: fix intermittent compression failure in do_package_write_rpm

[bkylerussell]

rpmbuild can start processing random memory when processing the value
provided by XZ_THREADS, and unintentionally disable encoding for a
file descriptor that in fact requires encoding to be enabled in order
for lzwrite() to actually create an rpm.

  *** Fdopen(0x7f2030002b30,w6T16.xzdio)  | fdio 23 fp (nil)
  ==> lzopen_internal("w6T16", 23, 1)
  ==> lzopen_internal set encoding
  ==> lzopen_internal clear encoding
  ==> Fdopen(0x7f2030002b30,"w6T16.xzdio") returns fd 0x7f2030002b30  | 
xzdio 0x7f2030004e30 fp 23 | fdio -1 fp (nil)
  ==> lzwrite(0x7f2030004e30, 0x7f20789d8070, 6) encoding 0
  ==> Fwrite(0x7f2030002b30,0x7f20789d8070,6) rc -1   | xzdio 
0x7f2030004e30 fp 23 | fdio -1 fp (nil)
  error: create archive failed: cpio: write

When the encoding bit gets cleared on the LZFILE* struct, lzwrite() then
rightfully complains when it detects !lzfile->encoding, which then gets
bubbled up as a write failure when we go to create the archive.

This fix is available in the rpm 4.17-release.

Upstream-Status: Backport 
[https://github.com/rpm-software-management/rpm/commit/405fc8998181353bd510864ca251dc233afec276]
---
 ..._internal-mode-parsing-when-Tn-is-us.patch | 31 +++
 meta/recipes-devtools/rpm/rpm_4.14.2.1.bb |  1 +
 2 files changed, 32 insertions(+)
 create mode 100644 
meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch

diff --git 
a/meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch
 
b/meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch
new file mode 100644
index 00..52110fa2b1
--- /dev/null
+++ 
b/meta/recipes-devtools/rpm/files/0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch
@@ -0,0 +1,31 @@
+From 405fc8998181353bd510864ca251dc233afec276 Mon Sep 17 00:00:00 2001
+From: Vitaly Chikunov 
+Date: Wed, 6 Jan 2021 23:43:41 +0300
+Subject: [PATCH] rpmio: Fix lzopen_internal mode parsing when 'Tn' is used
+
+When there is number after "T" (suggested number of threads or "0" for
+getncpus), lzopen_internal() mode parser would skip one byte, and when
+it's at the end of the string it would then parse undesired garbage from
+the memory, making intermittent compression failures.
+
+Fixes: 7740d1098 ("Add support for multithreaded xz compression")
+Signed-off-by: Vitaly Chikunov 
+---
+ rpmio/rpmio.c | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/rpmio/rpmio.c b/rpmio/rpmio.c
+index ed1e25140..9d32ec6d9 100644
+--- a/rpmio/rpmio.c
 b/rpmio/rpmio.c
+@@ -798,6 +798,7 @@ static LZFILE *lzopen_internal(const char *mode, int fd, 
int xz)
+* should've processed
+* */
+   while (isdigit(*++mode));
++  --mode;
+   }
+ #ifdef HAVE_LZMA_MT
+   else
+-- 
+2.25.1
+
diff --git a/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb 
b/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
index c39a5208e5..376021d913 100644
--- a/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
+++ b/meta/recipes-devtools/rpm/rpm_4.14.2.1.bb
@@ -44,6 +44,7 @@ SRC_URI = 
"git://github.com/rpm-software-management/rpm;branch=rpm-4.14.x;protoc

file://0001-mono-find-provides-requires-do-not-use-monodis-from-.patch \

file://0001-Rip-out-partial-support-for-unused-MD2-and-RIPEMD160.patch \
file://0001-rpmplugins.c-call-dlerror-prior-to-dlsym.patch \
+   
file://0001-rpmio-Fix-lzopen_internal-mode-parsing-when-Tn-is-us.patch \
file://CVE-2021-3421.patch \
file://CVE-2021-20266.patch \
"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161472): 
https://lists.openembedded.org/g/openembedded-core/message/161472
Mute This Topic: https://lists.openembedded.org/mt/88989796/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] rng-tools: add systemd-udev-settle wants to service

[bkylerussell]

Thanks, Claudius.  I really appreciate your responses.  I'm not trying to
be pedantic.  Since I don't have your test setup, I was just trying to make
sure I understood the context of the problem as I figure out how to deal
with issues this is causing in our setup.

I was also hoping one of the recipe maintainers of either systemd or
rng-tools would comment on systemd-udev-settle.

I'll take a look at the caam module to see if I can understand how it works.

On Thu, Feb 3, 2022 at 3:35 AM Claudius Heine  wrote:

> On 2022-02-02 17:26, Kyle Russell wrote:
> > Thanks, Claudius.
> >
> > On Wed, Feb 2, 2022 at 8:08 AM Claudius Heine  > > wrote:
> >
> > Hi Kyle,
> >
> > On 2022-02-02 13:38, Kyle Russell wrote:
> >  > Is this the correct approach?  Even the
> > systemd-udev-settle.service man
> >  > pages recommends not using its service.  Were the kernel modules
> > really
> >  > not loaded when rngd started?  Or is the original problem just a
> > matter
> >  > of waiting for sufficient entropy?
> >
> > IIRC, the rngd could not find any random source device node
> (/dev/hwrng
> > in that case), so the service failed to start.
> >
> >
> > If /dev/hwrng didn't exist, this feels like the original problem was a
> > misconfigured
> > kernel or module that wasn't being loaded properly.
>
> Yes, however it is a timing issue. The module was loaded properly at
> bootup, however at the time rngd was started the module was not loaded
> yet and thus the service fails to start. If it would be delayed until
> the module is loaded everything would be fine.
>
> It does not happen if the module is compiled into the kernel or if a
> initramfs is used which loads the module (I think). I our case it
> happend with the caam module as an external module loaded on boot from
> the real root file system.
>
> > The patch you are commenting on only adds `Wants` weak dependency to
> > make sure `systemd-udev-settle.service` is pulled in to the job
> queue,
> > the `After` ordering rule was already there.
> >
> >
> > Correct.  Just because an `After` exists does not mean the service gets
> > pulled into
> > the job queue, so prior to this change no other service was causing the
> > deprecated
> > systemd-udev-settle.service to be run during boot.  But now, every
> > device including
> > openssh (which has a default PACKAGECONFIG option for rng-tools) now
> depends
> > on this deprecated service, which may cause unexpected boot delays.
> >
> > So changing this service file to be triggered by a udev event or
> maybe
> > wrap it in a script, which makes sure the right modules are loaded
> and
> > device nodes are available, could be an improvement, but it would be
> > out
> > of scope of this patch IMO.
> >
> >
> > I'm more curious whether this change should be reverted from upstream.
> > It seems
> > like a drop-in file could have been applied to your distro instead of
> > adding a dependency
> > on a deprecated service for all openssh users.
>
> This patch just adds a missing entry into the service file. If you have
> solved the described issue in some way and can revert this patch and
> remove the `Wants=systemd-udev-settle.service` then you can also remove
> the `After=systemd-udev-settle.service` at the same time and at that
> point you can just remove both of those entries directly in the patch
> that solved the timing issue.
>
> I agree that `systemd-udev-settle.service` should probably not be used
> anymore, however that file already used it in a non-functional way and
> all this patch did was make it fulfill its intended function.
>
> In retrospect I probably should have tried to find a way to remove the
> usage of `systemd-udev-settle.service` completely, when I looked into
> the issue, however all this patch in essence does is revive dead code,
> which was already in place.
>
> Also I think at that time I couldn't find a more precise instrument in
> systemds massive toolbox to delay the start of rngd and services that
> should be started in succession until the just the hardware random
> generator device is ready and `After=systemd-udev-settle.service` was
> already there. I guess some `ExecStartPre=` script which delays the
> start until the conditions are met could be implemented, but that seems
> a bit hackish.
>
> regards,
> Claudius
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161284): 
https://lists.openembedded.org/g/openembedded-core/message/161284
Mute This Topic: https://lists.openembedded.org/mt/85671578/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] rng-tools: add systemd-udev-settle wants to service

[bkylerussell]

Thanks, Claudius.

On Wed, Feb 2, 2022 at 8:08 AM Claudius Heine  wrote:

> Hi Kyle,
>
> On 2022-02-02 13:38, Kyle Russell wrote:
> > Is this the correct approach?  Even the systemd-udev-settle.service man
> > pages recommends not using its service.  Were the kernel modules really
> > not loaded when rngd started?  Or is the original problem just a matter
> > of waiting for sufficient entropy?
>
> IIRC, the rngd could not find any random source device node (/dev/hwrng
> in that case), so the service failed to start.
>

If /dev/hwrng didn't exist, this feels like the original problem was a
misconfigured
kernel or module that wasn't being loaded properly.


> The patch you are commenting on only adds `Wants` weak dependency to
> make sure `systemd-udev-settle.service` is pulled in to the job queue,
> the `After` ordering rule was already there.
>

Correct.  Just because an `After` exists does not mean the service gets
pulled into
the job queue, so prior to this change no other service was causing the
deprecated
systemd-udev-settle.service to be run during boot.  But now, every device
including
openssh (which has a default PACKAGECONFIG option for rng-tools) now depends
on this deprecated service, which may cause unexpected boot delays.


> So changing this service file to be triggered by a udev event or maybe
> wrap it in a script, which makes sure the right modules are loaded and
> device nodes are available, could be an improvement, but it would be out
> of scope of this patch IMO.
>

I'm more curious whether this change should be reverted from upstream.  It
seems
like a drop-in file could have been applied to your distro instead of
adding a dependency
on a deprecated service for all openssh users.

Kyle


> >
> > On Fri, Sep 17, 2021 at 4:08 AM Claudius Heine  > > wrote:
> >
> > rngd needs to start after `systemd-udev-settle` in order for the
> kernel
> > modules of the random source hardware to be loaded before it is
> started.
> >
> > However, since the `rngd.service` does not require or want
> > `systemd-udev-settle.service` it might not be scheduled for start and
> > the `After=systemd-udev-settle.service` there has no effect.
> >
> > Adding `Wants=systemd-udev-settle.service` provides a weak
> requirement
> > to it, so that the `rngd` is started after it, if possible.
> >
> > Signed-off-by: Claudius Heine mailto:c...@denx.de>>
> > ---
> >
> > Hi,
> >
> > this is a fix, which should probably be backported to the maintained
> > releases.
> >
> > regards,
> > Claudius
> >
> >   meta/recipes-support/rng-tools/rng-tools/rngd.service | 1 +
> >   1 file changed, 1 insertion(+)
> >
> > diff --git a/meta/recipes-support/rng-tools/rng-tools/rngd.service
> > b/meta/recipes-support/rng-tools/rng-tools/rngd.service
> > index 0559b97991..568686e80e 100644
> > --- a/meta/recipes-support/rng-tools/rng-tools/rngd.service
> > +++ b/meta/recipes-support/rng-tools/rng-tools/rngd.service
> > @@ -3,6 +3,7 @@ Description=Hardware RNG Entropy Gatherer Daemon
> >   DefaultDependencies=no
> >   After=systemd-udev-settle.service
> >   Before=sysinit.target shutdown.target
> > +Wants=systemd-udev-settle.service
> >   Conflicts=shutdown.target
> >
> >   [Service]
> > --
> > 2.33.0
> >
> >
> > 
> >
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161220): 
https://lists.openembedded.org/g/openembedded-core/message/161220
Mute This Topic: https://lists.openembedded.org/mt/85671578/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] rng-tools: add systemd-udev-settle wants to service

[bkylerussell]

Is this the correct approach?  Even the systemd-udev-settle.service man
pages recommends not using its service.  Were the kernel modules really not
loaded when rngd started?  Or is the original problem just a matter of
waiting for sufficient entropy?

On Fri, Sep 17, 2021 at 4:08 AM Claudius Heine  wrote:

> rngd needs to start after `systemd-udev-settle` in order for the kernel
> modules of the random source hardware to be loaded before it is started.
>
> However, since the `rngd.service` does not require or want
> `systemd-udev-settle.service` it might not be scheduled for start and
> the `After=systemd-udev-settle.service` there has no effect.
>
> Adding `Wants=systemd-udev-settle.service` provides a weak requirement
> to it, so that the `rngd` is started after it, if possible.
>
> Signed-off-by: Claudius Heine 
> ---
>
> Hi,
>
> this is a fix, which should probably be backported to the maintained
> releases.
>
> regards,
> Claudius
>
>  meta/recipes-support/rng-tools/rng-tools/rngd.service | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/meta/recipes-support/rng-tools/rng-tools/rngd.service
> b/meta/recipes-support/rng-tools/rng-tools/rngd.service
> index 0559b97991..568686e80e 100644
> --- a/meta/recipes-support/rng-tools/rng-tools/rngd.service
> +++ b/meta/recipes-support/rng-tools/rng-tools/rngd.service
> @@ -3,6 +3,7 @@ Description=Hardware RNG Entropy Gatherer Daemon
>  DefaultDependencies=no
>  After=systemd-udev-settle.service
>  Before=sysinit.target shutdown.target
> +Wants=systemd-udev-settle.service
>  Conflicts=shutdown.target
>
>  [Service]
> --
> 2.33.0
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161193): 
https://lists.openembedded.org/g/openembedded-core/message/161193
Mute This Topic: https://lists.openembedded.org/mt/85671578/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] security_flags.inc: don't default to PIE if image-prelink is enabled

[bkylerussell]

Yes, we do use prelink.  I think our use case primarily benefits from CoW
memory savings, rather than load times.  Of course, GCCPIE can be
overridden in the distro layer, but seeing as image-prelink.bbclass still
exists upstream, the default definition should support configurations that
choose to enable it.

On Thu, Jan 20, 2022 at 3:30 AM Alexander Kanavin 
wrote:

> I think we pretty much abandoned prelink at this point, are you using it
> and do you see the benefits?
>
> Alex
>
> On Thu, 20 Jan 2022 at 04:30,  wrote:
>
>> Since a prelinked rootfs is in conflict with PIE, don't attempt the latter
>> if the image enables prelink.
>> ---
>>  meta/conf/distro/include/security_flags.inc | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/conf/distro/include/security_flags.inc
>> b/meta/conf/distro/include/security_flags.inc
>> index e469eadca1..be6feb9e5f 100644
>> --- a/meta/conf/distro/include/security_flags.inc
>> +++ b/meta/conf/distro/include/security_flags.inc
>> @@ -5,7 +5,7 @@
>>  # From a Yocto Project perspective, this file is included and tested
>>  # in the DISTRO="poky" configuration.
>>
>> -GCCPIE ?= "--enable-default-pie"
>> +GCCPIE ?= "${@bb.utils.contains('USER_CLASSES', 'image-prelink',
>> '--disable-default-pie', '--enable-default-pie', d)}"
>>  # If static PIE is known to work well, GLIBCPIE="--enable-static-pie"
>> can be set
>>
>>  # _FORTIFY_SOURCE requires -O1 or higher, so disable in debug builds as
>> they use
>> --
>> 2.25.1
>>
>>
>> 
>>
>>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160773): 
https://lists.openembedded.org/g/openembedded-core/message/160773
Mute This Topic: https://lists.openembedded.org/mt/88551948/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] security_flags.inc: don't default to PIE if image-prelink is enabled

[bkylerussell]

Since a prelinked rootfs is in conflict with PIE, don't attempt the latter
if the image enables prelink.
---
 meta/conf/distro/include/security_flags.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/distro/include/security_flags.inc 
b/meta/conf/distro/include/security_flags.inc
index e469eadca1..be6feb9e5f 100644
--- a/meta/conf/distro/include/security_flags.inc
+++ b/meta/conf/distro/include/security_flags.inc
@@ -5,7 +5,7 @@
 # From a Yocto Project perspective, this file is included and tested
 # in the DISTRO="poky" configuration.
 
-GCCPIE ?= "--enable-default-pie"
+GCCPIE ?= "${@bb.utils.contains('USER_CLASSES', 'image-prelink', 
'--disable-default-pie', '--enable-default-pie', d)}"
 # If static PIE is known to work well, GLIBCPIE="--enable-static-pie" can be 
set
 
 # _FORTIFY_SOURCE requires -O1 or higher, so disable in debug builds as they 
use
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160749): 
https://lists.openembedded.org/g/openembedded-core/message/160749
Mute This Topic: https://lists.openembedded.org/mt/88551948/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] cairo: use ${PN} in package name

[bkylerussell]

From: Steven Walter 

Without this, bitbake complains that the license for cairo-gobject is
incorrect, because the license is set using ${PN}

Signed-off-by: Kyle Russell 
---
 meta/recipes-graphics/cairo/cairo_1.16.0.bb | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/meta/recipes-graphics/cairo/cairo_1.16.0.bb 
b/meta/recipes-graphics/cairo/cairo_1.16.0.bb
index d76d935c30..b5d4b7eb22 100644
--- a/meta/recipes-graphics/cairo/cairo_1.16.0.bb
+++ b/meta/recipes-graphics/cairo/cairo_1.16.0.bb
@@ -77,17 +77,17 @@ do_install:append () {
rmdir -p --ignore-fail-on-non-empty ${D}${libdir}/cairo
 }
 
-PACKAGES =+ "cairo-gobject cairo-script-interpreter cairo-perf-utils"
+PACKAGES =+ "${PN}-gobject ${PN}-script-interpreter ${PN}-perf-utils"
 
-SUMMARY:cairo-gobject = "The Cairo library GObject wrapper library"
-DESCRIPTION:cairo-gobject = "A GObject wrapper library for the Cairo API."
+SUMMARY:${PN}-gobject = "The Cairo library GObject wrapper library"
+DESCRIPTION:${PN}-gobject = "A GObject wrapper library for the Cairo API."
 
-SUMMARY:cairo-script-interpreter = "The Cairo library script interpreter"
-DESCRIPTION:cairo-script-interpreter = "The Cairo script interpreter 
implements \
+SUMMARY:${PN}-script-interpreter = "The Cairo library script interpreter"
+DESCRIPTION:${PN}-script-interpreter = "The Cairo script interpreter 
implements \
 CairoScript.  CairoScript is used by tracing utilities to enable the ability \
 to replay rendering."
 
-DESCRIPTION:cairo-perf-utils = "The Cairo library performance utilities"
+DESCRIPTION:${PN}-perf-utils = "The Cairo library performance utilities"
 
 FILES:${PN} = "${libdir}/libcairo.so.*"
 FILES:${PN}-gobject = "${libdir}/libcairo-gobject.so.*"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156695): 
https://lists.openembedded.org/g/openembedded-core/message/156695
Mute This Topic: https://lists.openembedded.org/mt/86121102/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] gcc-runtime: do_check should execute before do_build

[bkylerussell]

I see. I had forgotten that the autobuilders don't run do_check as part of
the normal build.

We have a customization in our distro layer to run do_check on normal
builds. Internally, we use that as a hook to allow per-package test suites
to fail our integration pipeline during our normal builds. Maybe there's a
way we can prevent this conflict with rm_work from our own layer.

Sorry for the confusion,

Kyle


On Fri, Jul 30, 2021, 5:06 PM Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

> On Fri, 2021-07-30 at 14:16 -0700, Christopher Larson wrote:
> > This will result in a behavior change, making it so `bitbake
> gcc-runtime` will
> > run the check task. Are there implications to this?
>
> We definitely do not want to do that. This is used to run the testsuite
> which
> isn't part of a normal build.
>
> Cheers,
>
> Richard
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154292): 
https://lists.openembedded.org/g/openembedded-core/message/154292
Mute This Topic: https://lists.openembedded.org/mt/84557001/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] gcc-runtime: do_check should execute before do_build

[bkylerussell]

Otherwise, when rm_work is used, do_check may not execute before rm_work
causing do_check failures if the workdir has been removed underneath it.

Signed-off-by: Kyle Russell 
---
 meta/recipes-devtools/gcc/gcc-runtime.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/gcc/gcc-runtime.inc 
b/meta/recipes-devtools/gcc/gcc-runtime.inc
index 59d9a24686..607971adca 100644
--- a/meta/recipes-devtools/gcc/gcc-runtime.inc
+++ b/meta/recipes-devtools/gcc/gcc-runtime.inc
@@ -314,5 +314,5 @@ do_check() {
 
 oe_runmake -i ${MAKE_CHECK_TARGETS} 
RUNTESTFLAGS="${MAKE_CHECK_RUNTESTFLAGS}"
 }
-addtask check after do_compile do_populate_sysroot
+addtask check after do_compile do_populate_sysroot before do_build
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154288): 
https://lists.openembedded.org/g/openembedded-core/message/154288
Mute This Topic: https://lists.openembedded.org/mt/84557001/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] gstreamer: add libgst packages to PACKAGES_DYNAMIC

[bkylerussell]

This allows the dynamic libgst* plugin packages and libgstrtspserver-1.0
to be used correctly in RDEPENDS.

All gstreamer1.0-plugins-* recipes and gstreamer1.0-rtsp-server include
gstreamer1.0-plugins-packaging.inc, which creates multiple libgst*
packages through split_gstreamer10_packages(), but only one of the
do_split_packages() regexes was included in PACKAGES_DYNAMIC, and only
for the plugins packages (through gstreamer1.0-plugins-common.inc).
Now the PACKAGES_DYNAMIC definition exists with the do_split_packages()
regexes.

Signed-off-by: Kyle Russell 
---
 .../gstreamer/gstreamer1.0-plugins-common.inc  | 3 ---
 .../gstreamer/gstreamer1.0-plugins-packaging.inc   | 3 +++
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-common.inc 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-common.inc
index dba96e0f6e..cb1c9aaec1 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-common.inc
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-common.inc
@@ -45,6 +45,3 @@ EXTRA_OEMESON += " \
 
 GIR_MESON_ENABLE_FLAG = "enabled"
 GIR_MESON_DISABLE_FLAG = "disabled"
-
-# Dynamically generate packages for all enabled plugins
-PACKAGES_DYNAMIC = "^${PN}-.*"
diff --git 
a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-packaging.inc 
b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-packaging.inc
index 9a7a1b6afe..5b80a46b40 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-packaging.inc
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-packaging.inc
@@ -6,6 +6,9 @@
 # This is mainly used by the gstreamer1.0-plugins-* plugin set recipes,
 # but can be used in any recipe that produces GStreamer plugins.
 
+# Dynamically generate packages for all enabled plugins
+PACKAGES_DYNAMIC = "^${PN}-.* ^libgst.*"
+
 PACKAGESPLITFUNCS_prepend = " split_gstreamer10_packages "
 PACKAGESPLITFUNCS_append = " set_gstreamer10_metapkg_rdepends "
 
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153513): 
https://lists.openembedded.org/g/openembedded-core/message/153513
Mute This Topic: https://lists.openembedded.org/mt/83970377/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] uninative: allow PREMIRRORS to use versioned url subpath

[bkylerussell]

This allows users running their own mirrors to store uninative tarballs
under versioned paths just like the default UNINATIVE_URL without having
to completely override UNINATIVE_URL in their distro configuration and
manually update the versioned URL on each upgrade.

The hashed subpath is left for compatibility, but not practical for
mirror management.

Signed-off-by: Kyle Russell 
---
 meta/classes/uninative.bbclass   | 1 +
 meta/conf/distro/include/yocto-uninative.inc | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/uninative.bbclass b/meta/classes/uninative.bbclass
index 1e19917a97..543f747358 100644
--- a/meta/classes/uninative.bbclass
+++ b/meta/classes/uninative.bbclass
@@ -64,6 +64,7 @@ python uninative_event_fetchloader() {
 continue
 if find.startswith("http"):
 localdata.appendVar("PREMIRRORS", " 
${UNINATIVE_URL}${UNINATIVE_TARBALL} %s/uninative/%s/${UNINATIVE_TARBALL}" % 
(replace, chksum))
+localdata.appendVar("PREMIRRORS", " 
${UNINATIVE_URL}${UNINATIVE_TARBALL} 
%s/uninative/${UNINATIVE_VERSION}/${UNINATIVE_TARBALL}" % replace)
 
 srcuri = 
d.expand("${UNINATIVE_URL}${UNINATIVE_TARBALL};sha256sum=%s" % chksum)
 bb.note("Fetching uninative binary shim %s (will check PREMIRRORS 
first)" % srcuri)
diff --git a/meta/conf/distro/include/yocto-uninative.inc 
b/meta/conf/distro/include/yocto-uninative.inc
index 740cca0ecf..d7d1e2cb59 100644
--- a/meta/conf/distro/include/yocto-uninative.inc
+++ b/meta/conf/distro/include/yocto-uninative.inc
@@ -7,8 +7,9 @@
 #
 
 UNINATIVE_MAXGLIBCVERSION = "2.33"
+UNINATIVE_VERSION = "3.2"
 
-UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/3.2/;
+UNINATIVE_URL ?= 
"http://downloads.yoctoproject.org/releases/uninative/${UNINATIVE_VERSION}/;
 UNINATIVE_CHECKSUM[aarch64] ?= 
"4f0872cdca2775b637a8a99815ca5c8dd42146abe903a24a50ee0448358c764b"
 UNINATIVE_CHECKSUM[i686] ?= 
"e2eeab92e67263db37d9bb6d4c58579abd1f47ff4cded3171bde572fece124b2"
 UNINATIVE_CHECKSUM[x86_64] ?= 
"3ee8c7d55e2d4c7ae3887cddb97219f97b94efddfeee2e24923c0cb0e8ce84c6"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153245): 
https://lists.openembedded.org/g/openembedded-core/message/153245
Mute This Topic: https://lists.openembedded.org/mt/83744822/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[bkylerussell]

On Tue, Jun 8, 2021 at 7:16 AM Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

> > > There were specific issues with gcc 4.8 and 4.9 which required
> separate sstate. Do we
> > > know which gcc version switched the default and can we force older
> gcc's to use the
> > > same default or does it not work? I'm wondering if we could split off
> a "nopic" verison
> > > of sstate for example for the older gccs?
> > >
> > gcc 6 introduced an --enable-default-pie configure option, but I'm not
> exactly sure where
> > between Ubuntu 16.04 and Ubuntu 18.04 they picked that configure option
> up.  Since packages
> > based on the older gcc already exist in sstate, I think we would have to
> force the newer
> > behavior on each affected package.
> >
> > I realize we can just swizzle the compiler flags on the affected
> packages to make things happy,
> > but it seemed like this approach had at least been considered before, so
> I thought I would at least
> > get some feedback.
>
> As Andrea points out, we did fix this with a global include file which
> should have
> handled the issue. Are you using a release which doesn't have that or not
> using
> the include through your own custom distro?
>

Yes indeed, nice find.  bitbake -e confirms I'm including
uninative-flags.inc.  I do think this is exactly the
same issue, but there's a different set of dependencies involved in my
chain.  As someone suggested earlier,
I can force -fPIE on the packages where this is a problem, so I could of
course do something similar for those
packages.

But I still feel like there's an underlying uninative limitation that's not
being addressed, or highlighted.  For example,
the SSTATE_MIRRORS documentation seems to imply that differentiating
between uninative based on native gcc
version is the normal/correct behavior, even though it's clear to me now
that the example was chosen specifically
for the 4.8/4.9 issue, not just the general case.

"When pointing to sstate build artifacts on another machine that uses a
different GCC version for native builds,
you must configure SSTATE_MIRROR with a regular expression that maps local
search paths to server paths.
The paths need to take into account NATIVELSBSTRING set by the uninative
class."
 -
https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#var-SSTATE_MIRRORS

But I suppose even similar versions of gcc across different supported
distros could be configured differently.
For example, warrior claims support for Ubuntu 16.04, 18.04, and Fedora 28,
but only Ubuntu 18.04 seems to
use --enable-default-pie based on gcc -v.  (Even newer Fedoras seem to not
use this flag.)

I guess we could have a "nopic" split like you suggested.  Or maybe
uninative itself should be hashed against
the native gcc configure args and version, since version by itself also
doesn't seem to be sufficient based on the
Fedora example above.  Neither one of those sounds particularly appealing
the more I think about it, so maybe
I'll just opt for fixing the packages like we've done for binutils-native.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#152788): 
https://lists.openembedded.org/g/openembedded-core/message/152788
Mute This Topic: https://lists.openembedded.org/mt/83381901/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[bkylerussell]

On Mon, Jun 7, 2021 at 5:43 PM Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441
>
> This doesn't look quite right?
>

Yes, that's the one.  I assumed this was what you meant by, "Another option
would be to have multiple uninative feeds based on gcc
version rather than a single uninative one."

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441#c1

This is a huge performance penalty to put onto the build unfortunately and
> I really
> don't want to do that by default on the autobuilder. We haven't seen many
> of these
> errors on the autobuilder and I'm reluctant to take the performance hit on
> our testing.
>

I realize this is not ideal, but I guess the trade-off is dealing with
these issues as they arise on
a case-by-case basis.

There were specific issues with gcc 4.8 and 4.9 which required separate
> sstate. Do we
> know which gcc version switched the default and can we force older gcc's
> to use the
> same default or does it not work? I'm wondering if we could split off a
> "nopic" verison
> of sstate for example for the older gccs?
>

gcc 6 introduced an --enable-default-pie configure option, but I'm not
exactly sure where
between Ubuntu 16.04 and Ubuntu 18.04 they picked that configure option
up.  Since packages
based on the older gcc already exist in sstate, I think we would have to
force the newer
behavior on each affected package.

I realize we can just swizzle the compiler flags on the affected packages
to make things happy,
but it seemed like this approach had at least been considered before, so I
thought I would at least
get some feedback.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#152767): 
https://lists.openembedded.org/g/openembedded-core/message/152767
Mute This Topic: https://lists.openembedded.org/mt/83381901/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] lib: oe: utils: always append host gcc version to NATIVELSBSTRING

[bkylerussell]

By having multiple uninative feeds based on host gcc version (rather than a
single uninative feed for all possible host gcc versions), we avoid a general
class of problems (as first observed in the bug below) where changes in native
gcc don't cause native sstate to regenerate.  Below is another example where
this problem is encountered across multiple supported host distros for a given
poky version.

gcc 6 supports default generation of PIE, which gets enabled between Ubuntu
16.04 (default gcc 5.4) and 18.04 (default gcc 7.5), both supported distros
of warrior.  Consider a native package dependency chain as follows:

 A
   / | \
  B  C  D

If a complete set of native sstate for the above packages is built prior
to the default PIE change, but a workstation upgrades its distro to a version
that natively compiles PIE by default, the sstate hash is not currently tainted,
so the newer compiler version will pull from sstate not compiled for PIE.

For example, if a source change to C causes part of the above chain to rebuild
(C -> A), then this may result in non-PIE sstate (B and D) giving link errors
with A.

ld: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making 
a PIE object; recompile with -fPIC
ld: final link failed: Nonrepresentable section on output

This appears to have been considered previously as an option to address
the following bug, but appears to have been set aside only because of
proximity to the end of a release cycle.

[YOCTO #10441]

Signed-off-by: Kyle Russell 
---
 meta/lib/oe/utils.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index a84039f585..8afe19fd99 100644
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -426,7 +426,7 @@ def host_gcc_version(d, taskcontextonly=False):
 bb.fatal("Can't get compiler version from %s --version output" % 
compiler)
 
 version = match.group(1)
-return "-%s" % version if version in ("4.8", "4.9") else ""
+return "-%s" % version
 
 
 def get_multilib_datastore(variant, d):
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#152764): 
https://lists.openembedded.org/g/openembedded-core/message/152764
Mute This Topic: https://lists.openembedded.org/mt/83381901/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 1/1] patch.py: don't apply striplevel to git am command

[bkylerussell]

Don't pass a path removal parameter to git am since it always applies
patches from the root of the git repo, not from each individual patchdir
(like quilt).

GitApplyTree uses `git rev-parse --show-toplevel` to find the root of
the repo no matter which patchdir is passed into the GitApplyTree instance.
But for recipes who set S to a subdirectory within their repo, patches
may be added to SRC_URI using a striplevel parameter to match the depth of S.

Currently when these patches are applied with git am in GitApplyTree instead
of quilt, the striplevel still takes effect even though git applies the
patch itself from the repo root, not from within S.

In most cases when a patch modifies existing files within a repo, git am
works fine and the desired outcome is still achieved because the three-way
merge figures out which file the patch was intended to modify based on the
repo history.

An exception to this occurs if the patch adds a new file to the repo.

For example, consider a patch that adds a new file src/foo to a repo.
Additionally, assume the recipe for this repo sets S = "${WORKDIR}/git/src",
and the patch is added to SRC_URI using "striplevel=2" since the default
patch tool, quilt, applies each patch at its patchdir, ${S}.

  --- /dev/null
  +++ b/src/foo
  @@ -0,0 +1 @@
  +new file test

When the GitApplyTree patch class is used (through either devtool or
PATCHTOOL = "git"), the root of the repo is passed to git as the work tree,
in addition to the path removal paramter.  In the case above, GitApplyTree
creates ${WORKDIR}/git/foo instead of ${WORKDIR}/git/src/foo.

This is demonstrated with the included selftest.  By not passing the striplevel
parameter to git am, the patch creates the new file at the intended depth in
the repo.
---
 .../llvm/files/0001-Test-new-file.patch| 17 +
 .../recipes-test/llvm/llvm_%.bbappend  |  2 ++
 meta/lib/oe/patch.py   |  2 +-
 meta/lib/oeqa/selftest/cases/bbtests.py| 18 ++
 4 files changed, 38 insertions(+), 1 deletion(-)
 create mode 100644 
meta-selftest/recipes-test/llvm/files/0001-Test-new-file.patch
 create mode 100644 meta-selftest/recipes-test/llvm/llvm_%.bbappend

diff --git a/meta-selftest/recipes-test/llvm/files/0001-Test-new-file.patch 
b/meta-selftest/recipes-test/llvm/files/0001-Test-new-file.patch
new file mode 100644
index 00..8af0afeb12
--- /dev/null
+++ b/meta-selftest/recipes-test/llvm/files/0001-Test-new-file.patch
@@ -0,0 +1,17 @@
+From 057c3da44affb47c154ec1ef67bd11646a6429fa Mon Sep 17 00:00:00 2001
+From: Kyle Russell 
+Date: Tue, 28 Apr 2020 15:18:43 -0400
+Subject: [PATCH] Test new file
+
+---
+ llvm/README.new | 1 +
+ 1 file changed, 1 insertion(+)
+ create mode 100644 llvm/README.new
+
+diff --git a/llvm/README.new b/llvm/README.new
+new file mode 100644
+index 000..0527e6bd2d7
+--- /dev/null
 b/llvm/README.new
+@@ -0,0 +1 @@
++This is a test
diff --git a/meta-selftest/recipes-test/llvm/llvm_%.bbappend 
b/meta-selftest/recipes-test/llvm/llvm_%.bbappend
new file mode 100644
index 00..205720982c
--- /dev/null
+++ b/meta-selftest/recipes-test/llvm/llvm_%.bbappend
@@ -0,0 +1,2 @@
+# This bbappend is used to alter the recipe using the test_recipe.inc file 
created by tests.
+include test_recipe.inc
diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 7ca2e28b1f..870d183e8b 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -512,7 +512,7 @@ class GitApplyTree(PatchTree):
 try:
 shellcmd = [patchfilevar, "git", "--work-tree=%s" % reporoot]
 self.gitCommandUserOptions(shellcmd, self.commituser, 
self.commitemail)
-shellcmd += ["am", "-3", "--keep-cr", "-p%s" % 
patch['strippath']]
+shellcmd += ["am", "-3", "--keep-cr"]
 return _applypatchhelper(shellcmd, patch, force, reverse, run)
 except CmdError:
 # Need to abort the git am, or we'll still be within it at the 
end
diff --git a/meta/lib/oeqa/selftest/cases/bbtests.py 
b/meta/lib/oeqa/selftest/cases/bbtests.py
index dc423ec439..6d863b425a 100644
--- a/meta/lib/oeqa/selftest/cases/bbtests.py
+++ b/meta/lib/oeqa/selftest/cases/bbtests.py
@@ -4,6 +4,7 @@
 
 import os
 import re
+import tempfile
 
 import oeqa.utils.ftools as ftools
 from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars
@@ -81,6 +82,23 @@ class BitbakeTests(OESelftestTestCase):
 found = l
 self.assertTrue(found and found.startswith("ERROR:"), msg = 
"Incorrectly formed patch application didn't fail. bitbake output: %s" % 
result.output)
 
+def test_patchtool_git(self):
+test_recipe = 'llvm'
+tmpdir = tempfile.mkdtemp(prefix='bitbakeqa')
+bb_vars = get_bb_vars(['S'], test_recipe)
+src_dir = bb_vars['S']
+
+self.write_recipeinc('llvm', 'FILESEXTRAPATHS_prepend := 
"${THISDIR}/files:"\nSRC_URI += 

[OE-core] [PATCH v2 0/1] patch.py: don't apply striplevel to git am command

[bkylerussell]

I thought I had sent this out a month ago, but I can't find it in patchwork,
so I'm resending.  This is an updated patch that addresses feedback on v1, and
should hopefully be more clear.

I've dropped the change to 'git apply' since that wasn't behaving as I 
originally
thought, then I added a selftest on llvm that demonstrates the issue I'm trying 
to
resolve.  Without the change in patch.py included here, the selftest that I'm
proposing will fail.

One might argue, "Well, if you change the default PATCHTOOL, you could also
just change your patch SRC_URIs and drop the striplevel parameter."  While
that may be true, the same failure can also be demonstrated using devtool on
llvm without modifying PATCHTOOL at all, since by default devtool also uses
GitApplyTree.  So if you don't consider "modifying a recipe's PATCHTOOL without
also modify its SRC_URI patches" to be a valid workflow, devtool is another
common workflow where this behavior could be encountered.

Hope that helps,
Kyle

Kyle Russell (1):
  patch.py: don't apply striplevel to git am command

 .../llvm/files/0001-Test-new-file.patch| 17 +
 .../recipes-test/llvm/llvm_%.bbappend  |  2 ++
 meta/lib/oe/patch.py   |  2 +-
 meta/lib/oeqa/selftest/cases/bbtests.py| 18 ++
 4 files changed, 38 insertions(+), 1 deletion(-)
 create mode 100644 
meta-selftest/recipes-test/llvm/files/0001-Test-new-file.patch
 create mode 100644 meta-selftest/recipes-test/llvm/llvm_%.bbappend

-- 
2.17.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#139240): 
https://lists.openembedded.org/g/openembedded-core/message/139240
Mute This Topic: https://lists.openembedded.org/mt/74684793/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 0/1] patch.py: don't apply striplevel to git am command

[bkylerussell]

Here's an updated patch that I hope will be a little more clear.  I've dropped
the change to git apply since that wasn't behaving as I expected, then added
a selftest on llvm that demonstrates the issue I'm trying to resolve.  Without
the change in patch.py included here, the selftest that I'm proposing will fail.

One might argue, "Well, if you change the default PATCHTOOL, you could also
just change your patch SRC_URIs and drop the striplevel parameter."  While
that may be true, the same failure can also be demonstrated using devtool on
llvm without modifying PATCHTOOL at all, since by default devtool also uses
GitApplyTree.  So if you don't consider "modifying a recipe's PATCHTOOL without
also modify its SRC_URI patches" to be a valid workflow, devtool is another
common workflow where this behavior could be encountered.

Hope that helps,

Kyle

Kyle Russell (1):
  patch.py: don't apply striplevel to git am command

 .../llvm/files/0001-Test-new-file.patch| 17 +
 .../recipes-test/llvm/llvm_%.bbappend  |  2 ++
 meta/lib/oe/patch.py   |  2 +-
 meta/lib/oeqa/selftest/cases/bbtests.py| 18 ++
 4 files changed, 38 insertions(+), 1 deletion(-)
 create mode 100644 
meta-selftest/recipes-test/llvm/files/0001-Test-new-file.patch
 create mode 100644 meta-selftest/recipes-test/llvm/llvm_%.bbappend

-- 
2.17.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#137625): 
https://lists.openembedded.org/g/openembedded-core/message/137625
Mute This Topic: https://lists.openembedded.org/mt/73366011/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: 
https://lists.openembedded.org/g/openembedded-core/leave/8023207/1426099254/xyzzy
  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v2 1/1] patch.py: don't apply striplevel to git am command

[bkylerussell]

Don't pass a path removal parameter to git am since it always applies
patches from the root of the git repo, not from each individual patchdir
(like quilt).

GitApplyTree uses `git rev-parse --show-toplevel` to find the root of
the repo no matter which patchdir is passed into the GitApplyTree instance.
But for recipes who set S to a subdirectory within their repo, patches
may be added to SRC_URI using a striplevel parameter to match the depth of S.

Currently when these patches are applied with git am in GitApplyTree instead
of quilt, the striplevel still takes effect even though git applies the
patch itself from the repo root, not from within S.

In most cases when a patch modifies existing files within a repo, git am
works fine and the desired outcome is still achieved because the three-way
merge figures out which file the patch was intended to modify based on the
repo history.

An exception to this occurs if the patch adds a new file to the repo.

For example, consider a patch that adds a new file src/foo to a repo.
Additionally, assume the recipe for this repo sets S = "${WORKDIR}/git/src",
and the patch is added to SRC_URI using "striplevel=2" since the default
patch tool, quilt, applies each patch at its patchdir, ${S}.

  --- /dev/null
  +++ b/src/foo
  @@ -0,0 +1 @@
  +new file test

When the GitApplyTree patch class is used (through either devtool or
PATCHTOOL = "git"), the root of the repo is passed to git as the work tree,
in addition to the path removal paramter.  In the case above, GitApplyTree
creates ${WORKDIR}/git/foo instead of ${WORKDIR}/git/src/foo.

This is demonstrated with the included selftest.  By not passing the striplevel
parameter to git am, the patch creates the new file at the intended depth in
the repo.
---
 .../llvm/files/0001-Test-new-file.patch| 17 +
 .../recipes-test/llvm/llvm_%.bbappend  |  2 ++
 meta/lib/oe/patch.py   |  2 +-
 meta/lib/oeqa/selftest/cases/bbtests.py| 18 ++
 4 files changed, 38 insertions(+), 1 deletion(-)
 create mode 100644 
meta-selftest/recipes-test/llvm/files/0001-Test-new-file.patch
 create mode 100644 meta-selftest/recipes-test/llvm/llvm_%.bbappend

diff --git a/meta-selftest/recipes-test/llvm/files/0001-Test-new-file.patch 
b/meta-selftest/recipes-test/llvm/files/0001-Test-new-file.patch
new file mode 100644
index 00..8af0afeb12
--- /dev/null
+++ b/meta-selftest/recipes-test/llvm/files/0001-Test-new-file.patch
@@ -0,0 +1,17 @@
+From 057c3da44affb47c154ec1ef67bd11646a6429fa Mon Sep 17 00:00:00 2001
+From: Kyle Russell 
+Date: Tue, 28 Apr 2020 15:18:43 -0400
+Subject: [PATCH] Test new file
+
+---
+ llvm/README.new | 1 +
+ 1 file changed, 1 insertion(+)
+ create mode 100644 llvm/README.new
+
+diff --git a/llvm/README.new b/llvm/README.new
+new file mode 100644
+index 000..0527e6bd2d7
+--- /dev/null
 b/llvm/README.new
+@@ -0,0 +1 @@
++This is a test
diff --git a/meta-selftest/recipes-test/llvm/llvm_%.bbappend 
b/meta-selftest/recipes-test/llvm/llvm_%.bbappend
new file mode 100644
index 00..205720982c
--- /dev/null
+++ b/meta-selftest/recipes-test/llvm/llvm_%.bbappend
@@ -0,0 +1,2 @@
+# This bbappend is used to alter the recipe using the test_recipe.inc file 
created by tests.
+include test_recipe.inc
diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 2b1eee1003..7b9fd67194 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -517,7 +517,7 @@ class GitApplyTree(PatchTree):
 try:
 shellcmd = [patchfilevar, "git", "--work-tree=%s" % reporoot]
 self.gitCommandUserOptions(shellcmd, self.commituser, 
self.commitemail)
-shellcmd += ["am", "-3", "--keep-cr", "-p%s" % 
patch['strippath']]
+shellcmd += ["am", "-3", "--keep-cr"]
 return _applypatchhelper(shellcmd, patch, force, reverse, run)
 except CmdError:
 # Need to abort the git am, or we'll still be within it at the 
end
diff --git a/meta/lib/oeqa/selftest/cases/bbtests.py 
b/meta/lib/oeqa/selftest/cases/bbtests.py
index dc423ec439..6d863b425a 100644
--- a/meta/lib/oeqa/selftest/cases/bbtests.py
+++ b/meta/lib/oeqa/selftest/cases/bbtests.py
@@ -4,6 +4,7 @@
 
 import os
 import re
+import tempfile
 
 import oeqa.utils.ftools as ftools
 from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars
@@ -81,6 +82,23 @@ class BitbakeTests(OESelftestTestCase):
 found = l
 self.assertTrue(found and found.startswith("ERROR:"), msg = 
"Incorrectly formed patch application didn't fail. bitbake output: %s" % 
result.output)
 
+def test_patchtool_git(self):
+test_recipe = 'llvm'
+tmpdir = tempfile.mkdtemp(prefix='bitbakeqa')
+bb_vars = get_bb_vars(['S'], test_recipe)
+src_dir = bb_vars['S']
+
+self.write_recipeinc('llvm', 'FILESEXTRAPATHS_prepend := 
"${THISDIR}/files:"\nSRC_URI += 

Re: [OE-core] [PATCH 1/1] patch: don't strip GitApplyTree patches

[bkylerussell]

Sorry, it has taken me a while to get back to this.

> If you changed the patch tool to git for those recipes, doesn't it fail
with this patch applied?

Even without my patch, I don't think bash works normally with PATCHTOOL =
"git" since we unpack it from a tgz.  I added PATCHTOOL = "git" to the bash
recipe, and it seems to trash my poky repo's git history instead of the
source in my bash workdir. :( There might be a separate case we should
protect against for that, but I'll have to look at it separately.

llvm looks like the perfect example though, and is much more interesting.
Initially, it actually fails the git am attempt with "fatal: empty ident
name (for <>) not allowed".  I think this might be because git doesn't find
a "From" line first, or something like that.  (If I move Upstream-Status
and Signed-off-by below "From" then git am is happy.). But by default git
am fails, and the first fallback, git apply, doesn't work with my patch as
I expected.  git apply appears to only focus on its current directory,
unlike git am which I think applies from the work-tree root.  I may have
only exercised the git am path initially, but I'll investigate the
difference here further before providing an updated patch.

> We may need to teach GitApplyTree to calculate the difference between the
git repodir and patchdir and adjust the offsets accordingly?

That was the approach I started with, but I couldn't come up with a
scenario where we would end up with an offset other than the default (1)
for commands that work from the repo root (git am).  So maybe the git am
case is the only one that needs to ignore the striplevel, if it truly
always expects to work from the repo root.  Maybe git apply is the case I
unintentionally broke.

When I post a revised patch, I'll also update the commit message subject as
Peter pointed out.

On Thu, Apr 9, 2020 at 6:07 AM Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

> On Thu, 2020-04-09 at 00:41 -0400, bkyleruss...@gmail.com wrote:
> > Don't pass a path removal parameter to git am/git apply since
> GitApplyTree
> > always applies patches from the root of the git repo, not from each
> > individual patchdir (like quilt).
> >
> > GitApplyTree uses `git rev-parse --show-toplevel` to find the root of
> > the repo no matter which patchdir is passed into the GitApplyTree
> instance.
> > In most cases, this works fine when applying patches since the three-way
> > merge can figure out where the patch should be applied to existing files
> > based on the history, even if the file paths are excessively stripped.
> >
> > An exception to this occurs if a patch adds a new file to the repo
> > and a custom ${S} has been set to a subdirectory of the repo such
> > that the patch requires a non-default striplevel parameter.
> >
> > When processed by GitApplyTree, the patch will have its striplevel
> > parameter given to git even though it's applied at the root of the
> > tree.
> >
> > For example, consider a patch that adds a new file src/foo to a repo.
> >
> >   --- /dev/null
> >   +++ b/src/foo
> >   @@ -0,0 +1 @@
> >   +new file test
> >
> > The recipe sets S = "${WORKDIR}/git/src" and the patch is added to
> SRC_URI
> > using "striplevel=2" since the default patch tool, quilt, applies each
> > patch at its patchdir, which defaults to ${S}.
> >
> > When the GitApplyTree patch class is used (for example, through devtool),
> > the root of the repo is passed to git as the work tree, in addition to
> the
> > path removal paramter.  In the case above, GitApplyTree creates
> > ${WORKDIR}/git/foo instead of ${WORKDIR}/git/src/foo.
> >
> > Interestingly enough, you can take a more typical patch example that only
> > changes existing files (instead of adding a new file) and remove the -3
> > argument from the git am/git apply commands issued by GitApplyTree.  In
> > this scenario, the patch will also fail to apply because the path removal
> > arg is still being passed, and the resulting path does not exist.
> >
> > Since GitApplyTree always applies from the root of the repo, it should
> > ignore any striplevel arguments.
>
> I don't follow this argument. We have several recipes in OE-Core which
> set striplevel in SRC_URI (e.g. bash, llvm and a pointless looking one
> in build-compare). If you changed the patch tool to git for those
> recipes, doesn't it fail with this patch applied?
>
> I think you're right and there is an issue here since do_patch assumes
> ${S} is the root for patches whilst git assumes the root of the repo is
> the patch root. If those are different, it will break.
>
> Put another way, I think your patch fixes one case whilst breaking
> others :(.
>
> I'm not sure how we reconcile this. I certainly do not like hoping 'git
> am -3' figures it out. We may need to teach GitApplyTree to calculate
> the difference between the git repodir and patchdir and adjust the
> offsets accordingly?
>
> Cheers,
>
> Richard
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent 

[OE-core] [PATCH 0/1] patch: don't strip GitApplyTree patches

[bkylerussell]

I think that the GitApplyTree patch implementation may not be quite
performing as expected when dealing with a non-default ${S} and patch
striplevels.

I'm not quite sure there aren't scenarios where we *do* want to honor
striplevel in GitApplyTree, even though it always appears to apply
patches at the repository's root, so I wanted to send this out for
feedback.

The change does at least address the scenario I encountered, but there's
probably a legitimate use case I'm missing where we need striplevel here.

Kyle Russell (1):
  patch: don't strip GitApplyTree patches

 meta/lib/oe/patch.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-- 
2.17.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#137130): 
https://lists.openembedded.org/g/openembedded-core/message/137130
Mute This Topic: https://lists.openembedded.org/mt/72890513/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH 1/1] patch: don't strip GitApplyTree patches

[bkylerussell]

Don't pass a path removal parameter to git am/git apply since GitApplyTree
always applies patches from the root of the git repo, not from each
individual patchdir (like quilt).

GitApplyTree uses `git rev-parse --show-toplevel` to find the root of
the repo no matter which patchdir is passed into the GitApplyTree instance.
In most cases, this works fine when applying patches since the three-way
merge can figure out where the patch should be applied to existing files
based on the history, even if the file paths are excessively stripped.

An exception to this occurs if a patch adds a new file to the repo
and a custom ${S} has been set to a subdirectory of the repo such
that the patch requires a non-default striplevel parameter.

When processed by GitApplyTree, the patch will have its striplevel
parameter given to git even though it's applied at the root of the
tree.

For example, consider a patch that adds a new file src/foo to a repo.

  --- /dev/null
  +++ b/src/foo
  @@ -0,0 +1 @@
  +new file test

The recipe sets S = "${WORKDIR}/git/src" and the patch is added to SRC_URI
using "striplevel=2" since the default patch tool, quilt, applies each
patch at its patchdir, which defaults to ${S}.

When the GitApplyTree patch class is used (for example, through devtool),
the root of the repo is passed to git as the work tree, in addition to the
path removal paramter.  In the case above, GitApplyTree creates
${WORKDIR}/git/foo instead of ${WORKDIR}/git/src/foo.

Interestingly enough, you can take a more typical patch example that only
changes existing files (instead of adding a new file) and remove the -3
argument from the git am/git apply commands issued by GitApplyTree.  In
this scenario, the patch will also fail to apply because the path removal
arg is still being passed, and the resulting path does not exist.

Since GitApplyTree always applies from the root of the repo, it should
ignore any striplevel arguments.
---
 meta/lib/oe/patch.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 2b1eee1003..2e45a7ef89 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -517,7 +517,7 @@ class GitApplyTree(PatchTree):
 try:
 shellcmd = [patchfilevar, "git", "--work-tree=%s" % reporoot]
 self.gitCommandUserOptions(shellcmd, self.commituser, 
self.commitemail)
-shellcmd += ["am", "-3", "--keep-cr", "-p%s" % 
patch['strippath']]
+shellcmd += ["am", "-3", "--keep-cr"]
 return _applypatchhelper(shellcmd, patch, force, reverse, run)
 except CmdError:
 # Need to abort the git am, or we'll still be within it at the 
end
@@ -534,7 +534,7 @@ class GitApplyTree(PatchTree):
 runcmd(["sh", "-c", " ".join(shellcmd)], self.dir)
 
 # Fall back to git apply
-shellcmd = ["git", "--git-dir=%s" % reporoot, "apply", "-p%s" 
% patch['strippath']]
+shellcmd = ["git", "--git-dir=%s" % reporoot, "apply"]
 try:
 output = _applypatchhelper(shellcmd, patch, force, 
reverse, run)
 except CmdError:
-- 
2.17.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#137131): 
https://lists.openembedded.org/g/openembedded-core/message/137131
Mute This Topic: https://lists.openembedded.org/mt/72890514/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub  
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-