[OE-core][kirkstone][PATCH] tiff: CVE patch correction for CVE-2023-3576

2023-10-30 Thread Vijay Anusuri via lists.openembedded.org
From: Vijay Anusuri 

- The commit 
[https://gitlab.com/libtiff/libtiff/-/commit/881a070194783561fd209b7c789a4e75566f7f37]
fixes CVE-2023-3576
- Hence, renamed the CVE-2023-3618-1.patch to CVE-2023-3576.patch
- Reference: https://security-tracker.debian.org/tracker/CVE-2023-3576
 https://security-tracker.debian.org/tracker/CVE-2023-3618

Signed-off-by: Vijay Anusuri 
---
 .../tiff/{CVE-2023-3618-1.patch => CVE-2023-3576.patch}   | 3 ++-
 .../tiff/{CVE-2023-3618-2.patch => CVE-2023-3618.patch}   | 0
 meta/recipes-multimedia/libtiff/tiff_4.3.0.bb | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
 rename meta/recipes-multimedia/libtiff/tiff/{CVE-2023-3618-1.patch => 
CVE-2023-3576.patch} (93%)
 rename meta/recipes-multimedia/libtiff/tiff/{CVE-2023-3618-2.patch => 
CVE-2023-3618.patch} (100%)

diff --git a/meta/recipes-multimedia/libtiff/tiff/CVE-2023-3618-1.patch 
b/meta/recipes-multimedia/libtiff/tiff/CVE-2023-3576.patch
similarity index 93%
rename from meta/recipes-multimedia/libtiff/tiff/CVE-2023-3618-1.patch
rename to meta/recipes-multimedia/libtiff/tiff/CVE-2023-3576.patch
index 8f55d2b496..b17dd72170 100644
--- a/meta/recipes-multimedia/libtiff/tiff/CVE-2023-3618-1.patch
+++ b/meta/recipes-multimedia/libtiff/tiff/CVE-2023-3576.patch
@@ -4,8 +4,9 @@ Date: Tue, 7 Mar 2023 15:02:08 +0800
 Subject: [PATCH] Fix memory leak in tiffcrop.c
 
 Upstream-Status: Backport 
[https://gitlab.com/libtiff/libtiff/-/commit/881a070194783561fd209b7c789a4e75566f7f37]
-CVE: CVE-2023-3618
+CVE: CVE-2023-3576
 Signed-off-by: Hitendra Prajapati 
+Signed-off-by: Vijay Anusuri 
 ---
  tools/tiffcrop.c | 7 ++-
  1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-multimedia/libtiff/tiff/CVE-2023-3618-2.patch 
b/meta/recipes-multimedia/libtiff/tiff/CVE-2023-3618.patch
similarity index 100%
rename from meta/recipes-multimedia/libtiff/tiff/CVE-2023-3618-2.patch
rename to meta/recipes-multimedia/libtiff/tiff/CVE-2023-3618.patch
diff --git a/meta/recipes-multimedia/libtiff/tiff_4.3.0.bb 
b/meta/recipes-multimedia/libtiff/tiff_4.3.0.bb
index 8dcd73273e..e925b7d652 100644
--- a/meta/recipes-multimedia/libtiff/tiff_4.3.0.bb
+++ b/meta/recipes-multimedia/libtiff/tiff_4.3.0.bb
@@ -40,8 +40,8 @@ SRC_URI = 
"http://download.osgeo.org/libtiff/tiff-${PV}.tar.gz \
file://CVE-2023-26965.patch \
file://CVE-2023-2908.patch \
file://CVE-2023-3316.patch \
-   file://CVE-2023-3618-1.patch \
-   file://CVE-2023-3618-2.patch \
+   file://CVE-2023-3576.patch \
+   file://CVE-2023-3618.patch \
file://CVE-2023-26966.patch \
file://CVE-2022-40090.patch \
file://CVE-2023-1916.patch \
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189829): 
https://lists.openembedded.org/g/openembedded-core/message/189829
Mute This Topic: https://lists.openembedded.org/mt/102292263/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core][kirkstone][PATCH 1/1] libwebp: Fix CVE-2023-4863

2023-10-30 Thread Anuj Mittal
On Tue, 2023-10-31 at 04:37 +, Soumya via lists.openembedded.org
wrote:
> From: Soumya Sambu 
> 
> Heap buffer overflow in WebP in Google Chrome prior to
> 116.0.5845.187 allowed a remote attacker to perform an
> out of bounds memory write via a crafted HTML page.
> 
> References:
> https://nvd.nist.gov/vuln/detail/CVE-2023-4863
> https://security-tracker.debian.org/tracker/CVE-2023-4863
> https://bugzilla.redhat.com/show_bug.cgi?id=2238431#c12
> 
> Signed-off-by: Soumya Sambu 
> ---
>  .../webp/files/CVE-2023-4863.patch    | 109
> ++
>  meta/recipes-multimedia/webp/libwebp_1.2.4.bb |   1 +
>  2 files changed, 110 insertions(+)
>  create mode 100644 meta/recipes-multimedia/webp/files/CVE-2023-
> 4863.patch
> 
> diff --git a/meta/recipes-multimedia/webp/files/CVE-2023-4863.patch
> b/meta/recipes-multimedia/webp/files/CVE-2023-4863.patch
> new file mode 100644
> index 00..4c60cbc9a1
> --- /dev/null
> +++ b/meta/recipes-multimedia/webp/files/CVE-2023-4863.patch
> @@ -0,0 +1,109 @@
> +From 95ea5226c870449522240ccff26f0b006037c520 Mon Sep 17 00:00:00
> 2001
> +From: Vincent Rabaud 
> +Date: Mon, 11 Sep 2023 16:06:08 +0200
> +Subject: [PATCH] Fix invalid incremental decoding check.
> +
> +The first condition is only necessary if we have not read enough
> +(enough being defined by src_last, not src_end which is the end
> +of the image).
> +The second condition now fits the comment below: "if not
> +incremental, and we are past the end of buffer".
> +
> +BUG=oss-fuzz:62136
> +
> +Change-Id: I0700f67c62db8e1c02c2e429a069a71e606a5e4f
> +
> +CVE: CVE-2023-4863
> +
> +Upstream-Status: Backport
> [https://github.com/webmproject/libwebp/commit/95ea5226c870449522240c
> cff26f0b006037c520]
> +
> +Signed-off-by: Soumya Sambu 
> +---
> + ...x-invalid-incremental-decoding-check.patch | 48
> +++

Patch file included by mistake?

Thanks,

Anuj

> + src/dec/vp8l_dec.c    | 15 +-
> + 2 files changed, 61 insertions(+), 2 deletions(-)
> + create mode 100644 0001-Fix-invalid-incremental-decoding-
> check.patch
> +
> +diff --git a/0001-Fix-invalid-incremental-decoding-check.patch
> b/0001-Fix-invalid-incremental-decoding-check.patch
> +new file mode 100644
> +index 000..21f67f4
> +--- /dev/null
>  b/0001-Fix-invalid-incremental-decoding-check.patch
> +@@ -0,0 +1,48 @@
> ++From 95ea5226c870449522240ccff26f0b006037c520 Mon Sep 17 00:00:00
> 2001
> ++From: Vincent Rabaud 
> ++Date: Mon, 11 Sep 2023 16:06:08 +0200
> ++Subject: [PATCH] Fix invalid incremental decoding check.
> ++
> ++The first condition is only necessary if we have not read enough
> ++(enough being defined by src_last, not src_end which is the end
> ++of the image).
> ++The second condition now fits the comment below: "if not
> ++incremental, and we are past the end of buffer".
> ++
> ++BUG=oss-fuzz:62136
> ++
> ++Change-Id: I0700f67c62db8e1c02c2e429a069a71e606a5e4f
> ++---
> ++ src/dec/vp8l_dec.c | 15 +--
> ++ 1 file changed, 13 insertions(+), 2 deletions(-)
> ++
> ++diff --git a/src/dec/vp8l_dec.c b/src/dec/vp8l_dec.c
> ++index 5ab34f56..809b1aa9 100644
> ++--- a/src/dec/vp8l_dec.c
> + b/src/dec/vp8l_dec.c
> ++@@ -1233,9 +1233,20 @@ static int DecodeImageData(VP8LDecoder*
> const dec, uint32_t* const data,
> ++   }
> ++
> ++   br->eos_ = VP8LIsEndOfStream(br);
> ++-  if (dec->incremental_ && br->eos_ && src < src_end) {
> +++  // In incremental decoding:
> +++  // br->eos_ && src < src_last: if 'br' reached the end of the
> buffer and
> +++  // 'src_last' has not been reached yet, there is not enough
> data. 'dec' has to
> +++  // be reset until there is more data.
> +++  // !br->eos_ && src < src_last: this cannot happen as either the
> buffer is
> +++  // fully read, either enough has been read to reach 'src_last'.
> +++  // src >= src_last: 'src_last' is reached, all is fine. 'src'
> can actually go
> +++  // beyond 'src_last' in case the image is cropped and an LZ77
> goes further.
> +++  // The buffer might have been enough or there is some left. 'br-
> >eos_' does
> +++  // not matter.
> +++  assert(!dec->incremental_ || (br->eos_ && src < src_last) || src
> >= src_last);
> +++  if (dec->incremental_ && br->eos_ && src < src_last) {
> ++ RestoreState(dec);
> ++-  } else if (!br->eos_) {
> +++  } else if ((dec->incremental_ && src >= src_last) || !br->eos_)
> {
> ++ // Process the remaining rows corresponding to last row-block.
> ++ if (process_func != NULL) {
> ++   process_func(dec, row > last_row ? last_row : row);
> ++--
> ++2.40.0
> ++
> +diff --git a/src/dec/vp8l_dec.c b/src/dec/vp8l_dec.c
> +index 186b0b2..59a9e64 100644
> +--- a/src/dec/vp8l_dec.c
>  b/src/dec/vp8l_dec.c
> +@@ -1241,9 +1241,20 @@ static int DecodeImageData(VP8LDecoder* const
> dec, uint32_t* const data,
> +   }
> +
> +   br->eos_ = VP8LIsEndOfStream(br);
> +-  if (dec->incremental_ && br->eos_ && src < src_end) {
> ++  // In incremental decoding:
> ++  // br->eos_ && 

Patchtest results for [OE-core][kirkstone][PATCH 1/1] libwebp: Fix CVE-2023-4863

2023-10-30 Thread Soumya via lists.openembedded.org
Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch 
/home/patchtest/share/mboxes/kirkstone-1-1-libwebp-Fix-CVE-2023-4863.patch

FAIL: test CVE presence in commit message: A CVE tag should be provided in the 
commit message with format: "CVE: CVE--" 
(test_mbox.TestMbox.test_cve_presence_in_commit_message)

PASS: pretest lic files chksum modified not mentioned 
(test_metadata.TestMetadata.pretest_lic_files_chksum_modified_not_mentioned)
PASS: pretest src uri left files 
(test_metadata.TestMetadata.pretest_src_uri_left_files)
PASS: test CVE tag format (test_patch.TestPatch.test_cve_tag_format)
PASS: test Signed-off-by presence 
(test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test Signed-off-by presence 
(test_patch.TestPatch.test_signed_off_by_presence)
PASS: test Upstream-Status presence 
(test_patch.TestPatch.test_upstream_status_presence_format)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence 
(test_mbox.TestMbox.test_commit_message_presence)
PASS: test lic files chksum modified not mentioned 
(test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test src uri left files 
(test_metadata.TestMetadata.test_src_uri_left_files)

SKIP: pretest pylint: No python related patches, skipping test 
(test_python_pylint.PyLint.pretest_pylint)
SKIP: test bugzilla entry format: No bug ID found 
(test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: No python related patches, skipping test 
(test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now 
(test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test summary presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_summary_presence)
SKIP: test target mailing list: Series merged, no reason to check other mailing 
lists (test_mbox.TestMbox.test_target_mailing_list)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189827): 
https://lists.openembedded.org/g/openembedded-core/message/189827
Mute This Topic: https://lists.openembedded.org/mt/102291842/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core][kirkstone][PATCH 1/1] libwebp: Fix CVE-2023-4863

2023-10-30 Thread Soumya via lists.openembedded.org
From: Soumya Sambu 

Heap buffer overflow in WebP in Google Chrome prior to
116.0.5845.187 allowed a remote attacker to perform an
out of bounds memory write via a crafted HTML page.

References:
https://nvd.nist.gov/vuln/detail/CVE-2023-4863
https://security-tracker.debian.org/tracker/CVE-2023-4863
https://bugzilla.redhat.com/show_bug.cgi?id=2238431#c12

Signed-off-by: Soumya Sambu 
---
 .../webp/files/CVE-2023-4863.patch| 109 ++
 meta/recipes-multimedia/webp/libwebp_1.2.4.bb |   1 +
 2 files changed, 110 insertions(+)
 create mode 100644 meta/recipes-multimedia/webp/files/CVE-2023-4863.patch

diff --git a/meta/recipes-multimedia/webp/files/CVE-2023-4863.patch 
b/meta/recipes-multimedia/webp/files/CVE-2023-4863.patch
new file mode 100644
index 00..4c60cbc9a1
--- /dev/null
+++ b/meta/recipes-multimedia/webp/files/CVE-2023-4863.patch
@@ -0,0 +1,109 @@
+From 95ea5226c870449522240ccff26f0b006037c520 Mon Sep 17 00:00:00 2001
+From: Vincent Rabaud 
+Date: Mon, 11 Sep 2023 16:06:08 +0200
+Subject: [PATCH] Fix invalid incremental decoding check.
+
+The first condition is only necessary if we have not read enough
+(enough being defined by src_last, not src_end which is the end
+of the image).
+The second condition now fits the comment below: "if not
+incremental, and we are past the end of buffer".
+
+BUG=oss-fuzz:62136
+
+Change-Id: I0700f67c62db8e1c02c2e429a069a71e606a5e4f
+
+CVE: CVE-2023-4863
+
+Upstream-Status: Backport 
[https://github.com/webmproject/libwebp/commit/95ea5226c870449522240ccff26f0b006037c520]
+
+Signed-off-by: Soumya Sambu 
+---
+ ...x-invalid-incremental-decoding-check.patch | 48 +++
+ src/dec/vp8l_dec.c| 15 +-
+ 2 files changed, 61 insertions(+), 2 deletions(-)
+ create mode 100644 0001-Fix-invalid-incremental-decoding-check.patch
+
+diff --git a/0001-Fix-invalid-incremental-decoding-check.patch 
b/0001-Fix-invalid-incremental-decoding-check.patch
+new file mode 100644
+index 000..21f67f4
+--- /dev/null
 b/0001-Fix-invalid-incremental-decoding-check.patch
+@@ -0,0 +1,48 @@
++From 95ea5226c870449522240ccff26f0b006037c520 Mon Sep 17 00:00:00 2001
++From: Vincent Rabaud 
++Date: Mon, 11 Sep 2023 16:06:08 +0200
++Subject: [PATCH] Fix invalid incremental decoding check.
++
++The first condition is only necessary if we have not read enough
++(enough being defined by src_last, not src_end which is the end
++of the image).
++The second condition now fits the comment below: "if not
++incremental, and we are past the end of buffer".
++
++BUG=oss-fuzz:62136
++
++Change-Id: I0700f67c62db8e1c02c2e429a069a71e606a5e4f
++---
++ src/dec/vp8l_dec.c | 15 +--
++ 1 file changed, 13 insertions(+), 2 deletions(-)
++
++diff --git a/src/dec/vp8l_dec.c b/src/dec/vp8l_dec.c
++index 5ab34f56..809b1aa9 100644
++--- a/src/dec/vp8l_dec.c
+ b/src/dec/vp8l_dec.c
++@@ -1233,9 +1233,20 @@ static int DecodeImageData(VP8LDecoder* const dec, 
uint32_t* const data,
++   }
++
++   br->eos_ = VP8LIsEndOfStream(br);
++-  if (dec->incremental_ && br->eos_ && src < src_end) {
+++  // In incremental decoding:
+++  // br->eos_ && src < src_last: if 'br' reached the end of the buffer and
+++  // 'src_last' has not been reached yet, there is not enough data. 'dec' 
has to
+++  // be reset until there is more data.
+++  // !br->eos_ && src < src_last: this cannot happen as either the buffer is
+++  // fully read, either enough has been read to reach 'src_last'.
+++  // src >= src_last: 'src_last' is reached, all is fine. 'src' can actually 
go
+++  // beyond 'src_last' in case the image is cropped and an LZ77 goes further.
+++  // The buffer might have been enough or there is some left. 'br->eos_' does
+++  // not matter.
+++  assert(!dec->incremental_ || (br->eos_ && src < src_last) || src >= 
src_last);
+++  if (dec->incremental_ && br->eos_ && src < src_last) {
++ RestoreState(dec);
++-  } else if (!br->eos_) {
+++  } else if ((dec->incremental_ && src >= src_last) || !br->eos_) {
++ // Process the remaining rows corresponding to last row-block.
++ if (process_func != NULL) {
++   process_func(dec, row > last_row ? last_row : row);
++--
++2.40.0
++
+diff --git a/src/dec/vp8l_dec.c b/src/dec/vp8l_dec.c
+index 186b0b2..59a9e64 100644
+--- a/src/dec/vp8l_dec.c
 b/src/dec/vp8l_dec.c
+@@ -1241,9 +1241,20 @@ static int DecodeImageData(VP8LDecoder* const dec, 
uint32_t* const data,
+   }
+
+   br->eos_ = VP8LIsEndOfStream(br);
+-  if (dec->incremental_ && br->eos_ && src < src_end) {
++  // In incremental decoding:
++  // br->eos_ && src < src_last: if 'br' reached the end of the buffer and
++  // 'src_last' has not been reached yet, there is not enough data. 'dec' has 
to
++  // be reset until there is more data.
++  // !br->eos_ && src < src_last: this cannot happen as either the buffer is
++  // fully read, either enough has been read to reach 'src_last'.
++  // src >= src_last: 'src_last' is reached, all is fine. 

[OE-core] [PATCH] package: split strip cmd when ccache is used

2023-10-30 Thread Javier Tia
Using ccache stopped to work after 77497dbdca with following error:

  FileNotFoundError: [Errno 2] No such file or directory: 'ccache 
aarch64-trs-linux-strip'

Signed-off-by: Javier Tia 
---
 meta/lib/oe/package.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index 1dd20f85eb..2685da0af9 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -39,7 +39,7 @@ def runstrip(arg):
 newmode = origmode | stat.S_IWRITE | stat.S_IREAD
 os.chmod(file, newmode)
 
-stripcmd = [strip]
+stripcmd = strip.split() if "ccache" in strip else [strip]
 skip_strip = False
 # kernel module
 if elftype & 16:
-- 
2.42.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189825): 
https://lists.openembedded.org/g/openembedded-core/message/189825
Mute This Topic: https://lists.openembedded.org/mt/102291706/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core][dunfell][PATCH] tiff: CVE patch correction for CVE-2023-3576

2023-10-30 Thread Vijay Anusuri via lists.openembedded.org
From: Vijay Anusuri 

- The commit 
[https://gitlab.com/libtiff/libtiff/-/commit/881a070194783561fd209b7c789a4e75566f7f37]
fixes CVE-2023-3576
- Hence, renamed the CVE-2023-3618-1.patch to CVE-2023-3576.patch
- Reference: https://security-tracker.debian.org/tracker/CVE-2023-3576
 https://security-tracker.debian.org/tracker/CVE-2023-3618

Signed-off-by: Vijay Anusuri 
---
 .../files/{CVE-2023-3618-1.patch => CVE-2023-3576.patch}  | 3 ++-
 .../files/{CVE-2023-3618-2.patch => CVE-2023-3618.patch}  | 0
 meta/recipes-multimedia/libtiff/tiff_4.1.0.bb | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)
 rename meta/recipes-multimedia/libtiff/files/{CVE-2023-3618-1.patch => 
CVE-2023-3576.patch} (93%)
 rename meta/recipes-multimedia/libtiff/files/{CVE-2023-3618-2.patch => 
CVE-2023-3618.patch} (100%)

diff --git a/meta/recipes-multimedia/libtiff/files/CVE-2023-3618-1.patch 
b/meta/recipes-multimedia/libtiff/files/CVE-2023-3576.patch
similarity index 93%
rename from meta/recipes-multimedia/libtiff/files/CVE-2023-3618-1.patch
rename to meta/recipes-multimedia/libtiff/files/CVE-2023-3576.patch
index 35ed852519..67837fe142 100644
--- a/meta/recipes-multimedia/libtiff/files/CVE-2023-3618-1.patch
+++ b/meta/recipes-multimedia/libtiff/files/CVE-2023-3576.patch
@@ -4,8 +4,9 @@ Date: Tue, 7 Mar 2023 15:02:08 +0800
 Subject: [PATCH] Fix memory leak in tiffcrop.c
 
 Upstream-Status: Backport 
[https://gitlab.com/libtiff/libtiff/-/commit/881a070194783561fd209b7c789a4e75566f7f37]
-CVE: CVE-2023-3618
+CVE: CVE-2023-3576
 Signed-off-by: Hitendra Prajapati 
+Signed-off-by: Vijay Anusuri 
 ---
  tools/tiffcrop.c | 7 ++-
  1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-multimedia/libtiff/files/CVE-2023-3618-2.patch 
b/meta/recipes-multimedia/libtiff/files/CVE-2023-3618.patch
similarity index 100%
rename from meta/recipes-multimedia/libtiff/files/CVE-2023-3618-2.patch
rename to meta/recipes-multimedia/libtiff/files/CVE-2023-3618.patch
diff --git a/meta/recipes-multimedia/libtiff/tiff_4.1.0.bb 
b/meta/recipes-multimedia/libtiff/tiff_4.1.0.bb
index 6df4244697..d27381b4cd 100644
--- a/meta/recipes-multimedia/libtiff/tiff_4.1.0.bb
+++ b/meta/recipes-multimedia/libtiff/tiff_4.1.0.bb
@@ -43,8 +43,8 @@ SRC_URI = 
"http://download.osgeo.org/libtiff/tiff-${PV}.tar.gz \
file://CVE-2023-26966.patch \
file://CVE-2023-2908.patch \
file://CVE-2023-3316.patch \
-   file://CVE-2023-3618-1.patch \
-   file://CVE-2023-3618-2.patch \
+   file://CVE-2023-3576.patch \
+   file://CVE-2023-3618.patch \
   "
 SRC_URI[md5sum] = "2165e7aba557463acc0664e71a3ed424"
 SRC_URI[sha256sum] = 
"5d29f32517dadb6dbcd1255ea5bbc93a2b54b94fbf83653b4d65c7d6775b8634"
-- 
2.25.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189824): 
https://lists.openembedded.org/g/openembedded-core/message/189824
Mute This Topic: https://lists.openembedded.org/mt/102291704/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] qemuriscv: Add to common MACHINE_FEATURES instead of overriding them

2023-10-30 Thread Alistair Francis
On Tue, Oct 31, 2023 at 5:16 AM Khem Raj  wrote:
>
> machine features like vfat are needed for ptests to pass ( e..g. parted)
> This brings it closer to what x86 qemu config looks like as well.
>
> Signed-off-by: Khem Raj 

Reviewed-by: Alistair Francis 

Alistair

> ---
>  meta/conf/machine/include/riscv/qemuriscv.inc | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/conf/machine/include/riscv/qemuriscv.inc 
> b/meta/conf/machine/include/riscv/qemuriscv.inc
> index 46ae66b9e50..7024bd0a4e6 100644
> --- a/meta/conf/machine/include/riscv/qemuriscv.inc
> +++ b/meta/conf/machine/include/riscv/qemuriscv.inc
> @@ -3,7 +3,7 @@ PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
>  require conf/machine/include/qemu.inc
>  require conf/machine/include/riscv/tune-riscv.inc
>
> -MACHINE_FEATURES = "screen keyboard ext2 ext3 serial"
> +MACHINE_FEATURES += "keyboard ext2 ext3 serial"
>
>  KERNEL_IMAGETYPE = "Image"
>  KERNEL_IMAGETYPES += "uImage"
> --
> 2.42.0
>
>
> 
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189823): 
https://lists.openembedded.org/g/openembedded-core/message/189823
Mute This Topic: https://lists.openembedded.org/mt/102282740/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [meta-oe][PATCH v2] volatile-binds: Calculate the name of the /var/lib service

2023-10-30 Thread Randy MacLeod via lists.openembedded.org

On 2023-10-28 4:17 a.m., Stéphane Veyret via lists.openembedded.org wrote:

Hello,

This patch and the previous one 
(https://lists.openembedded.org/g/openembedded-core/message/186778) 
have not been integrated yet. Have they been forgotten, should I do 
some modifications, or have they been definitly rejected ?


Hi Stephane,

The [meta-oe] prefix may have confused people anyway,
the two commits were merged this morning:

https://git.openembedded.org/openembedded-core/log/

https://git.openembedded.org/openembedded-core/commit/?id=66f0c2a1678cb69cf8d50372b0592c55e2dc3e3c

https://git.openembedded.org/openembedded-core/commit/?id=1ca031b77546056ca1994469b0f2e93ea2018edf

Thanks,

../Randy



Thanks.





--
# Randy MacLeod
# Wind River Linux

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189822): 
https://lists.openembedded.org/g/openembedded-core/message/189822
Mute This Topic: https://lists.openembedded.org/mt/101015300/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core][PATCH v6 03/12] devtool: new ide plugin

2023-10-30 Thread Adrian Freihofer
Hi Ross

Thank you for the re-view.

Summary: I hope everything is fixed with v7:
https://lists.openembedded.org/g/openembedded-core/message/189812

On Thu, 2023-10-12 at 12:53 +, Ross Burton wrote:
> Finally looking at the code…
> 
> On 10 Sep 2023, at 16:52, Adrian Freihofer via lists.openembedded.org
>  wrote:
> > 
> > The new devtool ide plugin configures an IDE to work with the eSDK.
> > 
> > With this initial implementation VSCode is the default IDE.
> > The plugin works for recipes inheriting the cmake or the meson
> > bbclass.
> > Support for more programming languages and build tools may be added
> > in
> > the future.
> 
> Can the vscode pieces be split out to a separate file, to separate
> out the high level logic and the vscode specifics.  I’m pretty sure
> there would be interest in adding qtcreator, for example.  By doing
> this sooner rather than later we avoid adding cheeky “lets just
> handle vscode specially” blocks.  Also, splitting out the
> cmake/meson/none logic where it isn’t IDE-specific.
> 
> Also one thing I’d like to encourage more is decent code
> documentation, to make the code easier to both review and work on in
> the future.  Could you add at least basic documentation to the
> classes and major functions, with a summary of how the plugin works?
> 
> > +    def __gdbserver_start_cmd(self, binary, port):
> > +    if self.gdbserver_multi:
> > +    gdbserver_cmd = "/usr/bin/gdbserver --multi :%s" % (
> > +    port)
> > +    else:
> > +    gdbserver_cmd = "/usr/bin/gdbserver --once :%s %s" % (
> > +    port, binary)
> 
> If I’m being pedantic, that should be ${bindir}.
> 
> > +    if 'debuginfod' in
> > image_d.getVar('DISTRO_FEATURES').split():
> > +    # image_config.debuginfod = True
> > +    logger.warning("Support for debuginfod is not
> > implemented yet.")
> 
> What doesn’t work, and why is this warning needed?

I removed the warnings related to debuginfod. Currently the rootfs-dbg
is used, but debuginfod should not harm and no warning is needed.

> 
> > +    def solib_search_path_rootfs(self):
> > +    """Search for folders with shared libraries in the rootfs
> > and rootfs-dbg
> > +
> > +    This is based on the assumption that the
> > PACKAGE_DEBUG_SPLIT_STYLE variable from the image
> > +    is the global setting which is used by most packages. Even
> > if this variable does not seem
> > +    to make sense in the image context.
> > +    """
> > +    rootfs_solib_search_path = []
> > +    rootfs_dbg_solib_search_path = []
> > +    if self.package_debug_split_style in ['debug-with-srcpkg',
> > '.debug']:
> > +    if self.combine_dbg_image:
> > +    rootfs_dbg_solib_search_path = [
> > +    "/lib", "/lib/.debug", "/usr/lib",
> > "/usr/lib/.debug"]
> > +    else:
> > +    logger.warn(
> > +    'Adding IMAGE_CLASSES += "image-combined-dbg"
> > offers better remote debugging experience.')
> > +    rootfs_solib_search_path = [
> > +    "/lib", "/usr/lib"]
> > +    rootfs_dbg_solib_search_path = [
> > +    "/lib/.debug", "/usr/lib/.debug"]
> > +    elif self.package_debug_split_style == 'debug-file-
> > directory':
> > +    rootfs_dbg_solib_search_path = ["/usr/lib/debug"]
> > +    else:
> > +    logger.warning(
> > +    "Cannot find solib search path for a rootfs built
> > with PACKAGE_DEBUG_SPLIT_STYLE=%s." %
> > self.package_debug_split_style)
> > +
> > +    sym_dirs = []
> > +    for dbgdir in rootfs_solib_search_path:
> > +    sym_dirs.append(os.path.join(
> > +    self.rootfs, dbgdir.lstrip('/')))
> > +    for dbgdir in rootfs_dbg_solib_search_path:
> > +    sym_dirs.append(os.path.join(
> > +    self.rootfs_dbg, dbgdir.lstrip('/')))
> > +
> > +    return sym_dirs
> 
> Feels like there should be a much better way to do this.  Does the
> choice of debug split style actually matter?  Just search all of the
> candidates and avoid trying to special-case everything?

It's probably still not as simple as you would expect. But it's much
simpler because I dropped the combined-debug-dbg cases. I prefer to
handle the different cases separately and provide a reasonable error
message to the user. There are so many reasons why the debugger cannot
find the symbols and sources.

> 
> I’d not noticed image-combined-dbg existed and do wonder if that
> shoud be the behaviour of the debug rootfs.  Is there actually a use-
> case for a tarball which is _just_ the symbols?
> 
Yes, it seems best to remove the image-combined-dbg.bbclass and make it
the default. There is now a patch that does this. This solves many
problems when tools need debug symbols and sources.

> > +class RecipeMetaIdeSupport:
> > +    """Handle some meta-ide-support recipe related properties"""
> 
> 

[OE-core] [PATCH v7 8/8] docs: cover devtool ide

2023-10-30 Thread Adrian Freihofer
Cover the new devtool ide plugin in the extensible sdk section.

Many thanks to Enguerrand de Ribaucourt for his re-view and
contributions.

Signed-off-by: Adrian Freihofer 
---
 documentation/sdk-manual/extensible.rst | 158 +++-
 1 file changed, 157 insertions(+), 1 deletion(-)

diff --git a/documentation/sdk-manual/extensible.rst 
b/documentation/sdk-manual/extensible.rst
index 355c6cb0e4a..361ca091fbf 100644
--- a/documentation/sdk-manual/extensible.rst
+++ b/documentation/sdk-manual/extensible.rst
@@ -230,13 +230,15 @@ all the commands.
See the ":doc:`/ref-manual/devtool-reference`"
section in the Yocto Project Reference Manual.
 
-Three ``devtool`` subcommands provide entry-points into development:
+Four ``devtool`` subcommands provide entry-points into development:
 
 -  *devtool add*: Assists in adding new software to be built.
 
 -  *devtool modify*: Sets up an environment to enable you to modify
the source of an existing component.
 
+-  *devtool ide*: Generates a configuration for an IDE.
+
 -  *devtool upgrade*: Updates an existing recipe so that you can
build it for an updated set of source files.
 
@@ -614,6 +616,160 @@ command:
   decide you do not want to proceed with your work. If you do use this
   command, realize that the source tree is preserved.
 
+Use ``devtool ide`` to generate a configuration for the IDE
+---
+
+``devtool ide`` automatically configures IDEs for cross-compiling and remote 
debugging.
+
+Two different use cases are supported:
+
+#. *Recipe mode*: Generate the IDE configuration for a workspace created by 
``devtool modify``.
+
+   In order to use the tool, a few settings must be made.
+   As a starting example, the following lines of code can be added to the 
local.conf file.
+
+   .. code-block::
+
+  # Build the companion debug file system
+  IMAGE_GEN_DEBUGFS = "1"
+  # Optimize build time: with devtool ide the dbg tar is not needed
+  IMAGE_FSTYPES_DEBUGFS = ""
+
+  # ssh is mandatory, no password simplifies the usage
+  EXTRA_IMAGE_FEATURES += "\
+ ssh-server-openssh \
+ debug-tweaks \
+  "
+
+  # Remote debugging needs the gdbserver on the target device
+  IMAGE_INSTALL:append = " gdbserver"
+
+   Assuming the development environment is set up correctly and a workspace 
has been created
+   for the recipe using ``devtool modify recipe``, the following command can 
create the
+   configuration for VSCode in the recipe workspace:
+
+   .. code-block::
+
+  $ devtool ide recipe core-image-minimal --target root@192.168.7.2
+
+   What this command does exactly depends on the recipe or the build tool used 
by the recipe.
+   Currently, only CMake and Meson are supported natively.
+
+   For a recipe which inherits ``cmake`` it does:
+
+   - Prepare the SDK by calling ``bitbake core-image-minimal``, ``gdb-cross``, 
``qemu-native``...
+
+   - Generate a cmake-preset with configures CMake to use exactly the same 
environent and
+ the same cmake-cache configuration as used by ``bitbake recipe``. The 
cmake-preset referres
+ to the per-recipe-sysroot of the recipe.
+
+ Currently Configure, Build and Test presets are supported. Test presets 
execute the test
+ binaries with Qemu.
+
+   - Generates a helper script to handle the ``do_install`` with pseudo
+
+   - Generates some helper scripts to start ``gdbserver`` on the target device
+
+   - Generates the ``.vscode`` folder containing the following files:
+
+   - ``c_ccp_properties.json``: configure the code navigation
+
+   - ``extensions.json``: Recommend the extensions which are used.
+
+   - ``launch.json``: Provide a configuration for remote debugging with 
``gdb-cross`` and ``gdbserver``.
+ The debug-symbols are searched in the build-folder, the 
per-recipe-sysroot and the rootfs-dbg
+ folder which is provided by the image.
+
+   - ``settings.json``: configure the indexer to ignore the build folders.
+
+   - ``tasks.json``: Provide some helpers for running
+
+  - do_install and ``devtool deploy-target``
+
+  - start the ``gdbserver`` via ssh
+
+   For a recipe which inherits meson a similar configuration is generated.
+   Because there is nothing like a meson-preset a wrapper script for meson is 
generated.
+
+   It's possible to pass multiple recipes to the ``devtool ide`` command.
+   ``devtool ide`` tries to handle the recipes in a reasonable way if possible.
+
+   ``devtool ide`` aims to support multiple programming languages and multiple 
IDEs natively.
+   Native means that the IDE is configured to call the build tool (e.g. cmake 
or meson) directly.
+   This has several advantages. First of all, it is much faster than ``devtool 
build``, for example.
+   But it also allows to use the very good integration of tools like CMake or 
GDB directly with VSCode or other IDEs.
+   However, supporting many programming languages and 

[OE-core] [PATCH v7 7/8] oe-selftest devtool: ide tests

2023-10-30 Thread Adrian Freihofer
Add some oe-selftests for the new devtool ide plugin. Most of the
workflows are covered.

Many thanks to Enguerrand de Ribaucourt for testing and bug fixing.

Signed-off-by: Adrian Freihofer 
---
 meta/lib/oeqa/selftest/cases/devtool.py | 274 
 1 file changed, 274 insertions(+)

diff --git a/meta/lib/oeqa/selftest/cases/devtool.py 
b/meta/lib/oeqa/selftest/cases/devtool.py
index b5c488be8e8..f7c478423ce 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -11,6 +11,7 @@ import tempfile
 import glob
 import fnmatch
 import unittest
+import json
 
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer
@@ -2199,3 +2200,276 @@ class DevtoolUpgradeTests(DevtoolBase):
 
 #Step 4.5
 runCmd("grep %s %s" % (modconfopt, codeconfigfile))
+
+
+
+class DevtoolIdeTests(DevtoolBase):
+def __write_bb_config(self, recipe_names):
+"""Helper to write the bitbake local.conf file"""
+conf_lines = [
+'IMAGE_GEN_DEBUGFS = "1"',
+'IMAGE_INSTALL:append = " gdbserver %s"' % ' '.join([r + '-ptest' 
for r in recipe_names])
+]
+self.write_config("\n".join(conf_lines))
+
+def __devtool_ide_recipe(self, recipe_name, build_file, testimage):
+"""Setup a recipe for working with devtool ide
+
+Basically devtool modify -x followed by some tests
+"""
+tempdir = tempfile.mkdtemp(prefix='devtoolqa')
+self.track_for_cleanup(tempdir)
+self.track_for_cleanup(self.workspacedir)
+self.add_command_to_tearDown('bitbake -c clean %s' % recipe_name)
+self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+
+result = runCmd('devtool modify %s -x %s' % (recipe_name, tempdir))
+self.assertExists(os.path.join(tempdir, build_file),
+'Extracted source could not be found')
+self.assertExists(os.path.join(self.workspacedir, 'conf',
+'layer.conf'), 'Workspace directory not created')
+matches = glob.glob(os.path.join(self.workspacedir,
+'appends', recipe_name + '.bbappend'))
+self.assertTrue(matches, 'bbappend not created %s' % result.output)
+
+# Test devtool status
+result = runCmd('devtool status')
+self.assertIn(recipe_name, result.output)
+self.assertIn(tempdir, result.output)
+self._check_src_repo(tempdir)
+
+# Usually devtool ide would initiate the build of the SDK.
+# But there is a circular dependency with starting Qemu and passing 
the IP of runqemu to devtool ide.
+bitbake("%s qemu-native qemu-helper-native" % testimage)
+deploy_dir_image = get_bb_var('DEPLOY_DIR_IMAGE')
+self.add_command_to_tearDown('bitbake -c clean %s' % testimage)
+self.add_command_to_tearDown('rm -f %s/%s*' % (deploy_dir_image, 
testimage))
+
+return tempdir
+
+def __get_recipe_ids(self, recipe_name):
+bb_vars = get_bb_vars(['PACKAGE_ARCH'])
+package_arch = bb_vars['PACKAGE_ARCH']
+recipe_id = recipe_name + "-" + package_arch
+recipe_id_pretty = recipe_name + ": " + package_arch
+return (recipe_id, recipe_id_pretty)
+
+def __verify_install_script_code(self, tempdir, recipe_name):
+"""Verify the scripts referred by the tasks.json file are fine.
+
+This function does not depend on Qemu. Therefore it verifies the 
scripts
+exists and the install step works as expected. But it does not try to
+deploy to Qemu.
+"""
+recipe_id, recipe_id_pretty = self.__get_recipe_ids(recipe_name)
+with open(os.path.join(tempdir, '.vscode', 'tasks.json')) as tasks_j:
+tasks_d = json.load(tasks_j)
+tasks = tasks_d["tasks"]
+task_install = next((task for task in tasks if task["label"] == 
"install && deploy-target %s"  % recipe_id_pretty), None)
+self.assertIsNot(task_install, None)
+install_deploy_cmd = task_install["command"]
+# execute only the bb_run_do_install script since the deploy would 
require e.g. Qemu running.
+workspace_temp = os.path.join(self.builddir, 'workspace', 'temp', 
recipe_name)
+i_and_d_script = "install_and_deploy_" + recipe_id
+i_and_d_script_path = os.path.join(workspace_temp, i_and_d_script)
+self.assertExists(i_and_d_script_path)
+i_script = "bb_run_do_install_" + recipe_id
+install_cmd = install_deploy_cmd.replace(i_and_d_script, i_script)
+install_cmd_path = os.path.join(workspace_temp, install_cmd)
+self.assertExists(install_cmd_path)
+runCmd(install_cmd, cwd=tempdir)
+
+def __devtool_ide_qemu(self, tempdir, qemu, recipe_name, example_exe):
+"""Verify deployment and execution in Qemu system work for one recipe.
+
+This 

[OE-core] [PATCH v7 6/8] devtool: new ide plugin

2023-10-30 Thread Adrian Freihofer
The new devtool ide plugin configures an IDE to work with the eSDK.

With this initial implementation VSCode is the default IDE.
The plugin works for recipes inheriting the cmake or the meson bbclass.
Support for more programming languages and build tools may be added in
the future.

Using the plugin in recipe modes:
$ devtool modify a-recipe
$ devtool ide a-recipe a-image
$ code "$BUILDDIR/workspace/sources/a-recipe"
Work in VSCode, after installing the proposed plugins

Using the plugin without a recipe
$ devtool ide none a-image
vscode where/the/sources/are
Use the cross tool-chain which is provided as a cmake-kit.

The goal of this implementation is to create a configuration for VSCode
(or other IDEs) that allows to work on the code of a recipe completely
independent from bitbake. bitbake is only called if the configuration or
the whole SDK has to be regenerated. But bitbake should not need to be
called while working in the IDE. This has two major advantages over
calling devtool build from the IDE:
- The IDE provides plugins for integration with cmake, for example.
  These features are usable, which would not be the case if bitbake or
  devtool are called from within the IDE.
- It is much faster.

Many thanks to Enguerrand de Ribaucourt for testing and bug fixing.

Signed-off-by: Adrian Freihofer 
---
 scripts/lib/devtool/ide.py   | 1104 ++
 scripts/lib/devtool/ide_handlers/__init__.py |   10 +
 scripts/lib/devtool/ide_handlers/ide_base.py |   46 +
 scripts/lib/devtool/ide_handlers/ide_code.py |  420 +++
 scripts/lib/devtool/ide_handlers/ide_none.py |   91 ++
 5 files changed, 1671 insertions(+)
 create mode 100755 scripts/lib/devtool/ide.py
 create mode 100644 scripts/lib/devtool/ide_handlers/__init__.py
 create mode 100644 scripts/lib/devtool/ide_handlers/ide_base.py
 create mode 100644 scripts/lib/devtool/ide_handlers/ide_code.py
 create mode 100644 scripts/lib/devtool/ide_handlers/ide_none.py

diff --git a/scripts/lib/devtool/ide.py b/scripts/lib/devtool/ide.py
new file mode 100755
index 000..2067e72ceb0
--- /dev/null
+++ b/scripts/lib/devtool/ide.py
@@ -0,0 +1,1104 @@
+#! /usr/bin/env python3
+#
+# Copyright (C) 2023 Siemens AG
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+"""Devtool ide plugin"""
+
+import os
+import stat
+import sys
+import logging
+import json
+import re
+import shutil
+import subprocess
+from argparse import RawTextHelpFormatter
+from enum import IntEnum, auto
+
+import bb
+from devtool import exec_build_env_command, setup_tinfoil, 
check_workspace_recipe, DevtoolError, parse_recipe
+from devtool.standard import get_real_srctree
+import devtool.ide_handlers
+
+SHARED_SYSROOT_RECIPES = ['shared', 'none',
+  'meta-ide-support', 'build-sysroots']
+SUPPORTED_IDES = ['code', 'none']
+
+logger = logging.getLogger('devtool')
+
+
+class DevtoolIdeMode(IntEnum):
+UNDEFINED = auto()
+DEVTOOL_MODIFY = auto()
+SHARED_SYSROOT = auto()
+
+
+class TargetDevice:
+"""SSH remote login parameters"""
+
+def __init__(self, args):
+self.extraoptions = ''
+if args.no_host_check:
+self.extraoptions += '-o UserKnownHostsFile=/dev/null -o 
StrictHostKeyChecking=no'
+self.ssh_sshexec = 'ssh'
+if args.ssh_exec:
+self.ssh_sshexec = args.ssh_exec
+self.ssh_port = ''
+if args.port:
+self.ssh_port = "-p %s" % args.port
+if args.key:
+self.extraoptions += ' -i %s' % args.key
+
+self.target = args.target
+target_sp = args.target.split('@')
+if len(target_sp) == 1:
+self.login = ""
+self.host = target_sp[0]
+elif len(target_sp) == 2:
+self.login = target_sp[0]
+self.host = target_sp[1]
+else:
+logger.error("Invalid target argument: %s" % args.target)
+
+@staticmethod
+def get_devtool_deploy_opts(args):
+"""Filter args for devtool deploy-target args"""
+if not args.target:
+return None
+devtool_deploy_opts = [args.target]
+if args.no_host_check:
+devtool_deploy_opts += ["-c"]
+if args.show_status:
+devtool_deploy_opts += ["-s"]
+if args.no_preserve:
+devtool_deploy_opts += ["-p"]
+if args.no_check_space:
+devtool_deploy_opts += ["--no-check-space"]
+if args.ssh_exec:
+devtool_deploy_opts += ["-e", args.ssh.exec]
+if args.port:
+devtool_deploy_opts += ["-P", args.port]
+if args.key:
+devtool_deploy_opts += ["-I", args.key]
+if args.strip is False:
+devtool_deploy_opts += ["--no-strip"]
+return devtool_deploy_opts
+
+
+class RecipeNative:
+"""Base class for calling bitbake to provide a -native recipe"""
+
+def __init__(self, name, target_arch=None):
+self.name = name
+self.target_arch = 

[OE-core] [PATCH v7 5/8] devtool: refactor deploy-target

2023-10-30 Thread Adrian Freihofer
Make the deploy function independent from d. This allows to call the
function also from Python code not running in bitbake.
This is needed to for the devtool ide plugin which will call the
do_install task and the code from devtool deploy-target independently
from a bitbake server. This allows a much quicker workflow.

Signed-off-by: Adrian Freihofer 
---
 scripts/lib/devtool/__init__.py |   5 +-
 scripts/lib/devtool/deploy.py   | 230 +---
 2 files changed, 124 insertions(+), 111 deletions(-)

diff --git a/scripts/lib/devtool/__init__.py b/scripts/lib/devtool/__init__.py
index 702db669de3..7d64547616e 100644
--- a/scripts/lib/devtool/__init__.py
+++ b/scripts/lib/devtool/__init__.py
@@ -78,12 +78,15 @@ def exec_fakeroot(d, cmd, **kwargs):
 """Run a command under fakeroot (pseudo, in fact) so that it picks up the 
appropriate file permissions"""
 # Grab the command and check it actually exists
 fakerootcmd = d.getVar('FAKEROOTCMD')
+fakerootenv = d.getVar('FAKEROOTENV')
+exec_fakeroot_no_d(fakerootcmd, fakerootenv, cmd, kwargs)
+
+def exec_fakeroot_no_d(fakerootcmd, fakerootenv, cmd, **kwargs):
 if not os.path.exists(fakerootcmd):
 logger.error('pseudo executable %s could not be found - have you run a 
build yet? pseudo-native should install this and if you have run any build then 
that should have been built')
 return 2
 # Set up the appropriate environment
 newenv = dict(os.environ)
-fakerootenv = d.getVar('FAKEROOTENV')
 for varvalue in fakerootenv.split():
 if '=' in varvalue:
 splitval = varvalue.split('=', 1)
diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py
index e14a5874177..ea7e2cb1ae8 100644
--- a/scripts/lib/devtool/deploy.py
+++ b/scripts/lib/devtool/deploy.py
@@ -16,7 +16,7 @@ import bb.utils
 import argparse_oe
 import oe.types
 
-from devtool import exec_fakeroot, setup_tinfoil, check_workspace_recipe, 
DevtoolError
+from devtool import exec_fakeroot_no_d, setup_tinfoil, check_workspace_recipe, 
DevtoolError
 
 logger = logging.getLogger('devtool')
 
@@ -133,16 +133,11 @@ def _prepare_remote_script(deploy, verbose=False, 
dryrun=False, undeployall=Fals
 
 return '\n'.join(lines)
 
-
-
-def deploy(args, config, basepath, workspace):
-"""Entry point for the devtool 'deploy' subcommand"""
+def deploy_cached(srcdir, workdir, path, strip_cmd, libdir, base_libdir, 
max_process, fakerootcmd, fakerootenv, args):
 import math
 import oe.recipeutils
 import oe.package
 
-check_workspace_recipe(workspace, args.recipename, checksrc=False)
-
 try:
 host, destdir = args.target.split(':')
 except ValueError:
@@ -152,116 +147,131 @@ def deploy(args, config, basepath, workspace):
 if not destdir.endswith('/'):
 destdir += '/'
 
+recipe_outdir = srcdir
+if not os.path.exists(recipe_outdir) or not os.listdir(recipe_outdir):
+raise DevtoolError('No files to deploy - have you built the %s '
+'recipe? If so, the install step has not installed '
+'any files.' % args.recipename)
+
+if args.strip and not args.dry_run:
+# Fakeroot copy to new destination
+srcdir = recipe_outdir
+recipe_outdir = os.path.join(workdir, 'devtool-deploy-target-stripped')
+if os.path.isdir(recipe_outdir):
+exec_fakeroot_no_d(fakerootcmd, fakerootenv, "rm -rf %s" % 
recipe_outdir, shell=True)
+exec_fakeroot_no_d(fakerootcmd, fakerootenv, "cp -af %s %s" % 
(os.path.join(srcdir, '.'), recipe_outdir), shell=True)
+os.environ['PATH'] = ':'.join([os.environ['PATH'], path or ''])
+oe.package.strip_execs(args.recipename, recipe_outdir, strip_cmd, 
libdir, base_libdir, max_process)
+
+filelist = []
+inodes = set({})
+ftotalsize = 0
+for root, _, files in os.walk(recipe_outdir):
+for fn in files:
+fstat = os.lstat(os.path.join(root, fn))
+# Get the size in kiB (since we'll be comparing it to the output 
of du -k)
+# MUST use lstat() here not stat() or getfilesize() since we don't 
want to
+# dereference symlinks
+if fstat.st_ino in inodes:
+fsize = 0
+else:
+fsize = int(math.ceil(float(fstat.st_size)/1024))
+inodes.add(fstat.st_ino)
+ftotalsize += fsize
+# The path as it would appear on the target
+fpath = os.path.join(destdir, os.path.relpath(root, 
recipe_outdir), fn)
+filelist.append((fpath, fsize))
+
+if args.dry_run:
+print('Files to be deployed for %s on target %s:' % (args.recipename, 
args.target))
+for item, _ in filelist:
+print('  %s' % item)
+return 0
+
+extraoptions = ''
+if args.no_host_check:
+extraoptions += '-o UserKnownHostsFile=/dev/null -o 
StrictHostKeyChecking=no'
+if not 

[OE-core] [PATCH v7 4/8] tests: add a C++ example recipe

2023-10-30 Thread Adrian Freihofer
This simple C++ project supports compilation with cmake, meson and
autotools.
It's supposed to be used with oe-selftest for the devtool ide plugin.

Signed-off-by: Adrian Freihofer 
---
 meta-selftest/recipes-test/cpp/.gitignore |  1 +
 .../recipes-test/cpp/autotools-example.bb | 18 ++
 .../cpp/autotools-example/run-ptest   | 10 
 .../recipes-test/cpp/cmake-example.bb | 17 ++
 .../recipes-test/cpp/cmake-example/run-ptest  | 10 
 .../recipes-test/cpp/cpp-example.inc  | 24 
 .../recipes-test/cpp/files/CMakeLists.txt | 60 +++
 .../recipes-test/cpp/files/Makefile.am| 13 
 .../recipes-test/cpp/files/configure.ac   | 11 
 .../cpp/files/cpp-example-lib.cpp | 17 ++
 .../cpp/files/cpp-example-lib.hpp | 16 +
 .../recipes-test/cpp/files/cpp-example.cpp| 16 +
 .../recipes-test/cpp/files/meson.build| 34 +++
 .../cpp/files/test-cpp-example.cpp| 20 +++
 .../recipes-test/cpp/meson-example.bb | 17 ++
 .../recipes-test/cpp/meson-example/run-ptest  | 10 
 16 files changed, 294 insertions(+)
 create mode 100644 meta-selftest/recipes-test/cpp/.gitignore
 create mode 100644 meta-selftest/recipes-test/cpp/autotools-example.bb
 create mode 100644 meta-selftest/recipes-test/cpp/autotools-example/run-ptest
 create mode 100644 meta-selftest/recipes-test/cpp/cmake-example.bb
 create mode 100644 meta-selftest/recipes-test/cpp/cmake-example/run-ptest
 create mode 100644 meta-selftest/recipes-test/cpp/cpp-example.inc
 create mode 100644 meta-selftest/recipes-test/cpp/files/CMakeLists.txt
 create mode 100644 meta-selftest/recipes-test/cpp/files/Makefile.am
 create mode 100644 meta-selftest/recipes-test/cpp/files/configure.ac
 create mode 100644 meta-selftest/recipes-test/cpp/files/cpp-example-lib.cpp
 create mode 100644 meta-selftest/recipes-test/cpp/files/cpp-example-lib.hpp
 create mode 100644 meta-selftest/recipes-test/cpp/files/cpp-example.cpp
 create mode 100644 meta-selftest/recipes-test/cpp/files/meson.build
 create mode 100644 meta-selftest/recipes-test/cpp/files/test-cpp-example.cpp
 create mode 100644 meta-selftest/recipes-test/cpp/meson-example.bb
 create mode 100644 meta-selftest/recipes-test/cpp/meson-example/run-ptest

diff --git a/meta-selftest/recipes-test/cpp/.gitignore 
b/meta-selftest/recipes-test/cpp/.gitignore
new file mode 100644
index 000..30d388a12b7
--- /dev/null
+++ b/meta-selftest/recipes-test/cpp/.gitignore
@@ -0,0 +1 @@
+build*
\ No newline at end of file
diff --git a/meta-selftest/recipes-test/cpp/autotools-example.bb 
b/meta-selftest/recipes-test/cpp/autotools-example.bb
new file mode 100644
index 000..f5d8aa48154
--- /dev/null
+++ b/meta-selftest/recipes-test/cpp/autotools-example.bb
@@ -0,0 +1,18 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+SUMMARY = "A C++ example compiled with autotools."
+
+inherit autotools
+
+require cpp-example.inc
+
+SRC_URI += "\
+file://configure.ac \
+file://Makefile.am \
+"
+
+FILES:${PN}-ptest += "${bindir}/test-autotools-example"
diff --git a/meta-selftest/recipes-test/cpp/autotools-example/run-ptest 
b/meta-selftest/recipes-test/cpp/autotools-example/run-ptest
new file mode 100644
index 000..51548259886
--- /dev/null
+++ b/meta-selftest/recipes-test/cpp/autotools-example/run-ptest
@@ -0,0 +1,10 @@
+#!/bin/sh
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+testautoexample
+
+# Note: run-ptests exits with exit value from test-cmake-example
diff --git a/meta-selftest/recipes-test/cpp/cmake-example.bb 
b/meta-selftest/recipes-test/cpp/cmake-example.bb
new file mode 100644
index 000..96d543180b4
--- /dev/null
+++ b/meta-selftest/recipes-test/cpp/cmake-example.bb
@@ -0,0 +1,17 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+SUMMARY = "A C++ example compiled with cmake."
+
+inherit cmake
+
+require cpp-example.inc
+
+SRC_URI += "\
+file://CMakeLists.txt \
+"
+
+FILES:${PN}-ptest += "${bindir}/test-cmake-example"
diff --git a/meta-selftest/recipes-test/cpp/cmake-example/run-ptest 
b/meta-selftest/recipes-test/cpp/cmake-example/run-ptest
new file mode 100644
index 000..94b620a1984
--- /dev/null
+++ b/meta-selftest/recipes-test/cpp/cmake-example/run-ptest
@@ -0,0 +1,10 @@
+#!/bin/sh
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+test-cmake-example
+
+# Note: run-ptests exits with exit value from test-cmake-example
diff --git a/meta-selftest/recipes-test/cpp/cpp-example.inc 
b/meta-selftest/recipes-test/cpp/cpp-example.inc
new file mode 100644
index 000..39c61cf4ceb
--- /dev/null
+++ b/meta-selftest/recipes-test/cpp/cpp-example.inc
@@ -0,0 +1,24 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = 

[OE-core] [PATCH v7 3/8] image-combined-dbg: make this the default

2023-10-30 Thread Adrian Freihofer
Remove the image-combined-dbg.bbclass and make this the default
behavior for the rootfs-dbg. A rootfs-dbg with only debug symbols but
no executable binaries also causes problems with gdb, which is
probably the most common use case for the roofs-dbg. This change
simplifies and improves the user experience for a slightly larger
rootfs-dbg.

If the rootfs-dbg contains a complete copy of the rootfs, it is also
usable for booting the target device over the network. This in turn
simplifies other use cases with e.g. the use of perf on a device
booted over the network.

Signed-off-by: Adrian Freihofer 
---
 .../classes-recipe/image-combined-dbg.bbclass | 15 
 meta/lib/oe/rootfs.py | 35 ---
 scripts/crosstap  | 28 +--
 3 files changed, 7 insertions(+), 71 deletions(-)
 delete mode 100644 meta/classes-recipe/image-combined-dbg.bbclass

diff --git a/meta/classes-recipe/image-combined-dbg.bbclass 
b/meta/classes-recipe/image-combined-dbg.bbclass
deleted file mode 100644
index 729313739c1..000
--- a/meta/classes-recipe/image-combined-dbg.bbclass
+++ /dev/null
@@ -1,15 +0,0 @@
-#
-# Copyright OpenEmbedded Contributors
-#
-# SPDX-License-Identifier: MIT
-#
-
-IMAGE_PREPROCESS_COMMAND:append = " combine_dbg_image"
-
-combine_dbg_image () {
-if [ "${IMAGE_GEN_DEBUGFS}" = "1" -a -e ${IMAGE_ROOTFS}-dbg ]; then
-# copy target files into -dbg rootfs, so it can be used for
-# debug purposes directly
-tar -C ${IMAGE_ROOTFS} -cf - . | tar -C ${IMAGE_ROOTFS}-dbg 
-xf -
-fi
-}
diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py
index 1a48ed10b3f..a1cc28a6dd0 100644
--- a/meta/lib/oe/rootfs.py
+++ b/meta/lib/oe/rootfs.py
@@ -111,40 +111,17 @@ class Rootfs(object, metaclass=ABCMeta):
 if gen_debugfs != '1':
return
 
+rootfs_orig = self.image_rootfs + '-orig'
+
 bb.note("  Renaming the original rootfs...")
 try:
-shutil.rmtree(self.image_rootfs + '-orig')
+shutil.rmtree(rootfs_orig)
 except:
 pass
-bb.utils.rename(self.image_rootfs, self.image_rootfs + '-orig')
+bb.utils.rename(self.image_rootfs, rootfs_orig)
 
 bb.note("  Creating debug rootfs...")
-bb.utils.mkdirhier(self.image_rootfs)
-
-bb.note("  Copying back package database...")
-for path in package_paths:
-bb.utils.mkdirhier(self.image_rootfs + os.path.dirname(path))
-if os.path.isdir(self.image_rootfs + '-orig' + path):
-shutil.copytree(self.image_rootfs + '-orig' + path, 
self.image_rootfs + path, symlinks=True)
-elif os.path.isfile(self.image_rootfs + '-orig' + path):
-shutil.copyfile(self.image_rootfs + '-orig' + path, 
self.image_rootfs + path)
-
-# Copy files located in /usr/lib/debug or /usr/src/debug
-for dir in ["/usr/lib/debug", "/usr/src/debug"]:
-src = self.image_rootfs + '-orig' + dir
-if os.path.exists(src):
-dst = self.image_rootfs + dir
-bb.utils.mkdirhier(os.path.dirname(dst))
-shutil.copytree(src, dst)
-
-# Copy files with suffix '.debug' or located in '.debug' dir.
-for root, dirs, files in os.walk(self.image_rootfs + '-orig'):
-relative_dir = root[len(self.image_rootfs + '-orig'):]
-for f in files:
-if f.endswith('.debug') or '/.debug' in relative_dir:
-bb.utils.mkdirhier(self.image_rootfs + relative_dir)
-shutil.copy(os.path.join(root, f),
-self.image_rootfs + relative_dir)
+shutil.copytree(rootfs_orig, self.image_rootfs, symlinks=True)
 
 bb.note("  Install complementary '*-dbg' packages...")
 self.pm.install_complementary('*-dbg')
@@ -178,7 +155,7 @@ class Rootfs(object, metaclass=ABCMeta):
 bb.utils.rename(self.image_rootfs, self.image_rootfs + '-dbg')
 
 bb.note("  Restoring original rootfs...")
-bb.utils.rename(self.image_rootfs + '-orig', self.image_rootfs)
+bb.utils.rename(rootfs_orig, self.image_rootfs)
 
 def _exec_shell_cmd(self, cmd):
 try:
diff --git a/scripts/crosstap b/scripts/crosstap
index 5aa72f14d44..87dac33e064 100755
--- a/scripts/crosstap
+++ b/scripts/crosstap
@@ -170,18 +170,6 @@ class BitbakeEnv(object):
 return ret
 
 class ParamDiscovery(object):
-SYMBOLS_CHECK_MESSAGE = """
-WARNING: image '%s' does not have dbg-pkgs IMAGE_FEATURES enabled and no
-"image-combined-dbg" in inherited classes is specified. As result the image
-does not have symbols for user-land processes DWARF based probes. Consider
-adding 'dbg-pkgs' to EXTRA_IMAGE_FEATURES or adding "image-combined-dbg" to
-USER_CLASSES. I.e add this line 'USER_CLASSES += "image-combined-dbg"' to
-local.conf file.
-
-Or you 

[OE-core] [PATCH v7 2/8] cmake.bbclass: support qemu

2023-10-30 Thread Adrian Freihofer
Define the CMAKE_CROSSCOMPILING_EMULATOR variable similar to what the
meson bbclass does. This allows for example to execute cross compilied
unit tests on the build machine.

CMAKE_CROSSCOMPILING_EMULATOR is a semi colon separated list of
paramters which could directly handle the -L and the -E parameters.
Creating a wrapper script is not absolutely mandatory. But anyway lets
do it similar to what the meson.bbclass does and also disable pseudo.

Signed-off-by: Adrian Freihofer 
---
 meta/classes-recipe/cmake.bbclass | 20 
 1 file changed, 20 insertions(+)

diff --git a/meta/classes-recipe/cmake.bbclass 
b/meta/classes-recipe/cmake.bbclass
index d978b889440..911c237a3fd 100644
--- a/meta/classes-recipe/cmake.bbclass
+++ b/meta/classes-recipe/cmake.bbclass
@@ -4,6 +4,13 @@
 # SPDX-License-Identifier: MIT
 #
 
+inherit qemu
+
+EXEWRAPPER_ENABLED:class-native = "False"
+EXEWRAPPER_ENABLED:class-nativesdk = "False"
+EXEWRAPPER_ENABLED ?= "${@bb.utils.contains('MACHINE_FEATURES', 
'qemu-usermode', 'True', 'False', d)}"
+DEPENDS:append = "${@' qemu-native' if d.getVar('EXEWRAPPER_ENABLED') == 
'True' else ''}"
+
 # Path to the CMake file to process.
 OECMAKE_SOURCEPATH ??= "${S}"
 
@@ -156,6 +163,19 @@ EOF
 
 addtask generate_toolchain_file after do_patch before do_configure
 
+cmake_do_generate_toolchain_file:append:class-target() {
+if [ "${EXEWRAPPER_ENABLED}" = "True" ]; then
+# Write out a qemu wrapper that will be used as exe_wrapper so that 
camake
+# can run target helper binaries through that. This also allows to 
execute ctest.
+qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_HOST}', 
['${STAGING_DIR_HOST}/${libdir}','${STAGING_DIR_HOST}/${base_libdir}'])}"
+echo "#!/bin/sh" > "${WORKDIR}/cmake-qemuwrapper"
+echo "$qemu_binary \"\$@\"" >> "${WORKDIR}/cmake-qemuwrapper"
+chmod +x "${WORKDIR}/cmake-qemuwrapper"
+echo "set( CMAKE_CROSSCOMPILING_EMULATOR 
${WORKDIR}/cmake-qemuwrapper)" \
+  >> ${WORKDIR}/toolchain.cmake
+fi
+}
+
 CONFIGURE_FILES = "CMakeLists.txt *.cmake"
 
 do_configure[cleandirs] = "${@d.getVar('B') if d.getVar('S') != d.getVar('B') 
else ''}"
-- 
2.41.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189814): 
https://lists.openembedded.org/g/openembedded-core/message/189814
Mute This Topic: https://lists.openembedded.org/mt/102285616/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH v7 0/8] devtool ide plugin

2023-10-30 Thread Adrian Freihofer
Changes in comparison to v6:

- Fix findigs: https://lists.openembedded.org/g/openembedded-core/message/188995
  - Major refactoring to split IDE specific parts into separate files
  - Use bindir for gdbserver
  - Remove pontless warning for debuginfod
  - Simplified searching for debug symbols
  - Added a new patch that makes the behavior of image-combined-dbg.bbclass the
default behavior of rootfs-dbg. I think this solves a whole bunch of 
problems
not only for GDB but also for all the other tools that need the debug 
symbols.
- Dropped the workaround for the pseudo breakage with sources in tree
  After a cleanup and improving the script to better replicate what bitbake does
  I was no longer able to reproduce the issue.
- Improve the default behavior for not yet supported build tools. devtool build
  and devtool deploy-target are called.
- Improve the tests:
  - Start Qemu only once (it's expensive)
  - Test also the ide=none
- Improve documentation about the no recipe mode (to docs mailing list)
- Details and bug fixes

According to 
https://www.yoctoproject.org/community/yocto-project-engineering-request-for-quotation/
one of the proposed areas for development of the Yocto project is "VSCode IDE 
Integration - New developer tooling".
One aspect of this larger topic is helping application developers configure the 
IDE to work on a recipe's source code using Yocto's eSDK. This patchset 
provides a new devtool plugin (devtool ide) that does this fully automatically 
for recipes inheriting the cmake or the meson bbclass. Support for more 
programming languages and build tools may be added in the future.

Let's start with a brief introduction of how it looks like from a user's 
perspective.

Reference setup
---

$ . oe-init-build-env

Add the following layers to bblayers.conf file:
- meta
- meta-poky
- meta-yocto-bsp
- meta-selftest

Add the following to the local.conf

IMAGE_GEN_DEBUGFS = "1"
IMAGE_FEATURES += "ssh-server-dropbear debug-tweaks"
IMAGE_INSTALL:append = "\
cmake-example-ptest \
meson-example-ptest \
gdbserver \
"

Working on a recipe
---

Note for those who "hate cmake with passion": Replace cmake by meson in the 
following commands.

1. Add the recipe which should be modified to the workspace

   $ devtool modify cmake-example

2. Build the eSDK. This provides all the host tools as well as an image which 
can be used on the target device.
   Since everything is built with one bitbake command also all the debug 
symbols are expected to be consistent.

   $ devtool ide cmake-example core-image-minimal

3. Install the image on the target device or simply use Qemu

   $ runqemu

4. Start VSCode

   $ code "$BUILDDIR/workspace/sources/cmake-example"

5. Work in VSCode, after installing the proposed plugins

   Ctrl + Shift + p, cmake Select Configure Preset
   Ctrl + Shift + p, cmake Build

6. Execute the unit tests on the host works with Qemu even for cross compiled 
test executables.

   Ctrl + Shift + p, cmake Test

7. Remote debugging with GDB

   Ctrl + Shift + p, Run Task --> install && deploy-target cmake-example
   F5 (Debug configuration might be selected before)

8. Work on the recipes and call bitbake again to get the SDK updated.
   Building the application by calling cmake (from the IDE) or by calling 
bitbake or by calling devtool build is expected to produce the exact same 
results. To make that possible, the devtool ide plugin is designed to configure 
the IDE to call cmake with the exact same configuration as used by bitbake when 
building the recipe. Unlike the eSDK, the goal here is clearly that there is no 
longer a separation between the SDK and the application development process. 
Regardless of which tool is called, the same source files are edited and the 
same o-files are generated and re-used.

Working with a SDK without a recipe
---

If devtool ide is called for a recipe named none or meta-ide-support the eSDK 
with its generic environment file gets generated.
In case of using VSCode and cmake in addition to the well known environment 
file a cmake-kit
https://vector-of-bool.github.io/docs/vscode-cmake-tools/kits.html is added to 
the User-Local Kits.
This allows to work with cmake calling the cross-toolchain out of VSCode or a 
shell with the environment file sourced.

Design
--

The goal of this implementation is to create a configuration for VSCode (or 
other IDEs) that allows to work on the code of a recipe completely independent 
from bitbake. bitbake is only called if the configuration or the whole SDK has 
to be regenerated. But bitbake should not need to be called while working in 
the IDE. This has two major advantages over calling devtool build from the IDE:
- The IDE provides plugins for integration with cmake, for example. These 
features are usable, which would not be the case if bitbake or devtool are 
called from within the IDE.
- It is much faster.

Supporting other 

[OE-core] [PATCH v7 1/8] vscode: add minimal configuration

2023-10-30 Thread Adrian Freihofer
It is essential to configure VSCode indexer plugins to ignore the build
folder of bitbake. Otherwise, the indexer plugins run with 100% CPU load
until an OOM exception occurs. In practice, this makes VSCode more or
less unusable for working with Yocto until a file like the one added by
this commit is deployed before VSCode starts. From the user's point of
view, it is not obvious why the system runs at 100% CPU load and
eventually crashes.

It is even more misleading that VSCode starts the indexers immediately,
but does not stop or reconfigure them when the ignore list is updated.
In practice, this means that every time the ignore list is changed,
VSCode immediately starts indexing the build folder until the OOM
exception stops it. Depending on the system's OOM handler, the entire
build machine may crash.
Particularly annoying is the Python plugin that ignores the general
ignore list and requires an extra ignore section.

The settings are suitable for workflows like bitbake, devtool modify,
devtool reset. The settings are not intended to work on the source code
of a recipe. It is assumed that a separate instance of VSCode is used
per workspace folder. These per workspace instances can have different
settings depending on the details of the sources that come with the
recipe. The new devtool ide plugin will generate settings to match this.

Signed-off-by: Adrian Freihofer 
---
 .gitignore|  2 ++
 .vscode/settings.json | 32 
 2 files changed, 34 insertions(+)
 create mode 100644 .vscode/settings.json

diff --git a/.gitignore b/.gitignore
index 8f48d452dab..f6ce090b5fc 100644
--- a/.gitignore
+++ b/.gitignore
@@ -36,3 +36,5 @@ _toaster_clones/
 downloads/
 sstate-cache/
 toaster.sqlite
+.vscode/
+vscode-bitbake-build/
diff --git a/.vscode/settings.json b/.vscode/settings.json
new file mode 100644
index 000..517a86d1bfa
--- /dev/null
+++ b/.vscode/settings.json
@@ -0,0 +1,32 @@
+{
+"files.watcherExclude": {
+"**/.git/**": true,
+"**/cache/**": true,
+"**/tmp*/**": true,
+"**/downloads/**": true,
+"**/sstate-cache/**": true,
+"**/vscode-bitbake-build/**": true,
+"**/workspace/sources/**": true,
+"**/workspace/attic/**": true
+},
+"files.exclude": {
+"**/.git/**": true,
+"**/cache/**": true,
+"**/tmp*/**": true,
+"**/downloads/**": true,
+"**/sstate-cache/**": true,
+"**/vscode-bitbake-build/**": true,
+"**/workspace/sources/**": true,
+"**/workspace/attic/**": true
+},
+"python.analysis.exclude": [
+"**/.git/**",
+"**/cache/**",
+"**/tmp*/**",
+"**/downloads/**",
+"**/sstate-cache/**",
+"**/vscode-bitbake-build/**",
+"**/workspace/sources/**",
+"**/workspace/attic/**"
+]
+}
-- 
2.41.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189813): 
https://lists.openembedded.org/g/openembedded-core/message/189813
Mute This Topic: https://lists.openembedded.org/mt/102285615/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] All new features advertised in the Nanbield release notes?

2023-10-30 Thread Michael Opdenacker via lists.openembedded.org

Dear OE-core contributors,

I've reviewed all the commits specific to the Nanbield branch and came 
up with this draft or the release and migration notes for this upcoming 
release:


 * https://docs.yoctoproject.org/next/migration-guides/migration-4.3.html
 * https://docs.yoctoproject.org/next/migration-guides/release-notes-4.3.html

If you made a contribution that is featured in the new release, it would 
be great if you could check whether it's advertised in the release notes 
with sufficient detail. I'm pretty sure some noteworthy changes didn't 
catch my attention.


In particular, I'm pretty sure that the runqemu changes should be 
properly summarized, as well at the CVE management ones.


This way, new users will be aware of your new feature(s)!

Of course, I'm also interested in reviews of the text. Some sections are 
still empty, but some of them will be filled by our usual scripts when 
the release is ready. Some of them are also empty because I haven't 
found any noteworthy changes yet. The latter will be removed if they 
should indeed be empty.


And don't hesitate to send patches against the yocto-docs "master-next" 
branch!

Thanks in advance
Cheers
Michael.


--
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189811): 
https://lists.openembedded.org/g/openembedded-core/message/189811
Mute This Topic: https://lists.openembedded.org/mt/102284588/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH] kernel.bbclass: Use strip utility used for kernel build in do_package

2023-10-30 Thread Martin Jansa
This doesn't work well with ccache.bbclass when KERNEL_STRIP is prefixed
with ccache as:
KERNEL_STRIP="ccache aarch64-oe-linux-strip "

do_package then fails with:
ERROR: Fatal errors occurred in subprocesses:
[Errno 2] No such file or directory: 'ccache x86_64-oe-linux-strip':
Traceback (most recent call last):
  File "TOPDIR/oe-core/meta/lib/oe/utils.py", line 288, in run
ret = self._target(*self._args, **self._kwargs)
  ^
  File "TOPDIR/oe-core/meta/lib/oe/package.py", line 66, in runstrip
output = subprocess.check_output(stripcmd, stderr=subprocess.STDOUT)
 ^^^
  File "/usr/lib/python3.11/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
   ^
  File "/usr/lib/python3.11/subprocess.py", line 548, in run
with Popen(*popenargs, **kwargs) as process:
 ^^^
  File "/usr/lib/python3.11/subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.11/subprocess.py", line 1950, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'ccache
x86_64-oe-linux-strip'

On Wed, Oct 25, 2023 at 12:07 AM Khem Raj  wrote:

> os.environ does not pass this down to runstrip() function and in
> strip_execs() its using STRIP bitbake variable to find the strip utility
> to use. Since there might be a trailing whitespace in KERNEL_STRIP
> remove that otherwise python is not able to launch it.
> e.g.
>
> FileNotFoundError: [Errno 2] No such file or directory:
> 'riscv64-yoe-linux-strip '
>
> This is more evident when STRIP and KERNEL_STRIP are different utilities
> e.g. when using clang as default toolchain but using gcc+binutils only for
> kernel build.
>
> Signed-off-by: Khem Raj 
> Cc: Bruce Ashfield 
> ---
>  meta/classes-recipe/kernel.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-recipe/kernel.bbclass
> b/meta/classes-recipe/kernel.bbclass
> index 2ec9ea2091e..16b85dbca48 100644
> --- a/meta/classes-recipe/kernel.bbclass
> +++ b/meta/classes-recipe/kernel.bbclass
> @@ -336,7 +336,7 @@ kernel_do_transform_bundled_initramfs() {
>  do_transform_bundled_initramfs[dirs] = "${B}"
>
>  python do_package:prepend () {
> -os.environ['STRIP'] = d.getVar('KERNEL_STRIP')
> +d.setVar('STRIP',  d.getVar('KERNEL_STRIP').strip())
>  }
>
>  python do_devshell:prepend () {
> --
> 2.42.0
>
>
> 
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189810): 
https://lists.openembedded.org/g/openembedded-core/message/189810
Mute This Topic: https://lists.openembedded.org/mt/102167569/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] qemuriscv: Add to common MACHINE_FEATURES instead of overriding them

2023-10-30 Thread Khem Raj
machine features like vfat are needed for ptests to pass ( e..g. parted)
This brings it closer to what x86 qemu config looks like as well.

Signed-off-by: Khem Raj 
---
 meta/conf/machine/include/riscv/qemuriscv.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/machine/include/riscv/qemuriscv.inc 
b/meta/conf/machine/include/riscv/qemuriscv.inc
index 46ae66b9e50..7024bd0a4e6 100644
--- a/meta/conf/machine/include/riscv/qemuriscv.inc
+++ b/meta/conf/machine/include/riscv/qemuriscv.inc
@@ -3,7 +3,7 @@ PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot"
 require conf/machine/include/qemu.inc
 require conf/machine/include/riscv/tune-riscv.inc
 
-MACHINE_FEATURES = "screen keyboard ext2 ext3 serial"
+MACHINE_FEATURES += "keyboard ext2 ext3 serial"
 
 KERNEL_IMAGETYPE = "Image"
 KERNEL_IMAGETYPES += "uImage"
-- 
2.42.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189809): 
https://lists.openembedded.org/g/openembedded-core/message/189809
Mute This Topic: https://lists.openembedded.org/mt/102282740/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core][PATCH v2] license.bbclass: prefere plus singnal over append on IMAGE_CLASSES

2023-10-30 Thread Fabio Estevam
On Mon, Oct 30, 2023 at 3:54 PM Jose Quaresma  wrote:
>
> The append override is much harder to remove so change that.
>
> Signed-off-by: Jose Quaresma 
> ---
>
>  v2: fix typo in 'change'

There is another typo in the Subject in the word "signal".

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189808): 
https://lists.openembedded.org/g/openembedded-core/message/189808
Mute This Topic: https://lists.openembedded.org/mt/102282317/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core][PATCH v2] license.bbclass: prefere plus singnal over append on IMAGE_CLASSES

2023-10-30 Thread Jose Quaresma
The append override is much harder to remove so change that.

Signed-off-by: Jose Quaresma 
---

 v2: fix typo in 'change'

 meta/classes-global/license.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-global/license.bbclass 
b/meta/classes-global/license.bbclass
index b2e0d3faba..b915d39acd 100644
--- a/meta/classes-global/license.bbclass
+++ b/meta/classes-global/license.bbclass
@@ -418,7 +418,7 @@ SSTATETASKS += "do_populate_lic"
 do_populate_lic[sstate-inputdirs] = "${LICSSTATEDIR}"
 do_populate_lic[sstate-outputdirs] = "${LICENSE_DIRECTORY}/"
 
-IMAGE_CLASSES:append = " license_image"
+IMAGE_CLASSES += "license_image"
 
 python do_populate_lic_setscene () {
 sstate_setscene(d)
-- 
2.42.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189807): 
https://lists.openembedded.org/g/openembedded-core/message/189807
Mute This Topic: https://lists.openembedded.org/mt/102282317/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core][PATCH] license.bbclass: prefere plus singnal over append on IMAGE_CLASSES

2023-10-30 Thread Jose Quaresma
Khem Raj  escreveu no dia segunda, 30/10/2023 à(s)
18:18:

> On Mon, Oct 30, 2023 at 11:04 AM Jose Quaresma 
> wrote:
> >
> > The append override is much harder to remove so chnage that.
>
> typo in 'change'
> however, I think one can do :remove op to remove it intentionally,
> with this change it will
> need to be redefined by user who is using license bbclass explicitly,
> which is less intuitive.
>

The :remove is still possible to use and work as before.
One side effect of this patch is the order on IMAGE_CLASSES
where license_image can go from one of the last to one of the first
as well as what you pointed out of redefining the IMAGE_CLASSES.
However this change is not critical at all and can be discarded,
the main reason was because there has been some effort to use + instead of
the :append

Jose



>
> >
> > Signed-off-by: Jose Quaresma 
> > ---
> >  meta/classes-global/license.bbclass | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/meta/classes-global/license.bbclass
> b/meta/classes-global/license.bbclass
> > index b2e0d3faba..b915d39acd 100644
> > --- a/meta/classes-global/license.bbclass
> > +++ b/meta/classes-global/license.bbclass
> > @@ -418,7 +418,7 @@ SSTATETASKS += "do_populate_lic"
> >  do_populate_lic[sstate-inputdirs] = "${LICSSTATEDIR}"
> >  do_populate_lic[sstate-outputdirs] = "${LICENSE_DIRECTORY}/"
> >
> > -IMAGE_CLASSES:append = " license_image"
> > +IMAGE_CLASSES += "license_image"
> >
> >  python do_populate_lic_setscene () {
> >  sstate_setscene(d)
> > --
> > 2.42.0
> >
> >
> > 
> >
>


-- 
Best regards,

José Quaresma

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189806): 
https://lists.openembedded.org/g/openembedded-core/message/189806
Mute This Topic: https://lists.openembedded.org/mt/102281098/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core][PATCH] license.bbclass: prefere plus singnal over append on IMAGE_CLASSES

2023-10-30 Thread Khem Raj
On Mon, Oct 30, 2023 at 11:04 AM Jose Quaresma  wrote:
>
> The append override is much harder to remove so chnage that.

typo in 'change'
however, I think one can do :remove op to remove it intentionally,
with this change it will
need to be redefined by user who is using license bbclass explicitly,
which is less intuitive.

>
> Signed-off-by: Jose Quaresma 
> ---
>  meta/classes-global/license.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-global/license.bbclass 
> b/meta/classes-global/license.bbclass
> index b2e0d3faba..b915d39acd 100644
> --- a/meta/classes-global/license.bbclass
> +++ b/meta/classes-global/license.bbclass
> @@ -418,7 +418,7 @@ SSTATETASKS += "do_populate_lic"
>  do_populate_lic[sstate-inputdirs] = "${LICSSTATEDIR}"
>  do_populate_lic[sstate-outputdirs] = "${LICENSE_DIRECTORY}/"
>
> -IMAGE_CLASSES:append = " license_image"
> +IMAGE_CLASSES += "license_image"
>
>  python do_populate_lic_setscene () {
>  sstate_setscene(d)
> --
> 2.42.0
>
>
> 
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189805): 
https://lists.openembedded.org/g/openembedded-core/message/189805
Mute This Topic: https://lists.openembedded.org/mt/102281098/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core][PATCH] license.bbclass: prefere plus singnal over append on IMAGE_CLASSES

2023-10-30 Thread Jose Quaresma
The append override is much harder to remove so chnage that.

Signed-off-by: Jose Quaresma 
---
 meta/classes-global/license.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-global/license.bbclass 
b/meta/classes-global/license.bbclass
index b2e0d3faba..b915d39acd 100644
--- a/meta/classes-global/license.bbclass
+++ b/meta/classes-global/license.bbclass
@@ -418,7 +418,7 @@ SSTATETASKS += "do_populate_lic"
 do_populate_lic[sstate-inputdirs] = "${LICSSTATEDIR}"
 do_populate_lic[sstate-outputdirs] = "${LICENSE_DIRECTORY}/"
 
-IMAGE_CLASSES:append = " license_image"
+IMAGE_CLASSES += "license_image"
 
 python do_populate_lic_setscene () {
 sstate_setscene(d)
-- 
2.42.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189804): 
https://lists.openembedded.org/g/openembedded-core/message/189804
Mute This Topic: https://lists.openembedded.org/mt/102281098/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH 3/5] meta-yocto-bsp/conf/machine: remove SERIAL_CONSOLES_CHECK

2023-10-30 Thread Fabio Estevam
Hi Ross,

On Tue, Oct 10, 2023 at 9:37 AM Ross Burton  wrote:
>
> From: Ross Burton 
>
> There's no need for this variable anymore, as all consoles listed in
> SERIAL_CONSOLES are checked for their existence before a getty is
> started.
> ---

Your Signed-off-by tag is missing.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189803): 
https://lists.openembedded.org/g/openembedded-core/message/189803
Mute This Topic: https://lists.openembedded.org/mt/101873854/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[yocto][OE-core] Notification: Patchtest service going live

2023-10-30 Thread Trevor Gamblin

Hello all,

This is to notify the community that the Patchtest service will be going 
live today to provide patch feedback. For now, this is exclusive to 
submissions related to the openembedded-core mailing list, but support 
for other lists is planned in the future. If/when the service detects 
issues with a submission, a reply will be sent to the submitter (CCing 
the openembedded-core list) with a list of failing, passing, and skipped 
tests. If you have feedback or suggestions for patchtest, please submit 
an issue via Bugzilla (https://bugzilla.yoctoproject.org/) under the 
"Yocto Project Subprojects" -> "Patchtest" category.


For more information on patchtest, see the READMEs or the Wiki page:

1. 
https://git.openembedded.org/openembedded-core/tree/scripts/patchtest.README


2. https://git.yoctoproject.org/meta-patchtest/tree/README.md

3. https://wiki.yoctoproject.org/wiki/Patchtest

- Trevor



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189802): 
https://lists.openembedded.org/g/openembedded-core/message/189802
Mute This Topic: https://lists.openembedded.org/mt/102278882/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [PATCH 3/5] meta-yocto-bsp/conf/machine: remove SERIAL_CONSOLES_CHECK

2023-10-30 Thread Michael Opdenacker via lists.openembedded.org

Greetings,

On 10.10.23 at 14:36, Ross Burton wrote:

From: Ross Burton 

There's no need for this variable anymore, as all consoles listed in
SERIAL_CONSOLES are checked for their existence before a getty is
started.
---
  meta-yocto-bsp/conf/machine/beaglebone-yocto.conf | 1 -
  meta-yocto-bsp/conf/machine/genericx86-64.conf| 1 -
  2 files changed, 2 deletions(-)

diff --git a/meta-yocto-bsp/conf/machine/beaglebone-yocto.conf 
b/meta-yocto-bsp/conf/machine/beaglebone-yocto.conf
index 8b67cefef70..9f389711b3b 100644
--- a/meta-yocto-bsp/conf/machine/beaglebone-yocto.conf
+++ b/meta-yocto-bsp/conf/machine/beaglebone-yocto.conf
@@ -18,7 +18,6 @@ MACHINE_ESSENTIAL_EXTRA_RDEPENDS += "kernel-image 
kernel-devicetree"
  do_image_wic[depends] += "mtools-native:do_populate_sysroot 
dosfstools-native:do_populate_sysroot virtual/bootloader:do_deploy"
  
  SERIAL_CONSOLES ?= "115200;ttyS0 115200;ttyO0 115200;ttyAMA0"

-SERIAL_CONSOLES_CHECK = "${SERIAL_CONSOLES}"
  
  PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"

  PREFERRED_VERSION_linux-yocto ?= "6.1%"
diff --git a/meta-yocto-bsp/conf/machine/genericx86-64.conf 
b/meta-yocto-bsp/conf/machine/genericx86-64.conf
index 14913ea1f15..f19a1c1527c 100644
--- a/meta-yocto-bsp/conf/machine/genericx86-64.conf
+++ b/meta-yocto-bsp/conf/machine/genericx86-64.conf
@@ -6,6 +6,5 @@ DEFAULTTUNE ?= "core2-64"
  require conf/machine/include/x86/tune-core2.inc
  require conf/machine/include/genericx86-common.inc
  
-SERIAL_CONSOLES_CHECK = "ttyS0"

  #For runqemu
  QB_SYSTEM_NAME = "qemu-system-x86_64"



Oops, this patch hasn't been taken yet in poky-contrib. Any reason why?
Getting rid of SERIAL_CONSOLES_CHECK for good would allow me to remove 
it from the documentation.


Cheers
Michael.

--
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189801): 
https://lists.openembedded.org/g/openembedded-core/message/189801
Mute This Topic: https://lists.openembedded.org/mt/101873854/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core][PATCH] patchtest-send-results: send results to submitter

2023-10-30 Thread Trevor Gamblin
Modify patchtest-send-results so that it extracts the submitter's email
address and responds to them with the patch testresults. Also make a
minor adjustment to the suggestions provided with each email and include
a link to the Patchtest wiki page for additional clarification on
specific failures.

Signed-off-by: Trevor Gamblin 
---
 scripts/patchtest-send-results | 19 +--
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/scripts/patchtest-send-results b/scripts/patchtest-send-results
index 078651ec381..01b071159be 100755
--- a/scripts/patchtest-send-results
+++ b/scripts/patchtest-send-results
@@ -17,6 +17,7 @@ import boto3
 import configparser
 import mailbox
 import os
+import re
 import sys
 
 greeting = """Thank you for your submission. Patchtest identified one
@@ -25,10 +26,12 @@ more information:\n\n---\n"""
 
 suggestions = """\n---\n\nPlease address the issues identified and
 submit a new revision of the patch, or alternatively, reply to this
-email with an explanation of why the patch format should be accepted. If
-you believe these results are due to an error in patchtest, please
-submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest'
-category under 'Yocto Project Subprojects'). Thank you!"""
+email with an explanation of why the patch should be accepted. If you
+believe these results are due to an error in patchtest, please submit a
+bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
+under 'Yocto Project Subprojects'). For more information on specific
+failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
+you!"""
 
 parser = argparse.ArgumentParser(description="Send patchtest results to a 
submitter for a given patch")
 parser.add_argument("-p", "--patch", dest="patch", required=True, help="The 
patch file to summarize")
@@ -54,6 +57,10 @@ mbox = mailbox.mbox(args.patch)
 mbox_subject = mbox[0]['subject']
 subject_line = f"Patchtest results for {mbox_subject}"
 
+# extract the submitter email address and use it as the reply address
+# for the results
+reply_address = re.findall("<(.*)>", mbox[0]['from'])
+
 if "FAIL" in testresult:
 reply_contents = None
 if len(max(open(result_file, 'r'), key=len)) > 220:
@@ -66,9 +73,9 @@ if "FAIL" in testresult:
 response = ses_client.send_email(
 Source='patcht...@automation.yoctoproject.org',
 Destination={
-'ToAddresses': ['test-l...@lists.yoctoproject.org'],
+'CcAddresses': ['openembedded-core@lists.openembedded.org'],
 },
-ReplyToAddresses=['test-l...@lists.yoctoproject.org'],
+ReplyToAddresses=reply_address,
 Message={
 'Subject': {
 'Data': subject_line,
-- 
2.41.0


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189800): 
https://lists.openembedded.org/g/openembedded-core/message/189800
Mute This Topic: https://lists.openembedded.org/mt/102277940/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Patchtest results for [OE-core][PATCH] patchtest: shorten test result outputs

2023-10-30 Thread Trevor Gamblin
Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch 
/home/patchtest/share/mboxes/patchtest-shorten-test-result-outputs.patch

FAIL: test lic files chksum modified not mentioned: LIC_FILES_CHKSUM changed 
without "License-Update:" tag and description in commit message 
(test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)

PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test Signed-off-by presence 
(test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence 
(test_mbox.TestMbox.test_commit_message_presence)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)

SKIP: pretest lic files chksum modified not mentioned: No modified recipes, 
skipping pretest 
(test_metadata.TestMetadata.pretest_lic_files_chksum_modified_not_mentioned)
SKIP: pretest src uri left files: No modified recipes, skipping pretest 
(test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE presence in commit message: No new patches introduced 
(test_mbox.TestMbox.test_cve_presence_in_commit_message)
SKIP: test CVE tag format: No new CVE patches introduced 
(test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced 
(test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced 
(test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found 
(test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now 
(test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest 
(test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_summary_presence)
SKIP: test target mailing list: Series merged, no reason to check other mailing 
lists (test_mbox.TestMbox.test_target_mailing_list)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189799): 
https://lists.openembedded.org/g/openembedded-core/message/189799
Mute This Topic: https://lists.openembedded.org/mt/102275009/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-10-30 Thread Alexander Kanavin
On Mon, 30 Oct 2023 at 15:07, Richard Purdie
 wrote:
> > a) esdk sets a number of variables from its initialization script that
> > aid with cross-compiling components directly (e.g. the core use case
> > of SDKs). Normal mode doesn't do that, but recently added
> > meta-ide-support will generate a similar initialization script that
> > will set up the same environment from the normal mode. There are tests
> > and documentation for it.
>
> In that case, this one is something we can document as how to make the
> functionality available in the normal build.

This has been documented in the sdk manual:
https://docs.yoctoproject.org/sdk-manual/extensible.html#two-ways-to-install-the-extensible-sdk
https://docs.yoctoproject.org/sdk-manual/extensible.html#installing-additional-items-into-the-extensible-sdk

Actually, I have on purpose presented both options in the manual as
'esdk', the standalone installer option is simply locked down more
(task signatures and tooling wise) than the direct yocto build.

So here's what could be done:

- esdk tools become symlinks in poky/scripts/esdk-tools/. esdk
environment script puts that in PATH, rather than some custom
esdk-specific location (the code to generate that can then be
dropped).

- esdk tweaks to local.conf move into a dedicated include file, which
can be static and under version control, except for perhaps
METADATA_REVISION:poky = "4a1e0b9625729e422fcf24e632ee2a3c79f986d5" -
I need to check why is it there and how that is used.

Something else perhaps?

Alex

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189798): 
https://lists.openembedded.org/g/openembedded-core/message/189798
Mute This Topic: https://lists.openembedded.org/mt/101356420/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-10-30 Thread Richard Purdie
On Mon, 2023-10-30 at 14:50 +0100, Alexander Kanavin wrote:
> On Thu, 14 Sept 2023 at 14:56, Richard Purdie
>  wrote:
> > There are design elements to this work. We need to work out how we can
> > make eSDK and "normal" builds more similar and less of an overhead to
> > switch between one and the other. A "bblock all" command does partly
> > get you to an eSDK effectively so part of this may be switching eSDK to
> > use the new lock command. What other differences are there? What other
> > differences are necessary or make sense for the use cases eSDK was
> > designed for? How would you turn an existing build into an eSDK like
> > one? Could you provide a copy of a local build to someone else easily
> > using something like eSDK's tooling? What does the eSDK look like at
> > the end of this. One section we don't have good answers to yet is setup
> > and configuration although I know you've started on some of that.
> 
> So I see the following differences between esdk and normal modes:
> 
> 1. Environment and tooling availability.
> 
> a) esdk sets a number of variables from its initialization script that
> aid with cross-compiling components directly (e.g. the core use case
> of SDKs). Normal mode doesn't do that, but recently added
> meta-ide-support will generate a similar initialization script that
> will set up the same environment from the normal mode. There are tests
> and documentation for it.

In that case, this one is something we can document as how to make the
functionality available in the normal build.

> b) PATH. eSDK has a number of items in PATH that point to various
> locations inside tmp/sysroots/, collectively they provide the
> cross-toolchain.
> 
> eSDK also puts a selection of yocto tools into path - wic, devtool but
> not bitbake:
> 
> 
> alex@Zen2:~/poky_sdk$ ls -l sysroots/x86_64-pokysdk-linux/usr/bin/
> total 48
> lrwxrwxrwx 1 alex alex  39 Oct 30 12:52 devtool ->
> ../../../../layers/poky/scripts/devtool
> lrwxrwxrwx 1 alex alex  54 Oct 30 12:52 oe-find-native-sysroot ->
> ../../../../layers/poky/scripts/oe-find-native-sysroot
> lrwxrwxrwx 1 alex alex  42 Oct 30 12:52 recipetool ->
> ../../../../layers/poky/scripts/recipetool
> lrwxrwxrwx 1 alex alex  39 Oct 30 12:52 runqemu ->
> ../../../../layers/poky/scripts/runqemu
> lrwxrwxrwx 1 alex alex  55 Oct 30 12:52 runqemu-addptable2image ->
> ../../../../layers/poky/scripts/runqemu-addptable2image
> lrwxrwxrwx 1 alex alex  53 Oct 30 12:52 runqemu-export-rootfs ->
> ../../../../layers/poky/scripts/runqemu-export-rootfs
> lrwxrwxrwx 1 alex alex  51 Oct 30 12:52 runqemu-extract-sdk ->
> ../../../../layers/poky/scripts/runqemu-extract-sdk
> lrwxrwxrwx 1 alex alex  51 Oct 30 12:52 runqemu-gen-tapdevs ->
> ../../../../layers/poky/scripts/runqemu-gen-tapdevs
> lrwxrwxrwx 1 alex alex  46 Oct 30 12:52 runqemu-ifdown ->
> ../../../../layers/poky/scripts/runqemu-ifdown
> lrwxrwxrwx 1 alex alex  44 Oct 30 12:52 runqemu-ifup ->
> ../../../../layers/poky/scripts/runqemu-ifup
> lrwxrwxrwx 1 alex alex 100 Oct 30 12:52 unfsd ->
> ../../../../tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/recipe-sysroot-native/usr/bin/unfsd
> lrwxrwxrwx 1 alex alex  35 Oct 30 12:52 wic ->
> ../../../../layers/poky/scripts/wic
> ==
> 
> 'normal mode' puts bitbake/bin/ and oe-core/scripts in PATH.
> Cross-toolchain can be added by the same environment script made by
> meta-ide-support as mentioned in 1a.

Right, so in theory we can change PATH and change this which can also
easily be documented.

> 2. Configuration (e.g. local.conf).
> 
> eSDK local.conf is local.conf from the normal mode that was used to
> produce eSDK, stripped of all comments, and with a bunch of extra
> settings:
> 
> 
> INHERIT:remove = "buildhistory icecc"
> CONNECTIVITY_CHECK_URIS = ""
> 
> SIGGEN_LOCKEDSIGS_SSTATE_EXISTS_CHECK = "none"
> 
> SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "warn"
> 
> BB_HASHCONFIG_IGNORE_VARS:append = " SIGGEN_UNLOCKED_RECIPES"
> 
> BB_SETSCENE_ENFORCE_IGNORE_TASKS = "%:* *:do_shared_workdir
> *:do_rm_work wic-tools:* *:do_addto_recipe_sysroot"
> 
> BUILDCFG_HEADER = ""
> 
> METADATA_REVISION:poky = "4a1e0b9625729e422fcf24e632ee2a3c79f986d5"
> 
> # Provide a flag to indicate we are in the EXT_SDK Context
> WITHIN_EXT_SDK = "1"
> 
> SSTATE_MIRRORS += " file://universal/(.*) file://universal-4.9/\1
> file://universal-4.9/(.*) file://universal-4.8/\1"
> 

Perhaps some of this should become a generic include file which is
included which would then make it easy to document as how to lock
things down to match it? That would take the config out of the class
file too which is probably nicer.

> require conf/locked-sigs.inc
> require conf/unlocked-sigs.inc
> ===
> 
> Devtool also has a special configuration:
> =
> alex@Zen2:~/poky_sdk$ cat conf/devtool.conf
> [General]
> bitbake_subdir = layers/poky/bitbake
> init_path = layers/poky/oe-init-build-env
> 

Re: [OE-core] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?

2023-10-30 Thread Alexander Kanavin
On Thu, 14 Sept 2023 at 14:56, Richard Purdie
 wrote:
> There are design elements to this work. We need to work out how we can
> make eSDK and "normal" builds more similar and less of an overhead to
> switch between one and the other. A "bblock all" command does partly
> get you to an eSDK effectively so part of this may be switching eSDK to
> use the new lock command. What other differences are there? What other
> differences are necessary or make sense for the use cases eSDK was
> designed for? How would you turn an existing build into an eSDK like
> one? Could you provide a copy of a local build to someone else easily
> using something like eSDK's tooling? What does the eSDK look like at
> the end of this. One section we don't have good answers to yet is setup
> and configuration although I know you've started on some of that.

So I see the following differences between esdk and normal modes:

1. Environment and tooling availability.

a) esdk sets a number of variables from its initialization script that
aid with cross-compiling components directly (e.g. the core use case
of SDKs). Normal mode doesn't do that, but recently added
meta-ide-support will generate a similar initialization script that
will set up the same environment from the normal mode. There are tests
and documentation for it.

b) PATH. eSDK has a number of items in PATH that point to various
locations inside tmp/sysroots/, collectively they provide the
cross-toolchain.

eSDK also puts a selection of yocto tools into path - wic, devtool but
not bitbake:


alex@Zen2:~/poky_sdk$ ls -l sysroots/x86_64-pokysdk-linux/usr/bin/
total 48
lrwxrwxrwx 1 alex alex  39 Oct 30 12:52 devtool ->
../../../../layers/poky/scripts/devtool
lrwxrwxrwx 1 alex alex  54 Oct 30 12:52 oe-find-native-sysroot ->
../../../../layers/poky/scripts/oe-find-native-sysroot
lrwxrwxrwx 1 alex alex  42 Oct 30 12:52 recipetool ->
../../../../layers/poky/scripts/recipetool
lrwxrwxrwx 1 alex alex  39 Oct 30 12:52 runqemu ->
../../../../layers/poky/scripts/runqemu
lrwxrwxrwx 1 alex alex  55 Oct 30 12:52 runqemu-addptable2image ->
../../../../layers/poky/scripts/runqemu-addptable2image
lrwxrwxrwx 1 alex alex  53 Oct 30 12:52 runqemu-export-rootfs ->
../../../../layers/poky/scripts/runqemu-export-rootfs
lrwxrwxrwx 1 alex alex  51 Oct 30 12:52 runqemu-extract-sdk ->
../../../../layers/poky/scripts/runqemu-extract-sdk
lrwxrwxrwx 1 alex alex  51 Oct 30 12:52 runqemu-gen-tapdevs ->
../../../../layers/poky/scripts/runqemu-gen-tapdevs
lrwxrwxrwx 1 alex alex  46 Oct 30 12:52 runqemu-ifdown ->
../../../../layers/poky/scripts/runqemu-ifdown
lrwxrwxrwx 1 alex alex  44 Oct 30 12:52 runqemu-ifup ->
../../../../layers/poky/scripts/runqemu-ifup
lrwxrwxrwx 1 alex alex 100 Oct 30 12:52 unfsd ->
../../../../tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/recipe-sysroot-native/usr/bin/unfsd
lrwxrwxrwx 1 alex alex  35 Oct 30 12:52 wic ->
../../../../layers/poky/scripts/wic
==

'normal mode' puts bitbake/bin/ and oe-core/scripts in PATH.
Cross-toolchain can be added by the same environment script made by
meta-ide-support as mentioned in 1a.

2. Configuration (e.g. local.conf).

eSDK local.conf is local.conf from the normal mode that was used to
produce eSDK, stripped of all comments, and with a bunch of extra
settings:


INHERIT:remove = "buildhistory icecc"
CONNECTIVITY_CHECK_URIS = ""

SIGGEN_LOCKEDSIGS_SSTATE_EXISTS_CHECK = "none"

SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "warn"

BB_HASHCONFIG_IGNORE_VARS:append = " SIGGEN_UNLOCKED_RECIPES"

BB_SETSCENE_ENFORCE_IGNORE_TASKS = "%:* *:do_shared_workdir
*:do_rm_work wic-tools:* *:do_addto_recipe_sysroot"

BUILDCFG_HEADER = ""

METADATA_REVISION:poky = "4a1e0b9625729e422fcf24e632ee2a3c79f986d5"

# Provide a flag to indicate we are in the EXT_SDK Context
WITHIN_EXT_SDK = "1"

SSTATE_MIRRORS += " file://universal/(.*) file://universal-4.9/\1
file://universal-4.9/(.*) file://universal-4.8/\1"

require conf/locked-sigs.inc
require conf/unlocked-sigs.inc
===

Devtool also has a special configuration:
=
alex@Zen2:~/poky_sdk$ cat conf/devtool.conf
[General]
bitbake_subdir = layers/poky/bitbake
init_path = layers/poky/oe-init-build-env
core_meta_subdir = layers/poky/meta

[SDK]
sdk_targets = core-image-minimal
==
There is currently no tooling to add/remove these extras in either
esdk mode or normal mode as far as I understand. Their individual
purposes and effects are also not exactly clear to me, and need to be
investigated one by one.

3. Setting up a normal mode in a eSDK installation.

This is actually pretty easy: rather than sourcing the sdk environment
script, source the poky/oe-init-build-env:
==
alex@Zen2:~/poky_sdk$ . layers/poky/oe-init-build-env .

### Shell environment set up for builds. ###
...
===
Bitbake will then simply run, albeit with all those extra
esdk-specific items in 

Patchtest results for [OE-core][PATCH] patchtest: shorten test result outputs

2023-10-30 Thread Trevor Gamblin
Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch 
/home/patchtest/share/mboxes/patchtest-shorten-test-result-outputs.patch

FAIL: test lic files chksum modified not mentioned: LIC_FILES_CHKSUM changed 
without "License-Update:" tag and description in commit message 
(test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)

PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test Signed-off-by presence 
(test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence 
(test_mbox.TestMbox.test_commit_message_presence)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)

SKIP: pretest lic files chksum modified not mentioned: No modified recipes, 
skipping pretest 
(test_metadata.TestMetadata.pretest_lic_files_chksum_modified_not_mentioned)
SKIP: pretest src uri left files: No modified recipes, skipping pretest 
(test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE presence in commit message: No new patches introduced 
(test_mbox.TestMbox.test_cve_presence_in_commit_message)
SKIP: test CVE tag format: No new CVE patches introduced 
(test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced 
(test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced 
(test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found 
(test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now 
(test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest 
(test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_summary_presence)
SKIP: test target mailing list: Series merged, no reason to check other mailing 
lists (test_mbox.TestMbox.test_target_mailing_list)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch format should be accepted. If
you believe these results are due to an error in patchtest, please
submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest'
category under 'Yocto Project Subprojects'). Thank you!

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189795): 
https://lists.openembedded.org/g/openembedded-core/message/189795
Mute This Topic: https://lists.openembedded.org/mt/102275009/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] image_types.bbclass: Use xz default compression preset level

2023-10-30 Thread Niko Mauno via lists.openembedded.org
From: Niko Mauno 

Commit ef0654f1453ff0afe98d7e921626b2a96cf2f6f6
("Set XZ_COMPRESSION_LEVEL to -9") changed the xz compression preset
level from previous value of -3 to -9. The commit message explains that
the change was made in order to be consistent with other compressors
that also use their best compression. However looking at xz man page,
under the compression preset level selection chapter there is mentioned
that

  The  differences  between  the  presets  are more significant than with 
gzip(1) and
  bzip2(1).  The selected compression settings determine the memory  
requirements  of
  the  decompressor,  thus  using  a  too  high preset level might make it 
painful to
  decompress the file on an old system with little RAM.   Specifically,  it's  
not  a
  good  idea  to  blindly  use  -9  for  everything like it often is with 
gzip(1) and
  bzip2(1).

which is then followed by a table, which mentions that the decompressor
memory requirement for preset -9 is 65 MiB, whereas for xz default
preset -6 it is just 9 MiB. Given that the use case where a device
running a Yocto generated Linux OS decompresses an ext4 root filesystem
image to non-volatile memory as part of firmware upgrade process is not
far-fetched, and considering that a range of these devices can run low
on available RAM when there are other applications running at the same
time, the lower decompressor memory requirement of the default preset
level makes sense in order to prevent an OOM situation from occurring.

This change was tested on a 32 CPU core build host with 128 GB RAM by
issuing

  $ bitbake -c cleansstate core-image-minimal core-image-sato
  $ time bitbake core-image-minimal
  $ time bitbake core-image-sato

With MACHINE="qemux86-64" and IMAGE_FSTYPES="ext4 ext4.xz" using
XZ_COMPRESSION_LEVEL values "-6" and "-9". In both cases the resulting
'ext4' image size remained same, 38141952 bytes for core-image-minimal,
and 565043200 bytes for core-image-sato.

The observation was that with this change there is a small increase in
the resulting 'ext4.xz' file size, and a build speed improvement that
was significant for larger rootfs image.

core-image  XZ  real timetime deltaext4.xz size  size delta
---
minimal -9  0m44.992s  15932508
minimal -6  0m42.445s-5.66%16243484 +1.95%
sato-9  2m40.828s  85080416
sato-6  1m38.891s   -38.51%87447456 +2.78%

Regarding decompression speed, issuing following command in qemux86-64
target OS

  $ time xz -dkc --memlimit=MEMLIMIT core-image-sato-qemux86-64.rootfs.ext4.xz 
> /dev/null

using the lowest accepted value for MEMLIMIT for each case (providing a
lower value caused xz to exit with 'Memory usage limit reached' error)
showed that decompression time saw a minuscule improvement with the -6
compression preset level:

XZ   MEMLIMIT   real time
-
-965M0m43.83s
-6 9M0m43.28s

(In the above tables, XZ refers to XZ_COMPRESSION_LEVEL value used when
images were generated with Yocto).

Signed-off-by: Niko Mauno 
---
 meta/classes-recipe/image_types.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-recipe/image_types.bbclass 
b/meta/classes-recipe/image_types.bbclass
index 4aed64e27f..d615b41ed1 100644
--- a/meta/classes-recipe/image_types.bbclass
+++ b/meta/classes-recipe/image_types.bbclass
@@ -54,7 +54,7 @@ def imagetypes_getdepends(d):
 # Sort the set so that ordering is consistant
 return " ".join(sorted(deps))
 
-XZ_COMPRESSION_LEVEL ?= "-9"
+XZ_COMPRESSION_LEVEL ?= "-6"
 XZ_INTEGRITY_CHECK ?= "crc32"
 
 ZIP_COMPRESSION_LEVEL ?= "-9"
-- 
2.39.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189794): 
https://lists.openembedded.org/g/openembedded-core/message/189794
Mute This Topic: https://lists.openembedded.org/mt/102274378/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] [PATCH] qemu: Upgrade 8.1.0 -> 8.1.2

2023-10-30 Thread Richard Purdie
Drop three backport patches as they're applied upstream.

Signed-off-by: Richard Purdie 
---
 ...u-native_8.1.0.bb => qemu-native_8.1.2.bb} |   0
 ...e_8.1.0.bb => qemu-system-native_8.1.2.bb} |   0
 meta/recipes-devtools/qemu/qemu.inc   |   5 +-
 ...t-data-in-bounds-in-iotlb_to_section.patch |  42 -
 ...u-Use-async_run_on_cpu-in-tcg_commit.patch | 157 --
 .../qemu/qemu/CVE-2023-42467.patch|  49 --
 .../qemu/{qemu_8.1.0.bb => qemu_8.1.2.bb} |   0
 7 files changed, 1 insertion(+), 252 deletions(-)
 rename meta/recipes-devtools/qemu/{qemu-native_8.1.0.bb => 
qemu-native_8.1.2.bb} (100%)
 rename meta/recipes-devtools/qemu/{qemu-system-native_8.1.0.bb => 
qemu-system-native_8.1.2.bb} (100%)
 delete mode 100644 
meta/recipes-devtools/qemu/qemu/0001-softmmu-Assert-data-in-bounds-in-iotlb_to_section.patch
 delete mode 100644 
meta/recipes-devtools/qemu/qemu/0001-softmmu-Use-async_run_on_cpu-in-tcg_commit.patch
 delete mode 100644 meta/recipes-devtools/qemu/qemu/CVE-2023-42467.patch
 rename meta/recipes-devtools/qemu/{qemu_8.1.0.bb => qemu_8.1.2.bb} (100%)

diff --git a/meta/recipes-devtools/qemu/qemu-native_8.1.0.bb 
b/meta/recipes-devtools/qemu/qemu-native_8.1.2.bb
similarity index 100%
rename from meta/recipes-devtools/qemu/qemu-native_8.1.0.bb
rename to meta/recipes-devtools/qemu/qemu-native_8.1.2.bb
diff --git a/meta/recipes-devtools/qemu/qemu-system-native_8.1.0.bb 
b/meta/recipes-devtools/qemu/qemu-system-native_8.1.2.bb
similarity index 100%
rename from meta/recipes-devtools/qemu/qemu-system-native_8.1.0.bb
rename to meta/recipes-devtools/qemu/qemu-system-native_8.1.2.bb
diff --git a/meta/recipes-devtools/qemu/qemu.inc 
b/meta/recipes-devtools/qemu/qemu.inc
index 78c495516f2..5ab2cb83b4d 100644
--- a/meta/recipes-devtools/qemu/qemu.inc
+++ b/meta/recipes-devtools/qemu/qemu.inc
@@ -29,18 +29,15 @@ SRC_URI = "https://download.qemu.org/${BPN}-${PV}.tar.xz \

file://0009-Define-MAP_SYNC-and-MAP_SHARED_VALIDATE-on-needed-li.patch \

file://0010-hw-pvrdma-Protect-against-buggy-or-malicious-guest-d.patch \

file://0002-linux-user-Replace-use-of-lfs64-related-functions-an.patch \
-   file://0001-softmmu-Assert-data-in-bounds-in-iotlb_to_section.patch 
\
-   file://0001-softmmu-Use-async_run_on_cpu-in-tcg_commit.patch \
file://fixedmeson.patch \
file://fixmips.patch \
file://qemu-guest-agent.init \
file://qemu-guest-agent.udev \
-  file://CVE-2023-42467.patch \
"
 UPSTREAM_CHECK_REGEX = "qemu-(?P\d+(\.\d+)+)\.tar"
 
 
-SRC_URI[sha256sum] = 
"710c101198e334d4762eef65f649bc43fa8a5dd75303554b8acfec3eb25f0e55"
+SRC_URI[sha256sum] = 
"541526a764576eb494d2ff5ec46aeb253e62ea29035d1c23c0a8af4e6cd4f087"
 
 SRC_URI:append:class-target = " file://cross.patch"
 SRC_URI:append:class-nativesdk = " file://cross.patch"
diff --git 
a/meta/recipes-devtools/qemu/qemu/0001-softmmu-Assert-data-in-bounds-in-iotlb_to_section.patch
 
b/meta/recipes-devtools/qemu/qemu/0001-softmmu-Assert-data-in-bounds-in-iotlb_to_section.patch
deleted file mode 100644
index 7380e16ab32..000
--- 
a/meta/recipes-devtools/qemu/qemu/0001-softmmu-Assert-data-in-bounds-in-iotlb_to_section.patch
+++ /dev/null
@@ -1,42 +0,0 @@
-From 86e4f93d827d3c1efd00cd8a906e38a2c0f2b5bc Mon Sep 17 00:00:00 2001
-From: Richard Henderson 
-Date: Fri, 25 Aug 2023 14:06:58 -0700
-Subject: [PATCH] softmmu: Assert data in bounds in iotlb_to_section
-MIME-Version: 1.0
-Content-Type: text/plain; charset=UTF-8
-Content-Transfer-Encoding: 8bit
-
-Acked-by: Alex Bennée 
-Suggested-by: Alex Bennée 
-Signed-off-by: Richard Henderson 
-
-Upstream-Status: Backport 
[https://gitlab.com/qemu-project/qemu/-/commit/86e4f93d827d3c1efd00cd8a906e38a2c0f2b5bc]

- softmmu/physmem.c | 10 --
- 1 file changed, 8 insertions(+), 2 deletions(-)
-
-diff --git a/softmmu/physmem.c b/softmmu/physmem.c
-index 3df73542e1..7597dc1c39 100644
 a/softmmu/physmem.c
-+++ b/softmmu/physmem.c
-@@ -2413,9 +2413,15 @@ MemoryRegionSection *iotlb_to_section(CPUState *cpu,
- int asidx = cpu_asidx_from_attrs(cpu, attrs);
- CPUAddressSpace *cpuas = >cpu_ases[asidx];
- AddressSpaceDispatch *d = qatomic_rcu_read(>memory_dispatch);
--MemoryRegionSection *sections = d->map.sections;
-+int section_index = index & ~TARGET_PAGE_MASK;
-+MemoryRegionSection *ret;
-+
-+assert(section_index < d->map.sections_nb);
-+ret = d->map.sections + section_index;
-+assert(ret->mr);
-+assert(ret->mr->ops);
- 
--return [index & ~TARGET_PAGE_MASK];
-+return ret;
- }
- 
- static void io_mem_init(void)
--- 
-2.34.1
-
diff --git 
a/meta/recipes-devtools/qemu/qemu/0001-softmmu-Use-async_run_on_cpu-in-tcg_commit.patch
 
b/meta/recipes-devtools/qemu/qemu/0001-softmmu-Use-async_run_on_cpu-in-tcg_commit.patch
deleted file mode 100644
index 8289b459916..000
--- 

[OE-core] [PATCH] scripts/contrib/patchreview: fix commit identification

2023-10-30 Thread Ross Burton
From: Ross Burton 

git show-ref looks at the _remote_ ref called HEAD, which is fine when it
matches the local HEAD but problematic when you're iterating a series of
commits.

Use rev-parse to resolve the local name to a proper hash.

Signed-off-by: Ross Burton 
---
 scripts/contrib/patchreview.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/contrib/patchreview.py b/scripts/contrib/patchreview.py
index f95cadab0c6..bceae06561c 100755
--- a/scripts/contrib/patchreview.py
+++ b/scripts/contrib/patchreview.py
@@ -257,7 +257,7 @@ if __name__ == "__main__":
 row = collections.Counter()
 row["total"] = len(results)
 row["date"] = subprocess.check_output(["git", "-C", args.directory, 
"show", "-s", "--pretty=format:%cd", "--date=format:%s"], 
universal_newlines=True).strip()
-row["commit"] = subprocess.check_output(["git", "-C", args.directory, 
"show-ref", "--hash", "HEAD"], universal_newlines=True).strip()
+row["commit"] = subprocess.check_output(["git", "-C", args.directory, 
"rev-parse", "HEAD"], universal_newlines=True).strip()
 row['commit_count'] = subprocess.check_output(["git", "-C", 
args.directory, "rev-list", "--count", "HEAD"], universal_newlines=True).strip()
 row['recipe_count'] = count_recipes(layers)
 
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189792): 
https://lists.openembedded.org/g/openembedded-core/message/189792
Mute This Topic: https://lists.openembedded.org/mt/102273104/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core][PATCH] kernel.bbclass: add preceding space in appendVar setting

2023-10-30 Thread Chen Qi via lists.openembedded.org
From: Chen Qi 

The appendVar setting should have a preceding space, otherwise, when
KERNEL_MODULE_SPLIT is set to "0", we'll sometimes get dependency error
due to lacking of space.

Signed-off-by: Chen Qi 
---
 meta/classes-recipe/kernel.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-recipe/kernel.bbclass 
b/meta/classes-recipe/kernel.bbclass
index 2ec9ea2091..5eee87fee9 100644
--- a/meta/classes-recipe/kernel.bbclass
+++ b/meta/classes-recipe/kernel.bbclass
@@ -111,7 +111,7 @@ python __anonymous () {
 d.appendVar('RDEPENDS:%s-image' % kname, ' %s-modules (= 
${EXTENDPKGV})' % kname)
 d.appendVar('RDEPENDS:%s-image-%s' % (kname, typelower), ' 
%s-modules-${KERNEL_VERSION_PKG_NAME} (= ${EXTENDPKGV})' % kname)
 d.setVar('PKG:%s-modules' % kname, 
'%s-modules-${KERNEL_VERSION_PKG_NAME}' % kname)
-d.appendVar('RPROVIDES:%s-modules' % kname, 
'%s-modules-${KERNEL_VERSION_PKG_NAME}' % kname)
+d.appendVar('RPROVIDES:%s-modules' % kname, ' 
%s-modules-${KERNEL_VERSION_PKG_NAME}' % kname)
 
 d.setVar('PKG:%s-image-%s' % (kname,typelower), 
'%s-image-%s-${KERNEL_VERSION_PKG_NAME}' % (kname, typelower))
 d.setVar('ALLOW_EMPTY:%s-image-%s' % (kname, typelower), '1')
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189791): 
https://lists.openembedded.org/g/openembedded-core/message/189791
Mute This Topic: https://lists.openembedded.org/mt/102270002/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [OE-core] [qa-build-notification] QA notification for completed autobuilder build (yocto-3.1.29.rc1)

2023-10-30 Thread Jing Hui Tham
Hi all,
 
Intel and WR YP QA is planning for QA execution for YP build yocto-3.1.29.rc1. 
We are planning to execute following tests for this cycle:
 
OEQA-manual tests for following module:
1. OE-Core
2. BSP-hw
 
Runtime auto test for following platforms:
1. MinnowBoard Turbot - 32bit
2. Kaby Lake (7th Generation Intel(r) Core(tm) Processors)
3. Tiger Lake (11th Generation Intel(r) Core(tm) Processors)
4. Alder Lake-S (12th Generation Intel(r) Core(tm) Processors)
5. Raptor Lake-P (13th Generation Intel(r) Core(tm) Processors)
6. Edgerouter
7. Beaglebone

 
ETA for completion Thursday, 2 November 2023.
 
Best regards,
Jing Hui


> -Original Message-
> From: qa-build-notificat...@lists.yoctoproject.org  notificat...@lists.yoctoproject.org> On Behalf Of Pokybuild User
> Sent: Saturday, October 28, 2023 6:06 AM
> To: yo...@lists.yoctoproject.org
> Cc: qa-build-notificat...@lists.yoctoproject.org
> Subject: [qa-build-notification] QA notification for completed autobuilder
> build (yocto-3.1.29.rc1)
> 
> 
> A build flagged for QA (yocto-3.1.29.rc1) was completed on the autobuilder
> and is available at:
> 
> 
> https://autobuilder.yocto.io/pub/releases/yocto-3.1.29.rc1
> 
> 
> Build URL:
> https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6111
> 
> Build hash information:
> 
> bitbake: dd826595414c5dc1a649f45a9dd2430bf6d4699b
> meta-agl: 280f7e70af30eefd885f6e60bd29863b46bb2eab
> meta-arm: b1fe8443a7a72c65fa0fc3371f607c6671b3a882
> meta-aws: 6e3ace380b609dadf58c81c734ef2061e9636914
> meta-gplv2: 60b251c25ba87e946a0ca4cdc8d17b1cb09292ac
> meta-intel: e482213f37828216c7a7df68ff353652cc865ec1
> meta-mingw: 7bdc58e6c5d1054b1b6ad5c4e480a95e995ccbae
> meta-openembedded: 300be975359fdb3a3b2bf7c6fe15dea7acac575d
> meta-virtualization: 35c723774ee06b3c1831f00a2cbf25cbeae132e1
> oecore: 0dbf3a15321b8033ff8ed86c6aa261fdb9c3d5bb
> poky: aeac1034661725b5c83e79f76238429fb236b090
> 
> 
> 
> This is an automated message from the Yocto Project Autobuilder
> Git: git://git.yoctoproject.org/yocto-autobuilder2
> Email: richard.pur...@linuxfoundation.org
> 
> 
> 
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189790): 
https://lists.openembedded.org/g/openembedded-core/message/189790
Mute This Topic: https://lists.openembedded.org/mt/102269965/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-