[OE-core] [PATCH 1/2] tclibc-baremetal: Include qemu on the SDK

[Alejandro Enedino Hernandez Samaniego]

Since we now have an example recipe on meta-skeleton to
build baremetal applications using OpenEmbedded, a user
produced SDK should be able to run such application.

Include nativesdk-qemu on TOOLCHAIN_HOST_TASK so its
built inside the baremetal based SDK.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/conf/distro/include/tclibc-baremetal.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/distro/include/tclibc-baremetal.inc 
b/meta/conf/distro/include/tclibc-baremetal.inc
index e04f8c8..31d132e 100644
--- a/meta/conf/distro/include/tclibc-baremetal.inc
+++ b/meta/conf/distro/include/tclibc-baremetal.inc
@@ -27,7 +27,7 @@ BASEDEPENDS_remove_class-target = 
"virtual/${TARGET_PREFIX}compilerlibs"
 TARGET_OS = "elf"
 TARGET_OS_arm = "eabi"
 
-TOOLCHAIN_HOST_TASK ?= "packagegroup-cross-canadian-${MACHINE}"
+TOOLCHAIN_HOST_TASK ?= "packagegroup-cross-canadian-${MACHINE} nativesdk-qemu"
 TOOLCHAIN_HOST_TASK_ATTEMPTONLY ?= ""
 TOOLCHAIN_TARGET_TASK ?= "libgcc-dev"
 TOOLCHAIN_NEED_CONFIGSITE_CACHE_remove = "virtual/${MLPREFIX}libc zlib ncurses"
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] tclibc-newlib: Include qemu on the SDK

[Alejandro Enedino Hernandez Samaniego]

Since we now have an example recipe on meta-skeleton to
build baremetal applications using OpenEmbedded, a user
produced SDK should be able to run such application.

Include nativesdk-qemu on TOOLCHAIN_HOST_TASK so its
built inside the newlib based SDK.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/conf/distro/include/tclibc-newlib.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/distro/include/tclibc-newlib.inc 
b/meta/conf/distro/include/tclibc-newlib.inc
index 8338003..bf89b11 100644
--- a/meta/conf/distro/include/tclibc-newlib.inc
+++ b/meta/conf/distro/include/tclibc-newlib.inc
@@ -38,7 +38,7 @@ BASE_DEFAULT_DEPS_append_class-target = " ${NEWLIB_EXTENDED}"
 TARGET_OS = "elf"
 TARGET_OS_arm = "eabi"
 
-TOOLCHAIN_HOST_TASK ?= "packagegroup-cross-canadian-${MACHINE}"
+TOOLCHAIN_HOST_TASK ?= "packagegroup-cross-canadian-${MACHINE} nativesdk-qemu"
 TOOLCHAIN_TARGET_TASK ?= "${LIBC_DEPENDENCIES}"
 TOOLCHAIN_NEED_CONFIGSITE_CACHE_remove = "zlib ncurses"
 
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 1/2] baremetal-helloworld: Create recipe for baremetal examples on QEMU

[Alejandro Enedino Hernandez Samaniego]

Create HelloWorld examples that run on several of the QEMU architectures
supported by the build system.

This recipe can be used by anyone to understand how baremetal applications
can be built using OpenEmbedded and how the wiring to set them up could be.

This should also facilitate creating/extending the OE testing infrastructure
to allow baremetal applications or RTOSs to be tested in the same way that
Linux currently is.

This can easily be extended to work on other MACHINES in the future.

To run this example:

$ source oe-init-buildenv

$ bitbake-layers add-layer ../meta-skeleton

# TCLIBC="baremetal" would work as well
$ echo "TCLIBC = \"newlib\"" >> ./conf/local.conf

$ echo "MACHINE = \"qemuarm64\"" >> ./conf/local.conf

$ bitbake baremetal-helloworld

$ runqemu

runqemu - INFO - Running bitbake -e ...
runqemu - INFO - Continuing with the following parameters:
KERNEL: [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
MACHINE: [qemuarm64]
FSTYPE: [bin]
ROOTFS: [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
CONFFILE: 
[tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.qemuboot.conf]

Hello OpenEmbedded!

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 .../baremetal-examples/baremetal-helloworld_git.bb | 105 +
 1 file changed, 105 insertions(+)
 create mode 100644 
meta-skeleton/recipes-baremetal/baremetal-examples/baremetal-helloworld_git.bb

diff --git 
a/meta-skeleton/recipes-baremetal/baremetal-examples/baremetal-helloworld_git.bb
 
b/meta-skeleton/recipes-baremetal/baremetal-examples/baremetal-helloworld_git.bb
new file mode 100644
index 000..d863370
--- /dev/null
+++ 
b/meta-skeleton/recipes-baremetal/baremetal-examples/baremetal-helloworld_git.bb
@@ -0,0 +1,105 @@
+SUMMARY = "Baremetal examples to work with the several QEMU architectures 
supported on OpenEmbedded"
+HOMEPAGE = "https://github.com/aehs29/baremetal-helloqemu;
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=39346640a23c701e4f459e05f56f4449"
+
+SRCREV = "99f4fa4a3b266b42b52af302610b0f4f429ba5e3"
+PV = "0.1+git${SRCPV}"
+
+SRC_URI = 
"git://github.com/aehs29/baremetal-helloqemu.git;protocol=https;branch=master"
+
+S = "${WORKDIR}/git/"
+
+# These examples are not meant to be built when using either musl or glibc
+COMPATIBLE_HOST_libc-musl_class-target = "null"
+COMPATIBLE_HOST_libc-glibc_class-target = "null"
+
+# This will be translated automatically to the architecture and
+# machine that QEMU uses on OE, e.g. -machine virt -cpu cortex-a57
+# but the examples can also be run on other architectures/machines
+# such as vexpress-a15 by overriding the setting on the machine.conf
+COMPATIBLE_MACHINE = "qemuarmv5|qemuarm|qemuarm64"
+
+BAREMETAL_QEMUARCH ?= ""
+BAREMETAL_QEMUARCH_qemuarmv5 = "versatile"
+BAREMETAL_QEMUARCH_qemuarm = "arm"
+BAREMETAL_QEMUARCH_qemuarm64 = "aarch64"
+
+
+EXTRA_OEMAKE_append = " QEMUARCH=${BAREMETAL_QEMUARCH} V=1"
+
+do_install(){
+install -d ${D}/${datadir}
+install -m 755 ${B}build/hello_baremetal_${BAREMETAL_QEMUARCH}.bin 
${D}/${datadir}/hello_baremetal_${MACHINE}.bin
+install -m 755 ${B}build/hello_baremetal_${BAREMETAL_QEMUARCH}.elf 
${D}/${datadir}/hello_baremetal_${MACHINE}.elf
+}
+
+# Borrowed from meta-freertos
+inherit rootfs-postcommands
+inherit deploy
+do_deploy[dirs] = "${DEPLOYDIR} ${DEPLOY_DIR_IMAGE}"
+do_rootfs[dirs] = "${DEPLOYDIR} ${DEPLOY_DIR_IMAGE}"
+DEPLOYDIR = "${IMGDEPLOYDIR}"
+IMAGE_LINK_NAME ?= "baremetal-helloworld-image-${MACHINE}"
+IMAGE_NAME_SUFFIX ?= ""
+
+do_deploy(){
+install ${D}/${datadir}/hello_baremetal_${MACHINE}.bin 
${DEPLOYDIR}/${IMAGE_LINK_NAME}.bin
+install ${D}/${datadir}/hello_baremetal_${MACHINE}.elf 
${DEPLOYDIR}/${IMAGE_LINK_NAME}.elf
+}
+
+do_image(){
+:
+}
+
+FILES_${PN} += " \
+${datadir}/hello_baremetal_${MACHINE}.bin \
+${datadir}/hello_baremetal_${MACHINE}.elf \
+"
+
+python do_rootfs(){
+from oe.utils import execute_pre_post_process
+from pathlib import Path
+
+# Write empty manifest testdate file
+deploy_dir = d.getVar('DEPLOYDIR')
+link_name = d.getVar('IMAGE_LINK_NAME')
+manifest_name = d.getVar('IMAGE_MANIFEST')
+
+Path(manifest_name).touch()
+if os.path.exists(manifest_name) and link_name:
+manifest_link = deploy_dir + "/" + link_name + ".manifest"
+if os.path.lexists(manifest_link):
+os.remove(manifest_link)
+os.symlink(os.path.basename(manifest_name), manifest_link)
+execute_pre_post_process(d, d.getVar('ROOTFS_POSTPROCESS_COMMAND'))
+}
+
+# QEMU generic FreeRTOS parameters
+QB_DEFAULT_KERNEL = "${IMAGE_LINK_NAME}.bin"
+QB_MEM = "-m 256"
+QB_OPT_

Re: [OE-core] [PATCH] MEMORIAM: Add recognition for contributors no longer with us

[Alejandro Enedino Hernandez Samaniego]

Acked-by: Alejandro Hernandez Samaniego 

On Thu, 30 Jan 2020 at 15:02, akuster808  wrote:

> +1
>
> On 1/29/20 4:01 PM, Richard Purdie wrote:
> > Its sad to have this but as the project ages, also perhaps inevitable.
> > Besides their contributions, remember them here.
> >
> > Signed-off-by: Richard Purdie 
> > ---
> >  MEMORIAM | 5 +
> >  1 file changed, 5 insertions(+)
> >  create mode 100644 MEMORIAM
> >
> > diff --git a/MEMORIAM b/MEMORIAM
> > new file mode 100644
> > index 000..0b3ce4670fc
> > --- /dev/null
> > +++ b/MEMORIAM
> > @@ -0,0 +1,5 @@
> > +Some project contributors who are sadly no longer with us:
> > +
> > +Greg Gilbert (treke) - Ahead of his time with licensing
> > +Thomas Wood (thos) - Creator of the original sato
> > +Scott Rifenbark (scottrif) - Our long standing techwriter whose words
> live on
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] baremetal-helloworld: Create recipe for baremetal examples on QEMU

[Alejandro Enedino Hernandez Samaniego]

Hey Khem,

On Mon, Jan 27, 2020, 12:51 PM Khem Raj  wrote:

> On Mon, Jan 27, 2020 at 2:01 AM Alejandro Enedino Hernandez Samaniego
>  wrote:
> >
> > Create HelloWorld examples that run on several of the QEMU architectures
> > supported by the build system.
> >
> > This recipe can be used by anyone to understand how baremetal
> applications
> > can be built using OpenEmbedded and how the wiring to set them up could
> be.
> >
> > This should also facilitate creating/extending the OE testing
> infrastructure
> > to allow baremetal applications or RTOSs to be tested in the same way
> that
> > Linux currently is.
> >
> > This can easily be extended to work on other MACHINES in the future.
> >
> > To run this example:
> >
> > # Source the oe environment as usual
> > $ source oe-init-buildenv
> >
> > # Set TCLIBC to either newlib or baremetal
> > $ echo "TCLIBC = \"newlib\"" >> ./conf/local.conf
> >
> > # Use one of the supported architectures (qemuarm64, qemuarm or
> qemuarmv5)
> > $ echo "MACHINE = \"qemuarm64\"" >> ./conf/local.conf
> >
> > # Build
> > $ bitbake baremetal-helloworld
> >
> > # Launch QEMU
> > $ runqemu
> >
> > runqemu - INFO - Running bitbake -e ...
> > runqemu - INFO - Continuing with the following parameters:
> > KERNEL: [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
> > MACHINE: [qemuarm64]
> > FSTYPE: [bin]
> > ROOTFS: [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
> > CONFFILE:
> [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.qemuboot.conf]
> >
> > Hello OpenEmbedded!
> >
> > Signed-off-by: Alejandro Enedino Hernandez Samaniego <
> alejan...@enedino.org>
> > ---
> >  .../baremetal-examples/baremetal-helloworld_git.bb | 114
> +
> >  1 file changed, 114 insertions(+)
> >  create mode 100644 meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb
> >
> > diff --git a/meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb b/meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb
> > new file mode 100644
> > index 000..aab48b1
> > --- /dev/null
> > +++ b/meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb
> > @@ -0,0 +1,114 @@
> > +SUMMARY = "Baremetal examples to work with the several QEMU
> architectures supported on OpenEmbedded"
> > +HOMEPAGE = "https://github.com/aehs29/baremetal-helloqemu;
> > +LICENSE = "CC-BY-SA-4.0"
>
> I think MIT or Apache-2.0 would be much nicer.
>

I agree but this wasn't by choice, the original code written for the
versatile architecture was licensed on CC-BY-SA, which AFAIC wouldn't be
compatible with those, this is the best I could come up with since this
way, something that uses this could use GPLv3 if there's a better way let
me know


> > +LIC_FILES_CHKSUM = "file://LICENSE;md5=4084714af41157e38872e798eb3fe1b1"
> > +
> > +SRCREV = "fe81742f1159725408dd7305e40e7bd4dfd1e264"
> > +
> > +SRC_URI = "git://
> github.com/aehs29/baremetal-helloqemu.git;protocol=https "
> > +
> > +S = "${WORKDIR}/git"
> > +B = "${WORKDIR}/build"
> > +
> > +# These examples are not meant to be built when using either musl or
> glibc
> > +COMPATIBLE_HOST_libc-musl_class-target = "null"
> > +COMPATIBLE_HOST_libc-glibc_class-target = "null"
>
> Perhaps check for TARGET_OS instead of LIBC
>

We had decided this to avoid bitbake world errors, if you think TARGET_OS
is better I'm okay with that.



> > +# This will be translated automatically to the architecture and
> > +# machine that QEMU uses on OE, e.g. -machine virt -cpu cortex-a57
> > +# but the examples can also be run on other architectures/machines
> > +# such as vexpress-a15 by overriding the setting on the machine.conf
> > +COMPATIBLE_MACHINE = "qemuarmv5|qemuarm|qemuarm64"
> > +
> > +BAREMETAL_QEMUARCH ?= ""
> > +BAREMETAL_QEMUARCH_qemuarmv5 = "versatile"
> > +BAREMETAL_QEMUARCH_qemuarm = "arm"
> > +BAREMETAL_QEMUARCH_qemuarm64 = "aarch64"
> > +
>
> this setting should be abstracted into something like  baremetal.bbclass
>
>
> > +
> > +EXTRA_OEMAKE_append = " QEMUARCH=${BAREMETAL_QEMUARCH}"
> > +
> > +do_configure(){
> > +# Avoid building on the source directory
> > +cp ${S}/* ${B}
> >

Re: [OE-core] [PATCH 1/2] baremetal-helloworld: Create recipe for baremetal examples on QEMU

[Alejandro Enedino Hernandez Samaniego]

Hey guys,

On Mon, Jan 27, 2020, 1:11 PM Paul Barker  wrote:

> On Mon, 27 Jan 2020 at 15:42, Alejandro Enedino Hernandez Samaniego
>  wrote:
> >
> > Hey Paul,
> >
> >
> >
> > On Mon, Jan 27, 2020, 2:09 AM Paul Barker  wrote:
> >>
> >> On Mon, 27 Jan 2020 at 10:00, Alejandro Enedino Hernandez Samaniego
> >>  wrote:
> >> >
> >> > Create HelloWorld examples that run on several of the QEMU
> architectures
> >> > supported by the build system.
> >> >
> >> > This recipe can be used by anyone to understand how baremetal
> applications
> >> > can be built using OpenEmbedded and how the wiring to set them up
> could be.
> >> >
> >> > This should also facilitate creating/extending the OE testing
> infrastructure
> >> > to allow baremetal applications or RTOSs to be tested in the same way
> that
> >> > Linux currently is.
> >> >
> >> > This can easily be extended to work on other MACHINES in the future.
> >> >
> >> > To run this example:
> >> >
> >> > # Source the oe environment as usual
> >> > $ source oe-init-buildenv
> >> >
> >> > # Set TCLIBC to either newlib or baremetal
> >> > $ echo "TCLIBC = \"newlib\"" >> ./conf/local.conf
> >> >
> >> > # Use one of the supported architectures (qemuarm64, qemuarm or
> qemuarmv5)
> >> > $ echo "MACHINE = \"qemuarm64\"" >> ./conf/local.conf
> >> >
> >> > # Build
> >> > $ bitbake baremetal-helloworld
> >> >
> >> > # Launch QEMU
> >> > $ runqemu
> >> >
> >> > runqemu - INFO - Running bitbake -e ...
> >> > runqemu - INFO - Continuing with the following parameters:
> >> > KERNEL:
> [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
> >> > MACHINE: [qemuarm64]
> >> > FSTYPE: [bin]
> >> > ROOTFS:
> [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
> >> > CONFFILE:
> [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.qemuboot.conf]
> >> >
> >> > Hello OpenEmbedded!
> >> >
> >> > Signed-off-by: Alejandro Enedino Hernandez Samaniego <
> alejan...@enedino.org>
> >> > ---
> >> >  .../baremetal-examples/baremetal-helloworld_git.bb | 114
> +
> >> >  1 file changed, 114 insertions(+)
> >> >  create mode 100644 meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb
> >> >
> >> > diff --git a/meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb b/meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb
> >> > new file mode 100644
> >> > index 000..aab48b1
> >> > --- /dev/null
> >> > +++ b/meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb
> >> > @@ -0,0 +1,114 @@
> >> > +SUMMARY = "Baremetal examples to work with the several QEMU
> architectures supported on OpenEmbedded"
> >> > +HOMEPAGE = "https://github.com/aehs29/baremetal-helloqemu;
> >> > +LICENSE = "CC-BY-SA-4.0"
> >>
> >> I've raised https://github.com/aehs29/baremetal-helloqemu/issues/1 as
> >> CC licenses aren't recommended for software.
> >>
> >>
> >
> > I do agree, I originally had it as MIT, although I did base this on the
> original work of someone who made it work for the versatile architecture
> and they licensed that with CC-BY-SA and as I was reading I have to use the
> same license as them.
> >
> > I thought about it and for the Hello OpenEmbedded examples it would be
> ok since they should serve mostly as basis for the wiring that users can
> use to build, run and test their baremetal apps on OE, so the license
> wouldn't affect them, but I'm open to suggestions here.
>
> A bad license choice in a sample recipe is just going to encourage
> others to use it though. I'd recommend reviewing
> https://wiki.creativecommons.org/wiki/ShareAlike_compatibility:_GPLv3
> to see if you can use GPLv3 instead if you've got CC-BY-SA inputs. You
> could also contact the original author and ask about licensing the
> code more cleanly.
>

Agreed, I took a look at that already and it looks like the compatibility
starts with CC-BY-SA 4 which didn't exist when the original code was
written, but that's why I chose v4 because it was compatible with the older
ones and people that use this can use GPLv3 if they want to since this is
CC-BY-SA v4, again this wasn't my first choice either.

I tried contacting the author already but it's been a while and he hasn't
replied yet.

Alejandro

>
> Thanks,
> Paul
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] baremetal-helloworld: Create recipe for baremetal examples on QEMU

[Alejandro Enedino Hernandez Samaniego]

On Mon, Jan 27, 2020, 2:13 AM Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

> On Mon, 2020-01-27 at 02:00 -0800, Alejandro Enedino Hernandez Samaniego
> wrote:
> > Create HelloWorld examples that run on several of the QEMU architectures
> > supported by the build system.
> >
> > This recipe can be used by anyone to understand how baremetal
> applications
> > can be built using OpenEmbedded and how the wiring to set them up could
> be.
> >
> > This should also facilitate creating/extending the OE testing
> infrastructure
> > to allow baremetal applications or RTOSs to be tested in the same way
> that
> > Linux currently is.
> >
> > This can easily be extended to work on other MACHINES in the future.
> >
> > To run this example:
> >
> > # Source the oe environment as usual
> > $ source oe-init-buildenv
> >
> > # Set TCLIBC to either newlib or baremetal
> > $ echo "TCLIBC = \"newlib\"" >> ./conf/local.conf
> >
> > # Use one of the supported architectures (qemuarm64, qemuarm or
> qemuarmv5)
> > $ echo "MACHINE = \"qemuarm64\"" >> ./conf/local.conf
> >
> > # Build
> > $ bitbake baremetal-helloworld
> >
> > # Launch QEMU
> > $ runqemu
> >
> > runqemu - INFO - Running bitbake -e ...
> > runqemu - INFO - Continuing with the following parameters:
> > KERNEL: [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
> > MACHINE: [qemuarm64]
> > FSTYPE: [bin]
> > ROOTFS: [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
> > CONFFILE:
> [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.qemuboot.conf]
> >
> > Hello OpenEmbedded!
>
> Looks good.
>
> Should this be in meta-skeleton? That sublayer generally has our
> configuration example tests in as far as I can tell...
>
> Do you plan to add an oe-selftest for this?
>
> Cheers,
>
> Richard
>

Hey Richard,

I do, that's exactly the plan, I'm working on both oe-selftest and runtime
testing cases for this, I'm just doing it in chunks.

Cheers,

Alejandro


>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] baremetal-helloworld: Create recipe for baremetal examples on QEMU

[Alejandro Enedino Hernandez Samaniego]

Hey Paul,



On Mon, Jan 27, 2020, 2:09 AM Paul Barker  wrote:

> On Mon, 27 Jan 2020 at 10:00, Alejandro Enedino Hernandez Samaniego
>  wrote:
> >
> > Create HelloWorld examples that run on several of the QEMU architectures
> > supported by the build system.
> >
> > This recipe can be used by anyone to understand how baremetal
> applications
> > can be built using OpenEmbedded and how the wiring to set them up could
> be.
> >
> > This should also facilitate creating/extending the OE testing
> infrastructure
> > to allow baremetal applications or RTOSs to be tested in the same way
> that
> > Linux currently is.
> >
> > This can easily be extended to work on other MACHINES in the future.
> >
> > To run this example:
> >
> > # Source the oe environment as usual
> > $ source oe-init-buildenv
> >
> > # Set TCLIBC to either newlib or baremetal
> > $ echo "TCLIBC = \"newlib\"" >> ./conf/local.conf
> >
> > # Use one of the supported architectures (qemuarm64, qemuarm or
> qemuarmv5)
> > $ echo "MACHINE = \"qemuarm64\"" >> ./conf/local.conf
> >
> > # Build
> > $ bitbake baremetal-helloworld
> >
> > # Launch QEMU
> > $ runqemu
> >
> > runqemu - INFO - Running bitbake -e ...
> > runqemu - INFO - Continuing with the following parameters:
> > KERNEL: [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
> > MACHINE: [qemuarm64]
> > FSTYPE: [bin]
> > ROOTFS: [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
> > CONFFILE:
> [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.qemuboot.conf]
> >
> > Hello OpenEmbedded!
> >
> > Signed-off-by: Alejandro Enedino Hernandez Samaniego <
> alejan...@enedino.org>
> > ---
> >  .../baremetal-examples/baremetal-helloworld_git.bb | 114
> +
> >  1 file changed, 114 insertions(+)
> >  create mode 100644 meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb
> >
> > diff --git a/meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb b/meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb
> > new file mode 100644
> > index 000..aab48b1
> > --- /dev/null
> > +++ b/meta/recipes-extended/baremetal-examples/
> baremetal-helloworld_git.bb
> > @@ -0,0 +1,114 @@
> > +SUMMARY = "Baremetal examples to work with the several QEMU
> architectures supported on OpenEmbedded"
> > +HOMEPAGE = "https://github.com/aehs29/baremetal-helloqemu;
> > +LICENSE = "CC-BY-SA-4.0"
>
> I've raised https://github.com/aehs29/baremetal-helloqemu/issues/1 as
> CC licenses aren't recommended for software.
>
>
>
I do agree, I originally had it as MIT, although I did base this on the
original work of someone who made it work for the versatile architecture
and they licensed that with CC-BY-SA and as I was reading I have to use the
same license as them.

I thought about it and for the Hello OpenEmbedded examples it would be ok
since they should serve mostly as basis for the wiring that users can use
to build, run and test their baremetal apps on OE, so the license wouldn't
affect them, but I'm open to suggestions here.


> > +LIC_FILES_CHKSUM = "file://LICENSE;md5=4084714af41157e38872e798eb3fe1b1"
> > +
> > +SRCREV = "fe81742f1159725408dd7305e40e7bd4dfd1e264"
> > +
> > +SRC_URI = "git://
> github.com/aehs29/baremetal-helloqemu.git;protocol=https "
> > +
> > +S = "${WORKDIR}/git"
> > +B = "${WORKDIR}/build"
> > +
> > +# These examples are not meant to be built when using either musl or
> glibc
> > +COMPATIBLE_HOST_libc-musl_class-target = "null"
> > +COMPATIBLE_HOST_libc-glibc_class-target = "null"
> > +
> > +# This will be translated automatically to the architecture and
> > +# machine that QEMU uses on OE, e.g. -machine virt -cpu cortex-a57
> > +# but the examples can also be run on other architectures/machines
> > +# such as vexpress-a15 by overriding the setting on the machine.conf
> > +COMPATIBLE_MACHINE = "qemuarmv5|qemuarm|qemuarm64"
> > +
> > +BAREMETAL_QEMUARCH ?= ""
> > +BAREMETAL_QEMUARCH_qemuarmv5 = "versatile"
> > +BAREMETAL_QEMUARCH_qemuarm = "arm"
> > +BAREMETAL_QEMUARCH_qemuarm64 = "aarch64"
> > +
> > +
> > +EXTRA_OEMAKE_append = " QEMUARCH=${BAREMETAL_QEMUARCH}"
> > +
> > +do_configure(){
> > +# Avoid building on the source di

[OE-core] [PATCH 2/2] maintainers.inc: Add maintainer for baremetal-helloworld recipe

[Alejandro Enedino Hernandez Samaniego]

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/conf/distro/include/maintainers.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/conf/distro/include/maintainers.inc 
b/meta/conf/distro/include/maintainers.inc
index 01afc25..f2126bf 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -59,6 +59,7 @@ RECIPE_MAINTAINER_pn-automake = "Robert Yang 
"
 RECIPE_MAINTAINER_pn-avahi = "Yi Zhao "
 RECIPE_MAINTAINER_pn-avahi-ui = "Yi Zhao "
 RECIPE_MAINTAINER_pn-babeltrace = "Alexander Kanavin "
+RECIPE_MAINTAINER_pn-baremetal-helloworld = "Alejandro Enedino Hernandez 
Samaniego "
 RECIPE_MAINTAINER_pn-base-files = "Anuj Mittal "
 RECIPE_MAINTAINER_pn-base-passwd = "Anuj Mittal "
 RECIPE_MAINTAINER_pn-bash = "Hongxu Jia "
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] baremetal-helloworld: Create recipe for baremetal examples on QEMU

[Alejandro Enedino Hernandez Samaniego]

Create HelloWorld examples that run on several of the QEMU architectures
supported by the build system.

This recipe can be used by anyone to understand how baremetal applications
can be built using OpenEmbedded and how the wiring to set them up could be.

This should also facilitate creating/extending the OE testing infrastructure
to allow baremetal applications or RTOSs to be tested in the same way that
Linux currently is.

This can easily be extended to work on other MACHINES in the future.

To run this example:

# Source the oe environment as usual
$ source oe-init-buildenv

# Set TCLIBC to either newlib or baremetal
$ echo "TCLIBC = \"newlib\"" >> ./conf/local.conf

# Use one of the supported architectures (qemuarm64, qemuarm or qemuarmv5)
$ echo "MACHINE = \"qemuarm64\"" >> ./conf/local.conf

# Build
$ bitbake baremetal-helloworld

# Launch QEMU
$ runqemu

runqemu - INFO - Running bitbake -e ...
runqemu - INFO - Continuing with the following parameters:
KERNEL: [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
MACHINE: [qemuarm64]
FSTYPE: [bin]
ROOTFS: [tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.bin]
CONFFILE: 
[tmp/deploy/images/qemuarm64/baremetal-helloworld-qemuarm64.qemuboot.conf]

Hello OpenEmbedded!

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 .../baremetal-examples/baremetal-helloworld_git.bb | 114 +
 1 file changed, 114 insertions(+)
 create mode 100644 
meta/recipes-extended/baremetal-examples/baremetal-helloworld_git.bb

diff --git 
a/meta/recipes-extended/baremetal-examples/baremetal-helloworld_git.bb 
b/meta/recipes-extended/baremetal-examples/baremetal-helloworld_git.bb
new file mode 100644
index 000..aab48b1
--- /dev/null
+++ b/meta/recipes-extended/baremetal-examples/baremetal-helloworld_git.bb
@@ -0,0 +1,114 @@
+SUMMARY = "Baremetal examples to work with the several QEMU architectures 
supported on OpenEmbedded"
+HOMEPAGE = "https://github.com/aehs29/baremetal-helloqemu;
+LICENSE = "CC-BY-SA-4.0"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=4084714af41157e38872e798eb3fe1b1"
+
+SRCREV = "fe81742f1159725408dd7305e40e7bd4dfd1e264"
+
+SRC_URI = "git://github.com/aehs29/baremetal-helloqemu.git;protocol=https "
+
+S = "${WORKDIR}/git"
+B = "${WORKDIR}/build"
+
+# These examples are not meant to be built when using either musl or glibc
+COMPATIBLE_HOST_libc-musl_class-target = "null"
+COMPATIBLE_HOST_libc-glibc_class-target = "null"
+
+# This will be translated automatically to the architecture and
+# machine that QEMU uses on OE, e.g. -machine virt -cpu cortex-a57
+# but the examples can also be run on other architectures/machines
+# such as vexpress-a15 by overriding the setting on the machine.conf
+COMPATIBLE_MACHINE = "qemuarmv5|qemuarm|qemuarm64"
+
+BAREMETAL_QEMUARCH ?= ""
+BAREMETAL_QEMUARCH_qemuarmv5 = "versatile"
+BAREMETAL_QEMUARCH_qemuarm = "arm"
+BAREMETAL_QEMUARCH_qemuarm64 = "aarch64"
+
+
+EXTRA_OEMAKE_append = " QEMUARCH=${BAREMETAL_QEMUARCH}"
+
+do_configure(){
+# Avoid building on the source directory
+cp ${S}/* ${B}
+}
+
+do_install(){
+install -d ${D}/${datadir}
+install -m 755 ${B}/hello_baremetal_${BAREMETAL_QEMUARCH}.bin 
${D}/${datadir}/hello_baremetal_${MACHINE}.bin
+install -m 755 ${B}/hello_baremetal_${BAREMETAL_QEMUARCH}.elf 
${D}/${datadir}/hello_baremetal_${MACHINE}.elf
+}
+
+# Make sure there are no race conditions when
+# assembling compiling and linking steps
+PARALLEL_MAKE = ""
+
+# Borrowed from meta-freertos
+inherit rootfs-postcommands
+inherit deploy
+do_deploy[dirs] = "${DEPLOYDIR} ${DEPLOY_DIR_IMAGE}"
+do_rootfs[dirs] = "${DEPLOYDIR} ${DEPLOY_DIR_IMAGE}"
+DEPLOYDIR = "${IMGDEPLOYDIR}"
+IMAGE_LINK_NAME ?= "baremetal-helloworld-image-${MACHINE}"
+IMAGE_NAME_SUFFIX ?= ""
+
+do_deploy(){
+install ${D}/${datadir}/hello_baremetal_${MACHINE}.bin 
${DEPLOYDIR}/${IMAGE_LINK_NAME}.bin
+install ${D}/${datadir}/hello_baremetal_${MACHINE}.elf 
${DEPLOYDIR}/${IMAGE_LINK_NAME}.elf
+}
+
+do_image(){
+:
+}
+
+FILES_${PN} += " \
+${datadir}/hello_baremetal_${MACHINE}.bin \
+${datadir}/hello_baremetal_${MACHINE}.elf \
+"
+
+python do_rootfs(){
+from oe.utils import execute_pre_post_process
+from pathlib import Path
+
+# Write empty manifest testdate file
+deploy_dir = d.getVar('DEPLOYDIR')
+link_name = d.getVar('IMAGE_LINK_NAME')
+manifest_name = d.getVar('IMAGE_MANIFEST')
+
+Path(manifest_name).touch()
+if os.path.exists(manifest_name) and link_name:
+manifest_link = deploy_dir + "/" + link_name + ".manifest"
+if os.path.lexists(manifest_link):
+os.remove(manifest_link)
+os.symlink(os.path.basename(ma

Re: [OE-core] [PATCH] gcc-configure: Enable the use of different symbol versioning

[Alejandro Enedino Hernandez Samaniego]

Hey Khem,

On Sun, Jan 19, 2020, 4:22 PM Khem Raj  wrote:

> On Sat, Jan 18, 2020 at 7:02 PM Alejandro Enedino Hernandez Samaniego
>  wrote:
> >
> > While the gnu style for symbol versioning is the most usual,
> > --enable-symvers[=style] can be provided several values,
> > gnu, gnu-versioned-namespace, darwin, darwin-export, and sun,
> > depending on users needs.
> >
> > Introduce the SYMVERS_CONF variable to allow the user to
> > configure the symbol versioning in shared libraries.
> >
> > Signed-off-by: Alejandro Enedino Hernandez Samaniego <
> alejan...@enedino.org>
> > ---
> >  meta/recipes-devtools/gcc/gcc-configure-common.inc | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/recipes-devtools/gcc/gcc-configure-common.inc
> b/meta/recipes-devtools/gcc/gcc-configure-common.inc
> > index 24ba8ce..bb4f692 100644
> > --- a/meta/recipes-devtools/gcc/gcc-configure-common.inc
> > +++ b/meta/recipes-devtools/gcc/gcc-configure-common.inc
> > @@ -17,6 +17,8 @@ GCCTHREADS ?= "posix"
> >
> >  GCCPIE ??= ""
> >
> > +SYMVERS_CONF ?= "--enable-symvers=gnu"
> > +
>
> just turn the RHS into variable not full option
>

I did think about that, and I can do it, but I think I would have to also
create a PACKAGECONFIG and since AFAIC no one is changing the style I'm not
sure it's worth the complication of the code, and this still allows us to
remove the enablement entirely, not just changing the style.

Alejandro


> >  EXTRA_OECONF = "\
> >  ${@['--enable-clocale=generic', ''][d.getVar('USE_NLS') != 'no']} \
> >  --with-gnu-ld \
> > @@ -27,7 +29,7 @@ EXTRA_OECONF = "\
> >  ${GCCPIE} \
> >  --enable-c99 \
> >  --enable-long-long \
> > ---enable-symvers=gnu \
> > +${SYMVERS_CONF} \
> >  --enable-libstdcxx-pch \
> >  --program-prefix=${TARGET_PREFIX} \
> >  --without-local-prefix \
> > --
> > 2.7.4
> >
> > --
> > ___
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] newlib: Enable building libstdc++ for newlib based toolchains

[Alejandro Enedino Hernandez Samaniego]

Some baremetal applications might require support from libstdc++
On newlib based toolchains, libstdc++ can be built as a static
library that applications can then link against it.

Pass libsdtc++-(static)dev to LIBC_DEPENDENCIES allowing the
library to be present for cross compilation as well as on
sdk builds.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/conf/distro/include/tclibc-newlib.inc | 7 +++
 meta/recipes-devtools/gcc/gcc-runtime.inc  | 6 ++
 2 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/meta/conf/distro/include/tclibc-newlib.inc 
b/meta/conf/distro/include/tclibc-newlib.inc
index 896c0b1..8338003 100644
--- a/meta/conf/distro/include/tclibc-newlib.inc
+++ b/meta/conf/distro/include/tclibc-newlib.inc
@@ -25,12 +25,11 @@ LIBC_DEPENDENCIES = "\
 libgloss-dbg \
 libgcc-dev \
 libgcc-dbg \
+libstdc++-dev \
+libstdc++-staticdev \
 "
 
-# compilerlibs defaults to gcc-runtime, but we get runtime from libgloss
-# we set ASSUME_PROVIDED because we cant set PREFERRED_PROVIDER
-# for compilerlibs since its overridden by tcmode-default
-ASSUME_PROVIDED += "virtual/${TARGET_PREFIX}compilerlibs virtual/crypt"
+ASSUME_PROVIDED += "virtual/crypt"
 
 # Its useful to be able to extend newlib, but we dont provide a native variant 
of libgloss
 NEWLIB_EXTENDED ?=  "libgloss libgcc"
diff --git a/meta/recipes-devtools/gcc/gcc-runtime.inc 
b/meta/recipes-devtools/gcc/gcc-runtime.inc
index d3f8ae8..b2c5d43 100644
--- a/meta/recipes-devtools/gcc/gcc-runtime.inc
+++ b/meta/recipes-devtools/gcc/gcc-runtime.inc
@@ -16,10 +16,14 @@ EXTRA_OECONF_PATHS = "\
 
 EXTRA_OECONF_append_linuxstdbase = " --enable-clocale=gnu"
 EXTRA_OECONF_append = " --cache-file=${B}/config.cache"
+EXTRA_OECONF_append_libc-newlib = " --with-newlib"
 
 # Disable ifuncs for libatomic on arm conflicts -march/-mcpu
 EXTRA_OECONF_append_arm = " libat_cv_have_ifunc=no "
 
+# Newlib does not support symbol versioning on libsdtcc++
+SYMVERS_CONF_libc-newlib = ""
+
 # Building with thumb enabled on armv6t fails
 ARM_INSTRUCTION_SET_armv6 = "arm"
 
@@ -36,6 +40,8 @@ RUNTIMELIBSSP_mingw32 ?= "libssp"
 RUNTIMETARGET = "${RUNTIMELIBSSP} libstdc++-v3 libgomp libatomic 
${RUNTIMELIBITM} \
 ${@bb.utils.contains_any('FORTRAN', [',fortran',',f77'], 'libquadmath', 
'', d)} \
 "
+# Only build libstdc++ for newlib
+RUNTIMETARGET_libc-newlib = "libstdc++-v3"
 
 # libiberty
 # libmudflap
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] gcc-configure: Enable the use of different symbol versioning

[Alejandro Enedino Hernandez Samaniego]

While the gnu style for symbol versioning is the most usual,
--enable-symvers[=style] can be provided several values,
gnu, gnu-versioned-namespace, darwin, darwin-export, and sun,
depending on users needs.

Introduce the SYMVERS_CONF variable to allow the user to
configure the symbol versioning in shared libraries.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/recipes-devtools/gcc/gcc-configure-common.inc | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/gcc/gcc-configure-common.inc 
b/meta/recipes-devtools/gcc/gcc-configure-common.inc
index 24ba8ce..bb4f692 100644
--- a/meta/recipes-devtools/gcc/gcc-configure-common.inc
+++ b/meta/recipes-devtools/gcc/gcc-configure-common.inc
@@ -17,6 +17,8 @@ GCCTHREADS ?= "posix"
 
 GCCPIE ??= ""
 
+SYMVERS_CONF ?= "--enable-symvers=gnu"
+
 EXTRA_OECONF = "\
 ${@['--enable-clocale=generic', ''][d.getVar('USE_NLS') != 'no']} \
 --with-gnu-ld \
@@ -27,7 +29,7 @@ EXTRA_OECONF = "\
 ${GCCPIE} \
 --enable-c99 \
 --enable-long-long \
---enable-symvers=gnu \
+${SYMVERS_CONF} \
 --enable-libstdcxx-pch \
 --program-prefix=${TARGET_PREFIX} \
 --without-local-prefix \
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] newlib: Enable building libstdc++ for newlib based toolchains

[Alejandro Enedino Hernandez Samaniego]

On Sat, 18 Jan 2020 at 15:25, Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

> On Sat, 2020-01-18 at 13:22 -0800, Alejandro Enedino Hernandez
> Samaniego wrote:
> > Some baremetal applications might require support from libstdc++.
> >
> > On newlib based toolchains, libstdc++ can be built as a static
> > library that applications can then link against it.
> >
> > Pass libsdtc++-(static)dev to LIBC_DEPENDENCIES allowing the
> > library to be present for cross compilation as well as on
> > sdk builds.
> >
> > This also requires to modify the gcc-runtime recipe to
> > explictly build only libsdtc++ correctly.
> >
> > Signed-off-by: Alejandro Enedino Hernandez Samaniego <
> > alejan...@enedino.org>
> > ---
> >  meta/conf/distro/include/tclibc-newlib.inc | 7 +++
> >  meta/recipes-devtools/gcc/gcc-runtime.inc  | 6 ++
> >  2 files changed, 9 insertions(+), 4 deletions(-)
> >
> > diff --git a/meta/conf/distro/include/tclibc-newlib.inc
> > b/meta/conf/distro/include/tclibc-newlib.inc
> > index 896c0b1..8338003 100644
> > --- a/meta/conf/distro/include/tclibc-newlib.inc
> > +++ b/meta/conf/distro/include/tclibc-newlib.inc
> > @@ -25,12 +25,11 @@ LIBC_DEPENDENCIES = "\
> >  libgloss-dbg \
> >  libgcc-dev \
> >  libgcc-dbg \
> > +libstdc++-dev \
> > +libstdc++-staticdev \
> >  "
> >
> > -# compilerlibs defaults to gcc-runtime, but we get runtime from
> > libgloss
> > -# we set ASSUME_PROVIDED because we cant set PREFERRED_PROVIDER
> > -# for compilerlibs since its overridden by tcmode-default
> > -ASSUME_PROVIDED += "virtual/${TARGET_PREFIX}compilerlibs
> > virtual/crypt"
> > +ASSUME_PROVIDED += "virtual/crypt"
> >
> >  # Its useful to be able to extend newlib, but we dont provide a
> > native variant of libgloss
> >  NEWLIB_EXTENDED ?=  "libgloss libgcc"
> > diff --git a/meta/recipes-devtools/gcc/gcc-runtime.inc
> > b/meta/recipes-devtools/gcc/gcc-runtime.inc
> > index d3f8ae8..f035fbd 100644
> > --- a/meta/recipes-devtools/gcc/gcc-runtime.inc
> > +++ b/meta/recipes-devtools/gcc/gcc-runtime.inc
> > @@ -17,6 +17,9 @@ EXTRA_OECONF_PATHS = "\
> >  EXTRA_OECONF_append_linuxstdbase = " --enable-clocale=gnu"
> >  EXTRA_OECONF_append = " --cache-file=${B}/config.cache"
> >
> > +EXTRA_OECONF_remove_libc-newlib = "--enable-symvers=gnu"
> > +EXTRA_OECONF_append_libc-newlib = " --with-newlib"
> > +
>
> I really don't like using remove in core. Can we restructure this so
> that symvers is set something like:
>
> SYMVERS_CONF = " --enable-clocale=gnu"
> SYMVERS_CONF_libc-newlib = ""
> EXTRA_OECONF_append += "${SYMVERS_CONF}"
>
> ?
>

Hey Richard,

Absolutely agree, I will send a v2 with these changes.

Alejandro

>
> >  # Disable ifuncs for libatomic on arm conflicts -march/-mcpu
> >  EXTRA_OECONF_append_arm = " libat_cv_have_ifunc=no "
> >
> > @@ -33,6 +36,9 @@ RUNTIMELIBITM_riscv64 = ""
> >  RUNTIMELIBSSP ?= ""
> >  RUNTIMELIBSSP_mingw32 ?= "libssp"
> >
> > +# Only build libstdc++ for newlib
> > +RUNTIMETARGET_libc-newlib = "libstdc++-v3"
> > +
> >  RUNTIMETARGET = "${RUNTIMELIBSSP} libstdc++-v3 libgomp libatomic
> > ${RUNTIMELIBITM} \
> >  ${@bb.utils.contains_any('FORTRAN', [',fortran',',f77'],
> > 'libquadmath', '', d)} \
> >  "
>
> Convention is top put the overridden version under the original.
>

Will change this as well



>
> Cheers,
>
> Richard
>
>
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] newlib: Enable building libstdc++ for newlib based toolchains

[Alejandro Enedino Hernandez Samaniego]

Some baremetal applications might require support from libstdc++.

On newlib based toolchains, libstdc++ can be built as a static
library that applications can then link against it.

Pass libsdtc++-(static)dev to LIBC_DEPENDENCIES allowing the
library to be present for cross compilation as well as on
sdk builds.

This also requires to modify the gcc-runtime recipe to
explictly build only libsdtc++ correctly.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/conf/distro/include/tclibc-newlib.inc | 7 +++
 meta/recipes-devtools/gcc/gcc-runtime.inc  | 6 ++
 2 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/meta/conf/distro/include/tclibc-newlib.inc 
b/meta/conf/distro/include/tclibc-newlib.inc
index 896c0b1..8338003 100644
--- a/meta/conf/distro/include/tclibc-newlib.inc
+++ b/meta/conf/distro/include/tclibc-newlib.inc
@@ -25,12 +25,11 @@ LIBC_DEPENDENCIES = "\
 libgloss-dbg \
 libgcc-dev \
 libgcc-dbg \
+libstdc++-dev \
+libstdc++-staticdev \
 "
 
-# compilerlibs defaults to gcc-runtime, but we get runtime from libgloss
-# we set ASSUME_PROVIDED because we cant set PREFERRED_PROVIDER
-# for compilerlibs since its overridden by tcmode-default
-ASSUME_PROVIDED += "virtual/${TARGET_PREFIX}compilerlibs virtual/crypt"
+ASSUME_PROVIDED += "virtual/crypt"
 
 # Its useful to be able to extend newlib, but we dont provide a native variant 
of libgloss
 NEWLIB_EXTENDED ?=  "libgloss libgcc"
diff --git a/meta/recipes-devtools/gcc/gcc-runtime.inc 
b/meta/recipes-devtools/gcc/gcc-runtime.inc
index d3f8ae8..f035fbd 100644
--- a/meta/recipes-devtools/gcc/gcc-runtime.inc
+++ b/meta/recipes-devtools/gcc/gcc-runtime.inc
@@ -17,6 +17,9 @@ EXTRA_OECONF_PATHS = "\
 EXTRA_OECONF_append_linuxstdbase = " --enable-clocale=gnu"
 EXTRA_OECONF_append = " --cache-file=${B}/config.cache"
 
+EXTRA_OECONF_remove_libc-newlib = "--enable-symvers=gnu"
+EXTRA_OECONF_append_libc-newlib = " --with-newlib"
+
 # Disable ifuncs for libatomic on arm conflicts -march/-mcpu
 EXTRA_OECONF_append_arm = " libat_cv_have_ifunc=no "
 
@@ -33,6 +36,9 @@ RUNTIMELIBITM_riscv64 = ""
 RUNTIMELIBSSP ?= ""
 RUNTIMELIBSSP_mingw32 ?= "libssp"
 
+# Only build libstdc++ for newlib
+RUNTIMETARGET_libc-newlib = "libstdc++-v3"
+
 RUNTIMETARGET = "${RUNTIMELIBSSP} libstdc++-v3 libgomp libatomic 
${RUNTIMELIBITM} \
 ${@bb.utils.contains_any('FORTRAN', [',fortran',',f77'], 'libquadmath', 
'', d)} \
 "
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] machine_dict: Add i686 to the ELF machine dictionary

[Alejandro Enedino Hernandez Samaniego]

An error like the following is thrown when building
baremetal applications on some x86 architectures:

  (machine, osabi, abiversion, littleendian, bits) \
= oe.elf.machine_dict(d)[target_os][target_arch]

Exception: KeyError: i686

Since the i686 (target_arch) key does not exist in the dictionary.

Add the key to fix the error.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/lib/oe/elf.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/lib/oe/elf.py b/meta/lib/oe/elf.py
index 2562cea..e0c31ff 100644
--- a/meta/lib/oe/elf.py
+++ b/meta/lib/oe/elf.py
@@ -15,6 +15,7 @@ def machine_dict(d):
 "aarch64" :   (183,0,0,  True, 
 64),
 "aarch64_be" :(183,0,0,  False,
 64),
 "i586" :  (3,  0,0,  True, 
 32),
+"i686" :  (3,  0,0,  True, 
 32),
 "x86_64": (62, 0,0,  True, 
 64),
 "epiphany":   (4643,   0,0,  True, 
 32),
 "lm32":   (138,0,0,  False,
 32),
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] newlib: Upgrade newlib and libgloss to the yearly release 3.2.0

[Alejandro Enedino Hernandez Samaniego]

A new yearly snapshot was taken on January 2nd of 2020, this bumps
newlib to 3.2.0.

 - Mentor Graphics added tcl license (amdgcn-* targets)
 - BSD-Clause-2 was added as well by Dinux (pru-* targets)

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 .../newlib/{libgloss_3.1.0.bb => libgloss_3.2.0.bb}|  0
 meta/recipes-core/newlib/newlib.inc| 10 +-
 meta/recipes-core/newlib/{newlib_3.1.0.bb => newlib_3.2.0.bb}  |  0
 3 files changed, 5 insertions(+), 5 deletions(-)
 rename meta/recipes-core/newlib/{libgloss_3.1.0.bb => libgloss_3.2.0.bb} (100%)
 rename meta/recipes-core/newlib/{newlib_3.1.0.bb => newlib_3.2.0.bb} (100%)

diff --git a/meta/recipes-core/newlib/libgloss_3.1.0.bb 
b/meta/recipes-core/newlib/libgloss_3.2.0.bb
similarity index 100%
rename from meta/recipes-core/newlib/libgloss_3.1.0.bb
rename to meta/recipes-core/newlib/libgloss_3.2.0.bb
diff --git a/meta/recipes-core/newlib/newlib.inc 
b/meta/recipes-core/newlib/newlib.inc
index 2f0553b..d7ac8bf 100644
--- a/meta/recipes-core/newlib/newlib.inc
+++ b/meta/recipes-core/newlib/newlib.inc
@@ -3,21 +3,21 @@ HOMEPAGE = "https://sourceware.org/newlib/;
 DESCRIPTION = "C library intended for use on embedded systems. It is a 
conglomeration of several library parts, all under free software licenses that 
make them easily usable on embedded products."
 SECTION = "libs"
 
-LICENSE = "GPLv2 & LGPLv3 & GPLv3 & LGPLv2 & BSD-2-Clause"
+LICENSE = "GPLv2 & LGPLv3 & GPLv3 & LGPLv2 & BSD-2-Clause & tcl"
 LIC_FILES_CHKSUM = " \
file://COPYING;md5=59530bdf33659b29e73d4adb9f9f6552 \
file://COPYING3.LIB;md5=6a6a8e020838b23406c81b19c1d46df6 \
file://COPYING3;md5=d32239bcb673463ab874e80d47fae504 \
-   file://COPYING.LIBGLOSS;md5=73f5c98779aea7dba4a6c94a74ab0ae2 \
+   file://COPYING.LIBGLOSS;md5=54b778d585443cd7fbfa1b47cbd63a89 \
file://COPYING.LIB;md5=2d5025d4aa3495befef8f17206a5b0a1 \
-   file://COPYING.NEWLIB;md5=100d470af54e2d1d79fbd8498a32ff26 \
+   file://COPYING.NEWLIB;md5=ac17c68751aad7a5298ce3f249121070 \

file://newlib/libc/posix/COPYRIGHT;md5=103468ff1982be840fdf4ee9f8b51bbf \

file://newlib/libc/sys/linux/linuxthreads/LICENSE;md5=73640207fbc79b198c7ffd4ad4d97aa0
 \
"
 
 SRC_URI = "https://sourceware.org/pub/newlib/newlib-${PV}.tar.gz;
-SRC_URI[md5sum] = "f84263b7d524df92a9c9fb30b79e0134"
-SRC_URI[sha256sum] = 
"fb4fa1cc21e9060719208300a61420e4089d6de6ef59cf533b57fe74801d102a"
+SRC_URI[md5sum] = "afb3a01c8c82b6717ea55659a4de30ec"
+SRC_URI[sha256sum] = 
"f7b2322964a1e3b37bec0768da20f50dfb62247d729110974dd95756a53bb6d4"
 
 INHIBIT_DEFAULT_DEPS = "1"
 DEPENDS = "virtual/${TARGET_PREFIX}gcc"
diff --git a/meta/recipes-core/newlib/newlib_3.1.0.bb 
b/meta/recipes-core/newlib/newlib_3.2.0.bb
similarity index 100%
rename from meta/recipes-core/newlib/newlib_3.1.0.bb
rename to meta/recipes-core/newlib/newlib_3.2.0.bb
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [RFC PATCH] Add gnu testsuite execution for OEQA

[Alejandro Enedino Hernandez Samaniego]

Hey guys,

On Sat, Jul 6, 2019 at 5:52 AM Richard Purdie <
richard.pur...@linuxfoundation.org> wrote:

> On Sat, 2019-07-06 at 11:39 +, Nathan Rossi wrote:
> > This patch is an RFC for adding support to execute the gnu test suites
> for
> > binutils, gcc and glibc. With the intention for enabling automated test
> running
> > of these test suites within the OEQA framework such that they can be
> executed by
> > the Yocto Autobuilder.
> >
> > Please note that this patch is not a complete implementation and needs
> > additional work as well as changes based on comments and feedback from
> this RFC.
>
> This is rather cool, thanks!
>
> Looking at this was on my todo list once we got the existing OEQA,
> ptest and ltp setups working well. I'm very happy to have been beaten
> to it though.
>
> > The test suites covered need significant resources or build artifacts
> such
> > that running them on the target is undesirable which rules out the use
> of ptest.
> > Because of this the test suites can be run on the build host and if
> necessary
> > call out to the target.
> >
> > The following implementation creates a number of recipes that are used to
> > build/execute the test suites for the different components. The reason
> for
> > creating separate recipes is primarily due to dependencies and the need
> for
> > components in the sysroot. For example binutils has tests that use the C
> > compiler however binutils is a dependency for the C compiler and thus
> would
> > cause a dependency loop. The issue with sysroots occurs with dependence
> on
> > `*-initial` recipes and the test suites needing the non-initial version.
>
> I think this means you're working with something pre-warrior as we got
> rid of most of the *-initial recipes apart from libgcc-initial.
>

 Yup, I agree with this, and yes, we still have initial recipes, which is
in what Nathan based his work.


> > Some issues with splitting the recipes:
> >  - Rebuilds the recipe
> >- Like gcc-cross-testsuite in this patch, could use a stashed builddir
> >  - Source is duplicated
> >- gcc gets around this with shared source
> >  - Requires having the recipe files and maintaining them
> >- Multiple versions of recipes
> >- Multiple variants of recipes (-cross, -crosssdk, -native if desired)
>
> It might be possible to have multiple tasks in these recipes and have
> the later tasks depend on other pieces of the system like the C
> compiler, thereby avoiding the need for splitting if only the later
> tasks have the dependencies. Not sure if it would work or not but may
> be worth exploring.
>

Worth exploring but might end up being more convoluted than necessary IMO.
Benefit vs Complication issue.



> > Target execution is another issue with the test suites. Note that
> binutils
> > however does not require any target execution. In this patch both
> > qemu-linux-user and ssh target execution solutions are provided. For the
> > purposes of OE, qemu-linux-user may suffice as it has great success at
> executing
> > gcc and gcc-runtime tests with acceptable success at executing the glibc
> tests.
>
> I feel fairly strongly that we probably want to execute these kinds of
> tests under qemu system mode, not the user mode. The reason is that we
> want to be as close to the target environment as we can be and that
> qemu-user testing is at least as much of a test of qemu's emulation
> that it is the behaviour of the compiler or libc (libc in particular).
> I was thinking this and then later read you confirmed my suspicions
> below...
>

I believe the QEMU recipe splitting is also new in the tree, and Nathan
isn't basing his work on that,
so there might be some issues there.


>
> > The glibc test suite can be problematic to execute for a few reasons:
> >  - Requires access to the exact same filesystem as the build host
> >- On physical targets and QEMU this requires NFS mounts
>
> We do have unfs support already under qemu which might make this
> possible.
>
> >  - Relies on exact syscall behaviour
> >- Causes some issues where there are differences between
> qemu-linux-user and
> >  the target architectures kernel
>
> Right, this one worries me and pushes me to want to use qemu system
> mode.
>
> >  - Can consume significant resources (e.g. OOM, or worse trigger
> bugs/panics in
> >kernel drivers)
>
> Any rough guide to what significant is here? ptest needs 1GB memory for
> example. qemu-system mode should limit that to the VMs at least?
>
> >  - Slow to execute
> >- With QEMU system emulation it can take many hours
>
> We do have KVM acceleration for x86 and arm FWIW which is probably
> where we'd start testing this on the autobuilder.
>

Excuse me if I'm mistaken, but would this be something similar to what
we did for python3 optimization?


>
> >- With some physical target architectures it can take days (e.g.
> microblaze)
> >
> > The significantly increased execution speed of qemu-linux-user vs qemu
> system
> > with 

Re: [OE-core] [PATCH] python: make 'python' install everything instead of just the interpretter

[Alejandro Enedino Hernandez Samaniego]

Hey Ross,

On 6/20/19 7:53 AM, Ross Burton wrote:

Follow the python3 behaviour, and common sense, by making 'python' install
python-modules instead of python-core.  This means a user installing python gets
all of Python, instead of just a fraction of the library.

[ YOCTO #13402 ]

Signed-off-by: Ross Burton 
---
  meta/recipes-devtools/python/python_2.7.16.bb | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python_2.7.16.bb 
b/meta/recipes-devtools/python/python_2.7.16.bb
index d70342fe3a6..5f387b8af11 100644
--- a/meta/recipes-devtools/python/python_2.7.16.bb
+++ b/meta/recipes-devtools/python/python_2.7.16.bb
@@ -162,7 +162,7 @@ py_package_preprocess () {
  PACKAGES_remove = "${PN}"
  
  # manual dependency additions

-RPROVIDES_${PN}-core = "${PN}"
+RPROVIDES_${PN}-modules = "${PN}"


Is it just me or we already had this?, maybe we had this only for python3


Alejandro



  RRECOMMENDS_${PN}-core_append_class-nativesdk = " nativesdk-python-modules"
  RRECOMMENDS_${PN}-crypt = "openssl"
  

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] python-numpy: Avoid installing copy of f2py script

[Alejandro Enedino Hernandez Samaniego]

Hey Khem,


On 5/18/19 10:09 PM, Khem Raj wrote:



On 5/17/19 4:49 PM, Alejandro Enedino Hernandez Samaniego wrote:

Both python-numpy and python3-numpy try to install the file:

${D}/${bindir}/f2py

which causes an error during do_rootfs since the package manager 
complains about that.


The numpy packages create basically copies of f2py script with the 
following names:

f2py (on both)
f2py2 (on python 2)
f2py2.7 (on python 2)
f2py3 (on python 3)
f2py3.7 (on python 3)

Remove the copy of f2py from python-numpy in favor of using the one 
from python3

to avoid installation issues at do_rootfs().



what if one does not install py3 ?



If you only install py2, you could still execute f2py2 or f2py2.7, I'm 
simply


favoring the py3 installation by default but it should be mostly harmless

since there's still two other copies of the script with different names.

Cheers,

Alejandro



Signed-off-by: Alejandro Enedino Hernandez Samaniego 


---
  meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb | 4 
  1 file changed, 4 insertions(+)

diff --git 
a/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb 
b/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb

index 5013789..a1133da 100644
--- a/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb
+++ b/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb
@@ -4,3 +4,7 @@ require python-numpy.inc
  RDEPENDS_${PN}_class-target_append = " \
  ${PYTHON_PN}-subprocess \
  "
+
+do_install_append(){
+    rm ${D}/${bindir}/f2py
+}


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] python-numpy: Avoid installing copy of f2py script

[Alejandro Enedino Hernandez Samaniego]

Both python-numpy and python3-numpy try to install the file:

${D}/${bindir}/f2py

which causes an error during do_rootfs since the package manager complains 
about that.

The numpy packages create basically copies of f2py script with the following 
names:
f2py (on both)
f2py2 (on python 2)
f2py2.7 (on python 2)
f2py3 (on python 3)
f2py3.7 (on python 3)

Remove the copy of f2py from python-numpy in favor of using the one from python3
to avoid installation issues at do_rootfs().

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb | 4 
 1 file changed, 4 insertions(+)

diff --git a/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb 
b/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb
index 5013789..a1133da 100644
--- a/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb
+++ b/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb
@@ -4,3 +4,7 @@ require python-numpy.inc
 RDEPENDS_${PN}_class-target_append = " \
 ${PYTHON_PN}-subprocess \
 "
+
+do_install_append(){
+rm ${D}/${bindir}/f2py
+}
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] python-numpy: Avoid installing copy of f2py script

[Alejandro Enedino Hernandez Samaniego]

Both python-numpy and python3-numpy try to install the file:

${D}/${bindir}/f2py

which causes an error during do_rootfs since the package manager complains 
about that.

The numpy packages create basically copies of f2py script with the following 
names:
f2py (on both)
f2py2 (on python 2)
f2py2.7 (on python 2)
f2py3 (on python 3)
f2py3.7 (on python 3)

Remove the copy of f2py from python-numpy in favor of using the one from python3
to avoid installation issues at do_rootfs().

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb | 4 
 1 file changed, 4 insertions(+)

diff --git a/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb 
b/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb
index 5013789..a1133da 100644
--- a/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb
+++ b/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb
@@ -4,3 +4,7 @@ require python-numpy.inc
 RDEPENDS_${PN}_class-target_append = " \
 ${PYTHON_PN}-subprocess \
 "
+
+do_install_append(){
+rm ${D}/${bindir}/f2py
+}
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] python-numpy: Avoid installing copy of f2py script

[Alejandro Enedino Hernandez Samaniego]

Both python-numpy and python3-numpy try to install the file:

${D}/${bindir}/f2py

which causes an error during do_rootfs since the package manager complains 
about that.

The numpy packages create basically copies of f2py script with the following 
names:
f2py (on both)
f2py2 (on python 2)
f2py2.7 (on python 2)
f2py3 (on python 3)
f2py3.7 (on python 3)

Remove the copy of f2py from python-numpy in favor of using the one from python3
to avoid installation issues at do_rootfs().

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb | 4 
 1 file changed, 4 insertions(+)

diff --git a/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb 
b/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb
index 5013789..a1133da 100644
--- a/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb
+++ b/meta/recipes-devtools/python-numpy/python-numpy_1.16.3.bb
@@ -4,3 +4,7 @@ require python-numpy.inc
 RDEPENDS_${PN}_class-target_append = " \
 ${PYTHON_PN}-subprocess \
 "
+
+do_install_append(){
+rm ${D}/${bindir}/f2py
+}
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [OE-Core][PATCH] bmap-tools: Add RDEPENDS on python3-misc

[Alejandro Enedino Hernandez Samaniego]

On 5/3/19 11:32 AM, Alex Kiernan wrote:

On Fri, May 3, 2019 at 6:34 PM Burton, Ross  wrote:

On Fri, 3 May 2019 at 17:48, Alex Kiernan  wrote:

The newer python3 recipe no longer includes python3-misc in target by
default, leading to failures:


This statement isn't true, the python3 package does include 
python3-misc, but the bmap-tools recipe


does not RDEPEND on python3, it RDEPENDS on python3-core (which is not 
the same).




   Traceback (most recent call last):
 File "/usr/bin/bmaptool", line 6, in 
   from pkg_resources import load_entry_point
 File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 42, in 

   import ntpath
   ModuleNotFoundError: No module named 'ntpath'

Looks like ntpath should be in a different package if pkg_resources
won't run without it.


Sorry, should've looked more closely... I'll take a look at it.
Yeah, it might be worth checking if ntpath either belongs on its own 
package or in another one.



  RDEPENDS_${PN} = "python3-core python3-compression python3-mmap python3-setuptools 
python3-fcntl python3-six"
+RDEPENDS_${PN}_append_class-target = " python3-misc"

Why class-target only?


It doesn't exist for native (which was a surprise).


I think you're looking at this incorrectly, the python3-misc package 
does not exists for python3-native (because python3-native isn't split),


but, here, the class override you are using is stating that only 
bmap-tools(target) should RDEPEND on python3-misc, which I don't think


it should be the case, its likely that bmap-tools-native should RDEPEND 
on python3-misc-native, which although it doesn't actually exists


python3-native should be able to RPROVIDE.


Cheers,

Alejandro





--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] run-postinsts: Fix full execution of scripts at first boot

[Alejandro Enedino Hernandez Samaniego]

run-postinsts runs a given set of scripts during the first boot of the
device, when one of these scripts prints something to stdout (isnt
daemonized correctly), since stdout is not available at that time,
the script execution immediately returns with an error (exit_group()),
this error causes the script to terminate all threads within the process,
causing undesired behavior since the script might still had to execute
some other code.

Replace eval built-in with $(), since $() executes in a different shell,
even if one of the scripts exits, all threads of that process will only
be within that session, this ensures other scripts meant to be run are
still run afterwards.

This was only required on the line that actually executes the scripts:
"eval sh -c $i $append_log", other replacements were put for consistency,
and generally, it is recommended to use $() instead of eval anyway.

[YOCTO #13266]

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 .../recipes-devtools/run-postinsts/run-postinsts/run-postinsts | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts 
b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
index 95eff04..419796f 100755
--- a/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
+++ b/meta/recipes-devtools/run-postinsts/run-postinsts/run-postinsts
@@ -65,13 +65,13 @@ exec_postinst_scriptlets() {
for i in `ls $pi_dir`; do
i=$pi_dir/$i
echo "Running postinst $i..."
-   [ "$POSTINST_LOGGING" = "1" ] && eval echo "Running postinst 
$i..." $append_log
+   [ "$POSTINST_LOGGING" = "1" ] && $(echo "Running postinst 
$i..." $append_log)
if [ -x $i ]; then
-   eval sh -c $i $append_log
+   $(sh -c $i $append_log)
rm $i
else
echo "ERROR: postinst $i failed."
-   [ "$POSTINST_LOGGING" = "1" ] && eval echo "ERROR: 
postinst $i failed." $append_log
+   [ "$POSTINST_LOGGING" = "1" ] && $(echo "ERROR: 
postinst $i failed." $append_log)
remove_pi_dir=0
fi
done
@@ -81,11 +81,11 @@ remove_pi_dir=1
 if $pm_installed; then
case $pm in
"ipk")
-   eval opkg configure $append_log
+   $(opkg configure $append_log)
;;
 
"deb")
-   eval dpkg --configure -a $append_log
+   $(dpkg --configure -a $append_log)
;;
esac
 else
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] package_manager: Avoid looking at other multiconfigs for dependencies

[Alejandro Enedino Hernandez Samaniego]

When creating the rootfs, we look at direct and indirect dependencies
and then after processing them we create a link for rpm/ipk/deb packages
needed for each of them on the deploy directory.

The process looks at dependencies that we've already seen to avoid
copying them twice, but when BB_MULTICONFIG is enabled, the dependencies
themsevlves contain "mc:" in them, so duplicate packages are
not found if they come from different multiconfigs, causing an error
when linking the package file since it already exists.

Check which multiconfig we are currently building and avoid processing
dependencies from other multiconfigs.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/lib/oe/package_manager.py | 12 +++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/package_manager.py b/meta/lib/oe/package_manager.py
index 392fe7e..444da5d 100644
--- a/meta/lib/oe/package_manager.py
+++ b/meta/lib/oe/package_manager.py
@@ -643,10 +643,11 @@ def create_packages_dir(d, subrepo_dir, deploydir, 
taskname, filterbydependencie
 
 taskdepdata = d.getVar("BB_TASKDEPDATA", False)
 mytaskname = d.getVar("BB_RUNTASK")
+mc = d.getVar("BB_CURRENT_MC")
 pn = d.getVar("PN")
 seendirs = set()
 multilibs = {}
-   
+
 bb.utils.remove(subrepo_dir, recurse=True)
 bb.utils.mkdirhier(subrepo_dir)
 
@@ -660,6 +661,10 @@ def create_packages_dir(d, subrepo_dir, deploydir, 
taskname, filterbydependencie
 for dep in taskdepdata:
 data = taskdepdata[dep]
 if data[1] == mytaskname and data[0] == pn:
+if mc != 'default':
+depmc = dep.split(':')[1]
+if depmc != mc:
+continue
 start = dep
 break
 if start is None:
@@ -673,6 +678,11 @@ def create_packages_dir(d, subrepo_dir, deploydir, 
taskname, filterbydependencie
 next = []
 for dep2 in start:
 for dep in taskdepdata[dep2][3]:
+# We shouldnt care of other multiconfigs
+if mc != 'default':
+depmc = dep.split(':')[1]
+if depmc != mc:
+continue
 if taskdepdata[dep][0] != pn:
 if "do_" + taskname in dep:
 pkgdeps.add(dep)
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] staging.bbclass: Avoid looking at other multiconfigs on extend_recipe_sysroot

[Alejandro Enedino Hernandez Samaniego]

On extend_recipe_sysroot we only compare taskname and pn, but if we have
some dependency to a recipe from another multiconfig, these two will
still match if it is the same recipe, causing an error since the build
system believes it has already proccesed that dependency.

Check if this is a multiconfig build, if it is, make sure the current
dependency is coming from the same multiconfig as the one we are
building.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/classes/staging.bbclass | 4 
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
index 84e13ba..2dbe65a 100644
--- a/meta/classes/staging.bbclass
+++ b/meta/classes/staging.bbclass
@@ -277,6 +277,10 @@ python extend_recipe_sysroot() {
 for dep in taskdepdata:
 data = taskdepdata[dep]
 if data[1] == mytaskname and data[0] == pn:
+if mc != 'default':
+depmc = dep.split(':')[1]
+if depmc != mc:
+continue
 start = dep
 break
 if start is None:
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] create_manifest2: Dont match filenames which contain the directory name for new manifest

[Alejandro Enedino Hernandez Samaniego]

When creating a new python2 manifest, there is a corner case on which
the filepath for a certain dependency that was found, could contain
the path of an existing folder, e.g. ${libdir}/python2.7/xmlrpclib.py
module path contains ${libdir}/python2.7/xml, this causes an issue where
the dependency doesnt get eventually added on FILES for that module.

This patch checks if the dependency that was found is a directory, if it
is, it checks if it matches one of the existing directories on the
manifest, if it is not, then it checks if the dependency's path (without
the filename) matches one of the directories.

Also some misc indentation fixes.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 .../python/python/create_manifest2.py  | 28 +-
 1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/meta/recipes-devtools/python/python/create_manifest2.py 
b/meta/recipes-devtools/python/python/create_manifest2.py
index b674865..3465951 100644
--- a/meta/recipes-devtools/python/python/create_manifest2.py
+++ b/meta/recipes-devtools/python/python/create_manifest2.py
@@ -209,7 +209,13 @@ for key in old_manifest:
 
 inFolders=False
 for folder in allfolders:
-if folder in item:
+# The module could have a directory named after it, e.g. xml, 
if we take out the filename from the path
+# we'll end up with ${libdir}, and we want ${libdir}/xml
+if isFolder(item):
+check_path = item
+else:
+check_path = os.path.dirname(item)
+if folder in check_path :
 inFolders = True # Did we find a folder?
 folderFound = False # Second flag to break inner for
 # Loop only through packages which contain folders
@@ -262,16 +268,16 @@ for key in old_manifest:

new_manifest[key]['rdepends'].append(newkey)
 break
 else:
-  # Debug
-  print('Adding %s to %s FILES' % (item, key))
-  # Since it wasnt found on another package, its not an 
RDEP, so add it to FILES for this package
-  new_manifest[key]['files'].append(item)
-  if item.endswith('*'):
-  wildcards.append(item)
-  if item not in allfiles:
-  allfiles.append(item)
-  else:
-  repeated.append(item)
+# Debug
+print('Adding %s to %s FILES' % (item, key))
+# Since it wasnt found on another package, its not an 
RDEP, so add it to FILES for this package
+new_manifest[key]['files'].append(item)
+if item.endswith('*'):
+wildcards.append(item)
+if item not in allfiles:
+allfiles.append(item)
+else:
+repeated.append(item)
 
 print ('The following files are repeated (contained in more than one package), 
please check which package should get it:')
 print (repeated)
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] create_manifest3: Dont match filenames which contain the directory name for new manifest

[Alejandro Enedino Hernandez Samaniego]

When creating a new python3 manifest, there is a corner case on which
the filepath for a certain dependency that was found, could contain
the path of an existing folder, e.g. ${libdir}/python3/xmlrpclib.py
module path contains ${libdir}/python3/xml, this causes an issue where
the dependency doesnt get eventually added on FILES for that module.

This patch checks if the dependency that was found is a directory, if it
is, it checks if it matches one of the existing directories on the
manifest, if it is not, then it checks if the dependency's path (without
the filename) matches one of the directories.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/recipes-devtools/python/python3/create_manifest3.py | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3/create_manifest3.py 
b/meta/recipes-devtools/python/python3/create_manifest3.py
index f7d4587..4da02a2 100644
--- a/meta/recipes-devtools/python/python3/create_manifest3.py
+++ b/meta/recipes-devtools/python/python3/create_manifest3.py
@@ -310,7 +310,13 @@ for pypkg in old_manifest:
 pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}')
 inFolders = False
 for folder in allfolders:
-if folder in pymodule_dep:
+# The module could have a directory named after it, e.g. xml, 
if we take out the filename from the path
+# we'll end up with ${libdir}, and we want ${libdir}/xml
+if isFolder(pymodule_dep):
+check_path = pymodule_dep
+else:
+check_path = os.path.dirname(pymodule_dep)
+if folder in check_path :
 inFolders = True # Did we find a folder?
 folderFound = False # Second flag to break inner for
 # Loop only through packages which contain folders
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] python3: Adds instructions to the manifest file

[Alejandro Enedino Hernandez Samaniego]

While there is a bit of documentation regarding building a new
manifest file for python, it seems that users usually only read
the manifest file.

The manifest file is in JSON format which doesn't allow comments,
hence why instructions were initially put elsewhere.

This patch hacks the call to open the JSON manifest file by using a
marker to trick it into reading only part of the file as the manifest
itself, and keep the other part as comments, which contain instructions
for the user to run the create_manifest task after an upgrade or when
adding a new package.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 .../python/python3-native_3.5.6.bb |  6 +-
 .../python/python3/create_manifest3.py | 20 -
 .../python/python3/python3-manifest.json   | 94 +-
 meta/recipes-devtools/python/python3_3.5.6.bb  |  6 +-
 4 files changed, 119 insertions(+), 7 deletions(-)

diff --git a/meta/recipes-devtools/python/python3-native_3.5.6.bb 
b/meta/recipes-devtools/python/python3-native_3.5.6.bb
index d5953cf..e6ae2d3 100644
--- a/meta/recipes-devtools/python/python3-native_3.5.6.bb
+++ b/meta/recipes-devtools/python/python3-native_3.5.6.bb
@@ -84,7 +84,11 @@ python(){
 import json
 pythondir = d.getVar('THISDIR',True)
 with open(pythondir+'/python3/python3-manifest.json') as manifest_file:
-python_manifest=json.load(manifest_file)
+manifest_str =  manifest_file.read()
+json_start = manifest_str.find('# EOC') + 6
+manifest_file.seek(json_start)
+manifest_str = manifest_file.read()
+python_manifest = json.loads(manifest_str)
 
 rprovides = d.getVar('RPROVIDES').split()
 
diff --git a/meta/recipes-devtools/python/python3/create_manifest3.py 
b/meta/recipes-devtools/python/python3/create_manifest3.py
index fddb23cd..f7d4587 100644
--- a/meta/recipes-devtools/python/python3/create_manifest3.py
+++ b/meta/recipes-devtools/python/python3/create_manifest3.py
@@ -22,7 +22,7 @@
 #
 #
 # This way we will create a new manifest from the data structure that was 
built during
-# this process, ont this new manifest each package will contain specifically 
only
+# this process, on this new manifest each package will contain specifically 
only
 # what it needs to run.
 #
 # There are some caveats which we try to deal with, such as repeated files on 
different
@@ -36,7 +36,7 @@
 # Tha method to handle cached files does not work when a module includes a 
folder which
 # itself contains the pycache folder, gladly this is almost never the case.
 #
-# Author: Alejandro Enedino Hernandez Samaniego "aehs29" 
+# Author: Alejandro Enedino Hernandez Samaniego "aehs29" 
 
 
 import sys
@@ -78,9 +78,21 @@ def isCached(item):
 else:
 return False
 
+def prepend_comments(comments, json_manifest):
+with open(json_manifest, 'r+') as manifest:
+json_contents = manifest.read()
+manifest.seek(0, 0)
+manifest.write(comments + json_contents)
+
 # Read existing JSON manifest
 with open('python3-manifest.json') as manifest:
-old_manifest = json.load(manifest, 
object_pairs_hook=collections.OrderedDict)
+# The JSON format doesn't allow comments so we hack the call to keep the 
comments using a marker
+manifest_str =  manifest.read()
+json_start = manifest_str.find('# EOC') + 6 # EOC + \n
+manifest.seek(0)
+comments = manifest.read(json_start)
+manifest_str = manifest.read()
+old_manifest = json.loads(manifest_str, 
object_pairs_hook=collections.OrderedDict)
 
 #
 # First pass to get core-package functionality, because we base everything on 
the fact that core is actually working
@@ -402,6 +414,8 @@ with open('python3-manifest.json.new','w') as outfile:
 json.dump(new_manifest,outfile, indent=4)
 outfile.write('\n')
 
+prepend_comments(comments,'python3-manifest.json.new')
+
 if (repeated):
 error_msg = '\n\nERROR:\n'
 error_msg += 'The following files are repeated (contained in more than one 
package),\n'
diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index 5329bf7..685e3d5 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -1,3 +1,93 @@
+# DO NOT (entirely) modify this file manually, please read.
+#
+# IMPORTANT NOTE:
+# Please keep in mind that the create_manifest task relies on the fact the the
+# target and native Python packages are the same, and it also needs to be 
executed
+# with a fully working native package (with all the PACKAGECONFIGs enabled and 
all
+# and all the modules should be working, check log.do_compile), otherwise the 
script
+# will fail to find dependencies correctly, this note is valid either if you 
are
+# upgrading to a new Python version or adding a new package.
+#
+#
+# If you are adding a new package please f

[OE-core] [PATCH 2/2] python: Adds instructions to the manifest file

[Alejandro Enedino Hernandez Samaniego]

While there is a bit of documentation regarding building a new
manifest file for python, it seems that users usually only read
the manifest file.

The manifest file is in JSON format which doesn't allow comments,
hence why instructions were initially put elsewhere.

This patch hacks the call to open the JSON manifest file by using a
marker to trick it into reading only part of the file as the manifest
itself, and keep the other part as comments, which contain instructions
for the user to run the create_manifest task after an upgrade or
when adding a new package.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 .../python/python-native_2.7.15.bb |  6 +-
 .../python/python/create_manifest2.py  | 21 -
 .../python/python/python2-manifest.json| 92 +-
 meta/recipes-devtools/python/python_2.7.15.bb  |  6 +-
 4 files changed, 118 insertions(+), 7 deletions(-)

diff --git a/meta/recipes-devtools/python/python-native_2.7.15.bb 
b/meta/recipes-devtools/python/python-native_2.7.15.bb
index 7c491fa..de35104 100644
--- a/meta/recipes-devtools/python/python-native_2.7.15.bb
+++ b/meta/recipes-devtools/python/python-native_2.7.15.bb
@@ -69,7 +69,11 @@ python(){
 import json
 pythondir = d.getVar('THISDIR',True)
 with open(pythondir+'/python/python2-manifest.json') as manifest_file:
-python_manifest=json.load(manifest_file)
+manifest_str =  manifest_file.read()
+json_start = manifest_str.find('# EOC') + 6
+manifest_file.seek(json_start)
+manifest_str = manifest_file.read()
+python_manifest = json.loads(manifest_str)
 
 rprovides = d.getVar('RPROVIDES').split()
 
diff --git a/meta/recipes-devtools/python/python/create_manifest2.py 
b/meta/recipes-devtools/python/python/create_manifest2.py
index 879..b674865 100644
--- a/meta/recipes-devtools/python/python/create_manifest2.py
+++ b/meta/recipes-devtools/python/python/create_manifest2.py
@@ -22,7 +22,7 @@
 #
 #
 # This way we will create a new manifest from the data structure that was 
built during
-# this process, ont this new manifest each package will contain specifically 
only
+# this process, on this new manifest each package will contain specifically 
only
 # what it needs to run.
 #
 # There are some caveats which we try to deal with, such as repeated files on 
different
@@ -30,7 +30,7 @@
 # Its also important to note that this method only works for python files, and 
shared
 # libraries. Static libraries, header files and binaries need to be dealt with 
manually.
 #
-# Author: Alejandro Enedino Hernandez Samaniego "aehs29" 
+# Author: Alejandro Enedino Hernandez Samaniego "aehs29" 
 
 
 import sys
@@ -62,10 +62,21 @@ def isFolder(value):
   else:
 return False
 
+def prepend_comments(comments, json_manifest):
+with open(json_manifest, 'r+') as manifest:
+json_contents = manifest.read()
+manifest.seek(0, 0)
+manifest.write(comments + json_contents)
+
 # Read existing JSON manifest
 with open('python2-manifest.json') as manifest:
-  old_manifest = json.load(manifest, object_pairs_hook=collections.OrderedDict)
-
+# The JSON format doesn't allow comments so we hack the call to keep the 
comments using a marker
+manifest_str =  manifest.read()
+json_start = manifest_str.find('# EOC') + 6 # EOC + \n
+manifest.seek(0)
+comments = manifest.read(json_start)
+manifest_str = manifest.read()
+old_manifest = json.loads(manifest_str, 
object_pairs_hook=collections.OrderedDict)
 
 # First pass to get core-package functionality, because we base everything on 
the fact that core is actually working
 # Not exactly the same so it should not be a function
@@ -277,3 +288,5 @@ for key in new_manifest:
 # Create the manifest from the data structure that was built
 with open('python2-manifest.json.new','w') as outfile:
 json.dump(new_manifest,outfile, indent=4)
+
+prepend_comments(comments,'python2-manifest.json.new')
diff --git a/meta/recipes-devtools/python/python/python2-manifest.json 
b/meta/recipes-devtools/python/python/python2-manifest.json
index 260fa6f..3d1cf59 100644
--- a/meta/recipes-devtools/python/python/python2-manifest.json
+++ b/meta/recipes-devtools/python/python/python2-manifest.json
@@ -1,3 +1,93 @@
+# DO NOT (entirely) modify this file manually, please read.
+#
+# IMPORTANT NOTE:
+# Please keep in mind that the create_manifest task relies on the fact the the
+# target and native Python packages are the same, and it also needs to be 
executed
+# with a fully working native package (with all the PACKAGECONFIGs enabled and 
all
+# and all the modules should be working, check log.do_compile), otherwise the 
script
+# will fail to find dependencies correctly, this note is valid either if you 
are
+# upgrading to a new Python version or adding a new package.
+#
+#
+# If you are adding a new package please follow the next steps:
+# How 

Re: [OE-core] [PATCH 2/2] python: consolidate common patches

[Alejandro Enedino Hernandez Samaniego]

Hey Anuj,


On 10/03/2018 10:09 PM, Anuj Mittal wrote:

Move inclusion of common patches to .inc to simplify things a bit.

Signed-off-by: Anuj Mittal 
---
  meta/recipes-devtools/python/python-native_2.7.15.bb | 3 ---
  meta/recipes-devtools/python/python.inc  | 3 +++
  meta/recipes-devtools/python/python_2.7.15.bb| 3 ---
  3 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/meta/recipes-devtools/python/python-native_2.7.15.bb 
b/meta/recipes-devtools/python/python-native_2.7.15.bb
index 7c491fa3e0..ee2ec1082a 100644
--- a/meta/recipes-devtools/python/python-native_2.7.15.bb
+++ b/meta/recipes-devtools/python/python-native_2.7.15.bb
@@ -11,10 +11,7 @@ SRC_URI += "\
  file://debug.patch \
  file://unixccompiler.patch \
  file://nohostlibs.patch \
-file://multilib.patch \
-file://add-md5module-support.patch \
  file://builddir.patch \
-file://parallel-makeinst-create-bindir.patch \
  file://revert_use_of_sysconfigdata.patch \
 "
  
diff --git a/meta/recipes-devtools/python/python.inc b/meta/recipes-devtools/python/python.inc

index 66923678b1..38cc79cb41 100644
--- a/meta/recipes-devtools/python/python.inc
+++ b/meta/recipes-devtools/python/python.inc
@@ -13,6 +13,9 @@ SRC_URI = 
"http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
 
file://0003-bpo-34834-Fix-test_ssl.test_options-to-account-for-O.patch \
 
file://0004-bpo-34836-fix-test_default_ecdh_curve-needs-no-tlsv1.patch \
 
file://0001-bpo-33354-Fix-test_ssl-when-a-filename-cannot-be-enc.patch \


The multilib.patch is actually a different file for the native and 
target packages, I suggest we rename it and keep it on each recipe 
separately.


Cheers,

Alejandro


+   file://multilib.patch \
+   file://add-md5module-support.patch \
+   file://parallel-makeinst-create-bindir.patch \
 "
  
  SRC_URI[md5sum] = "a80ae3cc478460b922242f43a1b4094d"

diff --git a/meta/recipes-devtools/python/python_2.7.15.bb 
b/meta/recipes-devtools/python/python_2.7.15.bb
index 472c1da4bf..8bc0f273ec 100644
--- a/meta/recipes-devtools/python/python_2.7.15.bb
+++ b/meta/recipes-devtools/python/python_2.7.15.bb
@@ -12,10 +12,8 @@ SRC_URI += "\
file://03-fix-tkinter-detection.patch \
file://06-avoid_usr_lib_termcap_path_in_linking.patch \
${DISTRO_SRC_URI} \
-  file://multilib.patch \
file://cgi_py.patch \
file://setup_py_skip_cross_import_check.patch \
-  file://add-md5module-support.patch \
file://host_include_contamination.patch \
file://fix_for_using_different_libdir.patch \
file://setuptweaks.patch \
@@ -26,7 +24,6 @@ SRC_URI += "\
file://python-2.7.3-remove-bsdb-rpath.patch \
file://fix-makefile-for-ptest.patch \
file://run-ptest \
-  file://parallel-makeinst-create-bindir.patch \
file://use_sysroot_ncurses_instead_of_host.patch \
file://add-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch \
file://pass-missing-libraries-to-Extension-for-mul.patch \


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] python3{,-native}: update to 3.7.0

[Alejandro Enedino Hernandez Samaniego]

Hey Jens,


On 09/10/2018 11:58 PM, Jens Rehsack wrote:



Am 10.09.2018 um 23:33 schrieb Alejandro Enedino Hernandez Samaniego 
<mailto:alejandro.enedino.hernandez-samani...@xilinx.com>>:


Hey Jens,


As I explained before, when you create a manifest for python 
(target), it uses the native build as base (it literally runs the 
native python that was just built), it is assumed its the same 
version as target and contains all the modules provided by upstream, 
otherwise the missing modules cannot be checked for dependencies, and 
the manifest becomes incoherent, so its not an option to have an 
incomplete python native build.


In that case, uuid for target never gets deployed, but it is. And I 
didn't see any packaging issues for `python3` nor for `nativesdk-python3`



I don't see what that has to do with anything, fixing the native build 
should not affect what gets deployed on target, thats exactly why we 
have a manifest, so they user can decide what to install and what not to 
install.
I know you didn't see any packaging issues, but that doesn't mean they 
don't exist, just from the log I showed you, I can tell you that the 
python3-crypt package is not created correctly, for example, if you do:


IMAGE_INSTALL_append = " python3-crypt"

Boot the image, run python3 and you try to import sha3, it will fail, 
because the sha3*.so library won't be on the filesystem.



And thats because the sha3.*.so library was just introduced in this 
upgrade, and our manifest isn't aware it exists, hence it'll end up on 
python3-misc and you have just created an unnecessary dependency from 
python3-crypt to python3-misc (and worse, a dependency which were not 
even aware of, until we test it manually), which beats the whole purpose 
of the manifest.


The do_package function is not gonna fail just because, so you won't see 
errors, but the files will be packaged incorrectly, causing runtime 
errors as a consequence, the create_manifest task tries to solve these 
runtime issues before they happen.


Cheers,

Alejandro


Yes you probably need a patch to look at the correct directories for 
the h files, as well as a dependency to make the h files available on 
recipe-sysroot-native.



Please check the submission.


I did, its not checking inside recipe-sysroot-native.




Jens


Alejandro


On 09/10/2018 02:05 PM, Jens Rehsack wrote:

Hey Alejandro,

I fixed that for cross-compile only, since I would need add a patch 
and a dependency python3-native for one thing: calculate uuids.
When you can explain to me why the python-native needs that, I'll 
change that from -target & nativesdk to all.


From my point of view it's not a question of having every (unneeded) 
python module being built for the native python, which is used for 
cross-compiling python and some modules only.


Cheers,
Jens

Am 10.09.2018 um 22:17 schrieb Alejandro Enedino Hernandez 
Samaniego <mailto:alejandro.enedino.hernandez-samani...@xilinx.com>>:


Hey Jens,

The compilation log for python3-native still shows that it didn't 
build the uuid module



Python build finished successfully!
The necessary bits to build these optional modules were not found:
_uuid

Please look at my previous reply to find how this can be solved 
(its likely a missing DEPENDS).



Also, this patch is missing the new python3 manifest for this 
release, there appears to be a few new modules that we need to 
decide which package they belong to, this is the output of bitbake 
python3 -c create_manifest:



| ERROR:
| The following files are repeated (contained in more than one 
package),
| this is likely to happen when new files are introduced after an 
upgrade,

| please check which package should get it,
|  modify the manifest accordingly and re-run the create_manifest task:
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_blake2.*.so
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha3.*.so
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_contextvars.*.so
| ${libdir}/python${PYTHON_MAJMIN}/contextvars.py
| ${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextvars.*.pyc
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_queue.*.so


Cheers,


Alejandro


On 09/10/2018 09:38 AM, Burton, Ross wrote:

One thing to be aware of is that I've been fixing up Python's PGO
support and there's a slew of patches in master-next and more just
posted that this needs to be rebased on top of.  Good news is that my
patches remove two of the patches we've been carrying!

Ross

On 10 September 2018 at 17:36, Jens Rehsack <mailto:s...@netbsd.org>> wrote:


Am 10.09.2018 um 11:35 schrieb Alexander Kanavin 
mailto:alex.kana...@gmail.com>>:


Large parts of dnf and friends have been rewritten in c++. I have not
yet updated and reviewed that, that will happen in the next cycle.


If I can prepare something for you - drop me a note.
Otherwise - the perl-5.28 update ("." in @INC, regex buffer 
overflow, ...)

is also
awaiting some progress (I ca

Re: [OE-core] python test files installed in image on purpose?

[Alejandro Enedino Hernandez Samaniego]

Hey Andrew,


I seem to remember some of them were required, both unittest and and 
sqlite3-tests can be removed from the python-modules package,


as for the others, you would need to remove them at do_install() , and I 
would not mind making it optional.



Cheers,

Alejandro


On 09/07/2018 12:02 PM, Andrew Geissler wrote:

Hi,

We've been running into some flash space issues over in the openbmc
project. I noticed the python.bb file (python_2.7.14.bb for us) is
installing some test folders/files in our rootfs image.

# find /usr/lib/python2.7/ -name "test" -print
/usr/lib/python2.7/ctypes/test
/usr/lib/python2.7/email/test

I'm looking to see if there's any functional reason these tests are
included in the image.

In our compressed filesystem we're talking about ~150KB of space but
right now that's a pretty big deal for us. Would the community be
interested in a patch to not install (or make it an option like is
done with INCLUDE_PYCS) these test files?  Or is this something we
should just handle in a .bbappend file for our distro?

Here's a complete list of test directories in the python package.
./Python-2.7.14/Lib/bsddb/test
./Python-2.7.14/Lib/sqlite3/test
./Python-2.7.14/Lib/ctypes/test
./Python-2.7.14/Lib/unittest/test
./Python-2.7.14/Lib/lib-tk/test
./Python-2.7.14/Lib/email/test

Thanks,
Andrew


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] python3{,-native}: update to 3.7.0

[Alejandro Enedino Hernandez Samaniego]

Hey Jens,


As I explained before, when you create a manifest for python (target), 
it uses the native build as base (it literally runs the native python 
that was just built), it is assumed its the same version as target and 
contains all the modules provided by upstream, otherwise the missing 
modules cannot be checked for dependencies, and the manifest becomes 
incoherent, so its not an option to have an incomplete python native build.



Yes you probably need a patch to look at the correct directories for the 
h files, as well as a dependency to make the h files available on 
recipe-sysroot-native.


Alejandro


On 09/10/2018 02:05 PM, Jens Rehsack wrote:

Hey Alejandro,

I fixed that for cross-compile only, since I would need add a patch 
and a dependency python3-native for one thing: calculate uuids.
When you can explain to me why the python-native needs that, I'll 
change that from -target & nativesdk to all.


From my point of view it's not a question of having every (unneeded) 
python module being built for the native python, which is used for 
cross-compiling python and some modules only.


Cheers,
Jens

Am 10.09.2018 um 22:17 schrieb Alejandro Enedino Hernandez Samaniego 
<mailto:alejandro.enedino.hernandez-samani...@xilinx.com>>:


Hey Jens,

The compilation log for python3-native still shows that it didn't 
build the uuid module



Python build finished successfully!
The necessary bits to build these optional modules were not found:
_uuid

Please look at my previous reply to find how this can be solved (its 
likely a missing DEPENDS).



Also, this patch is missing the new python3 manifest for this 
release, there appears to be a few new modules that we need to decide 
which package they belong to, this is the output of bitbake python3 
-c create_manifest:



| ERROR:
| The following files are repeated (contained in more than one package),
| this is likely to happen when new files are introduced after an 
upgrade,

| please check which package should get it,
|  modify the manifest accordingly and re-run the create_manifest task:
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_blake2.*.so
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha3.*.so
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_contextvars.*.so
| ${libdir}/python${PYTHON_MAJMIN}/contextvars.py
| ${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextvars.*.pyc
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_queue.*.so


Cheers,


Alejandro


On 09/10/2018 09:38 AM, Burton, Ross wrote:

One thing to be aware of is that I've been fixing up Python's PGO
support and there's a slew of patches in master-next and more just
posted that this needs to be rebased on top of.  Good news is that my
patches remove two of the patches we've been carrying!

Ross

On 10 September 2018 at 17:36, Jens Rehsack <mailto:s...@netbsd.org>> wrote:


Am 10.09.2018 um 11:35 schrieb Alexander Kanavin 
mailto:alex.kana...@gmail.com>>:


Large parts of dnf and friends have been rewritten in c++. I have not
yet updated and reviewed that, that will happen in the next cycle.


If I can prepare something for you - drop me a note.
Otherwise - the perl-5.28 update ("." in @INC, regex buffer 
overflow, ...)

is also
awaiting some progress (I can keep "myself" busy).

There's already enough disruption to deal with (postinsts errors,
openssl 1.1, both caused by me :)


You know, corner, ash, ... things happen. But there is progress! Great!

Good that we got all the way to do_rootfs though with 3.7.


Yeah, but than came postinst (coreutils :P) :D

Cheers

Alex

2018-09-10 0:38 GMT+02:00 Tim Orling <mailto:ticot...@gmail.com>>:


I did not review the patches closely, but I did try to build
core-image-full-cmdline with the tip of poky and these patches applied.
Everything was fine until do_rootfs... I've attached the log.

Essentially, there are some bits of dnf and so on which are not 
ready for
Python 3.7. We have dnf version 2.7.5, but the latest upstream 
release is

3.4.0 (with a 3.5.0 just 3 days ago). Not sure yet if that would have
helped.

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org 
<mailto:Openembedded-core@lists.openembedded.org>

http://lists.openembedded.org/mailman/listinfo/openembedded-core


--
Jens Rehsack - rehs...@gmail.com


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org 
<mailto:Openembedded-core@lists.openembedded.org>

http://lists.openembedded.org/mailman/listinfo/openembedded-core


--
Jens Rehsack - rehs...@gmail.com <mailto:rehs...@gmail.com>



-- 
___
Openembedded-core mailing list
Ope

Re: [OE-core] [PATCH v2] python3{,-native}: update to 3.7.0

[Alejandro Enedino Hernandez Samaniego]

Hey Jens,

The compilation log for python3-native still shows that it didn't build 
the uuid module



Python build finished successfully!
The necessary bits to build these optional modules were not found:
_uuid

Please look at my previous reply to find how this can be solved (its 
likely a missing DEPENDS).



Also, this patch is missing the new python3 manifest for this release, 
there appears to be a few new modules that we need to decide which 
package they belong to, this is the output of bitbake python3 -c 
create_manifest:



| ERROR:
| The following files are repeated (contained in more than one package),
| this is likely to happen when new files are introduced after an upgrade,
| please check which package should get it,
|  modify the manifest accordingly and re-run the create_manifest task:
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_blake2.*.so
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_sha3.*.so
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_contextvars.*.so
| ${libdir}/python${PYTHON_MAJMIN}/contextvars.py
| ${libdir}/python${PYTHON_MAJMIN}/__pycache__/contextvars.*.pyc
| ${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_queue.*.so


Cheers,


Alejandro


On 09/10/2018 09:38 AM, Burton, Ross wrote:

One thing to be aware of is that I've been fixing up Python's PGO
support and there's a slew of patches in master-next and more just
posted that this needs to be rebased on top of.  Good news is that my
patches remove two of the patches we've been carrying!

Ross

On 10 September 2018 at 17:36, Jens Rehsack  wrote:


Am 10.09.2018 um 11:35 schrieb Alexander Kanavin :

Large parts of dnf and friends have been rewritten in c++. I have not
yet updated and reviewed that, that will happen in the next cycle.


If I can prepare something for you - drop me a note.
Otherwise - the perl-5.28 update ("." in @INC, regex buffer overflow, ...)
is also
awaiting some progress (I can keep "myself" busy).

There's already enough disruption to deal with (postinsts errors,
openssl 1.1, both caused by me :)


You know, corner, ash, ... things happen. But there is progress! Great!

Good that we got all the way to do_rootfs though with 3.7.


Yeah, but than came postinst (coreutils :P) :D

Cheers

Alex

2018-09-10 0:38 GMT+02:00 Tim Orling :

I did not review the patches closely, but I did try to build
core-image-full-cmdline with the tip of poky and these patches applied.
Everything was fine until do_rootfs... I've attached the log.

Essentially, there are some bits of dnf and so on which are not ready for
Python 3.7. We have dnf version 2.7.5, but the latest upstream release is
3.4.0 (with a 3.5.0 just 3 days ago). Not sure yet if that would have
helped.

--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core


--
Jens Rehsack - rehs...@gmail.com


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] python3: fix indentation on create_manifest3

[Alejandro Enedino Hernandez Samaniego]

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 .../python/python3/create_manifest3.py | 108 ++---
 1 file changed, 54 insertions(+), 54 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/create_manifest3.py 
b/meta/recipes-devtools/python/python3/create_manifest3.py
index 1849152..2db5e3b 100644
--- a/meta/recipes-devtools/python/python3/create_manifest3.py
+++ b/meta/recipes-devtools/python/python3/create_manifest3.py
@@ -50,8 +50,8 @@ pyversion = str(sys.argv[1])
 # Hack to get native python search path (for folders), not fond of it but it 
works for now
 pivot = 'recipe-sysroot-native'
 for p in sys.path:
-  if pivot in p:
-nativelibfolder = p[:p.find(pivot)+len(pivot)]
+if pivot in p:
+nativelibfolder = p[:p.find(pivot)+len(pivot)]
 
 # Empty dict to hold the whole manifest
 new_manifest = {}
@@ -65,21 +65,21 @@ hasfolders = []
 allfolders = []
 
 def isFolder(value):
-  value = value.replace('${PYTHON_MAJMIN}',pyversion)
-  if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or 
os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or 
os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')):
-return True
-  else:
-return False
+value = value.replace('${PYTHON_MAJMIN}',pyversion)
+if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or 
os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or 
os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')):
+return True
+else:
+return False
 
 def isCached(item):
-  if '__pycache__' in item:
-return True
-  else:
-return False
+if '__pycache__' in item:
+return True
+else:
+return False
 
 # Read existing JSON manifest
 with open('python3-manifest.json') as manifest:
-  old_manifest = json.load(manifest)
+old_manifest = json.load(manifest)
 
 #
 # First pass to get core-package functionality, because we base everything on 
the fact that core is actually working
@@ -114,50 +114,50 @@ for coredep in output.split():
 # pass them to the manifest directly.
 
 for filedep in old_manifest['core']['files']:
-  if isFolder(filedep):
-if isCached(filedep):
-if filedep not in old_manifest['core']['cached']:
-old_manifest['core']['cached'].append(filedep)
-else:
+if isFolder(filedep):
+if isCached(filedep):
+if filedep not in old_manifest['core']['cached']:
+old_manifest['core']['cached'].append(filedep)
+else:
+if filedep not in old_manifest['core']['files']:
+old_manifest['core']['files'].append(filedep)
+continue
+if '${bindir}' in filedep:
 if filedep not in old_manifest['core']['files']:
 old_manifest['core']['files'].append(filedep)
-continue
-  if '${bindir}' in filedep:
-if filedep not in old_manifest['core']['files']:
-  old_manifest['core']['files'].append(filedep)
-continue
-  if filedep == '':
-continue
-  if '${includedir}' in filedep:
-if filedep not in old_manifest['core']['files']:
-  old_manifest['core']['files'].append(filedep)
-continue
-
-  # Get actual module name , shouldnt be affected by libdir/bindir, etc.
-  pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0]
-
-
-  # We now know that were dealing with a python module, so we can import it
-  # and check what its dependencies are.
-  # We launch a separate task for each module for deterministic behavior.
-  # Each module will only import what is necessary for it to work in specific.
-  # The output of each task will contain each module's dependencies
-
-  print ('Getting dependencies for module: %s' % pymodule)
-  output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 
'%s' % pymodule]).decode('utf8')
-  print ('The following dependencies were found for module %s:\n' % pymodule)
-  print (output)
-
-
-  for pymodule_dep in output.split():
-pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}')
-
-if isCached(pymodule_dep):
-if pymodule_dep not in old_manifest['core']['cached']:
-old_manifest['core']['cached'].append(pymodule_dep)
-else:
-if pymodule_dep not in old_manifest['core']['files']:
-old_manifest['core']['files'].append(pymodule_dep)
+continue
+if filedep == '':
+continue
+if '${includedir}' in filedep:
+if filedep not in old_manifest['core']['files']:
+old_manifest['core']['files'].append(filedep)
+continue
+
+# Get actual module name , shouldnt be affected by libdir/bindir, etc.
+pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0]
+
+
+# We now know that were dealing with a python module, so we can import it
+# and check what its dependencies are.
+# We launch a separate task for each

[OE-core] [PATCH 3/4] python3: Clean up create manifest script

[Alejandro Enedino Hernandez Samaniego]

This patch intends to clean up the whole create_manifest script/task
for python3.

This is an effort to make the code more human friendly and facilitate
adoption, it not only cleans up the code but it also improves comments,
it should also be easier to upgrade the manifest after each python3
upgrade now, with these fixes the transition to python 3.7 should be
seamless.

It fixes a rather harmless bug where module dependencies were being
added twice to the core package and adds tests and sqlite3-tests
as special packages since we want specific dependencies on those.
It also fixes a bug that happened on a few packages that
contained a directory with the same name as the module itself
e.g. asyncio, where the script avoided checking that module for
dependencies.

Lastly, it improves the output, it errors out if a module is found
on more than one package, which is what usually happens when
python upstream introduces a new file, hence why the current
manifest is unaware of, it is better to exit with an error because
otherwise the user wouldnt know if anything went wrong unless the output
of the script was checked manually.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 .../python/python3/create_manifest3.py | 432 -
 1 file changed, 242 insertions(+), 190 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/create_manifest3.py 
b/meta/recipes-devtools/python/python3/create_manifest3.py
index 41a6bb0..1849152 100644
--- a/meta/recipes-devtools/python/python3/create_manifest3.py
+++ b/meta/recipes-devtools/python/python3/create_manifest3.py
@@ -48,21 +48,21 @@ import os
 pyversion = str(sys.argv[1])
 
 # Hack to get native python search path (for folders), not fond of it but it 
works for now
-pivot='recipe-sysroot-native'
+pivot = 'recipe-sysroot-native'
 for p in sys.path:
   if pivot in p:
-nativelibfolder=p[:p.find(pivot)+len(pivot)]
+nativelibfolder = p[:p.find(pivot)+len(pivot)]
 
 # Empty dict to hold the whole manifest
 new_manifest = {}
 
 # Check for repeated files, folders and wildcards
-allfiles=[]
-repeated=[]
-wildcards=[]
+allfiles = []
+repeated = []
+wildcards = []
 
-hasfolders=[]
-allfolders=[]
+hasfolders = []
+allfolders = []
 
 def isFolder(value):
   value = value.replace('${PYTHON_MAJMIN}',pyversion)
@@ -79,166 +79,204 @@ def isCached(item):
 
 # Read existing JSON manifest
 with open('python3-manifest.json') as manifest:
-  old_manifest=json.load(manifest)
-
+  old_manifest = json.load(manifest)
 
+#
 # First pass to get core-package functionality, because we base everything on 
the fact that core is actually working
 # Not exactly the same so it should not be a function
+#
+
 print ('Getting dependencies for package: core')
 
-# Special call to check for core package
+
+# This special call gets the core dependencies and
+# appends to the old manifest so it doesnt hurt what it
+# currently holds.
+# This way when other packages check for dependencies
+# on the new core package, they will still find them
+# even when checking the old_manifest
+
 output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 
'python-core-package']).decode('utf8')
-for item in output.split():
-item = item.replace(pyversion,'${PYTHON_MAJMIN}')
-# We append it so it doesnt hurt what we currently have:
-if isCached(item):
-if item not in old_manifest['core']['cached']:
-# We use the same data structure since its the one which will be 
used to check
-# dependencies for other packages
-old_manifest['core']['cached'].append(item)
+for coredep in output.split():
+coredep = coredep.replace(pyversion,'${PYTHON_MAJMIN}')
+if isCached(coredep):
+if coredep not in old_manifest['core']['cached']:
+old_manifest['core']['cached'].append(coredep)
 else:
-if item not in old_manifest['core']['files']:
-# We use the same data structure since its the one which will be 
used to check
-# dependencies for other packages
-old_manifest['core']['files'].append(item)
-
-for value in old_manifest['core']['files']:
-  value = value.replace(pyversion,'${PYTHON_MAJMIN}')
-  # Ignore folders, since we don't import those, difficult to handle multilib
-  if isFolder(value):
-# Pass it directly
-if isCached(value):
-if value not in old_manifest['core']['cached']:
-old_manifest['core']['cached'].append(value)
+if coredep not in old_manifest['core']['files']:
+old_manifest['core']['files'].append(coredep)
+
+
+# The second step is to loop through the existing files contained in the core 
package
+# according to the old manifest, identify if they are  modules, or some other 
type 
+# of file that we cant import (directories, binaries, configs) in which case we
+# can only assume they were added correctly (manually) so we ignore those and 
+# pass them to the manifest directly.
+
+for filedep

[OE-core] [PATCH 1/4] python3-manifest: Use PYTHON_MAJMIN instead of hard coded version

[Alejandro Enedino Hernandez Samaniego]

Currently the manifest contains hard coded paths with the version number,
e.g. python3.5, this patch changes the paths to use the variable
PYTHON_MAJMIN instead, this should make the python upgrades easier

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 .../python/python3/python3-manifest.json   | 804 ++---
 1 file changed, 402 insertions(+), 402 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index 3dda9f6..e332078 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -3,7 +3,7 @@
 "cached": [],
 "files": [
 "${bindir}/2to3-*",
-"${libdir}/python3.5/lib2to3"
+"${libdir}/python${PYTHON_MAJMIN}/lib2to3"
 ],
 "rdepends": [
 "core"
@@ -13,9 +13,9 @@
 "asyncio": {
 "cached": [],
 "files": [
-"${libdir}/python3.5/asyncio",
-"${libdir}/python3.5/concurrent",
-"${libdir}/python3.5/concurrent/futures"
+"${libdir}/python${PYTHON_MAJMIN}/asyncio",
+"${libdir}/python${PYTHON_MAJMIN}/concurrent",
+"${libdir}/python${PYTHON_MAJMIN}/concurrent/futures"
 ],
 "rdepends": [
 "compression",
@@ -36,18 +36,18 @@
 },
 "audio": {
 "cached": [
-"${libdir}/python3.5/__pycache__/chunk.*.pyc",
-"${libdir}/python3.5/__pycache__/sndhdr.*.pyc",
-"${libdir}/python3.5/__pycache__/sunau.*.pyc",
-"${libdir}/python3.5/__pycache__/wave.*.pyc"
+"${libdir}/python${PYTHON_MAJMIN}/__pycache__/chunk.*.pyc",
+"${libdir}/python${PYTHON_MAJMIN}/__pycache__/sndhdr.*.pyc",
+"${libdir}/python${PYTHON_MAJMIN}/__pycache__/sunau.*.pyc",
+"${libdir}/python${PYTHON_MAJMIN}/__pycache__/wave.*.pyc"
 ],
 "files": [
-"${libdir}/python3.5/chunk.py",
-"${libdir}/python3.5/lib-dynload/audioop.*.so",
-"${libdir}/python3.5/lib-dynload/ossaudiodev.*.so",
-"${libdir}/python3.5/sndhdr.py",
-"${libdir}/python3.5/sunau.py",
-"${libdir}/python3.5/wave.py"
+"${libdir}/python${PYTHON_MAJMIN}/chunk.py",
+"${libdir}/python${PYTHON_MAJMIN}/lib-dynload/audioop.*.so",
+"${libdir}/python${PYTHON_MAJMIN}/lib-dynload/ossaudiodev.*.so",
+"${libdir}/python${PYTHON_MAJMIN}/sndhdr.py",
+"${libdir}/python${PYTHON_MAJMIN}/sunau.py",
+"${libdir}/python${PYTHON_MAJMIN}/wave.py"
 ],
 "rdepends": [
 "core"
@@ -56,11 +56,11 @@
 },
 "codecs": {
 "cached": [
-"${libdir}/python3.5/__pycache__/xdrlib.*.pyc"
+"${libdir}/python${PYTHON_MAJMIN}/__pycache__/xdrlib.*.pyc"
 ],
 "files": [
-"${libdir}/python3.5/lib-dynload/_multibytecodec.*.so",
-"${libdir}/python3.5/xdrlib.py"
+
"${libdir}/python${PYTHON_MAJMIN}/lib-dynload/_multibytecodec.*.so",
+"${libdir}/python${PYTHON_MAJMIN}/xdrlib.py"
 ],
 "rdepends": [
 "core"
@@ -69,12 +69,12 @@
 },
 "compile": {
 "cached": [
-"${libdir}/python3.5/__pycache__/compileall.*.pyc",
-"${libdir}/python3.5/__pycache__/py_compile.*.pyc"
+"${libdir}/python${PYTHON_MAJMIN}/__pycache__/compileall.*.pyc",
+"${libdir}/python${PYTHON_MAJMIN}/__pycache__/py_compile.*.pyc"
 ],
 "files": [
-"${libdir}/python3.5/compileall.py",
-"${libdir}/python3.5/py_compile.py"
+"${libdir}/python${PYTHON_MAJMIN}/compileall.py",
+"${libdir}/python${PYTHON_MAJMIN}/py_compile.py"
 ],
 "rdepends": [
 "asyncio",
@@ -95,23 +95,23 @@
 },
 "compression": {
 "cached": [
-"${libdir}/python3.5/__pycache__/_compression.*.pyc",
-"${libdir}/python3.5/__pycache__/bz2.*.pyc",
-"${libdir}/python3.5/__pycache__/gzip.*.pyc",
-"${libdir}/p

[OE-core] [PATCH 2/4] python3: Modify create_manifest to make it versionless

[Alejandro Enedino Hernandez Samaniego]

This patch improves the create_manifest script by making it
use PYTHON_MAJMIN instead of hard coded paths containing the
version number when looking at the necessary modules for
every package, the script should now be independent of the
python(3) version on which were working

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/recipes-devtools/python/python3/create_manifest3.py | 10 +-
 meta/recipes-devtools/python/python3_3.5.5.bb|  2 +-
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/create_manifest3.py 
b/meta/recipes-devtools/python/python3/create_manifest3.py
index 2f944f9..41a6bb0 100644
--- a/meta/recipes-devtools/python/python3/create_manifest3.py
+++ b/meta/recipes-devtools/python/python3/create_manifest3.py
@@ -44,6 +44,9 @@ import subprocess
 import json
 import os
 
+# Get python version from ${PYTHON_MAJMIN}
+pyversion = str(sys.argv[1])
+
 # Hack to get native python search path (for folders), not fond of it but it 
works for now
 pivot='recipe-sysroot-native'
 for p in sys.path:
@@ -62,6 +65,7 @@ hasfolders=[]
 allfolders=[]
 
 def isFolder(value):
+  value = value.replace('${PYTHON_MAJMIN}',pyversion)
   if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or 
os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or 
os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')):
 return True
   else:
@@ -85,6 +89,7 @@ print ('Getting dependencies for package: core')
 # Special call to check for core package
 output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 
'python-core-package']).decode('utf8')
 for item in output.split():
+item = item.replace(pyversion,'${PYTHON_MAJMIN}')
 # We append it so it doesnt hurt what we currently have:
 if isCached(item):
 if item not in old_manifest['core']['cached']:
@@ -98,6 +103,7 @@ for item in output.split():
 old_manifest['core']['files'].append(item)
 
 for value in old_manifest['core']['files']:
+  value = value.replace(pyversion,'${PYTHON_MAJMIN}')
   # Ignore folders, since we don't import those, difficult to handle multilib
   if isFolder(value):
 # Pass it directly
@@ -131,6 +137,8 @@ for value in old_manifest['core']['files']:
   print ('The following dependencies were found for module %s:\n' % value)
   print (output)
   for item in output.split():
+item = item.replace(pyversion,'${PYTHON_MAJMIN}')
+
 # We append it so it doesnt hurt what we currently have:
 if isCached(item):
 if item not in old_manifest['core']['cached']:
@@ -250,6 +258,7 @@ for key in old_manifest:
 #   is folder_string inside path/folder1/folder2/filename?, 
 #   Yes, it works, but we waste a couple of milliseconds.
 
+item = item.replace(pyversion,'${PYTHON_MAJMIN}')
 inFolders=False
 for folder in allfolders:
 if folder in item:
@@ -265,7 +274,6 @@ for key in old_manifest:
 folderFound = True
 if keyfolder not in 
new_manifest[key]['rdepends'] and keyfolder != key:
 
new_manifest[key]['rdepends'].append(keyfolder)
-
 else:
 break
 
diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb 
b/meta/recipes-devtools/python/python3_3.5.5.bb
index bfe91fb..8223160 100644
--- a/meta/recipes-devtools/python/python3_3.5.5.bb
+++ b/meta/recipes-devtools/python/python3_3.5.5.bb
@@ -334,7 +334,7 @@ do_create_manifest() {
 
 cd ${WORKDIR}
 # This needs to be executed by python-native and NOT by HOST's python
-nativepython3 create_manifest3.py
+nativepython3 create_manifest3.py ${PYTHON_MAJMIN}
 cp python3-manifest.json.new ${THISDIR}/python3/python3-manifest.json
 }
 
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 4/4] python3-manifest: Avoid duplicate modules

[Alejandro Enedino Hernandez Samaniego]

There was a bug where modules were being added twice to the
core package, this patches the manifest to reflect the fix on
the create_manifest script.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/recipes-devtools/python/python3/python3-manifest.json | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index e332078..90a41fb 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -197,10 +197,6 @@
 "files": [
 "${bindir}/python*[!-config]",
 "${includedir}/python${PYTHON_BINABI}/pyconfig*.h",
-"${libdir}/python${PYTHON_MAJMIN}/_collections_abc.py",
-"${libdir}/python${PYTHON_MAJMIN}/_sitebuiltins.py",
-"${libdir}/python${PYTHON_MAJMIN}/collections",
-"${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py",
 "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
 "${libdir}/python${PYTHON_MAJMIN}/UserList.py",
 "${libdir}/python${PYTHON_MAJMIN}/UserString.py",
@@ -281,6 +277,7 @@
 "${libdir}/python${PYTHON_MAJMIN}/selectors.py",
 "${libdir}/python${PYTHON_MAJMIN}/signal.py",
 "${libdir}/python${PYTHON_MAJMIN}/site.py",
+"${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py",
 "${libdir}/python${PYTHON_MAJMIN}/sre_compile.py",
 "${libdir}/python${PYTHON_MAJMIN}/sre_constants.py",
 "${libdir}/python${PYTHON_MAJMIN}/sre_parse.py",
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] terminal.bbclass: use var-SHELL as the shebang of wrapper script

[Alejandro Enedino Hernandez Samaniego]

Hey Hongxu,


On 09/02/2018 10:52 PM, Hongxu Jia wrote:

On 2018年09月01日 08:14, Alejandro Enedino Hernandez Samaniego wrote:

Hey Hongxu,


On 08/24/2018 01:54 AM, Hongxu Jia wrote:
The devshell.bbclass set var-SHELL to var-DEVSHELL, and 
terminal.bbclass

initial var-SHELL with `bash'. Keep sync with it, use var-SHELL rather
than hardcoded `/bin/sh' as the shebang of wrapper script.

On Ubuntu host, default shell is dash (/bin/sh -> dash), even though
we assign var-SHELL with `/bin/bash', the wrapper script is still 
dashism.


Signed-off-by: Hongxu Jia 
---
  meta/classes/terminal.bbclass | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/terminal.bbclass 
b/meta/classes/terminal.bbclass

index a27e10c..73e765d 100644
--- a/meta/classes/terminal.bbclass
+++ b/meta/classes/terminal.bbclass
@@ -25,7 +25,8 @@ def emit_terminal_func(command, envdata, d):
  bb.utils.mkdirhier(os.path.dirname(runfile))
with open(runfile, 'w') as script:
-script.write('#!/bin/sh -e\n')
+script.write('#!/usr/bin/env %s\n' % d.getVar('SHELL'))
+script.write('set -e\n')
This is causing a bug on several systems, I believe that it is 
because the way our systems are set up,
basically, our running shell isn't necessarily $SHELL, so it is 
causing the process to exit immediately,

anything thats using this class is affected, devshell, menuconfig, etc.


$ echo $SHELL
/bin/csh

$ echo $0
/bin/bash



The csh does not support `export', in you case, you should correct 
var-SHELL to align with $0,


Such as:
$ usermod  -s $0


I absolutely agree, but as I was saying, because the way the system is 
set up, it does not allow me to do so.


Thanks!

Alejandro




But the script should be more robust, it should
work well in this situation. I will try to fix it.

//Hongxu




Sadly we are unable to set $SHELL correctly.

I wonder if there is another way of doing this that wouldn't cause 
this behavior.


Cheers,

Alejandro



  bb.data.emit_func(cmd_func, script, envdata)
  script.write(cmd_func)
  script.write("\n")






--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] terminal.bbclass: use var-SHELL as the shebang of wrapper script

[Alejandro Enedino Hernandez Samaniego]

Hey Hongxu,


On 08/24/2018 01:54 AM, Hongxu Jia wrote:

The devshell.bbclass set var-SHELL to var-DEVSHELL, and terminal.bbclass
initial var-SHELL with `bash'. Keep sync with it, use var-SHELL rather
than hardcoded `/bin/sh' as the shebang of wrapper script.

On Ubuntu host, default shell is dash (/bin/sh -> dash), even though
we assign var-SHELL with `/bin/bash', the wrapper script is still dashism.

Signed-off-by: Hongxu Jia 
---
  meta/classes/terminal.bbclass | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/terminal.bbclass b/meta/classes/terminal.bbclass
index a27e10c..73e765d 100644
--- a/meta/classes/terminal.bbclass
+++ b/meta/classes/terminal.bbclass
@@ -25,7 +25,8 @@ def emit_terminal_func(command, envdata, d):
  bb.utils.mkdirhier(os.path.dirname(runfile))
  
  with open(runfile, 'w') as script:

-script.write('#!/bin/sh -e\n')
+script.write('#!/usr/bin/env %s\n' % d.getVar('SHELL'))
+script.write('set -e\n')
This is causing a bug on several systems, I believe that it is because 
the way our systems are set up,
basically, our running shell isn't necessarily $SHELL, so it is causing 
the process to exit immediately,

anything thats using this class is affected, devshell, menuconfig, etc.


$ echo $SHELL
/bin/csh

$ echo $0
/bin/bash



Sadly we are unable to set $SHELL correctly.

I wonder if there is another way of doing this that wouldn't cause this 
behavior.


Cheers,

Alejandro



  bb.data.emit_func(cmd_func, script, envdata)
  script.write(cmd_func)
  script.write("\n")


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] python3{,-native}: update to 3.7.0

[Alejandro Enedino Hernandez Samaniego]

Hey Jens,

First of all good job!

On 08/31/2018 08:47 AM, Jens Rehsack wrote:

Update python3 to recent 3.7.0 release.

This is the first state of the package which compiles and
packages, further tests on target and in sysroot-native have
to be done before.

Please review, give feedback etc.
Can you please add a summary of what was modified on the commit message 
please?, rebased patches, upstreamed patches, deleted patches (and 
Why?), new patches, etc.


The native build says its missing uuid:
The necessary bits to build these optional modules were not found:
_uuid

perhaps there's a missing DEPENDS to whoever provides uuid.h... 
util-linux-native? or its not looking for the required h file in the 
correct location, check setup.py


Its really important that our python-native build is complete, since its 
the base on what the manifest is created, which btw I believe you forgot 
to run bitbake python3 -c create_manifest?
It would've produced a new manifest for this release, but I don't see 
any changes to it so I'm just assuming you didn't run it.


Also,
python3 (target) failed to build for me on qemuarm (do_install):
sed: can't read 
/poky/builds/qemuarm/tmp/work/armv5e-poky-linux-gnueabi/python3/3.7.0-r1.0/image//usr/lib/python3.7/_sysconfigdata_m_linux-gnueabi_arm-linux-gnueabi.py: 
No such file or directory



Cheers,

Alejandro



Signed-off-by: Jens Rehsack 
---
  meta/classes/python3-dir.bbclass  |   6 +-
  .../python/python3-native_3.5.5.bb| 112 --
  .../python/python3-native_3.7.0.bb|  74 
  meta/recipes-devtools/python/python3.inc  |  74 
  ...hell-version-of-python-config-that-w.patch |  21 +-
  ..._sysconfigdata.py-to-initialize-dist.patch |  66 
  .../python3/0001-cross-compile-support.patch  |  66 ++--
  ...d-target-to-split-profile-generation.patch |  40 --
  .../python3/03-fix-tkinter-detection.patch|  12 +-
  .../python3/030-fixup-include-dirs.patch  |   9 -
  .../080-distutils-dont_adjust_files.patch |   4 +-
  .../python/python3/150-fix-setupterm.patch|  17 -
  ...GS-for-extensions-when-cross-compili.patch |  53 ++-
  .../python3/avoid-ncursesw-include-path.patch |  18 +-
  .../python3/avoid_warning_about_tkinter.patch |  18 +-
  .../python3/configure.ac-fix-LIBPL.patch  |  21 +-
  ...ssing-libraries-to-Extension-for-mul.patch |  26 +-
  .../python/python3/python-3.3-multilib.patch  | 241 ++--
  ...CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch |  17 +-
  .../python/python3/regen-all.patch|  25 --
  ...port_SOURCE_DATE_EPOCH_in_py_compile.patch |  36 +-
  .../python3/sysroot-include-headers.patch |  23 +-
  meta/recipes-devtools/python/python3_3.5.5.bb | 346 --
  meta/recipes-devtools/python/python3_3.7.0.bb | 309 
  24 files changed, 747 insertions(+), 887 deletions(-)
  delete mode 100644 meta/recipes-devtools/python/python3-native_3.5.5.bb
  create mode 100644 meta/recipes-devtools/python/python3-native_3.7.0.bb
  create mode 100644 meta/recipes-devtools/python/python3.inc
  delete mode 100644 
meta/recipes-devtools/python/python3/0001-Issue-21272-Use-_sysconfigdata.py-to-initialize-dist.patch
  delete mode 100644 
meta/recipes-devtools/python/python3/0002-Makefile-add-target-to-split-profile-generation.patch
  delete mode 100644 
meta/recipes-devtools/python/python3/150-fix-setupterm.patch
  delete mode 100644 meta/recipes-devtools/python/python3/regen-all.patch
  delete mode 100644 meta/recipes-devtools/python/python3_3.5.5.bb
  create mode 100644 meta/recipes-devtools/python/python3_3.7.0.bb

diff --git a/meta/classes/python3-dir.bbclass b/meta/classes/python3-dir.bbclass
index 06bb046d9c..26d2b6668b 100644
--- a/meta/classes/python3-dir.bbclass
+++ b/meta/classes/python3-dir.bbclass
@@ -1,4 +1,8 @@
-PYTHON_BASEVERSION = "3.5"
+PYTHON_BASEVERSION = "3.7"
+# [d][m]
+# d: py_debug
+# m: my_malloc
+# u: wide-char unicode
  PYTHON_ABI = "m"
  PYTHON_DIR = "python${PYTHON_BASEVERSION}"
  PYTHON_PN = "python3"
diff --git a/meta/recipes-devtools/python/python3-native_3.5.5.bb 
b/meta/recipes-devtools/python/python3-native_3.5.5.bb
deleted file mode 100644
index 3098027d80..00
--- a/meta/recipes-devtools/python/python3-native_3.5.5.bb
+++ /dev/null
@@ -1,112 +0,0 @@
-require recipes-devtools/python/python.inc
-
-PR = "${INC_PR}.0"
-PYTHON_MAJMIN = "3.5"
-DISTRO_SRC_URI ?= "file://sitecustomize.py"
-DISTRO_SRC_URI_linuxstdbase = ""
-SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
-file://12-distutils-prefix-is-inside-staging-area.patch \
-file://python-config.patch \
-file://0001-cross-compile-support.patch \
-file://030-fixup-include-dirs.patch \
-file://070-dont-clean-ipkg-install.patch \
-file://080-distutils-dont_adjust_files.patch \
-file://130-readline-setup.patch \
-file://150-fix-setupterm.patch \
-file://python-3.3-multilib.patch \
-file://03-fix-tkinter-detection.patch \
-file://avoid_warning_about_tkinter.patch \

Re: [OE-core] Multiconfig builds broken in master

[Alejandro Enedino Hernandez Samaniego]

Hello Alex,

Firs of all, your issue is not a multiconfig problem, and is not caused 
by commit b8730f6 .



I was able to reproduce your problem before and after reverting the 
specified commit.



As your error log says, ostree is not in the list of valid image 
features, if you add debug flags to bitbake


you can see that the ostree-image.bbclass file is not even being parsed, 
hence why ostree was never added


as a valid image feature.


Why was it never parsed?

because theres no INHERITs/inherits pointing to it.

greping though the iot-refkit repo we can see that for example the 
refkit-image (I don't know what your production image is based on I'm 
just assuming) contains the following:


meta-refkit-core/classes/refkit-image.bbclass: inherit 
${@oe.utils.all_distro_features(d, 'ostree', 'ostree-image')}


So it will only be inherited if the DISTRO_FEATURE ostree is included.

Adding ostree to DISTRO_FEATURES on the .conf, allows the 
build to find the ostree IMAGE_FEATURE correctly.



Alejandro




On 08/29/2018 06:39 AM, Alex Kiernan wrote:

I've run into a weird multiconfig problem on master which seems to
have been introduced with

b8730f6 ("bitbake: bitbake: Add support for multiconfig dependencies")

Before that, I could happily use:

bitbake multiconfig:gplv2:production-image

whereas now it fails (gplv2 is a multiconfig which basically just sets
TMPDIR and INCOMPATIBLE_LICENSES).

The error suggests failure to populate IMAGE_FEATURES correctly:

ERROR: Nothing PROVIDES 'production-image'
production-image was skipped: 'ostree' in IMAGE_FEATURES is not a
valid image feature. Valid features: allow-empty-password
allow-root-login autologin dbg-pkgs debug-tweaks dev-pkgs doc doc-pkgs
eclipse-debug empty-root-password hwcodecs muted nfs-client nfs-server
package-management post-install-logging ptest-pkgs read-only-rootfs
splash ssh-server-dropbear ssh-server-openssh staticdev-pkgs
tools-debug tools-profile tools-sdk tools-testapps usb-nmc x11
x11-base x11-sato

(the ostree feature is introduced by a layer derived from the Intel
IoT refkit - 
https://github.com/intel/intel-iot-refkit/blob/master/meta-refkit-core/classes/ostree-image.bbclass)

If I revert that single commit everything works again.

Clearly there's nothing in that changeset which is touching
IMAGE_FEATURES, so it's something way more subtle!



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] adding python modules

[Alejandro Enedino Hernandez Samaniego]

Hey Bhaskar, Alex


On 08/14/2018 01:18 AM, Alexander Kanavin wrote:

Simple - add python3-modules to the packagegroup. Sadly this seems to
be 'common knowledge' that is poorly documented.

Alex
I'm pretty sure that the python3-modules package is included by default 
when python is installed (on master)

if you wont be using master then yes, install python3-modules


2018-08-14 0:51 GMT+02:00 bhas_purk :

Hello,
I am new to yocto, and we are using this for an Intel SOC.
I am creating a console-image, and wanted to get python3 in the rootfs.

So I added python3 in the recipe file -
layers/meta-openembedded/meta-oe/recipes-core/packagegroups/packagegroup-basic.bb
This gave me python3 in the rootfs and I can run ptyhon3. So that is great!


Now I would like to add some modules to python such that they are built in,
and I dont have to do a pip install for the modules.

I see that there is a script file which generates the manifest for all
modules will be there.
openembedded-core/scripts/contrib/python/generate-manifest-3.5.py


I would like to edit this file to some python modules, and am wondering how.

For example, one of the existing modules in the script is defined like this-
   m.addPackage( "${PN}-argparse", "Python command line argument parser",
"${PN}-core ${PN}-codecs ${PN}-textutils",
 "argparse.*" )

The generate-manifest-3.5 script is no longer used (again, on master),
there is a new one called create-manifest where you can add more granularity
to the python build, but in this case I don't believe you need it.

Alejandro



So my question is, for the modules I would like to add, how do I know the
dependencies etc? And more importantly, is this the right way to add modules
to python3 in the rootfs.


Thank you
Bhaskar





--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

[Alejandro Enedino Hernandez Samaniego]

On 08/02/2018 01:53 AM, Richard Purdie wrote:

On Thu, 2018-08-02 at 10:55 +0800, Anuj Mittal wrote:

On 08/02/2018 07:30 AM, Alejandro Enedino Hernandez Samaniego wrote:

Hey Anuj,


On 08/01/2018 04:25 AM, Anuj Mittal wrote:

On 07/31/2018 05:21 AM, Jaewon Lee wrote:

When using a recipe space kernel-meta, scc files are added
through
SRC_URI, but they may include corresponding kernel fragments
that are
not necessarily in SRC_URI.

For bitbake, this is not a problem because the kernel-yocto
class adds
the path where the .scc file was found to includes which
consequentially
makes the .cfg file available to the kernel build.

However, when using devtool, only files specified in SRC_URI
are copied
to oe-local-files in devtool's workspace. So if the cfg file is
not in
SRC_URI, it won't be copied, causing a kernel build failure
when trying
to find it.

This fix parses local .scc files in SRC_URI, copies the
corresponding .cfg
file to devtool's workdir, and also adds it to local_files so
it is
available when doing a devtool build for the kernel.

[YOCTO #12858]

Signed-off-by: Jaewon Lee 
Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
   meta/classes/devtool-source.bbclass | 12 
   1 file changed, 12 insertions(+)

diff --git a/meta/classes/devtool-source.bbclass
b/meta/classes/devtool-source.bbclass
index 56882a4..c70fea2 100644
--- a/meta/classes/devtool-source.bbclass
+++ b/meta/classes/devtool-source.bbclass
@@ -90,11 +90,23 @@ python devtool_post_unpack() {
   fname in files])
   return ret
   
+is_kernel_yocto = bb.data.inherits_class('kernel-yocto',

d)
   # Move local source files into separate subdir
   recipe_patches = [os.path.basename(patch) for patch in
   oe.recipeutils.get_recipe_patches(d)]
   local_files = oe.recipeutils.get_recipe_local_files(d)
   
+if is_kernel_yocto:

+  for key in local_files.copy():
+if key.endswith('scc'):
+  sccfile = open(local_files[key], 'r')
+  for l in sccfile:
+line = l.split()
+if line and line[0] == 'kconf' and line[-
1].endswith('.cfg'):
+  local_files[line[-1]] =
os.path.join(os.path.dirname(local_files[key]), line[-1])
+  shutil.copy2(os.path.join(os.path.dirname(local_
files[key]), line[-1]), workdir)
+  sccfile.close()
+

Would the patches included in these .scc files also need to be
handled
in the same way? Would this also work if there are other scc
files
included in a scc file?

Yes, I believe that the same mechanism should be used for patch
files as
well,
basically anything that may be needed to build but that its not
necessarily
explicitly listed on SRC_URI.

I believe it will work for other scc files and it doesnt have to
be
recursive,
because while cfg files arent required to be in SRC_URI , scc files
ARE
required
to be SRC_URI, which means that this will eventually find the other
scc
file on the list

I don't think they are required to be specified except for the top
level
one. At least, when I test it, I see problems. kernel-tools spp
script
parses them recursively and looks for a nested scc even if it is not
specified as part of SRC_URI. That is how the top level sccs from
kernel-cache are parsed too. Can you give it a try please?

Hey Anuj,
We can give it a try but do you have a specific example of how it fails 
for you?
Because I know theres lots of nested sccs on the yocto kernel cache, but 
in that case
it wouldn't be a problem since this bug is specifically introduced by 
devtool when it
copies local files from SRC_URI to a oe-local-files directory, it misses 
the corresponding cfg files (or patch files)
since they're not listed in SRC_URI and it fails to build, in the case 
of the yocto kernel cache, it
does not contain local files, so they wont be moved hence why it 
wouldn't be an issue.


From the kernel dev manual:
https://www.yoctoproject.org/docs/latest/kernel-dev/kernel-dev.html#recipe-space-metadata 
:
It is only necessary to specify the|.scc|files on the|SRC_URI| 
<http://www.yoctoproject.org/docs/2.5/ref-manual/ref-manual.html#var-SRC_URI>.


It specifies that only the scc files need to be on SRC_URI, which is why 
I'm saying the script

will eventually run into them from the list.

I want to be clear that I'm not against doing this recursively, I just 
want to understand your testcase better.

What would be really great here would be some test cases in the qa
framework! It'd be a good way to both illustrate the problems and then
test we've fixed it and that it stays fixed.

Hey Richard,
Sure that sounds like a good idea, well work on adding a testcase (or 
adding to the existing one) for devtool when

it is using a recipe-space kernel -meta.

Alejandro


Cheers,

Richard



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/li

Re: [OE-core] [PATCH] devtool-source.bbclass: Support kernel fragments not in SRC_URI

[Alejandro Enedino Hernandez Samaniego]

Hey Anuj,


On 08/01/2018 04:25 AM, Anuj Mittal wrote:

On 07/31/2018 05:21 AM, Jaewon Lee wrote:

When using a recipe space kernel-meta, scc files are added through
SRC_URI, but they may include corresponding kernel fragments that are
not necessarily in SRC_URI.

For bitbake, this is not a problem because the kernel-yocto class adds
the path where the .scc file was found to includes which consequentially
makes the .cfg file available to the kernel build.

However, when using devtool, only files specified in SRC_URI are copied
to oe-local-files in devtool's workspace. So if the cfg file is not in
SRC_URI, it won't be copied, causing a kernel build failure when trying
to find it.

This fix parses local .scc files in SRC_URI, copies the corresponding .cfg
file to devtool's workdir, and also adds it to local_files so it is
available when doing a devtool build for the kernel.

[YOCTO #12858]

Signed-off-by: Jaewon Lee 
Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
  meta/classes/devtool-source.bbclass | 12 
  1 file changed, 12 insertions(+)

diff --git a/meta/classes/devtool-source.bbclass 
b/meta/classes/devtool-source.bbclass
index 56882a4..c70fea2 100644
--- a/meta/classes/devtool-source.bbclass
+++ b/meta/classes/devtool-source.bbclass
@@ -90,11 +90,23 @@ python devtool_post_unpack() {
  fname in files])
  return ret
  
+is_kernel_yocto = bb.data.inherits_class('kernel-yocto', d)

  # Move local source files into separate subdir
  recipe_patches = [os.path.basename(patch) for patch in
  oe.recipeutils.get_recipe_patches(d)]
  local_files = oe.recipeutils.get_recipe_local_files(d)
  
+if is_kernel_yocto:

+  for key in local_files.copy():
+if key.endswith('scc'):
+  sccfile = open(local_files[key], 'r')
+  for l in sccfile:
+line = l.split()
+if line and line[0] == 'kconf' and line[-1].endswith('.cfg'):
+  local_files[line[-1]] = 
os.path.join(os.path.dirname(local_files[key]), line[-1])
+  shutil.copy2(os.path.join(os.path.dirname(local_files[key]), 
line[-1]), workdir)
+  sccfile.close()
+

Would the patches included in these .scc files also need to be handled
in the same way? Would this also work if there are other scc files
included in a scc file?
Yes, I believe that the same mechanism should be used for patch files as 
well,

basically anything that may be needed to build but that its not necessarily
explicitly listed on SRC_URI.

I believe it will work for other scc files and it doesnt have to be 
recursive,
because while cfg files arent required to be in SRC_URI , scc files ARE 
required
to be SRC_URI, which means that this will eventually find the other scc 
file on the list


Cheers,

Alejandro



Thanks,

Anuj


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] multiconfig: Enable multiconfig dependencies on oe-core

[Alejandro Enedino Hernandez Samaniego]

Hey Khem, Richard,


On 08/01/2018 10:44 AM, Khem Raj wrote:



On Tue, Jul 31, 2018 at 4:59 AM Richard Purdie 
<mailto:richard.pur...@linuxfoundation.org>> wrote:


On Mon, 2018-07-30 at 15:05 -0700, Alejandro Enedino Hernandez
Samaniego wrote:
> On 07/26/2018 08:36 PM, Khem Raj wrote:
> >
> > On 7/25/18 9:05 AM, Alejandro Enedino Hernandez Samaniego wrote:
> > > This patch enables multiconfig dependencies (mcdepends) to be
> > > used on
> > > recipes using the following format:
> > >
> > > task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-
> > > on"
> > >
> > > For the sake of simplicity consider the following example:
> > >
> > > Assuming we have set up multiconfig builds, one for qemux86 and
> > > one for
> > > qemuarm, named x86 and arm respectively.
> > >
> > > Adding the following line to an image recipe (core-image-sato):
> > > do_image[mcdepends] = "multiconfig:x86:arm:core-image-
> > > minimal:do_rootfs"
> > >
> >
> > Do we really need to add multiconfig to keyword namespace ? can we
> > deduce this from "arch1:arch2:..."
>
> Bitbake already uses this and I believed adopting it would make it
> easier for the user to understand its usage.
>
> For example, since the multiconfig names are also provided during
> setup, if you assume that someone set up the build and a different
> user is going through the recipe, arch1:arch2 wouldn't mean a
lot for
> that user because they may not map 1:1 to MACHINE names.

It is consistent with the rest of the system. I'm still torn on
whether
we should shorten "multiconfig:" to "mc:" but we need to be
consistent.


Given a choice I would suggest for using multiconfig for lesser chance 
of conflicts

I also agree here that its better to use multiconfig.



> > secondy, this need to be well documented, with examples for users
>
> I agree we could add more documentation to it, where do you
think it
> should go?

I'm going to queue the patches for testing in -next, they look good to
me just reading the patches.

Could you start to work with Scott (cc'd) to get this new dependency
type listed in the bitbake manual and anywhere else it needs to be
documented?


Sure, I'll sync with Scott

Thanks!,

Alejandro



Its great to see this finally working!

Cheers,

Richard



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] multiconfig: Enable multiconfig dependencies on oe-core

[Alejandro Enedino Hernandez Samaniego]

Hey Khem,


On 07/26/2018 08:36 PM, Khem Raj wrote:


On 7/25/18 9:05 AM, Alejandro Enedino Hernandez Samaniego wrote:

This patch enables multiconfig dependencies (mcdepends) to be used on
recipes using the following format:

task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on"

For the sake of simplicity consider the following example:

Assuming we have set up multiconfig builds, one for qemux86 and one for
qemuarm, named x86 and arm respectively.

Adding the following line to an image recipe (core-image-sato):
do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs"


Do we really need to add multiconfig to keyword namespace ? can we
deduce this from "arch1:arch2:..."
Bitbake already uses this and I believed adopting it would make it 
easier for the user to understand its usage.


For example, since the multiconfig names are also provided during setup, 
if you assume that someone set up
the build and a different user is going through the recipe, arch1:arch2 
wouldn't mean a lot for that user

because they may not map 1:1 to MACHINE names.



secondy, this need to be well documented, with examples for users
I agree we could add more documentation to it, where do you think it 
should go?



Would state that core-image-sato:do_image from x86 will depend on
core-image-minimal:do_rootfs from arm so it can be executed.

This patch makes modifications to bitbake.conf to enable mcdepends, and
to sstatesig and staging.bbclass to avoid conflicts between packages from
different multiconfigs.

[YOCTO #10681]

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
  meta/classes/staging.bbclass | 8 +++-
  meta/conf/bitbake.conf   | 2 +-
  meta/lib/oe/sstatesig.py | 9 -
  3 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
index 41df883..27b012e 100644
--- a/meta/classes/staging.bbclass
+++ b/meta/classes/staging.bbclass
@@ -256,7 +256,7 @@ python extend_recipe_sysroot() {
  workdir = d.getVar("WORKDIR")
  #bb.warn(str(taskdepdata))
  pn = d.getVar("PN")
-
+mc = d.getVar("BB_CURRENT_MC")
  stagingdir = d.getVar("STAGING_DIR")
  sharedmanifests = d.getVar("COMPONENTS_DIR") + "/manifests"
  recipesysroot = d.getVar("RECIPE_SYSROOT")
@@ -443,7 +443,13 @@ python extend_recipe_sysroot() {
  
  msg_exists = []

  msg_adding = []
+
  for dep in configuredeps:
+if mc != 'default':
+# We should not care about other multiconfigs
+depmc = dep.split(':')[1]
+if depmc != mc:
+continue
  c = setscenedeps[dep][0]
  if c not in installed:
  continue
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index e28f3c7..f68954c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -877,7 +877,7 @@ BB_SIGNATURE_EXCLUDE_FLAGS ?= "doc deps depends \
  file-checksums python func task export unexport noexec nostamp dirs 
cleandirs \
  sstate-lockfile-shared prefuncs postfuncs export_func deptask rdeptask \
  recrdeptask nodeprrecs stamp-extra-info sstate-outputdirs filename lineno 
\
-progress"
+progress mcdepends"
  
  MLPREFIX ??= ""

  MULTILIB_VARIANTS ??= ""
diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index 5dcc2f5..18c5a35 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -150,16 +150,23 @@ class 
SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
  if recipename in self.unlockedrecipes:
  unlocked = True
  else:
+def get_mc(tid):
+tid = tid.rsplit('.', 1)[0]
+if tid.startswith('multiconfig:'):
+elems = tid.split(':')
+return elems[1]
  def recipename_from_dep(dep):
  # The dep entry will look something like
  # /path/path/recipename.bb.task, 
virtual:native:/p/foo.bb.task,
  # ...
+
  fn = dep.rsplit('.', 1)[0]
  return dataCache.pkg_fn[fn]
  
+mc = get_mc(fn)

  # If any unlocked recipe is in the direct dependencies then the
  # current recipe should be unlocked as well.
-depnames = [ recipename_from_dep(x) for x in deps ]
+depnames = [ recipename_from_dep(x) for x in deps if mc == 
get_mc(x)]
  if any(x in y for y in depnames for x in self.unlockedrecipes):
  self.unlockedrecipes[recipename] = ''
  unlocked = True


Cheers,

Alejandro
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] multiconfig: Enable multiconfig dependencies on oe-core

[Alejandro Enedino Hernandez Samaniego]

This patch enables multiconfig dependencies (mcdepends) to be used on
recipes using the following format:

task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on"

For the sake of simplicity consider the following example:

Assuming we have set up multiconfig builds, one for qemux86 and one for
qemuarm, named x86 and arm respectively.

Adding the following line to an image recipe (core-image-sato):
do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs"

Would state that core-image-sato:do_image from x86 will depend on
core-image-minimal:do_rootfs from arm so it can be executed.

This patch makes modifications to bitbake.conf to enable mcdepends, and
to sstatesig and staging.bbclass to avoid conflicts between packages from
different multiconfigs.

[YOCTO #10681]

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/classes/staging.bbclass | 8 +++-
 meta/conf/bitbake.conf   | 2 +-
 meta/lib/oe/sstatesig.py | 9 -
 3 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
index 41df883..27b012e 100644
--- a/meta/classes/staging.bbclass
+++ b/meta/classes/staging.bbclass
@@ -256,7 +256,7 @@ python extend_recipe_sysroot() {
 workdir = d.getVar("WORKDIR")
 #bb.warn(str(taskdepdata))
 pn = d.getVar("PN")
-
+mc = d.getVar("BB_CURRENT_MC")
 stagingdir = d.getVar("STAGING_DIR")
 sharedmanifests = d.getVar("COMPONENTS_DIR") + "/manifests"
 recipesysroot = d.getVar("RECIPE_SYSROOT")
@@ -443,7 +443,13 @@ python extend_recipe_sysroot() {
 
 msg_exists = []
 msg_adding = []
+
 for dep in configuredeps:
+if mc != 'default':
+# We should not care about other multiconfigs
+depmc = dep.split(':')[1]
+if depmc != mc:
+continue
 c = setscenedeps[dep][0]
 if c not in installed:
 continue
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index e28f3c7..f68954c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -877,7 +877,7 @@ BB_SIGNATURE_EXCLUDE_FLAGS ?= "doc deps depends \
 file-checksums python func task export unexport noexec nostamp dirs 
cleandirs \
 sstate-lockfile-shared prefuncs postfuncs export_func deptask rdeptask \
 recrdeptask nodeprrecs stamp-extra-info sstate-outputdirs filename lineno \
-progress"
+progress mcdepends"
 
 MLPREFIX ??= ""
 MULTILIB_VARIANTS ??= ""
diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index 5dcc2f5..18c5a35 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -150,16 +150,23 @@ class 
SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
 if recipename in self.unlockedrecipes:
 unlocked = True
 else:
+def get_mc(tid):
+tid = tid.rsplit('.', 1)[0]
+if tid.startswith('multiconfig:'):
+elems = tid.split(':')
+return elems[1]
 def recipename_from_dep(dep):
 # The dep entry will look something like
 # /path/path/recipename.bb.task, virtual:native:/p/foo.bb.task,
 # ...
+
 fn = dep.rsplit('.', 1)[0]
 return dataCache.pkg_fn[fn]
 
+mc = get_mc(fn)
 # If any unlocked recipe is in the direct dependencies then the
 # current recipe should be unlocked as well.
-depnames = [ recipename_from_dep(x) for x in deps ]
+depnames = [ recipename_from_dep(x) for x in deps if mc == 
get_mc(x)]
 if any(x in y for y in depnames for x in self.unlockedrecipes):
 self.unlockedrecipes[recipename] = ''
 unlocked = True
-- 
2.7.4

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] maintainers: Assign ownership of newlib and libgloss to Alejandro Hernandez

[Alejandro Enedino Hernandez Samaniego]

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/conf/distro/include/maintainers.inc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/conf/distro/include/maintainers.inc 
b/meta/conf/distro/include/maintainers.inc
index 7547aef..1d3e059 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -327,6 +327,7 @@ RECIPE_MAINTAINER_pn-libgcc = "Khem Raj 
"
 RECIPE_MAINTAINER_pn-libgcc-initial = "Khem Raj "
 RECIPE_MAINTAINER_pn-libgcrypt = "Hongxu Jia "
 RECIPE_MAINTAINER_pn-libgfortran = "Khem Raj "
+RECIPE_MAINTAINER_pn-libgloss = "Alejandro Hernandez "
 RECIPE_MAINTAINER_pn-libglu = "Maxin B. John "
 RECIPE_MAINTAINER_pn-libgpg-error = "Hongxu Jia "
 RECIPE_MAINTAINER_pn-libgudev = "Maxin B. John "
@@ -512,6 +513,7 @@ RECIPE_MAINTAINER_pn-nativesdk-buildtools-perl-dummy = 
"Richard Purdie http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2 3/3] tclibc-newlib: Adds a new TCLIBC variant to build with newlib as C library

[Alejandro Enedino Hernandez Samaniego]

I was unable to reply to the thread, this is the updated patch, the 
contrib branch aehs29/newilb has also been rebased to latest master.



Cheers,

Alejandro


On 05/29/2018 03:38 PM, Alejandro Enedino Hernandez Samaniego wrote:

This patch adds the posibility to build using TCLIBC=newlib.

It allows users to build baremetal applications with the use of a
C library.

Newlib is a lightweight C library meant to be used on embedded systems,
it is meant to be easily portable for new platforms and to provide
basic functionality on them, by design, it provides stubs for some of
these core functions declared as weak, so they can be built correctly
and then linked against some other library which provides specifics
about the platform being used if need be, libgloss takes care of these
in some cases, but it can also be extended, this patch also allows the
user to easily add other libraries to it by adding them to
NEWLIB_EXTENDED for this specific reason.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
  meta/classes/cross-canadian.bbclass |  2 +-
  meta/conf/distro/include/tclibc-newlib.inc  | 44 +
  meta/recipes-core/newlib/newlib_3.0.0.bb|  3 ++
  meta/recipes-devtools/gcc/libgcc-common.inc |  5 
  4 files changed, 53 insertions(+), 1 deletion(-)
  create mode 100644 meta/conf/distro/include/tclibc-newlib.inc

diff --git a/meta/classes/cross-canadian.bbclass 
b/meta/classes/cross-canadian.bbclass
index ee8aa61..acde331 100644
--- a/meta/classes/cross-canadian.bbclass
+++ b/meta/classes/cross-canadian.bbclass
@@ -30,7 +30,7 @@ python () {
  if d.getVar("MODIFYTOS") != "1":
  return
  
-if d.getVar("TCLIBC") == "baremetal":

+if d.getVar("TCLIBC") in [ 'baremetal', 'newlib' ]:
  return
  
  tos = d.getVar("TARGET_OS")

diff --git a/meta/conf/distro/include/tclibc-newlib.inc 
b/meta/conf/distro/include/tclibc-newlib.inc
new file mode 100644
index 000..dc631d8
--- /dev/null
+++ b/meta/conf/distro/include/tclibc-newlib.inc
@@ -0,0 +1,44 @@
+#
+# Newlib configuration
+#
+
+LIBCEXTENSION = "-newlib"
+LIBCOVERRIDE = ":libc-newlib"
+
+PREFERRED_PROVIDER_virtual/libc ?= "newlib"
+PREFERRED_PROVIDER_virtual/libiconv ?= "newlib"
+PREFERRED_PROVIDER_virtual/libintl ?= "newlib"
+PREFERRED_PROVIDER_virtual/nativesdk-libintl ?= "nativesdk-glibc"
+PREFERRED_PROVIDER_virtual/nativesdk-libiconv ?= "nativesdk-glibc"
+
+DISTRO_FEATURES_BACKFILL_CONSIDERED += "ldconfig"
+
+#USE_NLS ?= "no"
+
+IMAGE_LINGUAS = ""
+
+LIBC_DEPENDENCIES = "\
+newlib-dbg \
+newlib-dev \
+libgloss \
+libgloss-dev \
+libgloss-dbg \
+libgcc-dev \
+libgcc-dbg \
+"
+
+# compilerlibs defaults to gcc-runtime, but we get runtime from libgloss
+# we set ASSUME_PROVIDED because we cant set PREFERRED_PROVIDER
+# for compilerlibs since its overridden by tcmode-default
+ASSUME_PROVIDED += "virtual/${TARGET_PREFIX}compilerlibs virtual/crypt"
+
+# Its useful to be able to extend newlib, but we dont provide a native variant 
of libgloss
+NEWLIB_EXTENDED ?=  "libgloss libgcc"
+BASE_DEFAULT_DEPS_append_class-target = " ${NEWLIB_EXTENDED}"
+
+TARGET_OS = "elf"
+TARGET_OS_arm = "eabi"
+
+TOOLCHAIN_HOST_TASK ?= "packagegroup-cross-canadian-${MACHINE}"
+TOOLCHAIN_TARGET_TASK ?= "${LIBC_DEPENDENCIES}"
+TOOLCHAIN_NEED_CONFIGSITE_CACHE_remove = "zlib ncurses"
diff --git a/meta/recipes-core/newlib/newlib_3.0.0.bb 
b/meta/recipes-core/newlib/newlib_3.0.0.bb
index 3380576..09238fe 100644
--- a/meta/recipes-core/newlib/newlib_3.0.0.bb
+++ b/meta/recipes-core/newlib/newlib_3.0.0.bb
@@ -14,3 +14,6 @@ do_install_append() {
# Remove original directory
rmdir ${D}${prefix}/${TARGET_SYS}
  }
+
+# No rpm package is actually created but -dev depends on it, avoid dnf error
+RDEPENDS_${PN}-dev_libc-newlib = ""
diff --git a/meta/recipes-devtools/gcc/libgcc-common.inc 
b/meta/recipes-devtools/gcc/libgcc-common.inc
index 848a476..d959566 100644
--- a/meta/recipes-devtools/gcc/libgcc-common.inc
+++ b/meta/recipes-devtools/gcc/libgcc-common.inc
@@ -47,8 +47,13 @@ do_install () {
  do_install_append_libc-baremetal () {
rmdir ${D}${base_libdir}
  }
+do_install_append_libc-newlib () {
+   rmdir ${D}${base_libdir}
+}
  
+# No rpm package is actually created but -dev depends on it, avoid dnf error

  RDEPENDS_${PN}-dev_libc-baremetal = ""
+RDEPENDS_${PN}-dev_libc-newlib = ""
  
  BBCLASSEXTEND = "nativesdk"
  


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2 3/3] tclibc-newlib: Adds a new TCLIBC variant to build with newlib as C library

[Alejandro Enedino Hernandez Samaniego]

This patch adds the posibility to build using TCLIBC=newlib.

It allows users to build baremetal applications with the use of a
C library.

Newlib is a lightweight C library meant to be used on embedded systems,
it is meant to be easily portable for new platforms and to provide
basic functionality on them, by design, it provides stubs for some of
these core functions declared as weak, so they can be built correctly
and then linked against some other library which provides specifics
about the platform being used if need be, libgloss takes care of these
in some cases, but it can also be extended, this patch also allows the
user to easily add other libraries to it by adding them to
NEWLIB_EXTENDED for this specific reason.

Signed-off-by: Alejandro Enedino Hernandez Samaniego 
---
 meta/classes/cross-canadian.bbclass |  2 +-
 meta/conf/distro/include/tclibc-newlib.inc  | 44 +
 meta/recipes-core/newlib/newlib_3.0.0.bb|  3 ++
 meta/recipes-devtools/gcc/libgcc-common.inc |  5 
 4 files changed, 53 insertions(+), 1 deletion(-)
 create mode 100644 meta/conf/distro/include/tclibc-newlib.inc

diff --git a/meta/classes/cross-canadian.bbclass 
b/meta/classes/cross-canadian.bbclass
index ee8aa61..acde331 100644
--- a/meta/classes/cross-canadian.bbclass
+++ b/meta/classes/cross-canadian.bbclass
@@ -30,7 +30,7 @@ python () {
 if d.getVar("MODIFYTOS") != "1":
 return

-if d.getVar("TCLIBC") == "baremetal":
+if d.getVar("TCLIBC") in [ 'baremetal', 'newlib' ]:
 return

 tos = d.getVar("TARGET_OS")
diff --git a/meta/conf/distro/include/tclibc-newlib.inc 
b/meta/conf/distro/include/tclibc-newlib.inc
new file mode 100644
index 000..dc631d8
--- /dev/null
+++ b/meta/conf/distro/include/tclibc-newlib.inc
@@ -0,0 +1,44 @@
+#
+# Newlib configuration
+#
+
+LIBCEXTENSION = "-newlib"
+LIBCOVERRIDE = ":libc-newlib"
+
+PREFERRED_PROVIDER_virtual/libc ?= "newlib"
+PREFERRED_PROVIDER_virtual/libiconv ?= "newlib"
+PREFERRED_PROVIDER_virtual/libintl ?= "newlib"
+PREFERRED_PROVIDER_virtual/nativesdk-libintl ?= "nativesdk-glibc"
+PREFERRED_PROVIDER_virtual/nativesdk-libiconv ?= "nativesdk-glibc"
+
+DISTRO_FEATURES_BACKFILL_CONSIDERED += "ldconfig"
+
+#USE_NLS ?= "no"
+
+IMAGE_LINGUAS = ""
+
+LIBC_DEPENDENCIES = "\
+newlib-dbg \
+newlib-dev \
+libgloss \
+libgloss-dev \
+libgloss-dbg \
+libgcc-dev \
+libgcc-dbg \
+"
+
+# compilerlibs defaults to gcc-runtime, but we get runtime from libgloss
+# we set ASSUME_PROVIDED because we cant set PREFERRED_PROVIDER
+# for compilerlibs since its overridden by tcmode-default
+ASSUME_PROVIDED += "virtual/${TARGET_PREFIX}compilerlibs virtual/crypt"
+
+# Its useful to be able to extend newlib, but we dont provide a native variant 
of libgloss
+NEWLIB_EXTENDED ?=  "libgloss libgcc"
+BASE_DEFAULT_DEPS_append_class-target = " ${NEWLIB_EXTENDED}"
+
+TARGET_OS = "elf"
+TARGET_OS_arm = "eabi"
+
+TOOLCHAIN_HOST_TASK ?= "packagegroup-cross-canadian-${MACHINE}"
+TOOLCHAIN_TARGET_TASK ?= "${LIBC_DEPENDENCIES}"
+TOOLCHAIN_NEED_CONFIGSITE_CACHE_remove = "zlib ncurses"
diff --git a/meta/recipes-core/newlib/newlib_3.0.0.bb 
b/meta/recipes-core/newlib/newlib_3.0.0.bb
index 3380576..09238fe 100644
--- a/meta/recipes-core/newlib/newlib_3.0.0.bb
+++ b/meta/recipes-core/newlib/newlib_3.0.0.bb
@@ -14,3 +14,6 @@ do_install_append() {
# Remove original directory
rmdir ${D}${prefix}/${TARGET_SYS}
 }
+
+# No rpm package is actually created but -dev depends on it, avoid dnf error
+RDEPENDS_${PN}-dev_libc-newlib = ""
diff --git a/meta/recipes-devtools/gcc/libgcc-common.inc 
b/meta/recipes-devtools/gcc/libgcc-common.inc
index 848a476..d959566 100644
--- a/meta/recipes-devtools/gcc/libgcc-common.inc
+++ b/meta/recipes-devtools/gcc/libgcc-common.inc
@@ -47,8 +47,13 @@ do_install () {
 do_install_append_libc-baremetal () {
rmdir ${D}${base_libdir}
 }
+do_install_append_libc-newlib () {
+   rmdir ${D}${base_libdir}
+}

+# No rpm package is actually created but -dev depends on it, avoid dnf error
 RDEPENDS_${PN}-dev_libc-baremetal = ""
+RDEPENDS_${PN}-dev_libc-newlib = ""

 BBCLASSEXTEND = "nativesdk"

--
2.7.4

This email and any attachments are intended for the sole use of the named 
recipient(s) and contain(s) confidential information that may be proprietary, 
privileged or copyrighted under applicable law. If you are not the intended 
recipient, do not read, copy, or forward this email message or any attachments. 
Delete this email message and any attachments immediately.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/3] newlib: Adds newlib and libgloss recipes

[Alejandro Enedino Hernandez Samaniego]

Newlib is a C library that is intended to be used on embedded systems.
It is a conglomeration of several library parts, all under free software
licenses that make them easily usable on embedded products.

Newlib provides a C library alternative that can run on baremetal, mainly
for resource constrained devices.

Libgloss is the BSP part of the C library, which can be easily modified
to port for new hardware platforms.

Signed-off-by: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>
---
 .../newlib/libgloss/fix-rs6000-crt0.patch  | 41 +++
 .../libgloss/fix_makefile_include_arm_h.patch  | 30 +++
 meta/recipes-core/newlib/libgloss_3.0.0.bb | 32 
 meta/recipes-core/newlib/newlib.inc| 58 ++
 meta/recipes-core/newlib/newlib_3.0.0.bb   | 16 ++
 meta/recipes-devtools/gcc/gcc-cross.inc|  1 +
 6 files changed, 178 insertions(+)
 create mode 100644 meta/recipes-core/newlib/libgloss/fix-rs6000-crt0.patch
 create mode 100644 
meta/recipes-core/newlib/libgloss/fix_makefile_include_arm_h.patch
 create mode 100644 meta/recipes-core/newlib/libgloss_3.0.0.bb
 create mode 100644 meta/recipes-core/newlib/newlib.inc
 create mode 100644 meta/recipes-core/newlib/newlib_3.0.0.bb

diff --git a/meta/recipes-core/newlib/libgloss/fix-rs6000-crt0.patch 
b/meta/recipes-core/newlib/libgloss/fix-rs6000-crt0.patch
new file mode 100644
index 000..c220fb8
--- /dev/null
+++ b/meta/recipes-core/newlib/libgloss/fix-rs6000-crt0.patch
@@ -0,0 +1,41 @@
+Upstream-Status: Inappropriate  [OE-Specific]
+
+Make install tries to perform install for all variants, but some of them use 
the same
+name for crt0 in specific, so performing all of them results in an error during
+do_install, we simply modify the name of the objects so the installation can 
proceed
+and leave it to the user to select which object files to use.
+
+Signed-off-by: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>
+
+Index: newlib-3.0.0/libgloss/rs6000/Makefile.in
+===
+--- newlib-3.0.0.orig/libgloss/rs6000/Makefile.in
 newlib-3.0.0/libgloss/rs6000/Makefile.in
+@@ -358,7 +358,7 @@ install-sim:
+   set -e; for x in ${SIM_CRT0} ${SIM_BSP} ${SIM_SCRIPTS}; do 
${INSTALL_DATA} $$x $(DESTDIR)${tooldir}/lib${MULTISUBDIR}/$$x; done
+
+ install-mvme:
+-  set -e; for x in ${MVME_CRT0} ${MVME_BSP} ${MVME_SCRIPTS}; do 
${INSTALL_DATA} $$x $(DESTDIR)${tooldir}/lib${MULTISUBDIR}/$$x; done
++  set -e; for x in ${MVME_CRT0} ${MVME_BSP} ${MVME_SCRIPTS}; do 
${INSTALL_DATA} $$x $(DESTDIR)${tooldir}/lib${MULTISUBDIR}/mvme-$$x; done
+
+ install-solaris:
+   set -e; for x in ${SOLARIS_CRT0} ${SOLARIS_BSP} ${SOLARIS_SCRIPTS}; do 
${INSTALL_DATA} $$x $(DESTDIR)${tooldir}/lib${MULTISUBDIR}/$$x; done
+@@ -367,15 +367,15 @@ install-linux:
+   set -e; for x in ${LINUX_CRT0} ${LINUX_BSP} ${LINUX_SCRIPTS}; do 
${INSTALL_DATA} $$x $(DESTDIR)${tooldir}/lib${MULTISUBDIR}/$$x; done
+
+ install-yellowknife:
+-  set -e; for x in ${YELLOWKNIFE_CRT0} ${YELLOWKNIFE_BSP}; do 
${INSTALL_DATA} $$x $(DESTDIR)${tooldir}/lib${MULTISUBDIR}/$$x; done
++  set -e; for x in ${YELLOWKNIFE_CRT0} ${YELLOWKNIFE_BSP}; do 
${INSTALL_DATA} $$x $(DESTDIR)${tooldir}/lib${MULTISUBDIR}/yellowknife-$$x; done
+   set -e; for x in ${YELLOWKNIFE_SCRIPTS}; do ${INSTALL_DATA} 
$(srcdir)/$$x $(DESTDIR)${tooldir}/lib${MULTISUBDIR}/$$x; done
+
+ install-ads:
+-  set -e; for x in ${ADS_CRT0} ${ADS_BSP}; do ${INSTALL_DATA} $$x 
$(DESTDIR)${tooldir}/lib${MULTISUBDIR}/$$x; done
++  set -e; for x in ${ADS_CRT0} ${ADS_BSP}; do ${INSTALL_DATA} $$x 
$(DESTDIR)${tooldir}/lib${MULTISUBDIR}/ads-$$x; done
+   set -e; for x in ${ADS_SCRIPTS}; do ${INSTALL_DATA} $(srcdir)/$$x 
$(DESTDIR)${tooldir}/lib${MULTISUBDIR}/$$x; done
+
+ install-mbx:
+-  set -e; for x in ${MBX_CRT0} ${MBX_BSP}; do ${INSTALL_DATA} $$x 
$(DESTDIR)${tooldir}/lib${MULTISUBDIR}/$$x; done
++  set -e; for x in ${MBX_CRT0} ${MBX_BSP}; do ${INSTALL_DATA} $$x 
$(DESTDIR)${tooldir}/lib${MULTISUBDIR}/mbx-$$x; done
+   set -e; for x in ${MBX_SCRIPTS} ${MBX_SPECS}; do ${INSTALL_DATA} 
$(srcdir)/$$x $(DESTDIR)${tooldir}/lib${MULTISUBDIR}/$$x; done
+
+ install-xil:
diff --git a/meta/recipes-core/newlib/libgloss/fix_makefile_include_arm_h.patch 
b/meta/recipes-core/newlib/libgloss/fix_makefile_include_arm_h.patch
new file mode 100644
index 000..7645be7
--- /dev/null
+++ b/meta/recipes-core/newlib/libgloss/fix_makefile_include_arm_h.patch
@@ -0,0 +1,30 @@
+Upstream-Status: Inappropriate  [OE-Specific]
+
+When trying to build libgloss for an arm target, the build system
+complains about missing some include files:
+
+| fatal error: acle-compiat.h: No such file or directory
+|  #include "acle-compat.h"
+|   ^~~
+| compilation terminated.
+
+These include files come from the newlib source, but since we

[OE-core] [PATCH 3/3] tclibc-newlib: Adds a new TCLIBC variant to build with newlib as C library

[Alejandro Enedino Hernandez Samaniego]

This patch adds the posibility to build using TCLIBC=newlib.

It allows users to build baremetal applications with the use of a
C library.

Newlib is a lightweight C library meant to be used on embedded systems,
it is meant to be easily portable for new platforms and to provide
basic functionality on them, by design, it provides stubs for some of
these core functions declared as weak, so they can be built correctly
and then linked against some other library which provides specifics
about the platform being used if need be, libgloss takes care of these
in some cases, but it can also be extended, this patch also allows the
user to easily add other libraries to it by adding them to
NEWLIB_EXTENDED for this specific reason.

Signed-off-by: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>
---
 meta/classes/cross-canadian.bbclass |  2 +-
 meta/conf/distro/include/tclibc-newlib.inc  | 47 +
 meta/recipes-core/newlib/newlib_3.0.0.bb|  3 ++
 meta/recipes-devtools/gcc/libgcc-common.inc |  5 +++
 4 files changed, 56 insertions(+), 1 deletion(-)
 create mode 100644 meta/conf/distro/include/tclibc-newlib.inc

diff --git a/meta/classes/cross-canadian.bbclass 
b/meta/classes/cross-canadian.bbclass
index ee8aa61..7f9a69b 100644
--- a/meta/classes/cross-canadian.bbclass
+++ b/meta/classes/cross-canadian.bbclass
@@ -30,7 +30,7 @@ python () {
 if d.getVar("MODIFYTOS") != "1":
 return

-if d.getVar("TCLIBC") == "baremetal":
+if (d.getVar("TCLIBC") == "baremetal") or (d.getVar("TCLIBC") == "newlib"):
 return

 tos = d.getVar("TARGET_OS")
diff --git a/meta/conf/distro/include/tclibc-newlib.inc 
b/meta/conf/distro/include/tclibc-newlib.inc
new file mode 100644
index 000..a5e63d0
--- /dev/null
+++ b/meta/conf/distro/include/tclibc-newlib.inc
@@ -0,0 +1,47 @@
+#
+# Newlib configuration
+#
+
+LIBCEXTENSION = "-newlib"
+
+# Add newlib libc overrides to the overrides.
+LIBCOVERRIDE = ":libc-newlib"
+OVERRIDES .= "${LIBCOVERRIDE}"
+
+PREFERRED_PROVIDER_virtual/libc ?= "newlib"
+PREFERRED_PROVIDER_virtual/libiconv ?= "newlib"
+PREFERRED_PROVIDER_virtual/libintl ?= "newlib"
+PREFERRED_PROVIDER_virtual/nativesdk-libintl ?= "nativesdk-glibc"
+PREFERRED_PROVIDER_virtual/nativesdk-libiconv ?= "nativesdk-glibc"
+
+DISTRO_FEATURES_BACKFILL_CONSIDERED += "ldconfig"
+
+#USE_NLS ?= "no"
+
+IMAGE_LINGUAS = ""
+
+LIBC_DEPENDENCIES = "\
+newlib-dbg \
+newlib-dev \
+libgloss \
+libgloss-dev \
+libgloss-dbg \
+libgcc-dev \
+libgcc-dbg \
+"
+
+# compilerlibs defaults to gcc-runtime, but we get runtime from libgloss
+# we set ASSUME_PROVIDED because we cant set PREFERRED_PROVIDER
+# for compilerlibs since its overriden by tcmode-default
+ASSUME_PROVIDED += "virtual/${TARGET_PREFIX}compilerlibs virtual/crypt"
+
+# Its useful to be able to extend newlib, but we dont provide a native variant 
of libgloss
+NEWLIB_EXTENDED ?=  "libgloss libgcc"
+BASE_DEFAULT_DEPS_append_class-target = " ${NEWLIB_EXTENDED}"
+
+TARGET_OS = "elf"
+TARGET_OS_arm = "eabi"
+
+TOOLCHAIN_HOST_TASK ?= "packagegroup-cross-canadian-${MACHINE}"
+TOOLCHAIN_TARGET_TASK ?= "${LIBC_DEPENDENCIES}"
+TOOLCHAIN_NEED_CONFIGSITE_CACHE_remove = "zlib ncurses"
diff --git a/meta/recipes-core/newlib/newlib_3.0.0.bb 
b/meta/recipes-core/newlib/newlib_3.0.0.bb
index 3380576..09238fe 100644
--- a/meta/recipes-core/newlib/newlib_3.0.0.bb
+++ b/meta/recipes-core/newlib/newlib_3.0.0.bb
@@ -14,3 +14,6 @@ do_install_append() {
# Remove original directory
rmdir ${D}${prefix}/${TARGET_SYS}
 }
+
+# No rpm package is actually created but -dev depends on it, avoid dnf error
+RDEPENDS_${PN}-dev_libc-newlib = ""
diff --git a/meta/recipes-devtools/gcc/libgcc-common.inc 
b/meta/recipes-devtools/gcc/libgcc-common.inc
index 848a476..d959566 100644
--- a/meta/recipes-devtools/gcc/libgcc-common.inc
+++ b/meta/recipes-devtools/gcc/libgcc-common.inc
@@ -47,8 +47,13 @@ do_install () {
 do_install_append_libc-baremetal () {
rmdir ${D}${base_libdir}
 }
+do_install_append_libc-newlib () {
+   rmdir ${D}${base_libdir}
+}

+# No rpm package is actually created but -dev depends on it, avoid dnf error
 RDEPENDS_${PN}-dev_libc-baremetal = ""
+RDEPENDS_${PN}-dev_libc-newlib = ""

 BBCLASSEXTEND = "nativesdk"

--
2.7.4

This email and any attachments are intended for the sole use of the named 
recipient(s) and contain(s) confidential information that may be proprietary, 
privileged or copyrighted under applicable law. If you are not the intended 
recipient, do not read, copy, or forward this email message or any attachments. 
Delete this email message and any attachments immediately.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 0/3] Add Newlib as a C library for baremetal

[Alejandro Enedino Hernandez Samaniego]

Newlib is a widely used C library for embedded systems, its mainly used
for resource constrained devices which run baremetal applications.

It provides basic C library functionality, and it is meant to be extended
by design, it contains several stubs for core functions that can be later
overriden with the symbols provided from another library at linking time.

Libgloss is the BSP part of the library which follows the same principle,
where a BSP layer can override functionality or even application runtime
depending on the processor architecture.

These patches add both the libgloss and newlib recipes, provide functionality
to make them extendable by other libraries if required, and also create a
new TCLIBC variant, which can now be set to newlib to build the toolchain for
applications where its suitable.

Newlib is also usually used by vendors to provide embedded toolchains, which is
interesting since, this enables OE to build such toolchains in a somewhat
automated way.

These patches have been tested on my end in the following ways:

- meta-toolchain builds successfully on qemux86, qemux86-64, qemuarm, qemuarm64,
  qemuppc, qemumips.
- A helloworld application that lays in another layer was built for qemux86, 
qemux86-64
  qemuarm, qemuarm64, qemuppc, qemumips, and it seems to have been built 
correctly.
- The hardware that I have available is from Xilinx, where Xilinx already had a
  baremetal application they ran on it, I was able to build the same application
  with a recipe, that used the toolchain coming from these patches 
(TCLIBC=newlib).
  I ran the application on hardware and confirmed that it was built and working 
correctly.
- meta-toolchain builds successfully for the previously mentioned HW MACHINE as 
well.
- I also have a meta-freertos (its a possibility) layer that builds a FreeRTOS 
application
  that runs on qemuarm, and instead of using the provided ARM Embedded 
Toolchain, I used
  the one coming from these patches and was able to obtain the same result.
- I built world for all the previously mentioned MACHINES (qemus and HW) just 
to make sure
  that this did not affect other OE components, and it did not seem to be the 
case.

The following changes since commit 7897fb35a3420c2555ab59dfa438217ada4a9784:

  libnss-mdns: fix postinst and postrm (2018-05-22 13:13:33 +0100)

are available in the git repository at:

  git://push.yoctoproject.org/poky-contrib aehs29/newlib

Alejandro Enedino Hernandez Samaniego (3):
  newlib: Adds newlib and libgloss recipes
  insane.bbclass: Adds powerpc to elf headers machine dictionary
  tclibc-newlib: Adds a new TCLIBC variant to build with newlib as C
library

 meta/classes/cross-canadian.bbclass|  2 +-
 meta/classes/insane.bbclass|  1 +
 meta/conf/distro/include/tclibc-newlib.inc | 47 ++
 .../newlib/libgloss/fix-rs6000-crt0.patch  | 41 +++
 .../libgloss/fix_makefile_include_arm_h.patch  | 30 +++
 meta/recipes-core/newlib/libgloss_3.0.0.bb | 32 
 meta/recipes-core/newlib/newlib.inc| 58 ++
 meta/recipes-core/newlib/newlib_3.0.0.bb   | 19 +++
 meta/recipes-devtools/gcc/gcc-cross.inc|  1 +
 meta/recipes-devtools/gcc/libgcc-common.inc|  5 ++
 10 files changed, 235 insertions(+), 1 deletion(-)
 create mode 100644 meta/conf/distro/include/tclibc-newlib.inc
 create mode 100644 meta/recipes-core/newlib/libgloss/fix-rs6000-crt0.patch
 create mode 100644 
meta/recipes-core/newlib/libgloss/fix_makefile_include_arm_h.patch
 create mode 100644 meta/recipes-core/newlib/libgloss_3.0.0.bb
 create mode 100644 meta/recipes-core/newlib/newlib.inc
 create mode 100644 meta/recipes-core/newlib/newlib_3.0.0.bb

--
2.7.4

This email and any attachments are intended for the sole use of the named 
recipient(s) and contain(s) confidential information that may be proprietary, 
privileged or copyrighted under applicable law. If you are not the intended 
recipient, do not read, copy, or forward this email message or any attachments. 
Delete this email message and any attachments immediately.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/3] insane.bbclass: Adds powerpc to elf headers machine dictionary

[Alejandro Enedino Hernandez Samaniego]

This patch adds makes it possible to build elf for powerpc by adding
this combination to the elf headers machine dictionary, this can be
useful when trying to build baremetal applications where the TARGET_OS=elf

Signed-off-by: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>
---
 meta/classes/insane.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index fa15460..bd7f519 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -71,6 +71,7 @@ def package_qa_get_machine_dict(d):
 "microblaze":  (189,   0,0,  False,
 32),
 "microblazeeb":(189,   0,0,  False,
 32),
 "microblazeel":(189,   0,0,  True, 
 32),
+"powerpc":(20, 0,0,  False,
 32),
 "riscv32":(243,0,0,  True, 
 32),
 "riscv64":(243,0,0,  True, 
 64),
   },
--
2.7.4

This email and any attachments are intended for the sole use of the named 
recipient(s) and contain(s) confidential information that may be proprietary, 
privileged or copyrighted under applicable law. If you are not the intended 
recipient, do not read, copy, or forward this email message or any attachments. 
Delete this email message and any attachments immediately.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] tclibc-baremetal: Adds virtual/crypt to ASSUME_PROVIDED

[Alejandro Enedino Hernandez Samaniego]

When trying to build meta-toolchain using TCLIBC = "baremetal"
bitbake throws an error due to a mising dependency:

ERROR: Nothing PROVIDES 'virtual/crypt'

glibc PROVIDES virtual/crypt but was skipped:
PREFERRED_PROVIDER_virtual/libc set to musl, not glibc
musl PROVIDES virtual/crypt but was skipped:
PREFERRED_PROVIDER_virtual/i586-poky-elf-libc-for-gcc set to baremetal,
not musl
libxcrypt PROVIDES virtual/crypt but was skipped: Recipe only applies in
nativesdk case for now

This is caused by the changes on commit:
29f65bda6d2c9fea4adb125c4857ee64f9312b9f
nativesdk-glibc: Split glibc and libcrypt to use libxcrypt instead

This is where the concept of virtual/crypt was introduced.

This patch adds virtual/crypt to ASSUME_PROVIDED on tclibc-baremetal,
providing the missing wiring to build meta-toolchain on baremetal
correctly.

Signed-off-by: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>
---
 meta/conf/distro/include/tclibc-baremetal.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/distro/include/tclibc-baremetal.inc 
b/meta/conf/distro/include/tclibc-baremetal.inc
index a8ce6d3..8d570f0 100644
--- a/meta/conf/distro/include/tclibc-baremetal.inc
+++ b/meta/conf/distro/include/tclibc-baremetal.inc
@@ -8,7 +8,7 @@ LIBCEXTENSION = ""
 LIBCOVERRIDE = ":libc-baremetal"
 OVERRIDES .= "${LIBCOVERRIDE}"

-ASSUME_PROVIDED += "virtual/libc virtual/libiconv"
+ASSUME_PROVIDED += "virtual/libc virtual/libiconv virtual/crypt"

 PREFERRED_PROVIDER_virtual/libc ?= "musl"
 PREFERRED_PROVIDER_virtual/libiconv ?= "musl"
--
2.7.4

This email and any attachments are intended for the sole use of the named 
recipient(s) and contain(s) confidential information that may be proprietary, 
privileged or copyrighted under applicable law. If you are not the intended 
recipient, do not read, copy, or forward this email message or any attachments. 
Delete this email message and any attachments immediately.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] python recipes with extensions fail to find/link with proper libraries

[Alejandro Enedino Hernandez Samaniego]

Hello Francesco,


Please hit the reply-all button, otherwise I wont be getting notified 
when you reply.



I think you're doing something very specific since you're building on 
target and tweaking the build system so to help you more I would need to 
take a look at the recipes you are using.



Alejandro


On 05/04/2018 01:19 AM, Francesco Giancane wrote:

Hi Alejandro,

Thank you for your response.
I know that openblas recipe is a pain, given that its Makefile is not 
correctly written with respect to cross-compilation. However, the 
library has been built locally on the target (sigh!) and then it is 
being deployed with a recipe which unpacks the built binary and copy 
them into the sys root.


So, from the openblas side, everything is OK.

What is actually wrong is the configuration step for python3-numpy and 
python3-scipy: in the setup.py they are instructed to search for some 
libraries (namely, openblas amongst them) but they fail to search for 
them given that the path they are searching for is the sysroot-native, 
rather than sysroot. (you can see it clearly from the logs attached in 
the first message).


Installing the packages using pip3 install scipy numpy on the target 
results on a correct execution (and libraries are properly found and 
linked).


Thank you,
Francesco Giancane




Il giorno ven 4 mag 2018 alle ore 03:01 Alejandro Enedino Hernandez 
Samaniego <aleja...@xilinx.com <mailto:aleja...@xilinx.com>> ha scritto:


Hello Francesco,

It would be helpful if you could provide more information about
this, e.g. access to your layer or your recipes, to make sure they
are actually being built correctly, also I would start by manually
linking it against the built library to see if the error is just
not having them on the correct path, but I seem to remember a lot
of issues when trying to cross compile openblas, specially for an
arm MACHINE.

Alejandro

*From:* openembedded-core-boun...@lists.openembedded.org
<mailto:openembedded-core-boun...@lists.openembedded.org>
[mailto:openembedded-core-boun...@lists.openembedded.org
<mailto:openembedded-core-boun...@lists.openembedded.org>] *On
Behalf Of *Francesco Giancane
*Sent:* Wednesday, May 2, 2018 2:33 PM
*To:* OE-core <openembedded-core@lists.openembedded.org
<mailto:openembedded-core@lists.openembedded.org>>
*Subject:* [OE-core] python recipes with extensions fail to
find/link with proper libraries

Hi Everyone,

I found a problem working with python(3) and C extensions.

The use-case is the recipe for python(3)-numpy, which can be
linked against a BLAS/LAPACK library to use accelerated matricial
and vectorial operations.

As it is currently shipped, the recipe fails to find any proper
library and then does not use extensions.

I wrote a custom recipe for openblas which is correctly built &
deploy using standard commands.

openblas is also added as a DEPENDS in a bbappend file for numpy.

However, the do_compile phase cannot find the aforementioned library.

Log attached:

DEBUG: Executing shell function do_compile

blas_opt_info:

blas_mkl_info:

customize UnixCCompiler

  libraries mkl_rt not found in

['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']

  NOT AVAILABLE

blis_info:

customize UnixCCompiler

  libraries blis not found in

['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']

  NOT AVAILABLE

openblas_info:

customize UnixCCompiler

customize UnixCCompiler

  libraries openblas not found in

['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']

  NOT AVAILABLE

atlas_3_10_blas_threads_info:

Setting PTATLAS=ATLAS

customize UnixCCompiler

  libraries tatlas not found in

['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']

  NOT AVAILABLE

atlas_3_10_blas_info:

customize UnixCCompiler

  libraries satlas not found in

['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']

  NOT AVAILABLE

atlas_blas_threads_info:

Setting PTATLAS=ATLAS

customize UnixCCompiler

  libraries ptf77blas,ptcblas,atlas not found in

['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']

  NOT AVAILABLE

atlas_blas_info:

customize UnixCCompiler

  libraries f77blas,cblas,atlas not found in

['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/pytho

Re: [OE-core] python recipes with extensions fail to find/link with proper libraries

[Alejandro Enedino Hernandez Samaniego]

Hello Francesco,

It would be helpful if you could provide more information about this, e.g. 
access to your layer or your recipes, to make sure they are actually being 
built correctly, also I would start by manually linking it against the built 
library to see if the error is just not having them on the correct path, but I 
seem to remember a lot of issues when trying to cross compile openblas, 
specially for an arm MACHINE.

Alejandro

From: openembedded-core-boun...@lists.openembedded.org 
[mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf Of 
Francesco Giancane
Sent: Wednesday, May 2, 2018 2:33 PM
To: OE-core 
Subject: [OE-core] python recipes with extensions fail to find/link with proper 
libraries

Hi Everyone,
I found a problem working with python(3) and C extensions.

The use-case is the recipe for python(3)-numpy, which can be linked against a 
BLAS/LAPACK library to use accelerated matricial and vectorial operations.

As it is currently shipped, the recipe fails to find any proper library and 
then does not use extensions.

I wrote a custom recipe for openblas which is correctly built & deploy using 
standard commands.
openblas is also added as a DEPENDS in a bbappend file for numpy.

However, the do_compile phase cannot find the aforementioned library.

Log attached:

DEBUG: Executing shell function do_compile
blas_opt_info:
blas_mkl_info:
customize UnixCCompiler
  libraries mkl_rt not found in 
['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']
  NOT AVAILABLE

blis_info:
customize UnixCCompiler
  libraries blis not found in 
['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']
  NOT AVAILABLE

openblas_info:
customize UnixCCompiler
customize UnixCCompiler
  libraries openblas not found in 
['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']
  NOT AVAILABLE

atlas_3_10_blas_threads_info:
Setting PTATLAS=ATLAS
customize UnixCCompiler
  libraries tatlas not found in 
['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']
  NOT AVAILABLE

atlas_3_10_blas_info:
customize UnixCCompiler
  libraries satlas not found in 
['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']
  NOT AVAILABLE

atlas_blas_threads_info:
Setting PTATLAS=ATLAS
customize UnixCCompiler
  libraries ptf77blas,ptcblas,atlas not found in 
['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']
  NOT AVAILABLE

atlas_blas_info:
customize UnixCCompiler
  libraries f77blas,cblas,atlas not found in 
['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']
  NOT AVAILABLE

blas_info:
customize UnixCCompiler
  libraries blas not found in 
['/home/fgp/poky/rpi3_64/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/python3-numpy/1.14.2-r0/recipe-sysroot-native/usr/lib']
  NOT AVAILABLE

blas_src_info:
  NOT AVAILABLE

  NOT AVAILABLE

...And it is correct for it to fail as it is searching for openblas in the 
wrong path (sysroot-native vs sysroot).

I would assume that this is a setuptools.bbclass issue, does anyone know what 
to do? Is it trivial to fix?

I noticed that all the shipped python extensions were built as if we were 
building for x86, even though the MACHINE is an arm target.

Thank you for your attention,

Francesco Giancane

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [OE-CORE][sumo][python3] Generated SDK missing sqlite3

[Alejandro Enedino Hernandez Samaniego]

Hey Mike,

It does make a little sense that you're not getting 
"sqlite_version_info" from sqlite3, the python3-sqlite3 is missing the 
__init__.py file (on purpose) for that module, and the only thing that 
does is


from sqlite3.dbapi2 import *

and sqlite_version_info comes from sqilte3.dbapi2, hence why you might 
not be getting it, but I would still like to know the steps you're 
following to reproduce this, since I couldn't do that either.



Alejandro

On 04/17/2018 09:27 AM, Burton, Ross wrote:

Work for me with plain oe-core:

$ ./poky-glibc-x86_64-core-image-minimal-corei7-64-toolchain-ext-2.4+snapshot.sh
Poky (Yocto Project Reference Distro) Extensible SDK installer version
2.4+snapshot
===
Enter target directory for SDK (default: ~/poky_sdk):
You are about to install the SDK to "/home/ross/poky_sdk". Proceed[Y/n]? y
Extracting 
SDK...done
Setting it up...
Extracting buildtools...
Preparing build system...
Parsing recipes: 100%
|##|
Time: 0:00:10
Initialising tasks: 100%
|###|
Time: 0:00:07
Checking sstate mirror object availability: 100%
|###| Time: 0:00:00
Loading cache: 100%
||
Time: 0:00:00
Initialising tasks: 100%
|###|
Time: 0:00:00
done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to
source the environment setup script e.g.
  $ . /home/ross/poky_sdk/environment-setup-corei7-64-poky-linux

Obviously something is different, how are you reproducing this?

Ross

On 17 April 2018 at 14:50, Davis, Michael <michael.da...@essvote.com> wrote:

I was on the HEAD of sumo(commit d012a9acdc3dce014705224a1261145628e75931)
so that commit was already included when I had the issue.  Just generated
another test today and the issue was still there.  If I remove
native-python3-misc from TOOLCHAIN_HOST_TASK it fails on installing the SDK.
With it there it succeeds.



Mike







From: Alejandro Enedino Hernandez Samaniego [mailto:aleja...@xilinx.com]
Sent: Monday, April 16, 2018 8:07 PM
To: Davis, Michael; openembedded-core@lists.openembedded.org
Subject: RE: [OE-CORE][sumo][python3] Generated SDK missing sqlite3



Which commit are you using?



https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=4379b29e7eede727993ee68c9eba2bdebace5108



In any case, that one should’ve solved it.



Alejandro



From: openembedded-core-boun...@lists.openembedded.org
[mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf Of
Davis, Michael
Sent: Monday, April 16, 2018 11:19 AM
To: Davis, Michael <michael.da...@essvote.com>;
openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [OE-CORE][sumo][python3] Generated SDK missing
sqlite3



Adding python3-misc to the SDK seems to have solved the issue.

I saw some commits about separating the two so perhaps this is a side
effect?



From: Davis, Michael
Sent: Monday, April 16, 2018 11:04 AM
To: 'openembedded-core@lists.openembedded.org'
Subject: [OE-CORE][sumo][python3] Generated SDK missing sqlite3



Testing out the upcoming sumo release and ran into an issue.  Whenever I
attempt to generate an sdk-ext with sumo I get a failure about missing
sqlite3.

Traceback (most recent call last):
  File "/home/mdavis/core_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/devtool",
line 344, in 
ret = main()
  File "/home/mdavis/core_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/devtool",
line 285, in main
scriptutils.logger_setup_color(logger, global_args.color)
  File "/home/mdavis/core_sdk/layers/core/scripts/lib/scriptutils.py", line
38, in logger_setup_color
from bb.msg import BBLogFormatter
  File "/home/mdavis/core_sdk/layers/core/bitbake/lib/bb/__init__.py", line
79, in 
from bb import fetch2 as fetch
  File "/home/mdavis/core_sdk/layers/core/bitbake/lib/bb/fetch2/__init__.py",
line 39, in 
import bb.persist_data, bb.utils
  File "/home/mdavis/core_sdk/layers/core/bitbake/lib/bb/persist_data.py",
line 33, in 
sqlversion = sqlite3.sqlite_version_info
AttributeError: module 'sqlite3' has no attribute 'sqlite_version_info'



The issue seems to be isolated only to python3. The sdk partially installs
and I am able to source it.  I created a small python sqlite app and was
able to run it properly with python2, but not 3.




--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listin

Re: [OE-core] [OE-CORE][sumo][python3] Generated SDK missing sqlite3

[Alejandro Enedino Hernandez Samaniego]

Which commit are you using?

https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=4379b29e7eede727993ee68c9eba2bdebace5108

In any case, that one should've solved it.

Alejandro

From: openembedded-core-boun...@lists.openembedded.org 
[mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf Of Davis, 
Michael
Sent: Monday, April 16, 2018 11:19 AM
To: Davis, Michael ; 
openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [OE-CORE][sumo][python3] Generated SDK missing sqlite3

Adding python3-misc to the SDK seems to have solved the issue.
I saw some commits about separating the two so perhaps this is a side effect?

From: Davis, Michael
Sent: Monday, April 16, 2018 11:04 AM
To: 'openembedded-core@lists.openembedded.org'
Subject: [OE-CORE][sumo][python3] Generated SDK missing sqlite3

Testing out the upcoming sumo release and ran into an issue.  Whenever I 
attempt to generate an sdk-ext with sumo I get a failure about missing sqlite3.
Traceback (most recent call last):
 File "/home/mdavis/core_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/devtool", 
line 344, in 
   ret = main()
 File "/home/mdavis/core_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/devtool", 
line 285, in main
   scriptutils.logger_setup_color(logger, global_args.color)
 File "/home/mdavis/core_sdk/layers/core/scripts/lib/scriptutils.py", line 38, 
in logger_setup_color
   from bb.msg import BBLogFormatter
 File "/home/mdavis/core_sdk/layers/core/bitbake/lib/bb/__init__.py", line 79, 
in 
   from bb import fetch2 as fetch
 File "/home/mdavis/core_sdk/layers/core/bitbake/lib/bb/fetch2/__init__.py", 
line 39, in 
   import bb.persist_data, bb.utils
 File "/home/mdavis/core_sdk/layers/core/bitbake/lib/bb/persist_data.py", line 
33, in 
   sqlversion = sqlite3.sqlite_version_info
AttributeError: module 'sqlite3' has no attribute 'sqlite_version_info'

The issue seems to be isolated only to python3. The sdk partially installs and 
I am able to source it.  I created a small python sqlite app and was able to 
run it properly with python2, but not 3.

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] python3: Improve logging capabilities for do_create_manifest

[Alejandro Enedino Hernandez Samaniego]

Hey Richard,

-Original Message-
From: Richard Purdie [mailto:richard.pur...@linuxfoundation.org] 
Sent: Tuesday, April 3, 2018 3:09 AM
To: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>; 
openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] python3: Improve logging capabilities for 
do_create_manifest

On Tue, 2018-04-03 at 11:04 +0100, Richard Purdie wrote:
> On Mon, 2018-04-02 at 13:08 -0700, Alejandro Enedino Hernandez 
> Samaniego wrote:
> > 
> > Adds a couple of prints to get a nicer log, and creates a small 
> > summary or report after checking every module, it makes it more 
> > feasible for adoption, easier to debug why a module ended at a 
> > certain package and see how the manifest was created.
> > 
> > Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xili 
> > nx .com>
> Can you rebase against master please as this clashes with one of your 
> other changes which already merged!

Actually, I have it applied now, thanks.

Richard

Thanks, I think my email server is to blame for this errors.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] python3: Improve logging capabilities for do_create_manifest

[Alejandro Enedino Hernandez Samaniego]

Adds a couple of prints to get a nicer log, and creates a
small summary or report after checking every module, it
makes it more feasible for adoption, easier to debug why
a module ended at a certain package and see how the
manifest was created.

Signed-off-by: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>
---
 .../python/python3/create_manifest3.py | 34 +-
 1 file changed, 27 insertions(+), 7 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/create_manifest3.py 
b/meta/recipes-devtools/python/python3/create_manifest3.py
index 212ddd4..44f3454 100644
--- a/meta/recipes-devtools/python/python3/create_manifest3.py
+++ b/meta/recipes-devtools/python/python3/create_manifest3.py
@@ -80,7 +80,7 @@ with open('python3-manifest.json') as manifest:

 # First pass to get core-package functionality, because we base everything on 
the fact that core is actually working
 # Not exactly the same so it should not be a function
-print ('Getting dependencies for core package:')
+print ('Getting dependencies for package: core')

 # Special call to check for core package
 output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 
'python-core-package']).decode('utf8')
@@ -128,6 +128,7 @@ for value in old_manifest['core']['files']:
   # Each module will only import what is necessary for it to work in specific
   print ('Getting dependencies for module: %s' % value)
   output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 
'%s' % value]).decode('utf8')
+  print ('The following dependencies were found for module %s:\n' % value)
   print (output)
   for item in output.split():
 # We append it so it doesnt hurt what we currently have:
@@ -170,7 +171,10 @@ for key in old_manifest:

 # Handle special cases, we assume that when they were manually added
 # to the manifest we knew what we were doing.
+print('\n')
+print('--')
 print ('Handling package %s' % key)
+print('--')
 special_packages=['misc', 'modules', 'dev']
 if key in special_packages or 'staticdev' in key:
 print('Passing %s package directly' % key)
@@ -219,11 +223,16 @@ for key in old_manifest:

 # Launch separate task for each module for deterministic behavior
 # Each module will only import what is necessary for it to work in 
specific
-print ('Getting dependencies for module: %s' % value)
+print ('\nGetting dependencies for module: %s' % value)
 output = subprocess.check_output([sys.executable, 
'get_module_deps3.py', '%s' % value]).decode('utf8')
 # We can print dependencies for debugging purposes
+print ('The following dependencies were found for module %s:\n' % 
value)
 print (output)
 # Output will have all dependencies
+
+reportFILES = []
+reportRDEPS = []
+
 for item in output.split():

 # Warning: This first part is ugly
@@ -255,7 +264,7 @@ for key in old_manifest:
 #print('Checking folder %s on package %s' % 
(item,keyfolder))
 for file_folder in 
old_manifest[keyfolder]['files'] or file_folder in 
old_manifest[keyfolder]['cached']:
 if file_folder==folder:
-print ('%s found in %s' % (folder, 
keyfolder))
+print ('%s folder found in %s' % (folder, 
keyfolder))
 folderFound = True
 if keyfolder not in 
new_manifest[key]['rdepends'] and keyfolder != key:
 
new_manifest[key]['rdepends'].append(keyfolder)
@@ -267,6 +276,7 @@ for key in old_manifest:
 if inFolders:
 continue

+
 # We might already have it on the dictionary since it could depend 
on a (previously checked) module
 if item not in new_manifest[key]['files'] and item not in 
new_manifest[key]['cached']:
 # Handle core as a special package, we already did it so we 
pass it to NEW data structure directly
@@ -287,6 +297,7 @@ for key in old_manifest:

 else:

+
 # Check if this dependency is already contained on another 
package, so we add it
 # as an RDEPENDS, or if its not, it means it should be 
contained on the current
 # package, so we should add it to FILES
@@ -298,19 +309,19 @@ for key in old_manifest:
 if(newkey!=key):
 if newkey not in 
new_manifest[key]['rdepends']:
# Add it to the new manifest data struct
-   # Debug
-   print('Adding %s to %s RDEPENDS, 
because it contains %s' % (newkey, key

Re: [OE-core] [PATCH] python3: consolidate sqlite3 files

[Alejandro Enedino Hernandez Samaniego]

Hey Ross,

I just sent you a patch that goes on top of this that fixes things.

Thanks

Alejandro

-Original Message-
From: openembedded-core-boun...@lists.openembedded.org 
[mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf Of 
Alejandro Enedino Hernandez Samaniego
Sent: Thursday, March 29, 2018 7:14 PM
To: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] python3: consolidate sqlite3 files

Hey Ross,


On 03/28/2018 01:19 PM, Ross Burton wrote:

python3-sqlite3 happens to be a very special and unique package, its folder 
inside ${libdir}/python3.5/ called sqlite3, which would be fine, we usually  
handle those packages by adding the whole folder, but in this case, we 
separated the sqlite3-tests (which is a folder inside the
sqlite3 folder)
into another package, so if we add the folder to sqlite3 there would be nothing 
for sqlite3-tests to get.

> Some of the sqlite3 module was in python3-misc by accident, move the 
> files into python3-sqlite3 where they belong.
>
> Signed-off-by: Ross Burton <ross.bur...@intel.com>
> ---
>   meta/recipes-devtools/python/python3/python3-manifest.json | 7 +--
>   1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git 
> a/meta/recipes-devtools/python/python3/python3-manifest.json 
> b/meta/recipes-devtools/python/python3/python3-manifest.json
> index 1914a4c43ba..95071c53f5a 100644
> --- a/meta/recipes-devtools/python/python3/python3-manifest.json
> +++ b/meta/recipes-devtools/python/python3/python3-manifest.json
> @@ -977,9 +977,12 @@
>   "summary": "Python Simple Mail Transport Daemon"
>   },
>   "sqlite3": {
> -"cached": [],
> +"cached": [
> +"${libdir}/python3.5/sqlite3/__pycache__/*.pyc"
> +],
>   "files": [
> -"${libdir}/python3.5/lib-dynload/_sqlite3.*.so"
> +"${libdir}/python3.5/lib-dynload/_sqlite3.*.so",

While this will work for splitting packages now, we dont support wildcards on 
the manifest file anymore, at least not for the whole filename, because when we 
try to get dependencies for a module the wildcard becomes useless, so this will 
likely kill the do_create_manifest task, it'll go fine but the next time you 
run it, it wont get the module name it will disregard that dependency.
> +"${libdir}/python3.5/sqlite3/*.py"
>   ],
>   "rdepends": [
>   "core"

The right way would be to get rid of sqlite3-tests but I know we cant do that, 
so I'm working on a fix for the do_create_manifest task that could handle this 
situation.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] python3: Fix do_create_manifest for python3-sqlite3

[Alejandro Enedino Hernandez Samaniego]

Some of the sqlite3 files ended up in python3-misc incorrectly,
this is caused becuse we couldnt add the whole ${libdir}/python3/sqlite3
folder on the package because we also have another sqlite3-tests
package that needs to include another folder from that directory.

This patch not only fixes the do_create_manifest script to handle this
situation, but also patches the manifest (created using the script)
which also fixes a hiddn runtime dependency that we wouldn't have seen.

Signed-off-by: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>
---
 .../python/python3/create_manifest3.py | 43 ++
 .../python/python3/python3-manifest.json   | 15 +---
 2 files changed, 37 insertions(+), 21 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/create_manifest3.py 
b/meta/recipes-devtools/python/python3/create_manifest3.py
index 43e95ce..212ddd4 100644
--- a/meta/recipes-devtools/python/python3/create_manifest3.py
+++ b/meta/recipes-devtools/python/python3/create_manifest3.py
@@ -124,7 +124,6 @@ for value in old_manifest['core']['files']:
   # Get module name , shouldnt be affected by libdir/bindir
   value = os.path.splitext(os.path.basename(os.path.normpath(value)))[0]

-
   # Launch separate task for each module for deterministic behavior
   # Each module will only import what is necessary for it to work in specific
   print ('Getting dependencies for module: %s' % value)
@@ -203,8 +202,20 @@ for key in old_manifest:
 if value not in new_manifest[key]['files']:
 new_manifest[key]['files'].append(value)
 continue
+
 # Get module name , shouldnt be affected by libdir/bindir
-value = os.path.splitext(os.path.basename(os.path.normpath(value)))[0]
+# We need to check if the imported module comes from another (e.g. 
sqlite3.dump)
+path,value = os.path.split(value)
+path = os.path.basename(path)
+value = os.path.splitext(os.path.basename(value))[0]
+
+# If this condition is met, it means we need to import it from another 
module
+# or its the folder itself (e.g. unittest)
+if path == key:
+  if value:
+value = path + '.' + value
+  else:
+value = path

 # Launch separate task for each module for deterministic behavior
 # Each module will only import what is necessary for it to work in 
specific
@@ -292,19 +303,20 @@ for key in old_manifest:

new_manifest[key]['rdepends'].append(newkey)
 break
 else:
-  # Debug
-  print('Adding %s to %s FILES' % (item, key))
-  # Since it wasnt found on another package, its not an 
RDEP, so add it to FILES for this package
-  if isCached(item):
-  new_manifest[key]['cached'].append(item)
-  else:
-  new_manifest[key]['files'].append(item)
-  if item.endswith('*'):
-  wildcards.append(item)
-  if item not in allfiles:
-  allfiles.append(item)
-  else:
-  repeated.append(item)
+  # A module shouldn't contain itself 
(${libdir}/python3/sqlite3 shouldnt be on sqlite3 files)
+  if os.path.basename(item) != key:
+print('Adding %s to %s FILES' % (item, key))
+# Since it wasnt found on another package, its not an 
RDEP, so add it to FILES for this package
+if isCached(item):
+new_manifest[key]['cached'].append(item)
+else:
+new_manifest[key]['files'].append(item)
+if item.endswith('*'):
+wildcards.append(item)
+if item not in allfiles:
+allfiles.append(item)
+else:
+repeated.append(item)

 print ('The following files are repeated (contained in more than one package), 
please check which package should get it:')
 print (repeated)
@@ -322,3 +334,4 @@ for key in new_manifest:
 # Create the manifest from the data structure that was built
 with open('python3-manifest.json.new','w') as outfile:
 json.dump(new_manifest,outfile,sort_keys=True, indent=4)
+outfile.write("\n")
diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index 911be8e..26fa613 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -591,8 +591,7 @@
 ],
 "rdepends&q

[OE-core] [PATCH 1/2] python3: fix create_manifest to handle pycache folders

[Alejandro Enedino Hernandez Samaniego]

We have a couple of python modules that contain folders themselves,
for that reason they also contain a __pycache__ folder inside those
directories, since we include the whole folder in the manifest, the
pycache directories end up with the files and not the cache files.

This patch catches that and adds the directories to the correct
structure.

Signed-off-by: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>
---
 .../python/python3/create_manifest3.py   |  5 -
 .../python/python3/python3-manifest.json | 20 +++-
 2 files changed, 15 insertions(+), 10 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/create_manifest3.py 
b/meta/recipes-devtools/python/python3/create_manifest3.py
index ead27e9..43e95ce 100644
--- a/meta/recipes-devtools/python/python3/create_manifest3.py
+++ b/meta/recipes-devtools/python/python3/create_manifest3.py
@@ -186,7 +186,10 @@ for key in old_manifest:
 # Ignore folders, since we don't import those, difficult to handle 
multilib
 if isFolder(value):
 # Pass folders directly
-new_manifest[key]['files'].append(value)
+if isCached(value):
+new_manifest[key]['cached'].append(value)
+else:
+new_manifest[key]['files'].append(value)
 # Ignore binaries, since we don't import those
 if '${bindir}' in value:
 # Pass it directly to the new manifest data structure
diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index 95071c5..911be8e 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -737,21 +737,21 @@
 "${libdir}/python3.5/__pycache__/poplib.*.pyc",
 "${libdir}/python3.5/__pycache__/smtplib.*.pyc",
 "${libdir}/python3.5/__pycache__/telnetlib.*.pyc",
-"${libdir}/python3.5/__pycache__/uuid.*.pyc"
+"${libdir}/python3.5/__pycache__/uuid.*.pyc",
+"${libdir}/python3.5/http/__pycache__",
+"${libdir}/python3.5/urllib/__pycache__"
 ],
 "files": [
 "${libdir}/python3.5/base64.py",
 "${libdir}/python3.5/ftplib.py",
 "${libdir}/python3.5/hmac.py",
 "${libdir}/python3.5/http",
-"${libdir}/python3.5/http/__pycache__",
 "${libdir}/python3.5/mimetypes.py",
 "${libdir}/python3.5/nntplib.py",
 "${libdir}/python3.5/poplib.py",
 "${libdir}/python3.5/smtplib.py",
 "${libdir}/python3.5/telnetlib.py",
 "${libdir}/python3.5/urllib",
-"${libdir}/python3.5/urllib/__pycache__",
 "${libdir}/python3.5/uuid.py"
 ],
 "rdepends": [
@@ -1088,11 +1088,12 @@
 "summary": "Python typing support"
 },
 "unittest": {
-"cached": [],
+"cached": [
+"${libdir}/python3.5/unittest/__pycache__"
+],
 "files": [
 "${libdir}/python3.5/unittest",
-"${libdir}/python3.5/unittest/",
-"${libdir}/python3.5/unittest/__pycache__"
+"${libdir}/python3.5/unittest/"
 ],
 "rdepends": [
 "core",
@@ -1132,11 +1133,12 @@
 "summary": "Python basic XML support"
 },
 "xmlrpc": {
-"cached": [],
-"files": [
-"${libdir}/python3.5/xmlrpc",
+"cached": [
 "${libdir}/python3.5/xmlrpc/__pycache__"
 ],
+"files": [
+"${libdir}/python3.5/xmlrpc"
+],
 "rdepends": [
 "core",
 "xml"
--
2.7.4

This email and any attachments are intended for the sole use of the named 
recipient(s) and contain(s) confidential information that may be proprietary, 
privileged or copyrighted under applicable law. If you are not the intended 
recipient, do not read, copy, or forward this email message or any attachments. 
Delete this email message and any attachments immediately.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] python3: consolidate sqlite3 files

[Alejandro Enedino Hernandez Samaniego]

Hey Ross,


On 03/28/2018 01:19 PM, Ross Burton wrote:

python3-sqlite3 happens to be a very special and unique package, its 
folder inside ${libdir}/python3.5/ called sqlite3, which would be fine,
we usually  handle those packages by adding the whole folder, but in 
this case, we separated the sqlite3-tests (which is a folder inside the 
sqlite3 folder)
into another package, so if we add the folder to sqlite3 there would be 
nothing for sqlite3-tests to get.



Some of the sqlite3 module was in python3-misc by accident, move the files
into python3-sqlite3 where they belong.

Signed-off-by: Ross Burton 
---
  meta/recipes-devtools/python/python3/python3-manifest.json | 7 +--
  1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index 1914a4c43ba..95071c53f5a 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -977,9 +977,12 @@
  "summary": "Python Simple Mail Transport Daemon"
  },
  "sqlite3": {
-"cached": [],
+"cached": [
+"${libdir}/python3.5/sqlite3/__pycache__/*.pyc"
+],
  "files": [
-"${libdir}/python3.5/lib-dynload/_sqlite3.*.so"
+"${libdir}/python3.5/lib-dynload/_sqlite3.*.so",


While this will work for splitting packages now, we dont support 
wildcards on the manifest file anymore,
at least not for the whole filename, because when we try to get 
dependencies for a module the
wildcard becomes useless, so this will likely kill the 
do_create_manifest task,
it'll go fine but the next time you run it, it wont get the module name 
it will disregard that dependency.

+"${libdir}/python3.5/sqlite3/*.py"
  ],
  "rdepends": [
  "core"


The right way would be to get rid of sqlite3-tests but I know we cant do 
that, so I'm working on a fix for the do_create_manifest task

that could handle this situation.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 1/2] package: Add INHIBIT_PACKAGE_STRIP_FILES to allow files to remain unstripped

[Alejandro Enedino Hernandez Samaniego]

Hey Richard,


Do we expect to backport this to older releases?


Cheers,


Alejandro


On 02/14/2018 07:07 AM, Richard Purdie wrote:

There are cases where its useful to allow only a select few files
to be excluded from the package stripping mechanism. Currently
this isn't possible so add a variable to allow this.

This is to be used sparingly as in general the core code should be
doing the right thing. This is better than the alternative of leaving the whole
package unstripped.

Signed-off-by: Richard Purdie 
---
  meta/classes/package.bbclass | 4 
  1 file changed, 4 insertions(+)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 6a7f35a..90e5236 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -946,6 +946,7 @@ python split_and_strip_files () {
  inodes = {}
  libdir = os.path.abspath(dvar + os.sep + d.getVar("libdir"))
  baselibdir = os.path.abspath(dvar + os.sep + d.getVar("base_libdir"))
+skipfiles = (d.getVar("INHIBIT_PACKAGE_STRIP_FILES") or "").split()
  if (d.getVar('INHIBIT_PACKAGE_STRIP') != '1' or \
  d.getVar('INHIBIT_PACKAGE_DEBUG_SPLIT') != '1'):
  for root, dirs, files in cpath.walk(dvar):
@@ -961,6 +962,9 @@ python split_and_strip_files () {
  if debugdir and debugdir in os.path.dirname(file[len(dvar):]):
  continue
  
+if file in skipfiles:

+continue
+
  try:
  ltarget = cpath.realpath(file, dvar, False)
  s = cpath.lstat(ltarget)


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] ✗ patchtest: failure for ninja: Upgrade from 1.7.2 to major release 1.8.2 (rev2)

[Alejandro Enedino Hernandez Samaniego]

Cool, thanks!

Alejandro

From: Burton, Ross [mailto:ross.bur...@intel.com]
Sent: Friday, February 2, 2018 1:35 PM
To: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] ✗ patchtest: failure for ninja: Upgrade from 1.7.2 to 
major release 1.8.2 (rev2)

So I'm 99% sure this is Exchange tramping the mail and git not handling that, 
as the patch is quoted-printable encoded.I'll pick from the branch :)

Ross

On 2 February 2018 at 21:28, Alejandro Enedino Hernandez Samaniego 
<alejandro.enedino.hernandez-samani...@xilinx.com<mailto:alejandro.enedino.hernandez-samani...@xilinx.com>>
 wrote:

I might be wrong but you can check it here as well:
https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=aehs29/ninja


On 02/02/2018 01:22 PM, Alejandro Enedino Hernandez Samaniego wrote:

This is what I got:



$ git remote show origin
* remote origin
  Fetch URL: git://git.yoctoproject.org/poky<http://git.yoctoproject.org/poky>
  Push  URL: git://git.yoctoproject.org/poky<http://git.yoctoproject.org/poky>
  HEAD branch: master

latest commit:

bca260ef4dca87c11fc9d30b2b8a17664dc2d879 ninja: Upgrade from 1.7.2 to major 
release 1.8.2
385944254d0ef88bd6450a221a54cbcb40020b42 tcmode-default.inc: drop preferred 
version of gzip-native



On 02/02/2018 12:58 PM, Burton, Ross wrote:
On 2 February 2018 at 19:06, Alejandro Enedino Hernandez Samaniego 
<aleja...@xilinx.com<mailto:aleja...@xilinx.com>> wrote:
This makes no sense to me, I rebased to master just before sending it.

Are you sure that the master you're rebasing is the actual upstream master?

Applying: ninja: Upgrade from 1.7.2 to major release 1.8.2
Using index info to reconstruct a base tree...
error: patch failed: 
meta/recipes-devtools/ninja/ninja_1.7.2.bb:1<http://ninja_1.7.2.bb:1>
error: meta/recipes-devtools/ninja/ninja_1.7.2.bb<http://ninja_1.7.2.bb>: patch 
does not apply
error: Did you hand edit your patch?
It does not apply to blobs recorded in its index.
Patch failed at 0001 ninja: Upgrade from 1.7.2 to major release 1.8.2





-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] ninja: Upgrade from 1.7.2 to major release 1.8.2

[Alejandro Enedino Hernandez Samaniego]

Hey Cal,

On 02/01/2018 02:10 PM, Alejandro Enedino Hernandez Samaniego wrote:
> Signed-off-by: Alejandro Enedino Hernandez Samaniego 
> <aleja...@xilinx.com>
> ---
>   meta/recipes-devtools/ninja/ninja_1.7.2.bb | 30 
> --
>   meta/recipes-devtools/ninja/ninja_1.8.2.bb | 30 
> ++
>   2 files changed, 30 insertions(+), 30 deletions(-)
>   delete mode 100644 meta/recipes-devtools/ninja/ninja_1.7.2.bb
>   create mode 100644 meta/recipes-devtools/ninja/ninja_1.8.2.bb
Adding the following to your .gitconfig will give you more readable patches for 
package upgrades:
[diff]
     renames = copies

It simplifies the delete -> creation to a rename, then shows what changed in 
the renamed file (SRC_URI and SRCREV).

Thanks,
Cal

Good to hear from you!, I thought git mv automatically did that, I was actually 
surprised it didnt but I didn’t spend too much time on that either.
Thanks!

Alejandro

>
> diff --git a/meta/recipes-devtools/ninja/ninja_1.7.2.bb 
> b/meta/recipes-devtools/ninja/ninja_1.7.2.bb
> deleted file mode 100644
> index 4d3b272..000
> --- a/meta/recipes-devtools/ninja/ninja_1.7.2.bb
> +++ /dev/null
> @@ -1,30 +0,0 @@
> -SUMMARY = "Ninja is a small build system with a focus on speed."
> -HOMEPAGE = "http://martine.github.com/ninja/;
> -LICENSE = "Apache-2.0"
> -LIC_FILES_CHKSUM = "file://COPYING;md5=a81586a64ad4e476c791cda7e2f2c52e"
> -
> -DEPENDS = "re2c-native ninja-native"
> -
> -SRCREV = "717b7b4a31db6027207588c0fb89c3ead384747b"
> -
> -SRC_URI = "git://github.com/martine/ninja.git;branch=release"
> -UPSTREAM_CHECK_GITTAGREGEX = "v(?P.*)"
> -
> -S = "${WORKDIR}/git"
> -
> -do_configure[noexec] = "1"
> -
> -do_compile_class-native() {
> -   ./configure.py --bootstrap
> -}
> -
> -do_compile() {
> -   ./configure.py
> -   ninja
> -}
> -
> -do_install() {
> -   install -D -m 0755  ${S}/ninja ${D}${bindir}/ninja
> -}
> -
> -BBCLASSEXTEND = "native nativesdk"
> diff --git a/meta/recipes-devtools/ninja/ninja_1.8.2.bb 
> b/meta/recipes-devtools/ninja/ninja_1.8.2.bb
> new file mode 100644
> index 000..c6fcfef
> --- /dev/null
> +++ b/meta/recipes-devtools/ninja/ninja_1.8.2.bb
> @@ -0,0 +1,30 @@
> +SUMMARY = "Ninja is a small build system with a focus on speed."
> +HOMEPAGE = "http://martine.github.com/ninja/;
> +LICENSE = "Apache-2.0"
> +LIC_FILES_CHKSUM = "file://COPYING;md5=a81586a64ad4e476c791cda7e2f2c52e"
> +
> +DEPENDS = "re2c-native ninja-native"
> +
> +SRCREV = "253e94c1fa511704baeb61cf69995bbf09ba435e"
> +
> +SRC_URI = "git://github.com/ninja-build/ninja.git;branch=release"
> +UPSTREAM_CHECK_GITTAGREGEX = "v(?P.*)"
> +
> +S = "${WORKDIR}/git"
> +
> +do_configure[noexec] = "1"
> +
> +do_compile_class-native() {
> +   ./configure.py --bootstrap
> +}
> +
> +do_compile() {
> +   ./configure.py
> +   ninja
> +}
> +
> +do_install() {
> +   install -D -m 0755  ${S}/ninja ${D}${bindir}/ninja }
> +
> +BBCLASSEXTEND = "native nativesdk"
> --
> 2.7.4
>
> This email and any attachments are intended for the sole use of the named 
> recipient(s) and contain(s) confidential information that may be proprietary, 
> privileged or copyrighted under applicable law. If you are not the intended 
> recipient, do not read, copy, or forward this email message or any 
> attachments. Delete this email message and any attachments immediately.

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] ✗ patchtest: failure for ninja: Upgrade from 1.7.2 to major release 1.8.2 (rev2)

[Alejandro Enedino Hernandez Samaniego]

I might be wrong but you can check it here as well:

https://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=aehs29/ninja


On 02/02/2018 01:22 PM, Alejandro Enedino Hernandez Samaniego wrote:


This is what I got:


$ git remote show origin
* remote origin
  Fetch URL: git://git.yoctoproject.org/poky
  Push  URL: git://git.yoctoproject.org/poky
  HEAD branch: master

latest commit:

bca260ef4dca87c11fc9d30b2b8a17664dc2d879 ninja: Upgrade from 1.7.2 to 
major release 1.8.2
385944254d0ef88bd6450a221a54cbcb40020b42 tcmode-default.inc: drop 
preferred version of gzip-native




On 02/02/2018 12:58 PM, Burton, Ross wrote:
On 2 February 2018 at 19:06, Alejandro Enedino Hernandez Samaniego 
<aleja...@xilinx.com <mailto:aleja...@xilinx.com>> wrote:


This makes no sense to me, I rebased to master just before
sending it.


Are you sure that the master you're rebasing is the actual upstream 
master?


Applying: ninja: Upgrade from 1.7.2 to major release 1.8.2
Using index info to reconstruct a base tree...
error: patch failed: meta/recipes-devtools/ninja/ninja_1.7.2.bb:1 
<http://ninja_1.7.2.bb:1>
error: meta/recipes-devtools/ninja/ninja_1.7.2.bb 
<http://ninja_1.7.2.bb>: patch does not apply

error: Did you hand edit your patch?
It does not apply to blobs recorded in its index.
Patch failed at 0001 ninja: Upgrade from 1.7.2 to major release 1.8.2





-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] ✗ patchtest: failure for ninja: Upgrade from 1.7.2 to major release 1.8.2 (rev2)

[Alejandro Enedino Hernandez Samaniego]

This is what I got:


$ git remote show origin
* remote origin
  Fetch URL: git://git.yoctoproject.org/poky
  Push  URL: git://git.yoctoproject.org/poky
  HEAD branch: master

latest commit:

bca260ef4dca87c11fc9d30b2b8a17664dc2d879 ninja: Upgrade from 1.7.2 to 
major release 1.8.2
385944254d0ef88bd6450a221a54cbcb40020b42 tcmode-default.inc: drop 
preferred version of gzip-native




On 02/02/2018 12:58 PM, Burton, Ross wrote:
On 2 February 2018 at 19:06, Alejandro Enedino Hernandez Samaniego 
<aleja...@xilinx.com <mailto:aleja...@xilinx.com>> wrote:


This makes no sense to me, I rebased to master just before sending it.


Are you sure that the master you're rebasing is the actual upstream 
master?


Applying: ninja: Upgrade from 1.7.2 to major release 1.8.2
Using index info to reconstruct a base tree...
error: patch failed: meta/recipes-devtools/ninja/ninja_1.7.2.bb:1 
<http://ninja_1.7.2.bb:1>
error: meta/recipes-devtools/ninja/ninja_1.7.2.bb 
<http://ninja_1.7.2.bb>: patch does not apply

error: Did you hand edit your patch?
It does not apply to blobs recorded in its index.
Patch failed at 0001 ninja: Upgrade from 1.7.2 to major release 1.8.2



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] ✗ patchtest: failure for ninja: Upgrade from 1.7.2 to major release 1.8.2 (rev2)

[Alejandro Enedino Hernandez Samaniego]

This makes no sense to me, I rebased to master just before sending it.

-Original Message-
From: Patchwork [mailto:patchw...@patchwork.openembedded.org] 
Sent: Thursday, February 1, 2018 2:33 PM
To: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>
Cc: openembedded-core@lists.openembedded.org
Subject: ✗ patchtest: failure for ninja: Upgrade from 1.7.2 to major release 
1.8.2 (rev2)

== Series Details ==

Series: ninja: Upgrade from 1.7.2 to major release 1.8.2 (rev2)
Revision: 2
URL   : https://patchwork.openembedded.org/series/10780/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is an 
automated response. Several tests have been executed on the proposed series by 
patchtest resulting in the following failures:



* Issue Series does not apply on top of target branch 
[test_series_merge_on_head] 
  Suggested fixRebase your series on top of targeted branch
  Targeted branch  master (currently at a0988c3374)



If you believe any of these test results are incorrect, please reply to the 
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new 
version of the patchset if applicable. Please ensure you add/increment the 
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> 
[PATCH v3] -> ...).

---
Guidelines: 
https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH v2] ninja: Upgrade from 1.7.2 to major release 1.8.2

[Alejandro Enedino Hernandez Samaniego]

Signed-off-by: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>
---
 meta/recipes-devtools/ninja/ninja_1.7.2.bb | 30 --
 meta/recipes-devtools/ninja/ninja_1.8.2.bb | 30 ++
 2 files changed, 30 insertions(+), 30 deletions(-)
 delete mode 100644 meta/recipes-devtools/ninja/ninja_1.7.2.bb
 create mode 100644 meta/recipes-devtools/ninja/ninja_1.8.2.bb

diff --git a/meta/recipes-devtools/ninja/ninja_1.7.2.bb 
b/meta/recipes-devtools/ninja/ninja_1.7.2.bb
deleted file mode 100644
index 4d3b272..000
--- a/meta/recipes-devtools/ninja/ninja_1.7.2.bb
+++ /dev/null
@@ -1,30 +0,0 @@
-SUMMARY = "Ninja is a small build system with a focus on speed."
-HOMEPAGE = "http://martine.github.com/ninja/;
-LICENSE = "Apache-2.0"
-LIC_FILES_CHKSUM = "file://COPYING;md5=a81586a64ad4e476c791cda7e2f2c52e"
-
-DEPENDS = "re2c-native ninja-native"
-
-SRCREV = "717b7b4a31db6027207588c0fb89c3ead384747b"
-
-SRC_URI = "git://github.com/martine/ninja.git;branch=release"
-UPSTREAM_CHECK_GITTAGREGEX = "v(?P.*)"
-
-S = "${WORKDIR}/git"
-
-do_configure[noexec] = "1"
-
-do_compile_class-native() {
-   ./configure.py --bootstrap
-}
-
-do_compile() {
-   ./configure.py
-   ninja
-}
-
-do_install() {
-   install -D -m 0755  ${S}/ninja ${D}${bindir}/ninja
-}
-
-BBCLASSEXTEND = "native nativesdk"
diff --git a/meta/recipes-devtools/ninja/ninja_1.8.2.bb 
b/meta/recipes-devtools/ninja/ninja_1.8.2.bb
new file mode 100644
index 000..c6fcfef
--- /dev/null
+++ b/meta/recipes-devtools/ninja/ninja_1.8.2.bb
@@ -0,0 +1,30 @@
+SUMMARY = "Ninja is a small build system with a focus on speed."
+HOMEPAGE = "http://martine.github.com/ninja/;
+LICENSE = "Apache-2.0"
+LIC_FILES_CHKSUM = "file://COPYING;md5=a81586a64ad4e476c791cda7e2f2c52e"
+
+DEPENDS = "re2c-native ninja-native"
+
+SRCREV = "253e94c1fa511704baeb61cf69995bbf09ba435e"
+
+SRC_URI = "git://github.com/ninja-build/ninja.git;branch=release"
+UPSTREAM_CHECK_GITTAGREGEX = "v(?P.*)"
+
+S = "${WORKDIR}/git"
+
+do_configure[noexec] = "1"
+
+do_compile_class-native() {
+   ./configure.py --bootstrap
+}
+
+do_compile() {
+   ./configure.py
+   ninja
+}
+
+do_install() {
+   install -D -m 0755  ${S}/ninja ${D}${bindir}/ninja
+}
+
+BBCLASSEXTEND = "native nativesdk"
--
2.7.4

This email and any attachments are intended for the sole use of the named 
recipient(s) and contain(s) confidential information that may be proprietary, 
privileged or copyrighted under applicable law. If you are not the intended 
recipient, do not read, copy, or forward this email message or any attachments. 
Delete this email message and any attachments immediately.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] ninja: Upgrade from 1.7.2 to major release 1.8.2

[Alejandro Enedino Hernandez Samaniego]

Sure, sorry for some reason I sent it to oe-core-bounce or something 
last week.


Interestingly I just rebased it on master cleanly...



On 01/31/2018 03:57 AM, Burton, Ross wrote:

Patch doesn't apply, can you rebase and resend please.

Ross

On 30 January 2018 at 21:09, Alejandro Enedino Hernandez Samaniego 
<alejandro.enedino.hernandez-samani...@xilinx.com 
<mailto:alejandro.enedino.hernandez-samani...@xilinx.com>> wrote:


Signed-off-by: Alejandro Enedino Hernandez Samaniego
<aleja...@xilinx.com <mailto:aleja...@xilinx.com>>

---
 meta/recipes-devtools/ninja/ninja_1.7.2.bb
<http://ninja_1.7.2.bb> | 30 --
 meta/recipes-devtools/ninja/ninja_1.8.2.bb
<http://ninja_1.8.2.bb> | 30 ++
 2 files changed, 30 insertions(+), 30 deletions(-)
 delete mode 100644 meta/recipes-devtools/ninja/ninja_1.7.2.bb
<http://ninja_1.7.2.bb>
 create mode 100644 meta/recipes-devtools/ninja/ninja_1.8.2.bb
<http://ninja_1.8.2.bb>

diff --git a/meta/recipes-devtools/ninja/ninja_1.7.2.bb
<http://ninja_1.7.2.bb>
b/meta/recipes-devtools/ninja/ninja_1.7.2.bb <http://ninja_1.7.2.bb>
deleted file mode 100644
index 4d3b272..000
--- a/meta/recipes-devtools/ninja/ninja_1.7.2.bb
<http://ninja_1.7.2.bb>
+++ /dev/null
@@ -1,30 +0,0 @@
-SUMMARY = "Ninja is a small build system with a focus on speed."
-HOMEPAGE = "http://martine.github.com/ninja/
<http://martine.github.com/ninja/>"
-LICENSE = "Apache-2.0"
-LIC_FILES_CHKSUM =
"file://COPYING;md5=a81586a64ad4e476c791cda7e2f2c52e"
-
-DEPENDS = "re2c-native ninja-native"
-
-SRCREV = "717b7b4a31db6027207588c0fb89c3ead384747b"
-
-SRC_URI = "git://github.com/martine/ninja.git;branch=release
<http://github.com/martine/ninja.git;branch=release>"
-UPSTREAM_CHECK_GITTAGREGEX = "v(?P.*)"
-
-S = "${WORKDIR}/git"
-
-do_configure[noexec] = "1"
-
-do_compile_class-native() {
-   ./configure.py --bootstrap
-}
-
-do_compile() {
-   ./configure.py
-   ninja
-}
-
-do_install() {
-   install -D -m 0755  ${S}/ninja ${D}${bindir}/ninja
-}
-
-BBCLASSEXTEND = "native nativesdk"
diff --git a/meta/recipes-devtools/ninja/ninja_1.8.2.bb
<http://ninja_1.8.2.bb>
b/meta/recipes-devtools/ninja/ninja_1.8.2.bb <http://ninja_1.8.2.bb>
new file mode 100644
index 000..c6fcfef
--- /dev/null
+++ b/meta/recipes-devtools/ninja/ninja_1.8.2.bb
<http://ninja_1.8.2.bb>
@@ -0,0 +1,30 @@
+SUMMARY = "Ninja is a small build system with a focus on speed."
+HOMEPAGE = "http://martine.github.com/ninja/
<http://martine.github.com/ninja/>"
+LICENSE = "Apache-2.0"
+LIC_FILES_CHKSUM =
"file://COPYING;md5=a81586a64ad4e476c791cda7e2f2c52e"
+
+DEPENDS = "re2c-native ninja-native"
+
+SRCREV = "253e94c1fa511704baeb61cf69995bbf09ba435e"
+
+SRC_URI = "git://github.com/ninja-build/ninja.git;branch=release
<http://github.com/ninja-build/ninja.git;branch=release>"
+UPSTREAM_CHECK_GITTAGREGEX = "v(?P.*)"
+
+S = "${WORKDIR}/git"
+
+do_configure[noexec] = "1"
+
+do_compile_class-native() {
+   ./configure.py --bootstrap
+}
+
+do_compile() {
+   ./configure.py
+   ninja
+}
+
+do_install() {
+   install -D -m 0755  ${S}/ninja ${D}${bindir}/ninja
+}
+
+BBCLASSEXTEND = "native nativesdk"
--
2.7.4

This email and any attachments are intended for the sole use of
the named recipient(s) and contain(s) confidential information
that may be proprietary, privileged or copyrighted under
applicable law. If you are not the intended recipient, do not
read, copy, or forward this email message or any attachments.
Delete this email message and any attachments immediately.
-- 
___

Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
<mailto:Openembedded-core@lists.openembedded.org>
http://lists.openembedded.org/mailman/listinfo/openembedded-core
<http://lists.openembedded.org/mailman/listinfo/openembedded-core>




-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] ninja: Upgrade from 1.7.2 to major release 1.8.2

[Alejandro Enedino Hernandez Samaniego]

Signed-off-by: Alejandro Enedino Hernandez Samaniego <aleja...@xilinx.com>

---
 meta/recipes-devtools/ninja/ninja_1.7.2.bb | 30 --
 meta/recipes-devtools/ninja/ninja_1.8.2.bb | 30 ++
 2 files changed, 30 insertions(+), 30 deletions(-)
 delete mode 100644 meta/recipes-devtools/ninja/ninja_1.7.2.bb
 create mode 100644 meta/recipes-devtools/ninja/ninja_1.8.2.bb

diff --git a/meta/recipes-devtools/ninja/ninja_1.7.2.bb 
b/meta/recipes-devtools/ninja/ninja_1.7.2.bb
deleted file mode 100644
index 4d3b272..000
--- a/meta/recipes-devtools/ninja/ninja_1.7.2.bb
+++ /dev/null
@@ -1,30 +0,0 @@
-SUMMARY = "Ninja is a small build system with a focus on speed."
-HOMEPAGE = "http://martine.github.com/ninja/;
-LICENSE = "Apache-2.0"
-LIC_FILES_CHKSUM = "file://COPYING;md5=a81586a64ad4e476c791cda7e2f2c52e"
-
-DEPENDS = "re2c-native ninja-native"
-
-SRCREV = "717b7b4a31db6027207588c0fb89c3ead384747b"
-
-SRC_URI = "git://github.com/martine/ninja.git;branch=release"
-UPSTREAM_CHECK_GITTAGREGEX = "v(?P.*)"
-
-S = "${WORKDIR}/git"
-
-do_configure[noexec] = "1"
-
-do_compile_class-native() {
-   ./configure.py --bootstrap
-}
-
-do_compile() {
-   ./configure.py
-   ninja
-}
-
-do_install() {
-   install -D -m 0755  ${S}/ninja ${D}${bindir}/ninja
-}
-
-BBCLASSEXTEND = "native nativesdk"
diff --git a/meta/recipes-devtools/ninja/ninja_1.8.2.bb 
b/meta/recipes-devtools/ninja/ninja_1.8.2.bb
new file mode 100644
index 000..c6fcfef
--- /dev/null
+++ b/meta/recipes-devtools/ninja/ninja_1.8.2.bb
@@ -0,0 +1,30 @@
+SUMMARY = "Ninja is a small build system with a focus on speed."
+HOMEPAGE = "http://martine.github.com/ninja/;
+LICENSE = "Apache-2.0"
+LIC_FILES_CHKSUM = "file://COPYING;md5=a81586a64ad4e476c791cda7e2f2c52e"
+
+DEPENDS = "re2c-native ninja-native"
+
+SRCREV = "253e94c1fa511704baeb61cf69995bbf09ba435e"
+
+SRC_URI = "git://github.com/ninja-build/ninja.git;branch=release"
+UPSTREAM_CHECK_GITTAGREGEX = "v(?P.*)"
+
+S = "${WORKDIR}/git"
+
+do_configure[noexec] = "1"
+
+do_compile_class-native() {
+   ./configure.py --bootstrap
+}
+
+do_compile() {
+   ./configure.py
+   ninja
+}
+
+do_install() {
+   install -D -m 0755  ${S}/ninja ${D}${bindir}/ninja
+}
+
+BBCLASSEXTEND = "native nativesdk"
--
2.7.4

This email and any attachments are intended for the sole use of the named 
recipient(s) and contain(s) confidential information that may be proprietary, 
privileged or copyrighted under applicable law. If you are not the intended 
recipient, do not read, copy, or forward this email message or any attachments. 
Delete this email message and any attachments immediately.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] python: Upgrade both python and python-native to 2.7.14

[Alejandro Enedino Hernandez Samaniego]

Awesome!, it seems that the script did save us from a couple of annoying 
RDEPEND bugs on this new release!



Thanks Derek!


On 01/29/2018 02:26 PM, Derek Straka wrote:
Thanks.  Last I checked it was just in master-next.  I'll send the 
patch for the manifest once I get off the train.


On Jan 29, 2018 5:18 PM, "Richard Purdie" 
> wrote:


On Mon, 2018-01-29 at 16:49 -0500, Derek Straka wrote:
> Rebased:
>   - python/01-use-proper-tools-for-cross-build.patch
>   - python/fix-makefile-for-ptest.patch
>   - python/parallel-makeinst-create-bindir.patch
>
> Removed Upstreamed Patch:
>   - python/Don-t-use-getentropy-on-Linux.patch
>
> Updated license checksum for changes in the copyright date.  The
> license
> terms remain unchanged
>
> Added an extra do_compile item to create the native pgen that no
> longer
> gets compiled by default
>
> Update the json manifest with the latest dependency set
>
> Signed-off-by: Derek Straka >

Your patch was merged into master so if there are tweaks they need to
be in a patch against master now please.

Cheers,

Richard





-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH] python: Upgrade both python and python-native to 2.7.14

[Alejandro Enedino Hernandez Samaniego]

Hey Derek,


Just a quick question, did you run the create manifest task along with 
the upgrade?



Cheers,


Alejandro


On 01/26/2018 11:25 AM, Derek Straka wrote:

Rebased:
   - python/01-use-proper-tools-for-cross-build.patch
   - python/fix-makefile-for-ptest.patch
   - python/parallel-makeinst-create-bindir.patch

Removed Upstreamed Patch:
   - python/Don-t-use-getentropy-on-Linux.patch

Updated license checksum for changes in the copyright date.  The license
terms remain unchanged

Added an extra do_compile item to create the native pgen that no longer
gets compiled by default

Signed-off-by: Derek Straka 
---
  ...on-native_2.7.13.bb => python-native_2.7.14.bb} |  6 
  meta/recipes-devtools/python/python.inc|  6 ++--
  .../01-use-proper-tools-for-cross-build.patch  | 17 +
  .../python/Don-t-use-getentropy-on-Linux.patch | 41 --
  .../python/python/fix-makefile-for-ptest.patch |  6 ++--
  .../python/parallel-makeinst-create-bindir.patch   | 12 +++
  .../python/{python_2.7.13.bb => python_2.7.14.bb}  |  1 -
  7 files changed, 28 insertions(+), 61 deletions(-)
  rename meta/recipes-devtools/python/{python-native_2.7.13.bb => 
python-native_2.7.14.bb} (94%)
  delete mode 100644 
meta/recipes-devtools/python/python/Don-t-use-getentropy-on-Linux.patch
  rename meta/recipes-devtools/python/{python_2.7.13.bb => python_2.7.14.bb} 
(99%)

diff --git a/meta/recipes-devtools/python/python-native_2.7.13.bb 
b/meta/recipes-devtools/python/python-native_2.7.14.bb
similarity index 94%
rename from meta/recipes-devtools/python/python-native_2.7.13.bb
rename to meta/recipes-devtools/python/python-native_2.7.14.bb
index 2b74003..622cb38 100644
--- a/meta/recipes-devtools/python/python-native_2.7.13.bb
+++ b/meta/recipes-devtools/python/python-native_2.7.14.bb
@@ -36,6 +36,12 @@ do_configure_append() {
autoreconf --verbose --install --force --exclude=autopoint 
../Python-${PV}/Modules/_ctypes/libffi
  }
  
+# Regenerate all of the generated files

+# This ensures that pgen and friends get created during the compile phase
+do_compile_prepend() {
+oe_runmake regen-all
+}
+
  do_install() {
oe_runmake 'DESTDIR=${D}' install
install -d ${D}${bindir}/${PN}
diff --git a/meta/recipes-devtools/python/python.inc 
b/meta/recipes-devtools/python/python.inc
index b40f551..979b601 100644
--- a/meta/recipes-devtools/python/python.inc
+++ b/meta/recipes-devtools/python/python.inc
@@ -5,12 +5,12 @@ SECTION = "devel/python"
  # bump this on every change in contrib/python/generate-manifest-2.7.py
  INC_PR = "r1"
  
-LIC_FILES_CHKSUM = "file://LICENSE;md5=6b60258130e4ed10d3101517eb5b9385"

+LIC_FILES_CHKSUM = "file://LICENSE;md5=f741e51de91d4eeea5930b9c3c7fa69d"
  
  SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz;
  
-SRC_URI[md5sum] = "53b43534153bb2a0363f08bae8b9d990"

-SRC_URI[sha256sum] = 
"35d543986882f78261f97787fd3e06274bfa6df29fac9b4a94f73930ff98f731"
+SRC_URI[md5sum] = "1f6db41ad91d9eb0a6f0c769b8613c5b"
+SRC_URI[sha256sum] = 
"71ffb26e09e78650e424929b2b457b9c912ac216576e6bd9e7d204ed03296a66"
  
  # python recipe is actually python 2.x

  # also, exclude pre-releases for both python 2.x and 3.x
diff --git 
a/meta/recipes-devtools/python/python/01-use-proper-tools-for-cross-build.patch 
b/meta/recipes-devtools/python/python/01-use-proper-tools-for-cross-build.patch
index 366ce3e..e795a74 100644
--- 
a/meta/recipes-devtools/python/python/01-use-proper-tools-for-cross-build.patch
+++ 
b/meta/recipes-devtools/python/python/01-use-proper-tools-for-cross-build.patch
@@ -9,6 +9,9 @@ Signed-off-by: Paul Eggleton 
  Rebased for python-2.7.9
  Signed-off-by: Alejandro Hernandez 
  
+Rebased for python-2.7.14

+Signed-off-by: Derek Straka 
+
  Index: Python-2.7.13/Makefile.pre.in
  ===
  --- Python-2.7.13.orig/Makefile.pre.in
@@ -30,14 +33,14 @@ Index: Python-2.7.13/Makefile.pre.in
   
   # Create build directory and generate the sysconfig build-time data there.

   # pybuilddir.txt contains the name of the build dir and is used for
-@@ -681,7 +682,7 @@ Modules/pwdmodule.o: $(srcdir)/Modules/p
-
- $(GRAMMAR_H): @GENERATED_COMMENT@ $(GRAMMAR_INPUT) $(PGEN)
+@@ -663,7 +663,7 @@
+   # Regenerate Include/graminit.h and Python/graminit.c
+   # from Grammar/Grammar using pgen
@$(MKDIR_P) Include
--  $(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C)
-+  $(HOSTPGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C)
- $(GRAMMAR_C): @GENERATED_COMMENT@ $(GRAMMAR_H)
-   touch $(GRAMMAR_C)
+-  $(PGEN) $(srcdir)/Grammar/Grammar \
++  $(HOSTPGEN) $(srcdir)/Grammar/Grammar \
+   $(srcdir)/Include/graminit.h \
+   $(srcdir)/Python/graminit.c
   
  @@ -1121,27 +1122,27 @@ libinstall:	build_all $(srcdir)/Lib/$(PL

 

Re: [OE-core] [oe-commits] [openembedded-core] 01/06: python: Restructure python packaging and replace it with autopackaging

[Alejandro Enedino Hernandez Samaniego]

Hey Paul,


On 01/22/2018 12:28 PM, Paul Eggleton wrote:

Hi Alejandro,

On Tuesday, 23 January 2018 8:02:07 AM NZDT Alejandro Enedino Hernandez 
Samaniego wrote:

Its backwards compatible, in the way that the usage should be the same,
in this case, this sort of failures are expected, since the mentioned
packages: enum, readline, textutils no longer exist, and they should
never have existed in the first place, they were always required to get
a minimal python working, so they are now part of python(3)-core, which
means the dependencies are actually met but recipes which RDEPEND on
them should be modified , please see
https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=7aaffbede46ce352b91c7a5c5f825a6f3b5b1c2e
regarding some of the RDEPEND fixes on oe-core, I expect there should be
several like this for meta-python.
I'd rather go with B) if thats ok, since the actual dependencies are 
being met and basically deleting the dependency for which its 
complaining from RDEPENDS would fix things.

It seems to me we need to do one of two things:

A) Add explicit RPROVIDES so that the core package provides these things
for compatibility with existing recipes

or

B) Add something to the migration guide to explain what users need to do in
order to fix their recipes after this change - the staging area for these is on 
the wiki
(doesn't have to be fully fleshed out, just the bare details would be 
sufficient)
   https://wiki.yoctoproject.org/wiki/FutureMigrationGuide

Could you please take care of this?

Sure, I can do this!


Thanks,
Paul



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/2] cmake: use Ninja by default

[Alejandro Enedino Hernandez Samaniego]

On 01/18/2018 04:38 AM, Ross Burton wrote:
> This changes the cmake class to use Ninja instead of Make by default.
>
> If a recipe is broken with Ninja then the recipe can set 
> OECMAKE_GENERATOR="Unix
> Makefiles" to change back to Make.
>
> Signed-off-by: Ross Burton 
> ---
>   meta/classes/cmake.bbclass | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
> index 74a952142ff..d60dad8d66b 100644
> --- a/meta/classes/cmake.bbclass
> +++ b/meta/classes/cmake.bbclass
> @@ -9,7 +9,7 @@ CCACHE = ""
>   
>   # What CMake generator to use.
>   # The supported options are "Unix Makefiles" or "Ninja".
> -OECMAKE_GENERATOR ?= "Unix Makefiles"
> +OECMAKE_GENERATOR ?= "Ninja"
>   
>   python() {
>   generator = d.getVar("OECMAKE_GENERATOR")

Awesome, testing it now!

Alejandro
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [meta-xilinx] liblzma: Memory allocation failed on do_package_rpm

[Alejandro Enedino Hernandez Samaniego]

Hey Manju,

I'd like to see if I can reproduce this, could you please send the the steps 
you followed?

Thanks!

Alejandro

-Original Message-
From: meta-xilinx-boun...@yoctoproject.org 
[mailto:meta-xilinx-boun...@yoctoproject.org] On Behalf Of Manjukumar 
Harthikote Matha
Sent: Sunday, January 14, 2018 10:20 PM
To: openembedded-core@lists.openembedded.org; yo...@yoctoproject.org; 
meta-xil...@yoctoproject.org
Subject: [meta-xilinx] liblzma: Memory allocation failed on do_package_rpm

All,

Has anybody seen this error?

Finished binary package job, result 0, filename (null)
error: create archive failed: cpio: write failed - Cannot allocate memory
error: liblzma: Memory allocation failederror: liblzma: Memory allocation 
failedFinished binary package job, result 2,

...

RPM build errors:
| Deprecated external dependency generator is used!
| Deprecated external dependency generator is used!
| Deprecated external dependency generator is used!
| Deprecated external dependency generator is used!
| Deprecated external dependency generator is used!
| Deprecated external dependency generator is used!
| Deprecated external dependency generator is used!
| Deprecated external dependency generator is used!
| Deprecated external dependency generator is used!
| liblzma: Memory allocation failedliblzma: Memory allocation failed
liblzma: Memory allocation failedliblzma: Memory allocation failed
liblzma: Memory allocation failedliblzma: Memory allocation failed

This happens in do_package_rpm task on various recipes, 
coreutils/libgpg/diffutils etc.

The host machine has good amount on memory, 256G to be exact. When I see the 
error and issue free -h command, I still see ~120G free memory available.
This is a RHEL 7.2 machine and I am running rocko baseline.

Any reason why this would happen? Where should I look to fix this issue?

Thanks,
Manju
This email and any attachments are intended for the sole use of the named 
recipient(s) and contain(s) confidential information that may be proprietary, 
privileged or copyrighted under applicable law. If you are not the intended 
recipient, do not read, copy, or forward this email message or any attachments. 
Delete this email message and any attachments immediately.
-- 
___
meta-xilinx mailing list
meta-xil...@yoctoproject.org
https://lists.yoctoproject.org/listinfo/meta-xilinx
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 2/2] python3: Fix native compilation of gdbm module and manifest

[Alejandro Enedino Hernandez Samaniego]

From: Alejandro Hernandez 

The gdbm module wasnt being built on python3-native showing the following
error during compilation:

Failed to build these modules:
_gdbm

This patch adds the required dependency to fix the compilation problem.

This issue on python3-native caused the manifest creation script to be
unaware of the gdbm library, so this patch also fixes the create_manifest
task for target python, and the manifest file to reflect the changes on
target python as well.

Signed-off-by: Alejandro Hernandez 
---
 meta/recipes-devtools/python/python3-native_3.5.3.bb  |  2 +-
 .../recipes-devtools/python/python3/python3-manifest.json | 15 +--
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-devtools/python/python3-native_3.5.3.bb 
b/meta/recipes-devtools/python/python3-native_3.5.3.bb
index 3053c4b..12f9f24 100644
--- a/meta/recipes-devtools/python/python3-native_3.5.3.bb
+++ b/meta/recipes-devtools/python/python3-native_3.5.3.bb
@@ -38,7 +38,7 @@ UPSTREAM_CHECK_REGEX = "[Pp]ython-(?P\d+(\.\d+)+).tar"
 S = "${WORKDIR}/Python-${PV}"

 EXTRANATIVEPATH += "bzip2-native"
-DEPENDS = "openssl-native bzip2-replacement-native zlib-native readline-native 
sqlite3-native"
+DEPENDS = "openssl-native bzip2-replacement-native zlib-native readline-native 
sqlite3-native gdbm-native"

 inherit native

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index 031745c..be020f8 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -366,7 +366,8 @@
 "db": {
 "cached": [],
 "files": [
-"${libdir}/python3.5/dbm"
+"${libdir}/python3.5/dbm",
+"${libdir}/python3.5/lib-dynload/_dbm.*.so"
 ],
 "rdepends": [
 "core"
@@ -493,6 +494,16 @@
 ],
 "summary": "Python's fcntl interface"
 },
+"gdbm": {
+"cached": [],
+"files": [
+"${libdir}/python3.5/lib-dynload/_gdbm.*.so"
+],
+"rdepends": [
+"core"
+],
+"summary": "Python GNU database support"
+},
 "html": {
 "cached": [
 "${libdir}/python3.5/__pycache__/formatter.*.pyc"
@@ -1128,4 +1139,4 @@
 ],
 "summary": "Python XML-RPC support"
 }
-}
+}
\ No newline at end of file
--
2.7.4

This email and any attachments are intended for the sole use of the named 
recipient(s) and contain(s) confidential information that may be proprietary, 
privileged or copyrighted under applicable law. If you are not the intended 
recipient, do not read, copy, or forward this email message or any attachments. 
Delete this email message and any attachments immediately.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH 1/2] python3-manifest.json: Add some binaries that were left out

[Alejandro Enedino Hernandez Samaniego]

From: Alejandro Hernandez 

With the introduction of a new manifest, since it had to be done manually
some binaries were left out, this patch adds them to their corresponding package
to fix the issue

Signed-off-by: Alejandro Hernandez 
---
 meta/recipes-devtools/python/python3/python3-manifest.json | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json 
b/meta/recipes-devtools/python/python3/python3-manifest.json
index 72d95d8..031745c 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -2,6 +2,7 @@
 "2to3": {
 "cached": [],
 "files": [
+"${bindir}/2to3-*",
 "${libdir}/python3.5/lib2to3"
 ],
 "rdepends": [
@@ -508,7 +509,7 @@
 "idle": {
 "cached": [],
 "files": [
-"${bindir}/idle",
+"${bindir}/idle*",
 "${libdir}/python3.5/idlelib"
 ],
 "rdepends": [
@@ -881,7 +882,7 @@
 "${libdir}/python3.5/__pycache__/pydoc.*.pyc"
 ],
 "files": [
-"${bindir}/pydoc",
+"${bindir}/pydoc*",
 "${libdir}/python3.5/pydoc.py",
 "${libdir}/python3.5/pydoc_data"
 ],
--
2.7.4

This email and any attachments are intended for the sole use of the named 
recipient(s) and contain(s) confidential information that may be proprietary, 
privileged or copyrighted under applicable law. If you are not the intended 
recipient, do not read, copy, or forward this email message or any attachments. 
Delete this email message and any attachments immediately.
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core