Re: [yocto] extensible SDK build failure

[RUSSELL PETERSON]

Frustrating in that I can't get the eSDK to fully build. I'm past the issue
with kernel-devsrc and harfbuzz... but now I see hundreds of messages like
below. Anyone seen this before? From what I can tell these tasks are being
executed out of the run queue.  Not all the messages are from cve-check but 
most.

Anyone seen this?



ERROR: Task attr-native.do_cve_check attempted to execute unexpectedly
ERROR: Task binutils-native.do_cve_check attempted to execute unexpectedly
ERROR: Task dbus-glib.do_cve_check attempted to execute unexpectedly
ERROR: Task libtool-native.do_cve_check attempted to execute unexpectedly
ERROR: Task fixesproto.do_cve_check attempted to execute unexpectedly
ERROR: Task bc.do_cve_check attempted to execute unexpectedly
ERROR: Task lsof.do_cve_check attempted to execute unexpectedly
ERROR: Task python-setuptools-native.do_cve_check attempted to execute
unexpectedly
ERROR: Task libxrandr-native.do_cve_check attempted to execute unexpectedly
ERROR: Task libpciaccess.do_cve_check attempted to execute unexpectedly
ERROR: Task xf86driproto.do_cve_check attempted to execute unexpectedly
ERROR: Task ncurses-native.do_cve_check attempted to execute unexpectedly

On August 1, 2017 at 8:55 PM Russell Peterson <bluehil...@comcast.net> wrote:

FYI: For those interested…

Just as a test/workaround I added a harfbuzz_%.bbappend file to my meta layer
and directly set the acpaths variable to the STAGING native directory and that
seems to work fine. Works for now but I will try to come up with a cleaner,
more generic fix.

acpaths = “-I ${STAGING_DATADIR_NATIVE}/aclocal/“

Regards,

Russell

> 
> On Aug 1, 2017, at 5:52 PM, RUSSELL PETERSON <bluehil...@comcast.net> 
> wrote:
> 
> Thank you for the response, Paul. You were correct that I had the
> TOOLCHAIN_*_TASK variables set in my machine.conf file. I put it into my
> image bb file and things seem far more sane... although I must admit I am
> not 100% sure why it works but I will study it a bit and figure it out.
> Thanks for your help.
> 
> I have run into another issue while trying to generate the eSDK in that
> there appears to be an issue with building harfbuzz. From what I can tell,
> this appears to be an issue with the search paths for the autoreconf call.
> The autoreconf tool is finding the pkg.m4 file in the harfbuzz m4 
> directory
> FIRST and therefore not getting the one in
> harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/share/aclocal which indeed has
> the correct m4_pattern_allow.
> 
> Do you think that's a bug or something with my recipe? Seems like I just
> want to reverse the -I options... but I don't know how, exactly.
> 
> 
> /scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/bin/autoconf
> 
> --include=/scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/harfbuzz-1.4.1/m4/
> 
> --include=/scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/share/aclocal/
> --force
> 
> ../../../autoconf-2.69/lib/autoconf/specific.m4:368:
> AC_USE_SYSTEM_EXTENSIONS is expanded from...
> | configure.ac:22: the top level
> | configure:18902: error: possibly undefined macro: PKG_CONFIG_SYSROOT_DIR
> | If this token and others are legitimate, please use m4_pattern_allow.
> | See the Autoconf documentation.
> | autoreconf:
> 
> /scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/bin/autoconf
> failed with exit status: 1
> | ERROR: Function failed: do_configure (log file is located at
> 
> /scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/temp/log.do_configure.14956)
> 
> > > 
> > On July 31, 2017 at 9:50 AM Paul Eggleton 
> > <paul.eggle...@linux.intel.com>
> > wrote:
> > 
> > Hi Russell,
> > 
> > It looks to me like you have added kernel-devsrc to the 
> > TOOLCHAIN_*_TASK
> > variable at the global level, which you really don't want to do if 
> > you're
> > going to be building buildtools-tarball (which eSDK will build as a
> > dependency). I'd suggest moving that setting to your image recipe.
> > 
> > Cheers,
> > Paul
> > 
> > On Sunday, 30 July 2017 4:22:24 PM CEST Russell Peterson wrote:
> > 
> > > > > 
> > > Hello, all.
> > > 
> > > I’m trying to build the eSDK for my platform and I just can’t 
> > > figure out
> > > why dnf can’t find kernel-devsrc. It’s there, I added it. The 
> 

Re: [yocto] extensible SDK build failure

[Russell Peterson]

FYI: For those interested…

Just as a test/workaround I added a harfbuzz_%.bbappend file to my meta layer 
and directly set the acpaths variable to the STAGING native directory and that 
seems to work fine.  Works for now but I will try to come up with a cleaner, 
more generic fix.

acpaths = “-I ${STAGING_DATADIR_NATIVE}/aclocal/“


Regards,

Russell


> On Aug 1, 2017, at 5:52 PM, RUSSELL PETERSON <bluehil...@comcast.net> wrote:
> 
> Thank you for the response, Paul. You were correct that I had the 
> TOOLCHAIN_*_TASK variables set in my machine.conf file. I put it into my 
> image bb file and things seem far more sane... although I must admit I am not 
> 100% sure why it works but I will study it a bit and figure it out. Thanks 
> for your help.
> 
> I have run into another issue while trying to generate the eSDK in that there 
> appears to be an issue with building harfbuzz. From what I can tell, this 
> appears to be an issue with the search paths for the autoreconf call. The 
> autoreconf tool is finding the pkg.m4 file in the harfbuzz m4 directory FIRST 
> and therefore not getting the one in 
> harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/share/aclocal which indeed has 
> the correct m4_pattern_allow.
> 
> Do you think that's a bug or something with my recipe? Seems like I just want 
> to reverse the -I options... but I don't know how, exactly.
> 
> /scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/bin/autoconf
>  
> --include=/scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/harfbuzz-1.4.1/m4/
>  
> --include=/scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/share/aclocal/
>  --force
> 
> ../../../autoconf-2.69/lib/autoconf/specific.m4:368: AC_USE_SYSTEM_EXTENSIONS 
> is expanded from...
> | configure.ac:22: the top level
> | configure:18902: error: possibly undefined macro: PKG_CONFIG_SYSROOT_DIR
> | If this token and others are legitimate, please use m4_pattern_allow.
> | See the Autoconf documentation.
> | autoreconf: 
> /scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/bin/autoconf
>  failed with exit status: 1
> | ERROR: Function failed: do_configure (log file is located at 
> /scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/temp/log.do_configure.14956)
> 
> > On July 31, 2017 at 9:50 AM Paul Eggleton <paul.eggle...@linux.intel.com> 
> > wrote:
> > 
> > 
> > Hi Russell,
> > 
> > It looks to me like you have added kernel-devsrc to the TOOLCHAIN_*_TASK 
> > variable at the global level, which you really don't want to do if you're 
> > going to be building buildtools-tarball (which eSDK will build as a 
> > dependency). I'd suggest moving that setting to your image recipe.
> > 
> > Cheers,
> > Paul
> > 
> > On Sunday, 30 July 2017 4:22:24 PM CEST Russell Peterson wrote:
> > > Hello, all.
> > > 
> > > I’m trying to build the eSDK for my platform and I just can’t figure out 
> > > why dnf can’t find kernel-devsrc. It’s there, I added it. The “normal 
> > > SDK” uses it and builds fine. Just no luck with the eSDK.
> > > Any help out there? Poky is pyro. aarch64 SoC target platform with an x86 
> > > build host (CentOS 7.3). Basically I just bitbake -c do_populate_sdk_ext 
> > > core-image-full. I do have a linux kernel recipe that pulls the kernel 
> > > from the standard kernel.org <http://kernel.org/> repo.
> > > 
> > > ERROR: buildtools-tarball-1.0-r0 do_populate_sdk: Could not invoke dnf. 
> > > Command 
> > > '/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/recipe-sysroot-native/usr/bin/dnf
> > >  -y -c 
> > > /scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/sdk/image/opt/poky/2.3.1/sysroots/none/etc/dnf/dnf.conf
> > >  
> > > --setopt=reposdir=/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/sdk/image/opt/poky/2.3.1/sysroots/none/etc/yum.repos.d
> > >  
> > > --repofrompath=oe-repo,/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/oe-rootfs-repo
> > >  
> > > --installroot=/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/sdk/image/opt/poky/2.3.1/sysroots/none
> > >  
> > > --setopt=logdir=/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/temp
> > >  --nogpgcheck install kernel-devsrc' returned 1:

Re: [yocto] extensible SDK build failure

[RUSSELL PETERSON]

Thank you for the response, Paul. You were correct that I had the 
TOOLCHAIN_*_TASK variables set in my machine.conf file. I put it into my image 
bb file and things seem far more sane... although I must admit I am not 100% 
sure why it works but I will study it a bit and figure it out. Thanks for your 
help.

I have run into another issue while trying to generate the eSDK in that there 
appears to be an issue with building harfbuzz. From what I can tell, this 
appears to be an issue with the search paths for the autoreconf call. The 
autoreconf tool is finding the pkg.m4 file in the harfbuzz m4 directory FIRST 
and therefore not getting the one in 
harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/share/aclocal which indeed has the 
correct m4_pattern_allow.

Do you think that's a bug or something with my recipe? Seems like I just want 
to reverse the -I options... but I don't know how, exactly.

/scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/bin/autoconf
 
--include=/scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/harfbuzz-1.4.1/m4/
 
--include=/scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/share/aclocal/
 --force

../../../autoconf-2.69/lib/autoconf/specific.m4:368: AC_USE_SYSTEM_EXTENSIONS 
is expanded from...
| configure.ac:22: the top level
| configure:18902: error: possibly undefined macro: PKG_CONFIG_SYSROOT_DIR
| If this token and others are legitimate, please use m4_pattern_allow.
| See the Autoconf documentation.
| autoreconf: 
/scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/bin/autoconf
 failed with exit status: 1
| ERROR: Function failed: do_configure (log file is located at 
/scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/temp/log.do_configure.14956)

> On July 31, 2017 at 9:50 AM Paul Eggleton <paul.eggle...@linux.intel.com> 
> wrote:
>
>
> Hi Russell,
>
> It looks to me like you have added kernel-devsrc to the TOOLCHAIN_*_TASK 
> variable at the global level, which you really don't want to do if you're 
> going to be building buildtools-tarball (which eSDK will build as a 
> dependency). I'd suggest moving that setting to your image recipe.
>
> Cheers,
> Paul
>
> On Sunday, 30 July 2017 4:22:24 PM CEST Russell Peterson wrote:
> > Hello, all.
> >
> > I’m trying to build the eSDK for my platform and I just can’t figure out 
> > why dnf can’t find kernel-devsrc. It’s there, I added it. The “normal SDK” 
> > uses it and builds fine. Just no luck with the eSDK.
> > Any help out there? Poky is pyro. aarch64 SoC target platform with an x86 
> > build host (CentOS 7.3). Basically I just bitbake -c do_populate_sdk_ext 
> > core-image-full. I do have a linux kernel recipe that pulls the kernel from 
> > the standard kernel.org <http://kernel.org/> repo.
> >
> > ERROR: buildtools-tarball-1.0-r0 do_populate_sdk: Could not invoke dnf. 
> > Command 
> > '/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/recipe-sysroot-native/usr/bin/dnf
> >  -y -c 
> > /scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/sdk/image/opt/poky/2.3.1/sysroots/none/etc/dnf/dnf.conf
> >  
> > --setopt=reposdir=/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/sdk/image/opt/poky/2.3.1/sysroots/none/etc/yum.repos.d
> >  
> > --repofrompath=oe-repo,/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/oe-rootfs-repo
> >  
> > --installroot=/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/sdk/image/opt/poky/2.3.1/sysroots/none
> >  
> > --setopt=logdir=/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/temp
> >  --nogpgcheck install kernel-devsrc' returned 1:
> > Added oe-repo repo from 
> > file:///scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/oe-rootfs-repo
> >  
> > 
> > Last metadata expiration check: 0:00:01 ago on Sun Jul 30 01:38:48 2017 UTC.
> > No package kernel-devsrc available.
> > Error: Unable to find a match
> >
> > ERROR: buildtools-tarball-1.0-r0 do_populate_sdk: Function failed: 
> > do_populate_sdk
> > ERROR: Logfile of failure stored in: 
> > /scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/temp/log.do_populate_sdk.4836
> > ERROR: Task 
> > (/labhome/russell/src/bf/poky/meta/recipes-core/meta/buildtools-tarball.bb:do_populate_sdk)
> >  failed with exit code '1'
>
>
> --
>
> Paul Eggleton
> Intel Open Source Technology Centre
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] release management

[Russell Peterson]

Hello.

As I learn more about yocto and more importantly gain practical experience with 
it I have started to think about my release structure.  Is there a “best 
practices” document or something like that that speaks to this?

For example, how does everyone deal with “external” meta layer dependencies?  
My software uses poky and meta-openembedded, of course.  It also relies on some 
recipes in meta-linaro and meta-virtualization.  I suspect there will be more 
as time goes by.  I have tweaked my layer priorities as well as used BBMASK to 
avoid unwanted bbappend files etc… works but seems slightly clunky… still 
better than duplicating recipes in my meta layer I think.

Also… I quickly came to the conclusion that “thou shall not modify poky or 
meta-openembedded”.  That is, ALWAYS use bbappend files instead of modifying 
external layers.  If I think that poky or some other layer has a recipe bug or 
want to change something in poky, for example, I plan to upstream a fix to the 
project and when that becomes available I rebase my poky and remove the 
bbappend from my meta layer. One thing about modifying poky and not using 
bbappend files is that I would then need to ship patches for poky instead of 
just directing users to to use this branch and this commit for release x.y.z.

Comments and suggestions welcome.

Regards,

Russell



-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] release management

[Russell Peterson]

Thanks very much, Gunnar, for your detailed and helpful response.

Not sure I will go the git submodule path.  My initial try is using a script in 
my build environment that pulls in whatever layers it needs via git clone and 
then using a specific commit in a specific branch for that specific build.  The 
cloned layers are considered “read only”.  Right now we clone directly from 
yocto or github but we could easily switch to local, non-public mirrors that we 
could populate to specific versions.

Thanks again!

Regards,

Russell


> On Aug 14, 2017, at 3:10 PM, Gunnar Andersson <ganders...@genivi.org> wrote:
> 
> 
> Hey Russell
> 
> I don't claim to be an authority on best practices but I will share some
> thoughts.
> 
> On Sun, 2017-08-13 at 06:58 -0400, Russell Peterson wrote:
>> Hello.
>> 
>> As I learn more about yocto and more importantly gain practical experience
>> with it I have started to think about my release structure.  Is there a
>> “best practices” document or something like that that speaks to this?
>> For example, how does everyone deal with “external” meta layer
>> dependencies?  My software uses poky and meta-openembedded, of course.  
> 
> We simply use a parent project [1] that includes git submodules, one per
> yocto layer.  I'm of the opinion that if git (only) can be used, why
> introduce other tools?   But it requires you to learn and master that
> feature.
> 
> For the primary layer that is unique to this project, meta-genivi-dev, we
> later on decided to merge it into the parent git repository instead, but
> keep all other layers as submodules.  Since there are frequent changes to
> the primary layer, it reduces the complexity of having to update a submodule
> for them continuously.  But the purpose is also to keep *only* unique things
> there, and thereby such things that are reusable by others will be naturally
> pushed to the other layers (those that are reusable, and therefore should be
> separate repos).
> 
> There's a natural tradeoff between layers being developed independently (and
> widely reused by other projects), vs. the complexity of hacking on your
> project.
> 
> Other projects use Google's "repo" tool but I have yet to understand what
> that adds compared to git-submodules (in a medium sized project at least),
> other than just additional hoops to jump through.
> 
>> It also relies on some recipes in meta-linaro and meta-virtualization.  I
>> suspect there will be more as time goes by.  I have tweaked my layer
>> priorities as well as used BBMASK to avoid unwanted bbappend files etc…
>> works but seems slightly clunky… still better than duplicating recipes in
>> my meta layer I think.
> 
> I think you've got it.  Those are the tools you have.  I haven't found too
> frequent needs to mask away that which is provided by other layers, but it
> depends on your needs...
> 
>> 
>> Also… I quickly came to the conclusion that “thou shall not modify poky or
>> meta-openembedded”.  
>> That is, ALWAYS use bbappend files instead of modifying external layers.  
> 
> Absolutely that's the Yocto way.  But .bbappends are a kind of patches,
> so who's to say what is right or wrong?  Some people might think that git is
> actually a better tool to keep track of local modifications - keeping those
> on branches that can be efficiently diffed and merged when upstream changes.
> 
> But clearly most Yocto proficient developers will be used to a .bbappend
> based workflow, and I would wager that you will be better received with any
> support questions than if you say "I have made my own branch of poky/meta-
> oe).  Strictly speaking the two option ought to be equivalent, but I'm just
> guessing it will still be perceived differently.  Yocto devs might agree or
> disagree.
> 
> 
>> If I think that poky or some other layer has a recipe bug or want to
>> change something in poky, for example, I plan to upstream a fix to the
>> project and when that becomes available I rebase my poky and remove the
>> bbappend from my meta layer. 
> 
>> One thing about modifying poky and not using bbappend files is that I
>> would then need to ship patches for poky instead of just directing users
>> to to use this branch and this commit for release x.y.z.
> 
> You're not really asking a question here - just drawing the standard open-
> source process conclusions.  I for one agree with you.  Use bbappends.  Talk
> to the upstream.  Always second-guess why you need to change anything at
> all.  Then if the appends become very complex, consider if finally
> overriding with a new .bb file fits better.  
> 
> In our policy that I will describe further (below) we also most

[yocto] SDK license manifest

[Russell Peterson]

Hello,

I am currently building an SDK and a disk image using morty.  Learning quite a 
bit about Yocto.

Eventually I will distribute both the SDK and the rootfs image to customers.  I 
see that there is a license manifest generated for the full rootfs which is 
very useful.  My question is, what about the SDK?  Is the SDK covered by the 
same manifest or is there something I’m missing.  I assume there are some 
things in the nativesdk rootfs, for example, that need to be tracked.

Thanks!

Russell

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] extensible SDK build failure

[Russell Peterson]

Thanks for the help but, no.  I’m just trying to build the eSDK via the normal 
bitbake process.

Russell

> On Aug 4, 2017, at 11:52 AM, Andrea Galbusera <giz...@gmail.com> wrote:
> 
> Hi,
> 
> On Wed, Aug 2, 2017 at 8:34 PM, RUSSELL PETERSON <bluehil...@comcast.net 
> <mailto:bluehil...@comcast.net>> wrote:
> Frustrating in that I can't get the eSDK to fully build. I'm past the issue
> with kernel-devsrc and harfbuzz... but now I see hundreds of messages like
> below. Anyone seen this before? From what I can tell these tasks are being
> executed out of the run queue.  Not all the messages are from cve-check but 
> most.
> 
> Anyone seen this?
> 
> Are you using devtool to work on recipes/sources? If so, try to run 'devtool 
> reset -a' before building the eSDK. 
> 
> 
> 
> 
> ERROR: Task attr-native.do_cve_check attempted to execute unexpectedly
> ERROR: Task binutils-native.do_cve_check attempted to execute unexpectedly
> ERROR: Task dbus-glib.do_cve_check attempted to execute unexpectedly
> ERROR: Task libtool-native.do_cve_check attempted to execute unexpectedly
> ERROR: Task fixesproto.do_cve_check attempted to execute unexpectedly
> ERROR: Task bc.do_cve_check attempted to execute unexpectedly
> ERROR: Task lsof.do_cve_check attempted to execute unexpectedly
> ERROR: Task python-setuptools-native.do_cve_check attempted to execute
> unexpectedly
> ERROR: Task libxrandr-native.do_cve_check attempted to execute unexpectedly
> ERROR: Task libpciaccess.do_cve_check attempted to execute unexpectedly
> ERROR: Task xf86driproto.do_cve_check attempted to execute unexpectedly
> ERROR: Task ncurses-native.do_cve_check attempted to execute unexpectedly
> 
> On August 1, 2017 at 8:55 PM Russell Peterson <bluehil...@comcast.net 
> <mailto:bluehil...@comcast.net>> wrote:
> 
> FYI: For those interested…
> 
> Just as a test/workaround I added a harfbuzz_%.bbappend file to my meta layer
> and directly set the acpaths variable to the STAGING native directory and that
> seems to work fine. Works for now but I will try to come up with a cleaner,
> more generic fix.
> 
> acpaths = “-I ${STAGING_DATADIR_NATIVE}/aclocal/“
> 
> Regards,
> 
> Russell
> 
> On Aug 1, 2017, at 5:52 PM, RUSSELL PETERSON <bluehil...@comcast.net 
> <mailto:bluehil...@comcast.net>> wrote:
> 
> Thank you for the response, Paul. You were correct that I had the
> TOOLCHAIN_*_TASK variables set in my machine.conf file. I put it into my
> image bb file and things seem far more sane... although I must admit I am
> not 100% sure why it works but I will study it a bit and figure it out.
> Thanks for your help.
> 
> I have run into another issue while trying to generate the eSDK in that
> there appears to be an issue with building harfbuzz. From what I can tell,
> this appears to be an issue with the search paths for the autoreconf call.
> The autoreconf tool is finding the pkg.m4 file in the harfbuzz m4 directory
> FIRST and therefore not getting the one in
> harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/share/aclocal which indeed has
> the correct m4_pattern_allow.
> 
> Do you think that's a bug or something with my recipe? Seems like I just
> want to reverse the -I options... but I don't know how, exactly.
> 
> /scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/bin/autoconf
> --include=/scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/harfbuzz-1.4.1/m4/
> --include=/scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/share/aclocal/
> --force
> 
> ../../../autoconf-2.69/lib/autoconf/specific.m4:368:
> AC_USE_SYSTEM_EXTENSIONS is expanded from...
> | configure.ac:22 <http://configure.ac:22/>: the top level
> | configure:18902: error: possibly undefined macro: PKG_CONFIG_SYSROOT_DIR
> | If this token and others are legitimate, please use m4_pattern_allow.
> | See the Autoconf documentation.
> | autoreconf:
> /scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/recipe-sysroot-native/usr/bin/autoconf
> failed with exit status: 1
> | ERROR: Function failed: do_configure (log file is located at
> /scratch/russell/yocto-00dc025f/work/aarch64-poky-linux/harfbuzz/1.4.1-r0/temp/log.do_configure.14956)
> 
> On July 31, 2017 at 9:50 AM Paul Eggleton <paul.eggle...@linux.intel.com 
> <mailto:paul.eggle...@linux.intel.com>>
> wrote:
> 
> Hi Russell,
> 
> It looks to me like you have added kernel-devsrc to the TOOLCHAIN_*_TASK
> variable at the global level, which you really don't want to do if you're
> going to be building buildtools-tarball (which eSDK will build as a
> dependency). I'd

[yocto] extensible SDK build failure

[Russell Peterson]

Hello, all.

I’m trying to build the eSDK for my platform and I just can’t figure out why 
dnf can’t find kernel-devsrc.  It’s there, I added it.  The “normal SDK” uses 
it and builds fine.  Just no luck with the eSDK.
Any help out there?  Poky is pyro.  aarch64 SoC target platform with an x86 
build host (CentOS 7.3).  Basically I just bitbake -c do_populate_sdk_ext 
core-image-full.  I do have a linux kernel recipe that pulls the kernel from 
the standard kernel.org  repo.

ERROR: buildtools-tarball-1.0-r0 do_populate_sdk: Could not invoke dnf. Command 
'/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/recipe-sysroot-native/usr/bin/dnf
 -y -c 
/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/sdk/image/opt/poky/2.3.1/sysroots/none/etc/dnf/dnf.conf
 
--setopt=reposdir=/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/sdk/image/opt/poky/2.3.1/sysroots/none/etc/yum.repos.d
 
--repofrompath=oe-repo,/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/oe-rootfs-repo
 
--installroot=/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/sdk/image/opt/poky/2.3.1/sysroots/none
 
--setopt=logdir=/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/temp
 --nogpgcheck install kernel-devsrc' returned 1:
Added oe-repo repo from 
file:///scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/oe-rootfs-repo
 

Last metadata expiration check: 0:00:01 ago on Sun Jul 30 01:38:48 2017 UTC.
No package kernel-devsrc available.
Error: Unable to find a match
 
ERROR: buildtools-tarball-1.0-r0 do_populate_sdk: Function failed: 
do_populate_sdk
ERROR: Logfile of failure stored in: 
/scratch/russell/yocto-00dc025f/work/x86_64-nativesdk-pokysdk-linux/buildtools-tarball/1.0-r0/temp/log.do_populate_sdk.4836
ERROR: Task 
(/labhome/russell/src/bf/poky/meta/recipes-core/meta/buildtools-tarball.bb:do_populate_sdk)
 failed with exit code '1'-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] extensible SDK build failure

[Russell Peterson]

Thanks, Paul.

That helped, however, I still see some of the same “unexpected” errors for perf 
tasks as well as my virtual/kernel tasks.  

I presume SDK_INHERIT_BLACKLIST excludes the use of the class for the SDK… but 
I am unclear as to why I see these errors with my linux-yocto-xxx.bb 
recipe/tasks.

Thanks again.

Russell


> On Aug 7, 2017, at 2:49 AM, Paul Eggleton <paul.eggle...@linux.intel.com> 
> wrote:
> 
> Hi Russell,
> 
> On Wednesday, 2 August 2017 8:34:11 PM CEST RUSSELL PETERSON wrote:
>> Frustrating in that I can't get the eSDK to fully build. I'm past the issue
>> with kernel-devsrc and harfbuzz... but now I see hundreds of messages like
>> below. Anyone seen this before? From what I can tell these tasks are being
>> executed out of the run queue.  Not all the messages are from cve-check but
>> most.
> 
> So there are certain classes that will interfere with the ability of the eSDK 
> to lock down the configuration, I wasn't aware of it but cve-check.bbclass is 
> apparently one of them. Try adding this to your configuration:
> 
> SDK_INHERIT_BLACKLIST = "buildhistory icecc cve-check"
> 
> Cheers,
> Paul
> 
> -- 
> 
> Paul Eggleton
> Intel Open Source Technology Centre

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] patch files

[RUSSELL PETERSON]

Hello,


I am fetching a src RPM file inside a simple library.  The RPM basically 
contains a tar file with the source and 2 patch files.  How to I apply those 
patch files?  If I put them in the SRC_URI the fetch fails.  Is there a way I 
can add them to some list that do_patche uses or do something in 
do_patch_prepend??


Regards,


Russell

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] patch files

[RUSSELL PETERSON]

Sorry.  I'm doing 5 things at once here.


I am fetching a source RPM inside a simple RECIPE.


> On June 20, 2017 at 1:33 PM RUSSELL PETERSON <bluehil...@comcast.net> wrote:
> 
> 
> Hello,
> 
> 
> I am fetching a src RPM file inside a simple library.  The RPM basically 
> contains a tar file with the source and 2 patch files.  How to I apply those 
> patch files?  If I put them in the SRC_URI the fetch fails.  Is there a way I 
> can add them to some list that do_patche uses or do something in 
> do_patch_prepend??
> 
> 
> Regards,
> 
> 
> Russell
> 
> 


 

___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] SRCPV and SRC_URI

[Russell Peterson]

Hello,

I’m writing a new recipe and doing something I think should be simple but 
instead causes an exception when expanding SRCPV.

SRCREV = “”
SRC_URI = “git://abc.com/xyz.git ”
PV = “1.2+git${SRCPV}

The above works fine.  Problem is, when I add a patch I get an exception when 
expanding SRCPV!?

SRC_URI_append = “file://myfile.patch ”

Adding that simple append line breaks my recipe.

Any clues?  Seems simple but no.  I see countless other recipes that do this.

Regards,

Russell

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] SRCPV and SRC_URI

[Russell Peterson]

Thanks.  My recipe was actually using += instead of _append and I tried various 
combinations using spaces. Not the issue. 
The issue turned out to be something about my using patchdir=${S}/x in the 
file:// statement.  I removed the patchdir and no more issues.  Wasn’t needed 
anyway.

Thanks for the response.

Russell


> On Jun 2, 2017, at 6:34 PM, Ayoub Zaki <ayoub.z...@embexus.com> wrote:
> 
> 
> 
> On 03.06.2017 00:26, Russell Peterson wrote:
>> Hello,
>> 
>> I’m writing a new recipe and doing something I think should be simple but 
>> instead causes an exception when expanding SRCPV.
>> 
>> SRCREV = “”
>> SRC_URI = “git://abc.com/xyz.git ”
>> PV = “1.2+git${SRCPV}
>> 
>> The above works fine.  Problem is, when I add a patch I get an exception 
>> when expanding SRCPV!?
>> 
>> SRC_URI_append = “file://myfile.patch ”
> 
> Try with a space right after the first double quote :
> SRC_URI_append = “ file://myfile.patch <>”
>> 
>> Adding that simple append line breaks my recipe.
>> 
>> Any clues?  Seems simple but no.  I see countless other recipes that do this.
>> 
>> Regards,
>> 
>> Russell
>> 
>> 
>> 
> 
> -- 
> 
> Ayoub Zaki
> 
> ayoub.z...@embexus.com <mailto:ayoub.z...@embexus.com>
> Mobile: +49(0)176-62901545
> https://embexus.com <https://embexus.com/>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] unmodified binary RPM installed on rootfs?

[Russell Peterson]

Hello,

I am fairly new to yocto and I think I’m having trouble installing an RPM on 
the rootfs. What I am trying to do is install an arm64 binary RPM file straight 
onto the root file system without a recipe… just use the native rpm tool to put 
it there.  There are several reasons why I’m experimenting with this.  Seems 
fairly simple but I have now been messing around for days with little to show 
for it.  Here is what I’m doing…

Added to the image bb file:



ROOTFS_POSTPROCESS_COMMAND += "install_mtt ; "
 
IMAGE_LINGUAS ?= "en-US”  <— Wondered if this was a UTF-8 issue
 
fakeroot install_mft () {
   ${STAGING_BINDIR_NATIVE}/rpm -i --nodeps --root=${IMAGE_ROOTFS}/ 
/mtt-4.7.0-21-arm64-rpm/RPMS/mtt-4.7.0-21.arm64.rpm
}
 
———

This is the error I see.   
 
DEBUG: Executing shell function install_mft
error: unpacking of archive failed: cpio: Bad magic
WARNING: 
/yocto-00dc025f/work/mymach-poky-linux/core-image-full/1.0-r0/temp/run.install_mtt.2846:1
 exit 1 from '/yocto-00dc025f/sysroots/x86_64-linux/usr/bin/rpm -i ---nodeps 
--root=/yocto-00dc025f/work/mymach-poky-linux/core-image-full/1.0-r0/rootfs/ 
/mtt-4.7.0-21-arm64-rpm/RPMS/mtt-4.7.0-21.arm64.rpm’

Seems like this should work.  I have no idea what magic number the file command 
doesn’t like.  I looked through the files and they all look legit.  Maybe this 
is an RPM version issue?  I think the original RPM was created with 4.8 and the 
poky version I’m using is 5.3?


-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] python & xml2

[Russell Peterson]

I am trying to install libxml2 for python2 in my image recipe.  Seems like it 
is being installed in the python3 site packages directory instead of the 
python2 site packages directory.  While looking into this I have become very 
confused.  Is there some documentation on this?  Do I need to set some kind of 
variable in the xml2 recipe??

Russell

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] python & xml2

[Russell Peterson]

I created a bbappend file with the following:

PYTHON_PN = "python"
DEPENDS += "python"
RDEPENDS_${PN}-ptest += "python-core"
PYTHON_BASEVERSION = "2.7"
PYTHON_DIR = "python2.7"

Seems to work although I have not tested it yet… the library shows up in the 
correct site dir anyway.

I would think this would be easier and more obvious to change but I’m not 
python or xml2 expert.
Our users have older python applications that reply on python2.

Thanks for the response.

Russell


> On Apr 28, 2018, at 8:20 AM, Burton, Ross <ross.bur...@intel.com> wrote:
> 
> The recipe specifies that Python 3 is to be used.  If you want to
> build it against Python 2 you'll have to patch the recipe.
> 
> Ross
> 
> On 28 April 2018 at 00:26, Russell Peterson <bluehil...@comcast.net> wrote:
>> I am trying to install libxml2 for python2 in my image recipe.  Seems like 
>> it is being installed in the python3 site packages directory instead of the 
>> python2 site packages directory.  While looking into this I have become very 
>> confused.  Is there some documentation on this?  Do I need to set some kind 
>> of variable in the xml2 recipe??
>> 
>> Russell
>> 
>> --
>> ___
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] dnf error

[Russell Peterson]

Thanks, Alex.  That was indeed the problem.

For those searching for answers to similar issues I will add what my particular 
issue was.

The package I was adding had python scripts in it.  The make process for the 
package used the python distutils.core setup function.  I didn’t write the 
code… I’m just creating a recipe for it.  This in turn copies the scripts to 
various locations but in the process of doing that modifies the #! at the start 
of the script to use the hosttools path to python.  Not very cross-build 
friendly.  I simply patched the makefile to use the —executable=/usr/bin/python 
argument.  All this does is prevent the #! modification.  Builds fine now.  Bit 
of a hack but I will speak with the developers to create a bit more cross build 
friendly options.

Appreciate the help, Alex.

Regards,

Russell


> On Feb 8, 2018, at 1:26 AM, Alexander Kanavin 
> <alexander.kana...@linux.intel.com> wrote:
> 
> On 02/08/2018 03:17 AM, Russell Peterson wrote:
>> core-image-initramfs-1.0-r0 do_rootfs: Could not invoke dnf
>> Last metadata expiration check: 0:00:00 ago on Thu 08 Feb 2018 12:07:06 AM 
>> UTC.
>> Error:
>>  Problem: conflicting requests
>>   - nothing provides /hosttools/python needed by 
>> ofa-kernel-4.2-r0.mymachine
>> ERROR: core-image-initramfs-1.0-r0 do_rootfs: Function failed: do_rootfs
> 
> 
> Something in ofa-kernel rpm package is erroneously referring to 
> .../hosttools/python. Rpm takes that as a dependency that must be satisfied.
> 
> Find out what it is and how it happens.
> 
> Alex

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] SDK and out of tree modules

[RUSSELL PETERSON]

I have started looking at this more closely and I have a few questions. Hope 
someone knows or has seen these issues before.

1. When running create_filtered task list I see failures. This is mainly due to 
the fact that the sdkbasepath getting renamed to tmp-renamed-sdk fails. I have 
bypassed this to get around it but it seems like it should work. Renaming 
<...>/sdk-ext/image/opt/poky/2.4.1 to <...>sdk-ext/image/tmp-renamed-sdk 
doesn't seem to work.

2. Understanding the sstate-cache took some time but I think I understand the 
basic idea now. The problem I was seeing relates to the fact that my build 
directory and my temp directory are on different disks. Hence, when I delete 
the build directory, the cache gets deleted. I would think that making 
core-image-full again would regenerate the sstate-cache but it does not. The 
only way I see the sstate-cache regenerated is by deleting the tmp directory 
completely and starting over. Without the cache, building the sdk-ext fails 
with about 5000 "should have been setscened" errors. As the ext sdk clearly has 
a dependency there... why isn't the state cache re-created?? Probably just 
don't quite understand state yet, I guess.

3. Finally, I have the ext sdk built... tried to install it... failed with an 
error telling me TMPDIR has changed and I need to rebuild or change it back. I 
assume this is related to me setting TMPDIR in the original local.conf? Anyone 
else see this? Seems like unless TMPDIR is set to the default (TOPDIR/tmp) the 
SDK won't work??

Regards,

Russell

> On July 25, 2018 at 9:01 AM RUSSELL PETERSON  wrote:
>
>
> So, this seems broken to me. I managed to get around this issue 
> (sstate-control/manifest-allarch-kernel-devsrc.populate_sysroot not found?) 
> by appending PACKAGE_EXTRA_ARCHS with the MACHINE_ARCH for my bsp. While 
> PACKAGE_ARCHS now has a duplicate in it (it already includes MACHINE_ARCH as 
> well as PACKAGE_EXTRA_ARCH), the python code in staging_populate_sysroot 
> seems to require this or it looks for the wrong manifest file.
>
> Also, when building the eSDK, dnf seems very confused about what packages are 
> compatible. It's looking for my SoC arch packages... but then won't accept 
> aarch64 as being compatible. I needed to set:
>
> ALL_MULTILIB_PACKAGE_ARCHS =+ " ${PACKAGE_EXTRA_ARCHS_tune-${DEFAULTTUNE}}"
>
> to get /etc/dnf/vars/arch to update correctly. This is a bit tedious. I'm now 
> having a similar issue because dnf doesn't seem to like an arch of 
> x86_64-nativesdk. Do I update ALL_MULTILIB_PACKAGE_ARCHS again?? Ug.
>
> --Russ
>
>
> > On July 24, 2018 at 8:36 AM RUSSELL PETERSON  wrote:
> >
> >
> > I am running Poky Rocko at the HEAD (just updated yesterday) and see the 
> > following when I attempt to build the eSDK:
> >
> > sstate-control/manifest-allarch-kernel-devsrc.populate_sysroot not found?
> >
> > I see this has been an issue with others as well. It looks like Paul had a 
> > fix around April but then the trail went silent so I'm not sure if there 
> > was a fix or if that fix went into Rocko. Anyone?
> >
> > --Russ
> >
> > > On July 21, 2018 at 4:42 PM Russell Peterson  
> > > wrote:
> > >
> > >
> > > No, just the standard SDK. I was having issues building the eSDK back 
> > > when we used pyro and never fully figured it out… but we have since 
> > > upgraded to rocko. I should revisit the eSDK and see if it works for me 
> > > now or find the root cause since it sounds useful.
> > >
> > > Thanks, Khem.
> > >
> > > —Russ
> > >
> > >
> > > > On Jul 21, 2018, at 1:34 PM, Khem Raj  wrote:
> > > >
> > > > On Sat, Jul 21, 2018 at 6:20 AM Russell Peterson 
> > > >  wrote:
> > > >>
> > > >> Hello,
> > > >>
> > > >> I have been building some modules using the SDK for a while now. This 
> > > >> is mostly for development flow purposes but we have had a few 
> > > >> customers doing this as well. To get this to work we always need to 
> > > >> run “make silentoldconfig scripts” inside the RFS of the SDK on the 
> > > >> build host. Many folks forget to do this this and thus many, many 
> > > >> questions come my way about the SDK being broken and they can’t build 
> > > >> their modules (not all users are kernel experts or even intermediates… 
> > > >> they just want to apply a patch and quickly move on to their app). Is 
> > > >> there a way to do this auto-magically during the installation of the 
> > > >> SDK by adding some type of scri

Re: [yocto] SDK and out of tree modules

[RUSSELL PETERSON]

So, this seems broken to me.  I managed to get around this issue 
(sstate-control/manifest-allarch-kernel-devsrc.populate_sysroot not found?) by 
appending PACKAGE_EXTRA_ARCHS with the MACHINE_ARCH for my bsp.  While 
PACKAGE_ARCHS now has a duplicate in it (it already includes MACHINE_ARCH as 
well as PACKAGE_EXTRA_ARCH), the python code in staging_populate_sysroot seems 
to require this or it looks for the wrong manifest file.

Also, when building the eSDK, dnf seems very confused about what packages are 
compatible.  It's looking for my SoC arch packages... but then won't accept 
aarch64 as being compatible.  I needed to set:

ALL_MULTILIB_PACKAGE_ARCHS =+ " ${PACKAGE_EXTRA_ARCHS_tune-${DEFAULTTUNE}}"

to get /etc/dnf/vars/arch to update correctly.  This is a bit tedious.  I'm now 
having a similar issue because dnf doesn't seem to like an arch of 
x86_64-nativesdk.  Do I update ALL_MULTILIB_PACKAGE_ARCHS again??  Ug.

--Russ
 

> On July 24, 2018 at 8:36 AM RUSSELL PETERSON  wrote:
> 
> 
> I am running Poky Rocko at the HEAD (just updated yesterday) and see the 
> following when I attempt to build the eSDK:
> 
> sstate-control/manifest-allarch-kernel-devsrc.populate_sysroot not found?
> 
> I see this has been an issue with others as well.  It looks like Paul had a 
> fix around April but then the trail went silent so I'm not sure if there was 
> a fix or if that fix went into Rocko.  Anyone?
> 
> --Russ
> 
> > On July 21, 2018 at 4:42 PM Russell Peterson  wrote:
> > 
> > 
> > No, just the standard SDK.  I was having issues building the eSDK back when 
> > we used pyro and never fully figured it out… but we have since upgraded to 
> > rocko.  I should revisit the eSDK and see if it works for me now or find 
> > the root cause since it sounds useful.
> > 
> > Thanks, Khem.
> > 
> > —Russ
> > 
> > 
> > > On Jul 21, 2018, at 1:34 PM, Khem Raj  wrote:
> > > 
> > > On Sat, Jul 21, 2018 at 6:20 AM Russell Peterson  
> > > wrote:
> > >> 
> > >> Hello,
> > >> 
> > >> I have been building some modules using the SDK for a while now.  This 
> > >> is mostly for development flow purposes but we have had a few customers 
> > >> doing this as well.  To get this to work we always need to run “make 
> > >> silentoldconfig scripts” inside the RFS of the SDK on the build host.  
> > >> Many folks forget to do this this and thus many, many questions come my 
> > >> way about the SDK being broken and they can’t build their modules (not 
> > >> all users are kernel experts or even intermediates… they just want to 
> > >> apply a patch and quickly move on to their app).  Is there a way to do 
> > >> this auto-magically during the installation of the SDK by adding some 
> > >> type of scripts etc… to the recipe?  I assume it needs to be done at 
> > >> install time since while the build host is x86… the exact linux distro 
> > >> is not known until then (or does that matter?).
> > >> 
> > > 
> > > are you using extensible SDK ? in that case I think do_make_scripts
> > > from module-base.bbclass should be helpful
> > > 
> > >> —Russ
> > >> 
> > >> --
> > >> ___
> > >> yocto mailing list
> > >> yocto@yoctoproject.org
> > >> https://lists.yoctoproject.org/listinfo/yocto
> > 
> > -- 
> > ___
> > yocto mailing list
> > yocto@yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/yocto
> -- 
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] openssl

[Russell Peterson]

Hello,

I’m looking to change to openssl version 1.1.  I’m using the rocko branch.  I 
have set PREFERRED_VERSION_openssl to 1.1%… but it doesn’t seem to work.

Anyone?

NOTE: Resolving any missing task queue dependencies
NOTE: preferred version 1.1.% of openssl not available (for item openssl10)
NOTE: versions of openssl available: 1.0.2m
NOTE: preferred version 1.1.% of openssl not available (for item openssl-misc)
NOTE: versions of openssl available: 1.0.2m
NOTE: preferred version 1.1.% of openssl not available (for item openssl-conf)
NOTE: versions of openssl available: 1.0.2m
NOTE: preferred version 1.1.% of openssl not available (for item libcrypto)
NOTE: versions of openssl available: 1.0.2m-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] SDK and out of tree modules

[Russell Peterson]

No, just the standard SDK.  I was having issues building the eSDK back when we 
used pyro and never fully figured it out… but we have since upgraded to rocko.  
I should revisit the eSDK and see if it works for me now or find the root cause 
since it sounds useful.

Thanks, Khem.

—Russ


> On Jul 21, 2018, at 1:34 PM, Khem Raj  wrote:
> 
> On Sat, Jul 21, 2018 at 6:20 AM Russell Peterson  
> wrote:
>> 
>> Hello,
>> 
>> I have been building some modules using the SDK for a while now.  This is 
>> mostly for development flow purposes but we have had a few customers doing 
>> this as well.  To get this to work we always need to run “make 
>> silentoldconfig scripts” inside the RFS of the SDK on the build host.  Many 
>> folks forget to do this this and thus many, many questions come my way about 
>> the SDK being broken and they can’t build their modules (not all users are 
>> kernel experts or even intermediates… they just want to apply a patch and 
>> quickly move on to their app).  Is there a way to do this auto-magically 
>> during the installation of the SDK by adding some type of scripts etc… to 
>> the recipe?  I assume it needs to be done at install time since while the 
>> build host is x86… the exact linux distro is not known until then (or does 
>> that matter?).
>> 
> 
> are you using extensible SDK ? in that case I think do_make_scripts
> from module-base.bbclass should be helpful
> 
>> —Russ
>> 
>> --
>> ___
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] SDK and out of tree modules

[Russell Peterson]

Hello,

I have been building some modules using the SDK for a while now.  This is 
mostly for development flow purposes but we have had a few customers doing this 
as well.  To get this to work we always need to run “make silentoldconfig 
scripts” inside the RFS of the SDK on the build host.  Many folks forget to do 
this this and thus many, many questions come my way about the SDK being broken 
and they can’t build their modules (not all users are kernel experts or even 
intermediates… they just want to apply a patch and quickly move on to their 
app).  Is there a way to do this auto-magically during the installation of the 
SDK by adding some type of scripts etc… to the recipe?  I assume it needs to be 
done at install time since while the build host is x86… the exact linux distro 
is not known until then (or does that matter?).

—Russ

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] SDK and out of tree modules

[RUSSELL PETERSON]

I am running Poky Rocko at the HEAD (just updated yesterday) and see the 
following when I attempt to build the eSDK:

sstate-control/manifest-allarch-kernel-devsrc.populate_sysroot not found?

I see this has been an issue with others as well.  It looks like Paul had a fix 
around April but then the trail went silent so I'm not sure if there was a fix 
or if that fix went into Rocko.  Anyone?

--Russ

> On July 21, 2018 at 4:42 PM Russell Peterson  wrote:
> 
> 
> No, just the standard SDK.  I was having issues building the eSDK back when 
> we used pyro and never fully figured it out… but we have since upgraded to 
> rocko.  I should revisit the eSDK and see if it works for me now or find the 
> root cause since it sounds useful.
> 
> Thanks, Khem.
> 
> —Russ
> 
> 
> > On Jul 21, 2018, at 1:34 PM, Khem Raj  wrote:
> > 
> > On Sat, Jul 21, 2018 at 6:20 AM Russell Peterson  
> > wrote:
> >> 
> >> Hello,
> >> 
> >> I have been building some modules using the SDK for a while now.  This is 
> >> mostly for development flow purposes but we have had a few customers doing 
> >> this as well.  To get this to work we always need to run “make 
> >> silentoldconfig scripts” inside the RFS of the SDK on the build host.  
> >> Many folks forget to do this this and thus many, many questions come my 
> >> way about the SDK being broken and they can’t build their modules (not all 
> >> users are kernel experts or even intermediates… they just want to apply a 
> >> patch and quickly move on to their app).  Is there a way to do this 
> >> auto-magically during the installation of the SDK by adding some type of 
> >> scripts etc… to the recipe?  I assume it needs to be done at install time 
> >> since while the build host is x86… the exact linux distro is not known 
> >> until then (or does that matter?).
> >> 
> > 
> > are you using extensible SDK ? in that case I think do_make_scripts
> > from module-base.bbclass should be helpful
> > 
> >> —Russ
> >> 
> >> --
> >> ___
> >> yocto mailing list
> >> yocto@yoctoproject.org
> >> https://lists.yoctoproject.org/listinfo/yocto
> 
> -- 
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] out of tree module dependencies

[RUSSELL PETERSON]

I have been debugging this issue and could use some advice.

The basic issue I see is that do_make_scripts executes and fails. I have seen 
different fails but in the end it all comes back to something being required in 
recipe-sysroot or recipe-sysroot-native and it's not there. For instance 
gcc-cross-aarch64 is missing from recipe-sysroot-native.

Yocto build dependencies is a rather advanced topic and I certainly don't feel 
I understand exactly what the issue is... but here is my guess.

1. I changed my linux kernel recipe.
2. Because of #1, the cross compiler decided to rebuild. Generally, this isn't 
an issue. In fact the old cross compiler should *still* be in 
recipe-sysroot-native.
3. Once the kernel build artifacts is done... my module recipes fire. The first 
thing they want to do is run do_cve_check.
4. This is the part that causes the problem... do_cve_check DELETES 
gcc-cross-aarch64 from the sysroot! This is because it cannot find 
gcc-cross-aarch64.complete... which points to a manifest file. I think this 
manifest file is created by the gcc-cross-aarch64.do_populate_sysroot. Problem 
is, cve_check does not depend on that?

This race condition does not always present itself so I am running tests... my 
solution?

do_cve_check[depends] += "${PN}:do_prepare_recipe_sysroot"

I do that because I believe do_prepare_recipe_sysroot already depends on 
whatever is required in the recipe-sysroot dirs so it's a more general fix.

Does this make sense to anyone?

> On April 3, 2018 at 7:42 PM Russell Peterson <bluehil...@comcast.net> wrote:
>
>
> Hello,
>
> I have some out of tree modules that I have added to my custom meta layer.
> Normally things work fine, however, when I modify these recipes every now and 
> then the do_configure fails because the recipe_sysroot directory is not 
> populated. This seems related to the kernel-build-artifacts and the 
> make_scripts task as well but I can’t quite figure it out. Seems like the 
> prepare_recipe_sysroot should execute but in these cases it does not.
>
> I realize I didn’t provide much info here but anyone see something like this 
> before? I do have an unpack_extra task as well that may somehow be related?
>
>
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] out of tree module dependencies

[Russell Peterson]

Hello,

I have some out of tree modules that I have added to my custom meta layer.
Normally things work fine, however, when I modify these recipes every now and 
then the do_configure fails because the recipe_sysroot directory is not 
populated.  This seems related to the kernel-build-artifacts and the 
make_scripts task as well but I can’t quite figure it out.  Seems like the 
prepare_recipe_sysroot should execute but in these cases it does not.

I realize I didn’t provide much info here but anyone see something like this 
before?  I do have an unpack_extra task as well that may somehow be related?


-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] make-mod-scripts

[Russell Peterson]

Hello,

We are seeing build failures every now and then regarding our out of tree
modules.

I have tracked the issue down to make-mod-scripts performing the make
oldconfig and prepare operations.  Normally this works, however, if for
some reason a clean is done on the linux kernel, the build artifacts
directory gets wiped out.  While the build artifacts directory does get
rebuilt, make-mod-scripts:do_configure is never triggered because it
doesn't think anything changed.  This breaks the module build because it
can't find auto.conf.

I think I can nostamp make-mod-scripts for now but is there a more elegant,
efficient solution here?  Seems like make-mod-scripts is designed to only
build once.

Regards,

Russell
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] TUNE_PKGARCH vs MACHINE_ARCH

[Russell Peterson]

Hello,

I’m trying to understand what the “normal” package arch should be when using 
yocto to build for a specific SoC.
In my local.conf file I set my MACHINE value to “xSoc”.  Because of that I see 
RPM files being generated with arch “xSoC” (as well as “aarch64”).
I forgot to mention in my machine/xSoC.conf file I include the aarch64 tune 
file (Rocko).  To me, I can’t quite understand what a rpm arch of xSoC is since 
the cores on this SoC are simply Arm A72 cores and I thought RPM arch meant 
micro-arch, no?  If I override PACKAGE_ARCH_xSoC = “aarch64” that sort of works 
but then all I see aarch64 rpm files (no allarch/noarch).  I also see some 
errors with PACKAGE_ARCH being set before inherit package group when I try 
override.

What’s the expected behavior here?  Everything seems to work except I get a few 
warning messages regarding staging files for kernel-devsrc when I build the 
eSDK.

Thanks,

Russell

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] kernel module override

[Russell Peterson]

Hello,

I have what I would think is an unusual requirement regarding a kernel
module and could use some advice.

I have a kernel module that has been heavily modified... to the point where
we have a separate recipe for it and treat it as an out-of-tree module.
This works by not setting the kernel config file to build that module and
our meta layer builds and installs the module.  No problem... works fine
for us.

The problem is that we now want to enable another module that relies on the
module we build out of tree.  The issue is that the check config sees we
have not enabled our out of tree module and thus the sanitized config does
not contain this module that we want to enable.  If we enable the modified
module in the config file there is, of course, a package conflict.

I have tried using "virtual/mymodule" preferred provider but that fails
since it seems to have a recipe "granularity" and we still need
virtual/kernel.  I have also messed around with
KERNEL_MODULE_PACKAGE_PREFIX and KERNEL_MODULES_RDEPENDS_BLACKLIST but I
get the sense I'm trying to put a square peg in a round hole.

I do realize this is an unusual situation and the question "why don't you
just patch the existing driver in the kernel?" is obvious.  We do have our
reasons that I won't go into.

Is there any obvious way we can simply "eclipse" an existing kernel module
with an out-of-tree module?  Is there a way I could add a config fragment
that doesn't get validated/sanitized, for example?

Regards,

Russell
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] PREMIRROR

[Russell Peterson]

Hello,

I am looking to have bitbake pick up files for a particular recipe from a
local git repository using the PREMIRROR functionality.

Basically, the recipe (bb file) points to github but in my local build I
add PREMIRROR_prepend = "git://.*/.*
git:///local/path/BASENAME;protocol=file\n"

I will probably make the git regular expression more exact for my specific
github repo but this works for now.

This all works (as I deleted the github download from the local download
directory) because I can see in the do_fetch log and the correct (local)
repo was found and placed in the DL_DIR.

Problem is, do_unpack fails because it appears to be looking for the
original (github) SRC_URI.  Then it complains about "no up to date source
found: clone or directory not available or not up to date (shallow clone
not enabled)"

Any help on what I am missing would be appreciated.

Regards,

Russell
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] PREMIRROR

[Russell Peterson]

Hi, Rudolf.

I apologize for not being clear.  The idea here is that my recipe points to
github while, for my local development environment, I set a premirror to
match a specific github repository and translate it to a local directory.
That works.  The fetch matches the PREMIRROR and places a copy of the LOCAL
git repo in my DL_DIR.  The problem is, the do_unpack task is looking for
the git repo in the DL_DIR (git2/etc...)... but it's looking for the git
repo from github, not my local repo that the fetch task just put in DL_DIR.

There are numerous reasons I'm doing this including the fact that I cannot
put a reference to my local repo in the bb file since I ship that recipe to
my customers and a local pathname is meaningless to them.  I prefer to
simply modify my local.conf with a PREMIRROR value that has a specific
regular expression that matches this particular github repo (and hence does
NOT effect all recipes).  This is why I wanted to use the PREMIRROR
function.  Unfortunately, it does not appear to work or at least not work
as I expected.

--Russ




On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif 
wrote:

> Hi Russell,
>
> devtool and eSDK are different things. The purpose of PREMIRRORS is to set
> a mirror for all recipes. It's a way for organizations to control where
> their YP builds download sources from. It's not intended to be used for a
> single recipe. There is no need for that. You simply set SRC_URI in your
> recipe to your local git repo. That is what devtool does after downloading
> the sources from a remote repo. If you already have the remote repo cloned
> locally you can just point devtool to it.
>
> You can do it manually by creating your recipe and setting SRC_URI like
> this:
>
> SRC_URI = "git:///local/path/${PN};protocol=file"
>
> SRCREV = "${AUTOREV}"
>
> S = "${WORKDIR}/git"
>
> PREMIRRORS is only relevant for do_fetch not for do_unpack.
>
> :rjs
>
>
> On 7/24/19 11:28 AM, Russell Peterson wrote:
>
> Hi, Rudolf.
>
> Thanks for the reply.  Yes, I am aware of the eSDK functionality, however,
> I have some unique requirements that I am trying to work around.
> Regardless... what I am doing should work, no?  I simply want to use a
> local git repo (the directory itself hence protocol=file) instead of what
> the recipe specifies.  Looks like the fetch is working but the do_unpack
> task is ignoring PREMIRRORS (or at least the localpath variable seems
> wrong).
>
> --Russ
>
>
> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif 
> wrote:
>
>> Russell,
>>
>> You don't need PREMIRROR for this functionality. It's not exactly
>> intended for that use.
>>
>> The simplest way to achieve what you are looking for is to use devtool.
>> If I understand you correctly you are downloading sources from a remote
>> repo on GitHub but want to have them locally to make modifications? If so
>> use:
>>
>> devtool add myrecipe localsrc fetchuri
>>
>> from your build environment (you have to source oe-init-build-env first).
>> devtool then fetches the source from fetchuri into a directory localsrc as
>> a git repo and automatically creates the recipe for it.
>>
>> :rjs
>>
>>
>> On 7/23/19 1:49 PM, Russell Peterson wrote:
>>
>> Hello,
>>
>> I am looking to have bitbake pick up files for a particular recipe from a
>> local git repository using the PREMIRROR functionality.
>>
>> Basically, the recipe (bb file) points to github but in my local build I
>> add PREMIRROR_prepend = "git://.*/.*
>> git:///local/path/BASENAME;protocol=file\n"
>>
>> I will probably make the git regular expression more exact for my
>> specific github repo but this works for now.
>>
>> This all works (as I deleted the github download from the local download
>> directory) because I can see in the do_fetch log and the correct (local)
>> repo was found and placed in the DL_DIR.
>>
>> Problem is, do_unpack fails because it appears to be looking for the
>> original (github) SRC_URI.  Then it complains about "no up to date source
>> found: clone or directory not available or not up to date (shallow clone
>> not enabled)"
>>
>> Any help on what I am missing would be appreciated.
>>
>> Regards,
>>
>> Russell
>>
>>
>>
>> --
>> -
>> Rudolf J Streif
>> CEO/CTO ibeeto
>> +1.855.442.3396 x700
>>
>> --
> -
> Rudolf J Streif
> CEO/CTO ibeeto
> +1.855.442.3396 x700
>
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] PREMIRROR

[Russell Peterson]

Thanks, Yi.  I did try that and I see the tar file is created... but again,
the do_unpack function seems to ignore it and go directly to the github
repository.

--Russ


On Wed, Jul 24, 2019 at 4:28 AM Yi Zhao  wrote:

>
> On 7/24/19 4:49 AM, Russell Peterson wrote:
>
> Hello,
>
> I am looking to have bitbake pick up files for a particular recipe from a
> local git repository using the PREMIRROR functionality.
>
> Basically, the recipe (bb file) points to github but in my local build I
> add PREMIRROR_prepend = "git://.*/.*
> git:///local/path/BASENAME;protocol=file\n"
>
> I will probably make the git regular expression more exact for my specific
> github repo but this works for now.
>
> This all works (as I deleted the github download from the local download
> directory) because I can see in the do_fetch log and the correct (local)
> repo was found and placed in the DL_DIR.
>
> Problem is, do_unpack fails because it appears to be looking for the
> original (github) SRC_URI.  Then it complains about "no up to date source
> found: clone or directory not available or not up to date (shallow clone
> not enabled)"
>
>
> Maybe you can use BB_GENERATE_MIRROR_TARBALLS to generate git repo tarball
> and put it to your download mirror.
>
> See:
> https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-bb-BB_GENERATE_MIRROR_TARBALLS
>
> and
>
>
> https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#var-DL_DIR
>
>
> Regards,
>
> Yi
>
>
>
> Any help on what I am missing would be appreciated.
>
> Regards,
>
> Russell
>
>
>
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] PREMIRROR

[Russell Peterson]

Hi, Rudolf.

Thanks for the reply.  Yes, I am aware of the eSDK functionality, however,
I have some unique requirements that I am trying to work around.
Regardless... what I am doing should work, no?  I simply want to use a
local git repo (the directory itself hence protocol=file) instead of what
the recipe specifies.  Looks like the fetch is working but the do_unpack
task is ignoring PREMIRRORS (or at least the localpath variable seems
wrong).

--Russ


On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif 
wrote:

> Russell,
>
> You don't need PREMIRROR for this functionality. It's not exactly intended
> for that use.
>
> The simplest way to achieve what you are looking for is to use devtool. If
> I understand you correctly you are downloading sources from a remote repo
> on GitHub but want to have them locally to make modifications? If so use:
>
> devtool add myrecipe localsrc fetchuri
>
> from your build environment (you have to source oe-init-build-env first).
> devtool then fetches the source from fetchuri into a directory localsrc as
> a git repo and automatically creates the recipe for it.
>
> :rjs
>
>
> On 7/23/19 1:49 PM, Russell Peterson wrote:
>
> Hello,
>
> I am looking to have bitbake pick up files for a particular recipe from a
> local git repository using the PREMIRROR functionality.
>
> Basically, the recipe (bb file) points to github but in my local build I
> add PREMIRROR_prepend = "git://.*/.*
> git:///local/path/BASENAME;protocol=file\n"
>
> I will probably make the git regular expression more exact for my specific
> github repo but this works for now.
>
> This all works (as I deleted the github download from the local download
> directory) because I can see in the do_fetch log and the correct (local)
> repo was found and placed in the DL_DIR.
>
> Problem is, do_unpack fails because it appears to be looking for the
> original (github) SRC_URI.  Then it complains about "no up to date source
> found: clone or directory not available or not up to date (shallow clone
> not enabled)"
>
> Any help on what I am missing would be appreciated.
>
> Regards,
>
> Russell
>
>
>
> --
> -
> Rudolf J Streif
> CEO/CTO ibeeto
> +1.855.442.3396 x700
>
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] PREMIRROR

[Russell Peterson]

I think I have a somewhat better understanding of what is going on.

First off, I was confused by the fact that the original error message I saw
from do_unpack referenced the file (URL) at
DL_DIR/git2/original_github_url.  What I didn't understand at the time was
that while that file existed,* it was actually a link* to
DL_DIR/git2/local_url.  So, do_unpack wasn't looking at the wrong bare
repository as I thought.  It was unhappy because it didn't see the git
commit in the local repo that it was looking for.  I use AUTOREV for
SRCREV.  Apparently, even with PREMIRROR set and matched, bitbake will
still fetch the git HEAD commit hash from the original URL in the recipe.
The local git repo doesn't contain that commit.  When I sync the local git
repo to the github repo things work as expected.

Here is my problem.  What I am ultimately trying to do here is have 2 git
repos.  1 public, 1 private.  For internal development we use the private
git repo (a local gerrit server).  Just before we ship we update the public
github repo.  The recipe always points to the github (public) repo so we
don't need to mess with that before a release.  This way we can do nightly
builds (I'm just talking nightly builds done by Jenkins... not developers
using eSDK and devtool) and test.  Customers using yocto will be given a
recipe that points to github and, importantly, those customers that do NOT
use yocto simply fetch the project from github and manually build it in
their own environment with their own tools.

I can't be the only one doing this.  Is there a best practices here
(private vs. public repo)?

--Russ


On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif 
wrote:

> Russell,
>
> That is exactly what devtool and the externalsrc class do. PREMIRROR is
> the wrong approach for that.
>
> :rjs
> On 7/24/19 12:53 PM, Russell Peterson wrote:
>
> Hi, Rudolf.
>
> I apologize for not being clear.  The idea here is that my recipe points
> to github while, for my local development environment, I set a premirror to
> match a specific github repository and translate it to a local directory.
> That works.  The fetch matches the PREMIRROR and places a copy of the LOCAL
> git repo in my DL_DIR.  The problem is, the do_unpack task is looking for
> the git repo in the DL_DIR (git2/etc...)... but it's looking for the git
> repo from github, not my local repo that the fetch task just put in DL_DIR.
>
> There are numerous reasons I'm doing this including the fact that I cannot
> put a reference to my local repo in the bb file since I ship that recipe to
> my customers and a local pathname is meaningless to them.  I prefer to
> simply modify my local.conf with a PREMIRROR value that has a specific
> regular expression that matches this particular github repo (and hence does
> NOT effect all recipes).  This is why I wanted to use the PREMIRROR
> function.  Unfortunately, it does not appear to work or at least not work
> as I expected.
>
> --Russ
>
>
>
>
> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif 
> wrote:
>
>> Hi Russell,
>>
>> devtool and eSDK are different things. The purpose of PREMIRRORS is to
>> set a mirror for all recipes. It's a way for organizations to control where
>> their YP builds download sources from. It's not intended to be used for a
>> single recipe. There is no need for that. You simply set SRC_URI in your
>> recipe to your local git repo. That is what devtool does after downloading
>> the sources from a remote repo. If you already have the remote repo cloned
>> locally you can just point devtool to it.
>>
>> You can do it manually by creating your recipe and setting SRC_URI like
>> this:
>>
>> SRC_URI = "git:///local/path/${PN};protocol=file"
>>
>> SRCREV = "${AUTOREV}"
>>
>> S = "${WORKDIR}/git"
>>
>> PREMIRRORS is only relevant for do_fetch not for do_unpack.
>>
>> :rjs
>>
>>
>> On 7/24/19 11:28 AM, Russell Peterson wrote:
>>
>> Hi, Rudolf.
>>
>> Thanks for the reply.  Yes, I am aware of the eSDK functionality,
>> however, I have some unique requirements that I am trying to work around.
>> Regardless... what I am doing should work, no?  I simply want to use a
>> local git repo (the directory itself hence protocol=file) instead of what
>> the recipe specifies.  Looks like the fetch is working but the do_unpack
>> task is ignoring PREMIRRORS (or at least the localpath variable seems
>> wrong).
>>
>> --Russ
>>
>>
>> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif <
>> rudolf.str...@ibeeto.com> wrote:
>>
>>> Russell,
>>>
>>> You don't need PREMIRROR for this functionality. It's not exactly
>>> i

Re: [yocto] PREMIRROR

[Russell Peterson]

Just tried the externalsrc feature.  Works perfectly.  Exactly what I was
looking for.  Thanks so much, Rudolf!

--Russ


On Thu, Jul 25, 2019 at 4:59 PM Rudolf J Streif 
wrote:

> Inlining below.
> On 7/25/19 8:14 AM, Russell Peterson wrote:
>
> I think I have a somewhat better understanding of what is going on.
>
> First off, I was confused by the fact that the original error message I
> saw from do_unpack referenced the file (URL) at
> DL_DIR/git2/original_github_url.  What I didn't understand at the time was
> that while that file existed,* it was actually a link* to
> DL_DIR/git2/local_url.  So, do_unpack wasn't looking at the wrong bare
> repository as I thought.  It was unhappy because it didn't see the git
> commit in the local repo that it was looking for.  I use AUTOREV for
> SRCREV.  Apparently, even with PREMIRROR set and matched, bitbake will
> still fetch the git HEAD commit hash from the original URL in the recipe.
> The local git repo doesn't contain that commit.  When I sync the local git
> repo to the github repo things work as expected.
>
> Correct. If the source repo or the correct commit cannot be found from the
> PREMIRROR, bitbake will use SRC_URI from the recipe. You can block that
> behavior by setting BB_NO_NETWORK = "1" in which case bitbake will not be
> able to connect to any remote server. However, that does not help if your
> PREMIRROR is a network server. In this case use BB_ALLOWED_HOSTS to allow
> access to only the servers you want.
>
>
> Here is my problem.  What I am ultimately trying to do here is have 2 git
> repos.  1 public, 1 private.  For internal development we use the private
> git repo (a local gerrit server).  Just before we ship we update the public
> github repo.  The recipe always points to the github (public) repo so we
> don't need to mess with that before a release.  This way we can do nightly
> builds (I'm just talking nightly builds done by Jenkins... not developers
> using eSDK and devtool) and test.  Customers using yocto will be given a
> recipe that points to github and, importantly, those customers that do NOT
> use yocto simply fetch the project from github and manually build it in
> their own environment with their own tools.
>
> That is a common problem. I do this all the time. To do this add to your
> conf/local.conf file:
>
> INHERIT += "externalsrc"
> EXTERNALSRC_pn-myrecipe = "/path/to/source/tree"
>
> Alternatively, you could use a bbappend file for you recipe in a
> development layer, myrecipe.bbappend:
>
> inherit externalsrc
> EXTERNALSRC = "/path/to/source/tree"
>
> Now, that is exactly what devtool does for you but you can of course do it
> manually.
>
> :rjs
>
>
>
> I can't be the only one doing this.  Is there a best practices here
> (private vs. public repo)?
>
> --Russ
>
>
> On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif 
> wrote:
>
>> Russell,
>>
>> That is exactly what devtool and the externalsrc class do. PREMIRROR is
>> the wrong approach for that.
>>
>> :rjs
>> On 7/24/19 12:53 PM, Russell Peterson wrote:
>>
>> Hi, Rudolf.
>>
>> I apologize for not being clear.  The idea here is that my recipe points
>> to github while, for my local development environment, I set a premirror to
>> match a specific github repository and translate it to a local directory.
>> That works.  The fetch matches the PREMIRROR and places a copy of the LOCAL
>> git repo in my DL_DIR.  The problem is, the do_unpack task is looking for
>> the git repo in the DL_DIR (git2/etc...)... but it's looking for the git
>> repo from github, not my local repo that the fetch task just put in DL_DIR.
>>
>> There are numerous reasons I'm doing this including the fact that I
>> cannot put a reference to my local repo in the bb file since I ship that
>> recipe to my customers and a local pathname is meaningless to them.  I
>> prefer to simply modify my local.conf with a PREMIRROR value that has a
>> specific regular expression that matches this particular github repo (and
>> hence does NOT effect all recipes).  This is why I wanted to use the
>> PREMIRROR function.  Unfortunately, it does not appear to work or at least
>> not work as I expected.
>>
>> --Russ
>>
>>
>>
>>
>> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif 
>> wrote:
>>
>>> Hi Russell,
>>>
>>> devtool and eSDK are different things. The purpose of PREMIRRORS is to
>>> set a mirror for all recipes. It's a way for organizations to control where
>>> their YP builds download sources from. It's not intended to be used for a