Re: [OE-core] [PATCH v2] uninative: Update to 2.7 release

[Stefan Agner]

On 2019-10-11 10:26, Stefan Agner wrote:
> On 2019-10-07 18:47, Michael Halstead wrote:
>> The 2.7 release updates glibc to version 2.30. Recently added to openSUSE
>> Tumbleweed and needed for Fedora Core 31.
> 
> Since we updated master to include this commit we see regularly the following
> issues:
> 
> 
> WARNING: Disabling uninative as unable to install uninative tarball:
> Command 'mkdir -p /workdir/oe/tmp/sysroots-uninative; cd
> /workdir/oe/tmp/sysroots-uninative; tar -xJf
> /workdir/downloads/uninative//9498d8bba04749a7310ac2576d0796461184965351a56f6d32c888a1f216/x86_64-nativesdk-libc.tar.xz;
> /workdir/oe/tmp/sysroots-uninative/relocate_sdk.py  
> /workdir/oe/tmp/sysroots-uninative/x86_64-linux
> /workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
>  
> /workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
>  
> /workdir/oe/tmp/sysroots-uninative/x86_64-linux//usr/bin/patchelf-uninative
>
> /workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libc*.so' returned
> non-zero exit status 1.
> WARNING: To build your own uninative loader, please bitbake
> uninative-tarball and set UNINATIVE_TARBALL appropriately.
> 
> [...]
> 
> ERROR: ldconfig-native-2.12.1-r2 do_populate_sysroot_setscene: Error
> executing a python function in exec_python_func() autogenerated:
> 
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_python_func() autogenerated', lineno: 2, function: 
>  0001:
>  *** 0002:uninative_changeinterp(d)
>  0003:
> File:
> '/workdir/oe/build/conf/../../layers/openembedded-core/meta/classes/uninative.bbclass',
> lineno: 165, function: uninative_changeinterp
>  0161:continue
>  0162:if not elf.isDynamic():
>  0163:continue
>  0164:
>  *** 0165:subprocess.check_output(("patchelf-uninative",
> "--set-interpreter", d.getVar("UNINATIVE_LOADER"), f),
> stderr=subprocess.STDOUT)
>  0166:}
> File: '/usr/lib/python3.6/subprocess.py', lineno: 336, function: check_output
>  0332:# empty string. That is maintained here for
> backwards compatibility.
>  0333:kwargs['input'] = '' if
> kwargs.get('universal_newlines', False) else b''
>  0334:
>  0335:return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
>  *** 0336:   **kwargs).stdout
>  0337:
>  0338:
>  0339:class CompletedProcess(object):
>  0340:"""A process that has finished running.
> File: '/usr/lib/python3.6/subprocess.py', lineno: 403, function: run
>  0399:if 'stdin' in kwargs:
>  0400:raise ValueError('stdin and input arguments may
> not both be used.')
>  0401:kwargs['stdin'] = PIPE
>  0402:
>  *** 0403:with Popen(*popenargs, **kwargs) as process:
>  0404:try:
>  0405:stdout, stderr = process.communicate(input,
> timeout=timeout)
>  0406:except TimeoutExpired:
>  0407:process.kill()
> File: '/usr/lib/python3.6/subprocess.py', lineno: 709, function: __init__
>  0705:startupinfo, creationflags, shell,
>  0706:p2cread, p2cwrite,
>  0707:c2pread, c2pwrite,
>  0708:errread, errwrite,
>  *** 0709:restore_signals, start_new_session)
>  0710:except:
>  0711:# Cleanup if the child failed starting.
>  0712:for f in filter(None, (self.stdin, self.stdout,
> self.stderr)):
>  0713:try:
> File: '/usr/lib/python3.6/subprocess.py', lineno: 1344, function: 
> _execute_child
>  1340:if errno_num != 0:
>  1341:err_msg = os.strerror(errno_num)
>  1342:if errno_num == errno.ENOENT:
>  1343:err_msg += ': ' + repr(err_filename)
>  *** 1344:raise child_exception_type(errno_num,
> err_msg, err_filename)
>  1345:raise child_exception_type(err_msg)
>  1346:
>  1347:
>  1348:def _handle_exitstatus(self, sts, 
> _WIFSIGNALED=os.WIFSIGNALED,
> Exception: FileNotFoundError: [Errno 2] No such file or directory:
> 'patchelf-uninative': 'patchelf-uninative'
> 
> ERROR: Logfile of failure stored in:
> /workdir/oe/tmp/work/x86_64-linux/ldconfig-native/2.12.1-r2/temp/log.do_populate_sysroot_setscene.14967
> 
> 
> 
> Looking into the directory shows that uninative only got extracted half-way:
> $ find /workdir/oe/tmp/sysroots-uninative
> /workdir/oe/tmp/sysroots-uninative
> /workdir/oe/tmp/sysroots-uninative/x86_64-linux
> /workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib
> /workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libresolv-2.30.so
> /workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libnsl.so.1
> /workdir/oe/tmp/sysroots-

Re: [OE-core] [PATCH v2] uninative: Update to 2.7 release

[Richard Purdie]

On Fri, 2019-10-11 at 10:26 +0200, Stefan Agner wrote:
> We run several build in parallel using the same downloads folder, it
> seems to me that
> there is a race condition. The history shows the warning several
> times already with
> exit code 127, but in those instances the bulid succeeded 
> 
> What I don't understand is how the update to 2.7 release could affect
> the behavior
> such that it suddenly leads to issues. Any idea?

We use the bitbake fetcher for uninative so there should be locking,
the same as any other download.

Are these builds in parallel on different machines with a shared
network filesystem for downloads or the same physical machine?

Are you using any special MIRRORS/PREMIRRORS?

FWIW we run many builds on the autobuilder and don't see this. I also
haven't seen any other reports of it.

I'm therefore at a loss as to why the locking would seem to be breaking
for you :(

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH v2] uninative: Update to 2.7 release

[Stefan Agner]

On 2019-10-07 18:47, Michael Halstead wrote:
> The 2.7 release updates glibc to version 2.30. Recently added to openSUSE
> Tumbleweed and needed for Fedora Core 31.

Since we updated master to include this commit we see regularly the following
issues:


WARNING: Disabling uninative as unable to install uninative tarball: Command 
'mkdir -p /workdir/oe/tmp/sysroots-uninative; cd 
/workdir/oe/tmp/sysroots-uninative; tar -xJf
/workdir/downloads/uninative//9498d8bba04749a7310ac2576d0796461184965351a56f6d32c888a1f216/x86_64-nativesdk-libc.tar.xz;
 /workdir/oe/tmp/sysroots-uninative/relocate_sdk.py   
/workdir/oe/tmp/sysroots-uninative/x86_64-linux  
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2   
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2   
/workdir/oe/tmp/sysroots-uninative/x86_64-linux//usr/bin/patchelf-uninative  
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libc*.so' returned non-zero 
exit status 1.
WARNING: To build your own uninative loader, please bitbake uninative-tarball 
and set UNINATIVE_TARBALL appropriately.

[...]

ERROR: ldconfig-native-2.12.1-r2 do_populate_sysroot_setscene: Error executing 
a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: 
 0001:
 *** 0002:uninative_changeinterp(d)
 0003:
File: 
'/workdir/oe/build/conf/../../layers/openembedded-core/meta/classes/uninative.bbclass',
 lineno: 165, function: uninative_changeinterp
 0161:continue
 0162:if not elf.isDynamic():
 0163:continue
 0164:
 *** 0165:subprocess.check_output(("patchelf-uninative", 
"--set-interpreter", d.getVar("UNINATIVE_LOADER"), f), stderr=subprocess.STDOUT)
 0166:}
File: '/usr/lib/python3.6/subprocess.py', lineno: 336, function: check_output
 0332:# empty string. That is maintained here for backwards 
compatibility.
 0333:kwargs['input'] = '' if kwargs.get('universal_newlines', 
False) else b''
 0334:
 0335:return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
 *** 0336:   **kwargs).stdout
 0337:
 0338:
 0339:class CompletedProcess(object):
 0340:"""A process that has finished running.
File: '/usr/lib/python3.6/subprocess.py', lineno: 403, function: run
 0399:if 'stdin' in kwargs:
 0400:raise ValueError('stdin and input arguments may not both 
be used.')
 0401:kwargs['stdin'] = PIPE
 0402:
 *** 0403:with Popen(*popenargs, **kwargs) as process:
 0404:try:
 0405:stdout, stderr = process.communicate(input, 
timeout=timeout)
 0406:except TimeoutExpired:
 0407:process.kill()
File: '/usr/lib/python3.6/subprocess.py', lineno: 709, function: __init__
 0705:startupinfo, creationflags, shell,
 0706:p2cread, p2cwrite,
 0707:c2pread, c2pwrite,
 0708:errread, errwrite,
 *** 0709:restore_signals, start_new_session)
 0710:except:
 0711:# Cleanup if the child failed starting.
 0712:for f in filter(None, (self.stdin, self.stdout, 
self.stderr)):
 0713:try:
File: '/usr/lib/python3.6/subprocess.py', lineno: 1344, function: _execute_child
 1340:if errno_num != 0:
 1341:err_msg = os.strerror(errno_num)
 1342:if errno_num == errno.ENOENT:
 1343:err_msg += ': ' + repr(err_filename)
 *** 1344:raise child_exception_type(errno_num, err_msg, 
err_filename)
 1345:raise child_exception_type(err_msg)
 1346:
 1347:
 1348:def _handle_exitstatus(self, sts, _WIFSIGNALED=os.WIFSIGNALED,
Exception: FileNotFoundError: [Errno 2] No such file or directory: 
'patchelf-uninative': 'patchelf-uninative'

ERROR: Logfile of failure stored in: 
/workdir/oe/tmp/work/x86_64-linux/ldconfig-native/2.12.1-r2/temp/log.do_populate_sysroot_setscene.14967



Looking into the directory shows that uninative only got extracted half-way:
$ find /workdir/oe/tmp/sysroots-uninative
/workdir/oe/tmp/sysroots-uninative
/workdir/oe/tmp/sysroots-uninative/x86_64-linux
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libresolv-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libnsl.so.1
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libanl.so.1
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libnss_compat-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/libpthread-2.30.so
/workdir/oe/tmp/sysroots-uninative/x86_64-linux/lib/l

Re: [OE-core] [PATCH v2] uninative: Update to 2.7 release

[Khem Raj]

this helps archlinux too

On Mon, Oct 7, 2019 at 9:48 AM Michael Halstead
 wrote:
>
> The 2.7 release updates glibc to version 2.30. Recently added to openSUSE
> Tumbleweed and needed for Fedora Core 31.
>
> Signed-off-by: Michael Halstead 
> ---
>  meta/conf/distro/include/yocto-uninative.inc | 10 +-
>  1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/meta/conf/distro/include/yocto-uninative.inc 
> b/meta/conf/distro/include/yocto-uninative.inc
> index df24346855..ad75d3e2a3 100644
> --- a/meta/conf/distro/include/yocto-uninative.inc
> +++ b/meta/conf/distro/include/yocto-uninative.inc
> @@ -6,9 +6,9 @@
>  # to the distro running on the build machine.
>  #
>
> -UNINATIVE_MAXGLIBCVERSION = "2.29"
> +UNINATIVE_MAXGLIBCVERSION = "2.30"
>
> -UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/2.6/";
> -UNINATIVE_CHECKSUM[aarch64] ?= 
> "a37118fc8b423f48146120707b81dd15017512c3e8ef9e6ca2cb3a033f4f4046"
> -UNINATIVE_CHECKSUM[i686] ?= 
> "3234fc3ded810225071f23a0e9a99f4f8c2480059945a848eff076ce78122ade"
> -UNINATIVE_CHECKSUM[x86_64] ?= 
> "133387753a9acf3e1b788103c59fac91e968e2ee331d7a4b9498e926ada7be57"
> +UNINATIVE_URL ?= "http://downloads.yoctoproject.org/releases/uninative/2.7/";
> +UNINATIVE_CHECKSUM[aarch64] ?= 
> "e76a45886ee8a0b3904b761c17ac8ff91edf9811ee455f1832d10763ba794dfc"
> +UNINATIVE_CHECKSUM[i686] ?= 
> "810d027dfb1c7675226afbcec07808770516c969ee7378f6d8240281083f8924"
> +UNINATIVE_CHECKSUM[x86_64] ?= 
> "9498d8bba04749a7310ac2576d0796461184965351a56f6d32c888a1f216"
> --
> 2.21.0
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core