Re: [OE-core] [PATCH 1/4] base.bbclass: Remove invalid ccache related code
On 12/11/18 6:12 PM, Robert Yang wrote: Hi RP, On 12/11/18 5:44 PM, richard.pur...@linuxfoundation.org wrote: On Tue, 2018-12-11 at 17:24 +0800, Robert Yang wrote: [snip] I did a test: $ ln -s /usr/bin/ccache /folk/lyang1/bin/gcc I got errors with/without the code: $ bitbake -p bb.BBHandledException ERROR: Error running gcc --version: I've sent a patch to fix/explain this problem: [OE-core] [PATCH 0/1] lib/oe/utils: Set stderr for host_gcc_version() // Robert $ ls -l tmp/hosttools/gcc tmp/hosttools/gcc -> /folk/lyang1/bin/gcc $ ls -l /folk/lyang1/bin/gcc /folk/lyang1/bin/gcc -> /usr/bin/ccache It worked well if I run gcc --version manually: $ gcc --version gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609 Copyright (C) 2015 Free Software Foundation, Inc. $ echo $? 0 There must be something wrong in such a case. I will fix this later. Let's get back to he code: srctool = bb.utils.which(path, tool, executable=True) if "ccache" in srctool: srctool = bb.utils.which(path, tool, executable=True, direction=1) if srctool: os.symlink(srctool, desttool) * If tool == "gcc", then srctool would be gcc, too. * If tool == "ccache", then srctool would be ccache, too. So I think that the code doesn't help when gcc links to ccache ? // Robert Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/4] base.bbclass: Remove invalid ccache related code
On 12/11/18 6:42 PM, richard.pur...@linuxfoundation.org wrote: On Tue, 2018-12-11 at 18:12 +0800, Robert Yang wrote: On 12/11/18 5:44 PM, richard.pur...@linuxfoundation.org wrote: On Tue, 2018-12-11 at 17:24 +0800, Robert Yang wrote: On 12/10/18 6:29 PM, Richard Purdie wrote: No, that isn't correct :( The problem is that "gcc" can be linked directly to ccache by the host distro. This means you end up using the host ccache just by calling "gcc". This code is still needed if we want to control ccache and be able to disable it and use ccache-native for determinism. I did a test: $ ln -s /usr/bin/ccache /folk/lyang1/bin/gcc I got errors with/without the code: $ bitbake -p bb.BBHandledException ERROR: Error running gcc --version: $ ls -l tmp/hosttools/gcc tmp/hosttools/gcc -> /folk/lyang1/bin/gcc $ ls -l /folk/lyang1/bin/gcc /folk/lyang1/bin/gcc -> /usr/bin/ccache It worked well if I run gcc --version manually: $ gcc --version gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609 Copyright (C) 2015 Free Software Foundation, Inc. $ echo $? 0 There must be something wrong in such a case. I will fix this later. Let's get back to he code: srctool = bb.utils.which(path, tool, executable=True) if "ccache" in srctool: srctool = bb.utils.which(path, tool, executable=True, direction=1) if srctool: os.symlink(srctool, desttool) * If tool == "gcc", then srctool would be gcc, too. * If tool == "ccache", then srctool would be ccache, too. So I think that the code doesn't help when gcc links to ccache ? https://www.unix.com/man-page/suse/1/ccache/ When the distros we tested on did this they placed it in something like bin/ccache/ so the code detects the "ccache" element of the path returned by which() (something like /usr/bin/ccache/gcc but I don't remember exactly). Try using /folk/lyang1/ccache/ or /folk/lyang1/bin/ccache/ in your example. Ah, sorry, I mixed 'ccache in srctool' with 'ccache == srctool', yes, we do need it. I will keep the original code, and add comments for it. // Robert Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/4] base.bbclass: Remove invalid ccache related code
Sent from mobile phone > 在 2018年12月11日，18:43，"richard.pur...@linuxfoundation.org" > 写道： > >> On Tue, 2018-12-11 at 18:12 +0800, Robert Yang wrote: >>> On 12/11/18 5:44 PM, richard.pur...@linuxfoundation.org wrote: On Tue, 2018-12-11 at 17:24 +0800, Robert Yang wrote: On 12/10/18 6:29 PM, Richard Purdie wrote: >>> No, that isn't correct :( >>> >>> The problem is that "gcc" can be linked directly to ccache by the >>> host >>> distro. This means you end up using the host ccache just by calling >>> "gcc". This code is still needed if we want to control ccache and >>> be >>> able to disable it and use ccache-native for determinism. >> >> I did a test: >> >> $ ln -s /usr/bin/ccache /folk/lyang1/bin/gcc >> I got errors with/without the code: >> >> $ bitbake -p >> bb.BBHandledException >> ERROR: Error running gcc --version: >> >> $ ls -l tmp/hosttools/gcc >> tmp/hosttools/gcc -> /folk/lyang1/bin/gcc >> >> $ ls -l /folk/lyang1/bin/gcc >> /folk/lyang1/bin/gcc -> /usr/bin/ccache >> >> It worked well if I run gcc --version manually: >> $ gcc --version >> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609 >> Copyright (C) 2015 Free Software Foundation, Inc. >> $ echo $? >> 0 >> >> There must be something wrong in such a case. I will fix this later. >> >> Let's get back to he code: >> >> srctool = bb.utils.which(path, tool, executable=True) >> if "ccache" in srctool: >> srctool = bb.utils.which(path, tool, >> executable=True, direction=1) >> if srctool: >> os.symlink(srctool, desttool) >> >> >> * If tool == "gcc", then srctool would be gcc, too. >> * If tool == "ccache", then srctool would be ccache, too. >> >> So I think that the code doesn't help when gcc links to ccache ? > > https://www.unix.com/man-page/suse/1/ccache/ > > When the distros we tested on did this they placed it in something like > bin/ccache/ so the code detects the "ccache" element of the path > returned by which() (something like /usr/bin/ccache/gcc but I don't > remember exactly). > > Try using /folk/lyang1/ccache/ or /folk/lyang1/bin/ccache/ in your > example. Thanks, I will try it tomorrow. // Robert > > Cheers, > > Richard > > > > -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/4] base.bbclass: Remove invalid ccache related code
On Tue, 2018-12-11 at 18:12 +0800, Robert Yang wrote: > On 12/11/18 5:44 PM, richard.pur...@linuxfoundation.org wrote: > > On Tue, 2018-12-11 at 17:24 +0800, Robert Yang wrote: > > > On 12/10/18 6:29 PM, Richard Purdie wrote: > > No, that isn't correct :( > > > > The problem is that "gcc" can be linked directly to ccache by the > > host > > distro. This means you end up using the host ccache just by calling > > "gcc". This code is still needed if we want to control ccache and > > be > > able to disable it and use ccache-native for determinism. > > I did a test: > > $ ln -s /usr/bin/ccache /folk/lyang1/bin/gcc > I got errors with/without the code: > > $ bitbake -p > bb.BBHandledException > ERROR: Error running gcc --version: > > $ ls -l tmp/hosttools/gcc > tmp/hosttools/gcc -> /folk/lyang1/bin/gcc > > $ ls -l /folk/lyang1/bin/gcc > /folk/lyang1/bin/gcc -> /usr/bin/ccache > > It worked well if I run gcc --version manually: > $ gcc --version > gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609 > Copyright (C) 2015 Free Software Foundation, Inc. > $ echo $? > 0 > > There must be something wrong in such a case. I will fix this later. > > Let's get back to he code: > > srctool = bb.utils.which(path, tool, executable=True) > if "ccache" in srctool: > srctool = bb.utils.which(path, tool, > executable=True, direction=1) > if srctool: > os.symlink(srctool, desttool) > > > * If tool == "gcc", then srctool would be gcc, too. > * If tool == "ccache", then srctool would be ccache, too. > > So I think that the code doesn't help when gcc links to ccache ? https://www.unix.com/man-page/suse/1/ccache/ When the distros we tested on did this they placed it in something like bin/ccache/ so the code detects the "ccache" element of the path returned by which() (something like /usr/bin/ccache/gcc but I don't remember exactly). Try using /folk/lyang1/ccache/ or /folk/lyang1/bin/ccache/ in your example. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/4] base.bbclass: Remove invalid ccache related code
Hi RP, On 12/11/18 5:44 PM, richard.pur...@linuxfoundation.org wrote: On Tue, 2018-12-11 at 17:24 +0800, Robert Yang wrote: On 12/10/18 6:29 PM, Richard Purdie wrote: On Sun, 2018-12-09 at 18:24 -0800, Robert Yang wrote: The code is in setup_hosttools_dir(), it searches ccache in PATH in reverse order, I can't figure out what it is used for, maybe it is aimed to use host's ccache rather than recipe-sysroot- native/usr/bin/ccache, but the later one isn't in original PATH, so remove the code. Signed-off-by: Robert Yang --- meta/classes/base.bbclass | 2 -- 1 file changed, 2 deletions(-) We should document it and I think its still needed. Some distros have a "gcc" which automatically wraps ccache behind the scenes. This code lets us find the real gcc and bypass ccache. Ah, thanks for the explanations, I got it now. I think that we don't need the code any more since we don't use host's ccache, and I have removed ccache from HOSTTOOLS. I will update commit message and send a V2. No, that isn't correct :( The problem is that "gcc" can be linked directly to ccache by the host distro. This means you end up using the host ccache just by calling "gcc". This code is still needed if we want to control ccache and be able to disable it and use ccache-native for determinism. I did a test: $ ln -s /usr/bin/ccache /folk/lyang1/bin/gcc I got errors with/without the code: $ bitbake -p bb.BBHandledException ERROR: Error running gcc --version: $ ls -l tmp/hosttools/gcc tmp/hosttools/gcc -> /folk/lyang1/bin/gcc $ ls -l /folk/lyang1/bin/gcc /folk/lyang1/bin/gcc -> /usr/bin/ccache It worked well if I run gcc --version manually: $ gcc --version gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609 Copyright (C) 2015 Free Software Foundation, Inc. $ echo $? 0 There must be something wrong in such a case. I will fix this later. Let's get back to he code: srctool = bb.utils.which(path, tool, executable=True) if "ccache" in srctool: srctool = bb.utils.which(path, tool, executable=True, direction=1) if srctool: os.symlink(srctool, desttool) * If tool == "gcc", then srctool would be gcc, too. * If tool == "ccache", then srctool would be ccache, too. So I think that the code doesn't help when gcc links to ccache ? // Robert Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/4] base.bbclass: Remove invalid ccache related code
On Tue, 2018-12-11 at 17:24 +0800, Robert Yang wrote: > > On 12/10/18 6:29 PM, Richard Purdie wrote: > > On Sun, 2018-12-09 at 18:24 -0800, Robert Yang wrote: > > > The code is in setup_hosttools_dir(), it searches ccache in PATH > > > in > > > reverse order, I can't figure out what it is used for, maybe it > > > is > > > aimed to use host's ccache rather than recipe-sysroot- > > > native/usr/bin/ccache, but the later one isn't in original PATH, > > > so > > > remove the code. > > > > > > Signed-off-by: Robert Yang > > > --- > > > meta/classes/base.bbclass | 2 -- > > > 1 file changed, 2 deletions(-) > > > > We should document it and I think its still needed. > > > > Some distros have a "gcc" which automatically wraps ccache behind > > the > > scenes. This code lets us find the real gcc and bypass ccache. > > Ah, thanks for the explanations, I got it now. I think that we don't > need the > code any more since we don't use host's ccache, and I have removed > ccache from > HOSTTOOLS. > > I will update commit message and send a V2. No, that isn't correct :( The problem is that "gcc" can be linked directly to ccache by the host distro. This means you end up using the host ccache just by calling "gcc". This code is still needed if we want to control ccache and be able to disable it and use ccache-native for determinism. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/4] base.bbclass: Remove invalid ccache related code
On 12/10/18 6:29 PM, Richard Purdie wrote: On Sun, 2018-12-09 at 18:24 -0800, Robert Yang wrote: The code is in setup_hosttools_dir(), it searches ccache in PATH in reverse order, I can't figure out what it is used for, maybe it is aimed to use host's ccache rather than recipe-sysroot- native/usr/bin/ccache, but the later one isn't in original PATH, so remove the code. Signed-off-by: Robert Yang --- meta/classes/base.bbclass | 2 -- 1 file changed, 2 deletions(-) We should document it and I think its still needed. Some distros have a "gcc" which automatically wraps ccache behind the scenes. This code lets us find the real gcc and bypass ccache. Ah, thanks for the explanations, I got it now. I think that we don't need the code any more since we don't use host's ccache, and I have removed ccache from HOSTTOOLS. I will update commit message and send a V2. // Robert There would be oe-selftest failures on such a distro if we didn't do that. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/4] base.bbclass: Remove invalid ccache related code
On Sun, 2018-12-09 at 18:24 -0800, Robert Yang wrote: > The code is in setup_hosttools_dir(), it searches ccache in PATH in > reverse order, I can't figure out what it is used for, maybe it is > aimed to use host's ccache rather than recipe-sysroot- > native/usr/bin/ccache, but the later one isn't in original PATH, so > remove the code. > > Signed-off-by: Robert Yang > --- > meta/classes/base.bbclass | 2 -- > 1 file changed, 2 deletions(-) We should document it and I think its still needed. Some distros have a "gcc" which automatically wraps ccache behind the scenes. This code lets us find the real gcc and bypass ccache. There would be oe-selftest failures on such a distro if we didn't do that. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
