Re: [linux-yocto] 3.10, standard/base at 3.10.17, but reverts back to 3.10.10...
On 13-11-07 02:22 AM, Darren Hart wrote: On Wed, 2013-11-06 at 21:07 -0500, Bruce Ashfield wrote: On 11/6/2013, 4:40 PM, Darren Hart wrote: On Tue, 2013-11-05 at 22:00 -0500, Bruce Ashfield wrote: On 13-11-05 6:36 PM, Darren Hart wrote: On Tue, 2013-11-05 at 17:15 -0600, Tom Zanussi wrote: On Tue, 2013-11-05 at 14:52 -0800, Darren Hart wrote: I'm working on rewriting the minnow-io feature to just apply patches. It's working but something is seriously horked with 3.10 - or my 3.10 tree or I don't know. HALP! The first problem was it was building PREEMPT_RT from a linux-yocto recipe. Turns out the eg20t feature has a branch (even in the yoctoproject.org git repo) which includes the preempt-rt sources, so those were all getting merged in. I removed the eg20t branch command and now preempt-rt isn't getting merged in. First I don't understand why eg20t even has a branch (did I do that?). Second, why would it contain I'm not sure how the eg20t branch could be getting merged in - the eg20t feature doesn't contain a 'git merge', and is just there for the .cfg. The eg20t branch originally contained about 70 eg20t patches before they'd gone upstream, and is now useless AFAIK, so should probably be removed. PREEMPT_RT? Fumbled upgrade? Sounds like it, but based on the above it shouldn't really matter as it shouldn't be used at all.. Right, it didn't have a merge command, just a branch: $ cat meta/cfg/kernel-cache/features/eg20t/eg20t.scc # changes should be staged on eg20t git branch eg20t master kconf hardware eg20t.cfg Still, including this scc brought in the eg20t branch. Perhaps if the branch name exists the branch command does a merge? Bug? We picked this up last week during some integration work @ Wind. The git branch was only used when staging the branch, and shouldn't have been in the .scc .. BSPs that are still including it are only working because they aren't adding any patches AFTER the command. In your case, you are .. hence why things are going insane. I have a change queued that drops the branch command, and moves eg20t to staging, and then to be killed, as Tom points out. I've confirmed this reverting to 3.10.10 happens in the do_patch() task, but I don't know where or why yet. Have you tried the same build with the branch command removed ? I removed eg20t from minnow.scc completely and I removed all features added in the recipe. The do_patch() stage still reverts from 3.10.17 to 3.10.10. I'll start instrumenting the tools I guess. It's not the tools, they don't mess with SRCREVs like this. My guess is that you are hitting a corner case in the do_validate_branches. To solve some issues with dynamic branches, they reset all branches that contain the machine's SRCREV to that value, and then start processing changes. This happens in patchme do_apply() somewhere. I'm currently instrumenting. I'll have a pile of cleanups and instrumentation patches ... hopefully tomorrow. My instrumentation tells this story so far: NOTE: git HEAD prior to patchme: c03195e kconfig: inhibit unitialized variable warning meta-dir(): meta-branch=meta meta-dir(): using meta_dir=.meta ktgt=minnow meta_series= branch=standard/base meta_series=.meta/cfg/scratch/minnow-standard-meta Git HEAD: c03195e kconfig: inhibit unitialized variable warning do_init() Git HEAD: c03195e kconfig: inhibit unitialized variable warning Git HEAD: c03195e kconfig: inhibit unitialized variable warning do_apply() [INFO] validating against known patches (minnow-standard-meta) [##] (completed in 6 seconds) Git HEAD: ebc8428 Merge tag 'v3.10.10' into standard/base NOTE: git HEAD after patchme: ebc8428 Merge tag 'v3.10.10' into standard/base So double check two things. What is your machine's SRCREV ? Is it the 3.10 hash ? It's the HEAD of my standard/base, which is 3.10.17 And try doing a build with AUTOREV, that disables the branch reset functionality and will point the smoking gun. OK, let's give that a try but it happens patchme, so it should fix it Same deal. BUT! AHA. Some git branch --contains on the commits listed in the output above reveal that standard/minnow HEAD is v3.10.10. Now, it is supposed to be using standard/base, but I think I've seen this before, if the tools try to create a branch that already exists, they just use the existing one. Delete standard/minnow from the publish tree... and vwhalla... no more resetting to an older tree. So some kind of 'bbwarn Hey dummy, this branch already exists' is in order. Digging in to see if I can find where and include it in my instrumentation patches. But typically, that is an expected / good thing :) If you are working from a tree with an existing git branch, that's the content you want to use. Fundamentally you work from branches as a human, and that's what the tools want to do .. let you work from the content you put in place. SRCREVs are fine for a build system ..
Re: [linux-yocto] 3.10, standard/base at 3.10.17, but reverts back to 3.10.10...
On 13-11-07 02:40 AM, Darren Hart wrote: On Wed, 2013-11-06 at 23:22 -0800, Darren Hart wrote: On Wed, 2013-11-06 at 21:07 -0500, Bruce Ashfield wrote: On 11/6/2013, 4:40 PM, Darren Hart wrote: On Tue, 2013-11-05 at 22:00 -0500, Bruce Ashfield wrote: On 13-11-05 6:36 PM, Darren Hart wrote: On Tue, 2013-11-05 at 17:15 -0600, Tom Zanussi wrote: On Tue, 2013-11-05 at 14:52 -0800, Darren Hart wrote: I'm working on rewriting the minnow-io feature to just apply patches. It's working but something is seriously horked with 3.10 - or my 3.10 tree or I don't know. HALP! The first problem was it was building PREEMPT_RT from a linux-yocto recipe. Turns out the eg20t feature has a branch (even in the yoctoproject.org git repo) which includes the preempt-rt sources, so those were all getting merged in. I removed the eg20t branch command and now preempt-rt isn't getting merged in. First I don't understand why eg20t even has a branch (did I do that?). Second, why would it contain I'm not sure how the eg20t branch could be getting merged in - the eg20t feature doesn't contain a 'git merge', and is just there for the .cfg. The eg20t branch originally contained about 70 eg20t patches before they'd gone upstream, and is now useless AFAIK, so should probably be removed. PREEMPT_RT? Fumbled upgrade? Sounds like it, but based on the above it shouldn't really matter as it shouldn't be used at all.. Right, it didn't have a merge command, just a branch: $ cat meta/cfg/kernel-cache/features/eg20t/eg20t.scc # changes should be staged on eg20t git branch eg20t master kconf hardware eg20t.cfg Still, including this scc brought in the eg20t branch. Perhaps if the branch name exists the branch command does a merge? Bug? We picked this up last week during some integration work @ Wind. The git branch was only used when staging the branch, and shouldn't have been in the .scc .. BSPs that are still including it are only working because they aren't adding any patches AFTER the command. In your case, you are .. hence why things are going insane. I have a change queued that drops the branch command, and moves eg20t to staging, and then to be killed, as Tom points out. I've confirmed this reverting to 3.10.10 happens in the do_patch() task, but I don't know where or why yet. Have you tried the same build with the branch command removed ? I removed eg20t from minnow.scc completely and I removed all features added in the recipe. The do_patch() stage still reverts from 3.10.17 to 3.10.10. I'll start instrumenting the tools I guess. It's not the tools, they don't mess with SRCREVs like this. My guess is that you are hitting a corner case in the do_validate_branches. To solve some issues with dynamic branches, they reset all branches that contain the machine's SRCREV to that value, and then start processing changes. This happens in patchme do_apply() somewhere. I'm currently instrumenting. I'll have a pile of cleanups and instrumentation patches ... hopefully tomorrow. My instrumentation tells this story so far: NOTE: git HEAD prior to patchme: c03195e kconfig: inhibit unitialized variable warning meta-dir(): meta-branch=meta meta-dir(): using meta_dir=.meta ktgt=minnow meta_series= branch=standard/base meta_series=.meta/cfg/scratch/minnow-standard-meta Git HEAD: c03195e kconfig: inhibit unitialized variable warning do_init() Git HEAD: c03195e kconfig: inhibit unitialized variable warning Git HEAD: c03195e kconfig: inhibit unitialized variable warning do_apply() [INFO] validating against known patches (minnow-standard-meta) [##] (completed in 6 seconds) Git HEAD: ebc8428 Merge tag 'v3.10.10' into standard/base NOTE: git HEAD after patchme: ebc8428 Merge tag 'v3.10.10' into standard/base So double check two things. What is your machine's SRCREV ? Is it the 3.10 hash ? It's the HEAD of my standard/base, which is 3.10.17 And try doing a build with AUTOREV, that disables the branch reset functionality and will point the smoking gun. OK, let's give that a try but it happens patchme, so it should fix it Same deal. BUT! AHA. Some git branch --contains on the commits listed in the output above reveal that standard/minnow HEAD is v3.10.10. Now, it is supposed to be using standard/base, but I think I've seen this before, if the tools try to create a branch that already exists, they just use the existing one. Delete standard/minnow from the publish tree... and vwhalla... no more resetting to an older tree. So some kind of 'bbwarn Hey dummy, this branch already exists' is in order. Digging in to see if I can find where and include it in my instrumentation patches. 2 days later now I can test the new minnow-io and 3.10 BSP sigh. OK, it occurs in wrap_meta_series() at: source $_series.wrap Which makes further instrumentation rather complicated. I had a discussion with Arjan about code that needed to be easily
Re: [linux-yocto] 3.10, standard/base at 3.10.17, but reverts back to 3.10.10...
On 13-11-07 11:28 AM, Darren Hart wrote: On Thu, 2013-11-07 at 08:58 -0500, Bruce Ashfield wrote: On 13-11-07 02:22 AM, Darren Hart wrote: BUT! AHA. Some git branch --contains on the commits listed in the output above reveal that standard/minnow HEAD is v3.10.10. Now, it is supposed to be using standard/base, but I think I've seen this before, if the tools try to create a branch that already exists, they just use the existing one. Delete standard/minnow from the publish tree... and vwhalla... no more resetting to an older tree. So some kind of 'bbwarn Hey dummy, this branch already exists' is in order. Digging in to see if I can find where and include it in my instrumentation patches. But typically, that is an expected / good thing :) If you are working from a tree with an existing git branch, that's the content you want to use. Fundamentally you work from branches as a human, and that's what the tools want to do .. let you work from the content you put in place. SRCREVs are fine for a build system .. but as we all know, by themselves they aren't something we can grab onto and remember. Hence why branches (iterative development) is balanced with SRCREV processing. The tools will already indicate if you are re-using a branch, but since things are no longer verbosely put to the build screen, you'd have to hunt that up from a build log. But it didn't. This change to standard/minnow happened in patchme as part of the do_patch task and there is next to nothing written to the log.do_patch. There is quite a bit of improvement to be done in terms of how logging interacts with bitbake - I have the start of a logging mechanisms similar to bitbake (but not dependent on it) which I'd like to be able to pass in debug level to from bitbake which would also allow it to be used from the command line. But, other than a bunch of hashes and control characters, very little is written to do_patch around the the code that changes branches. I've removed that for the verbose logging in the latest set of updates, or to be precise, a V=1 type logging that drops the spinner and shows you everything that happens in behind. With that, you'll see the existing logs that say branch foo (reuse), when reusing a branch. SRCREV does matter, and right now it only works in the opposite order that you were hitting ..if you branch is newer than the SRCREV, it is reset. If it isn't as far along, you get what is on the branch. Open a bug and I'll make that other case be handled in 1.6. OK, but I'm not quite sure what the nature of the bug is yet. So the scenario is this. KBRANCH is standard/base The BSP for minnow-standard.scc does: include ktypes/standard branch minnow There is some ambiguity in semantics of the branch command. From the kernel-dev manual: Notice the branch fri2 command, which creates a machine-specific branch into which source changes are applied. But also: Another method is to use the branch command in the BSP description: mybsp.scc: ... branch mynewbranch So, there appear to be two uses of the same command: 1) Create a branch from the current HEAD on which to apply additional features (merges,patches,etc). 2) Checkout an existing branch, and then do the patches/merges I wouldn't call those different uses, it is always: - create branch from current position, re-use if there - do more work. Trust me on this, we cannot error if the branch already exists. Each and every time you build what is described in the meta data is validated against the tree, so it absolutely expects and deals with being run many times on the same tree. Bruce In general, the problem for me is that branch implies checkout. THe instrumentation problem comes in that we are executing a generated file, rather than marching through it, which make instrumenting it cumbersome. Perhaps there is a good way, to put some print statements and if this then bail out with a horrible message code in the generated code, but the tooling is generic enough as make this difficult. I'll comment on this more in your other response after I drive in... ___ linux-yocto mailing list linux-yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/linux-yocto
Re: [linux-yocto] 3.10, standard/base at 3.10.17, but reverts back to 3.10.10...
On Tue, 2013-11-05 at 14:52 -0800, Darren Hart wrote: I'm working on rewriting the minnow-io feature to just apply patches. It's working but something is seriously horked with 3.10 - or my 3.10 tree or I don't know. HALP! The first problem was it was building PREEMPT_RT from a linux-yocto recipe. Turns out the eg20t feature has a branch (even in the yoctoproject.org git repo) which includes the preempt-rt sources, so those were all getting merged in. I removed the eg20t branch command and now preempt-rt isn't getting merged in. First I don't understand why eg20t even has a branch (did I do that?). Second, why would it contain PREEMPT_RT? Fumbled upgrade? I also dropped emgd to avoid any mess there. Now, the minnow-io patches are still applied, but instead of being at 3.10.17 from standard/base, we're back at 3.10.10. Any ideas what might be going on here? $ git l e683d6f minnowboard-keys: Bind MinnowBoard buttons to arrow keys 8ae33bc minnowboard-gpio: Export MinnowBoard expansion GPIO cebeec1 minnowboard: Add base platform driver for the MinnowBoard e3c69f2 gpio-sch: Add sch_gpio_resume_set_enable() ebc8428 Merge tag 'v3.10.10' into standard/base b76f445 yaffs2: disable procfs support ecfe5ed yaffs2: convert to kuid_t and kgid_t fa8efc9 yaffs2: restore multi-kernel version functionality 8bf3379 Linux 3.10.10 I'm building from master with my meta changes. It does this without my minnow-io patch as well -- Darren Hart Intel Open Source Technology Center Yocto Project - Linux Kernel ___ linux-yocto mailing list linux-yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/linux-yocto
Re: [linux-yocto] 3.10, standard/base at 3.10.17, but reverts back to 3.10.10...
On Tue, 2013-11-05 at 17:15 -0600, Tom Zanussi wrote: On Tue, 2013-11-05 at 14:52 -0800, Darren Hart wrote: I'm working on rewriting the minnow-io feature to just apply patches. It's working but something is seriously horked with 3.10 - or my 3.10 tree or I don't know. HALP! The first problem was it was building PREEMPT_RT from a linux-yocto recipe. Turns out the eg20t feature has a branch (even in the yoctoproject.org git repo) which includes the preempt-rt sources, so those were all getting merged in. I removed the eg20t branch command and now preempt-rt isn't getting merged in. First I don't understand why eg20t even has a branch (did I do that?). Second, why would it contain I'm not sure how the eg20t branch could be getting merged in - the eg20t feature doesn't contain a 'git merge', and is just there for the .cfg. The eg20t branch originally contained about 70 eg20t patches before they'd gone upstream, and is now useless AFAIK, so should probably be removed. PREEMPT_RT? Fumbled upgrade? Sounds like it, but based on the above it shouldn't really matter as it shouldn't be used at all.. Right, it didn't have a merge command, just a branch: $ cat meta/cfg/kernel-cache/features/eg20t/eg20t.scc # changes should be staged on eg20t git branch eg20t master kconf hardware eg20t.cfg Still, including this scc brought in the eg20t branch. Perhaps if the branch name exists the branch command does a merge? Bug? I've confirmed this reverting to 3.10.10 happens in the do_patch() task, but I don't know where or why yet. I also dropped emgd to avoid any mess there. Now, the minnow-io patches are still applied, but instead of being at 3.10.17 from standard/base, we're back at 3.10.10. Any ideas what might be going on here? $ git l e683d6f minnowboard-keys: Bind MinnowBoard buttons to arrow keys 8ae33bc minnowboard-gpio: Export MinnowBoard expansion GPIO cebeec1 minnowboard: Add base platform driver for the MinnowBoard e3c69f2 gpio-sch: Add sch_gpio_resume_set_enable() ebc8428 Merge tag 'v3.10.10' into standard/base b76f445 yaffs2: disable procfs support ecfe5ed yaffs2: convert to kuid_t and kgid_t fa8efc9 yaffs2: restore multi-kernel version functionality 8bf3379 Linux 3.10.10 I'm building from master with my meta changes. Happens from dora too. -- Darren Hart Intel Open Source Technology Center Yocto Project - Linux Kernel ___ linux-yocto mailing list linux-yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/linux-yocto
Re: [linux-yocto] 3.10, standard/base at 3.10.17, but reverts back to 3.10.10...
On 13-11-05 6:36 PM, Darren Hart wrote: On Tue, 2013-11-05 at 17:15 -0600, Tom Zanussi wrote: On Tue, 2013-11-05 at 14:52 -0800, Darren Hart wrote: I'm working on rewriting the minnow-io feature to just apply patches. It's working but something is seriously horked with 3.10 - or my 3.10 tree or I don't know. HALP! The first problem was it was building PREEMPT_RT from a linux-yocto recipe. Turns out the eg20t feature has a branch (even in the yoctoproject.org git repo) which includes the preempt-rt sources, so those were all getting merged in. I removed the eg20t branch command and now preempt-rt isn't getting merged in. First I don't understand why eg20t even has a branch (did I do that?). Second, why would it contain I'm not sure how the eg20t branch could be getting merged in - the eg20t feature doesn't contain a 'git merge', and is just there for the .cfg. The eg20t branch originally contained about 70 eg20t patches before they'd gone upstream, and is now useless AFAIK, so should probably be removed. PREEMPT_RT? Fumbled upgrade? Sounds like it, but based on the above it shouldn't really matter as it shouldn't be used at all.. Right, it didn't have a merge command, just a branch: $ cat meta/cfg/kernel-cache/features/eg20t/eg20t.scc # changes should be staged on eg20t git branch eg20t master kconf hardware eg20t.cfg Still, including this scc brought in the eg20t branch. Perhaps if the branch name exists the branch command does a merge? Bug? We picked this up last week during some integration work @ Wind. The git branch was only used when staging the branch, and shouldn't have been in the .scc .. BSPs that are still including it are only working because they aren't adding any patches AFTER the command. In your case, you are .. hence why things are going insane. I have a change queued that drops the branch command, and moves eg20t to staging, and then to be killed, as Tom points out. I've confirmed this reverting to 3.10.10 happens in the do_patch() task, but I don't know where or why yet. Have you tried the same build with the branch command removed ? I also dropped emgd to avoid any mess there. Now, the minnow-io patches are still applied, but instead of being at 3.10.17 from standard/base, we're back at 3.10.10. Any ideas what might be going on here? $ git l e683d6f minnowboard-keys: Bind MinnowBoard buttons to arrow keys 8ae33bc minnowboard-gpio: Export MinnowBoard expansion GPIO cebeec1 minnowboard: Add base platform driver for the MinnowBoard e3c69f2 gpio-sch: Add sch_gpio_resume_set_enable() ebc8428 Merge tag 'v3.10.10' into standard/base b76f445 yaffs2: disable procfs support ecfe5ed yaffs2: convert to kuid_t and kgid_t fa8efc9 yaffs2: restore multi-kernel version functionality 8bf3379 Linux 3.10.10 I'm building from master with my meta changes. Happens from dora too. dora and master are virtually identical, so it isn't surprising. Bruce ___ linux-yocto mailing list linux-yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/linux-yocto
