Apologies for the huge email: I'm afraid this is a long "tail" (as Alice thought).
So after some information from Richard, I've looked at the current Toaster task classification to see how we can: * present noexec tasks * present _setscene and their related sstate / executed tasks The later point is one of those pending design issues we have been dragging for a while. This is the problem: for each attempt to reuse an sstate object we have a task that ends in "_setscene" (we call them setscene tasks). If a setscene task succeeds, it becomes associated to a prebuilt task with outcome 'sstate', which means that an sstate object was found and restored. If a setscene task fails, the equivalent task will execute instead, which means that a failed setscene task becomes associated to an executed task that can in turn succeed or fail. All those tasks are separate tasks, but for presentation purposes it's probably better to show them together. Here is why. The question that is likely to be asked is: "why was this sstate object reused when I was expecting it to be built from scratch?" or "why is this being built from scratch when I was expecting it to be reused from sstate?". The relevant information to answer those questions is provided by both the setscene task and the task it is associated with (sstate or executed), and it would be nice to get all the information you need in a single place, instead of having to go back and forth between 2 tasks. Our current task classification cannot easily support the above. So I'd like to suggest some changes to it. Our current classification is as follows: we have 2 task types, executed and prebuilt. Executed tasks (which include setscene tasks) have the following outcomes: succeeded or failed. Prebuilt tasks have the following outcomes: existing, covered and sstate. Executed tasks have one of three script types: python, shell and noexec. Prebuilt tasks have (or will, once bug 5327 is fixed) their script type set to 'not applicable'. This creates 2 problems: * It shows setscene tasks separately from their associated sstate / executed tasks. * It bundles noexec tasks with executed tasks. Given that executed tasks are supposed to be the ones that execute, this makes no sense. This is what I propose we do instead. We will have still 2 task types, called 'executed' and 'not executed' (to make things crystal clear). Executed tasks can have the following outcomes: * succeeded * failed * sstate (why sstate here? Because the setscene tasks do execute. So these new executed-sstate tasks are in fact a successful setscene task + the prebuilt sstate task they are associated with). Succeeded and failed tasks can have script type "shell" or "python". Sstate tasks should have script type "not applicable". Not executed tasks can have the following outcomes: * prebuilt (these are what we now call 'existing' tasks, i.e, tasks for which a stamp file was found) * covered * noexec (these are the tasks with the [noexec] flag set to 1) All not executed tasks will have script type "not applicable". To see how sstate task information would look like, check: http://www.yoctoproject.org/toaster/task-details-prebuilt-sstate.html To see how executed tasks associated to a failed setscene task would look like, check: http://www.yoctoproject.org/toaster/task-details-sstate-missed.html There are advantages to these changes, but also some problems: * We'll need to rewrite the task order numbers. Both setscene and sstate tasks have a unique order number that indicate their sequence during the build process. Successful setscene tasks should probably adopt the sstate task order number. Failed setscene tasks should adopt the executed task order number. This effectively means that setscene tasks should not generate an order number. Since there can be no gaps in the order number sequence, we'll need to recalculate order numbers excluding the setscene tasks. * Warnings generated by failed setscene tasks will be hard to interpret, since nowhere in Toaster we will make explicit mention to setscene tasks or their order number. The warnings say: "Setscene task [order_number] ([path_to_bbfile], [task_name]) failed with exit code '1' - real task will be run instead". This is the biggest problem from my point of view. * Sstate tasks will have a task log ending in _setscene, but a task name without _setcene (for example, the log file of the task base-files.do_package-write will be called something like log.do_package_write_setscene). * This will not reflect what BitBake does (setscene and sstate tasks are separate tasks in the build system). This is not necessarily a bad thing, though. The goal of the GUI is not to present exactly what the build system does, but to present a coherent picture of what the build system does, a picture that is optimised to provide me with answers to my questions and problems. Let me know if you have any objections to the above. Also, it would be good to get a sense of how much work would be to implement the changes. We can easily test the new task classification with a few people before implementing it, to make sure it's the right way to go before doing any work. Thanks Belén On 12/11/2013 10:25, "Barros Pena, Belen" <[email protected]> wrote: >On 11/11/2013 18:13, "Richard Purdie" <[email protected]> >wrote: > >>On Mon, 2013-11-11 at 17:54 +0000, Barros Pena, Belen wrote: >>> It's been brought to my attention that we need to find a way to present >>> noexec tasks in Toaster (tasks with [noexec] = "1" set). I am trying to >>> figure out a way to fit them in the Toaster task classification, but I >>> think I am missing some context. >>> >>> Can someone explain how that [noexec] flag came to be and why was it >>> needed? A bit of history might help me come up with the right solution. >> >>When bitbake executes tasks, it forks off a new process which then sets >>up the environment and executes the task. This has overhead. >> >>We had a number of cases where we had "placeholder" tasks which were >>present for dependency reasons but were otherwise empty and had nothing >>to execute. We therefore decided to mark them "noexec" i.e. they have no >>executable content. >> >>Bitbake now knows that when it sees a noexec task, its there for >>dependency reasons only and it can skip the execution part which is good >>for performance. >> >>To see what I mean about dependency information, think about our >>packaging process. We have points in the build where we need to say >>"everything packaged" but don't care about specific package types. >>We therefore have a noexec do_package_write task which depends on >>do_package_write_{ipk,deb,rpm} depending on which are configured. >> >>There are also cases where we know the task will not do anything (e.g. >>package tasks of -native recipes). Marking them noexec means we can skip >>them faster. >> >>Does that help? > >It does help. A lot. It explains why they need to be shown (they are part >of the dependency chain) and confirms that they should not be bundled with >the executed tasks. > >Thanks!! > >> >>Cheers, >> >>Richard >> > >_______________________________________________ >toaster mailing list >[email protected] >https://lists.yoctoproject.org/listinfo/toaster _______________________________________________ toaster mailing list [email protected] https://lists.yoctoproject.org/listinfo/toaster
