On Sun, 2023-09-17 at 10:54 +0100, Richard Purdie via
lists.openembedded.org wrote:
> On Fri, 2023-09-15 at 22:08 +0100, Richard Purdie via
> lists.openembedded.org wrote:
> > Just to follow up with where I got to with this.
> >
> > The problem is that inotify isn't delivering change notifications in a
> > timely fashion.
> >
> > With the patch applied and BB_SERVER_TIMEOUT = "60", you can show an
> > issue with something like:
> >
> > rm bitbake-cookerdaemon.log
> > bitbake-layers add-layer ../meta-virtualization/
> > bitbake-layers add-layer ../meta-openembedded/meta-oe/
> > bitbake -m
> >
> > where the second add-layer fails with the same error as the first with
> > a message missing meta-virt dependencies. It doesn't notice
> > bblayers.conf has changed and hence doesn't re-parse the base
> > configuration. I've locally proven this is due to the inotify events
> > not arriving until after the parsing commands.
> >
> > os.sync() isn't enough to flush the inotify backlog. os.system("sync")
> > does but that may be the fork overhead. sys.stdout.flush() does too,
> > probably as side effects in the kernel IO workings.
> >
> > This basically brings me to the conclusion that it is hard to know when
> > the inotify events are in sync with the incoming command stream and the
> > current approach is flawed.
> >
> > I'm starting to think that we should drop inotify and go back to stat
> > calls and maybe require that UIs send "re-scan cache" events so we can
> > avoid the scanning when we don't need to. The client code does have a
> > pretty good idea of when it changes metadata, it just doesn't share
> > that at the moment. We can rescan at reconnection so it would just
> > affect tinfoil usecases.
>
> There are patches in master-next which replace inotify usage with
> stat() calls and mtime comparisons.
>
> For "bitbake" commands, I've made it revalidate the cache upon new
> commands so that part is relatively easy/safe.
>
> The tricky part is that we'd now require tinfoil usage to explicitly
> say when the client has modified metadata (or run all the stat calls
> between every tinfoil command which would have crazy overhead).
>
> This means we need to audit the tinfoil command usage to markup the
> places where it changes files and expects bitbake to notice.
>
> In master-next, I did audit bitbake-layers and recipetool but not
> devtool. I've run an autobuilder build just to get an idea of how close
> this gets us to a solution. Unsurprisingly devtool is exploding there
> but other areas look more promising.
>
> Ultimately we need to make a few more invasive changes to the way we
> control the server from the UI and be a bit more explicit about the
> command flow but that should wait until after release.
Final follow up to this. After some testing and sanity checking
my change with others I've merged a patch removing inotify usage in
bitbake. This then allowed the flush removal to merge too.
There are further things we should really fix but those are less urgent
and this removes the problems identified. It should hopefully help some
of the AB-INT bugs.
Anyone with custom tinfoil usage where they modify files will need to
add a function call but I doubt there are too many people doing that
(bitbake-layers, devtool and recipetool did).
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#1766):
https://lists.openembedded.org/g/openembedded-architecture/message/1766
Mute This Topic: https://lists.openembedded.org/mt/101375777/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
