Hello Willy,

Thank you for your feedbacks.

On Sat, May 9, 2020 at 6:39 AM Willy Tarreau <w...@1wt.eu> wrote:
> Your work could be extended to other files, like errorfiles, maps, ACLs,
> etc. However with the config parser being very linear, it's extremely
> unlikely that we could actually benefit from this in places other than
> certs. What makes certs specific is that a single config line can induce
> hundreds of thousands of file accesses and that these ones do benefit in
> being performed in parallel.

Yes, I take your comment as also something to improve in my commit
message explanation.

> Despite this I'm wondering if we should think about a generic approach
> like an async file open with a callback on what to call once the file
> is opened, or optionally an automatic async read and a callback set up
> for received data. This would bring the benefit of allowing to perform
> runtime FS accesses without destroying performance: right now, if you
> try to access the file system from Lua or maybe from within one of the
> device detection libs, or if you'd like to reload some files from the
> CLI, the whole thread totally freezes until the I/O is completed, which
> is why such accesses are absolutely forbidden. But done asynchronously
> it could be different. We could imagine an async read API to be used
> from Lua. We could even imagine being able to trigger reloading certain
> files (ACL patterns or error files maybe) from the CLI, or even to serve
> static files. This will of course not solve the security issues that
> come with doing this, but at least it could solve the performance issue.

wow I did not thought that far, but we could make things more generic
indeed. I will try to keep that in mind and improve things along the
way.
Regarding the CLI, I'm however a bit worried about the `chroot`
support though, which is important from my point of view.

> Probably that the work should be split into a functional and an
> implementation layer. The functional layer is the ability to perform
> async file operations. The implementation is done using uring. But for
> other OSes we could imagine using an emulation layer based on AIO or
> even a bunch of threads, which could still possibly provide some
> benefits during startup when loading tons of certificates, especially
> when that's from spinning disks (not sure if those are still in use
> though, last time I saw one was when I tried to repair my old Ultra5
> a few months ago).

ok will try to keep that in mind. not sure I will be able to come up
with a 100% nice interface but I will at least try to have something
we can iterate on.

> From day one I've been totally convinced it's possible to create a new
> poller entirely based on io-uring. We could avoid a number of the ugly
> tricks we use in epoll to avoid useless epoll_ctl() calls. What's nice
> about our pollers is that they're entirely modular so it is perfectly
> possible to create your own and experiment with it.

yes totally. For now I've duplicated to a new poller, but depending on
the result and diff, I might simply propose a similar #ifdef IO_URING
in ev_poll.c to activate it or not. I will see the final result.
Regarding ev_epoll, there is also an hybrid implementation to use
`epoll_wait` but uring for all `epoll_ctl` calls.

Thanks a lot for your inputs,
-- 
William

Reply via email to