On 3/3/23 06:36, Eshan Kelkar wrote:
Hi, I am Eshan Kelkar and would like to create the async sftp for libssh
as the GSOC project. I have gone through liburing to understand what
async i/o is and how it is implemented using io_uring. So in this async
sftp implementation I believe we can place calls to the liburing api
functions from inside of the async sftp api functions so that things
occur asynchronously.
Another approach that comes to my mind is that on a call to async sftp
api function a seperate thread gets created which does the waiting and
all and on completion places a call to the callback function notifying
that the operation has occurred. The second approach is async
conceptually as the user of api can continue his job after the call as
the waiting occurs on the separate thread but this approach seems a bit
"naive" as for each api call a new thread gets created which is resource
expensive.
Kindly comment on these two approaches and suggest any other approach
which you have in mind to implement the async sftp api, those
suggestions will help me prepare better before sending in the proposal.
Hello Eshan,
first of all, sorry for late reply. I saw your message on IRC, but
before I got back to reply, you were already away so thank you for
patience to reach out to us on other channels.
The async SFTP implementation is one of our priorities and one of the
more complicated tasks. I would suggest starting from the benchmark code
we have in tests/benchmarks/, which has an example of async donwload
(but there is only the download part -- the upload part does not exist now).
I do not think it is a good idea to spawn more threads as it would
require a lot of synchronization. The example in the benchmarks can run
several download requests from a single thread, which can help saturate
the network connection without the need for threads.
I did not read much about io_uring yet, but it sounds like it solves the
issues we have with speed of synchronous writes/reads caused by context
switching so this would be our preference. There are already some
hints/comments in the following issues, so if you will have some more
questions, comments, feel free to ask here or in either of the following
issues:
https://gitlab.com/libssh/libssh-mirror/-/issues/65
https://gitlab.com/libssh/libssh-mirror/-/issues/124
Regards,
--
Jakub Jelen
Crypto Team, Security Engineering
Red Hat, Inc.
