szetszwo commented on PR #6613: URL: https://github.com/apache/ozone/pull/6613#issuecomment-3452767811
> Suggested approach: Non-blocking API (gRPC) + Non-blocking threads More details: An application opens an input stream and calls read(L) (blocking call), where L is the requested data length: - Ozone client checks the buffer queue if it already has the data. - If not, send an onNext(L) request to a datanode. - Pull data from the buffer queue until L bytes and then return to the application When a datanode receives onNext(L) - Open the file if it is the first call. - Read the file at least L bytes of data. - It could read up to checksum/chunk boundary, or even a few chunks more for pre-read. - Return the data by one or more onNext() responses. When Ozone client receive onNext() responses - Put the data in the buffer queue. When the application closes the input stream: - The Ozone client sends an onComplete() to the datanode. - When the datanode receives onComplete(), close the file. When the datanode hits EOF of the block file: - The datanode closes the file and sends onComplete() to the Ozone client. - Ozone client closes the gRPC stream and continue with the next block. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
