szetszwo commented on PR #6613: URL: https://github.com/apache/ozone/pull/6613#issuecomment-3537172073
@sodonnel , Tried to add length to ReadBlockRequestProto. It works well. See https://github.com/szetszwo/ozone/commits/HDDS-10338b/ (please feel free to use the code.) Tested it with various client buffer sizes (17MB, 3.5MB, 4KB). Below is the case 3.5MB (=3565158). ``` XXX --------------------------------------------------------- XXX read 17825792 bytes with bufferSize 3565158 XXX --------------------------------------------------------- XXX initStreamRead conID: 1 locID: 115816896921600001 on datanode a85ea168-0eb0-46ed-9090-10c44ebd9c10(10.96.128.134/10.96.128.134) XXX position 0, length 3565158, requested 0, diff 3565158, rounded 3670016 XXX XceiverClientGrpc-1 -> dn-10c44ebd9c10_stream, send onNext request blockID { containerID: 1 localID: 115816896921600001 blockCommitSequenceId: 16 } offset: 0 length: 3670016 XXX server readBlock blockID { containerID: 1 localID: 115816896921600001 blockCommitSequenceId: 16 } offset: 0 length: 3670016 XXX server onNext response 0: dataLength=262144 XXX server onNext response 1: dataLength=262144 XXX server onNext response 2: dataLength=262144 XXX server onNext response 3: dataLength=262144 XXX server onNext response 4: dataLength=262144 XXX server onNext response 5: dataLength=262144 XXX server onNext response 6: dataLength=262144 XXX server onNext response 7: dataLength=262144 XXX server onNext response 8: dataLength=262144 XXX server onNext response 9: dataLength=262144 XXX server onNext response 10: dataLength=262144 XXX server onNext response 11: dataLength=262144 XXX server onNext response 12: dataLength=262144 XXX server onNext response 13: dataLength=262144 XXX server response ended: totalDataLength=3670016, numResponse=14 XXX position 3670016, length 3460300, requested 3670016, diff 3460300, rounded 3670016 XXX XceiverClientGrpc-1 -> dn-10c44ebd9c10_stream, send onNext request blockID { containerID: 1 localID: 115816896921600001 blockCommitSequenceId: 16 } offset: 3670016 length: 3670016 XXX server readBlock blockID { containerID: 1 localID: 115816896921600001 blockCommitSequenceId: 16 } offset: 3670016 length: 3670016 XXX server onNext response 0: dataLength=262144 XXX server onNext response 1: dataLength=262144 XXX server onNext response 2: dataLength=262144 XXX server onNext response 3: dataLength=262144 XXX server onNext response 4: dataLength=262144 XXX server onNext response 5: dataLength=262144 XXX server onNext response 6: dataLength=262144 XXX server onNext response 7: dataLength=262144 XXX server onNext response 8: dataLength=262144 XXX server onNext response 9: dataLength=262144 XXX server onNext response 10: dataLength=262144 XXX server onNext response 11: dataLength=262144 XXX server onNext response 12: dataLength=262144 XXX server onNext response 13: dataLength=262144 XXX server response ended: totalDataLength=3670016, numResponse=14 XXX position 7340032, length 1048576, requested 7340032, diff 1048576, rounded 1048576 XXX XceiverClientGrpc-1 -> dn-10c44ebd9c10_stream, send onNext request blockID { containerID: 1 localID: 115816896921600001 blockCommitSequenceId: 16 } offset: 7340032 length: 1048576 XXX server readBlock blockID { containerID: 1 localID: 115816896921600001 blockCommitSequenceId: 16 } offset: 7340032 length: 1048576 XXX server onNext response 0: dataLength=262144 XXX server onNext response 1: dataLength=262144 XXX server onNext response 2: dataLength=262144 XXX server onNext response 3: dataLength=262144 XXX server response ended: totalDataLength=1048576, numResponse=4 XXX initStreamRead conID: 2 locID: 115816896921600002 on datanode a85ea168-0eb0-46ed-9090-10c44ebd9c10(10.96.128.134/10.96.128.134) XXX position 0, length 2306866, requested 0, diff 2306866, rounded 2359296 XXX XceiverClientGrpc-1 -> dn-10c44ebd9c10_stream, send onNext request blockID { containerID: 2 localID: 115816896921600002 blockCommitSequenceId: 39 } offset: 0 length: 2359296 XXX server readBlock blockID { containerID: 2 localID: 115816896921600002 blockCommitSequenceId: 39 } offset: 0 length: 2359296 XXX server onNext response 0: dataLength=262144 XXX server onNext response 1: dataLength=262144 XXX server onNext response 2: dataLength=262144 XXX server onNext response 3: dataLength=262144 XXX server onNext response 4: dataLength=262144 XXX server onNext response 5: dataLength=262144 XXX server onNext response 6: dataLength=262144 XXX server onNext response 7: dataLength=262144 XXX server onNext response 8: dataLength=262144 XXX server response ended: totalDataLength=2359296, numResponse=9 XXX position 2359296, length 3512728, requested 2359296, diff 3512728, rounded 3670016 XXX XceiverClientGrpc-1 -> dn-10c44ebd9c10_stream, send onNext request blockID { containerID: 2 localID: 115816896921600002 blockCommitSequenceId: 39 } offset: 2359296 length: 3670016 XXX server readBlock blockID { containerID: 2 localID: 115816896921600002 blockCommitSequenceId: 39 } offset: 2359296 length: 3670016 XXX server onNext response 0: dataLength=262144 XXX server onNext response 1: dataLength=262144 XXX server onNext response 2: dataLength=262144 XXX server onNext response 3: dataLength=262144 XXX server onNext response 4: dataLength=262144 XXX server onNext response 5: dataLength=262144 XXX server onNext response 6: dataLength=262144 XXX server onNext response 7: dataLength=262144 XXX server onNext response 8: dataLength=262144 XXX server onNext response 9: dataLength=262144 XXX server onNext response 10: dataLength=262144 XXX server onNext response 11: dataLength=262144 XXX server onNext response 12: dataLength=262144 XXX server onNext response 13: dataLength=262144 XXX server response ended: totalDataLength=3670016, numResponse=14 XXX position 6029312, length 2359296, requested 6029312, diff 2359296, rounded 2359296 XXX XceiverClientGrpc-1 -> dn-10c44ebd9c10_stream, send onNext request blockID { containerID: 2 localID: 115816896921600002 blockCommitSequenceId: 39 } offset: 6029312 length: 2359296 XXX server readBlock blockID { containerID: 2 localID: 115816896921600002 blockCommitSequenceId: 39 } offset: 6029312 length: 2359296 XXX server onNext response 0: dataLength=262144 XXX server onNext response 1: dataLength=262144 XXX server onNext response 2: dataLength=262144 XXX server onNext response 3: dataLength=262144 XXX server onNext response 4: dataLength=262144 XXX server onNext response 5: dataLength=262144 XXX server onNext response 6: dataLength=262144 XXX server onNext response 7: dataLength=262144 XXX server onNext response 8: dataLength=262144 XXX server response ended: totalDataLength=2359296, numResponse=9 XXX initStreamRead conID: 3 locID: 115816896921600003 on datanode a85ea168-0eb0-46ed-9090-10c44ebd9c10(10.96.128.134/10.96.128.134) XXX position 0, length 1048574, requested 0, diff 1048574, rounded 1048576 XXX XceiverClientGrpc-1 -> dn-10c44ebd9c10_stream, send onNext request blockID { containerID: 3 localID: 115816896921600003 blockCommitSequenceId: 47 } offset: 0 length: 1048576 XXX server readBlock blockID { containerID: 3 localID: 115816896921600003 blockCommitSequenceId: 47 } offset: 0 length: 1048576 XXX server onNext response 0: dataLength=262144 XXX server onNext response 1: dataLength=262144 XXX server onNext response 2: dataLength=262144 XXX server onNext response 3: dataLength=262144 XXX server response ended: totalDataLength=1048576, numResponse=4 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
