ywcb00 opened a new pull request, #1601: URL: https://github.com/apache/systemds/pull/1601
Hi, this PR replaces the ObjectEncoders for encoding federated communication messages (i.e., FederatedRequest and FederatedResponse) with a respective FederatedRequestEncoder and FederatedResponseEncoder. These two new classes extent from ObjectEncoder and override the allocateBuffer method in order to allocate the buffer with an initial capacity corresponding to the FederatedRequest/FederatedResponse instead of the default value of 256. I've tried estimating the serialized size of the requests and responses to allocate the exact size needed, but haven't found a good solution for it because of the many cases to consider. Hence, the initial capacity in this PR only results from taking into consideration a static offset (FederatedRequest: 512, FederatedResponse: 312), the size of the CacheBlock parameters, and the length of the lineage trace (only for FederatedRequests). In my experiments with a matrix of dimensions 600.000x200, it reduces the encoding time from 50 seconds to 1.9 seconds. :tada: Thanks for review :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
