nirvanjhurreepriv commented on PR #2524: URL: https://github.com/apache/systemds/pull/2524#issuecomment-4936825394
Hi David @ywcb00 , Thank you for the continued feedback! I have addressed the comments in the latest commits. For configurable compression, I replaced the hardcoded TOPK / 0.01 config with three new static fields in DMLScript: FEDERATED_COMPRESSION_TYPE, FEDERATED_COMPRESSION_SPARSITY, and FEDERATED_COMPRESSION_BITS. FederationMap now reads from these fields, making the compression technique and parameters fully configurable at runtime. For network transfer, FederationMap.broadcast() now sends the CompressedMatrix object directly over the network instead of decompressing it on the sender side. FederatedWorkerHandler.putVariable() now detects incoming CompressedMatrix objects and decompresses them on the worker side using a new CompressionFactory.createForDecompression() method. For test coverage, I added FederatedMatrixCompressionTest; a full federated integration test that exercises the complete pipeline (compress, transfer, decompress) for both TopK and Probabilistic Quantization. All 14 existing unit tests still pass. All changes compile cleanly and pass checkstyle locally. Thank you for approving the workflow runs :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
