rickyma commented on PR #1519: URL: https://github.com/apache/incubator-uniffle/pull/1519#issuecomment-1938105129
> Maybe we can spit this pull requests to some smaller pull requests. It will be easier to review. Surely, it's good to summit a drat pull request at request to give reviewers an overall view. Smaller pull requests will be: 1. Upgrade Netty to latest stable version: [4.1.106.final](https://github.com/netty/netty/tags). 2. Upgrade GRPC to latest stable version: [1.61.1](https://github.com/grpc/grpc-java/releases/tag/v1.61.1) (including a lot of bug fixes). 3. No need to allocate a new direct `ByteBuf` in `Decoders.decodeShuffleBlockInfo()`, we can reuse it using `readSlice`. 4. Improve the robustness of methods `ShuffleDataResult.release()` and `ShuffleIndexResult.release()` to fix occasional IllegalReferenceCountException issues in extremely rare scenarios. 5. Add two metrics `allocated_direct_memory_size` and `pinned_direct_memory_size`. 6. 6.1 and 6.2 should be in one pull request, or else the UTs will fail. 6.1 When pre-allocating memory and flushing buffer, use the actual direct memory as the basis for judgment, instead of relying on the previous `usedMemory` metric. 6.2 Fix [#1008](https://github.com/apache/incubator-uniffle/pull/1008). It does not actually test `GRPC_NETTY` mode, because it uses `ShuffleServerGrpcClient` everywhere instead of `ShuffleServerGrpcNettyClient`. WDYT? @jerqi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
