dongjoon-hyun commented on PR #52893: URL: https://github.com/apache/spark/pull/52893#issuecomment-3623249677
To @mridulm , `Netty` community decided the functionality like `addLast` is unstable for the production due to the race condition. AFAIK, it has several issues before in `Netty` layer although we didn't get the error reports in Apache community yet because this is fairly new feature of Apache Spark 4.0.0, [SPARK-43987](https://issues.apache.org/jira/browse/SPARK-43987) https://github.com/apache/spark/blob/6bfb83315e4596037790020a48d0747c308a317d/common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ShuffleTransportContext.java#L106 I believe it's good to inform the users this unstable API dependency before getting this spread out further. Maybe, we had better deprecated this at Apache Spark 4.1.0. WDTY, @mridulm ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
