zjf2012 commented on code in PR #495:
URL: https://github.com/apache/incubator-uniffle/pull/495#discussion_r1095286000
##########
client-spark/common/src/test/java/org/apache/spark/shuffle/writer/WriteBufferManagerTest.java:
##########
@@ -41,14 +40,20 @@
public class WriteBufferManagerTest {
+ static {
+ // trigger spark config package initialization before RssSparkConfig to
avoid key, spark.shuffle.compress,
+ // duplication in spark3
+ org.apache.spark.internal.config.package$.MODULE$.EXECUTOR_MEMORY();
Review Comment:
Uniffle is shuffle plugin which gets startup after Spark internal config. As
tested, there is no issue in real spark job. And the UT tests with spark2 and
spark3 also passed without above code.
The only issue is you get duplicate key issue when you run the specific UT
from GUI editor, like intellij. Thus, I added above code to avoid the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
