eolivelli commented on code in PR #3592:
URL: https://github.com/apache/celeborn/pull/3592#discussion_r2757738112
##########
tests/spark-it/src/test/resources/log4j2-test.xml:
##########
@@ -21,7 +21,7 @@
<Console name="stdout" target="SYSTEM_OUT">
<PatternLayout pattern="%d{yy/MM/dd HH:mm:ss,SSS} %p [%t] %c{1}:
%m%n%ex"/>
<Filters>
- <ThresholdFilter level="ERROR"/>
+ <ThresholdFilter level="INFO"/>
Review Comment:
I changed it because by default you don't see any logs when running the tests
Reverted
##########
client/src/main/java/org/apache/celeborn/client/ShuffleClientImpl.java:
##########
@@ -231,6 +232,13 @@ public ShuffleClientImpl(String appUniqueId, CelebornConf
conf, UserIdentifier u
reviveManager = new ReviveManager(this, conf);
+ if (conf.hasS3Storage()) {
Review Comment:
we could do this check for all the external storage types
I did it only for S3 but we can loop over the available types and perform
the check
##########
worker/src/main/java/org/apache/celeborn/service/deploy/worker/memory/ChannelsLimiter.java:
##########
@@ -130,7 +130,7 @@ public void userEventTriggered(ChannelHandlerContext ctx,
Object evt) {
@Override
public void onPause(String moduleName) {
if (this.moduleName.equals(moduleName)) {
- logger.info("{} channels pause read.", this.moduleName);
+ logger.debug("{} channels pause read.", this.moduleName);
Review Comment:
reverted
I will move this to a separate patch, it is very noisy in production,
because it is logged lot of times, untile the channels are resumed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
