SteNicholas commented on code in PR #3346:
URL: https://github.com/apache/celeborn/pull/3346#discussion_r2168301622
##########
docs/migration.md:
##########
@@ -100,9 +100,14 @@ license: |
- Since 0.6.0, the client respects the spark.celeborn.storage.availableTypes
configuration,
ensuring revived partition locations no longer default to memory storage.
In contrast, clients prior
to 0.6.0 default to memory storage for revived partitions. This means that
if memory storage is enabled in
- worker nodes, pre-0.6.0 clients may inadvertently utilize memory storage
for an application even when memory
+ worker nodes, clients prior to 0.6.0 may inadvertently utilize memory
storage for an application even when memory
storage is not enabled for that app.
+- Since 0.6.0, we have added a new sink
`org.apache.celeborn.common.metrics.sink.LoggerSink` to make sure that Celeborn
+ metrics will be scraped periodically. It's recommend to enable this sink
to make sure that worker's metrics data won't
+ be too large to cause worker OOM if you don't have a collector to scrape
metrics periodically. Don't forget to update
+ the `metrics.conf`.
Review Comment:
```suggestion
the `celeborn.metrics.loggerSink.scrape.interval` and
`celeborn.metrics.loggerSink.output.enabled`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
