HeartSaVioR commented on issue #27557: [SPARK-30804][SS] Measure and log elapsed time for "compact" operation in CompactibleFileStreamLog URL: https://github.com/apache/spark/pull/27557#issuecomment-589409973 While I still think the latency information is important aspect, I agree the information about size of logs (length, memory usage) is not a thing to be interested unless there's an issue on latency. I've adjusted the severity of log message as WARN and DEBUG, and differentiate the information to provide according to the severity.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
