micheal-o commented on PR #51484: URL: https://github.com/apache/spark/pull/51484#issuecomment-3076444096
> Logging output does not count as user-facing output so you can remove that section, but you can update the PR description to add the example into the "what changes..." section instead. If you could include an actual examples of before and after log lines for both RocksDB, and HDFS that would be even better. Could you actually append the entire provider ID (including checkpointLoc, opId, partId, and storeName in addition to the queryRunId) as the logName? Sorry for the change, I did not realize that the HDFS logs included none of that. You can just verify this with inspecting unit test logs. Also, can you make sure that logs from RocksDB.scala and RocksDBFileManager.scala also have this change? > > @micheal-o -- a lot of HDFS logs manually include the StateStoreId, should we also do a quick audit of the HDFS logs to remove those so there is fewer duplicate info? Or is it fine to leave as is? @liviazhu @dylanwong250 yeah we can remove the duplicate in the logs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
