AngersZhuuuu commented on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-1006317362
> It is great to see that more metrics are being added to History server. Updating the listing.ldb is one of the major factors impacting the History server performance, there are several new metrics tracked for listing.ldb. But there is also another factor, which is generating each individual application levelDB when there is a user request. There are some metrics tracking the LevelDB generating time and loading time in this PR. Can you help add some more metrics, such as how many applications are being parsed to generated their own LevelDBs, and how many LevelDBs are currently stored in Disk, and what is the total size of them. Thanks. Update a change. And I want confirm with you that: 1. If we don't set `spark.history.store.path`, it will use memory to store app info and it's not belong to LevelDB metrics, right? 2. Only when support hybrid store disk backend, then we have in-memory LevelDBs, right? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
