gengliangwang commented on pull request #34092: URL: https://github.com/apache/spark/pull/34092#issuecomment-927024101
@zhouyejoe I use the benchmark code in the PR description, which is runnable under `AppStatusListenerSuite`. I tried 3000 and 5000 stages the result "after changes" is always slightly slow. When the number of stages reaches 1001, Spark will delete the oldest 100 stages UI data. I guess after changes reading 1001 stages from LevelDB is slow. Before changes Spark only reads 100 stages from LevelDB. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
