LantaoJin edited a comment on issue #24982: [SPARK-28181][CORE] Add a filter interface to KVStore to speed up the entities retrieve URL: https://github.com/apache/spark/pull/24982#issuecomment-506919132 > SPARK-28183 would be way more efficient if you instead scanned the tasks based on the status you want, applying the offset and limit, and sorted based on a different property after that (because of offset and limit, you wouldn't have a lot of elements to sort). Thanks for your reply @vanzin . IIUC, this still couldn’t address the target. The target is getting all tasks with particular `status` from all tasks. Even FAILED, KILLED will be getting in front with alphabetical order, or SUCCESS in front with reverse, it still needs additional filtering in upper application. SPARK-28183 wants to add a possibility in `taskList` API to get particular status. And this one could avoid filtering after pagination.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
