Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/1854#issuecomment-51596179
@YanTangZhai Yes, that's fine. The code in this case would just try to
remove 10 from `stageIdToData` several times, which is a bit redundant but
harmless.
My first comment was that, with this change, if I request 5 elements to be
trimmed, it will actually trim nothing. The first 5 elements of `stages` are
all 10, and none can be removed because 10 is active. I would more readily
expect that the behavior is to remove up to 5 elements that can be removed. 11
and 18 can be removed in this case. If you filter out active, *then* take, you
get this behavior.
The second potential issue is that, with this change, the method will
return [11,18] as `stages` after 5 values have been trimmed, even though 10 is
still active. Is that OK? If the comment above is addressed, then this return
value also has to change to reflect it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
