Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/654#issuecomment-52269494
@vanzin I think it would be useful to time stamp all events, either in this
patch or in the future. Is the reason why we have to add the timestamp field to
the event case classes themselves the following: we can't use the time when the
logger actually logs the event because it's technically inaccurate? E.g. the
block manager might have been added ages ago, and by the time we actually
process the event many milliseconds may have already passed. This isn't a huge
problem if the event queues are small, but we have seen them go up to O(100)
even after #1679. I guess this means we need to do the ugly thing of adding a
"timestamp" field to each of these case classes. Do you see a better
alternative?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
