Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19848
@squito Is there a requirement that it should be globally unique ? I am not
sure whether (some?) committers make this assumption : and the few I did take a
look at did not seem to care.
If there is such a requirement (a store which keeps track of commit
transactions by id ?), we can fallback to using uuid and propagating that to
executors via conf (as opposed to using rdd id).
The current situation of driver and executors having split brain was bad -
anything consistent (unique or this approach) would be preferable.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
