Github user ssaavedra commented on the issue:
https://github.com/apache/spark/pull/22392
@skonto this seems to be related to how the Checkpointing code works in
general. Variables that are not explicitly whitelisted as per the list I'm
changing in this proposal seems to be reloaded from the checklpointed data. I'm
not completely sure that it should be so hard to override the variables on job
restart, and I vaguely remember that there was an issue at JIRA for changing
more broadly how the Checkpointing code worked, but I can't find it right now.
@liyinan926 that was fast :)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
