Github user squito commented on the issue: https://github.com/apache/spark/pull/19194 one more thought -- have you considered adding stage properties? I know that is more involved since its an API change, but I dunno how many times I've wanted something like that, eg. for labelling a stage in the UI, for partitioning hints, etc. And if you had it, then it would remove all the complications in this of multiple jobs & threads etc., and allow the property to actually be set on just one stage, as opposed to the entire job, as was originally requested.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org