We've two (or more) jobs that write data into the same directory via a Dataframe.save method. We need to be able to figure out which job wrote which file. Maybe provide a 'prefix' to the file names. I was wondering if there's any 'option' that allows us to do this. Googling didn't come up with any solution so thought of asking the Spark experts on this mailing list.
Thanks in advance.