uncleGen commented on issue #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-616289728
Suppose there is a streaming job pipeline, and these streaming job comes from different end-users or department, if middle end-user upgrade their spark and use `CompactibleFileStreamLog V2` (as you said in default, we can read from version 1 and write to version 2 for smooth migration.), then downstream jobs will fail. Is there something I misunderstand？So, It is better to keep the version of `CompactibleFileStreamLog`, or leave a config to upgrade version of `CompactibleFileStreamLog` with default `false`? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
