mridulm commented on PR #43743: URL: https://github.com/apache/spark/pull/43743#issuecomment-1810293165
In a specific ecosystem, this can be designed away from being an issue (have narrow waist submission layer above spark which ensures uniqueness, among other things) - and so no risk to debugging in that ecosystem. Features in Apache Spark are used by all users - where others might not have such guarantees or might not even be aware of these requirement, and my comment about debuggability was in that context: how can a Spark user leverage this experimental feature ? What should they be aware of ? Currently, if they use this feature and there is an issue, as you mentioned behavior would be undefined. At a minimum, we should document the expectations/risks. Thoughts ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
