cloud-fan commented on PR #49163: URL: https://github.com/apache/spark/pull/49163#issuecomment-3184755668
Hi @steven-aerts and @dongjoon-hyun , while I agree that it's more ideal to pass a `SparkConf` parameter to `JsonProtocol`, I feel it's a bit too late to make this change now as `object JsonProtocol` has been widely used already. Moreover, we can have a surgical fix to mitigate the Jackson regression, by calling `setStreamReadConstraints`. The invasive changes here are mostly for a new feature: making the json max size configurable. It's arguable how valuable it is to make it configurable, but users can still configure it via the conf file, and `SparkConf` instaintation will load it. My proposal here is to revert this change and make a surgical fix for both master, 4.0 and 3.5. What do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
