LuciferYang commented on PR #40933: URL: https://github.com/apache/spark/pull/40933#issuecomment-1521933550
<img width="1060" alt="image" src="https://user-images.githubusercontent.com/1475305/234313784-5f6e54f0-5e2b-4725-a1c1-8f62e7cb6d41.png";> https://github.com/FasterXML/jackson-core/blob/a2c0bdcfb9aae8fca555240e63e57c1d9e6f8079/src/main/java/com/fasterxml/jackson/core/StreamReadConstraints.java#L30-L51 It seems to have added 4 restrictions, but `MAX_BIGINT_SCALE_MAGNITUDE ` is not configurable. May be we should add corresponding configurable items to `org.apache.spark.sql.catalyst.json.JSONOptions` and inject them in `JSONOptions#buildJsonFactory` by `JsonFactoryBuilder#streamReadConstraints(StreamReadConstraints)`. This may handle most scenarios, but there are still some places in Spark that call `new ObjectMapper()`, these will using `StreamReadConstraints.defaults()`, but for them, using the default value may be OK. If there is a problem with what I said, please correct me @pjfanning , thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/1475305/234313784-5f6e54f0-5e2b-4725-a1c1-8f62e7cb6d41.png"](https://user-images.githubusercontent.com/1475305/234313784-5f6e54f0-5e2b-4725-a1c1-8f62e7cb6d41.png"){kind=link}