gatorsmile commented on a change in pull request #26965:
URL: https://github.com/apache/spark/pull/26965#discussion_r411528492
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala
##########
@@ -101,8 +101,9 @@ private[this] object JsonPathParser extends RegexParsers {
private[this] object SharedFactory {
val jsonFactory = new JsonFactoryBuilder()
- // Enabled for Hive compatibility
+ // The two options below enabled for Hive compatibility
.enable(JsonReadFeature.ALLOW_UNESCAPED_CONTROL_CHARS)
+ .enable(JsonReadFeature.ALLOW_SINGLE_QUOTES)
Review comment:
I am afraid that end users expect all our JSON functions have the same
behavior. That means, either all the functions support SINGLE QUOTE or none of
them supports it. Make the behavior configurable? For reading the external JSON
data sources, users can control it
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala#L141-L149
BTW, will this change cause the query result difference?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
