HyukjinKwon commented on issue #23325: [SPARK-26376][SQL] Skip inputs without tokens by JSON datasource URL: https://github.com/apache/spark/pull/23325#issuecomment-447704171 Hm, the point of view was different. Yea, now I see they can't be consistent. To me, I didn't stay against in #22938 because the changes looked at least coherent because they look consistent at that time. > while for `from_json`, I do think treating empty token as invalid is better than returning null. I think this one is also kind of arguable as well .. Thing is, we currently return `null` for array types or map types. In that way, we should also produce empty array or empty map as @MaxGekk said at https://github.com/apache/spark/pull/22938#discussion_r231783277. If this is the change alone at https://github.com/apache/spark/pull/22938, I actually would have been hesitant about going ahead ..
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
