cloud-fan commented on issue #23325: [SPARK-26376][SQL] Skip inputs without tokens by JSON datasource URL: https://github.com/apache/spark/pull/23325#issuecomment-447689895 These 2 cases can't be consistent. `from_json` can't skip input. If it returns null, it's still not consistent with json data source which returns `Nil`. It's arguable if json data source should follow `from_json` and treat empty token as invalid input. But I think it's safer to not introduce behavior change if we are not sure one way is better than another. while for `from_json`, I do think treating empty token as invalid is better than returning null.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
