srowen commented on code in PR #36545: URL: https://github.com/apache/spark/pull/36545#discussion_r875033677
########## python/pyspark/sql/session.py: ########## @@ -570,10 +570,20 @@ def _inferSchemaFromList( if not data: raise ValueError("can not infer schema from empty dataset") infer_dict_as_struct = self._jconf.inferDictAsStruct() + infer_array_from_first_element = self._jconf.legacyInferArrayTypeFromFirstElement() Review Comment: Wait, this seems like the wrong behavior. StringType should be inferred for this column, as it is now. I don't see a reason to break this? don't we want to just find the widest applicable type? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org