physinet commented on code in PR #36545:
URL: https://github.com/apache/spark/pull/36545#discussion_r875142772
##########
python/pyspark/sql/session.py:
##########
@@ -570,10 +570,20 @@ def _inferSchemaFromList(
if not data:
raise ValueError("can not infer schema from empty dataset")
infer_dict_as_struct = self._jconf.inferDictAsStruct()
+ infer_array_from_first_element =
self._jconf.legacyInferArrayTypeFromFirstElement()
Review Comment:
This seems to be marked as a TODO (at least on the python side).
https://github.com/apache/spark/blob/master/python/pyspark/sql/types.py#L1377
Again, I think this should be addressed separately from this PR, but I'm
happy to hold off on this PR if you think type-widening logic should be
implemented first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
