HyukjinKwon commented on a change in pull request #29510:
URL: https://github.com/apache/spark/pull/29510#discussion_r475023959
##########
File path: python/pyspark/sql/session.py
##########
@@ -359,18 +359,14 @@ def range(self, start, end=None, step=1,
numPartitions=None):
def _inferSchemaFromList(self, data, names=None):
"""
- Infer schema from list of Row or tuple.
+ Infer schema from list of Row, dict, or tuple.
- :param data: list of Row or tuple
+ :param data: list of Row, dict, or tuple
:param names: list of column names
:return: :class:`pyspark.sql.types.StructType`
"""
if not data:
raise ValueError("can not infer schema from empty dataset")
- first = data[0]
- if type(first) is dict:
- warnings.warn("inferring schema from dict is deprecated,"
- "please use pyspark.sql.Row instead")
schema = reduce(_merge_type, (_infer_schema(row, names) for row in
data))
if _has_nulltype(schema):
raise ValueError("Some of types cannot be determined after
inferring")
Review comment:
Let's also remove the one below:
```python
if type(first) is dict:
warnings.warn("Using RDD of dict to inferSchema is deprecated. "
"Use pyspark.sql.Row instead")
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
