dgd_contributor created SPARK-36779:
---------------------------------------
Summary: Fix when list of data type tuples has len = 1
Key: SPARK-36779
URL: https://issues.apache.org/jira/browse/SPARK-36779
Project: Spark
Issue Type: Sub-task
Components: PySpark
Affects Versions: 3.3.0
Reporter: dgd_contributor
{code:python}
>>> ps.DataFrame[("a", int), [int]]
typing.Tuple[pyspark.pandas.typedef.typehints.IndexNameType, int]
>>> ps.DataFrame[("a", int), [("b", int)]]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/dgd/spark/python/pyspark/pandas/frame.py", line 11998, in
__class_getitem__
return create_tuple_for_frame_type(params)
File "/Users/dgd/spark/python/pyspark/pandas/typedef/typehints.py", line 685,
in create_tuple_for_frame_type
return Tuple[extract_types(params)]
File "/Users/dgd/spark/python/pyspark/pandas/typedef/typehints.py", line 755,
in extract_types
return (index_type,) + extract_types(data_types)
File "/Users/dgd/spark/python/pyspark/pandas/typedef/typehints.py", line 770,
in extract_types
raise TypeError(
TypeError: Type hints should be specified as one of:
- DataFrame[type, type, ...]
- DataFrame[name: type, name: type, ...]
- DataFrame[dtypes instance]
- DataFrame[zip(names, types)]
- DataFrame[index_type, [type, ...]]
- DataFrame[(index_name, index_type), [(name, type), ...]]
- DataFrame[dtype instance, dtypes instance]
- DataFrame[(index_name, index_type), zip(names, types)]
However, got [('b', <class 'int'>)].
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
