[
https://issues.apache.org/jira/browse/SPARK-36930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dch nguyen updated SPARK-36930:
-------------------------------
Description:
when MultiIndex.dtypes is supported, we can use:
{code:java}
>>> idx = pd.MultiIndex.from_arrays([[0, 1, 2, 3, 4, 5, 6, 7, 8], [1, 2, 3, 4,
>>> 5, 6, 7, 8, 9]], names=("zero", "one"))
>>> pdf = pd.DataFrame(
... {"a": [1, 2, 3, 4, 5, 6, 7, 8, 9], "b": [4, 5, 6, 3, 2, 1, 0, 0, 0]},
... index=idx,
... )
>>> psdf = ps.from_pandas(pdf)
>>> ps.DataFrame[psdf.ipsdf.dtypes]
psdf.iat psdf.idxmin( psdf.index psdf.insert(
psdf.isna( psdf.items( psdf.iterrows(
psdf.idxmax( psdf.iloc psdf.info( psdf.isin(
psdf.isnull( psdf.iteritems( psdf.itertuples(
>>> ps.DataFrame[psdf.index.dtypes, psdf.dtypes]
typing.Tuple[pyspark.pandas.typedef.typehints.IndexNameType,
pyspark.pandas.typedef.typehints.IndexNameType,
pyspark.pandas.typedef.typehints.NameType,
pyspark.pandas.typedef.typehints.NameType]
{code}
> Support ps.MultiIndex.dtypes
> ----------------------------
>
> Key: SPARK-36930
> URL: https://issues.apache.org/jira/browse/SPARK-36930
> Project: Spark
> Issue Type: Sub-task
> Components: PySpark
> Affects Versions: 3.3.0
> Reporter: dch nguyen
> Priority: Major
>
> when MultiIndex.dtypes is supported, we can use:
> {code:java}
> >>> idx = pd.MultiIndex.from_arrays([[0, 1, 2, 3, 4, 5, 6, 7, 8], [1, 2, 3,
> >>> 4, 5, 6, 7, 8, 9]], names=("zero", "one"))
> >>> pdf = pd.DataFrame(
> ... {"a": [1, 2, 3, 4, 5, 6, 7, 8, 9], "b": [4, 5, 6, 3, 2, 1, 0, 0, 0]},
> ... index=idx,
> ... )
> >>> psdf = ps.from_pandas(pdf)
> >>> ps.DataFrame[psdf.ipsdf.dtypes]
> psdf.iat psdf.idxmin( psdf.index psdf.insert(
> psdf.isna( psdf.items( psdf.iterrows(
> psdf.idxmax( psdf.iloc psdf.info( psdf.isin(
> psdf.isnull( psdf.iteritems( psdf.itertuples(
> >>> ps.DataFrame[psdf.index.dtypes, psdf.dtypes]
> typing.Tuple[pyspark.pandas.typedef.typehints.IndexNameType,
> pyspark.pandas.typedef.typehints.IndexNameType,
> pyspark.pandas.typedef.typehints.NameType,
> pyspark.pandas.typedef.typehints.NameType]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
