[
https://issues.apache.org/jira/browse/SPARK-38601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prakhar Sandhu updated SPARK-38601:
-----------------------------------
Description:
I am working on replacing Pandas library to pyspark.pandas Library in my python
repo in VS Code. But pyspark.pandas module does not seem to have DateOffset()
module similar to what pandas has.
I tried this :
{code:java}
import pyspark.pandas as pd
clean_df[self._config.general["info_col"]] = (clean_df["cycle_dt"] -
pd.DateOffset(months=cycle_info_gap)
).fillna(clean_df[self._config.general["time_col"]]) {code}
It results in the below error :
{code:java}
Un unanticipated error occurred: module 'pyspark.pandas' has no attribute
'DateOffset'{code}
Is there any alternative for this in pyspark.pandas?
was:
I am working on replacing Pandas library to Koalas Library in my python repo in
VS Code. But Koalas module does not seem to have DateOffset() module similar to
what pandas has.
I tried this :
{code:java}
import pyspark.pandas as pd
pdf["date_col_2"] = pdf["date_col_1"] - pd.DateOffset(months=cycle_info_gap)
{code}
It results in the below error :
{code:java}
AttributeError: module 'databricks.koalas' has no attribute 'DateOffset' {code}
Is there any alternative for this in Koalas?
> AttributeError: module 'pyspark.pandas' has no attribute 'DateOffset'
> ---------------------------------------------------------------------
>
> Key: SPARK-38601
> URL: https://issues.apache.org/jira/browse/SPARK-38601
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 3.2.1
> Reporter: Prakhar Sandhu
> Priority: Major
>
> I am working on replacing Pandas library to pyspark.pandas Library in my
> python repo in VS Code. But pyspark.pandas module does not seem to have
> DateOffset() module similar to what pandas has.
> I tried this :
> {code:java}
> import pyspark.pandas as pd
> clean_df[self._config.general["info_col"]] = (clean_df["cycle_dt"] -
> pd.DateOffset(months=cycle_info_gap)
> ).fillna(clean_df[self._config.general["time_col"]]) {code}
> It results in the below error :
> {code:java}
> Un unanticipated error occurred: module 'pyspark.pandas' has no attribute
> 'DateOffset'{code}
> Is there any alternative for this in pyspark.pandas?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
