[ https://issues.apache.org/jira/browse/SPARK-36722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Takuya Ueshin resolved SPARK-36722. ----------------------------------- Fix Version/s: 3.2.0 Assignee: dgd_contributor Resolution: Fixed Issue resolved by pull request 33968 https://github.com/apache/spark/pull/33968 > Problems with update function in koalas - pyspark pandas. > --------------------------------------------------------- > > Key: SPARK-36722 > URL: https://issues.apache.org/jira/browse/SPARK-36722 > Project: Spark > Issue Type: Bug > Components: PySpark > Affects Versions: 3.2.0, 3.3.0 > Reporter: Bjørn Jørgensen > Assignee: dgd_contributor > Priority: Major > Fix For: 3.2.0 > > > Hi I am using "from pyspark import pandas as ps" in a master build yesterday. > I do have some columns that I need to join to one. > In pandas I use update. > 54 FD_OBJECT_SUPPLIES_SERVICES_OBJECT_SUPPLY_SERVICE_ADDITIONAL_INFORMATION > 23 non-null object > 55 FD_OBJECT_SUPPLIES_SERVICES_OBJECT_SUPPLY_SERVICE_ADDITIONAL_INFORMATION.P > 24348 non-null object > > > > pd1['FD_OBJECT_SUPPLIES_SERVICES_OBJECT_SUPPLY_SERVICE_ADDITIONAL_INFORMATION'].update(pd1['FD_OBJECT_SUPPLIES_SERVICES_OBJECT_SUPPLY_SERVICE_ADDITIONAL_INFORMATION.P']) > > --------------------------------------------------------------------------- > AssertionError Traceback (most recent call last) > /tmp/ipykernel_73/391781247.py in <module> > ----> 1 > pd1['FD_OBJECT_SUPPLIES_SERVICES_OBJECT_SUPPLY_SERVICE_ADDITIONAL_INFORMATION'].update(pd1['FD_OBJECT_SUPPLIES_SERVICES_OBJECT_SUPPLY_SERVICE_ADDITIONAL_INFORMATION.P']) > /opt/spark/python/pyspark/pandas/series.py in update(self, other) > 4549 raise TypeError("'other' must be a Series") > 4550 > -> 4551 combined = combine_frames(self._psdf, other._psdf, how="leftouter") > 4552 > 4553 this_scol = > combined["this"]._internal.spark_column_for(self._column_label) > /opt/spark/python/pyspark/pandas/utils.py in combine_frames(this, how, > preserve_order_column, *args) > 139 elif len(args) == 1 and isinstance(args[0], DataFrame): > 140 assert isinstance(args[0], DataFrame) > --> 141 assert not same_anchor( > 142 this, args[0] > 143 ), "We don't need to combine. `this` and `that` are same." > AssertionError: We don't need to combine. `this` and `that` are same. > pd1.info() > 54 FD_OBJECT_SUPPLIES_SERVICES_OBJECT_SUPPLY_SERVICE_ADDITIONAL_INFORMATION > 23 non-null object > 55 FD_OBJECT_SUPPLIES_SERVICES_OBJECT_SUPPLY_SERVICE_ADDITIONAL_INFORMATION.P > 24348 non-null object -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org