Re: [DISCUSS][SPARK SQL] SPARK-51710: Using Dataframe.dropDuplicates with an empty array as argument behaves "unexpectedly"

This seems like the correct behavior to me. Every value of the null set of columns will match between any pair of Rows. On Thu, May 8, 2025 at 11:37 AM David Kunzmann wrote: > Hello everyone, > > Following the creation of this PR > and the discussio

Re: setuptools 78.0.0 does not work with pyspark 3.x releases

Perhaps it is sufficient to wait for setuptools to revert the change: https://github.com/pypa/setuptools/pull/4911 On Mon, Mar 24, 2025 at 11:38 AM Holden Karau wrote: > I think given the lack of 4.0 release and the amount of folks using > PySpark this is enough to trigger a 3.5 branch release.