bjornjorgensen opened a new pull request, #40913: URL: https://github.com/apache/spark/pull/40913
### What changes were proposed in this pull request? Remove `null_counts` from info() ### Why are the changes needed? Pandas 2.0 _Removed deprecated null_counts argument in [DataFrame.info()](https://pandas.pydata.org/pandas-docs/version/2.0/reference/api/pandas.DataFrame.info.html#pandas.DataFrame.info). Use show_counts instead ([GH37999](https://github.com/pandas-dev/pandas/issues/37999))_ ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Tested local ### Before this PR `F05.info()` ``` TypeError Traceback (most recent call last) Cell In[12], line 1 ----> 1 F05.info() File /opt/spark/python/pyspark/pandas/frame.py:12167, in DataFrame.info(self, verbose, buf, max_cols, null_counts) 12163 count_func = self.count 12164 self.count = ( # type: ignore[assignment] 12165 lambda: count_func()._to_pandas() # type: ignore[assignment, misc, union-attr] 12166 ) > 12167 return pd.DataFrame.info( 12168 self, # type: ignore[arg-type] 12169 verbose=verbose, 12170 buf=buf, 12171 max_cols=max_cols, 12172 memory_usage=False, 12173 null_counts=null_counts, 12174 ) 12175 finally: 12176 del self._data TypeError: DataFrame.info() got an unexpected keyword argument 'null_counts' ``` ### With this PR `F05.info()` ``` <class 'pyspark.pandas.frame.DataFrame'> Int64Index: 5257 entries, 0 to 5256 Data columns (total 203 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 DOFFIN_APPENDIX:EXPRESSION_OF_INTEREST_URL 471 non-null object (...) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
