Takuya Ueshin created SPARK-38882:
-------------------------------------
Summary: The usage logger attachment logic should handle static
methods properly.
Key: SPARK-38882
URL: https://issues.apache.org/jira/browse/SPARK-38882
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 3.2.1, 3.3.0
Reporter: Takuya Ueshin
The usage logger attachment logic has an issue when handling static methods.
For example,
{code}
$ PYSPARK_PANDAS_USAGE_LOGGER=pyspark.pandas.usage_logging.usage_logger
./bin/pyspark
{code}
{code:python}
>>> import pyspark.pandas as ps
>>> psdf = ps.DataFrame({"a": [1,2,3], "b": [4,5,6]})
>>> psdf.from_records([(1, 2), (3, 4)])
A function `DataFrame.from_records(data, index, exclude, columns, coerce_float,
nrows)` was failed after 2007.430 ms: 0
Traceback (most recent call last):
...
{code}
without usage logger:
{code:python}
>>> import pyspark.pandas as ps
>>> psdf = ps.DataFrame({"a": [1,2,3], "b": [4,5,6]})
>>> psdf.from_records([(1, 2), (3, 4)])
0 1
0 1 2
1 3 4
{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
