Maciej Szymkiewicz created SPARK-36617:
------------------------------------------
Summary: Inconsistencies in approxQuantile annotations
Key: SPARK-36617
URL: https://issues.apache.org/jira/browse/SPARK-36617
Project: Spark
Issue Type: Improvement
Components: PySpark, SQL
Affects Versions: 3.1.0, 3.2.0, 3.3.0
Reporter: Maciej Szymkiewicz
I've been reviewing PR in the legacy repo
(https://github.com/zero323/pyspark-stubs/pull/552) and it looks like we have
two problems with annotations for {{approxQuantile}}.
First of all {{DataFrame.approxQuantile}} should overload definition to match
input arguments ‒ if col is a sequence then result should be a list of lists:
{code:python}
@overload
def approxQuantile(
self,
col: str,
probabilities: Union[List[float], Tuple[float]],
relativeError: float
) -> List[float]: ...
@overload
def approxQuantile(
self,
col: Union[List[str], Tuple[str]],
probabilities: Union[List[float], Tuple[float]],
relativeError: float
) -> List[List[float]]: ...
{code}
Additionally {{DataFrameStatFunctions.approxQuantile}} should match whatever we
have in {{DataFrame}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
