Maciej Szymkiewicz created SPARK-33002:
------------------------------------------
Summary: Post-port removal of non-API stubs
Key: SPARK-33002
URL: https://issues.apache.org/jira/browse/SPARK-33002
Project: Spark
Issue Type: Sub-task
Components: PySpark
Affects Versions: 3.1.0
Reporter: Maciej Szymkiewicz
To simplify initial port we merge all existing stubs.
However, some of these cover non-API components and are usually dynamically
annotated (generated with stubgen).
This includes modules like {{serializers}}, {{utils}}, {{shell}}, {{worker}},
etc.
These can be safely removed as:
- MyPy can infer types from the source, where stub is not present.
- No longer provide value, when corresponding modules are present in the same
directory structure.
- Annotations are here primarily to help end users, not Spark developers and
many of the annotations cannot be meaningfully refined.
It should also reduce overhead of maintaining annotations (especially when
places where we don't guarantee signature stability).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
