zero323 commented on a change in pull request #34731:
URL: https://github.com/apache/spark/pull/34731#discussion_r759141957
##########
File path: python/pyspark/profiler.py
##########
@@ -31,54 +36,51 @@ class ProfilerCollector(object):
the different stages.
"""
- def __init__(self, profiler_cls, dump_path=None):
- self.profiler_cls = profiler_cls
- self.profile_dump_path = dump_path
- self.profilers = []
+ def __init__(self, profiler_cls: Type["Profiler"], dump_path:
Optional[str] = None):
+ self.profiler_cls: Type[Profiler] = profiler_cls
+ self.profile_dump_path: Optional[str] = dump_path
+ self.profilers: List[Tuple[int, Profiler, bool]] = []
Review comment:
@ueshin
Well,it it seems it is a `list` of `lists`
https://github.com/apache/spark/blob/e3256b838b5bd5c817bc95ba9d996b878078ad35/python/pyspark/profiler.py#L56
and it is used as such. In particular this part modifies records in place:
https://github.com/apache/spark/blob/e3256b838b5bd5c817bc95ba9d996b878078ad35/python/pyspark/profiler.py#L70
So we'd need type ignores in both places, just to cover for incorrect
signature.
Having list of typed might be nice, but I am not sure if it acceptable
change (do we depend on current mutable semantics, does some 3rd party code?)
and it is definitely something out of scope for this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
