gaogaotiantian commented on code in PR #53931:
URL: https://github.com/apache/spark/pull/53931#discussion_r2719754918
##########
python/pyspark/sql/context.py:
##########
@@ -43,6 +43,7 @@
from pyspark.errors.exceptions.captured import install_exception_handler
from pyspark.sql.types import AtomicType, DataType, StructType
from pyspark.sql.streaming import StreamingQueryManager
+from pyspark.sql.streaming import _StreamingCheckpointManager
Review Comment:
If `_StreamingCheckpointManager` is meant to be imported by other modules
(even just inside pyspark), it should not be prefixed with `_`. You can just do
`StreamingCheckpointManager` like `StreamingQueryManager` (and this line could
be combined with the previous one).
I can understand that this is "private" for now, but it's a breaking change
to change it in the future. It's totally fine to just not list it in `__all__`.
##########
python/pyspark/sql/context.py:
##########
@@ -699,6 +700,18 @@ def streams(self) -> StreamingQueryManager:
return StreamingQueryManager(self._ssql_ctx.streams())
+ @property
+ def __streamingCheckpointManager(self) -> _StreamingCheckpointManager:
Review Comment:
I don't believe we have any name mangling methods/properties (starting with
`__`) in pyspark. Is there a specific reason to have this? You are actually
using the mangled name in tests to access this variable which is against the
purpose of name mangling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
