jingz-db commented on code in PR #49560:
URL: https://github.com/apache/spark/pull/49560#discussion_r1953428900
##########
python/pyspark/sql/pandas/serializers.py:
##########
@@ -1180,6 +1179,7 @@ def load_stream(self, stream):
this function works in overall.
"""
import pyarrow as pa
+ from pyspark.sql.streaming.stateful_processor_util import
TransformWithStateInPandasFuncMode
Review Comment:
We will hit circular dependency so it has to be descoped:
```
File "/spark/python/pyspark/sql/pandas/serializers.py", line 39, in <module>
from pyspark.sql.streaming.stateful_processor_util import
TransformWithStateInPandasFuncMode
File "/spark/python/pyspark/sql/streaming/stateful_processor_util.py",
line 21, in <module>
from pyspark.sql.streaming.stateful_processor_api_client import (
ImportError: cannot import name 'StatefulProcessorApiClient' from partially
initialized module 'pyspark.sql.streaming.stateful_processor_api_client' (most
likely due to a circular import)
(/spark/python/pyspark/sql/streaming/stateful_processor_api_client.py)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
