Spenserrrr commented on PR #57407: URL: https://github.com/apache/spark/pull/57407#issuecomment-5040869900
Yeah that is a good point. For the StreamingQueryListener class, it is used by both classic and connect, but with different methods: - Classics use the `_jlistener` property, in sql/streaming.query.py, line 683: `self._jsqm.addListener(listener._jlistener)` - Connect use the `_set_spark_session(...)`, in sql/connect/streaming/query.py and connect/streaming/worker/listener_worker.py So the class is shared, but the _set_spark_session + spark getter/setter mechanism is only used by Connect. Another way I think of is to type it as something like Union[SparkSession, ConnectSparkSession], but it still imports connect into a non-connect dir. Given this, do you think the original way of neglecting this with # type: ignore[name-defined] is better? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
