Re: [PR] [SPARK-48062][PYTHON][SS][TESTS] Add pyspark test for SimpleDataSourceStreamingReader [spark]

2024-05-21 Thread via GitHub
chaoqin-li1123 commented on code in PR #46306: URL: https://github.com/apache/spark/pull/46306#discussion_r1609292918 ## python/pyspark/sql/tests/test_python_streaming_datasource.py: ## @@ -150,50 +151,90 @@ def check_batch(df, batch_id): q.awaitTermination Review

Re: [PR] [SPARK-48062][PYTHON][SS][TESTS] Add pyspark test for SimpleDataSourceStreamingReader [spark]

2024-05-21 Thread via GitHub
HyukjinKwon commented on code in PR #46306: URL: https://github.com/apache/spark/pull/46306#discussion_r1609289839 ## python/pyspark/sql/tests/test_python_streaming_datasource.py: ## @@ -150,50 +151,90 @@ def check_batch(df, batch_id): q.awaitTermination Review

Re: [PR] [SPARK-48062][PYTHON][SS][TESTS] Add pyspark test for SimpleDataSourceStreamingReader [spark]

2024-05-21 Thread via GitHub
chaoqin-li1123 commented on code in PR #46306: URL: https://github.com/apache/spark/pull/46306#discussion_r1609283800 ## python/pyspark/sql/tests/test_python_streaming_datasource.py: ## @@ -150,50 +151,90 @@ def check_batch(df, batch_id): q.awaitTermination Review

Re: [PR] [SPARK-48062][PYTHON][SS][TESTS] Add pyspark test for SimpleDataSourceStreamingReader [spark]

2024-05-21 Thread via GitHub
HyukjinKwon commented on code in PR #46306: URL: https://github.com/apache/spark/pull/46306#discussion_r1609264628 ## python/pyspark/sql/tests/test_python_streaming_datasource.py: ## @@ -150,50 +151,90 @@ def check_batch(df, batch_id): q.awaitTermination Review

Re: [PR] [SPARK-48062][PYTHON][SS][TESTS] Add pyspark test for SimpleDataSourceStreamingReader [spark]

2024-05-01 Thread via GitHub
HyukjinKwon closed pull request #46306: [SPARK-48062][PYTHON][SS][TESTS] Add pyspark test for SimpleDataSourceStreamingReader URL: https://github.com/apache/spark/pull/46306 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [SPARK-48062][PYTHON][SS][TESTS] Add pyspark test for SimpleDataSourceStreamingReader [spark]

2024-05-01 Thread via GitHub
HyukjinKwon commented on PR #46306: URL: https://github.com/apache/spark/pull/46306#issuecomment-2089292078 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [SPARK-48062][PYTHON][SS][TESTS] Add pyspark test for SimpleDataSourceStreamingReader [spark]

2024-04-30 Thread via GitHub
chaoqin-li1123 commented on code in PR #46306: URL: https://github.com/apache/spark/pull/46306#discussion_r1585867083 ## python/pyspark/sql/tests/test_python_streaming_datasource.py: ## @@ -136,6 +137,33 @@ def streamWriter(self, schema, overwrite): return

Re: [PR] [SPARK-48062][PYTHON][SS][TESTS] Add pyspark test for SimpleDataSourceStreamingReader [spark]

2024-04-30 Thread via GitHub
chaoqin-li1123 commented on code in PR #46306: URL: https://github.com/apache/spark/pull/46306#discussion_r1585866969 ## python/pyspark/sql/tests/test_python_streaming_datasource.py: ## @@ -150,6 +178,20 @@ def check_batch(df, batch_id): q.awaitTermination