HeartSaVioR opened a new pull request #26032: [SPARK-29361][SQL] Enable DataFrame with streaming source support on DSv1 URL: https://github.com/apache/spark/pull/26032 ### What changes were proposed in this pull request? This patch provides a new API on creating streaming DataFrame from RDD, which helps to create custom streaming data source in DSv1. This will be the one of interim solutions until new DSv2 has been stabilized and widely adopted. JavaRDD is not addressed since `Source` and its relevant interfaces are not Java-friendly. ### Why are the changes needed? The new API this patch proposes would help developers to choose DSv1 for streaming source: for now they're stuck between choosing old DSv2 (and have to work for Spark 3.0 again) and new DSv2 (wait for new API to be available - Spark 3.0 - and overcome huge learning curve). ### Does this PR introduce any user-facing change? No. ### How was this patch tested? New UT.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
