granthenke commented on issue #23354: [SPARK-26415][SQL] Mark StreamSinkProvider and StreamSourceProvider as stable URL: https://github.com/apache/spark/pull/23354#issuecomment-448767563 If I understand the progress of DatasourceV2 it sounds like there was a bit of a redesign and that DatasourceV2 wouldn't be available until Spark 3.0.0. If that is the case marking this as stable in the mean time should be minimal impact. Additionally using new interfaces for DatasourceV2 would allow for a much smoother transition for those who adopted structured streaming early. My context into DataSourceV2 comes from these locations: - Pull Request: https://github.com/apache/spark/pull/22547 - Umbrella Jira: [SPARK-25390](https://issues.apache.org/jira/browse/SPARK-25390) - API Design Doc: [here](https://docs.google.com/document/d/1uUmKCpWLdh9vHxP7AWJ9EgbwB_U6T3EJYNjhISGmiQg/edit#) Perhaps @cloud-fan could comment.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
