Hi all, I have a question about the future of custom data sinks in Structured Streaming. In particular, I want to know how continuous processing and the Datasource API V2 will impact them.
Right now, it is possible to have custom data sinks via the current Datasource API (V1) by implementing StreamSinkProvider and Sink traits. I am wondering if this approach is planned to be the right way or it is better to wait for the Datasource API V2 and the final implementation of continuous processing. As far as I understand SPARK-20928, there will be changes in the Source API in SS. But what about the Sink API? Is it safe to implement it now? Best regards, Anton
