[
https://issues.apache.org/jira/browse/SPARK-16407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15493936#comment-15493936
]
holdenk commented on SPARK-16407:
---------------------------------
I think its important to keep in mind that these APIs are already exposed - you
can pass in a custom sink by full class name as a string. So while the current
Sink API does indeed expose a micro-batch API that seems somewhat orthogonal to
allowing people to specify Sources/Sinks programmatically - the Source/Sink
APIs/interface its self needs to be improved but once it still makes sense to
allow people to specify their own Sources/Sinks programmatically.
It might make sense to go back in and mark them as experimental - but if
anything that is a reason to keep improving them.
> Allow users to supply custom StreamSinkProviders
> ------------------------------------------------
>
> Key: SPARK-16407
> URL: https://issues.apache.org/jira/browse/SPARK-16407
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Reporter: holdenk
>
> The current DataStreamWriter allows users to specify a class name as format,
> however it could be easier for people to directly pass in a specific provider
> instance - e.g. for user equivalent of ForeachSink or other sink with
> non-string parameters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
