[ https://issues.apache.org/jira/browse/SPARK-16407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15495006#comment-15495006 ]
Reynold Xin commented on SPARK-16407: ------------------------------------- The source/sink interface currently depends on RDDs, doesn't it? In that case, it has two issues: 1. The RDD interface is wide and difficult to stabilize across versions. This is similar to point 1 in https://issues.apache.org/jira/browse/SPARK-15689. Ideally, a source/sink implementation created for Spark 2.x should work in Spark 10.x, assuming the JVM is still around. 2. It is difficult to swap in/out a different execution engine. Actually I'm going to move the above into SPARK-10815 and just continue the discussion there. > Allow users to supply custom StreamSinkProviders > ------------------------------------------------ > > Key: SPARK-16407 > URL: https://issues.apache.org/jira/browse/SPARK-16407 > Project: Spark > Issue Type: Improvement > Components: Streaming > Reporter: holdenk > > The current DataStreamWriter allows users to specify a class name as format, > however it could be easier for people to directly pass in a specific provider > instance - e.g. for user equivalent of ForeachSink or other sink with > non-string parameters. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org