I authored the initial Spark(DoS) plugin but it was never released due to priorities. The initial implementation allows full duplex data exchange between Drill and Spark. That is, we can use Drill to query your data lake and do further iterative ML on it via Spark or vice versa.
I will need to rework in order to bring it to a publishable quality. Let me know if you are interested in contributing. -Hanifi 2016-05-27 11:23 GMT-07:00 Zhenrui(Jerry) Zhang < [email protected]>: > Hi, > > Does anyone has any updates on the integration with Spark. The feature > mentioned in https://drill.apache.org/blog/2014/12/16/whats-coming-in- > 2015/ > and > http://www.slideshare.net/SparkSummit/adding-complex- > data-to-spark-stackneeraja-rentachintala > ? Also there is an issue opened in JIRA( > https://issues.apache.org/jira/browse/DRILL-3184) for this. > > Thanks, > Jerry >
