I am afraid not. The whole point of Spark Streaming is to make it easy to do complicated processing on streaming data while interoperating with core Spark, MLlib, SQL without the operational overheads of maintain 4 different systems. As a slight cost of achieving that unification, there maybe some overheads compared to specialized systems that are designed to one specific thing.
If you have to do something simple that could have been done using Flume, then the resources needed by the Spark Streaming program shouldn't be too high. Can you provide more details? TD On Thu, Apr 2, 2015 at 11:51 AM, Harut Martirosyan < harut.martiros...@gmail.com> wrote: > Hi guys. > > Is there a more lightweight way of stream processing with Spark? What we > want is a simpler way, preferably with no scheduling, which just streams > the data to destinations multiple. > > We extensively use Spark Core, SQL, Streaming, GraphX, so it's our main > tool and don't want to add new things to the stack like Storm or Flume, but > from other side, it really takes much more resources on same streaming than > our previous setup with Flume, especially if we have multiple destinations > (triggers multiple actions/scheduling) > > > -- > RGRDZ Harut >
