[
https://issues.apache.org/jira/browse/SPARK-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Brewin updated SPARK-5267:
--------------------------------
Description:
The number of input stream protocols supported by Spark Streaming is quite
limited, which constrains the number of systems with which it can be integrated.
This proposal solves the problem by adding an optional module that integrates
Apache Camel, which supports many additional input protocols. Our tried and
tested implementation of this proposal is "spark-streaming-camel".
An Apache Camel service is run on a separate Thread, consuming each
http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/Message.html
and storing it into Spark's memory. The provider of the Message is specified
by any consuming component URI documented at
http://camel.apache.org/components.html, making all of these protocols
available to Spark Streaming.
Thoughts?
was:
The number of input stream protocols supported by Spark Streaming is quite
limited, which constrains the number of systems with which it can be integrated.
This proposal solves the problem by adding an optional module that integrates
Apache Camel, which support many more input protocols. Our tried and tested
implementation of this proposal is "spark-streaming-camel".
An Apache Camel service is run on a separate Thread, consuming each
http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/Message.html
and storing it into Spark's memory. The provider of the Message is specified
by any consuming component URI documented at
http://camel.apache.org/components.html, making all of these protocols
available to Spark Streaming.
Thoughts?
> Add a streaming module to ingest Apache Camel Messages from a configured
> endpoints
> ----------------------------------------------------------------------------------
>
> Key: SPARK-5267
> URL: https://issues.apache.org/jira/browse/SPARK-5267
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.2.0
> Reporter: Steve Brewin
> Labels: features
> Original Estimate: 120h
> Remaining Estimate: 120h
>
> The number of input stream protocols supported by Spark Streaming is quite
> limited, which constrains the number of systems with which it can be
> integrated.
> This proposal solves the problem by adding an optional module that integrates
> Apache Camel, which supports many additional input protocols. Our tried and
> tested implementation of this proposal is "spark-streaming-camel".
> An Apache Camel service is run on a separate Thread, consuming each
> http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/Message.html
> and storing it into Spark's memory. The provider of the Message is specified
> by any consuming component URI documented at
> http://camel.apache.org/components.html, making all of these protocols
> available to Spark Streaming.
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
