+1 on the order. @manuzhang, how difficult to have GEARPUMP-116 as you wrote the Storm compatibility layer?
NOTE: GEARPUMP-116 is DIFFERENT from our Storm app compatibility. GEARPUMP-116 tries to provide Storm compatible Source/Sink to Gearpump native application instead of Storm app. Thanks Weihua 在 16/5/5 上午1:34，“Karol Brejna”<[email protected]> 写入: >We have a series of jira tickets regarding Gearpump sinks/sources: > >https://issues.apache.org/jira/browse/GEARPUMP-116 - Compatibility >layer/adapter for Apache Storm >https://issues.apache.org/jira/browse/GEARPUMP-115 - Create MQTT source/sink >https://issues.apache.org/jira/browse/GEARPUMP-106 - Gearpump Redis >Integration >https://issues.apache.org/jira/browse/GEARPUMP-105 - Provide non-persistent >Sink Task so that examples like word count can materialize Sum results >within the Client >https://issues.apache.org/jira/browse/GEARPUMP-100 - Source task that emits >messages per a schedule (interval or otherwise) should be provided >https://issues.apache.org/jira/browse/GEARPUMP-95 - Add parquet datasource >and datasink connectors >https://issues.apache.org/jira/browse/GEARPUMP-91 - Apache Cassandra >Integration > >We also had a ticket for 'Add a HDFS Sink with secutiry' ( >https://github.com/gearpump/gearpump/issues/1547) - I am not sure as for >the outcome of this one. > >Most of them consider the medium (MQTT, Redis, Casandra, ...). Other talk >about the source mechanics (scheduled/repetative source). > >I'd like to discuss the order in wich we plan implementation for them. > >In my opinion Redis an MQTT (GEARPUMP-106, GEARPUMP-115) seems most >important to have. >Redis is well known and widely used. MQTT is a de facto standard in IoT >communications. > >Then I would like to have HDFS sink (if we didn't merged this already). > >Non-persistent datasink could be very useful for examples/demo purposes. >(Imagine we have capped collection that the application can send messages >to, kind of application console. In the dashboard there could be a section >that presents lates 'console' messages. This way a user could "watch" the >application progress. Especially if he/she doesn't have access to the >backend - as it happens often in YARN mode. But this is a topic for >dedicated discussion, I think.) > >On the other hand, if we start working on GEARPUMP-116, we'd probably >quickly have Redis, JMS, AMQP sources (adapted from Storm) > >Please, let me know what do you think. > >Karol
