[
https://issues.apache.org/jira/browse/S4-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186615#comment-13186615
]
Matthieu Morel commented on S4-22:
----------------------------------
In order to integrate Leo's work and easily test the approach, I copied and
reorganized the code for the test applications in a new branch, and reused the
mechanism for automatic deployment from S4-24 so that it is now possible to run
the test automatically from a single place.
see
https://git-wip-us.apache.org/repos/asf?p=incubator-s4.git;a=log;h=refs/heads/S4-22
One can deploy and run Leo's "showtime" and "counter" apps simply by running
the "TestProducerConsumer" test in s4-core. Apps get built, packaged and
deployed automatically.
The approach for handling streams looks good, we still have to (see the TODO
from the commit log) :
* make the code generic in Server.java
* update the fluent API
* test on different logical clusters
> Adaptor
> -------
>
> Key: S4-22
> URL: https://issues.apache.org/jira/browse/S4-22
> Project: Apache S4
> Issue Type: Improvement
> Affects Versions: 0.5
> Reporter: Leo Neumeyer
> Assignee: Bruce Robbins
> Fix For: 0.5
>
> Attachments: s4-subclusters.pdf
>
>
> Need an adaptor for v0.5
> Idea I posted earlier:
> What do you think of this idea for a simple adaptor:
> - Adaptor extends App
> - Adaptor can send events but not receive (for now)
> - Adaptor is deployed as a regular App to the S4 cluster and as an
> Adaptor type in a host (separate from the S4 cluster).
> - Adaptor, unlike regular apps, can accept event data (in any format)
> directly, not via comm layer.
> - Input data is transformed into S4 events using a modular approach
> and by providing standard modules such as JSON.
> - Output events are exposed using EventSource and consumed by other
> apps without even knowing that they are Adaptors (only the App type is
> exposed in the cluster).
> - S4 events can be processed locally using PEs and Streams as usual.
> (We kind of need to get a local Sender for the local PEs and a
> standard cluster Sender for the EventSource object.)
> So why this approach?
> The GOOD:
> - Seems to be the least disruptive way to inject external events
> - Apps can easily consume the events in a modular way without any
> dependencies. Getting events from an adaptor or from another app is
> identical.
> - The adaptor would be packaged and deployed to the cluster as if it
> was an App (no incremental cost)
> - The adaptor can do preprocessing using the same programming model
> and can reuse PEs.
> The CHALLENGE:
> - We need to also deploy the Adaptor in a separate host. On the other
> hand, this is inevitable. At least we use the same approach instead of
> creating a different system.
> - The Adaptor will need to be integrated with ZK to get the physical
> addresses.
> - We need to deal with two senders.
> for later: two-way communication and adapter clusters.
> thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
