Good morning, I have started using Apache S4 for a university project and I wanted to ask you some question about its architecture, mainly to be sure to do the modifications I need in the right way and to see if there is a cleaner and simpler one.
It's my understanding that a cluster is a group of nodes and each node has the same application-code copy; this means that if I want to partition the ProcessingElements between nodes I need to group them in different clusters. So, mapping S4 elements into a "classical" Stream Processing naming (Nodes, Operators, Slides...), would be having one application (Operator) per cluster and configure the ProcessingElements as singleton (1 Slide per Node). About inter-cluster streaming: Is it possible to have broadcast stream between one cluster and all nodes of another cluster? Or should I re-implement RemoteSenders to do that? In this last case, is there a way to unbind Module mappring between interface and class used to resolve @inject? Is there a way to have that feature as per-stream configuration rather than all-stream cluster-wide? Is there any functional difference (or limitation) between "RemoteStreams" and "Streams" beyond the naming to recognize inter-cluster vs intra-cluster streaming? I saw there is an ongoing integration with helix project, which has a slightly different concept for partition since it can host more than one on the same node, but I couldn't find any example. Is there any work on it? Regards, Sergio Vavassori
