I realize saying I have a superficial knowledge of this topic is an understatement, but I don't understand why, if:
The matterhorn-conductor module, which provides our workflow operation > handlers, has all dependencies we need to communicate with (possibly remote) > Matterhorn services > then What we lose is the ability to listen for other kinds of events, such as > users adding annotations, or jobs failing. > Users adding annotations are actually calling the annotation service, which falls into the realm of "communication with Matterhorn services". Are you saying that workflows can generate events that call services, but not vice-versa (i.e. services cannot generate events which somehow trigger a workflow, a part of a workflow, or another service) ? Hope I'm not asking something too obvious. Best regards Rubén 2011/6/28 Josh Holtzman <[email protected]> > I just finished working on a branch [1] that uses publish/subscribe to > update series metadata in our local distribution channel, even if the series > contains recordings that have been previously published. > > Before merging this work with the trunk, I want to discuss what's in the > branch and what used to be in the branch but was subsequently removed. My > goal is to obtain a general consensus that this is an adequate approach to > cluster-wide asynchronous eventing. This is a long message, so if you are > not interested in Matterhorn's services design, feel free to move on. > > When I think of pub-sub messaging across a cluster, I think of JMS. So > that was how I originally approached the problem. In order to make use of > the simplified eventing model of OSGI's EventAdmin, I added a Sakai-inspired > JMS to EventAdmin bridge to distribute OSGI events to other nodes in the > cluster. This was: > > a) a bit heavy, as it introduced an embedded activemq broker [2] into each > JVM > b) required another piece of configuration, so the brokers could find each > other over the network > c) problematic, in that each node needed to decide whether to take action > on an event. Since each service might be deployed on multiple nodes, it > wasn't clear which node should act on the event. > > Fortunately, we already have cluster coordination via the service registry, > so there should be a way around JMS style messaging. We should be able to > respond to events centrally by using the endpoints that the service registry > tells us are online and available. > > The matterhorn-conductor module, which provides our workflow operation > handlers, has all dependencies we need to communicate with (possibly remote) > Matterhorn services. By putting event handlers here, we can respond to any > workflow or admin-tool related events by calling any of the services in the > cluster. As long as the events we need to respond to come from the > admin-related services (workflow, series, scheduling) this removes the need > for JMS altogether. > > What we lose is the ability to listen for other kinds of events, such as > users adding annotations, or jobs failing. We can deal with the latter via > workflow. I don't know of any use cases related to the former, but I'm > confident that they can be handled. > > Another annoyance inherent in this approach is the EventAdmin's tolerance > for latency. If an EventHandler takes too long when handling an event, it > will be blacklisted, and will not be called again. Long running operations > are the rule in Matterhorn, not the exception, so we can either: > > a) disable timeouts on the event admin. I'd rather not do this, since it's > a reasonable expectation that handlers do not block for long periods of > time. > b) wrap service calls in a Callable or Runnable, using a > java.util.concurrent.ExecutorService to execute the event handling logic. > See [3] for an example. This isn't hard, but it a little ugly. > c) EventHandlers can be JobProducers, storing the details of the event in > the database, to be dispatched to a service that can handle it. This is how > we handle other kinds of long running jobs, but this would require rest > endpoints to handle events. Seems like overkill to me. > > If you have comments on specific lines of code in the branch, feel free to > join the review at [4]. Otherwise, let's discuss here. > > Thanks, > Josh > > [1] https://opencast.jira.com/svn/MH/branches/MH-7938 > [2] > http://activemq.apache.org/how-do-i-embed-a-broker-inside-a-connection.html > [3] > http://opencast.jira.com/svn/MH/branches/MH-7938/modules/matterhorn-conductor/src/main/java/org/opencastproject/event/handler/SeriesUpdatedEventHandler.java > [4] http://opencast.jira.com/source/cru/CR-MH-324 > > _______________________________________________ > Matterhorn mailing list > [email protected] > http://lists.opencastproject.org/mailman/listinfo/matterhorn > > > To unsubscribe please email > [email protected] > _______________________________________________ >
_______________________________________________ Matterhorn mailing list [email protected] http://lists.opencastproject.org/mailman/listinfo/matterhorn To unsubscribe please email [email protected] _______________________________________________
