Hi Erik I totally agree. I think a Scheduler which just sends JMS messages when its time to execute tasks is a great way to solve the problem. Then fault tolerance and load balancing can be achieved by having server farms to execute the tasks. (e.g. you could use Messenger with a cluster of Tomcat servers to execute the tasks which would take care of load balancing & fault tolerance for you).
The JMS provider will handle load balancing, acknowledgement, recovery, ensuring messages are delivered only once etc. If need be database-centric tasks could be implemented with XA 2 phase commit with the JMS provider. If 2 phase commit is required then it could be used by the scheduler as well, updating the schedule database and sending the JMS message in the same transaction. I guess for systems with less transactional requirements then HTTP, servlets or email could be used as alternatives to JMS. James ----- Original Message ----- From: "Erik Worth" <[EMAIL PROTECTED]> To: "Jakarta Commons Developers List" <[EMAIL PROTECTED]> Sent: Friday, November 30, 2001 7:26 PM Subject: RE: Workflow & Turbine design questions > Hi Gang, > The tricky thing with building a scheduler suitable for workflow is the need > to schedule tasks in the future (i.e. do this task 10 days from now at 2:35 > PM). When scheduling future tasks, one needs to persist the schedule. This > type of service is also difficult to support in a clustered or fault > tolerant environment because it is relatively difficult to keep multiple > instances of the same scheduler in sync so that when the main one (you only > want one of these scheduling tasks at a time or you get duplicate work > scheduled) goes down, the other one automatically picks up where the other > left off. > > I am not trying to dampen anyone's enthusiasm with regards to building one > of these for the open source community, but it would be nice if such an > endeavor was taken up, it would address these types of issues. I built a > CORBA-based implementation of such a scheduler a number of years back that > handled the persistence case, but when we considered the level of effort for > a fault tolerant version, we decided to live with the single point of > failure. > > One aspect of the approach I took worked out rather well. When scheduling a > task, I passed a command object along with the schedule time and recipient > (and periodicity if any). If this were implemented using JMS as the > transport, one could pass a serializable command object (an object > implementing an interface that specifies an execute method), a JMS > destination (as the recipient), the schedule time, and the periodicity using > an JMS ObjectMessage payload. The scheduler (in this case a stand-alone > daemon type application) keeps a time ordered list of tasks and when a > task's time comes due, sends the command object to the designated > destination and removes the task if is not a periodic task. If Message > Driven Beans are listening on the destination queues, the threading is > already taken care of and the listener simply invokes the execute method on > the command object (simple JMS listeners can also execute tasks, but they > need to manage their own thread pool). The execute method is implemented to > perform a task or invoke on something that performs the task. > > Using this model, a client is able to schedule some other service to perform > a task at a specific time in the future (or periodically starting at some > time in the future). It decouples the schedule maintenance from the task > execution, and it has a hope of being made fault tolerant. > > Food for thought :-) > -- Erik. > > > -----Original Message----- > From: James Strachan [mailto:[EMAIL PROTECTED]] > Sent: Friday, November 30, 2001 2:42 AM > To: Jakarta Commons Developers List > Subject: Re: Workflow & Turbine design questions > > > +1. > > I've wanted a scheduler component for some time - up until now everything > I've seen has been integrated into larger frameworks or has been payware. > Maybe a combination of a general ThreadPool that asynchronous tasks can be > dispatched to (probably using the Runnable interface to represent the tasks) > with a Scheduler capable of dispatching tasks at fixed or repeated > intervals. > > The Timer and TimerTask in JDK1.3 are a pretty good start for a Scheduler if > used with ThreadPools for the heavy-lifting. Though Timer/TimerTask is only > available on 1.3. Its a real shame we're not allowed to redistribute classes > from the newer JDKs so we can use them on older JDKs like 1.2. (Ditto for > other stuff in 1.4). > > James > ----- Original Message ----- > From: "Craig R. McClanahan" <[EMAIL PROTECTED]> > To: "Jakarta Commons Developers List" <[EMAIL PROTECTED]>; > <[EMAIL PROTECTED]> > Sent: Friday, November 30, 2001 2:32 AM > Subject: RE: Workflow & Turbine design questions > > > > One way to approach these issues (and would even fit into the Commons > > charter :-) would be to conceive of a Commons package to do background job > > scheduling and execution (what legacy OSs called "batch job queues") > > within a JVM. Executing a particular workflow script would be only one of > > the interesting things you could do -- ideally the definition of a "job" > > would be something like "call method X on object Y" so it could be fully > > generalized. > > > > The "jobs" package could be configured with limits on how many background > > threads it could run, support ways to schedule jobs repeatedly, and all > > the other stuff you can do with "cron" type utilities, if we wanted. It > > could be embedded in any sort of long-running JVM (such as in a webapp on > > a servlet container), as well as stand-alone background jobs. > > > > This is a bigger task than just something that can run Workflow scripts in > > background threads (which isn't terribly difficult to hard code in your > > application by starting a new thread and executing the Activity there), > > but it would be a generally useful package. What do you think? > > > > Craig > > > > > > On Thu, 29 Nov 2001, Brett Gullan wrote: > > > > > Date: Thu, 29 Nov 2001 09:24:55 +0800 > > > From: Brett Gullan <[EMAIL PROTECTED]> > > > Reply-To: Jakarta Commons Developers List > <[EMAIL PROTECTED]>, > > > [EMAIL PROTECTED] > > > To: 'Jakarta Commons Developers List' <[EMAIL PROTECTED]> > > > Subject: RE: Workflow & Turbine design questions > > > > > > Almost... > > > > > > > #1. User starts workflow process by performing action. > > > Let's say User A starts workflow process, by for instance, copying a > > > file from his/her desktop to a directory on a local/remote server (this > > > can/will be achieved by a variety of methods/protocols). > > > > > > > #2. Workflow performs tasks that may take time. > > > Yes > > > > > > > #3. User should not be held up waiting for a response from the server > > > by > > > background workflow tasks. > > > Yes. However, User A may never even use the Web UI. User B on the other > > > hand may be watching the queue/progress monitor for regular updates. > > > > > > The web UI is intended to be a monitoring and configuration tool for the > > > underlying workflow engine. For the most part, general users would > > > submit files (by desktop drag-n-drop, FTP, web-form, etc...) then use > > > the web UI to monitor progress or change the queue priority (if > > > authorised). > > > Administrative users would have access to web forms that configure the > > > workflow processing activity steps and the 'hot' folders that feed the > > > processing queue. > > > The application is intended to support deployment on a 'headless' server > > > where all interaction is via a web UI. > > > I hope that makes more sense? > > > Regards, > > > Brett > > > -- > > > Brett Gullan > > > [EMAIL PROTECTED] > > > > > > > > > > > > -- > > > To unsubscribe, e-mail: > <mailto:[EMAIL PROTECTED]> > > > For additional commands, e-mail: > <mailto:[EMAIL PROTECTED]> > > > > > > > > > > > > -- > > To unsubscribe, e-mail: > <mailto:[EMAIL PROTECTED]> > > For additional commands, e-mail: > <mailto:[EMAIL PROTECTED]> > > > _________________________________________________________ > Do You Yahoo!? > Get your free @yahoo.com address at http://mail.yahoo.com > > > -- > To unsubscribe, e-mail: > <mailto:[EMAIL PROTECTED]> > For additional commands, e-mail: > <mailto:[EMAIL PROTECTED]> > > -- > To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> > For additional commands, e-mail: <mailto:[EMAIL PROTECTED]> _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
