Re: Workflow Support Proposal
Craig R. McClanahan wrote: Thanks for your comments and thoughts. Embedded below are some responses about how the current thinking (of course, always subject to change) addresses some of them -- if they do. You might want to take a look at some initial code that has been checked in to the jakarta-commons-sandbox repository, if you haven't yet. The easiest way to get this is to use anonymous CVS (you'll need both jakarta-commons-sandbox and jakarta-commons to pick up dependencies), or I can make snapshots available. Will do - thanks. On Wed, 15 Aug 2001, John Sigler wrote: Craig, Your Workflow Proposal seems like a great start. Several of us have played around with some workflow concepts on our current project but it would be great to have something integrated into Struts. We have a few concepts that weren't mentioned that we think are worth considering: - bookmarkable workflow steps: This means being able to store all context related to a step in such a fashion that the user can go back to that same the same state in the workflow at a later time. Bookmarkability should be optional (some steps may not make sense to be bookmarked) and so it should be able to be indicated as such when being defined. Hmm, technically every time that a Step tells its associated Context to suspend (which causes the workflow engine to return control to the calling app), that sort of acts like a bookmark -- in the sense that, the next time this Context is executed, it picks up where it left off. In a web application, that happens each time you display a page and wait for the response. So, it seems we would need to provide a way for the user to select a different in-progress activity (or start a new one), and then come back to this Context later. Does that sound right? Yes. We also allow sending bookmarked workflows to other users. In our case, the only extra issues that we encounter are permissibility related (i.e., the user may not be allowed to view the workflow I've sent him) but thought I'd mention this in case you see other issues it introduces - a workflow history: keep a state of where the user is and/or has been. Allow this state to be displayed as breadcrumbs, a history list, or with another visual or navigational representation. A tag to render this information seems useful as well. Supporting a breadcrumb trail makes a lot of sense, at least in some applications (i.e. where you can legitimately branch to any of the previuos spots). There isn't any direct support for this in the current code, although Steps can have an identifier (and thus be the destination of a branch). Managing a breadcrumb trail in some use cases is going to get interesting ... especially if the user can loop as well as nest. You're right. We've had a variety of solutions for displaying breadcrumbs in complex workflows (looping, etc) and/or across workflows but none have been satisfactory for all situations. I still think it's useful if the framework has a centralized location which stores the workflow/steps which have been traversed so users can display if needed, etc. - workflows, especially scripted workflows, will often be customized. The workflow configuration files should support the ability to override a workflow definition - one possibility would be to use a mechanism similar to a Java classpath. The primary concern here is updating installations with new versions of out-of-the-box workflows without overwriting any customized versions of those workflows. This is an important point, and was raised in the jakarta-commons discussion as well. IMHO, this needs to be addressed at several levels. * In the core APIs, the key component abstractions are Java interfaces (Activity, Step, Context, and so on) that can accomodate multiple implementations ... the usual object-oriented design approach, but lets customized workflow engines create their own implementations. * Configuration of the static structure of a workflow Activity is separate from execution. An XML format (as illustrated in the proposal) is one very nice way to do it, but you can build up your own structures as well. * The point you are really making, though, has to do with the registry of scripted Activities. There should be a hierarchical registry so that the Activity I get when I ask for the Enter Purchase Orders activity can be customized for me, customized for my department, or the standard one. Is that what you had in mind? Yes exactly. (It would be nice to have similar support for the Actions and other current Struts elements defined in struts-config as well.) - when a workflow is started, it should be associated with a context that is specific to that *instance* of the workflow. This supports the ability for a single user to execute the
RE: Workflow Support Proposal
Yes. This is exactly the reason I proposed the struts:page addition to the schema a few days ago. If you are persisting the state across a workflow, you have to have some method of retrieving the state. My thought was to identify each FormBean (the data that creates the state of the workflow) by a unique ID, set as an attribute on the struts:page tag. The workflow could be similarly identified, allowing the management of multiple simultaneous workflows per user. If the workflow objects were stored at the application scope, organised by user, then we would also have a mechanism for transferring a workflow from one user to another. It seems to me that this addition to the schema would answer a lot of people's needs. Thoughts? Is something like this usable? I assume we'd probably need to add an additional tag/attribute that specified the persistence type, and perhaps the scope at which the workflow state objects are held. Cheers! -dan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]. org]On Behalf Of Colin Sharples Sent: Wednesday, August 15, 2001 5:16 AM To: [EMAIL PROTECTED] Subject: RE: Workflow Support Proposal Craig, First of all - great job on the workflow proposal! I had begun writing a workflow system based on basic Struts Actions with a bit of state management thrown in, but what you're suggesting is much closer to what I need. I have a couple of requirements which I'm sure are not unique to my application. The processes I'm modelling take a long time - the overall process takes a few weeks to finish, and individual steps may take a few minutes, a few hours or even a few days. This has a couple of implications: 1) there isn't going to be a user waiting for an interactive response; the user triggers the process by sending in some data, and will receive a notification when it's finally finished. In between, it's up to the system to ensure that each step gets executed. 2) Most of the steps will therefore be asynchronous (some of them involve getting approval, sometimes with delays of days). If the steps are asynchronous, this will require some state management, so that the workflow engine knows where to start off again. I had been thinking along the lines that an asynchronous step would actually consist of two steps, one which will send a message to initiate the process, and another to assimilate the results of the step (triggered by some event). Once the message initiating the step has been sent, the activity will be suspended until the end-of-step-event arrives. The workflow engine will invoke the activity, starting up at the point it left off (that's where the state management comes in). This also implies that instances of activities will need unique ids, so that the workflow engine knows which instance to invoke. The messaging bit could have several implementations - JMS, email, HTTP, etc. I realise it's early days - these are just a few points to ponder. I'll see if I can adapt my earlier work to what you've come up with. Regards Colin M Sharples I/T Architect Business Innovation Services Group IBM New Zealand email: [EMAIL PROTECTED] phone: 64-4-5769853 mobile: 64-21-402085 fax: 64-4-5765616 Craig R. McClanahan To: [EMAIL PROTECTED] craigmcc@apac cc: he.org Subject: RE: Workflow Support Proposal 14/08/2001 07:25 Please respond to struts-dev On Mon, 13 Aug 2001, Dan - Blue Lotus Software wrote: I've got a few ideas, and a few comments on the proposal. Reading over your workflow support, it looks not that far from what I had done for per-request workflow support. You've also got it extended to multiple pages. Nice. The suggestion below, of allowing people to share workflows, is an interesting one. Given this, here's a few of my suggestions. First of all, I'd like to see some consistent address space available across an entire workflow. I'd like to be able to request a form field value from a FormBean on the 2nd page of the workflow, even though I'm on the 7th page. I'd like to be able to transfer control for a workflow to another user, in a secure fashion. Given this, here's my proposal. In your multi-wizard schema, add a struts:page id=page1 (or whatever) to indicate a single page within the workflow. This would, in turn, contain the struts:forward and struts:navigate tags. The idea is that this provides a namespace for looking up FormBean values. From some other page in the workflow, you should be able to look up values from the FormBean associated with a given page that is identified by name. I guess I'm of a somewhat different opinion on how collaboration like this should happen: * ActionForm beans (from the perspective of the workflow system) are an implementation detail -- collaboration that requires
RE: Workflow Support Proposal
The initial version of the proposal is focused on the Activity and Step components (we refined the vocabulary a little based on feedback -- there is now a distinction between Process and Acivity). Activity is more for things that are relatively short term, and involving only a single person (such as filling out a multi-page wizard), while Process will be more in the long-term, asynchronous, multi-person space that Colin talks about. I confess that I haven't spent large amounts of time thinking about that part of the problem -- although it's certainly realistic to do so. However, at least I reserved a component name for it :-). Within an Activity, my thinking was that state information would be saved in between interactions within a Context -- either as anonymous values on an evaluation stack, or named variables stored in some Scope. Every Context has a local Scope that lasts for the lifetime of the execution of an Activity -- which makes it practical for saving temporary variables between Steps. Contexts also have the ability to have additional Scopes -- in a web implementation, you'd certainly assign Scopes that correspond to request, session, and application scope -- but I view those more as a convenient way to interact with the rest of the application than for managing the interaction state. ASIDE - when thinking about how a workflow engine would work, I was continually struck by how similar the concepts are to the way you design a central processing unit, where a Context contains the instruction pointer (what Step to execute next) plus all of the registers (contents of the Scopes) that represents the current state of an operating system process. A Context itself, in a Struts-based app, would most likely be stored as a session attribute, so that it survives past the end of each request, but stays unique to each individual user. Yes, we have to deal with a single user running multiple simultaneous Activities -- but that will be an implementation detail :-) from the point of view of designing the architecture. So, it seems to me that the Context satisfies the basic requirements for workflow objects that Dan is talking about, but does so using concepts that are independent of Struts (so that the basic machinery can be used in non-web-app environments as well). Does that make sense? In addition, I *don't* believe that the page designer (or the business logic designer, for that matter) should be burdened with any undue requirement to understand (or maybe even know about) the workflow engine. In particular, it is the business process designer, not the page developer, who knows what data has to be persisted -- and that person can simply write it into the script for the Activity. I don't see the need for tags that the designer needs to put on their page to instantiate the state-saving logic -- that should be something that is done by the Activity. What a page designer *does* need, though, is an easy way to create navigation controls within the Activity (i.e. Previous, Next, Finish, and Cancel buttons in a wizard). Those need to be part of the Struts layer on top of the basic workflow system machinery. Craig On Wed, 15 Aug 2001, Dan - Blue Lotus Software wrote: Yes. This is exactly the reason I proposed the struts:page addition to the schema a few days ago. If you are persisting the state across a workflow, you have to have some method of retrieving the state. My thought was to identify each FormBean (the data that creates the state of the workflow) by a unique ID, set as an attribute on the struts:page tag. The workflow could be similarly identified, allowing the management of multiple simultaneous workflows per user. If the workflow objects were stored at the application scope, organised by user, then we would also have a mechanism for transferring a workflow from one user to another. It seems to me that this addition to the schema would answer a lot of people's needs. Thoughts? Is something like this usable? I assume we'd probably need to add an additional tag/attribute that specified the persistence type, and perhaps the scope at which the workflow state objects are held. Cheers! -dan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]. org]On Behalf Of Colin Sharples Sent: Wednesday, August 15, 2001 5:16 AM To: [EMAIL PROTECTED] Subject: RE: Workflow Support Proposal Craig, First of all - great job on the workflow proposal! I had begun writing a workflow system based on basic Struts Actions with a bit of state management thrown in, but what you're suggesting is much closer to what I need. I have a couple of requirements which I'm sure are not unique to my application. The processes I'm modelling take a long time - the overall process takes a few weeks to finish, and individual steps may take a few minutes, a few hours or even a few days. This has a couple of implications: 1
RE: Workflow Support Proposal
Craig, Your Workflow Proposal seems like a great start. Several of us have played around with some workflow concepts on our current project but it would be great to have something integrated into Struts. We have a few concepts that weren't mentioned that we think are worth considering: - bookmarkable workflow steps: This means being able to store all context related to a step in such a fashion that the user can go back to that same the same state in the workflow at a later time. Bookmarkability should be optional (some steps may not make sense to be bookmarked) and so it should be able to be indicated as such when being defined. - a workflow history: keep a state of where the user is and/or has been. Allow this state to be displayed as breadcrumbs, a history list, or with another visual or navigational representation. A tag to render this information seems useful as well. - workflows, especially scripted workflows, will often be customized. The workflow configuration files should support the ability to override a workflow definition - one possibility would be to use a mechanism similar to a Java classpath. The primary concern here is updating installations with new versions of out-of-the-box workflows without overwriting any customized versions of those workflows. - when a workflow is started, it should be associated with a context that is specific to that *instance* of the workflow. This supports the ability for a single user to execute the same workflow multiple times simultaneously. In a web UI, this is usually associated with a user opening multiple windows. This implies that a workflow must have some kind of instance id to uniquely identify its context and also makes it clear that some kind of garbage-collection capability is needed for workflows. - the workflow configuration files/interfaces should support i18n especially for labels since the labels will likely appear in UI menus. - the ability to define pre- and post- conditions (inputs/outputs) for workflow steps would aid in robusting such as ensuring valid workflow definitions. A few basic questions that we have: - Can you give some more detail on your initial thoughts for the Struts implementation? The types of questions I have are: o Where do Actions fit in with workflows/activities in general? Are they used entirely behind the scenes when you write the XML for an activity? From the proposal description, it sounds as if writing an Activity will not require Struts coding... do this imply it will generate the necessary Actions, etc? o Would things like Actions, ActionForms, and JSP pages need to be written so they are workflow aware? o Are you thinking you'd extend ActionMappings to have another layer where they can handle Activities or something else? Here we're wondering if there's a way to retain the overall model programming in Struts but allowing non-programmers to create workflows. - Do you think browser issues can be handled properly? Caching has caused us problems in the areas of handling back, forward, refresh buttons... for example, the previous page displays correctly but the actual state of the workflow has not been updated due to the page being cached and the context state is incorrect. thanks, john sigler
RE: Workflow Support Proposal
Craig, First of all - great job on the workflow proposal! I had begun writing a workflow system based on basic Struts Actions with a bit of state management thrown in, but what you're suggesting is much closer to what I need. I have a couple of requirements which I'm sure are not unique to my application. The processes I'm modelling take a long time - the overall process takes a few weeks to finish, and individual steps may take a few minutes, a few hours or even a few days. This has a couple of implications: 1) there isn't going to be a user waiting for an interactive response; the user triggers the process by sending in some data, and will receive a notification when it's finally finished. In between, it's up to the system to ensure that each step gets executed. 2) Most of the steps will therefore be asynchronous (some of them involve getting approval, sometimes with delays of days). If the steps are asynchronous, this will require some state management, so that the workflow engine knows where to start off again. I had been thinking along the lines that an asynchronous step would actually consist of two steps, one which will send a message to initiate the process, and another to assimilate the results of the step (triggered by some event). Once the message initiating the step has been sent, the activity will be suspended until the end-of-step-event arrives. The workflow engine will invoke the activity, starting up at the point it left off (that's where the state management comes in). This also implies that instances of activities will need unique ids, so that the workflow engine knows which instance to invoke. The messaging bit could have several implementations - JMS, email, HTTP, etc. I realise it's early days - these are just a few points to ponder. I'll see if I can adapt my earlier work to what you've come up with. Regards Colin M Sharples I/T Architect Business Innovation Services Group IBM New Zealand email: [EMAIL PROTECTED] phone: 64-4-5769853 mobile: 64-21-402085 fax: 64-4-5765616 Craig R. McClanahan To: [EMAIL PROTECTED] craigmcc@apac cc: he.org Subject: RE: Workflow Support Proposal 14/08/2001 07:25 Please respond to struts-dev On Mon, 13 Aug 2001, Dan - Blue Lotus Software wrote: I've got a few ideas, and a few comments on the proposal. Reading over your workflow support, it looks not that far from what I had done for per-request workflow support. You've also got it extended to multiple pages. Nice. The suggestion below, of allowing people to share workflows, is an interesting one. Given this, here's a few of my suggestions. First of all, I'd like to see some consistent address space available across an entire workflow. I'd like to be able to request a form field value from a FormBean on the 2nd page of the workflow, even though I'm on the 7th page. I'd like to be able to transfer control for a workflow to another user, in a secure fashion. Given this, here's my proposal. In your multi-wizard schema, add a struts:page id=page1 (or whatever) to indicate a single page within the workflow. This would, in turn, contain the struts:forward and struts:navigate tags. The idea is that this provides a namespace for looking up FormBean values. From some other page in the workflow, you should be able to look up values from the FormBean associated with a given page that is identified by name. I guess I'm of a somewhat different opinion on how collaboration like this should happen: * ActionForm beans (from the perspective of the workflow system) are an implementation detail -- collaboration that requires information from multiple pages should be explicit * If you're using request scope for form beans, they will not in fact exist
RE: Workflow Support Proposal
I've got a few ideas, and a few comments on the proposal. Reading over your workflow support, it looks not that far from what I had done for per-request workflow support. You've also got it extended to multiple pages. Nice. The suggestion below, of allowing people to share workflows, is an interesting one. Given this, here's a few of my suggestions. First of all, I'd like to see some consistent address space available across an entire workflow. I'd like to be able to request a form field value from a FormBean on the 2nd page of the workflow, even though I'm on the 7th page. I'd like to be able to transfer control for a workflow to another user, in a secure fashion. Given this, here's my proposal. In your multi-wizard schema, add a struts:page id=page1 (or whatever) to indicate a single page within the workflow. This would, in turn, contain the struts:forward and struts:navigate tags. The idea is that this provides a namespace for looking up FormBean values. From some other page in the workflow, you should be able to look up values from the FormBean associated with a given page that is identified by name. You should also be able to set up role-based security for a workflow by the workflow name and the page name. I could, for instance, set the third page of an application open for my boss' signature. I would have some GUI that would add my boss to the role associated with the third page of the workflow. In such a case, the state (the collection of FormBeans for the workflow) must be persisted in some manner. I'm not sure if it would be easier/better to make this declarative--declare in the XML config file that the state is persistent--or if the act of assigning a workflow to another user would implicitly cause the state to be persisted. At that point, the other user would have to be able to go to the named page of the workflow. If the user has been assigned the workflow from more than one user, then the user will be allowed to choose which workflow should be used. Thus, runtime workflows are identified by workflow name, page name, and user name (the user name that assigned the workflow...probably set to null of the user initiated the workflow). Now, I think we would have a secure way to transfer workflows from one user to another. Your configuration file format only needs to change slightly. Workflow data can be interrogated during the workflow. I think it could work. It would require lots of JSP tags to facilitate things like selecting a workflow when more than one is assigned. What do you think? -dan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]. org]On Behalf Of Craig R. McClanahan Sent: Saturday, August 11, 2001 4:37 PM To: [EMAIL PROTECTED] Subject: RE: Workflow Support Proposal On Sat, 11 Aug 2001, Michael Rimov wrote: Craig: I read through your proposal. [Which I liked very much] Forgive me if I'm just missing it, but can the workflow support multiple users at any given particular time? What I mean is this: In other words, multiple users that participate in the *same* workflow instance? Yah, I didn't really address that one. I was focused on the individual user interaction - but the multiple-user case is very important as well. App Flow: -User A Performs Step 1 -User A Performs Step 2 -Application Mails User B URL to Step 3 (For example, a content approval message) -User B Clicks on URL to Step 3 User B Performs Step 3 etc... Now obviously, the mailing mechanism would have to be my own logic, but would the workflow cover the whole process with different users performing different steps or would you have to have a separate workflow for each user? In terms of the initial workflow proposal, it could be two individual workflows that are coordinated (and, at the detail level, this seems necessary anyway -- content approval, for example, could be a multi-step process in and of itself). There's also an interesting aspect of this - it is somewhat more asynchronous, because you don't really want person A to be locked out of doing anything else until person B is finished. If so, do you see any way that something like this could be integrated into the workflow engine? I think this is a very useful concept to consider adding. It might take a level of abstraction above the current Workflow component to represent it. There would also need to be a more formalized way to retrieve the current state of a particular flow instance that is in progress (for the web application single user case, you'd just store the Context object in the session). Now to figure out how to integrate it ... Thanks in advance, -Mike Craig
RE: Workflow Support Proposal
On Mon, 13 Aug 2001, Dan - Blue Lotus Software wrote: I've got a few ideas, and a few comments on the proposal. Reading over your workflow support, it looks not that far from what I had done for per-request workflow support. You've also got it extended to multiple pages. Nice. The suggestion below, of allowing people to share workflows, is an interesting one. Given this, here's a few of my suggestions. First of all, I'd like to see some consistent address space available across an entire workflow. I'd like to be able to request a form field value from a FormBean on the 2nd page of the workflow, even though I'm on the 7th page. I'd like to be able to transfer control for a workflow to another user, in a secure fashion. Given this, here's my proposal. In your multi-wizard schema, add a struts:page id=page1 (or whatever) to indicate a single page within the workflow. This would, in turn, contain the struts:forward and struts:navigate tags. The idea is that this provides a namespace for looking up FormBean values. From some other page in the workflow, you should be able to look up values from the FormBean associated with a given page that is identified by name. I guess I'm of a somewhat different opinion on how collaboration like this should happen: * ActionForm beans (from the perspective of the workflow system) are an implementation detail -- collaboration that requires information from multiple pages should be explicit * If you're using request scope for form beans, they will not in fact exist in between requests, or while other pages are being processed. In the jakarta-commons code that I'm about to commit, you'll see that a Context (the dynamic state of a particular computation) has a thing called Scopes associated with it. Every Context has a local scope in which it can stash beans by name, and the lifetime of these beans is the same as the lifetime of the Context (i.e. for as long as you are executing the steps in a particular activity, or until you explicitly remove them). This seems like a better way for various pieces of a computation to collaborate, IMHO. You should also be able to set up role-based security for a workflow by the workflow name and the page name. I could, for instance, set the third page of an application open for my boss' signature. I would have some GUI that would add my boss to the role associated with the third page of the workflow. In such a case, the state (the collection of FormBeans for the workflow) must be persisted in some manner. I'm not sure if it would be easier/better to make this declarative--declare in the XML config file that the state is persistent--or if the act of assigning a workflow to another user would implicitly cause the state to be persisted. That sounds like a reasonable sort of thing to do in the webapp layer on top of the core workflow engine. If we do, I'm a big believer in declarative security, so I'd probably lean that way (and also reuse the user principals and roles already defined in the servlet API). At that point, the other user would have to be able to go to the named page of the workflow. If the user has been assigned the workflow from more than one user, then the user will be allowed to choose which workflow should be used. Thus, runtime workflows are identified by workflow name, page name, and user name (the user name that assigned the workflow...probably set to null of the user initiated the workflow). Now, I think we would have a secure way to transfer workflows from one user to another. Your configuration file format only needs to change slightly. Workflow data can be interrogated during the workflow. I think it could work. It would require lots of JSP tags to facilitate things like selecting a workflow when more than one is assigned. Right now, I want to focus on getting something for the single-user activity scripting up and functional, and then address the cross-user stuff. (Yes, that might mean a partial redesign later, but I think some real code to play with will help crystalize our thinking). What do you think? -dan Craig -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]. org]On Behalf Of Craig R. McClanahan Sent: Saturday, August 11, 2001 4:37 PM To: [EMAIL PROTECTED] Subject: RE: Workflow Support Proposal On Sat, 11 Aug 2001, Michael Rimov wrote: Craig: I read through your proposal. [Which I liked very much] Forgive me if I'm just missing it, but can the workflow support multiple users at any given particular time? What I mean is this: In other words, multiple users that participate in the *same* workflow instance? Yah, I didn't really address that one. I was focused on the individual user interaction - but the multiple-user case is very important as well. App Flow: -User A Performs Step 1 -User A Performs Step 2 -Application Mails User B URL
RE: Workflow Support Proposal
On Mon, 13 Aug 2001, Moore, Robert wrote: I realize this may be *much* more than you wanted to tackle, but you might take into consideration the work of the Workflow Mgmt Coalition. They have a xml schema describing workflow: http://www.wfmc.org/standards/docs/Wf-XML%20Schema%20Description.htm. Just thought I'd pass it along in case it was of interest. Yep, they have a pretty interesting model. I didn't try to copy it directly because it seemed a little heavyweight for scripting a wizard interaction in a Struts app. A couple of other things that I looked at were a similar model at http://www.bpmi.org, and the Web Services Flow Language document from IBM. Rob Craig
