On Fri, Jun 20, 2008 at 1:30 PM, Hinde Bouziane <[EMAIL PROTECTED]> wrote:
> Simon Laws wrote: > >> On Fri, Jun 20, 2008 at 8:33 AM, Hinde Bouziane <[EMAIL PROTECTED]> >> wrote: >> >> >> >>> Hi Raymond, >>> >>> Yes :), you are right. That what we want to do. >>> >>> Hinde >>> >>> >>> Raymond Feng wrote: >>> >>> >>> >>>> Hi, >>>> >>>> Let me try to summarize my understandings in a few simple sentences: >>>> >>>> 1) We have an assembly describing the composition (logical structure) of >>>> components for an application. >>>> 2) We also have a workflow describing the sequence of component >>>> interactions in the assembly. >>>> 3) As the flow navigates through the composition, we want to select >>>> (based >>>> on the data/transition condition) a subset of the components in the >>>> assembly >>>> for the current window, instantiate them, feed input data to invoke the >>>> components, collect output data and then destroy the current window. >>>> 4) Repeat step 3 until the flow completes. >>>> >>>> Are I'm on the right track? >>>> >>>> Thanks, >>>> Raymond >>>> >>>> -------------------------------------------------- >>>> From: "Hinde Bouziane" <[EMAIL PROTECTED]> >>>> Sent: Thursday, June 19, 2008 9:32 AM >>>> To: <[email protected]> >>>> Subject: Re: dynamic assembly and deployment in SCA Tuscany Java >>>> implementation >>>> >>>> Hi, >>>> >>>> >>>>> Thank you for your reply. >>>>> >>>>> To explain our requirements, here is a very basic example of a use >>>>> case. >>>>> Please let me know if some thing is not clear. You can also find more >>>>> information about the principle of the spatio-temporal component model >>>>> we >>>>> are working on in : >>>>> >>>>> >>>>> https://www.irisa.fr/paris/bibadmin/uploads/pdf/0.21342000%201213723399_paper.pdf >>>>> In this paper, you can note that components are not SCA ones. However, >>>>> we >>>>> are applying the same principle on SCA and try to realize an >>>>> implementation. >>>>> >>>>> Raymond Feng wrote: >>>>> >>>>> >>>>> >>>>>> Hi, >>>>>> >>>>>> Thank you for sharing your thoughts with us. It's indeed a very >>>>>> interesting usage of Tuscany/SCA. Please see some comments inline. >>>>>> >>>>>> Raymond >>>>>> -------------------------------------------------- >>>>>> From: "Hinde Bouziane" <[EMAIL PROTECTED]> >>>>>> Sent: Wednesday, June 18, 2008 7:04 AM >>>>>> To: <[email protected]> >>>>>> Subject: dynamic assembly and deployment in SCA Tuscany Java >>>>>> implementation >>>>>> >>>>>> Dear All, >>>>>> >>>>>> >>>>>>> I'm a post-doc working on implementing an abstract >>>>>>> spatio-temporal component model on top of SCA/Tuscany Java >>>>>>> implementation v. 1.2. The main particularity of the model is that it >>>>>>> offer an assembly approach, based on describing spatial >>>>>>> (provides/uses) >>>>>>> and temporal (data-flow and control flow) dependencies between >>>>>>> components. In this implementation, components are SCA components. >>>>>>> The >>>>>>> assembly of components follows a dynamic approach. That means, the >>>>>>> structure of an application is modified during the >>>>>>> execution. Components are added/removed/connected/disconnected,mainly >>>>>>> depending on temporal dependencies. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> Can you provide us a real-world use case for this theory to help us >>>>>> better understand the requirements? >>>>>> >>>>>> Assume the following simplified form of assembly: >>>>>> >>>>>> >>>>> A(inA, outA) ; B (inB = inA, outB) >>>>> >>>>> It describes a sequence of execution of two components A and B, where >>>>> inX >>>>> and outX are respectively input and output data of components (They are >>>>> realized by >>>>> dedicated ports of the form references/services). The arrival of input >>>>> data triggers the execution of a task implemented by a component. >>>>> Output >>>>> data are produced by the task. Here I skip technical details and the >>>>> spatial >>>>> dimension as the example should be sufficient to illustrate our >>>>> requirements. >>>>> >>>>> For such an assembly our aim is to dynamically deploy components A and >>>>> B. >>>>> That means, we'll change the assembly during the execution. The >>>>> principle is >>>>> similar to what happens in workflows. For the example a suited behavior >>>>> is >>>>> to: >>>>> >>>>> 1. creat A. >>>>> 2. send inA, execute its task A, retrieve outA. >>>>> 3. remove A - create B. >>>>> 4. send inB=outA, execute its task B, retrieve outB. >>>>> >>>>> To manage the data movement between A and B, a proxy component is >>>>> introduced. So, according to the previous behavior, the assembly at >>>>> execution will be: >>>>> >>>>> • At time 0: >>>>> • At time 1: A connected to Proxy. >>>>> • At time 2: Proxy connected to B. >>>>> • At time 3: >>>>> >>>>> Where Proxy is the component receiving the outA data. It is sent to B >>>>> after B is created and connected to Proxy. (connection = wiring) >>>>> >>>>> To obtain the suited behavior, we need the ability to dynamically >>>>> create >>>>> new component instances. That means a first API is needed to be able to >>>>> dynamically add/remove nodes and a second API is needed to add/remove >>>>> new >>>>> components inside a node without preliminary knowledge of the content >>>>> of the >>>>> node. >>>>> For the example, if we assume that we want to have A, Proxy and B in >>>>> different nodes, we want to be able to programmatically do some thing >>>>> like: >>>>> >>>>> 1. Creat Node_A, create Node_Proxy >>>>> 2. Execute A ... >>>>> 3. create Node_B, connect Proxy to B, remove Node_A >>>>> 4. Execute B ... >>>>> >>>>> --> With the SCA/Tuscany Java we can add nodes, but I would like to >>>>> know >>>>> if it is possible to remove preliminary knowledge of the node by the >>>>> domain. >>>>> In particular, that is because both the content of a node and the >>>>> machine on >>>>> which I the node will be launched are not known before launching the >>>>> application and the domain. If it is possible, Can you please direct me >>>>> what >>>>> I have to do. >>>>> >>>>> Or if we assume that we want to have A, Proxy and B inside the same >>>>> node: >>>>> >>>>> 1. Create Node, add A and Proxy in Node >>>>> 2. Execute A ... >>>>> 3. add B in Node, connect Proxy to B, remove A from Node >>>>> 4. Execute B ... >>>>> >>>>> --> Here I would like to create an empty node (so, an empty composite) >>>>> in >>>>> which I can add/remove components, I'm quite lost in realizing that. I >>>>> saw >>>>> the node2-impl sources with the possibility of updating a node content >>>>> but I >>>>> don't know exactly how to proceed, if I have for example to provide in >>>>> all >>>>> cases an XML description of nodes before the execution of an >>>>> application. >>>>> Because that is problematic for us if we want to decide about the >>>>> placement >>>>> of nodes and their content during the exection. >>>>> >>>>> I respond inline now to your reply : >>>>> >>>>> >>>>> >>>>>> Independently of the particular usage of SCA/Tuscany implementation >>>>>>> for this work, we are facing some issues regarding dynamic >>>>>>> creation/removal of components for execution on distributed >>>>>>> resources. We aim to find a help to resolve these issues. >>>>>>> >>>>>>> Before I cite encountered issues. I would like just to mention what I >>>>>>> understand from the usage of SCA/Tuscany to execute an application on >>>>>>> distributed resources. If I'm wrong, please do not hesitate to >>>>>>> correct me. >>>>>>> >>>>>>> >>>>>>> * To execute an application in SCA a set of XML descriptors have to >>>>>>> be >>>>>>> provided to describe: components and connections, nodes hosting >>>>>>> components, domain content. >>>>>>> >>>>>>> >>>>>>> >>>>>> Actually, we only need the following information to execute a SCA >>>>>> composite application: >>>>>> >>>>>> 1) The deployable top-level composite (resolved in the SCA domain) >>>>>> 2) A collection of SCA contributions (archives containing artifacts, >>>>>> such as a JAR or ZIP) which are required by the composite. >>>>>> >>>>>> 1+2 is the install image and it's the input to the node. The node will >>>>>> be responsible to run the composite application. >>>>>> >>>>>> ok, >>>>>> >>>>>> >>>>>>> * The domain has a view of (or part of) the application structure and >>>>>>> its deployment through nodes. The domain is responsible to find a >>>>>>> service when this latter is asked. >>>>>>> >>>>>>> >>>>>>> >>>>>> The domain also knows about the contributions by URL. >>>>>> >>>>>> >>>>>> >>>>> yes >>>>> >>>>> >>>>> >>>>>> >>>>>> >>>>>>> * A node looks like a container encapsulating one or more >>>>>>> components. These components are grouped inside a composite >>>>>>> representing the node. >>>>>>> >>>>>>> >>>>>>> >>>>>> Node is modeled as a composite at this point. It configures the base >>>>>> URIs for the bindings. The node also references the deployable >>>>>> composite. >>>>>> >>>>>> >>>>>> >>>>> yes. >>>>> >>>>> >>>>> >>>>>> >>>>>> >>>>>>> * The set of defined nodes determines the deployment and placement >>>>>>> constraints of components (example: if tow components are defined >>>>>>> inside a node, they will be created within the same JVM). >>>>>>> >>>>>>> >>>>>>> >>>>>> Components are defined in composite, not node. It should be >>>>>> potentially >>>>>> possible that a node only picks a subset of the components in the >>>>>> deployable >>>>>> composite. >>>>>> >>>>>> >>>>>> >>>>> I mean "included" in the node, sorry. >>>>> >>>>> >>>>> >>>>>> >>>>>> >>>>>>> * To execute an application, a domain is first launched. Second, >>>>>>> nodes >>>>>>> are launched on targeted machines. Each node registers itself in the >>>>>>> domain. Last, the execution can be launched for example by a client >>>>>>> program invoking services on deployed components. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> Not necessarily. The domain is more an administrative concept. We have >>>>>> a >>>>>> web-based domain manager in Tuscany today. It's responsibility is to >>>>>> build/resolve the install image. The install image can be then saved >>>>>> to an >>>>>> XML file so that it can be loaded without the presence of the domain >>>>>> manager. Even though the domain manager can provide a button to start >>>>>> a node >>>>>> (push the install image to a node), the node can pull the install >>>>>> image from >>>>>> a URL (either live from the domain manager or pointing to a pre-saved >>>>>> file). >>>>>> >>>>>> The node doesn't have to explicitly register with the SCA domain, >>>>>> especially for the pull mode. >>>>>> >>>>>> >>>>>> >>>>> Is there an example of pull mode usage in the distribution or a >>>>> documentation? >>>>> >>>>> >>>>> >>>>>> >>>>>> >>>>>>> Encountered problems with some suggestions to try resolving some of >>>>>>> them: >>>>>>> >>>>>>> Issue 1. Component connections: there is no API provided to the user >>>>>>> to connect components. Connections are done by the runtime from >>>>>>> assembly descriptions. So there is no direct way for the user to >>>>>>> connect dynamically ports (references/services). For our >>>>>>> requirements, I adopted a solution to define a service with >>>>>>> connect/disconnect operations on a user side port (reference) rather >>>>>>> than define a reference port. Thanks to such methods I can pass a >>>>>>> service reference to the user component, which in reality realizes a >>>>>>> connection. The principle is similar to what we can find in other >>>>>>> component models like CCM. >>>>>>> >>>>>>> >>>>>>> >>>>>> I assume you are talking about "wiring" when you say "connection". The >>>>>> whole composite model is open to use, you can simply create the >>>>>> deployable >>>>>> composite programmatically. >>>>>> >>>>>> Do you mean that there exists an API that I can use to connect a >>>>>> >>>>>> >>>>> reference to a service rather than having to specify the connexion in >>>>> an XML >>>>> descriptor? >>>>> >>>>> >>>>> >>>>>> Issue 2. Deployment/placement for distributed execution: Components >>>>>>> within a node and nodes are statically specified. The concept of >>>>>>> node, >>>>>>> like it is realized, requires a whole application assembly to be >>>>>>> divided through node descriptions and to know execution resources in >>>>>>> advance. That also means that the hierarchy of an assembly should be >>>>>>> broken by the user when for example sub-components are to be deployed >>>>>>> on different machines. That can be tedious to do for the user and a >>>>>>> strong dependency between assembly and deployment is created. It can >>>>>>> be fine if an assembly is kept unchanged for different execution on >>>>>>> different resources. In this direction, if there is a way to create >>>>>>> "empty" nodes on which it is possible to add/remove components, part >>>>>>> of the issues can be resolved to avoid for example splitting an >>>>>>> assembly through nodes. I saw the Node implementation and there is no >>>>>>> API allowing that. >>>>>>> >>>>>>> >>>>>>> >>>>>> Again, the node is modeled as a SCA composite so you can create it >>>>>> programmatically too. Just look at the the node composite files in our >>>>>> store-tutorial. >>>>>> >>>>>> >>>>>> >>>>> Yes, probably I'm not clear, sorry. I'm using the concept of node to >>>>> realize our use cases for distributed execution. For me, the concept of >>>>> node >>>>> (which is not in the SCA specification) seems to be used for deployment >>>>> concern, right? If I have a whole application to deploy, I'll devide >>>>> its >>>>> assembly through nodes descriptions. Because in our use cases, we aim >>>>> to >>>>> decide about deployment during the execution (for node placement and/or >>>>> its >>>>> content) I would like to know if there is APIs to create empty nodes >>>>> add/remove within a node and what is required to be able to do that. >>>>> >>>>> >>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Issue 3. Behavior when a service is requested: The domain is aware >>>>>>> about the services provided by components. These services are known >>>>>>> thanks to nodes registration. However, it seems that the registration >>>>>>> is not sufficient to find a service. The knowledge of node placement >>>>>>> (host on which it is launched) is also required. That is why a host >>>>>>> is >>>>>>> specified in a node XML descriptor. In addition to the problems cited >>>>>>> in Issue 2, that represents an issue for dynamic deployment. Why >>>>>>> nodes >>>>>>> registration is not sufficient to be able to find a service? If a >>>>>>> node >>>>>>> can pass its placement information to a domain when it registers >>>>>>> itself, it should be possible to avoid the need of static description >>>>>>> of node placement. Isn't it the case? And if the concept of empty >>>>>>> nodes (see Issue 2) is considered, are XML descriptors of nodes still >>>>>>> necessary? Is the proposal feasible by light extensions or >>>>>>> modifications of the SCA Tuscany Java implementation? >>>>>>> >>>>>>> >>>>>>> Issue 4. Dynamic addition of nodes: Is it possible to add dynamically >>>>>>> a node definition in the domain? I'm revisiting the SCA >>>>>>> specification and in which there is a notion of Domain-level >>>>>>> composite. It seems to allow adding/removing nodes in this way, but >>>>>>> I'm not sure. Can that be a solution? if yes, is it supported in >>>>>>> SCA Tuscany? Otherwise, is the support of dynamic addition/removal of >>>>>>> nodes can be easily feasible? >>>>>>> >>>>>>> >>>>>>> >>>>>> Node is not a concept in the SCA spec. The domain-level composite has >>>>>> "include" for each deployable composite added to the SCA domain. >>>>>> Are you aware of the node2-api, node2-impl and node2-launcher modules >>>>>> in >>>>>> the code base? It provides a way to create node and launch it. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> Any help/suggestion is welcome. Thank you. >>>>>>> Hinde >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> Thank you very much for your help; >>>>> Hinde >>>>> >>>>> >>>>> >>>> >>>> >>> Hi Simon, hi all, > > Hi >> >> This is an interesting thread. Can you give me an idea of : >> >> 1 - how frequently the logical components (as opposed to the instances of >> those logical components) change. >> >> > Do you mean logical composition? > > For me the "logical" composition is the application description that > represents the designer point of view, before execution. It kept the same. > An instance of the composition will be the architecture of the application > at execution. It can be different from the logical. In the example (A;B), I > added the Proxy which is not in the logical composition. The result is an > instance of the original composition that have an architecture changing at > execution. The frequency of changes depends on the both the execution time > of tasks in components and the policy adopted to manage the evolution of the > workflow. > > For example: > - A policy can be the one that I cited in the (A;B) example, where a > component is created once the flow reaches it. The frequency of changes > depends on the flow and time execution of tasks. If the task of A takes one > day, the manager of the application architecture creates B once this task is > finished. > > - A policy can be based on prediction mechanisms to create in advance > components that may be used in a near future in the flow. Before the > completion of the task of A, B can be created. > > I hope that help understanding, Do not hesitate to tell me if something is > still ambiguous. > > Regards, > Hinde > > > 2 - how frequently the workflow that directs the order of execution of >> these >> components changes >> For example, Is it every request, every minute, every six months, etc. >> Regards >> >> Simon >> >> >> > > Thanks Hinde I don't know it this helps but let's talk about the distinction between how an application is assembled and the way that work flows through it at runtime. From an SCA point of view each time work flows through an assembled application it may take a different route. This depends on the business logic in each component in the application. This is true even though we may only change the description of the composite application every 6 months. An example, a digital film rendering application which, on a frame by frame basis does the following (I'm just making this up b.t.w). 1. Apply wire frame 2. Render background 3. If there are characters in the frame render characters. In the SCA composite for such and application I could imagine 4 components A. FrameRenderingCoordinator B. WireFrameProcessor C. BackgroundRenderer D. CharacterRenderer Where the FrameRenderingCoordinator contains a piece of workflow that is wired to components B, C & D and always calls components B & C and then makes a decision whether to call D or not depending on if there are characters in the scene. To spread load across machines we could deploy these components into a clustered environment that supports SCA (see demos/load-balancing-webapp). Alternatively we could write some new binding infrastructure that is able to spray work out based on some set of policy(we can't do this today). In both of these cases the application developer just assembles the business application and pays no attention to how the work is distributed. I suppose I'm trying to demonstrate here how I think about SCA applications. In your case you have some interesting infrastructure ideas, e.g. predictively creating component instances if it is anticipated that flow will reach them in the future. It's unclear to me how you want to represent these kinds of rules. Are you suggesting changes to the details of the Tuscany runtime infrastructure to accommodate these improvements without the application developer being aware? Are you suggesting that the application itself should be making these decisions, for example we could rearrange our example flow to read 1. If there are characters in the frame ask character renderer to get ready while I do steps 2 & 3 2. Apply wire frame 3. Render background 4. If there are characters in the frame render characters. Regards Simon
