Hi Rich Some comments in line...
Regards Simon On Tue, Aug 5, 2008 at 1:07 AM, Rich Smith (rjsmith2) <[EMAIL PROTECTED]>wrote: > Hi, Simon. Thanks for your reply. > > If I understand you correctly, the contents of the composite can be > configured before starting > the nodes, but once the nodes are running, the configuration is basically > fixed until restarting. > > Yes > I think this would probably work for our application, at least for now. > > The basic problem we are trying to solve is to create a scalable network > management > application that can manage anything from small to extremely large > networks. We are also > trying to modularize the architecture of the application so that > infrastructure components > and feature components can be developed separately, but run as a single > application. This > Sounds good. > is becoming more important as we consider allowing partner companies > develop features > that plugin to our application. > > SCA seems like a good fit for both of these criteria. > > We currently have the application running as a single, monolithic JVM, but > are running into > performance problems as customer networks get larger, and we are running > into stability > problems as the size of our development team increases. > > Components would consist of a mixture of infrastructure and application > code. > > Examples of infrastructure components would be: > > - Object Persistence (using Hibernate + RDBMS) > - Scheduler > - SNMP device communication > - Notification processing / fault correlation > - Poll-based data collection > - Report generation > Most of these sound like they could be represented as components. Is ObjectPersistence something you intend to continue to support through the existing Hibernate API. I don't know much about SNMP but I guess you could have a component providing coordination of this kind of traffic. Do you need to create a special SCA binding implementation for this. Is it socket based? > > Examples of application components would be: > > - Controller inventory > - Client management > - Topology / maps > > And the intentions is to allow these to be wired to the infrastructure components in order to build a bespoke management app? > > In the low-end case, ideally, we could deploy the entire application into a > single JVM. > > In the high-end case, I would like to split the application across several > servers, the number > of which is configurable based on the size of the network. > > What I would like to do is have a front-end server handling the webapp, a > database server, > a "coordinator", which is a combination of domain manager, fault > correlator, and controller > for configuration operations, and finally, 1..n back-end servers. > There is some support in Tuscany for representing Web2.0 type apps as SCA components and hence allowing them to be wired into other components using the familiar model. Take a look at the store tutorial if Web2.0 shakes you tree. > The back-end servers will be running some combination of data collection, > persistence, > scheduler, and device communication, as well as some notification > processing. In this case, > the data collection and device communication would probably be SCA > components having > remoteable interfaces, while the scheduler and persistence would be just > libraries that are > used internally to the components. > > When the front-end needs to communicate with a device, it will have to > select the > back-end server that has responsibility for that device, and make a remote > request. > > If I understand your E-mail, we could implement this by having "n" > components with different > names but the same implementation, and deploying them across "n" back-end > servers, > while using an appropriate domain manager configuration. > Yes, sounds right. You could of course include an interface in you back end components to provide information about which devides they manage. > Then, if the "controller" component > has a 1..n reference to the back-end components, it can get a collection of > the back-end > service instances wired to it, and query each one to determine which one it > is. At that point, > it could select whichever component (or service) it needs for a given > operation. > Sounds like we are saying the same thing ;-) > > If SCA can handle the communication between processes in a transparent way, > then > that would be an enormous help to us. If it can have components deployed > in a single > node or across multiple nodes without any code changes, then that would be > ideally > what we are looking for. (It would be amazing, really, because the ability > to do this seems > like a really recent development.) > That's the idea and it sounds like this would be a good exercise to make sure it does. > > It sounds like SCA will work for our application, but please correct me if > you see anything > here that is a bad fit or that will not work in an SCA model. > It looks good to me. The details of what makes a good component in your particular environment will obviously need some attention but I think you're going in the right direction. > > What I really wanted to do originally was to implement the libraries and > components as > OSGi modules, and use Spring DM to provide AOP, security, and webapp > integration. > From looking at the osgi-supplychain example, though, it seems to be that > serious support > for OSGi in Tuscany is not yet available. (This is why I sent my other > E-mail about how > hard it would be to get a Spring DM implementation type in Tuscany. I > cannot believe that > the OSGi sample wires its own service references.) So, I think our plan > might now be to try > the Spring implementation, and then switch to OSGi at some point in the > future. > Yeah, we're just catching up with the OSGi support. The first thing Raymond is looking at improving is OSGi support for tuscany itself so that the Tuscany jars can themsleves be easily deployed into and OSGi framework. Other improvements will be driven by whoever has the motivation and the time. You are very much invited to jump in if you get to the stage where there is an itch that needs scratching and no one seems to be doing anything about it. > I hope this makes sense, since I'm really new to SCA and Tuscany. I'd be > happy to provide > more details if you need them. > It looks plausible to me so far. Very interested to hear how you get on and of course post here if you get stuck and need more help. > --Rich > > > ------------------------------ > *From:* Simon Laws [mailto:[EMAIL PROTECTED] > *Sent:* Monday, August 04, 2008 2:35 PM > > *To:* [email protected] > *Subject:* Re: Question on dynamic configuration of distributed > application > > > > On Mon, Aug 4, 2008 at 5:14 PM, Rich Smith (rjsmith2) <[EMAIL > PROTECTED]>wrote: > >> Thanks, Raymond. I'll investigate your suggestions, and post back with >> any more >> questions. >> >> --Rich >> >> >> ------------------------------ >> *From:* Raymond Feng [mailto:[EMAIL PROTECTED] >> *Sent:* Monday, August 04, 2008 8:55 AM >> *To:* [email protected] >> *Subject:* Re: Question on dynamic configuration of distributed >> application >> >> Hi, >> >> Thank you for your interest in Tuscany. I have two quick comments inline. >> >> Raymond >> >> *From:* Rich Smith (rjsmith2) <[EMAIL PROTECTED]> >> *Sent:* Monday, August 04, 2008 7:57 AM >> *To:* [email protected] >> *Subject:* Question on dynamic configuration of distributed application >> >> Resending this question now that I am able to get on the mailing list. >> Apologies to >> anyone who gets it twice. >> >> Hi, all. I'm working on evaluation of Tuscany Java, and need some advice >> on a >> particular configuration/use-case that we are planning for our >> application. >> >> The application will be scalable from a low-end deployment where it runs >> on a >> single server to a high-end, distributed deployment, running on several >> servers. >> SCA and Tuscany seem like a good fit for this. >> >> In the high-end case, there will be several instances of "back-end >> servers", each >> of which will run different instances of the same components. The >> front-end will >> keep a mapping of which back-end server to use for which resources, and so >> this >> is _not_ going to require any kind of load balancing. >> >> So far, I've figured out that I can register the back-end server >> components using >> separate service names, and just get those service references using the >> appropriate >> names. >> >> <rfeng>In a SCA domain, the components in top-level composites should >> have unique names. But we can configure more than one components to use the >> same implementation. Would it help for your use case?</rfeng> >> >> However, I want to configure the back-ends using our application and not >> using >> static files (as in the calculator-distributed sample) because it will be >> necessary >> to allow customers to add back-end servers as they need to >> increase capacity. >> >> The thought of having to regenerate all of the .composite files does not >> appeal to >> me, and I don't really want to generate five copies of the same file with >> different >> service names. >> >> What is the best approach for this? Is there something I can do using SPI >> to >> dynamically configure a distributed Tuscany Java application? (It's >> probably okay >> if restart is required somewhere, but it would be even better if not.) >> >> <rfeng>SCA does have the deployment composite concept. It allows we use a >> dynamically configured composite (instead of being packaged in a SCA >> contribution) to deploy the application. In Tuscany, we have the following >> API: >> >> org.apache.tuscany.sca.node.SCANode2Factory.createSCANode(String, String, >> SCAContribution...) >> >> The 2nd argument is a String representing the XML for the deployment >> composite. >> >> We also have a SCA Domain Manager in Tuscany which provides the >> administration capability. It's built as a SCA composite application with >> Web 2.0 front end. The function can be used programmatically to introspect >> contributions/composites, resolve wirings and configure nodes for >> deployment. You might be able to explore it for your purpose. >> >> Please try the store tuturial to get a better understanding: >> https://svn.apache.org/repos/asf/tuscany/java/sca/tutorials/store/domain >> >> </rfeng> >> >> Please let me know. >> >> --Rich >> >> > > Hi Rich > > Both the calculator-distributed sample and the store tutorial, that Raymond > mentions, use the same underlying model of an SCA Domain. Here's how it > works. > > - You build your application as a set of composites where each composite > may run on a separate node > - References in one composite may target services in another by explicit > name > - The workspace.xml, domain.composite and cloud.composite file you have > looked at model these composites and their association with physical nodes > - A domain manager app uses this information to work out what the the real > URLs for services and the references that target them will be > - When you run up a node with the composite that is intended to run on that > node it is configured with the correct endpoints. Composites running on > other nodes are able to find it as they are also configured (written on > demand) by the domain manager. > > As you have seen from the calculator-distributed sample this is fairly > static. There is a domain manage GUI app that allows you to add composites > to your domain which does actually write the worskpace.xml, domain.composite > and cloud.composite files for you (start the calculator-distributed sample, > ant runDomain, and point your browser at http://l3aw203:9990/ui/home/). > > You add and remove composites to configure your domain and when you are > happy with it you can start nodes out there on the network where you expect > them to be and the node will use some configuration to find the domain > manager and suck down the information relating to the composite it is > supposed to be runnning. Once you've started a node it is currently stand > alone. It won't pick up any subsequent changes you make to the domain. To > pick up changes you have to restart the node. > > So how to do what you need to do? > > The wrinkle you face is that in SCA component names have to be unique > (actually they only need to be unique within the context of a composite but > for the purpose of this description it's the same thing). If you have 5 > different components that you want to wire to then you need to define 5 > different components with 5 different names. They can of course all have the > sample implementation. There is no sense in the SCA assembly model that you > wire to instances of compnents. So your 5 components running on 5 different > nodes are not different instances of the same component but separate > components. > > In your case you are suggesting that the front end can select a service > based on some knowledge of the resource that the service is running on. This > actually sounds like it would fit the model of explicitly named service > components well as you need to differentiate a component running on one node > from a component running on a different node with different resource > properties. To add a new component though would, as you say, require that > you create a new composite for the service component. We could get round the > need to manually update the front end composite by setting it to autowire to > available component services with the right sort of interface. You would > still need to restart the front end node though. > > How does your front end app differentiate between services? If we took the > approach above the front end component would have to have a collection of > references to services (a reference with a 1..N multiplicity) and hence no > easy way to identify one reference from another. You could of course have a > method on the service that returns information that allows you to identify > the properties of the service and relate these properties to the reference's > location in the reference collection. > > There are some other tricks you could think about, for example, take a look > at demo/load-balancing -webapp. This takes a different approach and defines > a very simple two component assembly > > client -> service. > > The same service composite is then deployed to several different nodes. The > client thinks it's only wired to one service and always sends it's request > to the same URL, the URL in the binding of that composite. However that URL > happens to be the URL of the front end of the load balancer that spreads the > load across the separate instances of the composite. Here you can start as > many back end nodes as you like dynamically but the front end doesn't get > any control over which one is called for any given request. > > That's probably enough for now but I'm interested to hear more about your > application and how we make it run on Tuscany. > > Regards > > Simon >
