Re: What's next for SCA?

[Jim Marino]

On Oct 26, 2006, at 3:11 AM, Meeraj Kunnumpurath wrote:
Hi,

I think when the dangling wires are resolved depends on, as you said,
what the definition of deployment is. The requirement is for the wires
to be all resolved, before a composite is active and service ready.
Regardless of what we mean by deployment, I think there should be  
still
ability to rewire components in active composite, to cater for meeting
SLA requirements, policy changes etc.

Yes absolutely. I view policy application as similar to AOP in except  
the meta-model is a service assembly and not a Java class. We should  
support the ability to to do this "live" such as say "I want to  
monitor traffic going over binding X". Jeremy and I were speaking  
about this the other day and it occurred to us what we are doing is  
also quite similar to subversion in that we are mutating the wiring  
fabric, or applying a diff against it. If we one of the things we  
support is deployment of SCDL artifacts, we should literally be able  
to use a diff ;-)
Roles are a kind of interesting thing. J2EE specs have always  
mentioned
these different roles. However, mostly the roles of developer/ 
assembler
and deployer/administrator amalgamate into one.

Yes and I think one of the problems with J2EE (EJB and packaging in  
particular) is that it never recognized that the roles often are  
carried out by the same individual. That said, I do think they are at  
least conceptually different.
Also, it terms of advertisement and discovery, would you prefer a
registry-based approach or a more peer-to-peer model, where components
auto-advertise their availability and resolving the wires can  
either be
auto-discovery or manual. From that perspective, deployment could be a
two phase process. First, activating all the components with dangling
wires and then resolving all the wires and activating the whole
composite.

I was thinking more distributed, i.e. non-registry based using  
multicast, for local-link communications but we may need some  
centralized mechanism for communication beyond a local link such as a  
DNS server (e.g. zeroconf used dynamic DNS queries for this). Do you  
have any preference?

On the wiring algorithm, at a high level that seems right since it  
roughly corresponds to what we do within a composite. One difference  
though is we are probably going to need to be able to handle cycles  
between composites.

Jim

Ta
Meeraj

-----Original Message-----
From: Jim Marino [mailto:[EMAIL PROTECTED]
Sent: 25 October 2006 08:23
To: tuscany-dev@ws.apache.org
Subject: Re: What's next for SCA?


On Oct 24, 2006, at 4:48 PM, Meeraj Kunnumpurath wrote:

Hi,

I have been having few thoughts on federated deployment as well.
Currently we wire services to references statically.
However, I think
this is more of a runtime concern rather than a
build/package/ deploy
time concern.
+1
An SCA system could be realised as a set of federated heterogenous
JVMs each hosting a different set of components.
Yes, I think this is a key difference between what Tuscany
is trying to do and more traditional application server
environments that have a more homogenous 2- or 3-tier
topology. Some nodes may not be VMs.
Each component would publish the services it provide based on
supported MEPs, IN/OUT parameters, supported transport
bindings and
policy intent. Based on the published information, an adminstrator
should be able to wire this services to a compatible
reference for a
consumer component at runtime. This wiring should be dynamically
amendable to cater for such requirements like meeting SLAs, policy
changes etc. I am not sure how much this currently fits
into the SCA
spec.
I think it will fit. In SCA we identified a couple of roles
(they may be the same person sometimes, or different
individuals), one of which is an "assembler". This person is
responsible for resolving unbound or dangling wires,
configuration, etc. Also, one of the changes being
considered is the introduction of "intents" which could
correspond to the things you mentioned above. As part of
these changes, we were also considering autowiring by
intent. This will provide even more flexibility since it
will allow the runtime to choose wire targets, bindings,
etc. based on a set of constraints. I believe these changes
will make things a lot easier as having to
"manually/explicitly" wire can get tedious (as well as
result in globs of XML).

So my picture of how things would work at a high level is:

1. Tuscany nodes come up

2. The nodes can either be pre-configured to know about
themselves or would autodiscover each other through a
protocol such as zeroconf. We should look at other protocols
as well but I am currently interested in zeroconf since it
is language-neutral, works with existing TCP/IP
infrastructure and does not require a central registry (the
service discovery part which Tuscany would use can be run
over multi-cast or uni-cast DNS), works in small devices
(e.g. printers and cameras), and is an IETF standard.  Nodes
could be different Tuscany instances running on a variety
host environments, just an agent talking to a middleware
instance with no Tuscany runtime, or represent some
middleware cluster.

3. An SCA system would be constituted from these nodes, and
would itself be represented as a composite.

4. There would be a provisioning infrastructure responsible
for mapping and deploying SCA assemblies to physical nodes.
This may entail breaking a composite apart and deploying
child components to various nodes according to some
(pluggable, extensible, etc.) algorithm, which itself could
use intents to determine where to deploy (or it could
explicitly be told where to go).

5. During the deployment, unbound wires would be resolved.
This can be done explicitly or through autowire.

However, as a user this is something I have looked for in ESBs. I
would say a consumer component only need to know about the
contract
and MEP (to a certain extent) of the service it use. The
assembly of
an available prospective service instance is more of a
runtime concern
than deployment time.

It may depend on the definition of "deployment". For me,
deployment occurs when someone (an assembler) takes an
assembly and instantiates it in an SCA system. As part of
that process, all dangling references and outstanding
configuration must be satisfied. Prior to that, a developer
could create components some of which were composites that
had local services "statically" wired. The key is there may
be reference targets which are only resolvable during
deployment into a live system and those targets may vary
depending on whether the SCA system were a production,
staging, or QA environment.

Does this fit with your thinking?


Another thing I am curious about is scope containers,
instance pooling
and thread confinement. This is kind of related to the JPA
integration
work I have been doing. Certain resources that are injected into
components may not be thread safe. In such scenarios, how
do we make
sure the components themselves are thread confined to avoid race
conditions. Is instance pooling for stateless components a
possible
solution?

Maybe we can kick off a separate thread on this? I'm not
sure thread confinement for component implementation
instances is something we want in all cases, particularly
for session and module scoped ones.
In cases where the resource being injected is not thread
safe, we may want to inject a proxy that resolves to the
correct instance on every invoke (similar to what we do when
a service is injected, e.g. a session-scoped service
injected on a module-scoped component). This would allow us
to have different scopes such as thread-confined or
transactional or re-use existing ones (request) which may
correspond to the lifetime of a Hibernate session or JPA
context more closely.
This would allow us to confine the specific resource as
opposed to the entire component. This will not solve the
issue of race conditions but sometimes the best thing to do
is not try and do too much.

The Java spec is also going to include some revised wording
on how client code should behave with respect to threading
(e.g. spawning threads and calling into SCA code may yield
unpredictable results; use WorkManager).

Jim


Ta
Meeraj


From: Jim Marino <[EMAIL PROTECTED]>
Reply-To: tuscany-dev@ws.apache.org
To: tuscany-dev@ws.apache.org
Subject: Re: What's next for SCA?
Date: Tue, 24 Oct 2006 10:21:45 -0700


On Oct 24, 2006, at 9:37 AM, Jeremy Boynes wrote:

With M2 mostly behind us, I'd like look forward to
things that we
can tackle next for SCA/Java.

I think there is still some cleanup to be done - fixing
bugs that
we find in M2, cleaning up some of the compromises we
made, and so
forth. As part of that I think we still need more coverage on
testing at all levels - unit, function, integration.
I'd like to  see us be in a position to regularly publish
development snapshots  that can be run through some form of test
suite.

I also think there's quite a few improvements we can
make to the
classloading infrastructure. The issues with Spring and
SDO show
the need for a controlled way to share classes between
application
code and the runtime; we also need to deal with the isolation
issues in the itest plugin (as in, dealing with the
current lack of
isolation). This is part of a general issue related to
packaging
formats and how composites correspond to physical artifacts in
general.

Yes I think this is critical, especially in multi-VM
environments.
In  the spec group we are dealing with the issue of
deployment and
packaging now so I think our work on this would
correspond nicely to
that.

This leads into the question of how we deploy things to
servers.
At  the moment we only support server functions by being
part of a
WAR  deployed to an application server; a user still needs to do
quite a  bit of work to set up the war (although the
plugin helps).
To  support users in general and integration testing I
think we need
a  way to deploy/undeploy SCA applications to running servers in
some  native form. We can do this either through extensions to
existing  servers (like the Tomcat integration from M1) or by
enhancing the  standalone environment.

Existing server integration is a nice-to-have but IMO we require
a  standalone environment that can support the full range
of SCA
features which is not easily done in the former.

Once we have the notion of multiple environments, I
think we're in
a position to tackle the more interesting aspects of SCA
dealing
with the composition of services across a network rather than
locally as we have now. Finding a way to map a logical
SCA system
onto a mesh of servers and handling the interconnections
between
the system's services.

Yes this will likely entail some type of service
discovery. I've been
looking at zeroconf and perhaps UPnP as ways of doing
this (support
should be pluggable) and when I have a better idea I will post a
write-up to the list.

Tied into that we should also start to look at the policy and
intention parts of the specification, providing support
for core
aspects such as security, reliability and transactions. We have
the  hooks in the fabric for attaching policy but we need the
implementations of these policies and the mechanisms for
ensuring
the right implementations get engaged.

We are going to need to deal with how intents are mapped
to policy
sets and then runtime artifacts. I see this "intent
engine" as a key
subsystem moving forward. I was discussing this the other
day with
Felix and I'll kick off another discussion on the list
related to
that since it is a broad topic.

We may also need to revisit the async programming model,
both from
the spec side and our implementation. I have a gut feeling that
there are issues lurking there that don't make this as easy for
users as we would like. We also need to extend what we have to
support async MEPs on the wire rather than the simple
sync MEP we
have now.

Yes plus handling of stateful conversations
Finally, there's a general issue with adding additional
bindings
and containers and finishing what we have already (there were
quite  a few in M2 that we didn't think were ready for binary
distribution). There seem to be quite a few in the pipe
with talk
of JMS and EJB, JDBC and declarative DAS.

This is just of brain dump of where my thinking is at
the moment,
I'm sure everyone has their own thoughts about things we should
tackle. It would be good to get to them all on the table :-)

It may make sense to start to categorize these in
buckets. My brain
dump would be:

1. SCA fabric, a.k.a. "SCA system"
- How an SCA System is (auto)constituted
- Service discovery

2. Deployment, provisioning and Management

- How to get things into an SCA system, how logical
artifacts are
mapped to physical runtime artifacts, and how to manage
what is going
on

3. Spec alignment
- Refactoring of core to support pending spec changes
around services
and references

4. Host environments
- A "real" standalone container, particularly to support 1 & 2
- OSGi support

5. Programming model
- Our programming model needs to support "high-value" SCA features
- Stateful conversations
- How databinding is specified
- Complex property support
- Improved autowire capability, including the ability to
autowire by
intents and exposure of autowire to applications
- Better web application support

6. Policy and intents
- We need to support the use of intents for allowing the
runtime to
apply policies as well as select wire targets
- Intent mapping engine
- Transactions

7. Extensions
- Classloader isolation
- Complete (or prune) existing extensions, add new
extensions such
as  JMS and AMQP binding support (we may want to use AMQP
as the
"default" SCA binding)

8. Integration test harness and integration tests

9. Better across-the-board test coverage

10. Better samples that demonstrate the features
mentioned above and
show why SCA makes things substantially easier

I'm planning on getting involved in service discovery, stateful
conversations, improved autowiring and integration testing...

Jim


--
Jeremy



--------------------------------------------------------------------
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]




-------------------------------------------------------------------- 
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


_________________________________________________________________
Download the new Windows Live Toolbar, including Desktop search!
http://toolbar.live.com/?mkt=en-gb



-------------------------------------------------------------------- 
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



-------------------------------------------------------------------- 
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


This message has been checked for all email viruses by MessageLabs

*****************************************************

    You can find us at www.voca.com

*****************************************************
This communication is confidential and intended for
the exclusive use of the addressee only. You should
not disclose its contents to any other person.
If you are not the intended recipient please notify
the sender named above immediately.

Registered in England, No 1023742,
Registered Office: Voca Limited
Drake House, Three Rivers Court,
Homestead Road, Rickmansworth,
Hertfordshire, WD3 1FX


This message has been checked for all email viruses by MessageLabs.

---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]