More comments inline.
Jim Marino wrote:
Comments inline
On Jul 6, 2006, at 6:17 PM, Jean-Sebastien Delfino wrote:
Jeremy,
I won't comment on your attacks at the bottom of this email. I was
hoping for a more constructive technical discussion. I added my
answers and comments on the specific technical issues inline.
Jeremy Boynes wrote:
On Jul 5, 2006, at 12:43 PM, Jean-Sebastien Delfino wrote:
My proposal is not to merge M1 and the core2 sandbox. I am
proposing to start a new fresh code stream and build the runtime
through baby steps. We may be able to reuse some pieces of existing
code, but more important is to engage our community in this
exercise and integrate the new ideas that will emerge from this.
I don't believe the two issues are necessarily coupled. Quite a few
members of the community are engaged on the sandbox code already and
we could work with you to improve that rather having to throw
everything out and start over with all new ideas.
Here's an example where I'm struggling with both M1 and the core2
sandbox and thinking that we can do better if we start with a new
fresh stream: our (recursive) assembly metadata model.
- M1 does not implement the recursive composition model and would
require significant changes to support it. Core2 is an attempt to
implement it but I'm not sure it's quite right, and also think that
it can be simplified.
It would really help if you could come up with concrete areas where
it is not right or where it could be simplified - for example, end
user scenarios that are not supported.
- M1 used Lists to represent relationships, Core2 uses Maps, I
think M1 was better since it allowed to keep the order in the
relationships.
There's nothing I remember in the assembly spec where order matters.
On the other hand there are many areas where things are keyed by a
name which has to be unique. This seems like a natural mapping
(sorry) to a Map. In M1 I started to move toward simple map
structures but you replaced it with what seemed like a fairly
complicated specialized List implementation that sent notifications
that updated a Map anyway. Given the desire for simplification, are
there any end-user scenarios that require ordering to be preserved
and that can't be supported with a basic HashMap or LinkedHashMap?
As an administrator I'll want my administration tool to display
components displayed in the order I declared them in SCDL.
SCDL isn't the only form assembly can be serialized to/from. Also, if
I were an admin, I'd probably want to sort the components according to
some useful criteria, not how they are listed in a SCDL as most admins
will never look at XML. One could always use LikedHashMap though.
Maybe SCDL isn't the only form but this is not relevant, we need to
support SCDL don't we? As soon as you put assembly elements in a
document that a user/developer can edit the order is relevant.
I disagree with your statement about administrators. They often look at
and work with XML configuration files. If you want to support other sort
criteria in addition that's fine, but admin, config and editing tools
need to at least support the order from the XML document.
I'll also want a configuration or admin tool loading/saving modified
SCDL to write things in the order that they were initially, not in a
random order. As an application developer I'd like to have an SCA
debugging tool showing me my components in a list in the right order
as well. Also if I want to implement the model defined by the XML
schemas in the spec using any of the DataBinding technologies out
there, I'll end up with Lists, not Maps.
We have been using StAX just fine for this and it accommodates a
number of databinding solutions for extensions. Are you proposing we
revisit this decision made back before the M1 release to go with STaX
loading? If so, for what reasons? BTW, not all databinding solutions
will have problems - XStream will work just fine with what we have.
Also, are you sure about XMLBeans and JAXB or are you just speaking
about a current implementation of SDO?
Not quite correct, the decision we made back before M1 was to go with
StAX loading, write the loaders by hand for now, and see how the SDO
team could generate this code after M1. Independent of that, I don't
want to tie us to any specific data binding, so we better pick
representations for model relationships that are commonly used by most
databindings to represent XSD <element... maxOccurs="unbounded"/>, i.e.
Lists, not Maps.
Finally even if we decided to use Maps in some cases to provide keyed
access to some elements of the model, we'd have to do it differently.
For example a single Map containing all components, references and
services in a composite (according to the spec they cannot have the
same names) instead of three Maps like you have in Core2.
And this is why LinkedHashMap will not help you here.
- Core2 only defines implementation classes for the model, I think
we should have interfaces + default implementation classes instead,
like we had in M1, to allow for alternate implementations of the
model.
One of the most complex things with the M1 model was all the
interfaces involved, the need to pass factory implementations
around, the number of different factories involved (one per
extension implementation) and the potential issues with code
assuming its implementation of the factory was the one used.
The core2 model uses concrete classes which are really just data
holders - there's no behaviour in them to be abstracted through the
interface. This gives a much simpler programming model for
extensions using the model.
Do you have any scenarios that would require different
implementations of the model? Are they so different that they might
as well just use different classes?
I don't think that having just implementation classes is much
simpler. If you interact with the model SPI, reading interfaces is
simpler IMO and more suitable for inclusion in a specification
document... allowing multiple implementations of these interfaces.
Also we have to support the whole lifecycle of an SCA application
(development, deploy/install, runtime, admin etc.) and I'd like to
allow some flexibility for different tools, running at different
times to use different implementations of the assembly model interfaces.
Oisin from the STP project said the POJO based approach would suit
them just fine. I don't see the complexity. On the contrary, all of
the AssemblyFactories we had in M1 lead IMO to a massive antipattern
where they were passed throughout the core. I'm happy to walk through
the relevant code if people are interested. All the factories did was
new up a POJO. Not worth the complexity in my opinion but I'm happy to
compare the work in the sandbox with your proposal if you'd like to
walk us through it.
When the runtime depends on too many factories, this is the
manifestation of bigger coupling problems. The factories for all the
extensions should not be visible at all from the core runtime, and if we
externalize the WSDL and Java interface support and the Java
implementation support our of core like I'm proposing, you're not
dealing with many factories.
I am sure that tooling projects will need to add much to this model,
support for events, change tracking, tracking between XML elements and
model objects to provider proper feedback to application developers,
integration with modeling technologies used in the tooling world,
support for cloning maybe... tons of things. I spent several years
developing tools so I think I know what I'm talking about here. The
first thing I'll ask as a tooling developer is: please give me
interfaces so I can hook what I need in the implementations.
I'm happy to walk people through the interfaces or answer any questions
on the list,
- Over usage of Java Generics breaks flexibility in some cases, for
example Component<I extends Implementation> will force you to
recreate an instance of Component to swap its implementation with
an implementation of a different type (and lose all the wires going
in/out of the component).
There may be cases where generics may be overkill but I don't think
that really requires us to throw out the model. There are other
cases where the use of wildcards would be appropriate; for example,
in the scenario you give here you could just create a
Component<Implementation> to allow different types of implementation
to be used.
Then instead of
Component<Implementation> {
Implementation getImplementation();
}
I think we can just do
Component {
Implementation getImplementation();
}
What we have now in core2 is overkill IMO.
then do we need to cast to the right impl type?
The core runtime should not have to cast, simply because it should not
depend on any component implementation type (not even the Java or System
implementation types).
- Core2 defines ReferenceDefinitions (without bindings) and
BoundReferenceDefinitions (with bindings). IMO there are Reference
types and Reference instances and both can have bindings.
or Reference.
I'm with you here - we need to refactor the way bindings are handled
for both Service and Reference. One thing the sandbox model is
missing is the ability to associate multiple bindings with a single
Service/Reference.
My main point is not about supporting multiple bindings on a Service
or Reference. I think this is secondary and the interfaces I put in
my sandbox to support a design discussion don't even have that
either. My point is that Services, References, and their
instantiation by Components are at the foundation of the SCA assembly
model... and therefore need to be modeled correctly. I'm proposing a
different design, illustrated by the interfaces I checked in.
Could you elaborate?
I think it should be very simple:
- Component types have service and reference types
- Components are instances of component types and have services and
references, which are instances of the service and reference types
- Service and reference types can have bindings
- Bindings can be overriden in service and reference instances
This is clear when you look at a Composite. A composite is a Component
Type, has service and reference types (aka composite services and
references) which can have bindings.
A component can be implemented by a Composite, has services and
references, which can use the (default) bindings from their respective
service and reference types, or specify (override) bindings.
- I think that Remotable should be on Interface and not Service.
I agree Service is wrong and that it should be on ServiceContract.
Thanks for catching it.
- Scope should be defined in the Java component implementation,
separate from the core model.
Scope is not a Java specific concept.
Interaction scope (stateless vs. stateful) can apply to any
ServiceContract.
Container scope is the contract between an implementation and a
ScopeContainer and applies to any implementation type that can
support stateful interactions. This would include JavaScript,
Groovy, C++, ... I think that means that support for state
management (which is what scope is configuring) belongs in the core
with the configuration metadata supplied by the implementation type.
I don't think it's quite right. First interaction scopes are defined
on interfaces and not service contracts. Also they control whether
an interface is conversational or not, independent from any state
management.
Anyway I was talking about a different scope, the implementation
scope defined in the Java C&I spec, which governs the lifecycle of
Java component implementation instances. I think the definition and
implementation of lifecycle management will vary greatly depending on
the component implementation type, for example Java component
implementations and BPEL component implementations typically deal
with this in a very different way.
Well, I don't think that's the case at all and actually there is a
concept of implementation scope in assembly - it just varies by
implementation type, which is entirely consistent with our design.
BPEL is the odd case, and this came up as we wrote the scope changes
into the spec (Ken did a lot of the work here). Across many
implementation types, e.g. Groovy, JavaScript, Ruby, Jython, etc.
(maybe even C++) I see use for the same scopes as in Java. Do you
disagree?
Also, I'm curious why you think the scope containers complicate the
core and need to be moved out? Or are you saying this based on your
reading of the spec? They seem quite simple to me.
I'm saying that scope management is specific to the implementation type
and therefore needs to be made pluggable, i.e. moved out of core. The
Java scope management is just one example of scope management.
Therefore, in my view state/lifecycle management should be left to
the component implementation contributions and not belong to core.
I think this would lead to over-complication, particularly for the
extension developer. Right now, scope containers can be reused. In
particular, how would conversational services be handled? If I want to
use module or session scope containers for my Groovy script, then I'd
have to write those rather then just reuse what the core gives me?
Also, be reusing, we also allow an additional extension type in terms
of scope. For example, someone could add a distributed cache scope and
have that shared by Groovy, Java, whatever.
I'll also note two things. Getting scope containers to work properly
with instance tracking is not trivial. I'd hate to push that on
extension developers. Second, this basic design has been there since
before M1. Why wasn't this brought up before since it is such a
significant issue?
- Java and WSDL interfaces should be defined separate from the core
model, we need to support multiple interface definition languages
supported by plugins, not in the core.
The model supports generic IDL through the use of ServiceContract.
Java and WSDL are two forms of IDL that are mandated by the
specification. This is really just a question of where those
implementations are packaged and again I don't think this warrants a
rewrite.
Packaging issues are important and often hide bigger
dependency/coupling problems. I think we should package the support
for Java and WSDL interfaces separate from the core to prevent any
coupling between the two, and also give people who will have to
support new interface definition languages a better template to follow.
Individual issues do not warrant a rewrite. What about the sum of
many issues?
None of the issues warrant a rewrite, not even the sum. Most of your
criticisms seem centered around the model which is fairly decoupled
from the bulk of the core2 runtime. Even if we adopted your changes
wholesale, I'd doubt they would change the core2 runtime
significantly. Even the scope containers could be moved out without
breaking anything and very little code changes, although that would be
a mistake IMO. I'm sorry but I fail to see the need for a rewrite.
I am not proposing a rewrite of the whole runtime, see my original
email, it's not a whole rewrite. I'm proposing a staged / baby step
approach integrating the good work from M1 and the sandbox and new
discussions where I think what we have is not right or where new ideas
from the group come up.
I'm starting with the model SPI because I think that having the assembly
model right is critical for an assembly runtime. Most of the ideas here
have an impact on the architecture of the runtime, so I thought this was
a good starting point, and also a good base of discussion to help all in
our community discuss and understand better the new recursive
composition model.
- Implementation should extend ComponentType IMO instead of
pointing to it, and we may even be able to simplify and just remove
Implementation. Also I am not sure why we need to distinguish
between AtomicImplementation and CompositeImplementation.
One of the problems the assembly spec has is that it is difficult to
do top-down design because you cannot link a component to a
componentType without having an implementation. I agree this is an
area that we (and the spec) need to sort out.
IMO a component is associated with one componentType but may have
multiple implementations so I don't think they are quite the same
thing or that either can be removed.
AtomicImplementation is a marker for implementations that cannot
have children.
In my view a component has a type. The ComponentType is either
abstract (just defining the types of services offered, references
used, and properties that can be configured), or concrete. A POJO
component implementation is a concrete ComponentType.
Perhaps we could walk through your model?
Yes, the interfaces I put under m2-design are there to illustrate ideas
and support a design discussion. I'm working on some UML diagrams that I
think will help too.
- Support for Composite Includes is missing, this is a significant
part of the recursive composition model (half of it, one of the two
ways to nest composites).
It's not really half - it's really just a very small part of the
model, comparable to the <import> element we used to support in M1.
Again, I don't see why we need to rewrite the model to add this in.
Quite the opposite: you've said you've been looking for way to
engage and this could be it.
I disagree. Includes are a very significant part of the assembly
model (the other part is the ability to use a composite as a
component implementation). Two examples:
- An included composite is the equivalent of a module fragment in the
0.9 spec. This concept is key to allowing a team to work on various
pieces of an application, split in multiple composites, included in a
composite representing the application.
- When (formerly subsystems) composites get deployed to an SCA
system, they are actually included in that system, rather than being
used as component implementations.
It's not "half" of the recursive model. In fact, most of the time we
spent in the spec group was grappling with other issues related to
recursion.
I don't see an immediate relation between the time spent by the spec
group on a specific item and its importance for application developers.
I am looking at this from an application developer point of view and
saying that I'll use Includes as much as composition through (composite)
components. Includes will allow a team to distribute work on an SCA
application and also represent a key concept for system composition. I
am starting to look at scenarios and can actually see the usage of
Includes in almost all of them, but having trouble finding good use
cases for the other form of composition (nested component
implementations). So I stand by my statement that understanding how
includes work is key here.
This list is not exhaustive... Another idea would be to externalize
support for Composites in a separate plugin not part of the core
service model (since there may be other ways to compose services in
addition to an SCA composite, with Spring or other similar
programming models), I'd like to know what people think about that.
Having the composite implementation type in the core does not
preclude that - again, it's just packaging for ease-of-use.
I think it's more significant than packaging. Are you saying that we
could move the code supporting composites out of core2 without
breaking the code in core2?
Why would we do this? We can already support multiple composite
implementation types - have a look at the Spring extension. That just
sounds like unnecessary complication.
Why? to avoid unecessary and dangerous coupling that will hurt us when
we try to evolve this runtime. How to illustrate that? how about trying
to move code supporting composites out of core2? I'm realizing I'm
asking the same question again... but I think it's an important
question, sill unanswered.
You seem to have the impression that the core is sealed and that we
only support things that are included within it. That is not the
case. The only place we need things bundled with the core is in the
bootstrap phase - specifically, we need them bundled with the
primordial deployer. The actual runtime is created from the SCDL
passed to that primordial deployer, can contain any mix of
components and need not contain any of the infrastructure used to
boot it.
I just checked in sandbox/sebastien/m2-design/model.spi a set of
new interfaces. This is just an initial strawman to trigger a
constructive discussion and ideas on how to best represent the
recursive model. I also need help to define a scenario (not unit
test cases, but an end to end sample application) to help put the
recursive composition model in perspective and make sure we all
understand it the same way.
I am troubled that you have chosen to start on your own codebase at
a time when most of us have been trying to have constructive
discussion on this list. Based on the approach you proposed in your
original email I would have hoped that we could have started with
your end-user scenarios and had a chance to explore how they could
be supported by M1, the sandbox, or some other code before starting
another codebase. I'm disappointed that, having started this very
thread nearly a week ago with the premise of community, your first
response on it was to commit a large chunk of independent code
rather than follow up with any of the other people who have already
contributed to the discussion.
I think discussion led to compromise and consensus on the
scenario-driven approach that you proposed. As shown above and in
other recent threads, there's plenty of room for improvements and/or
new features in our current code and a willingness to discuss them,
albeit in terms of technical merit rather than personal opinion. I
hope you can find a way to join in rather than forge your own path.
--Jeremy
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Jean-Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
