Re: [PROPOSAL] New pax project 'transx'

[mit_jones]

Hi Guillaume,

So as to openly declare by bias I will state my interest in this
conversation. My company commissioned Paremus to provide a DS solution for
transaction control of JPA and JDBC resources which ultimately led to Aries
tx-control. 

Your proposal sounds great, it really does, however I will explain how we
came to the decision to comission Paremus which I am hoping may then sway
your decsion as to where it should be implemented.

The product we develop was based upon SpringDM which given the future (or
lack of it) forced us to look for an alternative, we decided to migrate to a
DS based solution but soon realised there was a lack of transactional
solutions for DS. At the time I was aware of the Everit transaction-helper
and a partially completed transactional solution for DS, that being the
aries-jpa JpaTemplate solution. I had some early conversations with
Christian about adding new functionality to the JpaTemplate solution such as
support for JDBC, not rolling back for specific exceptions, raised a few
bugs on JIRA but  wasn't confident that I could recomend it as a solution
given the lack of tests and most importantly no specification to back the
implementation.

The solution we now have is tx-control backed by RFC-221 with a
implementation that covers the extra functionality we asked for such as last
gambit wins and has 2500+ lines of test code. 

Currently if one was to embark upon adopting OSGi we are faced with a
multitude of choices Blueprint/DS/CDI/iPOJO ... and solutions provided by
Aries/Pax/Enroute/Everit/Amdatu ... For you and perhaps many readers the
choices are obvious but in my opinion it is not obvious and even more
difficult to access the quality of the solutions. One thing that is
indisputable is whether a solution is backed by a specification and
therefore necessarily has controls to enforce a certain level of compliance
=> quality, this in my opinion is very important, although no guarantee it
is probably the best assurance an adopter can have that the
component/framework they are using will survive longer term.

I have followed the Aries/Karaf/Pax forums closely enough over the past few
years to see that there is tension between groups who belong and don't
belong to the OSGi Alliance. I think this is really unfortunate because it
has led and is continuing to led to fragmentation in the OSGi community as a
whole which manifests itself in multiple solutions/options making it really
difficult for an adopter of the technology to know which to choose.


To add JMS support into the tx-control project would add to an already well
designed solution, that already has good documentation, test support and not
lead to yet more fragmentation. I presume the specification could be added
to as needed if required.

Unfortunately my company does not have an immediate need for transactional
JMS otherwise I would be able offer more bribery in the form of $$ :)

Tim J.





--
View this message in context: 
http://aries.15396.n3.nabble.com/Re-PROPOSAL-New-pax-project-transx-tp4035366p4035371.html
Sent from the Aries - Dev mailing list archive at Nabble.com.

Re: [PROPOSAL] New pax project 'transx'

[Timothy Ward]

On 20 Jun 2017, at 15:28, Guillaume Nodet 
> wrote:

2017-06-20 12:53 GMT+02:00 Timothy Ward 
>:

Hi Guillaume,

The OSGi Alliance is an open organisation, and a number of OPS4j
developers are already members via their companies. There is absolutely
nothing preventing them from getting involved with the Alliance, nor
preventing any non-members from joining.

On the other hand to maintain the openness of its standards the OSGi
Alliance must have a strict IP policy, one that prevents it from consuming
arbitrary code or IP from external sources. If OPS4j are able to get to a
compatible place contribution-wise then I'm sure you'd see a flow of work
in the other direction too.

As for Aries Tx Control - a Narayana based XA implementation would be a
great addition, as would JMS support.


I agree, I may look at it in the future, but that would be easily based on
what I'm proposing here.  Aries tx-control does not necessarily have to
host the pooling code, but rather the rfc 220 integration code imho.



Wrapping the Geronimo transaction manager is deliberate for three reasons:

* the javax.transaction package is toxic due to its split package in the
JRE. Hiding all of the JTA code allows the impl to work without system
packages being declared when launching the OSGi framework.


That’s not specific to the Geronimo TM afaik.

This is not specific to the Geronimo TM, but it is a reason that wrapping a TM 
is preferable to consuming one from another bundle. Wrapping lets the JTA 
package usage be purely internal, and avoids the toxic class space issues.




* by being Geronimo specific the implementation can offer last participant
support


I don't think that's true either.  Geronimo TM itself offers no support for
enlisting local resources.  What tx-control does is wrap local resources
with the  LocalXAResourceImpl and just expect everything will be ok.   The
TM should at list make sure that such wrapped local resources should be
called last in the prepare phase.  Afaik, that's not the case with the
Geronimo TM.  I think the current code should work as is with other TM, or
better of some can offer real support for this use case.
I think Narayana simply requires the XAResource to implement a specific
interface Last in order to be called last in the prepare phase and lessen
the possibilities of something bad happening.


The Aries TX control implementation wraps the resource and adds it to the last 
place in the resource list. It does this safe in the knowledge that Geronimo 
calls resources in a FIFO order when preparing. This is not required to be true 
for other implementations (which may optimise their calls in different ways), 
and so requires knowledge of the specific implementation logic. Similarly, 
implementing a Narayana interface requires you to know that the implementation 
will pay attention to the interface, and cannot be done speculatively.



* by being Geronimo specific the implementation can support XA recovery


Yes, it's really unfortunate that the JTA spec has not covered this part.
I'm wondering if we there's a place for a small project which would offer
an api and wrappers around existing TM so that they could be switched if
needed, and more importantly, so that code can access those non standard
features without dealing with the specifics.
I may try working on this part next, then maybe integrate both into
tx-control.

I think that this would need to be custom per-provider, but a Narayana 
implementation would definitely be useful.



This model gives a great level of functionality in an easy to access way
for users, and I would be keen to keep this option. A pluggable model is
possible, but would need to be done carefully to ensure that scopes could
cope with external parties "messing with" the transaction. It would also
lose the benefits described above, although neither of these things mean
that it would not be worth adding as an alternative implementation.

Finally - I am not sure why tx Control would have a dependency on pax jdbc
(other than as a source of DataSourceFactory services)? This sounds like it
would make the Aries project harder to configure and deploy, and I can't
immediately see what additional benefits it would provide. Can you clarify?


From a high level, pax-jdbc aims at providing DataSourceFactory while
tx-control aims at integrating those into the transaction api. So it could
make sense to layer them.  I haven’t looked at the specifics though...

I think that this layering already exists. Right now the Tx Control JDBC and 
JPA providers expect to find and make use of a standard DataSourceFactory 
service.

Regards,

Tim


Guillaume



Regards,

Tim

Sent from my iPhone

On 20 Jun 2017, at 11:00, Guillaume Nodet 
> wrote:

2017-06-16 11:16 GMT+02:00 Richard Nicholson 

Re: [PROPOSAL] New pax project 'transx'

[Timothy Ward]

Hi Achim,

I am certain that that this is a conversation that has been had before, and 
that it would be better for any revisit of this discussion to be held between 
OPS4j and the Alliance rather than on the Karaf/Aries dev lists. I am also not 
an OSGi board representative, nor am I corporate officer of the OSGi Alliance, 
so I can’t speak on their behalf. Finally, I wasn’t part of any previous 
discussion between OPS4j and the OSGi Alliance about accepting implementations 
from OPS4j, so I do not know what any specific sticking points might have been.

I do know that in order for an “external” community to contribute reference 
implementations to the OSGi Alliance (which seems to be what we’re talking 
about here) there are rules about acceptable Open Source licences, levels of 
community diversity, legal IP governance and guarantees of originality for the 
code. There are probably other important requirements that I am not aware of. I 
know that Apache and Eclipse are examples of acceptable external communities 
which are regularly used to provide reference implementations, and that OPS4j 
is currently not on that list.

There may be very little for OPS4j to do to become another such community, or 
there may be some thornier problems that would need to be solved before that 
could be the case. If you want anything more specific I strongly recommend 
contacting the OSGi Alliance, either through a board member or the CTO. See 
https://www.osgi.org/about-us/board-officers/ for contact details.

Best Regards,

Tim


On 20 Jun 2017, at 12:12, Achim Nierbeck 
> wrote:

Hi Tim,

could you please elaborate on this a bit more?

On the other hand to maintain the openness of its standards the OSGi
Alliance must have a strict IP policy, one that prevents it from consuming
arbitrary code or IP from external sources. If OPS4j are able to get to a
compatible place contribution-wise then I'm sure you'd see a flow of work
in the other direction too.


especially the "If OPS4j are able to get to a compatible place
contribution-wise"

what in your view is missing for the OPS4j community to be regarded a
compatible place?

regards, Achim



2017-06-20 12:53 GMT+02:00 Timothy Ward 
>:

Hi Guillaume,

The OSGi Alliance is an open organisation, and a number of OPS4j
developers are already members via their companies. There is absolutely
nothing preventing them from getting involved with the Alliance, nor
preventing any non-members from joining.

On the other hand to maintain the openness of its standards the OSGi
Alliance must have a strict IP policy, one that prevents it from consuming
arbitrary code or IP from external sources. If OPS4j are able to get to a
compatible place contribution-wise then I'm sure you'd see a flow of work
in the other direction too.

As for Aries Tx Control - a Narayana based XA implementation would be a
great addition, as would JMS support.

Wrapping the Geronimo transaction manager is deliberate for three reasons:

* the javax.transaction package is toxic due to its split package in the
JRE. Hiding all of the JTA code allows the impl to work without system
packages being declared when launching the OSGi framework.

* by being Geronimo specific the implementation can offer last participant
support

* by being Geronimo specific the implementation can support XA recovery

This model gives a great level of functionality in an easy to access way
for users, and I would be keen to keep this option. A pluggable model is
possible, but would need to be done carefully to ensure that scopes could
cope with external parties "messing with" the transaction. It would also
lose the benefits described above, although neither of these things mean
that it would not be worth adding as an alternative implementation.

Finally - I am not sure why tx Control would have a dependency on pax jdbc
(other than as a source of DataSourceFactory services)? This sounds like it
would make the Aries project harder to configure and deploy, and I can't
immediately see what additional benefits it would provide. Can you clarify?

Regards,

Tim

Sent from my iPhone

On 20 Jun 2017, at 11:00, Guillaume Nodet 
> wrote:

2017-06-16 11:16 GMT+02:00 Richard Nicholson 
>:


Doesn’t this directly clash with OSGi Alliance Transaction Control
Specification work going on in Aries?

If so, wouldn’t it make more sense for this community to input into that
work rather than cause needless confusion / fragmentation?


Just a thought.


Yeah, I'm a bit skeptic about the relationship between the OPS4j
community
and the OSGi Alliance work.  It seems to always go in the same
direction...
i.e. the guys working at OPS4j should help working on the project defined
by the guys working at the OSGi Alliance.

That being said, the work in Aries is 

Re: [PROPOSAL] New pax project 'transx'

[Achim Nierbeck]

Hi Tim,

could you please elaborate on this a bit more?

On the other hand to maintain the openness of its standards the OSGi
> Alliance must have a strict IP policy, one that prevents it from consuming
> arbitrary code or IP from external sources. If OPS4j are able to get to a
> compatible place contribution-wise then I'm sure you'd see a flow of work
> in the other direction too.


especially the "If OPS4j are able to get to a compatible place
contribution-wise"

what in your view is missing for the OPS4j community to be regarded a
compatible place?

regards, Achim



2017-06-20 12:53 GMT+02:00 Timothy Ward :

> Hi Guillaume,
>
> The OSGi Alliance is an open organisation, and a number of OPS4j
> developers are already members via their companies. There is absolutely
> nothing preventing them from getting involved with the Alliance, nor
> preventing any non-members from joining.
>
> On the other hand to maintain the openness of its standards the OSGi
> Alliance must have a strict IP policy, one that prevents it from consuming
> arbitrary code or IP from external sources. If OPS4j are able to get to a
> compatible place contribution-wise then I'm sure you'd see a flow of work
> in the other direction too.
>
> As for Aries Tx Control - a Narayana based XA implementation would be a
> great addition, as would JMS support.
>
> Wrapping the Geronimo transaction manager is deliberate for three reasons:
>
> * the javax.transaction package is toxic due to its split package in the
> JRE. Hiding all of the JTA code allows the impl to work without system
> packages being declared when launching the OSGi framework.
>
> * by being Geronimo specific the implementation can offer last participant
> support
>
> * by being Geronimo specific the implementation can support XA recovery
>
> This model gives a great level of functionality in an easy to access way
> for users, and I would be keen to keep this option. A pluggable model is
> possible, but would need to be done carefully to ensure that scopes could
> cope with external parties "messing with" the transaction. It would also
> lose the benefits described above, although neither of these things mean
> that it would not be worth adding as an alternative implementation.
>
> Finally - I am not sure why tx Control would have a dependency on pax jdbc
> (other than as a source of DataSourceFactory services)? This sounds like it
> would make the Aries project harder to configure and deploy, and I can't
> immediately see what additional benefits it would provide. Can you clarify?
>
> Regards,
>
> Tim
>
> Sent from my iPhone
>
> > On 20 Jun 2017, at 11:00, Guillaume Nodet  wrote:
> >
> > 2017-06-16 11:16 GMT+02:00 Richard Nicholson :
> >
> >>
> >> Doesn’t this directly clash with OSGi Alliance Transaction Control
> >> Specification work going on in Aries?
> >>
> >> If so, wouldn’t it make more sense for this community to input into that
> >> work rather than cause needless confusion / fragmentation?
> >
> >
> >> Just a thought.
> >>
> >
> > Yeah, I'm a bit skeptic about the relationship between the OPS4j
> community
> > and the OSGi Alliance work.  It seems to always go in the same
> direction...
> > i.e. the guys working at OPS4j should help working on the project defined
> > by the guys working at the OSGi Alliance.
> >
> > That being said, the work in Aries is about defining a new programming
> > model for transactions.  That's something I'm not really interested in at
> > this point.  In addition, my initial goal is to have support for JMS +
> > Narayana and both aspects are not covered.  In particular, it does create
> > and wrap the geronimo TransactionManager instead of re-using an external
> > one (even the one defined in Aries Transaction for example).
> >
> > In theory, things should be layered.  For example, pax-jdbc provides a
> way
> > to expose DataSourceFactory objects in the OSGi registry.Imho,
> pooling
> > should be done at this level, as specified in the DataSourceFactory
> > interface.  So pooling inside aries-tx-control is irrelevant.
> >
> > This project is even at a lower level and I plan to integrate it below
> > pax-jdbc for the jdbc part.
> >
> > That said, I may have a look at aries-tx-control and see if I can replace
> > some of the code there to leverage pax-jdbc and pax-transx more to help
> > avoiding confusion and fragmentation.
> >
> >
> >>
> >>> On 15 Jun 2017, at 13:55, Toni Menzel  wrote:
> >>>
> >>> Sounds interesting!
> >>> Two comments:
> >>>
> >>>  - i find the whole space of "pooling resources" a not confusing and
> >> hard
> >>>  to find out what you actually really need. So, say once you know you
> >> want
> >>>  takaricp, which other bridges and matching configs do you need so that
> >> the
> >>>  DataSource proxy (for JDBC) appears in your Service Registry. Maybe
> >> it's
> >>>  just me not following bridge provider-projects like Aries 

Re: [PROPOSAL] New pax project 'transx'

[Timothy Ward]

Hi Christian

The Transaction Control API has no OSGi framework dependency, so a Java SE mode 
of operation would be possible (just like Aries blueprint no osgi). Possibly 
something worth exploring?

Tim

Sent from my iPhone

> On 20 Jun 2017, at 11:35, Christian Schneider  wrote:
> 
> Aries transaction control has a lot of good parts. Unfortunately though it is 
> an all or nothing solution.
> It works well if you directly depend on Aries transaction control in your 
> user code. Unfortunately this makes the code OSGi only. So this is not an 
> option for many projects like CXF or Camel.
> 
> The benefit of pax-jdbc-config and pax-jdbc-pool* ist that it provides a XA 
> ready DataSource that can be leveraged by code that is OSGi agnostic.
> 
> So I think there is a need for the same in the JMS space. So why not focus on 
> jms only and call the project pax-jms? In fact there already is such a 
> project that we could build on and extend for XA features.
> 
> Christian
> 
> 
>> On 16.06.2017 11:29, Timothy Ward wrote:
>> The Transaction Control project in Aries does have a pretty complete overlap 
>> with what’s being proposed here. There are already resource providers for 
>> JPA and JDBC which provide connection pooling, resource local transactions 
>> and XA transactions. It would be great to get some input into a JMS resource 
>> provider to extend the set of supported resource types.
>> 
>> The Aries TX control code already includes a base resource project for 
>> adding Resource providers which should help to ensure correct lifecycle 
>> management - I’d be happy to talk through that code in more detail. 
>> Contributing JMS support should therefore be a (relatively) simple process 
>> of providing the necessary JMS customisation much like with JDBC.
>> 
>> Happy to help,
>> 
>> Tim
>> 
>> 
>>> On 16 Jun 2017, at 10:20, Grzegorz Grzybek  wrote:
>>> 
>>> +1
>>> 
>>> Great about forking tranql - finally ;)
>>> 
>>> regards
>>> Grzegorz
>>> 
>>> 2017-06-16 11:16 GMT+02:00 Richard Nicholson :
>>> 
 Doesn’t this directly clash with OSGi Alliance Transaction Control
 Specification work going on in Aries?
 
 If so, wouldn’t it make more sense for this community to input into that
 work rather than cause needless confusion / fragmentation?
 
 Just a thought.
 
 
> On 15 Jun 2017, at 13:55, Toni Menzel  wrote:
> 
> Sounds interesting!
> Two comments:
> 
>  - i find the whole space of "pooling resources" a not confusing and
 hard
>  to find out what you actually really need. So, say once you know you
 want
>  takaricp, which other bridges and matching configs do you need so that
 the
>  DataSource proxy (for JDBC) appears in your Service Registry. Maybe
 it's
>  just me not following bridge provider-projects like Aries too closely.
>  Anything that makes setup simpler and offers a wider range of options
 is
>  highly welcome. (particularly in the OPS4J community, or how Bndtools
>  people say "P A X" ;)
>  - Any reason why this is not Pax Tx (org.ops4j.pax.tx) ?Find the
>  Transx a bit alien. just an idea.
> 
> Thanks for your heads up, JB about karaf-boot. Was wondering what
 happened
> to it.
> 
> Toni
> 
> 
> On Thu, Jun 15, 2017 at 1:58 PM, Achim Nierbeck  
> wrote:
> 
>> Hi Guillaume,
>> 
>> sounds like a good idea to me, and the pax space like the perfect eco
>> system :)
>> 
>> regards, Achim
>> 
>> 2017-06-15 10:20 GMT+02:00 Jean-Baptiste Onofré :
>> 
>>> +1
>>> 
>>> It sounds like a good idea and definitely a good candidate for PAX.
>>> 
>>> By the way, on my side, I did good progress on:
>>> - karaf sample & new dev guide
>>> - some new updates on karaf-boot
>>> - ServiceMix APIMan for API/Service Discovery, Management, Gateway
>>> But I will send an update in separate threads.
>>> 
>>> Regards
>>> JB
>>> 
>>> 
 On 06/15/2017 09:57 AM, Guillaume Nodet wrote:
 
 I began to work on a small project which aims at providing support for
 pooled XA-enabled connections for JDBC and JMS.
 
 For JDBC, the problem was already solved in pax-jdbc by using either
 pax-jdbc-pool-aries when deploying the Aries/Geronimo transaction
>> manager,
 and by using pax-jdbc-pool-narayana when using the Narayana
 transaction
 manager.
 
 However, there's absolutely no support for JMS.
 
 So what I've been doing is to reuse the geronimo JCA connector, make
 it
 independent on Geronimo TM and add support for Narayana, use a clone
 of
 the
 old tranql adapter for JDBC and rewrite a new JMS 2.0 

Re: [PROPOSAL] New pax project 'transx'

[Guillaume Nodet]

2017-06-16 11:16 GMT+02:00 Richard Nicholson :

>
> Doesn’t this directly clash with OSGi Alliance Transaction Control
> Specification work going on in Aries?
>
> If so, wouldn’t it make more sense for this community to input into that
> work rather than cause needless confusion / fragmentation?


> Just a thought.
>

Yeah, I'm a bit skeptic about the relationship between the OPS4j community
and the OSGi Alliance work.  It seems to always go in the same direction...
i.e. the guys working at OPS4j should help working on the project defined
by the guys working at the OSGi Alliance.

That being said, the work in Aries is about defining a new programming
model for transactions.  That's something I'm not really interested in at
this point.  In addition, my initial goal is to have support for JMS +
Narayana and both aspects are not covered.  In particular, it does create
and wrap the geronimo TransactionManager instead of re-using an external
one (even the one defined in Aries Transaction for example).

In theory, things should be layered.  For example, pax-jdbc provides a way
to expose DataSourceFactory objects in the OSGi registry.Imho, pooling
should be done at this level, as specified in the DataSourceFactory
interface.  So pooling inside aries-tx-control is irrelevant.

This project is even at a lower level and I plan to integrate it below
pax-jdbc for the jdbc part.

That said, I may have a look at aries-tx-control and see if I can replace
some of the code there to leverage pax-jdbc and pax-transx more to help
avoiding confusion and fragmentation.


>
> > On 15 Jun 2017, at 13:55, Toni Menzel  wrote:
> >
> > Sounds interesting!
> > Two comments:
> >
> >   - i find the whole space of "pooling resources" a not confusing and
> hard
> >   to find out what you actually really need. So, say once you know you
> want
> >   takaricp, which other bridges and matching configs do you need so that
> the
> >   DataSource proxy (for JDBC) appears in your Service Registry. Maybe
> it's
> >   just me not following bridge provider-projects like Aries too closely.
> >   Anything that makes setup simpler and offers a wider range of options
> is
> >   highly welcome. (particularly in the OPS4J community, or how Bndtools
> >   people say "P A X" ;)
> >   - Any reason why this is not Pax Tx (org.ops4j.pax.tx) ?Find the
> >   Transx a bit alien. just an idea.
> >
> > Thanks for your heads up, JB about karaf-boot. Was wondering what
> happened
> > to it.
> >
> > Toni
> >
> >
> > On Thu, Jun 15, 2017 at 1:58 PM, Achim Nierbeck  >
> > wrote:
> >
> >> Hi Guillaume,
> >>
> >> sounds like a good idea to me, and the pax space like the perfect eco
> >> system :)
> >>
> >> regards, Achim
> >>
> >> 2017-06-15 10:20 GMT+02:00 Jean-Baptiste Onofré :
> >>
> >>> +1
> >>>
> >>> It sounds like a good idea and definitely a good candidate for PAX.
> >>>
> >>> By the way, on my side, I did good progress on:
> >>> - karaf sample & new dev guide
> >>> - some new updates on karaf-boot
> >>> - ServiceMix APIMan for API/Service Discovery, Management, Gateway
> >>> But I will send an update in separate threads.
> >>>
> >>> Regards
> >>> JB
> >>>
> >>>
> >>> On 06/15/2017 09:57 AM, Guillaume Nodet wrote:
> >>>
>  I began to work on a small project which aims at providing support for
>  pooled XA-enabled connections for JDBC and JMS.
> 
>  For JDBC, the problem was already solved in pax-jdbc by using either
>  pax-jdbc-pool-aries when deploying the Aries/Geronimo transaction
> >> manager,
>  and by using pax-jdbc-pool-narayana when using the Narayana
> transaction
>  manager.
> 
>  However, there's absolutely no support for JMS.
> 
>  So what I've been doing is to reuse the geronimo JCA connector, make
> it
>  independent on Geronimo TM and add support for Narayana, use a clone
> of
>  the
>  old tranql adapter for JDBC and rewrite a new JMS 2.0 compatible
> adapter
>  for JMS.
> 
>  It's not in a usable state yet, but I wanted to give an heads-up.
>  My plan is to make the pooling almost transparent in OSGi, and reuse
> it
>  instead of the connection pooling I added to Karaf a few weeks ago
> which
>  does not support XA or recovery:
>    https://github.com/apache/karaf/tree/master/jms/pool
>  and maybe to plug it into pax-jdbc to replace pax-jdbc-pool-aries and
>  pax-jdbc-pool-narayana.
> 
>  The source code is currently available at:
>    https://github.com/gnodet/org.ops4j.pax.transx
> 
> 
>  Cheers,
> 
> 
> >>> --
> >>> Jean-Baptiste Onofré
> >>> jbono...@apache.org
> >>> http://blog.nanthrax.net
> >>> Talend - http://www.talend.com
> >>>
> >>
> >>
> >>
> >> --
> >>
> >> Apache Member
> >> Apache Karaf  Committer & PMC
> >> OPS4J Pax Web 
> Committer &
> >>