Re: Features resolver investigation for 4.3.0 and 4.2.11

2020-10-16 Thread Romain Manni-Bucau 
Thought that as well but camel, cxf, jaxrs whiteboard etc are other
examples where order is importantbut the symbolic names should be
sufficient to guarantee a good order in most cases too

Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book



Le ven. 16 oct. 2020 à 10:24, Jean-Baptiste Onofre  a
écrit :

> Yeah, agree that it’s weird to not have seen before ;)
>
> My guess is that people maybe have more "splitter/dynamic" features than
> "our" scenario (even if it’s a valid one) and also have super fast machine.
> Here, the application "imposes" a startup order which is not the case in
> pure OSGi (by design, it’s dynamic, so it can react later).
>
> So, I’m pretty sure, it’s not really a problem with "dynamic ready"
> modules. Here’s we identified the issue due to SpiFly and CDI.
>
> Don’t get my wrong, it should be improved, but not a big deal for OSGi
> designed module.
>
> Regards
> JB
>
> > Le 16 oct. 2020 à 07:30, Romain Manni-Bucau  a
> écrit :
> >
> > To try to help on the different points you mention:
> >
> > 1.a prerequisite is good when  you have two stagings - order to respect
> to
> > install a full feature - so it is a very rare and limited case (in my
> app I
> > have 5 for ex)
> > 1.b requirements doesn't help with the order (until a framework
> extension),
> > just with "start" which fails so you can still have a bundle starting and
> > not having some dependency so at the end order is still required
> > 2. the comparator is not sufficient by itself, all the Set and Map are
> not
> > sorted in karaf featureservice/felix resolver so they also break the
> order
> >
> > So at the end I think the order is a requirement of the features - but
> > still amazed it didnt pop up before.
> >
> > Romain Manni-Bucau
> > @rmannibucau  |  Blog
> >  | Old Blog
> >  | Github <
> https://github.com/rmannibucau> |
> > LinkedIn  | Book
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> >
> >
> > Le ven. 16 oct. 2020 à 05:59, Jean-Baptiste Onofre  a
> > écrit :
> >
> >> The comparator based on symbolic name is to have something deterministic
> >> but doesn’t necessary match.
> >> I would like to check If using requirement in a feature would help (I’m
> >> afraid it would block the installation but not guarantee the order
> though).
> >>
> >> So, I think it’s worth to evaluate/improve the comparator to have
> >> something more "user logic".
> >>
> >> I will create a Jira and check different scenario.
> >>
> >> Regards
> >> JB
> >>
> >>> Le 15 oct. 2020 à 17:43, Łukasz Dywicki  a écrit
> :
> >>>
> >>> What you say makes a lot of sense. I was just refering to behavior
> which
> >>> I think is there since new resolver come into play.
> >>> Making it a bit more deterministic is definitely fair (yet expensive in
> >>> debugging) point to take.
> >>>
> >>> Cheers,
> >>> Łukasz
> >>>
> >>>
> >>> On 15.10.2020 17:32, Romain Manni-Bucau wrote:
>  Fact is if features are not installed in reversed graph order it is
> not
>  really reliable an you can't install an app and be sure it will work,
> >> even
>  if you sorted manually the bundles.
>  The ResourceComparator sorts by symbolic names which makes the
> >> deployment
>  deterministic but potentially random from an user perspective
>  (uncontrollable).
>  Then the featureservice+resolve use a list of hashmap (or hashset)
> which
>  also randomizes the installation.
>  Guess using at least a reversed dijkstra distance (or tree deepness in
> >> some
>  simple cases) from the feature to sort bundles installation order is
> >> worth
>  it to make it more natural and controlled for end users.
> 
>  Hope it makes sense.
> 
>  Romain Manni-Bucau
>  @rmannibucau  |  Blog
>   | Old Blog
>   | Github <
> >> https://github.com/rmannibucau> |
>  LinkedIn  | Book
>  <
> >>
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >>>
> 
> 
>  Le jeu. 15 oct. 2020 à 17:28, Łukasz Dywicki  a
> >> écrit :
> 
> > I am not sure if that's feature resolver issue. It is rather related
> to
> > bundle resolver which determines installation order.
> > As far I remember from my last debugging session in this area bundles
> > are installed in computed "dependency" order. It might be that leaf/1
> > has no direct dependencies on any of its children hence

Re: Features resolver investigation for 4.3.0 and 4.2.11

2020-10-16 Thread Jean-Baptiste Onofre 
Yeah, agree that it’s weird to not have seen before ;)

My guess is that people maybe have more "splitter/dynamic" features than "our" 
scenario (even if it’s a valid one) and also have super fast machine.
Here, the application "imposes" a startup order which is not the case in pure 
OSGi (by design, it’s dynamic, so it can react later).

So, I’m pretty sure, it’s not really a problem with "dynamic ready" modules. 
Here’s we identified the issue due to SpiFly and CDI.

Don’t get my wrong, it should be improved, but not a big deal for OSGi designed 
module.

Regards
JB

> Le 16 oct. 2020 à 07:30, Romain Manni-Bucau  a écrit :
> 
> To try to help on the different points you mention:
> 
> 1.a prerequisite is good when  you have two stagings - order to respect to
> install a full feature - so it is a very rare and limited case (in my app I
> have 5 for ex)
> 1.b requirements doesn't help with the order (until a framework extension),
> just with "start" which fails so you can still have a bundle starting and
> not having some dependency so at the end order is still required
> 2. the comparator is not sufficient by itself, all the Set and Map are not
> sorted in karaf featureservice/felix resolver so they also break the order
> 
> So at the end I think the order is a requirement of the features - but
> still amazed it didnt pop up before.
> 
> Romain Manni-Bucau
> @rmannibucau  |  Blog
>  | Old Blog
>  | Github  |
> LinkedIn  | Book
> 
> 
> 
> Le ven. 16 oct. 2020 à 05:59, Jean-Baptiste Onofre  a
> écrit :
> 
>> The comparator based on symbolic name is to have something deterministic
>> but doesn’t necessary match.
>> I would like to check If using requirement in a feature would help (I’m
>> afraid it would block the installation but not guarantee the order though).
>> 
>> So, I think it’s worth to evaluate/improve the comparator to have
>> something more "user logic".
>> 
>> I will create a Jira and check different scenario.
>> 
>> Regards
>> JB
>> 
>>> Le 15 oct. 2020 à 17:43, Łukasz Dywicki  a écrit :
>>> 
>>> What you say makes a lot of sense. I was just refering to behavior which
>>> I think is there since new resolver come into play.
>>> Making it a bit more deterministic is definitely fair (yet expensive in
>>> debugging) point to take.
>>> 
>>> Cheers,
>>> Łukasz
>>> 
>>> 
>>> On 15.10.2020 17:32, Romain Manni-Bucau wrote:
 Fact is if features are not installed in reversed graph order it is not
 really reliable an you can't install an app and be sure it will work,
>> even
 if you sorted manually the bundles.
 The ResourceComparator sorts by symbolic names which makes the
>> deployment
 deterministic but potentially random from an user perspective
 (uncontrollable).
 Then the featureservice+resolve use a list of hashmap (or hashset) which
 also randomizes the installation.
 Guess using at least a reversed dijkstra distance (or tree deepness in
>> some
 simple cases) from the feature to sort bundles installation order is
>> worth
 it to make it more natural and controlled for end users.
 
 Hope it makes sense.
 
 Romain Manni-Bucau
 @rmannibucau  |  Blog
  | Old Blog
  | Github <
>> https://github.com/rmannibucau> |
 LinkedIn  | Book
 <
>> https://www.packtpub.com/application-development/java-ee-8-high-performance
>>> 
 
 
 Le jeu. 15 oct. 2020 à 17:28, Łukasz Dywicki  a
>> écrit :
 
> I am not sure if that's feature resolver issue. It is rather related to
> bundle resolver which determines installation order.
> As far I remember from my last debugging session in this area bundles
> are installed in computed "dependency" order. It might be that leaf/1
> has no direct dependencies on any of its children hence it is installed
> first.
> Note, that I don't know details on how present feature resolution works
> and if its just supplier for graph elements or separate graph computed
> by same algorithm as bundles.
> 
> Best,
> Łukasz
> 
> 
> On 15.10.2020 15:47, Jean-Baptiste Onofre wrote:
>> Hi guys,
>> 
>> Romain found an issue on the feature resolver about the features
> ordering.
>> 
>> For instance, if we have the following features descriptor:
>> 
>> 
>>   
>>   mvn:foo/nested1/2
>>   
>>   
>>   mvn:foo/nested21/3
>>   
>>   
>>   nested21
>>   mvn:foo/nested2/2
>>   
>>   
>>   nested1
>>   nested2
>>   mvn:foo/leaf/1
>>   
>> 
>>