Re: [cross-project-issues-dev] Building SimRel with Tycho for 2021-03

[Mickael Istria]

On Tue, Nov 10, 2020 at 5:21 PM Jonah Graham  wrote:

> - The gerrit today only has a subset of the parts of CDT that are in
> simrel.aggr/cdt.aggrcom (the only contribution I looked at closely). AFAICT
> only the items that are categorized are present in category.xml. Where do
> uncategorized features end up? Similar question about uncategorized
> bundles. I could add them all to a new category if needed.
>

This is now fixed.
___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To unsubscribe from this list, visit 
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev

Re: [cross-project-issues-dev] Building SimRel with Tycho for 2021-03

[Mickael Istria]

On Wed, Nov 11, 2020 at 10:06 AM Christoph Läubrich 
wrote:

> So it does not really fix anything in the file itself?


It's better if it modifies the file directly IMO.


> Currently, if tycho can't resolve it simply fails with: "can't resolve
> because missing requirement XYZ and so on"
> It would be really useful (and might solve the issue for simrel also) if
> we can add some pice of code that emits if there is an alternative
> version or no version at all for the missing requirement, eclipse (or
> P2?) even computes "alternative solutions", maybe this feature can be
> reused to give the required hints.


p2 already has such features to compute remediations; but it's long to
process and requires some user decision. IMO it does not fit well in what
we expect as a build tool.
___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To unsubscribe from this list, visit 
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev

Re: [cross-project-issues-dev] Building SimRel with Tycho for 2021-03

[Christoph Läubrich]

> Fix version will fix the version of the features in category.xml
> according to p2 repos. Eg for each feature in category.xml, it would
> verify the feature for the given version is available in the p2 repos
> (target platform), and if not, it would suggest a change to make.

So it does not really fix anything in the file itself? I could then 
think about an addition to tycho that might be useful in general as well:


Currently, if tycho can't resolve it simply fails with: "can't resolve 
because missing requirement XYZ and so on"


It would be really useful (and might solve the issue for simrel also) if 
we can add some pice of code that emits if there is an alternative 
version or no version at all for the missing requirement, eclipse (or 
P2?) even computes "alternative solutions", maybe this feature can be 
reused to give the required hints.


The the simrel validation will fail with a message suggesting the right 
versions to use in category.xml



Am 11.11.20 um 09:29 schrieb Mickael Istria:



On Wed, Nov 11, 2020 at 9:00 AM Christoph Läubrich 
mailto:lae...@laeubi-soft.de>> wrote:


As I'm currently working on category.xml adding support for source
features/bundles if someone can explain what exactly this "fix
versions"
does I can investigate if it is possible to be added to tycho.


Fix version will fix the version of the features in category.xml 
according to p2 repos. Eg for each feature in category.xml, it would 
verify the feature for the given version is available in the p2 repos 
(target platform), and if not, it would suggest a change to make.
This is similar and a slight variation of 
https://github.com/jbosstools/jbosstools-maven-plugins/blob/master/tycho-plugins/target-platform-utils/src/main/java/org/jboss/tools/tycho/targets/FixVersionsMojo.java 
 
which enables some similar feature for .target file (however it also 
replaced 0.0.0, which is not desired here); so it's technically totally 
doable.


Tycho has already a version-updater for bundles/features so one for
category.xml seems valid for me.


It's a bit different, as it's about fixing the content of the file. Not 
sure we're talking about the same thing here.


Another area I'm currently investigating is creating an update-site of
everything in the current reactor build so category.xml is generated
automatically, not sure if this is applicable/useful for simrel as well.


Nope, SimRel just takes as input the p2 repos and a category.xml to 
create a new p2 repo aggregating content. There is nothing else built in 
the same reactor.


___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To unsubscribe from this list, visit 
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev


___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To unsubscribe from this list, visit 
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev

Re: [cross-project-issues-dev] Building SimRel with Tycho for 2021-03

[Mickael Istria]

On Wed, Nov 11, 2020 at 9:00 AM Christoph Läubrich 
wrote:

> As I'm currently working on category.xml adding support for source
> features/bundles if someone can explain what exactly this "fix versions"
> does I can investigate if it is possible to be added to tycho.
>

Fix version will fix the version of the features in category.xml according
to p2 repos. Eg for each feature in category.xml, it would verify the
feature for the given version is available in the p2 repos (target
platform), and if not, it would suggest a change to make.
This is similar and a slight variation of
https://github.com/jbosstools/jbosstools-maven-plugins/blob/master/tycho-plugins/target-platform-utils/src/main/java/org/jboss/tools/tycho/targets/FixVersionsMojo.java
which enables some similar feature for .target file (however it also
replaced 0.0.0, which is not desired here); so it's technically totally
doable.

Tycho has already a version-updater for bundles/features so one for
> category.xml seems valid for me.
>

It's a bit different, as it's about fixing the content of the file. Not
sure we're talking about the same thing here.

Another area I'm currently investigating is creating an update-site of
> everything in the current reactor build so category.xml is generated
> automatically, not sure if this is applicable/useful for simrel as well.
>

Nope, SimRel just takes as input the p2 repos and a category.xml to create
a new p2 repo aggregating content. There is nothing else built in the same
reactor.
___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To unsubscribe from this list, visit 
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev

Re: [cross-project-issues-dev] Building SimRel with Tycho for 2021-03

[Christoph Läubrich]

> There is no such tool for Tycho/category.xml

As I'm currently working on category.xml adding support for source 
features/bundles if someone can explain what exactly this "fix versions" 
does I can investigate if it is possible to be added to tycho.


Tycho has already a version-updater for bundles/features so one for 
category.xml seems valid for me.


Another area I'm currently investigating is creating an update-site of 
everything in the current reactor build so category.xml is generated 
automatically, not sure if this is applicable/useful for simrel as well.


> pom.xml / update URLs

Maybe it would be better to collect the URLs in a target definition file 
as it is more flexible (you can choose what features/versions to 
include) and has IDE Editing support?



Am 11.11.20 um 08:48 schrieb Mickael Istria:

Hello Jonah,

Thanks for having a look, answer inline


On Tue, Nov 10, 2020 at 5:21 PM Jonah Graham > wrote:


   1. Merge conflicts - this is probably not such a big deal as the
tycho validation seems very fast (2 minutes!).


I think the case of merge conflicts are almost non-existent for SimRel: 
projects typically modify a few lines, or add a few lines, which are 
independent from others. Git or even plain diff would be able to merge 
the various change without pain.


   2. Tracking who is paying attention. Until now the rule was touch
the .aggrcon file, which then was easy to identify with git log. Is
this rule important going forward? I believe so, and if so can we
agree on a simple standard for what this looks like? e.g. The 
of the repository in the pom.xml should have the simrel version that
it was contributed for in the id?


We can indeed set up some new convention in the pom.xml file, either 
with id, or with some comment convention such as  to be replaced by  and make 
the inactive contribution tool report the url of repo for which is just 
after a "Not there yet" line.


- Updating version number ranges. The b3 aggregator has a very
useful "fix versions" action that can set all the version ranges
quickly and accurately. Is there some way to make this easier with
Tycho?


There is no such tool for Tycho/category.xml at the moment; and I think 
it's mostly a use-case that is related by some existing PDE and m2e 
(requiring a lot of work though). One alternative would be to implement 
a SimRel/Tycho specific tool; that would be more affordable. I'll 
investigate about it.


- The gerrit today only has a subset of the parts of CDT that are in
simrel.aggr/cdt.aggrcom (the only contribution I looked at closely).
AFAICT only the items that are categorized are present in
category.xml. Where do uncategorized features end up? Similar
question about uncategorized bundles. I could add them all to a new
category if needed.


That seems like a bug in the routine that turns the .aggr model to a 
category.xml/pom.xml pair. I'll fix it.


___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To unsubscribe from this list, visit 
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev


___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To unsubscribe from this list, visit 
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev

Re: [cross-project-issues-dev] Building SimRel with Tycho for 2021-03

[Mickael Istria]

Hello Jonah,

Thanks for having a look, answer inline


On Tue, Nov 10, 2020 at 5:21 PM Jonah Graham 
wrote:

>   1. Merge conflicts - this is probably not such a big deal as the tycho
> validation seems very fast (2 minutes!).
>

I think the case of merge conflicts are almost non-existent for SimRel:
projects typically modify a few lines, or add a few lines, which are
independent from others. Git or even plain diff would be able to merge the
various change without pain.


>   2. Tracking who is paying attention. Until now the rule was touch the
> .aggrcon file, which then was easy to identify with git log. Is this rule
> important going forward? I believe so, and if so can we agree on a simple
> standard for what this looks like? e.g. The  of the repository in the
> pom.xml should have the simrel version that it was contributed for in the
> id?
>

We can indeed set up some new convention in the pom.xml file, either with
id, or with some comment convention such as  to be replaced by  and make the inactive contribution tool report the url of
repo for which is just after a "Not there yet" line.


> - Updating version number ranges. The b3 aggregator has a very useful "fix
> versions" action that can set all the version ranges quickly and
> accurately. Is there some way to make this easier with Tycho?
>

There is no such tool for Tycho/category.xml at the moment; and I think
it's mostly a use-case that is related by some existing PDE and m2e
(requiring a lot of work though). One alternative would be to implement a
SimRel/Tycho specific tool; that would be more affordable. I'll investigate
about it.


> - The gerrit today only has a subset of the parts of CDT that are in
> simrel.aggr/cdt.aggrcom (the only contribution I looked at closely). AFAICT
> only the items that are categorized are present in category.xml. Where do
> uncategorized features end up? Similar question about uncategorized
> bundles. I could add them all to a new category if needed.
>

That seems like a bug in the routine that turns the .aggr model to a
category.xml/pom.xml pair. I'll fix it.
___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To unsubscribe from this list, visit 
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev

Re: [cross-project-issues-dev] Building SimRel with Tycho for 2021-03

[Jonah Graham]

Hi Mickael and all the other people who worked hard to get to this point,

This is great, well done! I had a look at the gerrit and it is quite nice.
It certainly looks familiar. And despite my concerns listed below, moving
to Tycho seems fairly obviously the way forward.

There are a few questions I have:
- B3 is split into multiple files, this Tycho solution has everything in a
single file (two actually, pom.xml and category.xml). This has two
disadvantages:
  1. Merge conflicts - this is probably not such a big deal as the tycho
validation seems very fast (2 minutes!).
  2. Tracking who is paying attention. Until now the rule was touch the
.aggrcon file, which then was easy to identify with git log. Is this rule
important going forward? I believe so, and if so can we agree on a simple
standard for what this looks like? e.g. The  of the repository in the
pom.xml should have the simrel version that it was contributed for in the
id?
- Updating version number ranges. The b3 aggregator has a very useful "fix
versions" action that can set all the version ranges quickly and
accurately. Is there some way to make this easier with Tycho?
- The gerrit today only has a subset of the parts of CDT that are in
simrel.aggr/cdt.aggrcom (the only contribution I looked at closely). AFAICT
only the items that are categorized are present in category.xml. Where do
uncategorized features end up? Similar question about uncategorized
bundles. I could add them all to a new category if needed.

BTW I don't know what other features of b3 may be important - but I don't
know what this was: "sending emails for example wouldn't be available
anymore." That is something I have not heard of.

Thanks,
Jonah


~~~
Jonah Graham
Kichwa Coders
www.kichwacoders.com


On Tue, 10 Nov 2020 at 10:43, Mickael Istria  wrote:

> Hi all,
>
> With a recent patch, Tycho has achieved a major milestone that allows it
> to be a viable replacement to build SimRel repo in place of the very
> specific CBI aggregator. Specifically, Tycho now can validate a build plan
> without downloading all artifacts, so it can allow the Gerrit patch/CI
> feedback short feedback loop.
>
> The principle and benefits of this change consists in getting rid of CBI
> aggregator specific files, models, builders... which no-one beyond a few
> privileged (?) ones know how to deal with. By moving it to more mainstream
> and better known Maven+Tycho+category.xml stack, we allow more people to
> participate in the maintenance of the SimRel aggregation; and we also get
> rid of one brick to maintain (CBI aggregator) in favor of a well maintained
> and active one (Tycho). This will overall result in some resources saving
> and SimRel entry-barrier being much lowered.
> Of course, there are some features of CBI aggregator that don't have
> equivalent in Tycho, and the other way round; sending emails for example
> wouldn't be available anymore. But times have changed since the
> introduction of CBI aggregator (called b3 back then): CI rose, Code Review
> rose, all this DevOps things basically matured and has brought solutions
> for the use-case that were directly supported by b3. Not so objectively, I
> don't know of any feature of CBI aggregator for SimRel build that we would
> really miss when moving to Tycho. I don't the replacement this will either
> increase difficulty of contributing to SimRel or to maintain it.
>
> This Gerrit patch shows it in action
> https://git.eclipse.org/r/c/simrel/org.eclipse.simrel.build/+/170553 .
> You see it adds 1 category.xml file with the content of SimRel, and the
> list of p2 repositories in the pom.xml.
>
> I think all that stack is mature enough, and the value/savings in changing
> to Tycho is compelling enough to consider this migration for 2021-03.
>
> Cheers,
> --
> Mickael Istria
> Eclipse IDE 
> developer, for Red Hat Developers 
> ___
> cross-project-issues-dev mailing list
> cross-project-issues-dev@eclipse.org
> To unsubscribe from this list, visit
> https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev
>
___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To unsubscribe from this list, visit 
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev

[cross-project-issues-dev] Building SimRel with Tycho for 2021-03

[Mickael Istria]

Hi all,

With a recent patch, Tycho has achieved a major milestone that allows it to
be a viable replacement to build SimRel repo in place of the very specific
CBI aggregator. Specifically, Tycho now can validate a build plan without
downloading all artifacts, so it can allow the Gerrit patch/CI feedback
short feedback loop.

The principle and benefits of this change consists in getting rid of CBI
aggregator specific files, models, builders... which no-one beyond a few
privileged (?) ones know how to deal with. By moving it to more mainstream
and better known Maven+Tycho+category.xml stack, we allow more people to
participate in the maintenance of the SimRel aggregation; and we also get
rid of one brick to maintain (CBI aggregator) in favor of a well maintained
and active one (Tycho). This will overall result in some resources saving
and SimRel entry-barrier being much lowered.
Of course, there are some features of CBI aggregator that don't have
equivalent in Tycho, and the other way round; sending emails for example
wouldn't be available anymore. But times have changed since the
introduction of CBI aggregator (called b3 back then): CI rose, Code Review
rose, all this DevOps things basically matured and has brought solutions
for the use-case that were directly supported by b3. Not so objectively, I
don't know of any feature of CBI aggregator for SimRel build that we would
really miss when moving to Tycho. I don't the replacement this will either
increase difficulty of contributing to SimRel or to maintain it.

This Gerrit patch shows it in action
https://git.eclipse.org/r/c/simrel/org.eclipse.simrel.build/+/170553 . You
see it adds 1 category.xml file with the content of SimRel, and the list of
p2 repositories in the pom.xml.

I think all that stack is mature enough, and the value/savings in changing
to Tycho is compelling enough to consider this migration for 2021-03.

Cheers,
-- 
Mickael Istria
Eclipse IDE 
developer, for Red Hat Developers 
___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To unsubscribe from this list, visit 
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev