Re: [cross-project-issues-dev] On the issue of building withthelatest Orbit repository

[David M Williams]

 > 3. Alternatively, you can create a root feature that includes your 
> actual project feature and the Orbit bundles you require. You point 
> your build at this feature, but you do not contribute this feature 
> to simrel and you do not making it visible in your own categories 
> file. For the cleanest result, you’d remove this feature as a post-
> build step, but it wouldn’t hurt anything if it were to remain as 
> “hidden” feature in your repository that never gets aggregated or 
installed.

I have a lot more to say on the general topic being discussed, but thought 
I'd try to clear up this one point quickly. 

If there is something "in your repository" that you name in your b3aggrcon 
file, then p2 considers anything in there as "fair game" to pull, if it 
satisfies some constraint it is trying to satisfy. 

It may pick something from there rarely, but it has been a "real life" 
problem over the years, where people mistakenly think that only the 
features they name are installed from their repository. Whether not it 
would be a problem depends on many things -- some of which is what other 
projects do or specify, which you could never know or be in control of. 

While some are surprised at this, it is also the way p2 works when a user 
installs something. 



___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from 
this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev

Re: [cross-project-issues-dev] On the issue of building withthelatest Orbit repository

[Konstantin Komissarchik]

> 1) Ensure that none of the features contributed to simrel include any Orbit 
> bundles,
> but only refer to them via dependencies (and ensure that proper version 
> constraints
> are used in addition).

+1 to not using feature includes on Orbit bundles from project’s features. The 
actual presence of Orbit bundles in project’s repository is not particularly 
harmful as long as the dependencies on these bundles are expressed using 
version ranges.

> 2) Use specific all-in-one features that include the required Orbit bundles 
> (and the
> features deployed to simrel), but only use them to produce self-contained 
> local drop
> files (zip).

That’s one implementation approach of achieving #1. There are other options 
that I have outlined in another response.

> 3) Do not publish all-in-one features on your local update sites either, but 
> rather
> have your local update-sites refer to the simrel update-site via an 
> associate-site entry
> (so Orbit dependencies get pulled in from there).

It’s not a good idea to mix project release repositories and simrel update 
stream. The project’s release should be a fixed point in time, so it’s entirely 
appropriate for the repository to contain the necessary dependendencies at 
versions used during the release process. 

Using an associate-site in the manner you are proposing would mix release 
artifacts and a certain particular update policy in a way that would make it 
difficult for other update policies to exist. The problems with this are 
discussed in more details in other threads on this mailing list. In short, to 
maintain the flexibility of supporting multiple update policies, project’s 
release repositories should never use associate sites (referenced 
repositories). Only an update stream repository (such as simrel or one 
maintained by the project) should do this.

Thanks,

- Konstantin


From: Alexander Nyßen
Sent: Saturday, February 6, 2016 6:14 AM
To: Cross project issues
Subject: Re: [cross-project-issues-dev] On the issue of building withthelatest 
Orbit repository

What about the following policy?

1) Ensure that none of the features contributed to simrel include any Orbit 
bundles, but only refer to them via dependencies (and ensure that proper 
version constraints are used in addition).
2) Use specific all-in-one features that include the required Orbit bundles 
(and the features deployed to simrel), but only use them to produce 
self-contained local drop files (zip).
3) Do not publish all-in-one features on your local update sites either, but 
rather have your local update-sites refer to the simrel update-site via an 
associate-site entry (so Orbit dependencies get pulled in from there).

With 1) we would have a single authority that controls which Orbit bundles are 
actually provided in a release, namely the simrel aggregator. 2) would cover 
your use case, Ed. 3) would ensure that newer Orbit bundles (added in a 
maintenance release) would also get picked up when installing from a local 
update-site.

Cheers
Alexander

Am 05.02.2016 um 20:22 schrieb Ed Willink :

Hi Konstantin

I have no idea how a download ZIP can easily mirror something.

But I presume what you really mean is that we should produce two 
ZIPs/categories.

- A contribution ZIP/category that replicates the features contributed to 
SimRel.
- An All-in-One ZIP/category that additionally bundles Orbit facilities 
avoiding the need for users to learn how to include Orbit in the available 
sites.

    Regards

        Ed Willink

On 05/02/2016 17:33, Konstantin Komissarchik wrote:
I would advise strongly against using the feature includes directive for Orbit 
bundles. While doing so provides a cheap way to mirror the Orbit bundle into 
the produced project repository, it also locks you down to using that one 
version of the bundle and we end up with multiple versions of various bundles 
in the install, often for no good reason. It’s easy enough to mirror the 
required Orbit bundles using a separate step at the end of the project build 
and be free to require the bundle via whatever version range is most 
appropriate.
 
If everyone did this and simrel aggregation included the latest Orbit 
repository, the result of aggregation would contain fewer duplicate Orbit 
bundles without requiring projects to issue a new release just to move to a 
newer version of an Orbit bundle.
 
Thanks,
 
- Konstantin
 
 
From: Ed Willink
Sent: Thursday, February 4, 2016 11:11 PM
To: cross-project-issues-dev@eclipse.org
Subject: Re: [cross-project-issues-dev] On the issue of building with thelatest 
Orbit repository
 
Hi David

Interesting if you're right. I thought projects needed to bundle what wasn't 
bundled upstream. Thus OCL bundles LPG since no one else does. OCL includes 
ASM, Guava that someone else bundles.

You suggest that OCL could stop bundling LPG since LPG would appear in the 
SimRel repo automatically.

However OCL still needs to bundle LPG in order for the install 

Re: [cross-project-issues-dev] On the issue of building withthelatest Orbit repository

[Konstantin Komissarchik]

> I have no idea how a download ZIP can easily mirror something.

The exact details depend on the build system you are using. Here are some 
options:

1. If you can inject logic before repository metadata is generated, all you 
need to do is copy the Orbit bundle into your repository. That’s what I do in 
the Sapphire build to include ASM. 

2. If you can’t inject logic before repository metadata is generated, you can 
use a p2 mirror command as a post-processing step. The mirroring command can do 
a partial mirroring (the bundles you specify).

http://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Fguide%2Fp2_mirror.html=2_0_20_3_0
https://wiki.eclipse.org/Equinox/p2/Ant_Tasks#Partial_Mirroring

3. Alternatively, you can create a root feature that includes your actual 
project feature and the Orbit bundles you require. You point your build at this 
feature, but you do not contribute this feature to simrel and you do not making 
it visible in your own categories file. For the cleanest result, you’d remove 
this feature as a post-build step, but it wouldn’t hurt anything if it were to 
remain as “hidden” feature in your repository that never gets aggregated or 
installed.

> But I presume what you really mean is that we should produce two 
> ZIPs/categories.
> 
> - A contribution ZIP/category that replicates the features contributed to 
> SimRel.
> - An All-in-One ZIP/category that additionally bundles Orbit facilities 
> avoiding
> the need for users to learn how to include Orbit in the available sites.

There is no need to prepare two separate zips. The presence of a particular 
version of an Orbit bundle in a project’s release repository will not prevent 
aggregation from using a newer version, if one is available elsewhere, as long 
as your project’s bundles and features specify the dependency via a version 
range. That is, you aren’t using the feature’s include facility in your 
aggregated project features, which ties you to one exact version of the bundle.

Hope that helps,

- Konstantin


From: Ed Willink
Sent: Friday, February 5, 2016 11:23 AM
To: cross-project-issues-dev@eclipse.org
Subject: Re: [cross-project-issues-dev] On the issue of building withthelatest 
Orbit repository

Hi Konstantin

I have no idea how a download ZIP can easily mirror something.

But I presume what you really mean is that we should produce two 
ZIPs/categories.

- A contribution ZIP/category that replicates the features contributed to 
SimRel.
- An All-in-One ZIP/category that additionally bundles Orbit facilities 
avoiding the need for users to learn how to include Orbit in the available 
sites.

    Regards

        Ed Willink
On 05/02/2016 17:33, Konstantin Komissarchik wrote:
I would advise strongly against using the feature includes directive for Orbit 
bundles. While doing so provides a cheap way to mirror the Orbit bundle into 
the produced project repository, it also locks you down to using that one 
version of the bundle and we end up with multiple versions of various bundles 
in the install, often for no good reason. It’s easy enough to mirror the 
required Orbit bundles using a separate step at the end of the project build 
and be free to require the bundle via whatever version range is most 
appropriate.
 
If everyone did this and simrel aggregation included the latest Orbit 
repository, the result of aggregation would contain fewer duplicate Orbit 
bundles without requiring projects to issue a new release just to move to a 
newer version of an Orbit bundle.
 
Thanks,
 
- Konstantin
 
 
From: Ed Willink
Sent: Thursday, February 4, 2016 11:11 PM
To: cross-project-issues-dev@eclipse.org
Subject: Re: [cross-project-issues-dev] On the issue of building with thelatest 
Orbit repository
 
Hi David

Interesting if you're right. I thought projects needed to bundle what wasn't 
bundled upstream. Thus OCL bundles LPG since no one else does. OCL includes 
ASM, Guava that someone else bundles.

You suggest that OCL could stop bundling LPG since LPG would appear in the 
SimRel repo automatically.

However OCL still needs to bundle LPG in order for the install from ZIPs to 
work offline. AFAIK there is no SimRel ZIP distribution. If there was, it would 
probably be so big that few would use it. However if there was a 
used-in-SimRel-Orbit ZIP distribution that could be useful and could help you 
enforce limited usage.

    Regards

        Ed Willink
On 04/02/2016 21:35, David M Williams wrote:
Off the top of my head, I think features are suppose to 'include' them, since 
that is the only way to have a reproducible build or install. If it was left up 
to "requires" then who knows what you would get. 
Granted, there should not be anything breaking, if you simply took a what ever 
was there, within some specified range, but especially with third party 
bundles, you never know. Some are real good at following good versioning 
practices, some are not.  Plus, keep in mind, the "aggregated repository" is 

Re: [cross-project-issues-dev] On the issue of building withthelatestOrbit repository

[Konstantin Komissarchik]

Yes, if another project references the hidden all-in-one feature, it will be 
slurped up by aggregation and we are back to square one. If you aren’t able to 
remove this feature from your project’s repository completely as a post-build 
step (the ideal solution), I would recommend naming it in a way that 
discourages its use…

org.eclipse.project.Internal_Root_DoNotReference or some such

Thanks,

- Konstantin



From: David M Williams
Sent: Monday, February 8, 2016 10:17 AM
To: Cross project issues
Subject: Re: [cross-project-issues-dev] On the issue of building 
withthelatestOrbit repository

 
> 3. Alternatively, you can create a root feature that includes your 
> actual project feature and the Orbit bundles you require. You point 
> your build at this feature, but you do not contribute this feature 
> to simrel and you do not making it visible in your own categories 
> file. For the cleanest result, you’d remove this feature as a post-
> build step, but it wouldn’t hurt anything if it were to remain as 
> “hidden” feature in your repository that never gets aggregated or installed.

I have a lot more to say on the general topic being discussed, but thought I'd 
try to clear up this one point quickly. 

If there is something "in your repository" that you name in your b3aggrcon 
file, then p2 considers anything in there as "fair game" to pull, if it 
satisfies some constraint it is trying to satisfy. 

It may pick something from there rarely, but it has been a "real life" problem 
over the years, where people mistakenly think that only the features they name 
are installed from their repository. Whether not it would be a problem depends 
on many things -- some of which is what other projects do or specify, which you 
could never know or be in control of. 

While some are surprised at this, it is also the way p2 works when a user 
installs something. 


___
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from 
this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev