Comments below.
On 29.01.2020 10:48, Mickael Istria wrote:
It seems mostly based on the the principle/assumption that moving
the problem will simplify the problem or make existing problems
disappear by magic.
Indeed, I believe that moving the problem to a better functioning
project according to EDP practices, and by the way getting rid of a
lot of questionable effort, will make many projects disappear.
Making project disappears will definitely help make their contributed
problems disappears. Also, centralization of responsibilities seems
helpful.
Currently the train repo is a prerequisite for producing the
packages and it composes a large set of repositories into a single
aggregate at which point a high level of consistency is checked
and ensured. In the end, ensuring that all the artifacts that
comprise each package are coherent (and stable) does not go away
even if somehow the packages were produced by directly pulling
content from something other than the train repository.
With EPP, we ensure that the packages are consistent and of good
enough quality to be released. I don't think that would change much,
and EPP could also add some tests verifying all features can install
in the same IDE.
The user's often install additional things the consistency of which is
also important it seems to me. The train has helped a great deal with
that. E.g., you can be sure that when you install Xtext you also install
the corresponding EMF version...
Nothing changes with regard to ensuring consistent licenses,
signed content, proper inter-operation, stable repositories, and
mutual instability.
Right, but at least, if it's in EPP, then we're sure the projects that
are integrated have at least 1 person (the EPP maintainer for this
package) caring about those issues and dealing with them. While with
current push model in SimRel, some project push outdated stuff and
aren't reachable.
Yes, though it's not clear that this point what subset remains based on
the transitive closure of all EPP package dependencies. Often there
are surprises. E.g., I tried to remove BIRT, but there remain charting
dependencies so I could only reduce the subset aggregated from BIRT...
In the end, I'm not even sure if you're suggesting that there
needs to be no aggregation at all, but simply a very large set of
direct references to various project repositories. But I can
assure you that loading 50 repositories instead of 2 when doing an
install will not improve the experience, and that getting n
projects to maintain long-term stable sites is a new problem that
will also turn into yet another cat herding exercise and when it
fails (as all things do on occasion), the users will notice
immediately.
I suggest EPP builds the aggregated and categorized p2 repository,
containing only stuff that are included in packages.
We'd need to understand the transitive closure of all packages and
presume that consistency for anything in a package is just not important
to anyone, not even users.
As such, I did quick test, resolve all EPP IUs for 2020-03 into a target
platform, which suggests that only 10% of the IUs are in packages. It's
not clear which cross section of projects this covers. But it's clear
looking at which EMF things are in the TP that only a very small subset
of what EMF contributes to the aggregate is transitively required by the
EPP packages. I'd not be happy with a train aggregate that did not
include all the things I currently contribute to the train. One stop
shopping from the aggregate, an aggregate that's bigger than what's
transitively pre-installed in the union of all EPP packages, seems to me
to have significant value...
To build. EPP references different source p2 repositories, just like
SimRel reference other repositories.
So how will the exact repositories to use be managed? I see later you
suggest a *.target file, but it's not clear if that's a per-package
*.target, in which case keeping it consistent across packages is
important, or if it's shared single *.target, in which case multiple
packages need to main the union of IUs to pull from the URLs.
It feel as if you've joined the discussion years after all the
reasons for having a train the first place were had, and that you
assume there really are no good reasons because you were not part
of those discussions. So all the reasons need to be reiterated, at
which point you are highly inclined to try to shoot each one down
because they don't fit you current conclusion.
I know the reasons, and participated in integration of a project in
SimRel 11 years ago and was very excited about SimRel and invested in
it. I think it was interesting back then, but times have changed: many
project are almost inactive in SimRel, SimRel build has lost
maintenance workforce and there doesn't seem to be much hope for more,
here is Marketplace, there are EPP packages, there is an installer...
So I think it's just time to question again whether we can
collectively afford SimRel as it is now, and even whether this is
still the appropriate solution for past problems.
So at best you are suggesting the transitive closure of the EPP
dependencies is much smaller and will be easier to manage by a yet-to-be
determined someone. That seems not unlikely but it presumes there is no
other value for the aggregate.
Those who need more need to invest more; but if no-one wishes to
invest more despite the urge, then it's that there is actually no
strong enough need to keep things as they are now.
Certainly centralized cohesive management seems a necessary part of any
solution, but distributed management of contributed content (and
organizing it) will remain a problem for any solution.
In any case, no matter exactly all the concrete details of what
you are suggesting, the question of who does that work remains the
same one.
If we keep separated SimRel and EPP and keep projects that are
irrelevant to EPP in a push mode, it's a lot of work and no-one will
want to do this work.
In the end though, the projects will need to push what's intended to be
pushed. E.g., a bug like this is opened every few weeks asking me
which EMF repo to push into the platform's build:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=559488
This is still work for me, and I imagine someone needing to manage what
ends up in EPP packages will need to do this type of thing for a
significantly large number of projects, which seems incredibly painful
to me. I think that whatever the process, it needs further automated to
make this all less painful. E.g., why can't we have staging EPP
repos/packages daily?
If we make things simpler and better address current issues instead of
sticking to older ones, then it's less work and it's more interesting,
some people will continue it.
Which issues will be simpler and simpler for whom? I already easily
push a new repo into simrel and that for me is simpler than being
prompted by Bug 559488. For anyone producing an EPP package it's very
simple to find what's aggregated into the train repo because all the
providers just push content into it... I would not want to manage
finding what should be pulled... I would expect a push process to make
those available. And I would not want to be faced with resolution
failures because someone pushed something incompatible with something
someone else pushed; this something the aggregator prevents right now.
My reasons to support that is that:
* Marketplace would still be available -> no loss for users
I also pointed out that you could fix your marketplace entry:
https://www.eclipse.org/setups/marketplace/?id=3394048
This is far from being top-priority, I have more valuable work in the
backlog (like lobbying for the end of SimRel as we know it ;)
Apparently a consistent marketplace is no one's priority, so the users
end up with a perception that much of it is crap, and therefore Eclipse
is crap.
That hasn't happened and the fully 1/3 of the marketplace entries
are completely broken or somewhat broken. Consistent/correct
marketplace listings is yet another exercise of cat herding.
There are stats about installation issues in Marketplace already, and
IIRC there is even a system to notify owner in case of too many
install failures. That seems far enough to me, and my current usage of
Marketplace is pretty pleasant and leads to good experience. (while I
never use the SimRel site...)
There is also a full report:
https://www.eclipse.org/setups/marketplace/?style=all
It suggests that randomly choosing something from the marketplace will
result in 1/3 of those things failing. Do doubt many marketplace
contributors do an excellent job maintaining their listings, but those
that don't reflect poorly on all of those that do.
* packages would still be available -> no loss for users
So at least we agree that packages are needed. Unfortunately
there's no one to produce them.
On the epp-dev mailing-lists, some questions are pending for others to
evaluate how much the can invest.
But EPP packages are a trivial-ish Tycho project, that builds itself
on CI on top of active technologies and reactive, communities. It's
not a too big deal (compared to SimRel), and once the actual
maintenance steps are clarified, there will be some people to do the work.
It seems a lot of +1 activities around 2013-03 M1 are absent.
* installer would still be available -? no loss for users
Here the question is: Which repositories will contain all the
artifacts?
I guess the same download.eclipse.org/2020-06
<http://download.eclipse.org/2020-06> repo, expect that it's built by
EPP, not by SimRel.
So there will still be aggregation, it will just be implemented
differently. Presumably the aggregate will be a smaller set of projects
so some project-specific problems will disappear, which would be good.
But overall the same nature of problems will remain for whatever
projects remain, there might well be fewer of those projects. Someone
will just need to implement a new processes and manage contributing
repositories differently.
How much work will I personally (Oomph) have because of a complete
change in design in EPP structure?
Hopefully none.
* SimRel and its strange governance and all the discussions that
have emerged with it disappear -> time saved
It will only disappear from view, but the identical technical
problems will simply migrate somewhere else. Somewhere
decentralized? Somewhere with no central oversight? This doesn't
not sound like the problems will go away, but rather will become
invisible to most of us, but not for the users.
Indeed, some checks need to remain; but most of them can be automated,
and in EPP, much is already done by package testers.
Yes, automation is key, including automation of more problem detection
and automation of problem reporting...
* EPP starts handing everything, and EPP governance is working
well -> EDP used efficiently.
The person who does not exist will restructure everything and will
manage everything personally and none of us will have to do
anything at all anymore to help that person. That sounds great in
principle, except for that person. And I'm often that person, and
I can assure you it's not great at all; I often cannot solve
problems that come from elsewhere.
That's fair, however, I'm convinced the proposed change is profitable.
But it indeed requires investment; but it's always the same, if we can
convince people there is a return on investment (and IMO there is as
SimRel day-to-day maintenance will be drastically reduced), then we'll
have them more likely to help.
I think the key driving force for you is to see the simrel aggregator go
away, correct? But I still feel that there needs to be an automatic
process by which contributing projects make their repositories known.
The categorization of the features also remains. So a number of things
that the aggregator solves well already needs to be rearchitected via a
new process that will have new problems as well as many of the same old
ones. And, someone must step up to do that.
* Active contributors like you stop spending effort on projects
that are not worth it (to you): many projects are in SimRel just
by the force of habit, but the value for the user community is
arguable and the cost for maintainers is present (more things to
check, more bugs to open, more files to watch, longer build
time....).
Removing projects from the train will be somewhat helpful in
reducing the overhead. We could start with EMF and finish with
Oomph; that would save me personally one hell of a lot of work.
Or did you have specific projects in mind that are not worth the
effort?
Every project that is not part of an EPP package and not investing in
testing the EPP package is IMO not worth the effort.
That seems a reasonable position.
With this proposal, the maintenance cost would be drastically
reduced and the process be made more streamlined with typical
EDP. Hopefully this will become simple enough for the few active
contributors on SimRel (build & infra, not contributions) and EPP
to be able to cope with this for some years.
Are you volunteering to step up to prototype and demonstrate all
the new infrastructure that would be involved in your proposal so
that we may concretely assess how that alternative would work in
detail rather than in the abstract?
:)
Yes and no. I may try it at some point, but cannot guarantee it nor
estimate when. But as I'm convinced of the ROI and my suggestion is
mostly about adding a .target file into EPP build that happens to be a
simple Tycho build, then I may find some time for it when I'm bored
with other things.
I.e., implement something that captures the same information as in
captured by the aggr/aggrecon model. The URLs in that *.target file
will need to be maintained by a distributed set of projects/people.
And you'll need a site.xml for categorization which also must be managed
in distributed way.
About enforcing or checking SimRel rules, then they are not
really SimRel rules and checking that or declaring compatibility
should be handled by projects, as part of their releases; not by
a downstream consumption.
I've seen that projects are so very very responsive in addressing
the issues raised for them, not!
Then it becomes the responsibility and choice of the ones who "pull"
the project: if i maintain an EPP and see a dependency in a bad state
(let's say Mylyn), it becomes my duty to get it fixed or get it
removed from the EPP packages/aggregation. But as long as Mylyn
doesn't react and comply with the necessary rules, they'll just be
removed from EPP and aggregated site.
This is effectively what I've been doing for the past months. Mostly
hunting down problems and contributing fixes for them whenever
possible. Between that and contributing my own projects, it's been a
full time job.
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@eclipse.org
To change your delivery options, retrieve your password, or unsubscribe from
this list, visit
https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev
