On 05/15/2018 05:58 AM, Austin Macdonald wrote:
Here's another complexity, how do 2 plugins create a single publication?
The plugin API could make this seamless.
We basically have the same problem of 2 parallel operations creating
content from a single source.
I don't think so. plugins should not manipulate content outside of
their domain (other plugins content) so either serial or parallel should
be safe.
On Tue, May 15, 2018, 06:27 Ina Panova <ipan...@redhat.com
<mailto:ipan...@redhat.com>> wrote:
+1 on not introducing dependencies between plugins.
What will be the behavior in case there is a composed repo of rpm
and ks trees but just the rpm plugin is installed?
Do we fail and say we cannot sync this repo at all or we just sync
the rpm part?
Assuming plugins do not depend on each other, I think that when each
plugin looks at the upstream repo, they will only "see" the content of
that type. Conceptually, we will have 2 remotes, so it will feel like
we are syncing from 2 totally distinct repositories.
The solution I've been imagining is a lot like 2. Each plugin would
sync to a *separate repository.* These separate repositories are then
published creating *separate publications*. This approach allows the
plugins to live completely in ignorance of each other.
The final step is to associate *both publications to one
distribution*, which composes the publications as they are served.
The downside is that we have to sync and publish twice, and that the
resulting versions and publications aren't locked together. But I
think this is better than leaving versions and publications unfinished
with the assumption that another plugin will finish the job. Maybe
linking them together could be a good use of the notes field.
Pulp should support repositories with composed (mixed) content for the
same reason RH does. The repository is a collection of content that
users want to manage together. Consider the promotion cases: dev, test,
prod.
Depends how we plan this ^ i guess we'll decide which option 1 or
2 fits better.
Don't want to go wild, but what if notion of composed repos will
be so popular in the future that's its amount will increase? I
think we do want to at least partially being able to sync it and
not take the approach all or nothing?
#2 speaks to me more for now.
--------
Regards,
Ina Panova
Software Engineer| Pulp| Red Hat Inc.
"Do not go where the path may lead,
go instead where there is no path and leave a trail."
On Mon, May 14, 2018 at 9:44 PM, Jeff Ortel <jor...@redhat.com
<mailto:jor...@redhat.com>> wrote:
Let's brainstorm on something.
Pulp needs to deal with remote repositories that are composed
of multiple content types which may span the domain of a
single plugin. Here are a few examples. Some Red Hat RPM
repositories are composed of: RPMs, DRPMs, , ISOs and
Kickstart Trees. Some OSTree repositories are composed of
OSTrees & Kickstart Trees. This raises a question:
How can pulp3 best support syncing with remote repositories
that are composed of multiple (unrelated) content types in a
way that doesn't result in plugins duplicating support for
content types?
Few approaches come to mind:
1. Multiple plugins (Remotes) participate in the sync flow to
produce a new repository version.
2. Multiple plugins (Remotes) are sync'd successively each
producing a new version of a repository. Only the last
version contains the fully sync'd composition.
3. Plugins share code.
4. Other?
Option #1: Sync would be orchestrated by core or the user so
that multiple plugins (Remotes) participate in populating a
new repository version. For example: the RPM plugin (Remote)
and the Kickstart Tree plugin (Remote) would both be sync'd
against the same remote repository that is composed of both
types. The new repository version would be composed of the
result of both plugin (Remote) syncs. To support this, we'd
need to provide a way for each plugin to operate seamlessly on
the same (new) repository version. Perhaps something internal
to the RepositoryVersion. The repository version would not be
marked "complete" until the last plugin (Remote) sync has
succeeded. More complicated than #2 but results in only
creating truly complete versions or nothing. No idea how this
would work with current REST API whereby plugins provide sync
endpoints.
Option #2: Sync would be orchestrated by core or the user so
that multiple plugins (Remotes) create successive repository
versions. For example: the RPM plugin (Remote) and the
Kickstart Tree plugin (Remote) would both be sync'd against
the same remote repository that is a composition including
both types. The intermediate versions would be incomplete.
Only the last version contains the fully sync'd composition.
This approach can be supported by core today :) but will
produce incomplete repository versions that are marked
complete=True. This /seems/ undesirable, right? This may not
be a problem for distribution since I would imaging that only
the last (fully composed) version would be published. But
what about other usages of the repository's "latest" version?
Option #3: requires a plugin to be aware of specific
repository composition(s); other plugins and creates a code
dependency between plugins. For example, the RPM plugin could
delegate ISOs to the File plugin and Kickstart Trees to the
KickStart Tree plugin.
For all options, plugins (Remotes) need to limit sync to
affect only those content types within their domain. For
example, the RPM (Remote) sync cannot add/remove ISO or KS Trees.
I am an advocate of some from of options #1 or #2. Combining
plugins (Remotes) as needed to deal with arbitrary
combinations within remote repositories seems very powerful;
does not impose complexity on plugin writers; and does not
introduce code dependencies between plugins.
Thoughts?
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com <mailto:Pulp-dev@redhat.com>
https://www.redhat.com/mailman/listinfo/pulp-dev
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com <mailto:Pulp-dev@redhat.com>
https://www.redhat.com/mailman/listinfo/pulp-dev
_______________________________________________
Pulp-dev mailing list
Pulp-dev@redhat.com
https://www.redhat.com/mailman/listinfo/pulp-dev
