Here's another complexity, how do 2 plugins create a single publication? We basically have the same problem of 2 parallel operations creating content from a single source.
On Tue, May 15, 2018, 06:27 Ina Panova <ipan...@redhat.com> wrote: > +1 on not introducing dependencies between plugins. > > What will be the behavior in case there is a composed repo of rpm and ks > trees but just the rpm plugin is installed? > Do we fail and say we cannot sync this repo at all or we just sync the rpm > part? > Assuming plugins do not depend on each other, I think that when each plugin looks at the upstream repo, they will only "see" the content of that type. Conceptually, we will have 2 remotes, so it will feel like we are syncing from 2 totally distinct repositories. The solution I've been imagining is a lot like 2. Each plugin would sync to a *separate repository.* These separate repositories are then published creating *separate publications*. This approach allows the plugins to live completely in ignorance of each other. The final step is to associate *both publications to one distribution*, which composes the publications as they are served. The downside is that we have to sync and publish twice, and that the resulting versions and publications aren't locked together. But I think this is better than leaving versions and publications unfinished with the assumption that another plugin will finish the job. Maybe linking them together could be a good use of the notes field. > Depends how we plan this ^ i guess we'll decide which option 1 or 2 fits > better. > > Don't want to go wild, but what if notion of composed repos will be so > popular in the future that's its amount will increase? I think we do want > to at least partially being able to sync it and not take the approach all > or nothing? > > #2 speaks to me more for now. > > > > > -------- > Regards, > > Ina Panova > Software Engineer| Pulp| Red Hat Inc. > > "Do not go where the path may lead, > go instead where there is no path and leave a trail." > > On Mon, May 14, 2018 at 9:44 PM, Jeff Ortel <jor...@redhat.com> wrote: > >> Let's brainstorm on something. >> >> Pulp needs to deal with remote repositories that are composed of multiple >> content types which may span the domain of a single plugin. Here are a few >> examples. Some Red Hat RPM repositories are composed of: RPMs, DRPMs, , >> ISOs and Kickstart Trees. Some OSTree repositories are composed of OSTrees >> & Kickstart Trees. This raises a question: >> >> How can pulp3 best support syncing with remote repositories that are >> composed of multiple (unrelated) content types in a way that doesn't result >> in plugins duplicating support for content types? >> >> Few approaches come to mind: >> >> 1. Multiple plugins (Remotes) participate in the sync flow to produce a >> new repository version. >> 2. Multiple plugins (Remotes) are sync'd successively each producing a >> new version of a repository. Only the last version contains the fully >> sync'd composition. >> 3. Plugins share code. >> 4. Other? >> >> >> Option #1: Sync would be orchestrated by core or the user so that >> multiple plugins (Remotes) participate in populating a new repository >> version. For example: the RPM plugin (Remote) and the Kickstart Tree >> plugin (Remote) would both be sync'd against the same remote repository >> that is composed of both types. The new repository version would be >> composed of the result of both plugin (Remote) syncs. To support this, >> we'd need to provide a way for each plugin to operate seamlessly on the >> same (new) repository version. Perhaps something internal to the >> RepositoryVersion. The repository version would not be marked "complete" >> until the last plugin (Remote) sync has succeeded. More complicated than >> #2 but results in only creating truly complete versions or nothing. No >> idea how this would work with current REST API whereby plugins provide sync >> endpoints. >> >> Option #2: Sync would be orchestrated by core or the user so that >> multiple plugins (Remotes) create successive repository versions. For >> example: the RPM plugin (Remote) and the Kickstart Tree plugin (Remote) >> would both be sync'd against the same remote repository that is a >> composition including both types. The intermediate versions would be >> incomplete. Only the last version contains the fully sync'd >> composition. This approach can be supported by core today :) but will >> produce incomplete repository versions that are marked complete=True. This >> /seems/ undesirable, right? This may not be a problem for distribution >> since I would imaging that only the last (fully composed) version would be >> published. But what about other usages of the repository's "latest" >> version? >> >> Option #3: requires a plugin to be aware of specific repository >> composition(s); other plugins and creates a code dependency between >> plugins. For example, the RPM plugin could delegate ISOs to the File >> plugin and Kickstart Trees to the KickStart Tree plugin. >> >> For all options, plugins (Remotes) need to limit sync to affect only >> those content types within their domain. For example, the RPM (Remote) >> sync cannot add/remove ISO or KS Trees. >> >> I am an advocate of some from of options #1 or #2. Combining plugins >> (Remotes) as needed to deal with arbitrary combinations within remote >> repositories seems very powerful; does not impose complexity on plugin >> writers; and does not introduce code dependencies between plugins. >> >> Thoughts? >> >> _______________________________________________ >> Pulp-dev mailing list >> Pulp-dev@redhat.com >> https://www.redhat.com/mailman/listinfo/pulp-dev >> >> > _______________________________________________ > Pulp-dev mailing list > Pulp-dev@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-dev >
_______________________________________________ Pulp-dev mailing list Pulp-dev@redhat.com https://www.redhat.com/mailman/listinfo/pulp-dev
