Yes I get that perspective and I was a bit surprised to read it like that - I assume when toby wrote it down the need for mutable content as seeding content wasn't really present (or an alternative approach discussed like in sling with repoinit - yet we still have the need to support those for a while). The question is now if relaxing the definition would be the right way to go - I guess we don#t have any logic depending on those constraints right now beside of the analysers failing on those. So iterating over the definition might be the right thing to do.
Not sure if Toby has a specific opinion on that one. Cheers Dominik On Tue, Jan 21, 2020 at 3:40 PM Konrad Windszus <[email protected]> wrote: > That is not what is stated in JCRVLT-170. > It is a bit tricky though because JCRVLT-170 combines blacklisting and > whitelisting rules and doesn't say what the baseline is. > ... > Container Packages > > As for the container / deployment packages they have the characteristics > that: > > - they don't contain any content > - they contain a set of application content packages > - they don't have dependencies on other deployment packages (i.e. the > dependencies are implicit given by the relations of the included artifacts) > > > .... > > I think everyone would derive from this info that "content" type content > package are not allowed, because only "application" content packages are > mentioned here. > But I am fine with a clarification, that content packages may be contained > in there as well. For that we should have a new FileVault ticket! > > Then we don't necessarily need to have another package type. But we should > then also clarify if nesting of container packages should be allowed or not. > IMHO this would be a useful feature and I don't see why we should disallow > that. > > Konrad > > > On 21. Jan 2020, at 15:21, Dominik Süß <[email protected]> wrote: > > well TBH - the original understanding I had about container was what I was > describing with the collection with the difference that it has an identity > (yet it isn't defined anywhere and not declared that you can have > dependencies on containers. Core concept that we discussed for JCRVLT-170 > is that the containers would never directly install anything in the > repository directly but is a container for packages and other installables > as well as meta that may be installed by what ever processes them. This is > the origin of not having "any" content in them - basically, anything that > is to be installed would have an own identity so you can declare to any > content or installable artifact by referencing their specific identity. > > So if we refine the definition of container and relax it what is currently > enforced with the analyzers today we would be able to follow this > deployment scheme and are able to enforce to not base on antipatterns > purely by blocking mixed packages for scenarios with composite nodestore > (btw. being able to configure the analysers to enforce certain packageTypes > would be really useful for that). > > WDYT? > > Cheers > Dominik > > > > On Tue, Jan 21, 2020 at 1:50 PM Konrad Windszus <[email protected]> wrote: > >> Hi Dominik, >> This is still very vague to me. >> >> What is your idea about the current packageType: container (which is a >> content-package with an identity)? What should be allowed in there? >> What about the new propsed type PackageType.Collection? How does that >> relate to the existing packageType? Does it replace it? >> >> Can you try to answer those questions, so that everyone can get a clearer >> picture at what you are aiming? >> >> I would strongly suggest not to come up with something which is not a >> content-package itself, because it seems no one has the time to implement >> new features in FileVault (like. a package which can only be unwrapped, but >> no one can depend on). >> Let's rather enforce this with some validators: >> https://jackrabbit.apache.org/filevault/validation.html. Way less effort >> IMHO and much better backwards compatible! >> Thanks, >> Konrad >> >> >> On 21. Jan 2020, at 09:20, Dominik Süß <[email protected]> wrote: >> >> Hi Konrad, >> >> what worries me about mixed is that is doesn't enforce any separation >> which is the scenario that we want to prevent. >> A mixed content-package has an identity which another package can depend >> on - yet in a composite nodestore the mutable and immutable parts are >> installed at different times making it impossible to model the dependencies >> of immutable packages against immutable content tha would be served from a >> mixed package. >> This is why supporting mixed content packages cannot work in a composite >> nodestore scenario. >> >> Yes it is possible to abuse mixed packages as what I propose as a >> collection type - yet no one prevents you from adding content directly to >> that package which in turn might require to set direct dependencies for it >> making it again something you cannot simply drop >> >> Therefore the suggestion is to have a type that allows bundling together >> everything you want to eventually get installed into the system, where all >> packages that have an identity are clearly either application or content >> (so separation of mutable and immutable) so the dependency ordering is no >> longer ambiguous between deployment (change of immutable content state) and >> runtime (change of mutable state). >> >> I understand that the perspective you have is from what is the state when >> you would upload this to packagemanager - and from my pov this would only >> alow you to unfold that package so you have all the contained packages >> available. And yes we might even add a ui / rest api feature to triggere >> installation for all of the contained packages automatically (e.g. the >> mutable subset or all if there is no CNS) yet the package itself would not >> have an installation state on its own as it is only a shipping container. >> >> Does this clarify my thoughts about this a bit better? >> >> When it comes to nesting of packages of same type it is ambiguous so we >> might want to clarify this for sure. >> >> Cheers >> Dominik >> >> On Tue, Jan 21, 2020 at 8:47 AM Konrad Windszus <[email protected]> wrote: >> >>> Hi Dominik, >>> I am not getting exactly what you propose here: >>> Which types should be allowed in "container"? >>> a) If there is already a support for "content" type packages I don't see >>> the need for another type within "container" >>> b) If there is no support for nested content type packages, the question >>> is how to aggregate multiple content type packages together so that they >>> are actually installed via one big content-package? >>> >>> As I already said, I am not too much worried about a mixture of mutable >>> and immutable packages (that can still be done via 'mixed') but if we >>> support that in FileVault via a dedicated type, this should be installable >>> as well (i.e. be a regular content package). >>> Having a pure package format without supporting installation sounds like >>> a unnecessary limitation (even having AEM as a Cloud Service in mind) >>> because a lot of consumers right now don't care about mutable/immutable >>> (i.e. old AEM instances not leveraging the Composite Node Store). >>> As I already said, I would like to see a clarification ticket for >>> JCRVLT-170, because that clearly does neither mention nested containers nor >>> content type packages within containers (therefore everyone assumes, those >>> are forbidden) >>> >>> Thanks, >>> Konrad >>> >>> On 21. Jan 2020, at 08:34, Dominik Süß <[email protected]> wrote: >>> >>> Hi Konrad, >>> >>> I get your point yet that was not really the origin. Probably we can >>> split this off in another way. >>> Let's introduce a PackageType.Collection - basically this would supposed >>> to be never installed but is just a shipping unit holding a set of other >>> packages. These could either be in a flat list or for compat reasons also >>> be placed in an /etc/packages structure. >>> This packageType would be excluded of evaluation and is supposed to >>> never be installed directly. It basically is a zip file acting as a wrapper >>> around a collection of other packages. >>> No further semantics beside the fact that it has no vault identity on >>> its own and won't be visible after being installed - it's a pure shipping >>> unit. >>> >>> Cheers >>> Dominik >>> >>> On Thu, Jan 16, 2020 at 5:21 PM Konrad Windszus <[email protected]> wrote: >>> >>>> Hi Dominik, >>>> For me container was always immutable by only embedding immutable >>>> content (bundles, config, app-packages, as indicated in JCRVLT-170) >>>> Especially when we allow nesting of containers, we should probably >>>> think of two different container types: >>>> - container (immutable), with the current restrictions, but in addition >>>> allows nested containers (immutable) >>>> - content-container (mutable), only allows content and >>>> content-container sub packages >>>> >>>> That way we have a clean separation even on the container level. >>>> Whether we need one container which allows embedding both container >>>> (immutable) and content-container (mutable), I am not sure about. WDYT? >>>> Also not very sure about the naming of such a construct... >>>> >>>> Konrad >>>> >>>> On 16. Jan 2020, at 16:40, Dominik Süß <[email protected]> wrote: >>>> >>>> Hi Konrad, >>>> >>>> mixed is a legacy structure that basically allows all kinds of things. >>>> What we would want to avoid is mixing content and application content in >>>> the same package - building a container that allow application and content >>>> package to be grouped together (as installable entities) but making sure >>>> they are not mashed together helps to install both at distinct times. If >>>> you check setups working with the composite nodestore you would have 2 >>>> different times at which the application and the content packages would be >>>> installed. Crucial is that containers basically don't really define an own >>>> identity that you would be depending on - you should only define >>>> dependencies on the actual installables . The problem of a mixed package is >>>> that people start to add content to it and therefore you have to define >>>> dependencies on it, if they then also contain application on content there >>>> is ambiguity when this package actually is in an installed state as the >>>> application part would need to be installed when preparing the immutable >>>> part of the repository while the content only can be installed at runtime >>>> with the mutable content being accessible. >>>> Containers would be the tool to declare a grouping of entities that are >>>> supposed to be installed while (optimally - not sure if we should enforce) >>>> not playing a role in the dependency handling at all. >>>> >>>> Cheers >>>> Dominik >>>> >>>> On Thu, Jan 16, 2020 at 3:33 PM Konrad Windszus <[email protected]> >>>> wrote: >>>> >>>>> Hi Dominik, >>>>> better late feedback than no feedback at all. >>>>> Let me comment on your points >>>>> >>>>> On 16. Jan 2020, at 15:09, Dominik Süß <[email protected]> >>>>> wrote: >>>>> >>>>> Based on the last changes in the filevault plugin some of the setups >>>>> that were in place are no longer working - this is specifically by >>>>> enforcing to not embed osgi bundles and configs. >>>>> >>>>> IMHO not if one explicitly sets the packageType to 'mixed' >>>>> >>>>> >>>>> My suggestion to make sure this is not breaking each and >>>>> every existing setup would be 2fold >>>>> a) make sure we can override those rules to run it in some kind of >>>>> legacy mode >>>>> >>>>> That is implicitly done by setting the packageType to 'mixed'. >>>>> But maybe I just don't understand correctly. I would help a lot to >>>>> discuss on a concrete example. Can you share one which is no longer >>>>> working? >>>>> >>>>> b) we optimize the tooling to support a merging of container packages >>>>> into one container or provide a nesting capability >>>>> >>>>> There was a question just raised today on what a container is supposed >>>>> to support. According to >>>>> https://issues.apache.org/jira/browse/JCRVLT-170 neither nested >>>>> container packages nor content packages. >>>>> What exactly do you propose? Can you formulate your proposal in a >>>>> dedicated JIRA ticket for FileVault? >>>>> >>>>> >>>>> As an endresult it should be possible to slowly migrate existing >>>>> structures (that currently are postprocessed by a converter) towards >>>>> something that natively enables a suitable structure while the >>>>> codestructures would not need to change beyond the pom definitions. >>>>> >>>>> With 'mixed' this is already the case IMHO, other changes might need >>>>> further refactorings.... >>>>> >>>>> >>>>> Right now in most cases because of compatiblity reasons with the osgi >>>>> installer the osgi configs and osgi bundles are part of the application >>>>> packages. This is failing for almost everyone - and if we want them to be >>>>> moved out into the container we must make sure that the structures can be >>>>> reflected. >>>>> >>>>> Not necessarily. Even the OSGi installer can deal with container >>>>> packages containing both app subpackages and bundles/configs. >>>>> >>>>> >>>>> As a first step I would suggest we establish some property to >>>>> establish some legacy mode for application packages and as a next step >>>>> make >>>>> sure we get container packages that allow the same that is currently >>>>> partially reflected by application packages with mutliple install folders >>>>> (e.g. with runmode support). >>>>> >>>>> What exactly is wrong about 'mixed'? >>>>> >>>>> >>>>> As long as we want to support the sling installer for a while the >>>>> container structure itself would need be able to locate the binaries and >>>>> packages in locations the installer is aware of (install and config >>>>> folders) or we need to patch the sling installer to be able to process the >>>>> new container packages accurately >>>>> >>>>> Container packages allow exactly that IMHO >>>>> >>>>> >>>>> Cheers >>>>> Dominik >>>>> >>>>> Konrad >>>>> >>>> >>>> >>> >> >
