I created https://issues.apache.org/jira/browse/JCRVLT-401 <https://issues.apache.org/jira/browse/JCRVLT-401> for a clarification of allowed contents of a container package.
> On 21. Jan 2020, at 16:07, Dominik Süß <[email protected]> wrote: > > Yes I get that perspective and I was a bit surprised to read it like that - I > assume when toby wrote it down the need for mutable content as seeding > content wasn't really present (or an alternative approach discussed like in > sling with repoinit - yet we still have the need to support those for a > while). > The question is now if relaxing the definition would be the right way to go - > I guess we don#t have any logic depending on those constraints right now > beside of the analysers failing on those. So iterating over the definition > might be the right thing to do. > > Not sure if Toby has a specific opinion on that one. > > Cheers > Dominik > > > > On Tue, Jan 21, 2020 at 3:40 PM Konrad Windszus <[email protected] > <mailto:[email protected]>> wrote: > That is not what is stated in JCRVLT-170. > It is a bit tricky though because JCRVLT-170 combines blacklisting and > whitelisting rules and doesn't say what the baseline is. > ... > Container Packages > As for the container / deployment packages they have the characteristics that: > they don't contain any content > they contain a set of application content packages > they don't have dependencies on other deployment packages (i.e. the > dependencies are implicit given by the relations of the included artifacts) > > .... > > I think everyone would derive from this info that "content" type content > package are not allowed, because only "application" content packages are > mentioned here. > But I am fine with a clarification, that content packages may be contained in > there as well. For that we should have a new FileVault ticket! > > Then we don't necessarily need to have another package type. But we should > then also clarify if nesting of container packages should be allowed or not. > IMHO this would be a useful feature and I don't see why we should disallow > that. > > Konrad > > >> On 21. Jan 2020, at 15:21, Dominik Süß <[email protected] >> <mailto:[email protected]>> wrote: >> >> well TBH - the original understanding I had about container was what I was >> describing with the collection with the difference that it has an identity >> (yet it isn't defined anywhere and not declared that you can have >> dependencies on containers. Core concept that we discussed for JCRVLT-170 is >> that the containers would never directly install anything in the repository >> directly but is a container for packages and other installables as well as >> meta that may be installed by what ever processes them. This is the origin >> of not having "any" content in them - basically, anything that is to be >> installed would have an own identity so you can declare to any content or >> installable artifact by referencing their specific identity. >> >> So if we refine the definition of container and relax it what is currently >> enforced with the analyzers today we would be able to follow this deployment >> scheme and are able to enforce to not base on antipatterns purely by >> blocking mixed packages for scenarios with composite nodestore (btw. being >> able to configure the analysers to enforce certain packageTypes would be >> really useful for that). >> >> WDYT? >> >> Cheers >> Dominik >> >> >> >> On Tue, Jan 21, 2020 at 1:50 PM Konrad Windszus <[email protected] >> <mailto:[email protected]>> wrote: >> Hi Dominik, >> This is still very vague to me. >> >> What is your idea about the current packageType: container (which is a >> content-package with an identity)? What should be allowed in there? >> What about the new propsed type PackageType.Collection? How does that relate >> to the existing packageType? Does it replace it? >> >> Can you try to answer those questions, so that everyone can get a clearer >> picture at what you are aiming? >> >> I would strongly suggest not to come up with something which is not a >> content-package itself, because it seems no one has the time to implement >> new features in FileVault (like. a package which can only be unwrapped, but >> no one can depend on). >> Let's rather enforce this with some validators: >> https://jackrabbit.apache.org/filevault/validation.html >> <https://jackrabbit.apache.org/filevault/validation.html>. Way less effort >> IMHO and much better backwards compatible! >> Thanks, >> Konrad >> >> >>> On 21. Jan 2020, at 09:20, Dominik Süß <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hi Konrad, >>> >>> what worries me about mixed is that is doesn't enforce any separation which >>> is the scenario that we want to prevent. >>> A mixed content-package has an identity which another package can depend on >>> - yet in a composite nodestore the mutable and immutable parts are >>> installed at different times making it impossible to model the dependencies >>> of immutable packages against immutable content tha would be served from a >>> mixed package. >>> This is why supporting mixed content packages cannot work in a composite >>> nodestore scenario. >>> >>> Yes it is possible to abuse mixed packages as what I propose as a >>> collection type - yet no one prevents you from adding content directly to >>> that package which in turn might require to set direct dependencies for it >>> making it again something you cannot simply drop >>> >>> Therefore the suggestion is to have a type that allows bundling together >>> everything you want to eventually get installed into the system, where all >>> packages that have an identity are clearly either application or content >>> (so separation of mutable and immutable) so the dependency ordering is no >>> longer ambiguous between deployment (change of immutable content state) and >>> runtime (change of mutable state). >>> >>> I understand that the perspective you have is from what is the state when >>> you would upload this to packagemanager - and from my pov this would only >>> alow you to unfold that package so you have all the contained packages >>> available. And yes we might even add a ui / rest api feature to triggere >>> installation for all of the contained packages automatically (e.g. the >>> mutable subset or all if there is no CNS) yet the package itself would not >>> have an installation state on its own as it is only a shipping container. >>> >>> Does this clarify my thoughts about this a bit better? >>> >>> When it comes to nesting of packages of same type it is ambiguous so we >>> might want to clarify this for sure. >>> >>> Cheers >>> Dominik >>> >>> On Tue, Jan 21, 2020 at 8:47 AM Konrad Windszus <[email protected] >>> <mailto:[email protected]>> wrote: >>> Hi Dominik, >>> I am not getting exactly what you propose here: >>> Which types should be allowed in "container"? >>> a) If there is already a support for "content" type packages I don't see >>> the need for another type within "container" >>> b) If there is no support for nested content type packages, the question is >>> how to aggregate multiple content type packages together so that they are >>> actually installed via one big content-package? >>> >>> As I already said, I am not too much worried about a mixture of mutable and >>> immutable packages (that can still be done via 'mixed') but if we support >>> that in FileVault via a dedicated type, this should be installable as well >>> (i.e. be a regular content package). >>> Having a pure package format without supporting installation sounds like a >>> unnecessary limitation (even having AEM as a Cloud Service in mind) because >>> a lot of consumers right now don't care about mutable/immutable (i.e. old >>> AEM instances not leveraging the Composite Node Store). >>> As I already said, I would like to see a clarification ticket for >>> JCRVLT-170, because that clearly does neither mention nested containers nor >>> content type packages within containers (therefore everyone assumes, those >>> are forbidden) >>> >>> Thanks, >>> Konrad >>> >>>> On 21. Jan 2020, at 08:34, Dominik Süß <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> Hi Konrad, >>>> >>>> I get your point yet that was not really the origin. Probably we can split >>>> this off in another way. >>>> Let's introduce a PackageType.Collection - basically this would supposed >>>> to be never installed but is just a shipping unit holding a set of other >>>> packages. These could either be in a flat list or for compat reasons also >>>> be placed in an /etc/packages structure. >>>> This packageType would be excluded of evaluation and is supposed to never >>>> be installed directly. It basically is a zip file acting as a wrapper >>>> around a collection of other packages. >>>> No further semantics beside the fact that it has no vault identity on its >>>> own and won't be visible after being installed - it's a pure shipping unit. >>>> >>>> Cheers >>>> Dominik >>>> >>>> On Thu, Jan 16, 2020 at 5:21 PM Konrad Windszus <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> Hi Dominik, >>>> For me container was always immutable by only embedding immutable content >>>> (bundles, config, app-packages, as indicated in JCRVLT-170) >>>> Especially when we allow nesting of containers, we should probably think >>>> of two different container types: >>>> - container (immutable), with the current restrictions, but in addition >>>> allows nested containers (immutable) >>>> - content-container (mutable), only allows content and content-container >>>> sub packages >>>> >>>> That way we have a clean separation even on the container level. >>>> Whether we need one container which allows embedding both container >>>> (immutable) and content-container (mutable), I am not sure about. WDYT? >>>> Also not very sure about the naming of such a construct... >>>> >>>> Konrad >>>> >>>>> On 16. Jan 2020, at 16:40, Dominik Süß <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> Hi Konrad, >>>>> >>>>> mixed is a legacy structure that basically allows all kinds of things. >>>>> What we would want to avoid is mixing content and application content in >>>>> the same package - building a container that allow application and >>>>> content package to be grouped together (as installable entities) but >>>>> making sure they are not mashed together helps to install both at >>>>> distinct times. If you check setups working with the composite nodestore >>>>> you would have 2 different times at which the application and the content >>>>> packages would be installed. Crucial is that containers basically don't >>>>> really define an own identity that you would be depending on - you should >>>>> only define dependencies on the actual installables . The problem of a >>>>> mixed package is that people start to add content to it and therefore you >>>>> have to define dependencies on it, if they then also contain application >>>>> on content there is ambiguity when this package actually is in an >>>>> installed state as the application part would need to be installed when >>>>> preparing the immutable part of the repository while the content only can >>>>> be installed at runtime with the mutable content being accessible. >>>>> Containers would be the tool to declare a grouping of entities that are >>>>> supposed to be installed while (optimally - not sure if we should >>>>> enforce) not playing a role in the dependency handling at all. >>>>> >>>>> Cheers >>>>> Dominik >>>>> >>>>> On Thu, Jan 16, 2020 at 3:33 PM Konrad Windszus <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> Hi Dominik, >>>>> better late feedback than no feedback at all. >>>>> Let me comment on your points >>>>> >>>>>> On 16. Jan 2020, at 15:09, Dominik Süß <[email protected] >>>>>> <mailto:[email protected]>> wrote: >>>>>> >>>>>> Based on the last changes in the filevault plugin some of the setups >>>>>> that were in place are no longer working - this is specifically by >>>>>> enforcing to not embed osgi bundles and configs. >>>>> IMHO not if one explicitly sets the packageType to 'mixed' >>>>>> >>>>>> My suggestion to make sure this is not breaking each and every existing >>>>>> setup would be 2fold >>>>>> a) make sure we can override those rules to run it in some kind of >>>>>> legacy mode >>>>> That is implicitly done by setting the packageType to 'mixed'. >>>>> But maybe I just don't understand correctly. I would help a lot to >>>>> discuss on a concrete example. Can you share one which is no longer >>>>> working? >>>>> >>>>>> b) we optimize the tooling to support a merging of container packages >>>>>> into one container or provide a nesting capability >>>>> There was a question just raised today on what a container is supposed to >>>>> support. According to https://issues.apache.org/jira/browse/JCRVLT-170 >>>>> <https://issues.apache.org/jira/browse/JCRVLT-170> neither nested >>>>> container packages nor content packages. >>>>> What exactly do you propose? Can you formulate your proposal in a >>>>> dedicated JIRA ticket for FileVault? >>>>>> >>>>>> As an endresult it should be possible to slowly migrate existing >>>>>> structures (that currently are postprocessed by a converter) towards >>>>>> something that natively enables a suitable structure while the >>>>>> codestructures would not need to change beyond the pom definitions. >>>>> With 'mixed' this is already the case IMHO, other changes might need >>>>> further refactorings.... >>>>>> >>>>>> Right now in most cases because of compatiblity reasons with the osgi >>>>>> installer the osgi configs and osgi bundles are part of the application >>>>>> packages. This is failing for almost everyone - and if we want them to >>>>>> be moved out into the container we must make sure that the structures >>>>>> can be reflected. >>>>> Not necessarily. Even the OSGi installer can deal with container packages >>>>> containing both app subpackages and bundles/configs. >>>>>> >>>>>> As a first step I would suggest we establish some property to establish >>>>>> some legacy mode for application packages and as a next step make sure >>>>>> we get container packages that allow the same that is currently >>>>>> partially reflected by application packages with mutliple install >>>>>> folders (e.g. with runmode support). >>>>> What exactly is wrong about 'mixed'? >>>>>> >>>>>> As long as we want to support the sling installer for a while the >>>>>> container structure itself would need be able to locate the binaries and >>>>>> packages in locations the installer is aware of (install and config >>>>>> folders) or we need to patch the sling installer to be able to process >>>>>> the new container packages accurately >>>>> Container packages allow exactly that IMHO >>>>>> >>>>>> Cheers >>>>>> Dominik >>>>>> >>>>> Konrad >>>> >>> >> >
