Cool, gonna take a look. Regards JB
Le 26 sept. 2018 à 11:06, à 11:06, Thomas Draier <tdra...@jahia.com> a écrit: >Sorry, I forgot to mention : it's pushed in the patch branch : >https://github.com/apache/incubator-unomi/tree/patch > > >On Wed, Sep 26, 2018 at 3:14 PM Thomas Draier <tdra...@jahia.com> >wrote: > >> Hi, >> >> I've committed the changes for the patch. The files need to be put in >the >> same folder as the definitions ( META-INF/cxs/xxx ), and named with a >> suffix -patch.json . They will be deployed only once, if the >definitions >> was already there. Each patch has an id, so if you need to deploy a >new >> patch, just use a new id. The list of deployed patches is stored in >ES. >> Patches can also be manually executed with the >"unomi:deploy-definition" >> karaf command, or through a rest endpoint at /cxs/patches/apply . >I've >> removed the behaviour with -SNAPSHOT - definitions are never >redeployed, >> even in snapshot versions, so you'll need to explicitely provide a >patch to >> override. >> >> The syntax is slightly different from what has been sent previously : >> >> { >> "itemId": "firstName-patch1", >> "patchedItemId": "firstName", >> "operation": "patch", >> "data": [ >> >> >> { >> "op": "replace", >> "path": "/defaultValue", >> "value": "foo" >> } >> ] >> } >> >> For an override : >> >> { >> "itemId": "gender-patch1", >> "patchedItemId": "gender", >> "operation": "override", >> "data": { >> "metadata": { >> "id": "gender", >> "name": "Gender", >> >> "systemTags": [ >> "properties", >> "profileProperties" >> ] >> }, >> "type": "string", >> "defaultValue": "", >> "automaticMappingsFrom": [ ], >> >> "rank": "105.0" >> } >> } >> >> Or a remove : >> >> { >> "itemId": "firstName-patch3", >> "patchedItemId": "firstName", >> "operation": "remove" >> } >> >> >> Regards >> >> >> >> >> On Mon, Sep 24, 2018 at 11:00 AM Thomas Draier <tdra...@jahia.com> >wrote: >> >>> Hi, >>> >>> I did a quick test with json patch and actually it looks quite easy >to >>> integrate and seems to fill more use cases than just a simple >"override". >>> However I think it can still be useful to have a complete override >and also >>> the possisbility to completely remove the definition, so I was >thinking of >>> some "migration" or "update" json file wrapping the patch, within >the same >>> folder as the definitions : >>> >>> { >>> "itemId": "firstName", >>> "updateId": "firstName-patch1", >>> "operation": "patch", >>> "patch": [ >>> { >>> "op": "replace", "path": "/defaultValue", "value": "foo" >>> } >>> ] >>> } >>> >>> >>> A removal : >>> >>> { >>> "itemId": "firstName", >>> "updateId": "firstName-patch1", >>> "operation": "remove" >>> } >>> >>> Or a complete override : >>> >>> { >>> "itemId": "firstName", >>> "updateId": "firstName-patch1", >>> "operation": "override", >>> "definition": { >>> "metadata": { >>> "id": "firstName", >>> "name": "First name", >>> "systemTags": [ >>> "properties", >>> "profileProperties", >>> "basicProfileProperties", >>> "personalIdentifierProperties" >>> ] >>> }, >>> "type": "string", >>> "defaultValue": "", >>> "automaticMappingsFrom": [ ], >>> "rank": "101.0" >>> } >>> } >>> >>> >>> This could actually be extended with other type of operation. >>> >>> For now I do store "updateId" in the metadata of the item, so that >it's >>> not executed multiple times. But we have have an issue with the >delete if >>> the item is recreated after.. Maybe I should store the list of >updates in a >>> new structure ? That could also be accessible with karaf commands, >to see >>> which patch have been executed and reexecute them if needed. >>> >>> Thomas >>> >>> >>> On Tue, Sep 18, 2018 at 11:46 AM Francois Papon < >>> francois.pa...@openobject.fr> wrote: >>> >>>> Hi, >>>> >>>> Very interesting flow :) >>>> >>>> What is the format to update the definitions, it's just a new JSON, >is >>>> there some directives about the changes ? >>>> >>>> For example, in Karaf we have something like this to override >>>> configuration files : >>>> >>>> <edits> >>>> <edit> >>>> <file>config.properties</file> >>>> <operation>put</operation> >>>> <key>karaf.framework</key> >>>> <value>equinox</value> >>>> </edit> >>>> <edit> >>>> <file>config.properties</file> >>>> <operation>extend</operation> >>>> <key>org.osgi.framework.system.capabilities</key> >>>> <value>my-magic-capability</value> >>>> </edit> >>>> <edit> >>>> <file>config.properties</file> >>>> <operation>remove</operation> >>>> <key>org.apache.karaf.security.providers</key> >>>> </edit> >>>> </edits> >>>> >>>> May be a solution is a file with a patch description of the changes >? >>>> >>>> regards, >>>> >>>> François Papon >>>> fpa...@apache.org >>>> >>>> Le 18/09/2018 à 13:38, Serge Huber a écrit : >>>> > Not sure if you are talking about the patching system, but if yes >you >>>> > would provide a patch file that removes all the contents of file. >>>> > >>>> > If we are just using versions it is indeed a bit more tricky. We >would >>>> > also need to have a way to list the definitions to remove. >>>> > >>>> > Regards, >>>> > Serge... >>>> > >>>> > On Tue, Sep 18, 2018 at 11:23 AM Romain Gauthier ><rgauth...@jahia.com> >>>> wrote: >>>> >> Hi Serge, >>>> >> >>>> >> In that context, how would we manage objects deletion from a >plugin? >>>> >> >>>> >> Thanks! >>>> >> >>>> >> Romain >>>> >> >>>> >> On Tue, Sep 18, 2018 at 10:45 AM Serge Huber <shu...@apache.org> >>>> wrote: >>>> >> >>>> >>> Sorry, I didn't get around to this discussion but I think that >>>> >>> versions would probably be cleaner. I was even looking at >solutions >>>> >>> such as JSON patch >(https://github.com/java-json-tools/json-patch) >>>> to >>>> >>> be able to perform modifications on parts of definitions >instead of >>>> >>> having to deploy a whole definition again. >>>> >>> >>>> >>> We had discussions about adding lots of metadata, including >last >>>> >>> modification dates, maybe even hashes for the definition files, >to >>>> >>> know if a definition has been changed or not. >>>> >>> >>>> >>> I think that one of the advantages of the patch idea is that >you >>>> could >>>> >>> try to patch something if it's compatible, and if it's not you >would >>>> >>> not deploy it. But this idea would mostly be for use cases >where you >>>> >>> are interested in modifying something that comes out of the box >in >>>> >>> Unomi, not for upgrading a plugin. >>>> >>> >>>> >>> With versions at least we could also track if a definition is >on the >>>> >>> "default" version or has been modified somehow. >>>> >>> >>>> >>> For the cfg idea it feels weird to use a config file that way, >maybe >>>> >>> we should put the marker file elsewhere but to be honest I >don't have >>>> >>> a better idea :) >>>> >>> >>>> >>> Would love some input from JB or other Karaf experts :) >>>> >>> >>>> >>> Regards, >>>> >>> Serge... >>>> >>> >>>> >>> On Tue, Sep 18, 2018 at 9:51 AM Thomas Draier ><tdra...@jahia.com> >>>> wrote: >>>> >>>> Hi, >>>> >>>> >>>> >>>> I have to check, but the idea would be to be able to package >this >>>> cfg >>>> >>> file >>>> >>>> beside the bundle. I'm not familiar with unomi:migrate >command, but >>>> we >>>> >>> can >>>> >>>> imagine that this command would extract and deploy the cfg >file >>>> from the >>>> >>>> bundle - which would make the packaging easier (cfg directly >inside >>>> the >>>> >>>> plugin). The thing is that we need to be able to remove the >file >>>> once the >>>> >>>> definition has been "overridden" so that it's only done once, >so >>>> the cfg >>>> >>>> file should not be deployed automatically on each startup. >>>> >>>> >>>> >>>> Another idea I had was to store sort of "version" of the >definition >>>> >>> inside >>>> >>>> the metadata, and check if the json file has a greater >"version" >>>> defined. >>>> >>>> That makes things even more easier when developing plugin and >looks >>>> a >>>> >>>> little more clean, but I don't know if it would cover >correctly all >>>> use >>>> >>>> cases. >>>> >>>> >>>> >>>> We should also extends the DeployDefinition shell command to >be >>>> able to >>>> >>>> redeploy all definitions of a bundle. >>>> >>>> >>>> >>>> Thomas >>>> >>>> >>>> >>>> >>>> >>>> On Tue, Sep 18, 2018 at 1:33 AM Romain Gauthier < >>>> rgauth...@jahia.com> >>>> >>> wrote: >>>> >>>>> Hello Thomas, >>>> >>>>> >>>> >>>>> Nice proposition.How would it work for developers that are >building >>>> >>>>> plugins? Would we have to add our own cfg file or somehow >update >>>> the >>>> >>> cfg >>>> >>>>> file that you want to add? >>>> >>>>> >>>> >>>>> Since there is already a "unomi:migrate" command would you >reuse >>>> that >>>> >>>>> trigger to update / delete the definitions of the objects? >>>> >>>>> >>>> >>>>> Same question regarding the plugins, should we also implement >>>> "migrate" >>>> >>>>> command? >>>> >>>>> >>>> >>>>> Thanks, >>>> >>>>> >>>> >>>>> Romain >>>> >>>>> >>>> >>>>> On Thu, Sep 13, 2018 at 6:22 PM Thomas Draier ><tdra...@jahia.com> >>>> >>> wrote: >>>> >>>>>> Hi, >>>> >>>>>> >>>> >>>>>> I'm having some issues with the deployment and redeployment >of the >>>> >>> json >>>> >>>>>> files containing default definitions (segments, properties, >>>> etc..). >>>> >>> Many >>>> >>>>>> services have these loadPredefined* methods which load data >into >>>> ES, >>>> >>>>> based >>>> >>>>>> on json definition files. These methods only load the >content >>>> once, >>>> >>> if >>>> >>>>> the >>>> >>>>>> object does not exist yet in ES. So if a bundle comes with >an >>>> update >>>> >>> of >>>> >>>>> one >>>> >>>>>> of these file, it will just be ignored. >>>> >>>>>> >>>> >>>>>> Actually there was one ticket about this : >>>> >>>>>> https://issues.apache.org/jira/browse/UNOMI-182 , where >>>> definitions >>>> >>> are >>>> >>>>>> always redeployed when a bundle is in "snapshot" version - >but >>>> never >>>> >>> for >>>> >>>>> a >>>> >>>>>> release version. So that's nice when you are in development, >but >>>> not >>>> >>>>>> possible when upgrading a production environment. >>>> >>>>>> >>>> >>>>>> There also a console command to manually deploy a definition >- >>>> which >>>> >>> is >>>> >>>>>> also nice but cannot be automated in a migration. >>>> >>>>>> >>>> >>>>>> Of course there's a reason - these objects can be modified >>>> >>> afterwards by >>>> >>>>>> using the API, and so restarting should not overwrite the >>>> >>> modifications >>>> >>>>> by >>>> >>>>>> reimporting the file. Even if a module is updated and object >is >>>> >>> modified, >>>> >>>>>> the user may not want to overwrite his changes. >>>> >>>>>> >>>> >>>>>> However in some cases the developer may want to force the >update >>>> of >>>> >>> some >>>> >>>>>> specific objects ( it actually happened to us ). I'm looking >for >>>> some >>>> >>>>>> additional solution to force redeployment of a definition. >>>> >>>>>> >>>> >>>>>> So far i was thinking of adding a cfg file, which would list >the >>>> >>>>> properties >>>> >>>>>> to redeploy. This file would be removed once the properties >are >>>> ><https://maps.google.com/?q=ould+be+removed+once+the+properties+are&entry=gmail&source=g> >>>> >>> updated, >>>> >>>>> so >>>> >>>>>> that they are reloaded only once and not on every startup. >>>> >>>>>> >>>> >>>>>> As a bonus, I would also need to be able to remove >definitions the >>>> >>> same >>>> >>>>>> way. There's no way to do that now except using the REST API >, it >>>> >>> could >>>> >>>>> be >>>> >>>>>> good to also have a console command and to be able to remove >>>> >>> definitions >>>> >>>>>> with the same cfg file. >>>> >>>>>> >>>> >>>>>> What do you think .. ? >>>> >>>>>> >>>> >>>>>> Regards, >>>> >>>>>> >>>> >>>>> >>>> >>>>> -- >>>> >>>>> Romain Gauthier >>>> >>>>> Product Manager at Jahia >>>> >>>>> + 33 6 20 18 35 25 <+33%206%2020%2018%2035%2025> >>>> <+33%206%2020%2018%2035%2025> >>>> >>>>> 8 Rue du sentier | 75002 Paris | France >>>> >>>>> < >>>> >https://www.jahia.com/about-us/news-events/events-webinars/jahiadays> >>>> >>>>> >>>> >> >>>> >> -- >>>> >> Romain Gauthier >>>> >> Product Manager at Jahia >>>> >> + 33 6 20 18 35 25 <+33%206%2020%2018%2035%2025> >>>> >> 8 Rue du sentier | 75002 Paris | France >>>> >> ><https://www.jahia.com/about-us/news-events/events-webinars/jahiadays >>>> > >>>> >>>> >>>>
