It look's very nice :) regards,
François Papon [email protected] Le 26/09/2018 à 19:06, Thomas Draier a écrit : > Sorry, I forgot to mention : it's pushed in the patch branch : > https://github.com/apache/incubator-unomi/tree/patch > > > On Wed, Sep 26, 2018 at 3:14 PM Thomas Draier <[email protected]> wrote: > >> Hi, >> >> I've committed the changes for the patch. The files need to be put in the >> same folder as the definitions ( META-INF/cxs/xxx ), and named with a >> suffix -patch.json . They will be deployed only once, if the definitions >> was already there. Each patch has an id, so if you need to deploy a new >> patch, just use a new id. The list of deployed patches is stored in ES. >> Patches can also be manually executed with the "unomi:deploy-definition" >> karaf command, or through a rest endpoint at /cxs/patches/apply . I've >> removed the behaviour with -SNAPSHOT - definitions are never redeployed, >> even in snapshot versions, so you'll need to explicitely provide a patch to >> override. >> >> The syntax is slightly different from what has been sent previously : >> >> { >> "itemId": "firstName-patch1", >> "patchedItemId": "firstName", >> "operation": "patch", >> "data": [ >> >> >> { >> "op": "replace", >> "path": "/defaultValue", >> "value": "foo" >> } >> ] >> } >> >> For an override : >> >> { >> "itemId": "gender-patch1", >> "patchedItemId": "gender", >> "operation": "override", >> "data": { >> "metadata": { >> "id": "gender", >> "name": "Gender", >> >> "systemTags": [ >> "properties", >> "profileProperties" >> ] >> }, >> "type": "string", >> "defaultValue": "", >> "automaticMappingsFrom": [ ], >> >> "rank": "105.0" >> } >> } >> >> Or a remove : >> >> { >> "itemId": "firstName-patch3", >> "patchedItemId": "firstName", >> "operation": "remove" >> } >> >> >> Regards >> >> >> >> >> On Mon, Sep 24, 2018 at 11:00 AM Thomas Draier <[email protected]> wrote: >> >>> Hi, >>> >>> I did a quick test with json patch and actually it looks quite easy to >>> integrate and seems to fill more use cases than just a simple "override". >>> However I think it can still be useful to have a complete override and also >>> the possisbility to completely remove the definition, so I was thinking of >>> some "migration" or "update" json file wrapping the patch, within the same >>> folder as the definitions : >>> >>> { >>> "itemId": "firstName", >>> "updateId": "firstName-patch1", >>> "operation": "patch", >>> "patch": [ >>> { >>> "op": "replace", "path": "/defaultValue", "value": "foo" >>> } >>> ] >>> } >>> >>> >>> A removal : >>> >>> { >>> "itemId": "firstName", >>> "updateId": "firstName-patch1", >>> "operation": "remove" >>> } >>> >>> Or a complete override : >>> >>> { >>> "itemId": "firstName", >>> "updateId": "firstName-patch1", >>> "operation": "override", >>> "definition": { >>> "metadata": { >>> "id": "firstName", >>> "name": "First name", >>> "systemTags": [ >>> "properties", >>> "profileProperties", >>> "basicProfileProperties", >>> "personalIdentifierProperties" >>> ] >>> }, >>> "type": "string", >>> "defaultValue": "", >>> "automaticMappingsFrom": [ ], >>> "rank": "101.0" >>> } >>> } >>> >>> >>> This could actually be extended with other type of operation. >>> >>> For now I do store "updateId" in the metadata of the item, so that it's >>> not executed multiple times. But we have have an issue with the delete if >>> the item is recreated after.. Maybe I should store the list of updates in a >>> new structure ? That could also be accessible with karaf commands, to see >>> which patch have been executed and reexecute them if needed. >>> >>> Thomas >>> >>> >>> On Tue, Sep 18, 2018 at 11:46 AM Francois Papon < >>> [email protected]> wrote: >>> >>>> Hi, >>>> >>>> Very interesting flow :) >>>> >>>> What is the format to update the definitions, it's just a new JSON, is >>>> there some directives about the changes ? >>>> >>>> For example, in Karaf we have something like this to override >>>> configuration files : >>>> >>>> <edits> >>>> <edit> >>>> <file>config.properties</file> >>>> <operation>put</operation> >>>> <key>karaf.framework</key> >>>> <value>equinox</value> >>>> </edit> >>>> <edit> >>>> <file>config.properties</file> >>>> <operation>extend</operation> >>>> <key>org.osgi.framework.system.capabilities</key> >>>> <value>my-magic-capability</value> >>>> </edit> >>>> <edit> >>>> <file>config.properties</file> >>>> <operation>remove</operation> >>>> <key>org.apache.karaf.security.providers</key> >>>> </edit> >>>> </edits> >>>> >>>> May be a solution is a file with a patch description of the changes ? >>>> >>>> regards, >>>> >>>> François Papon >>>> [email protected] >>>> >>>> Le 18/09/2018 à 13:38, Serge Huber a écrit : >>>>> Not sure if you are talking about the patching system, but if yes you >>>>> would provide a patch file that removes all the contents of file. >>>>> >>>>> If we are just using versions it is indeed a bit more tricky. We would >>>>> also need to have a way to list the definitions to remove. >>>>> >>>>> Regards, >>>>> Serge... >>>>> >>>>> On Tue, Sep 18, 2018 at 11:23 AM Romain Gauthier <[email protected]> >>>> wrote: >>>>>> Hi Serge, >>>>>> >>>>>> In that context, how would we manage objects deletion from a plugin? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> Romain >>>>>> >>>>>> On Tue, Sep 18, 2018 at 10:45 AM Serge Huber <[email protected]> >>>> wrote: >>>>>>> Sorry, I didn't get around to this discussion but I think that >>>>>>> versions would probably be cleaner. I was even looking at solutions >>>>>>> such as JSON patch (https://github.com/java-json-tools/json-patch) >>>> to >>>>>>> be able to perform modifications on parts of definitions instead of >>>>>>> having to deploy a whole definition again. >>>>>>> >>>>>>> We had discussions about adding lots of metadata, including last >>>>>>> modification dates, maybe even hashes for the definition files, to >>>>>>> know if a definition has been changed or not. >>>>>>> >>>>>>> I think that one of the advantages of the patch idea is that you >>>> could >>>>>>> try to patch something if it's compatible, and if it's not you would >>>>>>> not deploy it. But this idea would mostly be for use cases where you >>>>>>> are interested in modifying something that comes out of the box in >>>>>>> Unomi, not for upgrading a plugin. >>>>>>> >>>>>>> With versions at least we could also track if a definition is on the >>>>>>> "default" version or has been modified somehow. >>>>>>> >>>>>>> For the cfg idea it feels weird to use a config file that way, maybe >>>>>>> we should put the marker file elsewhere but to be honest I don't have >>>>>>> a better idea :) >>>>>>> >>>>>>> Would love some input from JB or other Karaf experts :) >>>>>>> >>>>>>> Regards, >>>>>>> Serge... >>>>>>> >>>>>>> On Tue, Sep 18, 2018 at 9:51 AM Thomas Draier <[email protected]> >>>> wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have to check, but the idea would be to be able to package this >>>> cfg >>>>>>> file >>>>>>>> beside the bundle. I'm not familiar with unomi:migrate command, but >>>> we >>>>>>> can >>>>>>>> imagine that this command would extract and deploy the cfg file >>>> from the >>>>>>>> bundle - which would make the packaging easier (cfg directly inside >>>> the >>>>>>>> plugin). The thing is that we need to be able to remove the file >>>> once the >>>>>>>> definition has been "overridden" so that it's only done once, so >>>> the cfg >>>>>>>> file should not be deployed automatically on each startup. >>>>>>>> >>>>>>>> Another idea I had was to store sort of "version" of the definition >>>>>>> inside >>>>>>>> the metadata, and check if the json file has a greater "version" >>>> defined. >>>>>>>> That makes things even more easier when developing plugin and looks >>>> a >>>>>>>> little more clean, but I don't know if it would cover correctly all >>>> use >>>>>>>> cases. >>>>>>>> >>>>>>>> We should also extends the DeployDefinition shell command to be >>>> able to >>>>>>>> redeploy all definitions of a bundle. >>>>>>>> >>>>>>>> Thomas >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Sep 18, 2018 at 1:33 AM Romain Gauthier < >>>> [email protected]> >>>>>>> wrote: >>>>>>>>> Hello Thomas, >>>>>>>>> >>>>>>>>> Nice proposition.How would it work for developers that are building >>>>>>>>> plugins? Would we have to add our own cfg file or somehow update >>>> the >>>>>>> cfg >>>>>>>>> file that you want to add? >>>>>>>>> >>>>>>>>> Since there is already a "unomi:migrate" command would you reuse >>>> that >>>>>>>>> trigger to update / delete the definitions of the objects? >>>>>>>>> >>>>>>>>> Same question regarding the plugins, should we also implement >>>> "migrate" >>>>>>>>> command? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Romain >>>>>>>>> >>>>>>>>> On Thu, Sep 13, 2018 at 6:22 PM Thomas Draier <[email protected]> >>>>>>> wrote: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I'm having some issues with the deployment and redeployment of the >>>>>>> json >>>>>>>>>> files containing default definitions (segments, properties, >>>> etc..). >>>>>>> Many >>>>>>>>>> services have these loadPredefined* methods which load data into >>>> ES, >>>>>>>>> based >>>>>>>>>> on json definition files. These methods only load the content >>>> once, >>>>>>> if >>>>>>>>> the >>>>>>>>>> object does not exist yet in ES. So if a bundle comes with an >>>> update >>>>>>> of >>>>>>>>> one >>>>>>>>>> of these file, it will just be ignored. >>>>>>>>>> >>>>>>>>>> Actually there was one ticket about this : >>>>>>>>>> https://issues.apache.org/jira/browse/UNOMI-182 , where >>>> definitions >>>>>>> are >>>>>>>>>> always redeployed when a bundle is in "snapshot" version - but >>>> never >>>>>>> for >>>>>>>>> a >>>>>>>>>> release version. So that's nice when you are in development, but >>>> not >>>>>>>>>> possible when upgrading a production environment. >>>>>>>>>> >>>>>>>>>> There also a console command to manually deploy a definition - >>>> which >>>>>>> is >>>>>>>>>> also nice but cannot be automated in a migration. >>>>>>>>>> >>>>>>>>>> Of course there's a reason - these objects can be modified >>>>>>> afterwards by >>>>>>>>>> using the API, and so restarting should not overwrite the >>>>>>> modifications >>>>>>>>> by >>>>>>>>>> reimporting the file. Even if a module is updated and object is >>>>>>> modified, >>>>>>>>>> the user may not want to overwrite his changes. >>>>>>>>>> >>>>>>>>>> However in some cases the developer may want to force the update >>>> of >>>>>>> some >>>>>>>>>> specific objects ( it actually happened to us ). I'm looking for >>>> some >>>>>>>>>> additional solution to force redeployment of a definition. >>>>>>>>>> >>>>>>>>>> So far i was thinking of adding a cfg file, which would list the >>>>>>>>> properties >>>>>>>>>> to redeploy. This file would be removed once the properties are >>>> <https://maps.google.com/?q=ould+be+removed+once+the+properties+are&entry=gmail&source=g> >>>>>>> updated, >>>>>>>>> so >>>>>>>>>> that they are reloaded only once and not on every startup. >>>>>>>>>> >>>>>>>>>> As a bonus, I would also need to be able to remove definitions the >>>>>>> same >>>>>>>>>> way. There's no way to do that now except using the REST API , it >>>>>>> could >>>>>>>>> be >>>>>>>>>> good to also have a console command and to be able to remove >>>>>>> definitions >>>>>>>>>> with the same cfg file. >>>>>>>>>> >>>>>>>>>> What do you think .. ? >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> Romain Gauthier >>>>>>>>> Product Manager at Jahia >>>>>>>>> + 33 6 20 18 35 25 <+33%206%2020%2018%2035%2025> >>>> <+33%206%2020%2018%2035%2025> >>>>>>>>> 8 Rue du sentier | 75002 Paris | France >>>>>>>>> < >>>> https://www.jahia.com/about-us/news-events/events-webinars/jahiadays> >>>>>> -- >>>>>> Romain Gauthier >>>>>> Product Manager at Jahia >>>>>> + 33 6 20 18 35 25 <+33%206%2020%2018%2035%2025> >>>>>> 8 Rue du sentier | 75002 Paris | France >>>>>> <https://www.jahia.com/about-us/news-events/events-webinars/jahiadays >>>> >>>>
