Hi,
I've committed the changes for the patch. The files need to be put in the
same folder as the definitions ( META-INF/cxs/xxx ), and named with a
suffix -patch.json . They will be deployed only once, if the definitions
was already there. Each patch has an id, so if you need to deploy a new
patch, just use a new id. The list of deployed patches is stored in ES.
Patches can also be manually executed with the "unomi:deploy-definition"
karaf command, or through a rest endpoint at /cxs/patches/apply . I've
removed the behaviour with -SNAPSHOT - definitions are never redeployed,
even in snapshot versions, so you'll need to explicitely provide a patch to
override.
The syntax is slightly different from what has been sent previously :
{
"itemId": "firstName-patch1",
"patchedItemId": "firstName",
"operation": "patch",
"data": [
{
"op": "replace",
"path": "/defaultValue",
"value": "foo"
}
]
}
For an override :
{
"itemId": "gender-patch1",
"patchedItemId": "gender",
"operation": "override",
"data": {
"metadata": {
"id": "gender",
"name": "Gender",
"systemTags": [
"properties",
"profileProperties"
]
},
"type": "string",
"defaultValue": "",
"automaticMappingsFrom": [ ],
"rank": "105.0"
}
}
Or a remove :
{
"itemId": "firstName-patch3",
"patchedItemId": "firstName",
"operation": "remove"
}
Regards
On Mon, Sep 24, 2018 at 11:00 AM Thomas Draier <[email protected]> wrote:
> Hi,
>
> I did a quick test with json patch and actually it looks quite easy to
> integrate and seems to fill more use cases than just a simple "override".
> However I think it can still be useful to have a complete override and also
> the possisbility to completely remove the definition, so I was thinking of
> some "migration" or "update" json file wrapping the patch, within the same
> folder as the definitions :
>
> {
> "itemId": "firstName",
> "updateId": "firstName-patch1",
> "operation": "patch",
> "patch": [
> {
> "op": "replace", "path": "/defaultValue", "value": "foo"
> }
> ]
> }
>
>
> A removal :
>
> {
> "itemId": "firstName",
> "updateId": "firstName-patch1",
> "operation": "remove"
> }
>
> Or a complete override :
>
> {
> "itemId": "firstName",
> "updateId": "firstName-patch1",
> "operation": "override",
> "definition": {
> "metadata": {
> "id": "firstName",
> "name": "First name",
> "systemTags": [
> "properties",
> "profileProperties",
> "basicProfileProperties",
> "personalIdentifierProperties"
> ]
> },
> "type": "string",
> "defaultValue": "",
> "automaticMappingsFrom": [ ],
> "rank": "101.0"
> }
> }
>
>
> This could actually be extended with other type of operation.
>
> For now I do store "updateId" in the metadata of the item, so that it's
> not executed multiple times. But we have have an issue with the delete if
> the item is recreated after.. Maybe I should store the list of updates in a
> new structure ? That could also be accessible with karaf commands, to see
> which patch have been executed and reexecute them if needed.
>
> Thomas
>
>
> On Tue, Sep 18, 2018 at 11:46 AM Francois Papon <
> [email protected]> wrote:
>
>> Hi,
>>
>> Very interesting flow :)
>>
>> What is the format to update the definitions, it's just a new JSON, is
>> there some directives about the changes ?
>>
>> For example, in Karaf we have something like this to override
>> configuration files :
>>
>> <edits>
>> <edit>
>> <file>config.properties</file>
>> <operation>put</operation>
>> <key>karaf.framework</key>
>> <value>equinox</value>
>> </edit>
>> <edit>
>> <file>config.properties</file>
>> <operation>extend</operation>
>> <key>org.osgi.framework.system.capabilities</key>
>> <value>my-magic-capability</value>
>> </edit>
>> <edit>
>> <file>config.properties</file>
>> <operation>remove</operation>
>> <key>org.apache.karaf.security.providers</key>
>> </edit>
>> </edits>
>>
>> May be a solution is a file with a patch description of the changes ?
>>
>> regards,
>>
>> François Papon
>> [email protected]
>>
>> Le 18/09/2018 à 13:38, Serge Huber a écrit :
>> > Not sure if you are talking about the patching system, but if yes you
>> > would provide a patch file that removes all the contents of file.
>> >
>> > If we are just using versions it is indeed a bit more tricky. We would
>> > also need to have a way to list the definitions to remove.
>> >
>> > Regards,
>> > Serge...
>> >
>> > On Tue, Sep 18, 2018 at 11:23 AM Romain Gauthier <[email protected]>
>> wrote:
>> >> Hi Serge,
>> >>
>> >> In that context, how would we manage objects deletion from a plugin?
>> >>
>> >> Thanks!
>> >>
>> >> Romain
>> >>
>> >> On Tue, Sep 18, 2018 at 10:45 AM Serge Huber <[email protected]>
>> wrote:
>> >>
>> >>> Sorry, I didn't get around to this discussion but I think that
>> >>> versions would probably be cleaner. I was even looking at solutions
>> >>> such as JSON patch (https://github.com/java-json-tools/json-patch) to
>> >>> be able to perform modifications on parts of definitions instead of
>> >>> having to deploy a whole definition again.
>> >>>
>> >>> We had discussions about adding lots of metadata, including last
>> >>> modification dates, maybe even hashes for the definition files, to
>> >>> know if a definition has been changed or not.
>> >>>
>> >>> I think that one of the advantages of the patch idea is that you could
>> >>> try to patch something if it's compatible, and if it's not you would
>> >>> not deploy it. But this idea would mostly be for use cases where you
>> >>> are interested in modifying something that comes out of the box in
>> >>> Unomi, not for upgrading a plugin.
>> >>>
>> >>> With versions at least we could also track if a definition is on the
>> >>> "default" version or has been modified somehow.
>> >>>
>> >>> For the cfg idea it feels weird to use a config file that way, maybe
>> >>> we should put the marker file elsewhere but to be honest I don't have
>> >>> a better idea :)
>> >>>
>> >>> Would love some input from JB or other Karaf experts :)
>> >>>
>> >>> Regards,
>> >>> Serge...
>> >>>
>> >>> On Tue, Sep 18, 2018 at 9:51 AM Thomas Draier <[email protected]>
>> wrote:
>> >>>> Hi,
>> >>>>
>> >>>> I have to check, but the idea would be to be able to package this cfg
>> >>> file
>> >>>> beside the bundle. I'm not familiar with unomi:migrate command, but
>> we
>> >>> can
>> >>>> imagine that this command would extract and deploy the cfg file from
>> the
>> >>>> bundle - which would make the packaging easier (cfg directly inside
>> the
>> >>>> plugin). The thing is that we need to be able to remove the file
>> once the
>> >>>> definition has been "overridden" so that it's only done once, so the
>> cfg
>> >>>> file should not be deployed automatically on each startup.
>> >>>>
>> >>>> Another idea I had was to store sort of "version" of the definition
>> >>> inside
>> >>>> the metadata, and check if the json file has a greater "version"
>> defined.
>> >>>> That makes things even more easier when developing plugin and looks a
>> >>>> little more clean, but I don't know if it would cover correctly all
>> use
>> >>>> cases.
>> >>>>
>> >>>> We should also extends the DeployDefinition shell command to be able
>> to
>> >>>> redeploy all definitions of a bundle.
>> >>>>
>> >>>> Thomas
>> >>>>
>> >>>>
>> >>>> On Tue, Sep 18, 2018 at 1:33 AM Romain Gauthier <[email protected]
>> >
>> >>> wrote:
>> >>>>> Hello Thomas,
>> >>>>>
>> >>>>> Nice proposition.How would it work for developers that are building
>> >>>>> plugins? Would we have to add our own cfg file or somehow update the
>> >>> cfg
>> >>>>> file that you want to add?
>> >>>>>
>> >>>>> Since there is already a "unomi:migrate" command would you reuse
>> that
>> >>>>> trigger to update / delete the definitions of the objects?
>> >>>>>
>> >>>>> Same question regarding the plugins, should we also implement
>> "migrate"
>> >>>>> command?
>> >>>>>
>> >>>>> Thanks,
>> >>>>>
>> >>>>> Romain
>> >>>>>
>> >>>>> On Thu, Sep 13, 2018 at 6:22 PM Thomas Draier <[email protected]>
>> >>> wrote:
>> >>>>>> Hi,
>> >>>>>>
>> >>>>>> I'm having some issues with the deployment and redeployment of the
>> >>> json
>> >>>>>> files containing default definitions (segments, properties, etc..).
>> >>> Many
>> >>>>>> services have these loadPredefined* methods which load data into
>> ES,
>> >>>>> based
>> >>>>>> on json definition files. These methods only load the content once,
>> >>> if
>> >>>>> the
>> >>>>>> object does not exist yet in ES. So if a bundle comes with an
>> update
>> >>> of
>> >>>>> one
>> >>>>>> of these file, it will just be ignored.
>> >>>>>>
>> >>>>>> Actually there was one ticket about this :
>> >>>>>> https://issues.apache.org/jira/browse/UNOMI-182 , where
>> definitions
>> >>> are
>> >>>>>> always redeployed when a bundle is in "snapshot" version - but
>> never
>> >>> for
>> >>>>> a
>> >>>>>> release version. So that's nice when you are in development, but
>> not
>> >>>>>> possible when upgrading a production environment.
>> >>>>>>
>> >>>>>> There also a console command to manually deploy a definition -
>> which
>> >>> is
>> >>>>>> also nice but cannot be automated in a migration.
>> >>>>>>
>> >>>>>> Of course there's a reason - these objects can be modified
>> >>> afterwards by
>> >>>>>> using the API, and so restarting should not overwrite the
>> >>> modifications
>> >>>>> by
>> >>>>>> reimporting the file. Even if a module is updated and object is
>> >>> modified,
>> >>>>>> the user may not want to overwrite his changes.
>> >>>>>>
>> >>>>>> However in some cases the developer may want to force the update of
>> >>> some
>> >>>>>> specific objects ( it actually happened to us ). I'm looking for
>> some
>> >>>>>> additional solution to force redeployment of a definition.
>> >>>>>>
>> >>>>>> So far i was thinking of adding a cfg file, which would list the
>> >>>>> properties
>> >>>>>> to redeploy. This file would be removed once the properties are
>> <https://maps.google.com/?q=ould+be+removed+once+the+properties+are&entry=gmail&source=g>
>> >>> updated,
>> >>>>> so
>> >>>>>> that they are reloaded only once and not on every startup.
>> >>>>>>
>> >>>>>> As a bonus, I would also need to be able to remove definitions the
>> >>> same
>> >>>>>> way. There's no way to do that now except using the REST API , it
>> >>> could
>> >>>>> be
>> >>>>>> good to also have a console command and to be able to remove
>> >>> definitions
>> >>>>>> with the same cfg file.
>> >>>>>>
>> >>>>>> What do you think .. ?
>> >>>>>>
>> >>>>>> Regards,
>> >>>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Romain Gauthier
>> >>>>> Product Manager at Jahia
>> >>>>> + 33 6 20 18 35 25 <+33%206%2020%2018%2035%2025>
>> <+33%206%2020%2018%2035%2025>
>> >>>>> 8 Rue du sentier | 75002 Paris | France
>> >>>>> <
>> https://www.jahia.com/about-us/news-events/events-webinars/jahiadays>
>> >>>>>
>> >>
>> >> --
>> >> Romain Gauthier
>> >> Product Manager at Jahia
>> >> + 33 6 20 18 35 25 <+33%206%2020%2018%2035%2025>
>> >> 8 Rue du sentier | 75002 Paris | France
>> >> <https://www.jahia.com/about-us/news-events/events-webinars/jahiadays>
>>
>>
>>
