Re: [openstack-dev] [congress] generic push driver

[Afek, Ifat (Nokia - IL/Kfar Sava)]

From: Eric K 
Date: Tuesday, 9 January 2018 at 0:43

Hi Ifat,

From: "Afek, Ifat (Nokia - IL/Kfar Sava)" 
>
Date: Sunday, January 7, 2018 at 4:00 AM


Hi Eric,

I have two questions:


1.  An alarm is usually raised on a resource, and in Vitrage we can send 
you the details of that resource. Is there a way in Congress for the alarm to 
reference a resource that exists in another table? And what if the resource 
does not exist in Congress?
First, the columns I chose are just a minimal sample to illustrate the generic 
nature of the driver. In use with vitrage, we would probably also want to 
include columns such as `resource_id`. Does that address the need to reference 
a resource? That resource referenced by ID may or may not exist in another part 
of Congress. It would be the job of the policy to resolve references when 
taking appropriate actions. If referential integrity is needed, additional 
policy rules can be specified to catch breakage.

[Ifat] Ok, sounds good.

This brings up a related question I had about vitrage:
Looking at the vertex properties listed here: 
https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py#L17

Where can I find more information about the type and content of data in each 
property?

Exapmle:
- is the `resource` property an ID string or a python object reference?

[Ifat] Most of the properties are key-value strings on the vertex in the entity 
graph. The RESOURCE is a special property that is added on an alarm for the use 
of the notifier. It holds the entire resource object, so the notifier could use 
its properties when sending notifications.

- what does the property `is_real_vitrage_id`
represent?

[Ifat] It represents old code that should be deleted ;-) please ignore it

- what is the difference between `resource_id` and `vitrage_resource_id` ?

[Ifat] resource_id is the id of the resource as retrieved by the datasource, 
e.g. the Nova instance id
 vitrage_id is the id of the resource inside Vitrage. This is the id 
that Vitrage uses to identify its resources. For a Nova instance, vitrage_id 
will be different from its resource_id.
 vitrage_resource_id is used only on alarms, and holds the vitrage_id 
of the resource of the alarm.



2.  Do you plan to support also updateRows? This can be useful for alarm 
state changes.
Are you thinking about updating an entire row or updating a specific field of a 
row? That is, update
Row {"id":"1-1", "name":"name1", "state":"active", "severity":1} to become 
{"id":"1-1", "name":"name1", "state":"active", "severity":100}
Vs
Update the severity field of row with id "1-1" to severity 100.
Both could be supported, but the second one is more complex to support 
efficiently.

[Ifat] It’s really up to you, I think both would satisfy the use case. The 
Congress notifier will be written based on your selected implementation.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [congress] generic push driver

[Eric K]

From:  Tim Hinrichs 
Date:  Monday, January 8, 2018 at 7:31 AM
To:  Eric Kao 
Cc:  "OpenStack Development Mailing List (not for usage questions)"

Subject:  Re: [congress] generic push driver

> It's probably worth considering PATCH instead of PUT for updating the table.
Ah right of course. PATCH makes more sense here.
> 
> http://restcookbook.com/HTTP%20Methods/patch/
> 
> You could also think about using JSON-patch to describe the requested update.
> It provides fine-grained update semantics:
> 
> https://tools.ietf.org/html/rfc6902
Hmm it would be very nice to follow an existing standard. Unfortunately the
json patch path specifications seem like an awkward fit with the set
semantics of congress tables. Removal, for example, must be done by
specifying the array index of the row to be removed. But perhaps we can
borrow the style of json patch for patching sets. For example:
PATCH '/v1/data-sources/vitrage/tables/alarms' with body:
[
  {
"op":"add",
"path":"/",
"value":{
  "id":"1-1",
  "name":"name1",
  "state":"active",
  "severity":1
}
  },
  {
"op":"add",
"path":"/",
"value":[
  "1-2",
  "name2",
  "active",
  2
]
  },
  {
"op":"remove",
"path":"/",
"value":[
  "1-2",
  "name2",
  "active",
  2
]
  }
]

Would that work well? At least there will be well-defined semantic based on
sequential operation.
> 
> Tim
> 
> On Fri, Jan 5, 2018 at 5:50 PM Eric K  wrote:
>> We've been discussing generic push drivers for Congress for quite a while.
>> Finally sketching out something concrete and looking for some preliminary
>> feedback. Below are sample interactions with a proposed generic push driver.
>> A generic push driver could be used to receive push updates from vitrage,
>> monasca, and many other sources.
>> 
>> 1. creating a datasource:
>> 
>> congress datasource create generic_push_driver vitrage --config schema='
>> {
>>   "tables":[
>> {
>>   "name":"alarms",
>>   "columns":[
>> "id",
>> "name",
>> "state",
>> "severity",
>>   ]
>> }
>>   ]
>> }
>> '
>> 
>> 2. Update an entire table:
>> 
>> PUT '/v1/data-sources/vitrage/tables/alarms' with body:
>> {
>>   "rows":[
>> {
>>   "id":"1-1",
>>   "name":"name1",
>>   "state":"active",
>>   "severity":1
>> },
>> [
>>   "1-2",
>>   "name2",
>>   "active",
>>   2
>> ]
>>   ]
>> }
>> Note that a row can be either a {} or []
>> 
>> 
>> 3. perform differential update:
>> 
>> PUT '/v1/data-sources/vitrage/tables/alarms' with body:
>> {
>>   "addrows":[
>> {
>>   "id":"1-1",
>>   "name":"name1",
>>   "state":"active",
>>   "severity":1
>> },
>> [
>>   "1-2",
>>   "name2",
>>   "active",
>>   2
>> ]
>>   ]
>> }
>> 
>> OR
>> 
>> {
>>   "deleterows":[
>> {
>>   "id":"1-1",
>>   "name":"name1",
>>   "state":"active",
>>   "severity":1
>> },
>> [
>>   "1-2",
>>   "name2",
>>   "active",
>>   2
>> ]
>>   ]
>> }
>> 
>> Note 1: we may allow 'rows', 'addrows', and 'deleterows' to be used together
>> with some well defined semantics. Alternatively we may mandate that each
>> request can have only one of the three pieces.
>> 
>> Note 2: we leave it as the responsibility of the sender to send and confirm
>> the requests for differential updates in correct order. We could add
>> sequencing in future work.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [congress] generic push driver

[Eric K]

Hi Ifat,
From:  "Afek, Ifat (Nokia - IL/Kfar Sava)" <ifat.a...@nokia.com>
Reply-To:  "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev@lists.openstack.org>
Date:  Sunday, January 7, 2018 at 4:00 AM
To:  "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [congress] generic push driver

> Hi Eric,
>  
> I have two questions:
>  
> 1.  An alarm is usually raised on a resource, and in Vitrage we can send
> you the details of that resource. Is there a way in Congress for the alarm to
> reference a resource that exists in another table? And what if the resource
> does not exist in Congress?

First, the columns I chose are just a minimal sample to illustrate the
generic nature of the driver. In use with vitrage, we would probably also
want to include columns such as `resource_id`. Does that address the need to
reference a resource? That resource referenced by ID may or may not exist in
another part of Congress. It would be the job of the policy to resolve
references when taking appropriate actions. If referential integrity is
needed, additional policy rules can be specified to catch breakage.

This brings up a related question I had about vitrage:
Looking at the vertex properties listed here:
https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py
#L17

Where can I find more information about the type and content of data in each
property?

Exapmle:
- is the `resource` property an ID string or a python object reference?
- what does the property `is_real_vitrage_id` represent?
- what is the difference between `resource_id` and `vitrage_resource_id` ?

> 2.  Do you plan to support also updateRows? This can be useful for alarm
> state changes.

Are you thinking about updating an entire row or updating a specific field
of a row? That is, update
Row {"id":"1-1", "name":"name1", "state":"active", "severity":1} to become
{"id":"1-1", "name":"name1", "state":"active", "severity":100}
Vs
Update the severity field of row with id "1-1" to severity 100.
Both could be supported, but the second one is more complex to support
efficiently.

Thanks!
Eric
>  
> Thanks,
> Ifat
>  
>  
> 
> From: Eric K <ekcs.openst...@gmail.com>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org>
> Date: Saturday, 6 January 2018 at 3:50
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org>
> Subject: [openstack-dev] [congress] generic push driver
> 
>  
> 
> We've been discussing generic push drivers for Congress for quite a while.
> Finally sketching out something concrete and looking for some preliminary
> feedback. Below are sample interactions with a proposed generic push driver. A
> generic push driver could be used to receive push updates from vitrage,
> monasca, and many other sources.
> 
>  
> 
> 1. creating a datasource:
> 
>  
> 
> congress datasource create generic_push_driver vitrage --config schema='
> 
> {
> 
>   "tables":[
> 
> {
> 
>   "name":"alarms",
> 
>   "columns":[
> 
> "id",
> 
> "name",
> 
> "state",
> 
> "severity",
> 
>   ]
> 
> }
> 
>   ]
> 
> }
> 
> '
> 
>  
> 
> 2. Update an entire table:
> 
>  
> 
> PUT '/v1/data-sources/vitrage/tables/alarms' with body:
> 
> {
> 
>   "rows":[
> 
> {
> 
>   "id":"1-1",
> 
>   "name":"name1",
> 
>   "state":"active",
> 
>   "severity":1
> 
> },
> 
> [
> 
>   "1-2",
> 
>   "name2",
> 
>   "active",
> 
>   2
> 
> ]
> 
>   ]
> 
> }
> 
> Note that a row can be either a {} or []
> 
>  
> 
>  
> 
> 3. perform differential update:
> 
>  
> 
> PUT '/v1/data-sources/vitrage/tables/alarms' with body:
> 
> {
> 
>   "addrows":[
> 
> {
> 
>   "id":"1-1",
> 
>   "name":"name1",
> 
>   "state":"active",
> 
>   "severity":1
> 
> },
> 
> [
> 
>   "1-2",
> 
>   "name2",
> 
>   "

Re: [openstack-dev] [congress] generic push driver

[Tim Hinrichs]

It's probably worth considering PATCH instead of PUT for updating the table.

http://restcookbook.com/HTTP%20Methods/patch/

You could also think about using JSON-patch to describe the requested
update.  It provides fine-grained update semantics:

https://tools.ietf.org/html/rfc6902

Tim

On Fri, Jan 5, 2018 at 5:50 PM Eric K  wrote:

> We've been discussing generic push drivers for Congress for quite a while.
> Finally sketching out something concrete and looking for some preliminary
> feedback. Below are sample interactions with a proposed generic push
> driver. A generic push driver could be used to receive push updates from
> vitrage, monasca, and many other sources.
>
> 1. creating a datasource:
>
> congress datasource create generic_push_driver vitrage --config schema='
> {
>   "tables":[
> {
>   "name":"alarms",
>   "columns":[
> "id",
> "name",
> "state",
> "severity",
>   ]
> }
>   ]
> }
> '
>
> 2. Update an entire table:
>
> PUT '/v1/data-sources/vitrage/tables/alarms' with body:
> {
>   "rows":[
> {
>   "id":"1-1",
>   "name":"name1",
>   "state":"active",
>   "severity":1
> },
> [
>   "1-2",
>   "name2",
>   "active",
>   2
> ]
>   ]
> }
> Note that a row can be either a {} or []
>
>
> 3. perform differential update:
>
> PUT '/v1/data-sources/vitrage/tables/alarms' with body:
> {
>   "addrows":[
> {
>   "id":"1-1",
>   "name":"name1",
>   "state":"active",
>   "severity":1
> },
> [
>   "1-2",
>   "name2",
>   "active",
>   2
> ]
>   ]
> }
>
> OR
>
> {
>   "deleterows":[
> {
>   "id":"1-1",
>   "name":"name1",
>   "state":"active",
>   "severity":1
> },
> [
>   "1-2",
>   "name2",
>   "active",
>   2
> ]
>   ]
> }
>
> Note 1: we may allow 'rows', 'addrows', and 'deleterows' to be used
> together with some well defined semantics. Alternatively we may mandate
> that each request can have only one of the three pieces.
>
> Note 2: we leave it as the responsibility of the sender to send and
> confirm the requests for differential updates in correct order. We could
> add sequencing in future work.
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [congress] generic push driver

[Afek, Ifat (Nokia - IL/Kfar Sava)]

Hi Eric,

I have two questions:


1.   An alarm is usually raised on a resource, and in Vitrage we can send 
you the details of that resource. Is there a way in Congress for the alarm to 
reference a resource that exists in another table? And what if the resource 
does not exist in Congress?

2.   Do you plan to support also updateRows? This can be useful for alarm 
state changes.

Thanks,
Ifat


From: Eric K <ekcs.openst...@gmail.com>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Date: Saturday, 6 January 2018 at 3:50
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Subject: [openstack-dev] [congress] generic push driver

We've been discussing generic push drivers for Congress for quite a while. 
Finally sketching out something concrete and looking for some preliminary 
feedback. Below are sample interactions with a proposed generic push driver. A 
generic push driver could be used to receive push updates from vitrage, 
monasca, and many other sources.

1. creating a datasource:

congress datasource create generic_push_driver vitrage --config schema='
{
  "tables":[
{
  "name":"alarms",
  "columns":[
"id",
"name",
"state",
"severity",
  ]
}
  ]
}
'

2. Update an entire table:

PUT '/v1/data-sources/vitrage/tables/alarms' with body:
{
  "rows":[
{
  "id":"1-1",
  "name":"name1",
  "state":"active",
  "severity":1
},
[
  "1-2",
  "name2",
  "active",
  2
]
  ]
}
Note that a row can be either a {} or []


3. perform differential update:

PUT '/v1/data-sources/vitrage/tables/alarms' with body:
{
  "addrows":[
{
  "id":"1-1",
  "name":"name1",
  "state":"active",
  "severity":1
},
[
  "1-2",
  "name2",
  "active",
  2
]
  ]
}

OR

{
  "deleterows":[
{
  "id":"1-1",
  "name":"name1",
  "state":"active",
  "severity":1
},
[
  "1-2",
  "name2",
  "active",
  2
]
  ]
}

Note 1: we may allow 'rows', 'addrows', and 'deleterows' to be used together 
with some well defined semantics. Alternatively we may mandate that each 
request can have only one of the three pieces.

Note 2: we leave it as the responsibility of the sender to send and confirm the 
requests for differential updates in correct order. We could add sequencing in 
future work.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [congress] generic push driver

[Eric K]

We've been discussing generic push drivers for Congress for quite a while.
Finally sketching out something concrete and looking for some preliminary
feedback. Below are sample interactions with a proposed generic push
driver. A generic push driver could be used to receive push updates from
vitrage, monasca, and many other sources.

1. creating a datasource:

congress datasource create generic_push_driver vitrage --config schema='
{
  "tables":[
{
  "name":"alarms",
  "columns":[
"id",
"name",
"state",
"severity",
  ]
}
  ]
}
'

2. Update an entire table:

PUT '/v1/data-sources/vitrage/tables/alarms' with body:
{
  "rows":[
{
  "id":"1-1",
  "name":"name1",
  "state":"active",
  "severity":1
},
[
  "1-2",
  "name2",
  "active",
  2
]
  ]
}
Note that a row can be either a {} or []


3. perform differential update:

PUT '/v1/data-sources/vitrage/tables/alarms' with body:
{
  "addrows":[
{
  "id":"1-1",
  "name":"name1",
  "state":"active",
  "severity":1
},
[
  "1-2",
  "name2",
  "active",
  2
]
  ]
}

OR

{
  "deleterows":[
{
  "id":"1-1",
  "name":"name1",
  "state":"active",
  "severity":1
},
[
  "1-2",
  "name2",
  "active",
  2
]
  ]
}

Note 1: we may allow 'rows', 'addrows', and 'deleterows' to be used
together with some well defined semantics. Alternatively we may mandate
that each request can have only one of the three pieces.

Note 2: we leave it as the responsibility of the sender to send and confirm
the requests for differential updates in correct order. We could add
sequencing in future work.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev