Re: [openstack-dev] [charms] evolving the ha interface type
Hi James, I like option 2 but I think there is a problem with it. I don't think the hacluster charm sets any data down the relation with the principle until it has first received data from the principle. As I understand it option 2 would change this behaviour so that hacluster immediately sets an api-version option for the principle to consume. The only problem is the principle does not know whether to wait for this api-version information or not. eg when the principle is deciding whether to json encode its data it cannot differentiate between: a) An old version of the hacluster charm which does not support api-version or json b) A new version of the hacluster charm which has not set the api-version yet. Thanks Liam __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [charms] evolving the ha interface type
On 11/28/2017 01:50 PM, Dmitrii Shcherbakov wrote:
Hi James,
The side effect of using JSON is that we will lose type information:
In [0]: json.loads(json.dumps([{1: 2}]))
Out[0]: [{'1': 2}]
In [1]: ast.literal_eval(repr([{1: 2}]))
Out[1]: [{1: 2}]
I'm not sure this is really an issue for the hacluster interface. To my
knowledge, all of the keys used for resources today are strings defining
the pacemaker resource options (clones, groups, colocation options etc).
The hacluster charm simply formats crm commands with the name/value
pairs anyways [0]. It is true that type information is lost, but I don't
think its an issue for this interface.
[0] -
https://github.com/openstack/charm-hacluster/blob/master/hooks/hooks.py#L314
This is a hard requirement in the JSON spec:
https://tools.ietf.org/html/rfc7159#section-4
"An object structure is represented as a pair of curly brackets
surrounding zero or more name/value pairs (or members). A name is
a string. "
I wonder if we could use some type-aware serialization format but I do
not have any suggestions without trade-offs because we need to keep in
mind that, in general, charms may not be python charms, may run on
hosts with different protocol parsing libraries etc.
If we only cared about python I would suggest using "pickle" but this
is not universal and not human-readable:
https://docs.python.org/3/library/pickle.html#comparison-with-json
"JSON, by default, can only represent a subset of the Python built-in
types, and no custom classes; pickle can represent an extremely large
number of Python types"
JSON has an advantage of being readable as opposed to a binary protocol
so we can explore what's available in relation data by doing relation-
get without additional tools. It is also ubiquitous so we don't have to
worry about spec changes, parsing bugs and other potential problems.
Thanks for bringing this topic up.
Dmitrii Shcherbakov
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [charms] evolving the ha interface type
Hi James,
The side effect of using JSON is that we will lose type information:
In [0]: json.loads(json.dumps([{1: 2}]))
Out[0]: [{'1': 2}]
In [1]: ast.literal_eval(repr([{1: 2}]))
Out[1]: [{1: 2}]
This is a hard requirement in the JSON spec:
https://tools.ietf.org/html/rfc7159#section-4
"An object structure is represented as a pair of curly brackets
surrounding zero or more name/value pairs (or members). A name is
a string. "
I wonder if we could use some type-aware serialization format but I do
not have any suggestions without trade-offs because we need to keep in
mind that, in general, charms may not be python charms, may run on
hosts with different protocol parsing libraries etc.
If we only cared about python I would suggest using "pickle" but this
is not universal and not human-readable:
https://docs.python.org/3/library/pickle.html#comparison-with-json
"JSON, by default, can only represent a subset of the Python built-in
types, and no custom classes; pickle can represent an extremely large
number of Python types"
JSON has an advantage of being readable as opposed to a binary protocol
so we can explore what's available in relation data by doing relation-
get without additional tools. It is also ubiquitous so we don't have to
worry about spec changes, parsing bugs and other potential problems.
Thanks for bringing this topic up.
Dmitrii Shcherbakov
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [charms] evolving the ha interface type
Hi Team Whilst working on migrating to using Python 3 as the default charm execution environment, I've hit upon a snag with presentation of data from principle charms to the hacluster subordinate charm. Specifically the data presented on the relation is simple str representation of python dicts which are then parsed used ast in the hacluster charm. This has worked OK under Python 2, but due to the non-deterministic iteration of dict keys, the data presented from the principle charm can change over time when the ha_joined function is re-exec'ed (such as during a config-changed hook execution). I'd like to propose that we move to using json to serialise this data so that we can used ordered dicts to present data in a consistent fashion. We need to evolve the interface to deal with this - we could potentially take two approaches: a) Attempt to parse presented data using json, fallback to ast for backwards compat b) Present a version flag from the hacluster charm, allowing the principle to switch to the new approach when the required version of the hacluster charm is presented to it. We'd still do a) but it would avoid hook failures in the event that a user does not upgrade hacluster application instances prior to upgrading principle charms. Anyway - that's my current thoughts. I have a prototype for a) which works nicely, but I think adding b) will provide a better UX (thanks gnuoy for pointing to this solution). Thoughts? James __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
