> Data migration from where to where? We access the current state by retrieving the data from leader db, or am I missing something here?

In case there are changes in how data is stored in one version of a charm vs the other.


Another problem is application versioning: we do have version-specific templates but this data may be versioned too. Old entries may not be simple strings, e.g. they can be small objects which can change following version changes (new data added or removed in a pre-defined way).

I can also see potential scenarios where you would need to gracefully retire old data as features get deprecated and, eventually, removed. So, during an upgrade you would have two copies of stateful data and a charm would react differently depending on the current application version set. After an upgrade the old copy could be automatically discarded.

> Perhaps I'm being naive but I don't see these developing into data sets large enough to cause performance problems.

I don't think it's going to be used for large data sets either but you never know.

> Each time the action is run the context associated with the action is deleted and recreated. If an action argument is unset I guess we could interpret that as leave-unchanged.

Leave unchanged - yes. Still need to be able to delete completely though.

What I like about actions is that you can clearly express imperative steps with arguments that you have to perform after a deployment and they have a very specific type of data in mind which is fetched from stateful applications out of band by an operator.

On 19.02.2018 18:11, Liam Young wrote:
On Mon, Feb 19, 2018 at 9:05 AM, Dmitrii Shcherbakov
<dmitrii.shcherba...@canonical.com> wrote:
Hi Liam,

I was recently looking at how to support custom configuration that relies
on post deployment setup.
I would describe the problem in general as follows:

1) charms can get context not only from Juju (config options, relation data,
leader data), environment (operating system release, OpenStack release,
services running etc.) but also from a stateful data store (e.g. a Keystone
database);
2) it's not easy to track application state from a charm because:
authentication is needed to fetch persistent state, notifications from a
data store cannot be reliably set up because charm code is ran periodically
and it is not always present in memory (polling is neither timely nor
efficient). Another problem is that software that holds the state needs to
support data change notifications which raises version compatibility
questions.

By using actions we move the responsibility for data retrieval and change
notifications to an operator but a more generic scenario would be modeling a
feedback loop from an application to Juju as a modeling system where changes
can be either automatic or gated by an operator (an orchestrator). Making it
automatic would mean that a service would get notifications/poll data from a
state store and would be authorized to use Juju client to make certain
changes.
This is an interesting idea, but there is no such mechanism within
Juju that I know of.

Another problem to solve is maintenance of that state: if we start
maintaining a key-value DB in leader settings we need to think about data
migration over time and how to access the current state.
Data migration from where to where? We access the current state by retrieving
the data from leader db, or am I missing something here?

In other words, in
CRUD, the "C" part is relatively straightforward, "R" is more complicated
with large data sets (if I have a lot of leader data, how do I interpret it
efficiently?),
Perhaps I'm being naive but I don't see these developing into data
sets large enough
to cause performance problems.

"UD" is less clear - seems like there will have to be 3 or 4
actions per feature for C, [R], U and D or one action that can multiplex
commands.
Each time the action is run the context associated with the action is deleted
and recreated. If an action argument is unset I guess we could interpret that as
leave-unchanged.

This brings me to the question of how is it different from state-specific
config values with a complex structure.
To my mind the difference is complexity for the end user. An action has clearly
defined arguments and the charm action code looks after forming this into
the correct context.

Instead of leader data, a per-charm
config option could hold state data in some format namespaced by a feature
name or config file name to render. A data model would be needed to make
sure we can create versioned application-specific state buckets (e.g. for
upgrades, hold both states, then remove the old one).

Application version-specific config values is something not modeled in Juju
although custom application versions are present
(https://jujucharms.com/docs/2.3/reference-hook-tools#application-version-set).
Version information has to be set via a hook tool which means that it has to
come from a custom config option anyway. Each charm has its own method to
specify an application version and config dependencies are not modeled
explicitly - one has to implement that logic in a charm without any Juju API
for charms present the way I see it.

config('key', 'app-version') - would be something to aim for.

Do you have any thoughts about leader data vs a special complex config
option per charm and versioning?

Thanks!
Thanks for the feedback Dmitrii


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to