Merlijn,
Nice summary of these issues. Especially for the #2, we have been using
Ansible (called from charm) for executions. I think there are 3 points
here:
1. confg mgt has solved idempotency, great!
2. many organizations have accumulated an asset of recipes using these
config tools (puppet, chef, ansible, and so on)s. Being able to re-use
them makes transition and adoption within these orgs easier.
3. Betting on technology. Puppet, chef, ansible all have/had their days.
New things will inevitably emerge. Having two in the stack, though
inefficient, mitigate risk. This is just sad reality.
Using an analogy. A charm is like a cooked dish prepared by a wonderful
chef, tastes great, but few can do that at home yet. I have awed many by
showing them how easy it is to deploy an Openstack with a click of
button using Juju GUI. But ppl fear things they don't understand. Config
mgt tools are like giving you a set of cooking utensils and a recipe
book, you can follow the book, or be an iron chef yourself, and
surprisingly, many believe they have that talent to be the iron chef,
though over optimistically often ~~
On 10/30/2017 11:56 AM, Merlijn Sebrechts wrote:
Hi all
Great discussion in this thread. I sense there are two issues here:
1. The transactional nature of charms.reactive and Juju needs to be
explained better
<https://github.com/juju-solutions/charms.reactive/issues/137>. We
can't change the transactional nature from the charms.reactive side
since this is a Juju core feature, but we can provide a lot better
docs and change function names to better match their actual behavior.
This is already discussed for relationships as part of the Endpoint PR
<https://github.com/juju-solutions/charms.reactive/pull/123>.
2. Idempotency is hard, not commonly understood outside of config mgmt
community and charms.reactive isn't helping.
<https://github.com/juju-solutions/charms.reactive/issues/138> Idempotency
is a solved issue in config mgmt tools. I don't think it's the job of
Juju and charms.reactive to provide ways to do this because we operate
on a higher level (service orchestration, not config mgmt). What we
should to is make it easier to use charms.reactive together with
config mgmt tools like Puppet and Chef. This will keep us from
reinventing the wheel and will provide a number of additional benefits
(such as being able to leverage existing Puppet scripts and quicker
charms).
Please create more issues if I've missed something, and add your
comments to the issues.
Kind regards
Merlijn
2017-10-05 20:50 GMT+02:00 fengxia <[email protected]
<mailto:[email protected]>>:
" An assumption is being made that the state changes get committed
immediately, but these changes are actually transactional and
following the same transactional behaviour as the Juju hook
environment [1]."
To chip in my experience as 6-month into learning charms and
writing a few simple charms.
1. The "transactional" nature of Juju hook needs better explained.
To be honest I have no idea what this means, and what implication
it has to a charm writer. Any reference would be helpful.
2. I like Mike Wilson's approach to provide a list of
"set_state_xxx" functions so new writer can better guess what this
function will do. Further, a different name calls for further
study why they are different, thus learning important concept of
whatever Juju thinks charm writer needs to understand.
Otherwise, I will expect "set_state" will set that state/flag
asap. If there is a scanning cycle (which I heard there is some
kind of 5-min cycle, which document has not sufficiently made it
clear to a writer either), charm writer needs to have better doc
to learn what it means for design. I came from embedded system
world in which a timer loop is common. It calls for a different
thinking than user space script user. I think such implication
should be emphasized more.
3. idempotent
Again, this is a concept me (or many new writer) will fail to
grasp. Looking at "apt install" as example, my reaction was that
the package manager is taking care of "doing nothing" if called
multiple times. But how this translate to my system in which charm
is expected to "do something"? Does it mean I need a gatekeeper
like the package manager so to guard these "multiple calls"?
Again, this feels like a work around because "set_state" will call
the same function block multiple times, which is unintuitive to
writer -- when I set a state, the action for that state is
executed once, not over and over again until I turn it off.
Further, even "remove_state" doesn't take effect immediately, so
it feels arbitrary how many cycles a block of code is executed.
This is a design pattern I'm afraid many are not familiar with, so
some tutorial examples will be much appreciated.
Best,
Feng
On 10/04/2017 08:59 AM, Marco Ceppi wrote:
So, I've not actually checked the logs in a while, but if
visibility is an issue, it seems reasonable that the reactive
framework should print in the log that X flags are being reset
due to hook failure. Things like set_flag_immediate has farther
reaching consequences than simply stating that flags only get set
on success of a hook run.
I know there are further reaching initiatives to alleviate this
by decoupling the reactive state engine from the juju hooks
system. In this case each successive loop of the reactive runtime
could better snapshot state and make failures more granular.
* state is being renamed to flag in the next major version of
reactive.
On Wed, Oct 4, 2017 at 8:52 AM Mike Wilson
<[email protected] <mailto:[email protected]>> wrote:
So as a new charm writer coming to Juju I would first do this:
def get_ready():
func0()
func1_fails()
Then I would, hopefully, test and notice issues. I would
investigate and see that I needed to be idempotent. My next
attempt would be to wrap those functions inside state checks
with sets after they complete. This would also fail and now
the charm creator is left with nothing in the api that can
help. They are now off to their own devices to start doing
random things to attempt to make this work the way they want
it to work. Hopefully, the solution is as straight-forward as
touching random files, but we just never know.
I would expect the name of set_state to be something like
set_state_on_success and I would further expect some sort of
immediate state thing like set_state or set_state_immediate.
This would give the user the tools we know that they need in
order to create bug-free charms.
Now to compound that confusion, we have the issue of a hook
can call multiple functions inside the charm code and if any
of those functions have something that fails the whole thing
is unwrapped. I understand from a Juju perspective why this
is the case, but as a user, I would be completely taken by
surprise here. The only real fix here is documentation so
that we can set expectations, but people will most likely
look at examples instead of documentation. This means that we
need to make sure to call out any potential issues like this
in the example charms we release.
On Wed, Oct 4, 2017 at 6:34 AM Stuart Bishop
<[email protected]
<mailto:[email protected]>> wrote:
On 4 October 2017 at 00:51, Mike Wilson
<[email protected]
<mailto:[email protected]>> wrote:
> So the best practice here is to touch a file and test
for the existence of
> that file before running must_be_called_exactly_once()?
>
> I think part of the issue here is that without knowing
the extent of the
> hook it is hard to enforce idempotency as a charm
writer. It's easy to look
> at the code above and say that is it idempotent since
the init function is
> wrapped in a when_not and the initialized state is set
at the bottom of
> init.
Individual handlers should be idempotent, so it doesn't
matter about
the extent of the hook, or even if the chained handlers
being triggers
are running in the same hook. Assume your handlers get
called multiple
times, because they may be. Yes, it looks idempotent but
it isn't. An
assumption is being made that the state changes get committed
immediately, but these changes are actually transactional and
following the same transactional behaviour as the Juju hook
environment [1]. I think this can certainly be explained
better in the
docs, but I can't think of a way to stop this being an
easy error to
make.
[1] spot the DBA
--
Stuart Bishop <[email protected]
<mailto:[email protected]>>
--
Juju mailing list
[email protected] <mailto:[email protected]>
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
<https://lists.ubuntu.com/mailman/listinfo/juju>
--
Feng Xia
Advisory Engineer
Datacenter Group (DCG), Lenovo US
8000 Development Dr, Morrisiville, NC 27509
W:http://www.lenovo.com
--
Juju mailing list
[email protected] <mailto:[email protected]>
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
<https://lists.ubuntu.com/mailman/listinfo/juju>
--
Feng Xia
Advisory Engineer
Datacenter Group (DCG), Lenovo US
8000 Development Dr, Morrisiville, NC 27509
W: http://www.lenovo.com
--
Juju mailing list
[email protected]
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju
