Re: My beginners experience of juju charm development

[fengxia]

I'm a researcher at Lenovo US. I myself started researching Juju and
charms about 4 months ago and can relate to many of Erik's comments. To
add to this discussion, here are my experience and thoughts:

1. Design document.

Reading official document and tutorials are good but not satisfying to
engineer's curiosity. Copy and paste someone's "hello world" example
often got me only to the length of that "hello world", because it lacks
in-depth context that will help beginner to understand how these pieces
come together.

Take reactive framework for example, I want to know who kicks off charm
execution? in what type of running environment -- using system's Python
or a virtualenv? which state will be the first in line? states are
queued, are their order sorted by their names? When it says "reactive",
I was expecting to see mechanisms such as signals or some preemptive
methods. I used to do embedded systems and am familiar with such
designs. But there is no doc explaining how it "react". Looking into
charm-helper code the main function has two loops -- one to scan all
hooks exhaustively, and another is a for(1..100) loop(!?), and that's
very puzzling. Is this by design, or am I looking at the wrong place? I
wish there were "official" voice on these topics so users can learn.

2. Juju -> charm.

Juju and charms go hand in hand. But I haven't been able to find much
information on how they work together. Writing charm and looking at
charm codes actually make me realize that hook sequence are hardcoded in
Juju code. If so, how about a state diagram? I made one and attached it
here. To beginners a visual is much easier than line item definitions.
Canonical did a great job defining these, let's take one step further to
make a diagram.

3. Tutorial of common design pattern.

I'm a 15+ year Python developer. I look at a problem naturally in an OOD
lens -- define a class Car, then inherit it to define a class FordFocus.
Learning a new technology like charm isn't only about syntax, but the
way to think and analyze problem at hand. Therefore tutorials on common
patterns will be of great help. How do I do inheritance -- charm layers.
How do I do aggregation? how do I do composition? In my research,
realizing that charm layers can be nested but bundles can not is
significant factor that will impact how I approach a design.

4. Comparison framework.

There are many orchestration projects out there. To many teams who take
interest in this subject, the first step is not to dive into a
particular one, but to read about it horizontally and make a comparison
sense of their strength and weakness -- in technical term, not marketing
term.

I always feel there needs to be a framework, however flawed it might be,
to establish a scope and categories so one can see technically how one
is different from another. To start, agent vs. agent less, pre-built vs.
runtime interpreted, reuse mechanism (layer, include, import), rich
pre-defined syntax (TOSCA) vs. user-defined (charm states). A voice from
domain expert like Canonical, though biased it may be, will help
beginner to grasp the landscape and what consists of this subject. It's
laundry list, like when I look at programming language, a list of "+-*/,
and, or, xor, if, else, while...".. check check, since I know what I
mostly use, exotic topics are interesting but not necessary.

For orchestration I wish there were such a thing put out by one these
teams. Otherwise, they all look powerful and can do the job, and that's
certainly not helping user adoption since we have no starting point to
evaluate.

Last, just want to add to Erik's comment that Juju and charm are good
technologies. Canonical did a good job implementing them and is in lead.
I hear the word "juju" "charm" more and more these days in various
discussions. So the user interest is strong. We want to learn, to
practice, and to share our experience. Help us and the community to make
the technology better understood.

Best,

Feng


On 05/11/2017 02:19 PM, Erik Lönroth wrote:

Hello!

I've been trying to become friends with juju for the last month or so
and was asked in the #juju IRC channel on Freenode to share my
experiences from a "beginner perspective on juju charming".

Generally, I believe JuJu is a great concept. I makes alot of sense
and the architecture and "thought" gone into the tool is impressive.

BUT:

Having some 10+ years in programmer/sysadmin/linux - I think the high
learning curve of developing charms is a problem. Frankly, without the
help from the guys in the IRC-channel - I would have given up a long
time ago. Let me explain my problems and share my experiences.

1. The target developers are sysadmins - not programmers.

Most developers new to JuJu charms are likely sysadmins or devops.
Likely with -limited- amount of programming knowledge or even self
taught scripters. These professionals are not always deep in with
advanced development concepts like "interfaces", "stubs", "layers",
"reactive", "hooks" etc.

Re: My beginners experience of juju charm development

[Charles Butler]

Thank you so much for the detailed view into your journey and sharing this
feedback Erik.

Things like this will go far to help us improve our community resources. I
offer no excuses, only empathy that I understand where you're coming from.
As a 3 year veteran of the Juju ecosystem, I too felt this pain in the
early days. It got better, but we can certainly improve this experience,
and many of your suggestions are solid thoughts we've had on our side as
well. There's a big overlap between where you wish us to go and where we
want to be.

I'll make sure additional eyes make their way to the list to see this
detailed write-up.

Thanks and all the best,

lazyPower

On Thu, May 11, 2017 at 1:48 PM Erik Lönroth  wrote:

> Hello!
>
> I've been trying to become friends with juju for the last month or so and
> was asked in the #juju IRC channel on Freenode to share my experiences from
> a "beginner perspective on juju charming".
>
> Generally, I believe JuJu is a great concept. I makes alot of sense and
> the architecture and "thought" gone into the tool is impressive.
>
> BUT:
>
> Having some 10+ years in programmer/sysadmin/linux - I think the high
> learning curve of developing charms is a problem. Frankly, without the help
> from the guys in the IRC-channel - I would have given up a long time ago.
> Let me explain my problems and share my experiences.
>
> 1. The target developers are sysadmins - not programmers.
>
> Most developers new to JuJu charms are likely sysadmins or devops. Likely
> with -limited- amount of programming knowledge or even self taught
> scripters. These professionals are not always deep in with advanced
> development concepts like "interfaces", "stubs", "layers", "reactive",
> "hooks" etc.
> The need for a clear path in how to learn juju charming is critical
>
> 2. The documentation site really need some love and thought.
> A) Where are the tutorials?
> B) What is the "flow" of reading the documentation? What is the intended
> purpose and audience? At the moment its a mix between references,
> commercial, examples and feels just like some place you go to NOT really
> find what you are looking for. Perhaps split the content into developer
> sites, user sites, tutorial sites or something like that.
>
> I'm better off reading blogs trying to figure out what goes on in juju.
>
> 3. There are no "best practice" available for common tasks.
>
> Developing a simple useful charm today requires you to cover alot of
> ground before you even can produce your first "hello-world" charm. Concepts
> like "interfaces", "reactive programming", "layers" and juju-state-stuck
> debugging needs to be understood well before you write a decent charm. This
> would normally be OK if there was a extremely good, documented, examples -
> using a set of "best practices" clearly available to developers. Offcourse
> there are, but those are only known by the "elite developers" in the IRC
> channels.
>
> Beginners like me, have to learn these best-practices by either:
>
> A) Dwell in the IRC for extensive periods of time and have the courage to
> ask.
> B) Fight with the juju documentation and figure out "my own
> best-practises".
> C) Google and reading multiple of blogs with "others best practices" and
> selecting what ever works for you.
>
> In this process, I found myself getting confused over "old ways of doing
> things" and whatever new concepts was king today. As a beginner - I can't
> figure out if I was doing things wrong, or if juju was just volatile by
> nature. I'm still partially there...
>
> Quoting from #juju:
>
> "We've refactored our charms couple times after we got something working..
> First it was just bash that was run through Juju hooks And after getting
> comfortable with reactive we rebuilt everything from scratch using reactive
> only"
>
> 3. Reactive programming is great, but not intuitive.
>
> As I understand from some blogs, reactive programming is the way juju is
> transcending into (or already has?). It seems to make sense for most
> experienced charmers, but how about beginners?
>
> When I was starting to develop my first "hello-world" example charm. I
> used only hooks, which quickly got me to a working example. Happy thinking
> I was on the right track, I then discovered  I needed to "relearn/redo" all
> my knowledge because "reactive" was "the real" way of doing charms. Bump.
>
> So, taking on a new challenge on relearning that naturally got me into
> deep holes mixing these concepts up (hooks vs reactive) and not using them
> properly and leading up to new challenges and messy code. To make things
> even worse, I learned just today that in certain cases, these concepts are
> mixed up "deliberately"/"by design". Getting through all this is a hard
> task.
>
> Now, I want to be clear that I'm not criticising juju as such here. My
> message  is that the success of juju is highly dependent on the
> availability to clear paths through difficult terrain. A multitude of

My beginners experience of juju charm development

[Erik Lönroth]

Hello!

I've been trying to become friends with juju for the last month or so and
was asked in the #juju IRC channel on Freenode to share my experiences from
a "beginner perspective on juju charming".

Generally, I believe JuJu is a great concept. I makes alot of sense and the
architecture and "thought" gone into the tool is impressive.

BUT:

Having some 10+ years in programmer/sysadmin/linux - I think the high
learning curve of developing charms is a problem. Frankly, without the help
from the guys in the IRC-channel - I would have given up a long time ago.
Let me explain my problems and share my experiences.

1. The target developers are sysadmins - not programmers.

Most developers new to JuJu charms are likely sysadmins or devops. Likely
with -limited- amount of programming knowledge or even self taught
scripters. These professionals are not always deep in with advanced
development concepts like "interfaces", "stubs", "layers", "reactive",
"hooks" etc.
The need for a clear path in how to learn juju charming is critical

2. The documentation site really need some love and thought.
A) Where are the tutorials?
B) What is the "flow" of reading the documentation? What is the intended
purpose and audience? At the moment its a mix between references,
commercial, examples and feels just like some place you go to NOT really
find what you are looking for. Perhaps split the content into developer
sites, user sites, tutorial sites or something like that.

I'm better off reading blogs trying to figure out what goes on in juju.

3. There are no "best practice" available for common tasks.

Developing a simple useful charm today requires you to cover alot of ground
before you even can produce your first "hello-world" charm. Concepts like
"interfaces", "reactive programming", "layers" and juju-state-stuck
debugging needs to be understood well before you write a decent charm. This
would normally be OK if there was a extremely good, documented, examples -
using a set of "best practices" clearly available to developers. Offcourse
there are, but those are only known by the "elite developers" in the IRC
channels.

Beginners like me, have to learn these best-practices by either:

A) Dwell in the IRC for extensive periods of time and have the courage to
ask.
B) Fight with the juju documentation and figure out "my own best-practises".
C) Google and reading multiple of blogs with "others best practices" and
selecting what ever works for you.

In this process, I found myself getting confused over "old ways of doing
things" and whatever new concepts was king today. As a beginner - I can't
figure out if I was doing things wrong, or if juju was just volatile by
nature. I'm still partially there...

Quoting from #juju:

"We've refactored our charms couple times after we got something working..
First it was just bash that was run through Juju hooks And after getting
comfortable with reactive we rebuilt everything from scratch using reactive
only"

3. Reactive programming is great, but not intuitive.

As I understand from some blogs, reactive programming is the way juju is
transcending into (or already has?). It seems to make sense for most
experienced charmers, but how about beginners?

When I was starting to develop my first "hello-world" example charm. I used
only hooks, which quickly got me to a working example. Happy thinking I was
on the right track, I then discovered  I needed to "relearn/redo" all my
knowledge because "reactive" was "the real" way of doing charms. Bump.

So, taking on a new challenge on relearning that naturally got me into deep
holes mixing these concepts up (hooks vs reactive) and not using them
properly and leading up to new challenges and messy code. To make things
even worse, I learned just today that in certain cases, these concepts are
mixed up "deliberately"/"by design". Getting through all this is a hard
task.

Now, I want to be clear that I'm not criticising juju as such here. My
message  is that the success of juju is highly dependent on the
availability to clear paths through difficult terrain. A multitude of
tutorials showing "best practices" and covering common tasks would be a
great start. Deciding what those "common tasks" are is important too I
guess as this is the way you tutor developers and cultivate communities.
Like - why isn't there a super-clear-mega-awesome-tutorial around
demonstrating how to create  a database and an example client application,
that connects to each other with juju? That must be something of the most
common scenarios for juju and I'm just chocked that its no such "tutorial"
around via the official juju documentation site.  Comparing with the
Android community where there tutorials tied to each end every concept
available in the framework. Juju deserves this.

I'm going to continue struggle with juju regardless. Thanx again for all
the great help in the IRC channel from @kjackal , @stub, @lazyPower. Thanx
for taking your time explaining and helping me get through