Re: [DISCUSSION] Kubernetes Helm

[Richard Downer]

Firstly I'd like to make it clear that I *do* appreciate the work that
Andrea has done here! I never want to turn down a good contribution, and
this is a good contribution, so I'm very reluctant but have to agree with
Thomas that the Helm PRs should be reverted (as I'm sure that Thomas was
very reluctant to revert them). Please don't be put off!

As the Apacheites say frequently, "patches welcome" :-)

As we know, the existing Helm contribution has some issues which prevents
in being in master and in any releases. But if the contribution can have
these issues resolved, then it can be merged. I would say that if it works
in our Karaf build (since the classic build is now obsolete) without
requiring platform-specific downloads, then it can be merged without much
issue.

If it's *not* possible to avoid the platform-specific downloads - then it's
still not the end of the road, but the contribution needs to address this
problem and make it workable for the whole project. Personally I'd be
reluctant to accept any proposal which removed our platform-independent
binary releases, unless a significant number of *users* came and told us
that it's what they want. Perhaps the main Apache Brooklyn build excludes
Helm, but there's a small, optional, drop-in pack of platform-specific code
that is an official Brooklyn release artifact. However, the effort around
process, documentation and website updates needed to support multi-platform
releases is not small, and I'd much rather that effort was directed into
creating a pure-Java solution.

But, I know nothing about Kubernetes Helm, and not very much about this
specific problem, so ignore me if I am barking up the wrong tree :-)

Remember we do already have a platform-specific component in Brooklyn - the
`br` CLI. It's optional, simple to package the different platforms, and
simple to install separately, so adding it was a relatively low-friction
change to our processes. If Helm can be supported in the same way then I'd
be quite happy.

It's up to the contributor, Andrea - and anyone else here who is interested
in the work and wants to work with Andrea - to decide where to go from
here. In it's present form, we can't merge it. It is up to contributors to
decide if they want to change the contribution to make it suitable, or to
maintain it out-of-tree for a while. Or drop it, but I really hope that
doesn't happen (see first paragraph).

Richard.


On 18 April 2018 at 19:56, Geoff Macartney 
wrote:

> hi all,
>
> just to check what is the resolution on this, is the k8s-helm location
> going to be left out of Brooklyn, and done in a community repo?
>
> regards
> Geoff
>
>
>
> On Tue, 17 Apr 2018 at 14:03 Andrea Turli  wrote:
>
> > Thomas,
> >
> > thanks for taking care of this.
> >
> > Geomacy,
> >
> > I've built brooklyn-{server,dist} on OSX and ran it from CentOS 7 and it
> > works fine as I think `os-maven-plugin` is doing the right incantation
> >
> > Richard,
> >
> > I think helm is a well appreciated tool for kubernetes ecoystem.
> >
> > I think I can donate my work as community-supported addon although it is
> > designed as a Brooklyn location not a simple application blueprint, but
> > worth noticing that it was not part of the core but was committed in
> > `brooklyn-server/locations/container` module, which seems a perfect
> match
> > to me,
> > I undestand the helm location works with the classic launcher, but there
> is
> > an extra complexity to OSGify grpc (a transitive dependency of
> > microbean-helm) to make it work with brooklyn-karaf.
> >
> > I may resubmit the contribution without bringing in the microbean-helm
> java
> > client but writing a simpler restful client for Helm/Tiller, but not sure
> > how hard it would be to refactor the location yet.
> >
> > Best,
> > Andrea
> >
> >
> >
> > On Tue, 17 Apr 2018 at 11:50 Richard Downer  wrote:
> >
> > > All,
> > >
> > > I'm sorry but I'm not really aware of the Kubernetes ecosystem in much
> > > detail. Do we have an indication of how much demand there is for Helm
> > > support in Brooklyn?
> > >
> > > This sounds like a significant change - both to our build process, but
> > also
> > > our release process and website (supporting multiple platform
> downloads).
> > > I'd be opposed to doing it unless there's a significant demand for
> Helm.
> > > I'd prefer to see this as a community-supported addon (like most of our
> > > blueprints are) instead of being core Brooklyn, until there's evidence
> of
> > > demand for this to be in core.
> > >
> > > Richard.
> > >
> > >
> > >
> > > On 17 April 2018 at 09:07, Geoff Macartney <
> geoff.macart...@cloudsoft.io
> > >
> > > wrote:
> > >
> > > > hi Thomas
> > > >
> > > > It certainly doesn't sound right to have to have separate builds of
> > > > Brooklyn for different platforms, and especially not just for this
> > > > feature.  Can we build the dependency package into a bundle for each
> > > > 

Re: [DISCUSSION] Kubernetes Helm

[Geoff Macartney]

hi all,

just to check what is the resolution on this, is the k8s-helm location
going to be left out of Brooklyn, and done in a community repo?

regards
Geoff



On Tue, 17 Apr 2018 at 14:03 Andrea Turli  wrote:

> Thomas,
>
> thanks for taking care of this.
>
> Geomacy,
>
> I've built brooklyn-{server,dist} on OSX and ran it from CentOS 7 and it
> works fine as I think `os-maven-plugin` is doing the right incantation
>
> Richard,
>
> I think helm is a well appreciated tool for kubernetes ecoystem.
>
> I think I can donate my work as community-supported addon although it is
> designed as a Brooklyn location not a simple application blueprint, but
> worth noticing that it was not part of the core but was committed in
> `brooklyn-server/locations/container` module, which seems a perfect match
> to me,
> I undestand the helm location works with the classic launcher, but there is
> an extra complexity to OSGify grpc (a transitive dependency of
> microbean-helm) to make it work with brooklyn-karaf.
>
> I may resubmit the contribution without bringing in the microbean-helm java
> client but writing a simpler restful client for Helm/Tiller, but not sure
> how hard it would be to refactor the location yet.
>
> Best,
> Andrea
>
>
>
> On Tue, 17 Apr 2018 at 11:50 Richard Downer  wrote:
>
> > All,
> >
> > I'm sorry but I'm not really aware of the Kubernetes ecosystem in much
> > detail. Do we have an indication of how much demand there is for Helm
> > support in Brooklyn?
> >
> > This sounds like a significant change - both to our build process, but
> also
> > our release process and website (supporting multiple platform downloads).
> > I'd be opposed to doing it unless there's a significant demand for Helm.
> > I'd prefer to see this as a community-supported addon (like most of our
> > blueprints are) instead of being core Brooklyn, until there's evidence of
> > demand for this to be in core.
> >
> > Richard.
> >
> >
> >
> > On 17 April 2018 at 09:07, Geoff Macartney  >
> > wrote:
> >
> > > hi Thomas
> > >
> > > It certainly doesn't sound right to have to have separate builds of
> > > Brooklyn for different platforms, and especially not just for this
> > > feature.  Can we build the dependency package into a bundle for each
> > > platform, so that it is the only thing that is platform specific, and
> > > provide all three bundles along with the distribution of Brooklyn? Then
> > on
> > > whatever target platform you are on, OSX, Linux, Windows, you install
> the
> > > right dependency bundle for that platform into Brooklyn's Karaf?
> > >
> > > Geoff
> > >
> > >
> > >
> > > On Mon, 16 Apr 2018 at 12:18 Thomas Bouron
>  > > com>
> > > wrote:
> > >
> > > > Hi Brooklyners.
> > > >
> > > > You might have noticed that the Brooklyn builds started to fail more
> > than
> > > > usual recently. I spent some time last week to fix those issues but I
> > > just
> > > > realised that there is a deeper one with the recent change I merged
> > > (about
> > > > Kubernetes Helm)
> > > >
> > > > This change requires a dependency, which depends on some native code.
> > > Now,
> > > > this dependency exists for 3 platforms: macOS, Linux and Window. The
> > > issue
> > > > is that the "right" dependency is included at build time via the
> maven
> > > > classifier, and the way it is picked is by looking at the current
> build
> > > OS
> > > > and selecting the corresponding one[1]. Obviously, this leads to
> nasty
> > > > problem: as all our builds are done on Linux, Brooklyn artifacts only
> > > work
> > > > on Linux. Not only that, the classifier is dynamically picked and set
> > as
> > > > envvar by a plugin. This is also an issue for any downstream
> projects.
> > > >
> > > > While we want this feature in Brooklyn, I don't think this is
> > acceptable
> > > > for our users therefore I reverted the changes and started this
> > > > conversation. What do you think would be the best approach to fix
> this?
> > > > Having 1 build per platform doesn't sound good as it won't be
> portable
> > > > anymore. Maybe we can find another dependency without a link to
> native
> > > > code?
> > > >
> > > > Best.
> > > >
> > > > [1]
> > > >
> > > > https://github.com/apache/brooklyn-dist/pull/119/files#diff-
> > > 01b5eceed5fb6499e99a42e711695648R139
> > > > --
> > > >
> > > > Thomas Bouron • Senior Software Engineer @ Cloudsoft Corporation •
> > > > https://cloudsoft.io/
> > > > Github: https://github.com/tbouron
> > > > Twitter: https://twitter.com/eltibouron
> > > >
> > >
> >
>

Re: [DISCUSSION] Kubernetes Helm

[Andrea Turli]

Thomas,

thanks for taking care of this.

Geomacy,

I've built brooklyn-{server,dist} on OSX and ran it from CentOS 7 and it
works fine as I think `os-maven-plugin` is doing the right incantation

Richard,

I think helm is a well appreciated tool for kubernetes ecoystem.

I think I can donate my work as community-supported addon although it is
designed as a Brooklyn location not a simple application blueprint, but
worth noticing that it was not part of the core but was committed in
`brooklyn-server/locations/container` module, which seems a perfect match
to me,
I undestand the helm location works with the classic launcher, but there is
an extra complexity to OSGify grpc (a transitive dependency of
microbean-helm) to make it work with brooklyn-karaf.

I may resubmit the contribution without bringing in the microbean-helm java
client but writing a simpler restful client for Helm/Tiller, but not sure
how hard it would be to refactor the location yet.

Best,
Andrea



On Tue, 17 Apr 2018 at 11:50 Richard Downer  wrote:

> All,
>
> I'm sorry but I'm not really aware of the Kubernetes ecosystem in much
> detail. Do we have an indication of how much demand there is for Helm
> support in Brooklyn?
>
> This sounds like a significant change - both to our build process, but also
> our release process and website (supporting multiple platform downloads).
> I'd be opposed to doing it unless there's a significant demand for Helm.
> I'd prefer to see this as a community-supported addon (like most of our
> blueprints are) instead of being core Brooklyn, until there's evidence of
> demand for this to be in core.
>
> Richard.
>
>
>
> On 17 April 2018 at 09:07, Geoff Macartney 
> wrote:
>
> > hi Thomas
> >
> > It certainly doesn't sound right to have to have separate builds of
> > Brooklyn for different platforms, and especially not just for this
> > feature.  Can we build the dependency package into a bundle for each
> > platform, so that it is the only thing that is platform specific, and
> > provide all three bundles along with the distribution of Brooklyn? Then
> on
> > whatever target platform you are on, OSX, Linux, Windows, you install the
> > right dependency bundle for that platform into Brooklyn's Karaf?
> >
> > Geoff
> >
> >
> >
> > On Mon, 16 Apr 2018 at 12:18 Thomas Bouron  > com>
> > wrote:
> >
> > > Hi Brooklyners.
> > >
> > > You might have noticed that the Brooklyn builds started to fail more
> than
> > > usual recently. I spent some time last week to fix those issues but I
> > just
> > > realised that there is a deeper one with the recent change I merged
> > (about
> > > Kubernetes Helm)
> > >
> > > This change requires a dependency, which depends on some native code.
> > Now,
> > > this dependency exists for 3 platforms: macOS, Linux and Window. The
> > issue
> > > is that the "right" dependency is included at build time via the maven
> > > classifier, and the way it is picked is by looking at the current build
> > OS
> > > and selecting the corresponding one[1]. Obviously, this leads to nasty
> > > problem: as all our builds are done on Linux, Brooklyn artifacts only
> > work
> > > on Linux. Not only that, the classifier is dynamically picked and set
> as
> > > envvar by a plugin. This is also an issue for any downstream projects.
> > >
> > > While we want this feature in Brooklyn, I don't think this is
> acceptable
> > > for our users therefore I reverted the changes and started this
> > > conversation. What do you think would be the best approach to fix this?
> > > Having 1 build per platform doesn't sound good as it won't be portable
> > > anymore. Maybe we can find another dependency without a link to native
> > > code?
> > >
> > > Best.
> > >
> > > [1]
> > >
> > > https://github.com/apache/brooklyn-dist/pull/119/files#diff-
> > 01b5eceed5fb6499e99a42e711695648R139
> > > --
> > >
> > > Thomas Bouron • Senior Software Engineer @ Cloudsoft Corporation •
> > > https://cloudsoft.io/
> > > Github: https://github.com/tbouron
> > > Twitter: https://twitter.com/eltibouron
> > >
> >
>

[DISCUSSION] Kubernetes Helm

[Thomas Bouron]

Hi Brooklyners.

You might have noticed that the Brooklyn builds started to fail more than
usual recently. I spent some time last week to fix those issues but I just
realised that there is a deeper one with the recent change I merged (about
Kubernetes Helm)

This change requires a dependency, which depends on some native code. Now,
this dependency exists for 3 platforms: macOS, Linux and Window. The issue
is that the "right" dependency is included at build time via the maven
classifier, and the way it is picked is by looking at the current build OS
and selecting the corresponding one[1]. Obviously, this leads to nasty
problem: as all our builds are done on Linux, Brooklyn artifacts only work
on Linux. Not only that, the classifier is dynamically picked and set as
envvar by a plugin. This is also an issue for any downstream projects.

While we want this feature in Brooklyn, I don't think this is acceptable
for our users therefore I reverted the changes and started this
conversation. What do you think would be the best approach to fix this?
Having 1 build per platform doesn't sound good as it won't be portable
anymore. Maybe we can find another dependency without a link to native code?

Best.

[1]
https://github.com/apache/brooklyn-dist/pull/119/files#diff-01b5eceed5fb6499e99a42e711695648R139
-- 

Thomas Bouron • Senior Software Engineer @ Cloudsoft Corporation •
https://cloudsoft.io/
Github: https://github.com/tbouron
Twitter: https://twitter.com/eltibouron