Re: [osgi-dev] Best way to move an OSGi application to Docker plus K8 ?

[Peter Kriens via osgi-dev]

Christian,

Yes, for JPM, which feels very long ago, I had a base image and then added only 
an executable JAR as a new layer.

However, since I've seen the stats for porn sites on the Internet the size of 
images got me a lot less worried :-) I've spent a lot of my working life trying 
to make things work in small spaces and trying to make them efficient. However, 
in the last decade I've been bypassed by several people that just could not 
care less about these issues and thereby gained speed. So I am trying to also 
become less obsessed with efficiency because it does remove a lot of 
complexity. Today, quite often keeping it simple and stupid is actually good 
enough.

Kind regards,

Peter Kriens



> On 7 Aug 2019, at 09:49, Christian Schneider  wrote:
> 
> Hi Peter,
> 
> I also thought a bit about how to possibly make the deployments smaller.
> 
> I see three possible solutions:
> 1. Put the main version of a system in a docker layer and upgrades in a layer 
> on top.
> 
> An example where this might make sense is Adobe Experience Manager. You put 
> the bundles of AEM 6.5 in one docker layer. Each file name must uniquely map 
> to a bundle symbolic name + version (or maven coordinates).
> Then for any updates you overwrite or add the changed bundles in a layer on 
> top and overwrite the runbundles list.
> 
> Even if you write out the full list of bundles again for any update docker 
> will be able to create an effective diff that makes your new docker image 
> pretty small.
> 
> 2. Put your dependencies in one docker layer and your own bundles in another
> 
> Typically you have more changes to your own code than changes to 
> dependencies. Also typically your own code is much smaller than your 
> dependencies. 
> 
> So this approach can also make your docker images (needed for an upgrade) a 
> lot smaller.
> 
> 3. Pull bundles from maven repository (maybe with local caching)
> 
> In this case you only deploy the list of runbundles with maven coordinates 
> and a starter in the docker image. The starter then downloads the bundles 
> from a maven repo. To make this fast you would need a local cache of bundles 
> that survives restart of a pod. (Not sure how to best achieve this for k8s 
> but I am sure it can be done).
> 
> What do you think? What concept do you have in mind?
> 
> Christian
> 
> Am Di., 6. Aug. 2019 um 16:58 Uhr schrieb Peter Kriens via osgi-dev 
> mailto:osgi-dev@mail.osgi.org>>:
> I don't see any reason why upgrading your existing workflow with a static 
> Docker container and then updating the bundles would not work.
> 
> However ...
> 
> Just look at the number of moving parts that you then need in runtime. 
> Creating a Docker image in the build is trivial and deploying it to 
> Kubernetes is, well less than trivial. You need to have a lot of things going 
> right on different systems and much more configuration to make the dynamic 
> update work. All these moving parts can fail.
> 
> The only issue is size & time. If you need to deploy a full docker image to 
> hundreds of thousands of machines the update can be done more efficiently 
> using OSGi bundles. And actually that is a solution I'm working on at the 
> moment. However, if time/size is not a concern I find a full docker image 
> disturbingly hard to beat. About 5 years ago I ran the 'Java Package Manager' 
> web site on Kubernetes and it worked scarily easy and reliable.
> 
> When I am hired to help one of the first things I look for is reduce as many 
> of the moving parts as possible. Yes, you can get anything to work but 
> reducing the possible errors cases really increases reliability imho.
> 
> Kind regards,
> 
> Peter Kriens
> 
> 
> > On 6 Aug 2019, at 16:28, Cristiano via osgi-dev  > > wrote:
> > 
> > Hello all,
> > 
> > I have a challenged POC to do in order to dockerize an existent OSGi
> > based application and then deploy it to a Kubernetes based cloud.
> > 
> > I'm not totally aware of k8 features yet, so I have some doubts that I
> > would like discuss here.
> > 
> > The main doubt is related to our existent upgrading process which
> > currently we upload a R5 repository to a webserver and then a node
> > management agent bundle access it and upgrades the necessary app bundles.
> > 
> > Many examples I saw in the web creates a docker image in their building
> > process and delivery an image at each dev cycle. I don't like much of
> > this idea, so initially I thought to mimic our existent process in a
> > docker container just setting up a Volume in order to upload the newer
> > repositories.
> > 
> > So I have created two docker images for testing this locally. One image
> > contains the OSGI container and framework bundle (that do not change
> > much) and other image that extends the other for only the apps bundles.
> > it have worked well running locally.
> > 
> > Would this also work in K8 ?  What would happen if I need to scale and
> > then create 

Re: [osgi-dev] Best way to move an OSGi application to Docker plus K8 ?

[Christian Schneider via osgi-dev]

Hi Peter,

I also thought a bit about how to possibly make the deployments smaller.

I see three possible solutions:
1. Put the main version of a system in a docker layer and upgrades in a
layer on top.

An example where this might make sense is Adobe Experience Manager. You put
the bundles of AEM 6.5 in one docker layer. Each file name must uniquely
map to a bundle symbolic name + version (or maven coordinates).
Then for any updates you overwrite or add the changed bundles in a layer on
top and overwrite the runbundles list.

Even if you write out the full list of bundles again for any update docker
will be able to create an effective diff that makes your new docker image
pretty small.

2. Put your dependencies in one docker layer and your own bundles in another

Typically you have more changes to your own code than changes to
dependencies. Also typically your own code is much smaller than your
dependencies.

So this approach can also make your docker images (needed for an upgrade) a
lot smaller.

3. Pull bundles from maven repository (maybe with local caching)

In this case you only deploy the list of runbundles with maven coordinates
and a starter in the docker image. The starter then downloads the bundles
from a maven repo. To make this fast you would need a local cache of
bundles that survives restart of a pod. (Not sure how to best achieve this
for k8s but I am sure it can be done).

What do you think? What concept do you have in mind?

Christian

Am Di., 6. Aug. 2019 um 16:58 Uhr schrieb Peter Kriens via osgi-dev <
osgi-dev@mail.osgi.org>:

> I don't see any reason why upgrading your existing workflow with a static
> Docker container and then updating the bundles would not work.
>
> However ...
>
> Just look at the number of moving parts that you then need in runtime.
> Creating a Docker image in the build is trivial and deploying it to
> Kubernetes is, well less than trivial. You need to have a lot of things
> going right on different systems and much more configuration to make the
> dynamic update work. All these moving parts can fail.
>
> The only issue is size & time. If you need to deploy a full docker image
> to hundreds of thousands of machines the update can be done more
> efficiently using OSGi bundles. And actually that is a solution I'm working
> on at the moment. However, if time/size is not a concern I find a full
> docker image disturbingly hard to beat. About 5 years ago I ran the 'Java
> Package Manager' web site on Kubernetes and it worked scarily easy and
> reliable.
>
> When I am hired to help one of the first things I look for is reduce as
> many of the moving parts as possible. Yes, you can get anything to work but
> reducing the possible errors cases really increases reliability imho.
>
> Kind regards,
>
> Peter Kriens
>
>
> > On 6 Aug 2019, at 16:28, Cristiano via osgi-dev 
> wrote:
> >
> > Hello all,
> >
> > I have a challenged POC to do in order to dockerize an existent OSGi
> > based application and then deploy it to a Kubernetes based cloud.
> >
> > I'm not totally aware of k8 features yet, so I have some doubts that I
> > would like discuss here.
> >
> > The main doubt is related to our existent upgrading process which
> > currently we upload a R5 repository to a webserver and then a node
> > management agent bundle access it and upgrades the necessary app bundles.
> >
> > Many examples I saw in the web creates a docker image in their building
> > process and delivery an image at each dev cycle. I don't like much of
> > this idea, so initially I thought to mimic our existent process in a
> > docker container just setting up a Volume in order to upload the newer
> > repositories.
> >
> > So I have created two docker images for testing this locally. One image
> > contains the OSGI container and framework bundle (that do not change
> > much) and other image that extends the other for only the apps bundles.
> > it have worked well running locally.
> >
> > Would this also work in K8 ?  What would happen if I need to scale and
> > then create multiple PODS for this application?
> >
> > thanks for any help.
> >
> > best regards,
> >
> > Cristiano
> >
> >
> > ___
> > OSGi Developer Mail List
> > osgi-dev@mail.osgi.org
> > https://mail.osgi.org/mailman/listinfo/osgi-dev
>
> ___
> OSGi Developer Mail List
> osgi-dev@mail.osgi.org
> https://mail.osgi.org/mailman/listinfo/osgi-dev
>


-- 
-- 
Christian Schneider
http://www.liquid-reality.de

Computer Scientist
http://www.adobe.com
___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev

Re: [osgi-dev] Best way to move an OSGi application to Docker plus K8 ?

[Christian Schneider via osgi-dev]

The basic idea of docker as well as k8s are immutable images.
The advantage of these is that you can give them a version and this way
uniquely identify what is running in the cloud at any time.

The upgrade process in k8s typically is that a new pod is installed with
the new docker image. As soon as it reports ready the old pod is taken
down. So this ensures your system is always available.

The simplest way to apply this pattern in OSGi is to create a an
application using a bndrun file. The resulting jar is easy to package into
a docker container. The only thing I miss there is to not have a single jar
but a starter and individual bundles as files. This would make it easier to
use docker layers to minimize the size of a docker image.

You can try to use you current upgrade scheme in docker / k8s but it will
be very different from its normal operation and I suspect you will hit many
small issues.

Of course giving up live upgrades takes away one OSGi feature but to be
honest I never used this much for OSGi production systems.

Christian



Am Di., 6. Aug. 2019 um 16:29 Uhr schrieb Cristiano via osgi-dev <
osgi-dev@mail.osgi.org>:

> Hello all,
>
> I have a challenged POC to do in order to dockerize an existent OSGi
> based application and then deploy it to a Kubernetes based cloud.
>
> I'm not totally aware of k8 features yet, so I have some doubts that I
> would like discuss here.
>
> The main doubt is related to our existent upgrading process which
> currently we upload a R5 repository to a webserver and then a node
> management agent bundle access it and upgrades the necessary app bundles.
>
> Many examples I saw in the web creates a docker image in their building
> process and delivery an image at each dev cycle. I don't like much of
> this idea, so initially I thought to mimic our existent process in a
> docker container just setting up a Volume in order to upload the newer
> repositories.
>
> So I have created two docker images for testing this locally. One image
> contains the OSGI container and framework bundle (that do not change
> much) and other image that extends the other for only the apps bundles.
> it have worked well running locally.
>
> Would this also work in K8 ?  What would happen if I need to scale and
> then create multiple PODS for this application?
>
> thanks for any help.
>
> best regards,
>
> Cristiano
>
>
> ___
> OSGi Developer Mail List
> osgi-dev@mail.osgi.org
> https://mail.osgi.org/mailman/listinfo/osgi-dev



-- 
-- 
Christian Schneider
http://www.liquid-reality.de

Computer Scientist
http://www.adobe.com
___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev

Re: [osgi-dev] Best way to move an OSGi application to Docker plus K8 ?

[Peter Kriens via osgi-dev]

I don't see any reason why upgrading your existing workflow with a static 
Docker container and then updating the bundles would not work.

However ...

Just look at the number of moving parts that you then need in runtime. Creating 
a Docker image in the build is trivial and deploying it to Kubernetes is, well 
less than trivial. You need to have a lot of things going right on different 
systems and much more configuration to make the dynamic update work. All these 
moving parts can fail.

The only issue is size & time. If you need to deploy a full docker image to 
hundreds of thousands of machines the update can be done more efficiently using 
OSGi bundles. And actually that is a solution I'm working on at the moment. 
However, if time/size is not a concern I find a full docker image disturbingly 
hard to beat. About 5 years ago I ran the 'Java Package Manager' web site on 
Kubernetes and it worked scarily easy and reliable.

When I am hired to help one of the first things I look for is reduce as many of 
the moving parts as possible. Yes, you can get anything to work but reducing 
the possible errors cases really increases reliability imho.

Kind regards,

Peter Kriens


> On 6 Aug 2019, at 16:28, Cristiano via osgi-dev  
> wrote:
> 
> Hello all,
> 
> I have a challenged POC to do in order to dockerize an existent OSGi
> based application and then deploy it to a Kubernetes based cloud.
> 
> I'm not totally aware of k8 features yet, so I have some doubts that I
> would like discuss here.
> 
> The main doubt is related to our existent upgrading process which
> currently we upload a R5 repository to a webserver and then a node
> management agent bundle access it and upgrades the necessary app bundles.
> 
> Many examples I saw in the web creates a docker image in their building
> process and delivery an image at each dev cycle. I don't like much of
> this idea, so initially I thought to mimic our existent process in a
> docker container just setting up a Volume in order to upload the newer
> repositories.
> 
> So I have created two docker images for testing this locally. One image
> contains the OSGI container and framework bundle (that do not change
> much) and other image that extends the other for only the apps bundles.
> it have worked well running locally.
> 
> Would this also work in K8 ?  What would happen if I need to scale and
> then create multiple PODS for this application?
> 
> thanks for any help.
> 
> best regards,
> 
> Cristiano
> 
> 
> ___
> OSGi Developer Mail List
> osgi-dev@mail.osgi.org
> https://mail.osgi.org/mailman/listinfo/osgi-dev

___
OSGi Developer Mail List
osgi-dev@mail.osgi.org
https://mail.osgi.org/mailman/listinfo/osgi-dev