Hi All,
Following up recent discussions that I had with Imesh, Isuru and Indika on
the improvements that we can make
on our container based deployment space, I will be first doing a PoC (Proof
of Concept) on
a little bit, re-structured deployment artifact management hierarchy as
follows.
Above diagram splits our deployment space into three layers, namely :
[1] Configuration Management Space ( Container-based / Non-container-based
) :
For a particular product, this layer would support configuring "N"
number of deployment patterns for both container and non-container-based
environments.
As of now, we are mostly dependent on puppet for building
configurations / distributions for a specific deployment pattern of a
product. But here, in an important note: I am bringing in
a separate repository as being discussed at [1] to keep directly
deployable configurations + supporting artifacts for each deployment
pattern. We can use this repository, i.e.
for example [2] as the default entry-point for configuring a
deployment, so that the existing usage of puppet for doing the same becomes
optional.
Configurations maintained in this repository would either be a direct
input for the container image management space in building product images
or be an input for building product
distributions that will then be used at the container image management
space for building the images.
[2] Container Image Management Space :
This layer would support the same "N" number of deployment patterns,
referenced above for container-based environments.
It would take in configurations or product distributions, specific to
a particular deployment pattern from configuration management space as
input and produce production ready container images
to be utilized by container-based deployment management space, as
output.
Docker (Using Dockerfile) is our current implementation of this space,
but in future, we may also have to bring in other technologies ( such as
Rocket ) too in to this space depending on the need.
As an important note: Currently we are keeping docker-compose related
artifacts too with in this space [3]. But my personal preference would be
to keep it separate and decoupled from the
docker image management layer, as a separate repository (as positioned
in the above diagram), so that those artifacts would only be downloaded
by customers who are really in need of them. Please share your
thoughts on this.
[3] Container-based Deployment Management Space :
This layer would also support the same "N" number of deployment
patterns, referenced above for container-based environments.
It would take in a set of container images, specific to a particular
deployment pattern from container image management space, as input and
produce a container-based deployment
as output. Docker-compose, kubernetes and OpenShift are our current
implementations of this space.
As an important note: Currently, If we refer to the recent
kubernetes-apim 2.1.0 [4] effort, we are keeping all these technologies,
combined and coupled into one repository. But my personal reference would
be to
keep them separate and decoupled from each other in separate
repositories as visualized in the above diagram, so that we can direct our
customers to the exact technology that they prefer and those
artifacts would only be downloaded by those who are really in need of
them. Please share your thoughts on this.
A work-in-progress effort, adhering to the above re-structure can be found
at [5] for kubernetes-ei with related configuration artifacts at conf-ei
[6].
To build the required images, currently I am using the existing approach of
docker-ei dockerfiles with default provisioning.
Your feedback is deeply appreciated.
References :
- - - - - - - - - - -
[1] [Strategy] Proposal to create Configuration Repositories for all
products
[2] conf-ei repository: https://github.com/DilanUA/conf-ei/,
wso2-apim-conf: https://github.com/imesh/wso2-apim-conf
[3] docker-ei repository: https://github.com/wso2/docker-ei, docker-apim:
https://github.com/wso2/docker-apim
[4] kubernetes-apim: https://github.com/wso2/kubernetes-apim/tree/2.1.0
[5] kubernetes-ei effort:
https://github.com/DilanUA/kubernetes-ei/tree/master/pattern-2/wso2ei-integrator-ha-analytics-stdln
[6] conf-ei effort:
https://github.com/DilanUA/conf-ei/tree/master/pattern-2/wso2ei-integrator-ha-analytics-stdln
[7] docker-ei dockerfile:
https://github.com/wso2/docker-ei/tree/master/dockerfile
Thanks,
Dilan.
*Dilan U. Ariyaratne*
Senior Software Engineer
WSO2 Inc. <http://wso2.com/>
Mobile: +94766405580 <%2B94766405580>
lean . enterprise . middleware
On Wed, Jul 26, 2017 at 8:57 AM, Dilan Udara Ariyaratne <[email protected]>
wrote:
> Hi All,
>
> With respect to "container-based" deployment patterns related to EI,
> following were identified as the first set to be supported and are
> currently under development.
>
> Pattern 1: Standalone Integrator Profile with external DB,
> Scalable, But Minimum High Availability Cluster for
> Integrator Profile with external DB
>
> Pattern 2: Standalone MB Profile with external DB,
> Scalable, But Minimum High Availability Cluster for MB
> Profile with external DB
>
> Pattern 3: Standalone BPS Profile with external DB,
> Scalable, But Minimum High Availability Cluster for BPS
> Profile with external DB
>
> Pattern 4: Standalone [ Integrator Profile + EI Analytics Profile ] with
> external DB,
> Scalable, But Minimum [ High Availability Cluster for
> Integrator Profile + High Availability Cluster for EI Analytics Profile ]
> with external DB
>
> Pattern 5: Standalone [ Integrator Profile + MB Profile ] with external
> DB,
> Scalable, But Minimum [ High Availability Cluster for
> Integrator Profile + High Availability Cluster for MB Profile ] with
> external DB
>
> Pattern 6: Standalone [ Integrator Profile + EI Analytics Profile + MB
> Profile ] with external DB,
> Scalable, But Minimum [ High Availability Cluster for
> Integrator Profile + High Availability Cluster for MB Profile + High
> Availability Cluster for MB Profile ] with external DB
>
> Please add and correct, if I have missed any.
>
> Thanks,
> Dilan.
>
>
> *Dilan U. Ariyaratne*
> Senior Software Engineer
> WSO2 Inc. <http://wso2.com/>
> Mobile: +94766405580 <%2B94766405580>
> lean . enterprise . middleware
>
>
> On Mon, Jul 24, 2017 at 8:46 AM, Dilan Udara Ariyaratne <[email protected]>
> wrote:
>
>>
>> Hi Isuru and Imesh,
>>
>> On Thu, Jul 20, 2017 at 7:20 PM, Imesh Gunaratne <[email protected]> wrote:
>>
>>>
>>>
>>> On Thu, Jul 20, 2017 at 6:06 AM, Isuru Haththotuwa <[email protected]>
>>> wrote:
>>>
>>>> [ += Architecture ]
>>>>
>>>> On Thu, Jul 20, 2017 at 2:57 PM, Isuru Haththotuwa <[email protected]>
>>>> wrote:
>>>>
>>>>> Hi Dilan,
>>>>>
>>>>> Apologies for the delayed response.
>>>>>
>>>>> A couple of changes that I can think of:
>>>>>
>>>>> - Rather than using Puppet to build the Docker images, use a
>>>>> simple copy based approach.
>>>>> - IMHO using puppet to build docker images is an overkill. We
>>>>> can have one product base images, and build the images relevant to
>>>>> product
>>>>> specific deployment patterns extending from the base image.
>>>>>
>>>>>
>>> +1 Isuru! We also had an offline discussion on this with Lakmal. Better
>>> to make usage of Puppet optional for K8S based deployments.
>>>
>>
>> I am also +1 to the idea that we should totally decouple Puppet from
>> Docker as well as K8s and treat each of these separately as they cater
>> different layers of the deployment spectrum.
>>
>> When it comes to a "deployment pattern", each product pack participating
>> in that deployment pattern, has some specific configuration to use.
>>
>> Configuration can also differ depending on the fact that deployment is
>> hosted in a container-based environment or not.
>>
>> We can support automating both these environments either by using
>> [1] a simple copy based approach, [2] puppet or any other configuration /
>> artifact update mechanism that we may bring in future.
>>
>> IMO, we can keep this layer totally decoupled from either the Docker or
>> Kubernetes layers and what we can expect as the output, would be a fully
>> configured product pack to cater a specific deployment pattern.
>>
>> Once the pack is there, we can use it directly at the docker layer to
>> build an image as the output and that image will then be an input to the
>> K8s layer to host the specific deployment pattern for which it was built.
>>
>> WDYT?
>>
>> Thanks,
>> Dilan.
>>
>>
>>> Thanks
>>> Imesh
>>>
>>>>
>>>>> -
>>>>> - Without using an intermediate set of scripts do build docker
>>>>> images (currently we have our own docker build, run scripts), let the
>>>>> user
>>>>> directly use Docker API for building images, running them, etc.
>>>>> - For a Docker user its more natural to use the Docker API.
>>>>> Additionally, there would be no need to maintain our own build
>>>>> scripts.
>>>>>
>>>>> Please share your thoughts.
>>>>> I have started this effort for APIM the new deployment patterns
>>>>> discussed in thread [1] in APIM group, and the WIP artifacts can be found
>>>>> at [2] and [3]. Note that these artifacts are not finalized.
>>>>>
>>>>> [1]. API-M perf results to share with a customer
>>>>> [2]. https://github.com/isurulucky/docker-apim/tree/new-deploymen
>>>>> t-patterns
>>>>> [3]. https://github.com/isurulucky/kubernetes-apim/tree/new-deplo
>>>>> yment-patterns
>>>>>
>>>>> On Thu, Jul 13, 2017 at 7:56 PM, Dilan Udara Ariyaratne <
>>>>> [email protected]> wrote:
>>>>>
>>>>>>
>>>>>> ---------- Forwarded message ----------
>>>>>> From: Dilan Udara Ariyaratne <[email protected]>
>>>>>> Date: Thu, Jul 13, 2017 at 7:48 PM
>>>>>> Subject: [Architecture] [Kubernetes] Improving Kubernetes Deployment
>>>>>> Support for WSO2 Products
>>>>>> To: architecture <[email protected]>
>>>>>>
>>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I am currently working on $subject. Initial idea is to deliver a
>>>>>> fully automated, stable Kubernetes deployment experience for the end
>>>>>> users
>>>>>> of both WSO2 EI and APIM products
>>>>>> such that during the process, we can understand how we can improve
>>>>>> Kubernetes-common, the base Kubernetes deployment enabling layer as its
>>>>>> platform.
>>>>>>
>>>>>> Earlier with our old product strategy, we did maintain a repository
>>>>>> called, kubernetes-artifacts
>>>>>> <https://github.com/wso2/kubernetes-artifacts> where all the product
>>>>>> specific artifacts were also kept.
>>>>>>
>>>>>> Now, we have split this out in to two levels, namely
>>>>>> [1] kubernetes-common <https://github.com/wso2/kubernetes-common> -
>>>>>> base Kubernetes deployment enabling layer
>>>>>> [2] kubernetes-<product-label>, i.e. for example, kubernetes-ei -
>>>>>> Product specific kubernetes artifacts that integrates with
>>>>>> kubernetes-common
>>>>>>
>>>>>> However, with our new product strategy and architectural changes,
>>>>>> most of the product-specific kubernetes artifacts are not yet done and
>>>>>> WSO2
>>>>>> EI, WSO2 APIM are two such products.
>>>>>>
>>>>>> While there is an on-going effort by Isuru (Isuruh) on finalizing
>>>>>> kubernetes-apim related deployment artifacts, I will be working on
>>>>>> kubernetes-ei related artifacts
>>>>>> as an effort to improve Kubernetes Deployment Support for WSO2
>>>>>> Products.
>>>>>>
>>>>>> As mentioned above, during this process, we will be also accessing
>>>>>> ways of improving Kubernetes-common layer for a much-developer friendly
>>>>>> framework in building
>>>>>> more customer centric production ready kubernetes deployment
>>>>>> artifacts with very minimum effort.
>>>>>>
>>>>>> Thanks,
>>>>>> Dilan.
>>>>>>
>>>>>> *Dilan U. Ariyaratne*
>>>>>> Senior Software Engineer
>>>>>> WSO2 Inc. <http://wso2.com/>
>>>>>> Mobile: +94766405580 <%2B94766405580>
>>>>>> lean . enterprise . middleware
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Thanks and Regards,
>>>>>
>>>>> Isuru H.
>>>>> +94 716 358 048 <071%20635%208048>* <http://wso2.com/>*
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Thanks and Regards,
>>>>
>>>> Isuru H.
>>>> +94 716 358 048 <+94%2071%20635%208048>* <http://wso2.com/>*
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Imesh Gunaratne*
>>> WSO2 Inc: http://wso2.com
>>> T: +94 11 214 5345 M: +94 77 374 2057 <+94%2077%20374%202057>
>>> W: https://medium.com/@imesh TW: @imesh
>>> lean. enterprise. middleware
>>>
>>>
>>
>
_______________________________________________
Architecture mailing list
[email protected]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
