Re: [Architecture] [X509 Authenticator] Certificate Revocation Verification with CRL and OCSP

[Afkham Azeez]

Approved

On Feb 12, 2018 10:26 AM, "Indunil Upeksha Rathnayake" 
wrote:

Hi Maheshika,

Can you please create a new git repository with the name "
identity-x509-revocation" under wso2-extensions, for moving this feature
implementation.

Thanks and Regards

On Wed, Jan 17, 2018 at 11:20 AM, Indunil Upeksha Rathnayake <
indu...@wso2.com> wrote:

> Hi,
>
> Please find the following considerations on proceeding with the CRL/OCSP
> certificate validation. Appreciate your ideas and comments on this.
>
> *Store root CA and intermediate CA certificates in registry*
>
>- As per the current implementation, trust stores which are having CA
>certificates that should be used for CRL and OCSP validation, can be
>configured file based as follows.
>
>
>
>http://wso2.org/project
>s/carbon/certificate-validation.xml
>">
>
>…….
>
>
>
>   *truststorePass="cacertspassword" type="JKS"/>*
>
>
>
>
>
>
>- In server start up and in tenant initial activation, all the
>certificates in the trust stores will added to the registry.
>
> Registry path : 
> *repository/security/certificate/certificate-authorities/ Issuer DN>/*
>
> Normalized Issuer DN : UTF-8 URL encoded issuer DN while replacing the "%"
> with ":" in the URL encoded value, in order to support the Illegal
> characters for registry resource path (Check mail in [1]).
> Certificate Serial Num : Positive integer assigned by the CA to each
> certificate which is unique for each certificate issued by a given CA
> (i.e., the issuer name and serial number identify a unique certificate)
>
> Registry Content : *X509Certificate CA Certificate*
> Registry Properties :
> *crl - comma separated CRL URLs*
> *ocsp - comma separated OCSP URLs*
>
>- There can be certificates of Intermediate CAs and root CAs in the
>trust store (An intermediate CA is an entity that can sign certificates on
>behalf of the root CA. The root CA signs the intermediate certificate,
>forming a chain of trust). All those intermediate  and root certificates
>will be added into the registry considering following scenarios.
>
>
>1. Intermediate CAs have their own CRL and OCSP URLs
>
> Revocation of certificate is not propagated across the CA tree. Each CA
> (root and intermediate) is responsible of the publication of the CRL
> containing the list of only the revoked certificates that were issued by
> that CA and OCSP response for only the certificates that were issued by
> that CA.
>
>2. Observer intermediate CA certificates in the chain
> of trust
> For full-chain CRL/OCSP validation as mention in below, need to have
> access to intermediate CA certificates in the chain of trust to retrieve
> the CRL and OCSP Urls
>
>
> *Need of CA and intermediate CA Certificates for CRL/OCSP validation*
>
>- CRL Validation :
>   - If the CA certificate available in the registry which issued the
>   client certificate, retrieve the CRL URLs from the CA cert. If not use 
> the
>   URLs in the client certificate.
>   - In order to validate intermediate CAs when Full-chain CRL/OCSP
>   validation enabled, CRL URls will be extracted from corresponding root 
> CA.
>
>
>- OCSP Validation :
>   - For OCSP validation, CA certificate which issued the client
>   certificate should be available in the registry. Otherwise consider as a
>   unsuccessful validation.
>   - In order to validate intermediate CAs when Full-chain CRL/OCSP
>   validation enabled, OCSP URLs will be extracted from corresponding root 
> CA.
>
>
>
> *Configure Full-chain CRL/OCSP Validation *
>
> 
>
> http://wso2.org/project
> s/carbon/certificate-validation.xml">
>
> 
>
>  name="org.wso2.carbon.identity.x509Certificate.validation.validator.CRLValidator"
> displayName="CRLValidator" enable="false">
>
>   1
>
>*true*
>
>
> 
>
>  name="org.wso2.carbon.identity.x509Certificate.validation.validator.OCSPValidator"
> displayName="OCSPValidator" enable="true">
>
>   2
>
>true
>
> 
>
> 
>
> 
>
>- *Full-chain CRL/OCSP validation disabled:* validate the client
>certificate with the CRL/OCSP URLs of the issuer CA
>
>
>- *Full-chain CRL/OCSP validation enabled*: validate with the CRL/OCSP
>of every intermediate certificate within the chain of trust for the client,
>except for the root CA certificate.
>
> Ex:  Root CA (root CA CRL) Cert ==> Intermediate CA Cert(inter CA CRL) ==>
> Client Cert
> The intermediate CA CRL is used to verify whether the client certificate
> is valid. The root CA CRL is used to verify whether Intermediate CA Cert is
> valid.
>
> If Full-chain CRL/OCSP checking enabled, If the intermediate certificates
> are not available in the registry or, if a CRL is not available in any of
> the intermediate CA in the chain of trust, or the certificate is 

Re: [Architecture] Get rid of java2wsdl, wsdl2java sh/bat scripts from IS 5.5.0 bin directory

[Afkham Azeez]

Yeah, please get rid of these. We discussed the same sometime back and I
was under the impression that these were already removed.

On Thu, Jan 11, 2018 at 7:52 AM, Sagara Gunathunga <sag...@wso2.com> wrote:

> IS bin directory contains following set of sh/bat files, ATM these are
> exists due to historical reasons only couldn't find any real usage. If
> there is no objection I would like to discard them from 5.5.0 WDYT ?
>
> java2wsdl.sh
> java2wsdl.bat
> wsdl2java.bat
> wsdl2java.sh
> tcpmon-1.0.jar
> tcpmon.sh
> tcpmon.bat
>
>
> Thanks !
> --
> Sagara Gunathunga
>
> Director; WSO2, Inc.;  http://wso2.com
> Linkedin; http://www.linkedin.com/in/ssagara
> Blog ;  http://ssagara.blogspot.com
> Mobile : +9471 <+94%2071%20565%209887>2149951
>
>


-- 
*Afkham Azeez*
Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] C5 clustering

[Afkham Azeez]

No we don't encourage distributed caches anymore. As you may have heard "There
are only two hard things in Computer Science: cache invalidation and naming
things. -- Phil Karlton" :)

So you are supposed to use a product specific DB and if required, a local
cache.

Thanks
Azeez

On Fri, Dec 22, 2017 at 7:12 PM, Lahiru Sandaruwan <lahi...@wso2.com> wrote:

> Thank you for the clarification Azeez. I believe each product has to build
> or use a library for distributed cache requirements. Isn’t it a strong case
> to have a common way of clustering?
>
> Thanks.
>
> On Fri, Dec 22, 2017 at 3:53 AM Afkham Azeez <az...@wso2.com> wrote:
>
>> There is no such thing as C5 clustering. For each product, clustering
>> would mean a different thing. Some of the core decisions we have made
>> related to multi-tenancy include, each tenant will be running in separate
>> containers/processes and there will be no in-VM multi-tenancy.
>>
>> So when it comes to clustering, you need to refer to the product specific
>> docs.
>>
>> Thanks
>> Azeez
>>
>> On Fri, Dec 22, 2017 at 12:08 PM, Lahiru Sandaruwan <lahi...@wso2.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> Do we have any docs/ details on $subject?
>>>
>>> Thanks.
>>>
>>> --
>>> --
>>>
>>> Lahiru Sandaruwan
>>> Associate Technical Lead,
>>> WSO2 Inc., http://wso2.com
>>>
>>> lean.enterprise.middleware
>>>
>>> m: +1 901 530 2379 <+1%20901-530-2379>
>>> e: lahi...@wso2.com b: https://medium.com/@lahirugmg
>>> in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>>>
>>>
>>
>>
>> --
>> *Afkham Azeez*
>> Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
>> Member; Apache Software Foundation; http://www.apache.org/
>> * <http://www.apache.org/>*
>> *email: **az...@wso2.com* <az...@wso2.com>
>> * cell: +94 77 3320919 <077%20332%200919>blog: **http://blog.afkham.org*
>> <http://blog.afkham.org>
>> *twitter: **http://twitter.com/afkham_azeez*
>> <http://twitter.com/afkham_azeez>
>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>> <http://lk.linkedin.com/in/afkhamazeez>*
>>
>> *Lean . Enterprise . Middleware*
>>
> --
> --
>
> Lahiru Sandaruwan
> Associate Technical Lead,
> WSO2 Inc., http://wso2.com
>
> lean.enterprise.middleware
>
> m: +1 901 530 2379 <+1%20901-530-2379>
> e: lahi...@wso2.com b: https://medium.com/@lahirugmg
> in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
>


-- 
*Afkham Azeez*
Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] C5 clustering

[Afkham Azeez]

There is no such thing as C5 clustering. For each product, clustering would
mean a different thing. Some of the core decisions we have made related to
multi-tenancy include, each tenant will be running in separate
containers/processes and there will be no in-VM multi-tenancy.

So when it comes to clustering, you need to refer to the product specific
docs.

Thanks
Azeez

On Fri, Dec 22, 2017 at 12:08 PM, Lahiru Sandaruwan <lahi...@wso2.com>
wrote:

> Hi All,
>
> Do we have any docs/ details on $subject?
>
> Thanks.
>
> --
> --
>
> Lahiru Sandaruwan
> Associate Technical Lead,
> WSO2 Inc., http://wso2.com
>
> lean.enterprise.middleware
>
> m: +1 901 530 2379 <+1%20901-530-2379>
> e: lahi...@wso2.com b: https://medium.com/@lahirugmg
> in: http://lk.linkedin.com/pub/lahiru-sandaruwan/16/153/146
>
>


-- 
*Afkham Azeez*
Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [IAM] SCIM 2.0 Outbound Connector

[Afkham Azeez]

Now you need approval for 2 repos of 3 repos?

On Tue, Nov 21, 2017 at 2:53 PM, Isura Karunaratne <is...@wso2.com> wrote:

> Repo name for outbound connector
>
>- * identity-outbound-provisioning-scim2*
>
> Repo name for the scim2 client.
>
>-
> * identity-client-scim2 *
>
>
> @isuraranga,
>
> Why do we need the scim2-commons repo? Can't we use Charon for that?
>
> Thanks
> Isura.
>
> On Mon, Nov 20, 2017 at 3:04 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> What is the repo name?
>>
>> On Tue, Nov 7, 2017 at 1:06 PM, Maheshika Goonetilleke <
>> mahesh...@wso2.com> wrote:
>>
>>> Hi Azeez
>>>
>>> Please confirm.
>>>
>>> On Tue, Nov 7, 2017 at 11:23 AM, Johann Nallathamby <joh...@wso2.com>
>>> wrote:
>>>
>>>> Hi Maheshika,
>>>>
>>>> Can we have following 3 repos for this project under wso2-extensions
>>>> organization?
>>>>
>>>> 1. *identity-outbound-provisioning-scim2*
>>>>
>>>> For the outbound connector
>>>>
>>>> 2. *identity-scim2-common*
>>>>
>>>> For common utilities for inbound and outbound connectors. E.g.
>>>> AttributeMapper class in inbound connector which is needed for outbound
>>>> connector as well.
>>>>
>>>> 3. *identity-client-scim2*
>>>>
>>>> For SCIM2 client generated using SCIM2 swagger files. This will be used
>>>> by outbound connector as well as can be used by anyone as standalone
>>>> client. Ideally this should be used for the scim2 compliance test suite as
>>>> well, but have failed to do so.
>>>>
>>>> Regards,
>>>> Johann.
>>>>
>>>> On Mon, Oct 16, 2017 at 2:21 PM, Johann Nallathamby <joh...@wso2.com>
>>>> wrote:
>>>>
>>>>> Yes, I also think we need to take the approach of using the Swagger
>>>>> files and generate SDK because that is what standard Rest API world will 
>>>>> be
>>>>> doing. We can find any issues early.
>>>>>
>>>>> Regards,
>>>>> Johann.
>>>>>
>>>>> On Mon, Oct 16, 2017 at 2:18 PM, Gayan Gunawardana <ga...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Oct 16, 2017 at 1:21 PM, Isuranga Perera <
>>>>>> isurangamper...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Gayan,
>>>>>>>
>>>>>>> In that case, I'll try to create an SDK from swagger and use it as
>>>>>>> the client.
>>>>>>>
>>>>>> That would be great.
>>>>>>
>>>>>>>
>>>>>>> Best Regards
>>>>>>>
>>>>>>> On Mon, Oct 16, 2017 at 9:12 AM, Gayan Gunawardana <ga...@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Since you are looking for abstraction layer, can implement
>>>>>>>> something like [1] for SCIM2 as well.
>>>>>>>>
>>>>>>>> [1] https://github.com/wso2-extensions/identity-inbound-provisio
>>>>>>>> ning-scim/blob/master/components/org.wso2.carbon.identity.sc
>>>>>>>> im.common/src/main/java/org/wso2/carbon/identity/scim/common
>>>>>>>> /impl/ProvisioningClient.java
>>>>>>>>
>>>>>>>> On Sun, Oct 15, 2017 at 11:16 PM, Gayan Gunawardana <ga...@wso2.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sun, Oct 15, 2017 at 8:39 PM, Johann Nallathamby <
>>>>>>>>> joh...@wso2.com> wrote:
>>>>>>>>>
>>>>>>>>>> *[+ IsharaK, Omindu, Farasath]*
>>>>>>>>>>
>>>>>>>>>> On Sun, Oct 15, 2017 at 7:34 PM, Isuranga Perera <
>>>>>>>>>> isurangamper...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I went through the scim2-compliance-test-suite [1] source code,
&

Re: [Architecture] [IAM] SCIM 2.0 Outbound Connector

[Afkham Azeez]

On Tue, Nov 21, 2017 at 9:12 PM, Johann Nallathamby <joh...@wso2.com> wrote:

>
>
> On Tue, Nov 21, 2017 at 2:53 PM, Isura Karunaratne <is...@wso2.com> wrote:
>
>> Repo name for outbound connector
>>
>>- * identity-outbound-provisioning-scim2*
>>
>> Repo name for the scim2 client.
>>
>>-
>> * identity-client-scim2 *
>>
>>
> +1 for above two repos.
>
> *@Azeez*: please approve above two repos in wso2-extensions.
>

Approved

>
>>-
>>
>>
>> @isuraranga,
>>
>> Why do we need the scim2-commons repo? Can't we use Charon for that?
>>
>
> Discussed offline with Isuranga and decided we can use Charon repo for
> those common packages because they are SCIM2 specific and not carbon
> specific. So no need scim2-common for now.
>
> Regards,
> Johann.
>
>
>>
>> Thanks
>> Isura.
>>
>> On Mon, Nov 20, 2017 at 3:04 PM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> What is the repo name?
>>>
>>> On Tue, Nov 7, 2017 at 1:06 PM, Maheshika Goonetilleke <
>>> mahesh...@wso2.com> wrote:
>>>
>>>> Hi Azeez
>>>>
>>>> Please confirm.
>>>>
>>>> On Tue, Nov 7, 2017 at 11:23 AM, Johann Nallathamby <joh...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Maheshika,
>>>>>
>>>>> Can we have following 3 repos for this project under wso2-extensions
>>>>> organization?
>>>>>
>>>>> 1. *identity-outbound-provisioning-scim2*
>>>>>
>>>>> For the outbound connector
>>>>>
>>>>> 2. *identity-scim2-common*
>>>>>
>>>>> For common utilities for inbound and outbound connectors. E.g.
>>>>> AttributeMapper class in inbound connector which is needed for outbound
>>>>> connector as well.
>>>>>
>>>>> 3. *identity-client-scim2*
>>>>>
>>>>> For SCIM2 client generated using SCIM2 swagger files. This will be
>>>>> used by outbound connector as well as can be used by anyone as standalone
>>>>> client. Ideally this should be used for the scim2 compliance test suite as
>>>>> well, but have failed to do so.
>>>>>
>>>>> Regards,
>>>>> Johann.
>>>>>
>>>>> On Mon, Oct 16, 2017 at 2:21 PM, Johann Nallathamby <joh...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Yes, I also think we need to take the approach of using the Swagger
>>>>>> files and generate SDK because that is what standard Rest API world will 
>>>>>> be
>>>>>> doing. We can find any issues early.
>>>>>>
>>>>>> Regards,
>>>>>> Johann.
>>>>>>
>>>>>> On Mon, Oct 16, 2017 at 2:18 PM, Gayan Gunawardana <ga...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Oct 16, 2017 at 1:21 PM, Isuranga Perera <
>>>>>>> isurangamper...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Gayan,
>>>>>>>>
>>>>>>>> In that case, I'll try to create an SDK from swagger and use it as
>>>>>>>> the client.
>>>>>>>>
>>>>>>> That would be great.
>>>>>>>
>>>>>>>>
>>>>>>>> Best Regards
>>>>>>>>
>>>>>>>> On Mon, Oct 16, 2017 at 9:12 AM, Gayan Gunawardana <ga...@wso2.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Since you are looking for abstraction layer, can implement
>>>>>>>>> something like [1] for SCIM2 as well.
>>>>>>>>>
>>>>>>>>> [1] https://github.com/wso2-extensions/identity-inbound-provisio
>>>>>>>>> ning-scim/blob/master/components/org.wso2.carbon.identity.sc
>>>>>>>>> im.common/src/main/java/org/wso2/carbon/identity/scim/common
>>>>>>>>> /impl/ProvisioningClient.java
>>>>>>>>>
>>>>>>>>> On Sun, Oct 15, 2017 at 11:16 PM, Gayan Gunawardana <
>>>>>>>>> ga...@wso2.com> wrote:
>>>>>&

Re: [Architecture] [IAM] SCIM 2.0 Outbound Connector

[Afkham Azeez]

>>> 2.0 client.
>>>>>>>>>
>>>>>>>> Most feasible way is to go with apache commons HttpClient  but
>>>>>>> better to give a try with swagger doc as well.
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>1. Separate SCIM 2.0 client from [1]
>>>>>>>>>2. Separate SCIM 2.0 client from [2]
>>>>>>>>>3. Use swagger doc [3] to generate client
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1] https://github.com/wso2-incubator/scim2-compliance-test-suite
>>>>>>>>> [2] https://github.com/HansageeSJ/scim-client
>>>>>>>>> [3] https://wso2.org/jira/browse/IDENTITY-5695
>>>>>>>>>
>>>>>>>>> Appreciate any suggestions.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best Regards
>>>>>>>>> Isuranga Perera
>>>>>>>>>
>>>>>>>>> On Fri, Oct 13, 2017 at 9:42 AM, Gayan Gunawardana <ga...@wso2.com
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Oct 12, 2017 at 5:33 PM, Johann Nallathamby <
>>>>>>>>>> joh...@wso2.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Oct 12, 2017 at 1:28 PM, Isuranga Perera <
>>>>>>>>>>> isurangamper...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi IAM Team,
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Currently, there is no $subject. Therefore I'm looking at
>>>>>>>>>>>> implementing a SCIM2 Outbound Connector. I'm looking at
>>>>>>>>>>>> identity-outbound-provisioning-scim [1]
>>>>>>>>>>>> and scim2-compliance-test-suite [2]. Appreciate further
>>>>>>>>>>>> suggestions.
>>>>>>>>>>>>
>>>>>>>>>>> Hi Isuranga,
>>>>>>>>>>
>>>>>>>>>> It should be same as [1] you just have to think SCIM provider is
>>>>>>>>>> version 2 and send http requests according to SCIM2 format. As a 
>>>>>>>>>> starting
>>>>>>>>>> point you can setup existing SCIM provisioning connector and debug 
>>>>>>>>>> from
>>>>>>>>>> point [1] so you will understand the flow.
>>>>>>>>>>
>>>>>>>>>> [1] https://github.com/wso2-extensions/identity-outbound-provisi
>>>>>>>>>> oning-scim/blob/master/components/org.wso2.carbon.identity.p
>>>>>>>>>> rovisioning.connector.scim/src/main/java/org/wso2/carbon/ide
>>>>>>>>>> ntity/provisioning/connector/scim/SCIMProvisioningConnector.
>>>>>>>>>> java#L99
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://github.com/wso2-extensions/identity-outbound-pro
>>>>>>>>>>>> visioning-scim
>>>>>>>>>>>> [2] https://github.com/wso2-incubator/scim2-compliance-test-
>>>>>>>>>>>> suite
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Best Regards
>>>>>>>>>>>> Isuranga Perera
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ___
>>>>>>>>>>>> Architecture mailing list
>>>>>>>>>>>> Architecture@wso2.org
>>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Thanks & Regards,
>>>>>>>>>>>
>>>>>>>>>>> *Johann Dilantha Nallathamby*
>>>>>>>>>>> Senior Lead Solutions Engineer
>>>>>>>>>>> WSO2, Inc.
>>>>>>>>>>> lean.enterprise.middleware
>>>>>>>>>>>
>>>>>>>>>>> Mobile - *+9476950*
>>>>>>>>>>> Blog - *http://nallaa.wordpress.com
>>>>>>>>>>> <http://nallaa.wordpress.com>*
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Gayan Gunawardana
>>>>>>>>>> Senior Software Engineer; WSO2 Inc.; http://wso2.com/
>>>>>>>>>> Email: ga...@wso2.com
>>>>>>>>>> Mobile: +94 (71) 8020933
>>>>>>>>>>
>>>>>>>>>> ___
>>>>>>>>>> Architecture mailing list
>>>>>>>>>> Architecture@wso2.org
>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Thanks & Regards,
>>>>>>>>
>>>>>>>> *Johann Dilantha Nallathamby*
>>>>>>>> Senior Lead Solutions Engineer
>>>>>>>> WSO2, Inc.
>>>>>>>> lean.enterprise.middleware
>>>>>>>>
>>>>>>>> Mobile - *+9476950*
>>>>>>>> Blog - *http://nallaa.wordpress.com <http://nallaa.wordpress.com>*
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Gayan Gunawardana
>>>>>>> Senior Software Engineer; WSO2 Inc.; http://wso2.com/
>>>>>>> Email: ga...@wso2.com
>>>>>>> Mobile: +94 (71) 8020933
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Gayan Gunawardana
>>>>>> Senior Software Engineer; WSO2 Inc.; http://wso2.com/
>>>>>> Email: ga...@wso2.com
>>>>>> Mobile: +94 (71) 8020933
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Gayan Gunawardana
>>>> Senior Software Engineer; WSO2 Inc.; http://wso2.com/
>>>> Email: ga...@wso2.com
>>>> Mobile: +94 (71) 8020933
>>>>
>>>
>>>
>>>
>>> --
>>> Thanks & Regards,
>>>
>>> *Johann Dilantha Nallathamby*
>>> Senior Lead Solutions Engineer
>>> WSO2, Inc.
>>> lean.enterprise.middleware
>>>
>>> Mobile - *+9476950*
>>> Blog - *http://nallaa.wordpress.com <http://nallaa.wordpress.com>*
>>>
>>
>>
>>
>> --
>> Thanks & Regards,
>>
>> *Johann Dilantha Nallathamby*
>> Senior Lead Solutions Engineer
>> WSO2, Inc.
>> lean.enterprise.middleware
>>
>> Mobile - *+9476950*
>> Blog - *http://nallaa.wordpress.com <http://nallaa.wordpress.com>*
>>
>
>
>
> --
>
> Thanks & Best Regards,
>
> Maheshika Goonetilleke
> Senior Engineering Process Coordinator
>
> *WSO2 Inc*
> *email   : mahesh...@wso2.com <mahesh...@wso2.com>*
> *mobile : +94 773 596707 <077%20359%206707>*
> *www: :http://wso2.com <http://wso2.com/>*lean . enterprise . middleware
>
>
>
>
>


-- 
*Afkham Azeez*
Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] License header in confi files

[Afkham Azeez]

On Wed, Jun 28, 2017 at 11:59 AM, KasunG Gajasinghe <kas...@wso2.com> wrote:

>
> +1 Azeez. And, we should not blindly add them either. In the latest
> carbon-commons version, the INFO message box displays the license text. [1]
> :(
>

Yes, I have not seen many other projects adding license headers to conf
files users edit. I think we should implement this for all future releases.

>
> [1] https://github.com/wso2/carbon-commons/issues/273
>
> On Wed, Jun 28, 2017 at 11:55 AM, Afkham Azeez <az...@wso2.com> wrote:
>
>> I was wondering whether we really need the license header in config
>> files. These are files that will be edited by the users, and it really
>> doesn't make much sense to have the license there, and also it makes the
>> files unnecessarily long.
>>
>> How about removing the license headers from config files, going forward?
>>
>> --
>> *Afkham Azeez*
>> Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
>> Member; Apache Software Foundation; http://www.apache.org/
>> * <http://www.apache.org/>*
>> *email: **az...@wso2.com* <az...@wso2.com>
>> * cell: +94 77 3320919 <077%20332%200919>blog: **http://blog.afkham.org*
>> <http://blog.afkham.org>
>> *twitter: **http://twitter.com/afkham_azeez*
>> <http://twitter.com/afkham_azeez>
>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>> <http://lk.linkedin.com/in/afkhamazeez>*
>>
>> *Lean . Enterprise . Middleware*
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
>
> *Kasun Gajasinghe*Associate Technical Lead, WSO2 Inc.
> email: kasung AT spamfree wso2.com
> linked-in: http://lk.linkedin.com/in/gajasinghe
> blog: http://kasunbg.org
> phone: +1 650-745-4499 <+1%20650-745-4499>, 77 678 0813
>
>
> ___
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
*Afkham Azeez*
Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

[Architecture] License header in confi files

[Afkham Azeez]

I was wondering whether we really need the license header in config files.
These are files that will be edited by the users, and it really doesn't
make much sense to have the license there, and also it makes the files
unnecessarily long.

How about removing the license headers from config files, going forward?

-- 
*Afkham Azeez*
Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [APIM] [MSF4J] Reason for creating separate micro service for each path defined swagger

[Afkham Azeez]

mpl}
> - Added microservice: org.wso2.carbon.apimgt.rest.
> api.store.PoliciesApi@50ba8085
> [2017-03-09 12:44:44,380]  INFO 
> {org.wso2.msf4j.internal.MicroservicesRegistryImpl}
> - Added microservice: org.wso2.carbon.apimgt.rest.
> api.store.PoliciesApi@50ba8085
> [2017-03-09 12:44:44,424]  INFO 
> {org.wso2.msf4j.internal.MicroservicesRegistryImpl}
> - Added microservice: org.wso2.carbon.apimgt.rest.
> api.store.SubscriptionsApi@18da9675
> [2017-03-09 12:44:44,474]  INFO 
> {org.wso2.msf4j.internal.MicroservicesRegistryImpl}
> - Added microservice: org.wso2.carbon.apimgt.rest.
> api.store.SubscriptionsApi@18da9675
> [2017-03-09 12:44:44,525]  INFO 
> {org.wso2.msf4j.internal.MicroservicesRegistryImpl}
> - Added microservice: org.wso2.carbon.apimgt.rest.
> api.store.TagsApi@1e835f96
> [2017-03-09 12:44:44,573]  INFO 
> {org.wso2.msf4j.internal.MicroservicesRegistryImpl}
> - Added microservice: org.wso2.carbon.apimgt.rest.
> api.store.TagsApi@1e835f96
> [2017-03-09 12:44:44,843]  INFO 
> {org.wso2.msf4j.internal.MicroservicesServerSC}
> - All microservices are available
> [2017-03-09 12:44:44,844]  INFO {org.wso2.carbon.transport.
> http.netty.internal.NettyTransportServiceComponent} - All
> CarbonNettyServerInitializers are available
> [2017-03-09 12:44:44,856]  INFO 
> {org.wso2.msf4j.internal.MicroservicesRegistryImpl}
> - Added microservice: org.wso2.carbon.uuf.httpconnector.msf4j.
> UUFMicroservice@1a3c0a9c
> [2017-03-09 12:44:44,856]  INFO 
> {org.wso2.msf4j.internal.MicroservicesRegistryImpl}
> - Added microservice: org.wso2.carbon.uuf.httpconnector.msf4j.
> UUFMicroservice@1a3c0a9c
> [2017-03-09 12:44:44,857]  WARN {org.wso2.carbon.kernel.
> internal.startupresolver.StartupComponentManager} - You are trying to add
> an available capability org.wso2.msf4j.Microservice from
> bundle(org.wso2.carbon.uuf.httpconnector.msf4j:1.0.0.m13) to an already
> activated startup listener component wso2-microservices-server in
> bundle(msf4j-core:2.1.1). Refer the Startup Order Resolver documentation
> and validated your configuration
> [2017-03-09 12:44:44,857]  INFO {org.wso2.carbon.uuf.
> httpconnector.msf4j.internal.MSF4JHttpConnector} - UUF app
> 'org.wso2.carbon.apimgt.web.store' is available at '/store'.
> [2017-03-09 12:44:44,859]  INFO 
> {org.wso2.msf4j.internal.MicroservicesRegistryImpl}
> - Added microservice: org.wso2.carbon.uuf.httpconnector.msf4j.
> UUFMicroservice@3178a583
> [2017-03-09 12:44:44,859]  INFO 
> {org.wso2.msf4j.internal.MicroservicesRegistryImpl}
> - Added microservice: org.wso2.carbon.uuf.httpconnector.msf4j.
> UUFMicroservice@3178a583
> [2017-03-09 12:44:44,859]  WARN {org.wso2.carbon.kernel.
> internal.startupresolver.StartupComponentManager} - You are trying to add
> an available capability org.wso2.msf4j.Microservice from
> bundle(org.wso2.carbon.uuf.httpconnector.msf4j:1.0.0.m13) to an already
> activated startup listener component wso2-microservices-server in
> bundle(msf4j-core:2.1.1). Refer the Startup Order Resolver documentation
> and validated your configuration
> [2017-03-09 12:44:44,860]  INFO {org.wso2.carbon.uuf.
> httpconnector.msf4j.internal.MSF4JHttpConnector} - UUF app
> 'org.wso2.carbon.apimgt.web.publisher' is available at '/publisher'.
> [2017-03-09 12:44:44,861]  INFO 
> {org.wso2.msf4j.internal.MicroservicesRegistryImpl}
> - Added microservice: org.wso2.carbon.uuf.httpconnector.msf4j.
> UUFMicroservice@2d9caac3
> [2017-03-09 12:44:44,861]  INFO 
> {org.wso2.msf4j.internal.MicroservicesRegistryImpl}
> - Added microservice: org.wso2.carbon.uuf.httpconnector.msf4j.
> UUFMicroservice@2d9caac3
>
> Do we really need to create new micro service for each path? What's the
> reason behind this?
>
> Some of the services below can be combined to one, for example
> org.wso2.carbon.apimgt.rest.api.publisher.ApisApi,
> org.wso2.carbon.apimgt.rest.api.publisher.ApplicationsApi,
> org.wso2.carbon.apimgt.rest.api.publisher.EndpointsApi,
> org.wso2.carbon.apimgt.rest.api.publisher.EnvironmentsApi,
> org.wso2.carbon.apimgt.rest.api.publisher.LabelsApi,
> org.wso2.carbon.apimgt.rest.api.publisher.SubscriptionsApi can be
> combined to one micro service, since they can be functionally categorized
> together as publisher micro service.
>
> org.wso2.carbon.apimgt.rest.api.store.ApisApi,
> org.wso2.carbon.apimgt.rest.api.store.ApplicationsApi,
> org.wso2.carbon.apimgt.rest.api.store.PoliciesApi,
> org.wso2.carbon.apimgt.rest.api.store.SubscriptionsApi,
> org.wso2.carbon.apimgt.rest.api.store.TagsApi can be combined to one
> micro service, since they can be functionally categorized together as store
> micro service.
>
> Also, we need to eliminate micro service on which runs on HTTP and need to
> have only HTTP

Re: [Architecture] Carbon C5 - Server Configuration Model

[Afkham Azeez]

Yes we need to do that.


On Thu, Feb 2, 2017 at 10:32 AM, Thusitha Thilina Dayaratne <
thusit...@wso2.com> wrote:

> Hi All,
>
> Since MSF4J supports non-OSGi mode, IMHO it would be great if we move this
> out from the carbon.core and create a separate top level component which
> will support both OSGi and standalone mode. Then we can use this to
> configure the executor pool, according to the new carbon transport
> architecture.
> WDYT?
>
> Thanks
> Thusitha
>
> On Thu, Feb 2, 2017 at 10:16 AM, Sagara Gunathunga <sag...@wso2.com>
> wrote:
>
>>
>> We (myself and Azeez) had a offline chat about this requirement, it seems
>> it would be much flexible for both users and product teams if
>> deployment.yaml processing framework can support 3 levels instead of 2 :
>> Component, Product, User
>>
>> Component level  : - Component author define default values within the
>> bean class if  'default values' concept is applicable.
>> Product level:-  Product team override component level default
>> values using 'defaults.yaml' file ( within the WSO2 file space )
>> User level :-  User can override component or product level
>> default values using deployment.yaml file
>>
>> The main advantage here is we can clearly separate responsibilities and
>> scope of each group and possible to ship deployment.yaml with absolutely
>> zero content.  This is the same concept suggested by Ruwan with few
>> modifications.
>>
>> Thanks !
>>
>> On Thu, Feb 2, 2017 at 9:36 AM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> This is out of the scope of the deployment.yaml processing framework
>>> Danesh wrote. If you want to have default connectors or config, you can
>>> either write a product-specific component which programmatically creates
>>> those, or you can ship that in the product specific deployment.yaml.
>>>
>>> On Thu, Feb 2, 2017 at 7:24 AM, Sagara Gunathunga <sag...@wso2.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Wed, Feb 1, 2017 at 11:11 PM, Danesh Kuruppu <dan...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Jayanga,
>>>>>
>>>>> it is defaulted to the component. any product which is using the
>>>>> component will get the same default values. If a product need values other
>>>>> than the default value, they have to override it in the deployment.yaml.
>>>>> default values should be component related values, not related for the
>>>>> specific product.
>>>>>
>>>>
>>>> This is true in ideal cases but practically we have more complex use
>>>> cases, taking the same example identity-mgt is a generic F/W kind of a
>>>> component which allows to register/manage IdentityStore connectors and
>>>> there is no component level default connector concept, it's just a
>>>> registration/management capability, only in product level(IS) we ship
>>>> default  IdentityStore connectors.  Here we have 2 options ..
>>>>
>>>> 1.) As there is no default connector in component level, all the
>>>> products including IS has to provide default connectors  in
>>>>  deployment.yaml, then this is not kind of value overridden and when number
>>>> of such components increase default size of the deployment.yaml will
>>>> increase which again result into deviate from original motivation of
>>>> deployment.yaml.
>>>>
>>>>
>>>> 2. ) In cases of components do not have default values we can hard cord
>>>> default values according to the base product, in this way at least base
>>>> product  can keep deployment.yaml clean.
>>>>
>>>> WDYT ?
>>>>
>>>>
>>>> Thanks !
>>>>
>>>>
>>>>>
>>>>> Thanks
>>>>> Danesh
>>>>>
>>>>> On Wed, Feb 1, 2017 at 3:58 AM, Jayanga Kaushalya <jayan...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> If we are hard-coding default values to the bean class, are those
>>>>>> values should be default to the component or to the product which is
>>>>>> (frequently) using that component ? If we use default values related to 
>>>>>> the
>>>>>> product then we can use that values directly in the specific product and 
>>>>>> if
>>

Re: [Architecture] Carbon C5 - Server Configuration Model

[Afkham Azeez]

This is out of the scope of the deployment.yaml processing framework Danesh
wrote. If you want to have default connectors or config, you can either
write a product-specific component which programmatically creates those, or
you can ship that in the product specific deployment.yaml.

On Thu, Feb 2, 2017 at 7:24 AM, Sagara Gunathunga <sag...@wso2.com> wrote:

>
>
> On Wed, Feb 1, 2017 at 11:11 PM, Danesh Kuruppu <dan...@wso2.com> wrote:
>
>> Hi Jayanga,
>>
>> it is defaulted to the component. any product which is using the
>> component will get the same default values. If a product need values other
>> than the default value, they have to override it in the deployment.yaml.
>> default values should be component related values, not related for the
>> specific product.
>>
>
> This is true in ideal cases but practically we have more complex use
> cases, taking the same example identity-mgt is a generic F/W kind of a
> component which allows to register/manage IdentityStore connectors and
> there is no component level default connector concept, it's just a
> registration/management capability, only in product level(IS) we ship
> default  IdentityStore connectors.  Here we have 2 options ..
>
> 1.) As there is no default connector in component level, all the products
> including IS has to provide default connectors  in  deployment.yaml, then
> this is not kind of value overridden and when number of such components
> increase default size of the deployment.yaml will increase which again
> result into deviate from original motivation of deployment.yaml.
>
>
> 2. ) In cases of components do not have default values we can hard cord
> default values according to the base product, in this way at least base
> product  can keep deployment.yaml clean.
>
> WDYT ?
>
>
> Thanks !
>
>
>>
>> Thanks
>> Danesh
>>
>> On Wed, Feb 1, 2017 at 3:58 AM, Jayanga Kaushalya <jayan...@wso2.com>
>> wrote:
>>
>>> Hi,
>>>
>>> If we are hard-coding default values to the bean class, are those values
>>> should be default to the component or to the product which is
>>> (frequently) using that component ? If we use default values related to the
>>> product then we can use that values directly in the specific product and if
>>> some other product is using that component, they have to override it in the
>>> deployment.yaml. For example product-is is using component identity-mgt. So
>>> what should be the default values for the config files coming from
>>> identity-mgt component ? Are those should be defaulted to the product-is
>>> related values or to the component related values and product-is should
>>> always override them from deployment.yaml.
>>>
>>> Thanks!
>>>
>>> *Jayanga Kaushalya*
>>> Software Engineer
>>> Mobile: +94777860160 <+94%2077%20786%200160>
>>> WSO2 Inc. | http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> On Wed, Nov 30, 2016 at 10:57 AM, Danesh Kuruppu <dan...@wso2.com>
>>> wrote:
>>>
>>>> Hi Dilan,
>>>>
>>>> If all user-configurable properties are not readily available in the
>>>>> .yaml file by default, how would a user know which
>>>>> properties are configurable and which are not ?
>>>>>
>>>>
>>>> All the configurable properties and their default values will be
>>>> documented. We are going to create this config document automatically by
>>>> reading the config bean class (using maven plugin).
>>>> We need to decide whether we pack those config documents in the product
>>>> or add to central location (doc page etc)
>>>>
>>>> Thanks
>>>> --
>>>>
>>>> *Danesh Kuruppu*
>>>> Senior Software Engineer | WSO2
>>>>
>>>> Email: dan...@wso2.com
>>>> Mobile: +94 (77) 1690552 <+94%2077%20169%200552>
>>>> Web: WSO2 Inc <https://wso2.com/signature>
>>>>
>>>>
>>>> ___
>>>> Architecture mailing list
>>>> Architecture@wso2.org
>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>
>>>>
>>>
>>
>>
>> --
>>
>> *Danesh Kuruppu*
>> Senior Software Engineer | WSO2
>>
>> Email: dan...@wso2.com
>> Mobile: +94 (77) 1690552 <+94%2077%20169%200552>
>> Web: WSO2 Inc <https://wso2.com/signature>
>>
>>
>
>
> --
> Sagara Gunathunga
>
> Associate Director / Architect; WSO2, Inc.;  http://wso2.com
> V.P Apache Web Services;http://ws.apache.org/
> Linkedin; http://www.linkedin.com/in/ssagara
> Blog ;  http://ssagara.blogspot.com
>
>


-- 
*Afkham Azeez*
Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [C5] [Carbon-Feature-Plugin] Dynamic Creation of carbon.product via a Template

[Afkham Azeez]

Yeah that approach sounds good

On Jan 27, 2017 4:30 PM, "Dilan Udara Ariyaratne" <dil...@wso2.com> wrote:

>
>
> *Dilan U. Ariyaratne*
> Senior Software Engineer
> WSO2 Inc. <http://wso2.com/>
> Mobile: +94766405580 <%2B94766405580>
> lean . enterprise . middleware
>
>
> On Thu, Jan 26, 2017 at 10:06 AM, Kishanthan Thangarajah <
> kishant...@wso2.com> wrote:
>
>>
>>
>> On Mon, Jan 23, 2017 at 11:49 AM, Dilan Udara Ariyaratne <dil...@wso2.com
>> > wrote:
>>
>>> Hi Azeez,
>>>
>>> Even with the pom based approach (as noted by Kishanthan), we do not
>>> have the luxury of totally getting rid of this file, carbon.product
>>>
>>
>> I believe this is only needed at product build time, so we do not need to
>> keep a file in the repo but only create the file during build time and then
>> delete it (or may be use the target directory so that it will be anyway
>> removed).
>>
> +1 to create the file at target directory during build time, I just tested
> the implementation and it works fine. So, in going forward, we can
> deprecate the static carbon.product file that exists in distribution folder
> of the repo.
>
>>
>> since both underlying, but external tycho and equinox launcher plug-ins
>>> used by our carbon-feature-plugin require this file as an input.
>>>
>>> So IMO, the only improvement that we can introduce here is supporting
>>> templated-dynamic-creation of the file at carbon-feature-plugin level
>>> using the standard carbon kernel version values available in the
>>> distribution pom.
>>>
>>> Thanks,
>>> Dilan.
>>>
>>> *Dilan U. Ariyaratne*
>>> Senior Software Engineer
>>> WSO2 Inc. <http://wso2.com/>
>>> Mobile: +94766405580 <%2B94766405580>
>>> lean . enterprise . middleware
>>>
>>>
>>> On Mon, Jan 23, 2017 at 7:33 AM, Kishanthan Thangarajah <
>>> kishant...@wso2.com> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Jan 20, 2017 at 5:57 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>
>>>>> I would suggest totally getting rid of it.
>>>>>
>>>>
>>>> To maintain backward compatibility of the plugin, we need to have the
>>>> file based supported. But from next major release of the plugin, we can
>>>> remove the usage of this file and use the pom based approach only.
>>>>
>>>> Thanks,
>>>>
>>>>>
>>>>> On Fri, Jan 20, 2017 at 5:24 PM, KasunG Gajasinghe <kas...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> +1. carbon.product file hasn't really been used by the products. So,
>>>>>> +1 to make it optional.
>>>>>>
>>>>>> On Fri, Jan 20, 2017 at 3:06 PM, Dilan Udara Ariyaratne <
>>>>>> dil...@wso2.com> wrote:
>>>>>>
>>>>>>> Hi Folks,
>>>>>>>
>>>>>>> In the process of building C5, we currently require carbon.product
>>>>>>> for the following goals.
>>>>>>> [1] publish-product
>>>>>>> [2] generate-runtime
>>>>>>>
>>>>>>> This file maintains current version of the carbon kernel to be
>>>>>>> utilized by "carbon-feature-plugin" in the build process.
>>>>>>> Keeping this value in carbon.product prevents the kernel from been
>>>>>>> auto-released as it requires manual intervention to bump
>>>>>>> version values as necessary during the release process.
>>>>>>>
>>>>>>> In order to solve this issue, we are currently in the process of
>>>>>>> improving Carbon-Feature-Plugin to dynamically create this file during
>>>>>>> build time using
>>>>>>> a template where the necessary version value information is read
>>>>>>> from corresponding distribution pom file.
>>>>>>>
>>>>>>> In order to support backward compatibility, we will still maintain
>>>>>>> the original approach of keeping a carbon.product file somewhere
>>>>>>> appropriate
>>>>>>> in the distribution folder and read it accordingly when
>>>>>>>  tag is present in the pom file.
>>>>>>>
>>>>

Re: [Architecture] [C5] [Carbon-Feature-Plugin] Dynamic Creation of carbon.product via a Template

[Afkham Azeez]

I would suggest totally getting rid of it.

On Fri, Jan 20, 2017 at 5:24 PM, KasunG Gajasinghe <kas...@wso2.com> wrote:

>
> +1. carbon.product file hasn't really been used by the products. So, +1
> to make it optional.
>
> On Fri, Jan 20, 2017 at 3:06 PM, Dilan Udara Ariyaratne <dil...@wso2.com>
> wrote:
>
>> Hi Folks,
>>
>> In the process of building C5, we currently require carbon.product for
>> the following goals.
>> [1] publish-product
>> [2] generate-runtime
>>
>> This file maintains current version of the carbon kernel to be utilized
>> by "carbon-feature-plugin" in the build process.
>> Keeping this value in carbon.product prevents the kernel from been
>> auto-released as it requires manual intervention to bump
>> version values as necessary during the release process.
>>
>> In order to solve this issue, we are currently in the process of
>> improving Carbon-Feature-Plugin to dynamically create this file during
>> build time using
>> a template where the necessary version value information is read from
>> corresponding distribution pom file.
>>
>> In order to support backward compatibility, we will still maintain the
>> original approach of keeping a carbon.product file somewhere appropriate
>> in the distribution folder and read it accordingly when
>>  tag is present in the pom file.
>>
>> In the meantime, as the way to go forward, we will introduce the
>> following.
>>
>> Carbon-Feature-Plugin will be updated to read version and other optional
>> values that were originally persisted in the file, from the pom itself.
>> After reading these values, plugin will dynamically create the
>> carbon.product which will then be taken into reference by underlying
>> eclipse.tycho plugin as in the usual way of execution.
>>
>> WDYT ?
>>
>> Thank You.
>>
>> *Dilan U. Ariyaratne*
>> Senior Software Engineer
>> WSO2 Inc. <http://wso2.com/>
>> Mobile: +94766405580 <%2B94766405580>
>> lean . enterprise . middleware
>>
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
>
> *Kasun Gajasinghe*Associate Technical Lead, WSO2 Inc.
> email: kasung AT spamfree wso2.com
> linked-in: http://lk.linkedin.com/in/gajasinghe
> blog: http://kasunbg.org
> phone: +1 650-745-4499 <+1%20650-745-4499>, 77 678 0813
>
>
> ___
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
*Afkham Azeez*
Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [C5] [Carbon-Feature-Plugin] Dynamic Creation of carbon.product via a Template

[Afkham Azeez]

Sounds good. We can simply rely on the pom and not duplicate the version in
different files.

On Fri, Jan 20, 2017 at 3:06 PM, Dilan Udara Ariyaratne <dil...@wso2.com>
wrote:

> Hi Folks,
>
> In the process of building C5, we currently require carbon.product for the
> following goals.
> [1] publish-product
> [2] generate-runtime
>
> This file maintains current version of the carbon kernel to be utilized by
> "carbon-feature-plugin" in the build process.
> Keeping this value in carbon.product prevents the kernel from been
> auto-released as it requires manual intervention to bump
> version values as necessary during the release process.
>
> In order to solve this issue, we are currently in the process of improving
> Carbon-Feature-Plugin to dynamically create this file during build time
> using
> a template where the necessary version value information is read from
> corresponding distribution pom file.
>
> In order to support backward compatibility, we will still maintain the
> original approach of keeping a carbon.product file somewhere appropriate
> in the distribution folder and read it accordingly when
>  tag is present in the pom file.
>
> In the meantime, as the way to go forward, we will introduce the following.
>
> Carbon-Feature-Plugin will be updated to read version and other optional
> values that were originally persisted in the file, from the pom itself.
> After reading these values, plugin will dynamically create the
> carbon.product which will then be taken into reference by underlying
> eclipse.tycho plugin as in the usual way of execution.
>
> WDYT ?
>
> Thank You.
>
> *Dilan U. Ariyaratne*
> Senior Software Engineer
> WSO2 Inc. <http://wso2.com/>
> Mobile: +94766405580 <%2B94766405580>
> lean . enterprise . middleware
>
>
> _______
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
*Afkham Azeez*
Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [Dev] [C5] MSF4J Interceptors need to be configurable.

[Afkham Azeez]

On Wed, Dec 7, 2016 at 5:17 PM, Ishara Cooray <isha...@wso2.com> wrote:

> Hi Thilina,
>
> And also if there are multiple interceptors and one interceptor returns
> false from its' preCaall then the invocation chain will not continue
> further.
>
> So Is this implies if preCall returns 'true' then the invocation chain
> will continue further?
>

Yes


> If that is the case we can return true in our overridden preCall method so
> that it goes to next Interceptor.
>
>
> Thanks & Regards,
> Ishara Cooray
> Senior Software Engineer
> Mobile : +9477 262 9512 <077%20262%209512>
> WSO2, Inc. | http://wso2.com/
> Lean . Enterprise . Middleware
>
> On Wed, Dec 7, 2016 at 2:33 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> How about supporting JAXRS filters?
>>
>> On Wed, Dec 7, 2016 at 12:52 PM, Thusitha Thilina Dayaratne <
>> thusit...@wso2.com> wrote:
>>
>>> Hi Ishara,
>>>
>>> As you have mentioned, with the current architecture we can't set the
>>> specific interceptor for a particular service but rather to all services in
>>> the registry. And also if there are multiple interceptors and one
>>> interceptor returns false from its' preCaall then the invocation chain will
>>> not continue further.
>>>
>>> IMHO we have few options
>>>
>>>- We can implement a way to register specific interceptors to
>>>specific services
>>>- We can support JAX-RS Filters
>>>- We can provide a way to skip some interceptors for specific
>>>services
>>>
>>> @Azeez WDYT?
>>>
>>> Thanks
>>> Thusitha
>>>
>>>
>>> On Wed, Dec 7, 2016 at 10:56 AM, Ishara Cooray <isha...@wso2.com> wrote:
>>>
>>>> HI,
>>>>
>>>> We are using MSF4J interceptor for securing REST APIs in API Manager.
>>>> [1] As for now Interceptor registration happens at the class level
>>>> @Component annotation as below.
>>>>
>>>> @Component(
>>>> name = "org.wso2.carbon.apimgt.rest.a
>>>> pi.common.interceptors.OAUTH2SecurityInterceptor",
>>>> service = Interceptor.class,
>>>> immediate = true
>>>> )
>>>> The limitations here are
>>>>
>>>>1. it is not possible to have more than one interceptor that will
>>>>dynamically pick when an api call is received(Because the order matters 
>>>> and
>>>>we are not certain which interceptor will take into effect ).
>>>>2. We cannot explicitly configure to use Custom interceptors
>>>>because of the above[1] reason.
>>>>
>>>> Do we have any plans for these limitations?
>>>>
>>>> Thanks & Regards,
>>>> Ishara Cooray
>>>> Senior Software Engineer
>>>> Mobile : +9477 262 9512 <+94%2077%20262%209512>
>>>> WSO2, Inc. | http://wso2.com/
>>>> Lean . Enterprise . Middleware
>>>>
>>>>
>>>> ___
>>>> Dev mailing list
>>>> d...@wso2.org
>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Thusitha Dayaratne
>>> Software Engineer
>>> WSO2 Inc. - lean . enterprise . middleware |  wso2.com
>>>
>>> Mobile  +94712756809 <071%20275%206809>
>>> Blog  alokayasoya.blogspot.com
>>> Abouthttp://about.me/thusithathilina
>>> <http://wso2.com/signature>
>>>
>>>
>>> ___
>>> Architecture mailing list
>>> Architecture@wso2.org
>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>
>>>
>>
>>
>> --
>> *Afkham Azeez*
>> Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
>> Member; Apache Software Foundation; http://www.apache.org/
>> * <http://www.apache.org/>*
>> *email: **az...@wso2.com* <az...@wso2.com>
>> * cell: +94 77 3320919 <+94%2077%20332%200919>blog: *
>> *http://blog.afkham.org* <http://blog.afkham.org>
>> *twitter: **http://twitter.com/afkham_azeez*
>> <http://twitter.com/afkham_azeez>
>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>> <http://lk.linkedin.com/in/afkhamazeez>*
>>
>> *Lean . Enterprise . Middleware*
>>
>
>


-- 
*Afkham Azeez*
Senior Director, Platform Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Carbon C5 - Server Configuration Model

[Afkham Azeez]

On Tue, Nov 22, 2016 at 10:19 AM, SajithAR Ariyarathna <sajit...@wso2.com>
wrote:

> Hi Danesh,
>
> Do we really need the @Ignore annotation? IMO, we can ignore frields wich
> are not marked with @Element annotation, thus we don't need a separate
> annotation.
>

@Element is optional. By default all fields will become config elements.
That is how it works in popular frameworks such as JAXB. So we are giving a
simple way to specify that some fileds shouldn't become config elements.


>
> Thanks.
>
> On Mon, Nov 21, 2016 at 11:32 PM, Danesh Kuruppu <dan...@wso2.com> wrote:
>
>> Hi all,
>>
>> In Carbon C5, we are going to introduce new global configuration model
>> where we have only one configuration file for all server configurations.
>> So that user have to maintain only one config file for a server.
>>
>> With New Configuration Model,
>>
>>- Global configuration file (deployment.yaml) includes minimal sets
>>of configurations which need to override very often by default (e.g: 
>> server
>>ports etc).
>>- All user settable configurations must have default values and they
>>are burnt into compile codes.
>>- Any default configuration can overridden by adding the particular
>>configuration to the global configuration file(deployment.yaml).
>>- Defining all configurations in the configuration file in not
>>required. add only if we need to override the default value.
>>- If configurations are not specified in the configuration file,
>>values defined in the bean class will apply by default.
>>
>> In global configuration file (deployment.yaml),
>>
>>- Each component will have their own configurations with a unique
>>namespace (e.g: wso2.carbon, wso2.transports.netty etc) and it will be a
>>fragment of deployment.yaml file. Please find the sample file below.
>>- At runtime, the relevant config bean will be generated by
>>overriding the default values specified in bean class with values 
>> mentioned
>>in deployment.yaml file.
>>
>> Sample file looks like,
>>
>>   # Carbon Configuration Parameters
>> wso2.carbon:
>>   id: carbon-kernel
>>   version: 5.2.0-SNAPSHOT
>>   ports:
>> offset: 0
>>   ...
>>
>>   # Netty Transport Configurations
>> wso2.transports.netty:
>>   listeners:
>> - id: msf4j-http
>>   host: 127.0.0.1
>>   port: 8080
>>   bossThreadPoolSize: 2
>>   workerThreadPoolSize: 250
>>   parameters:
>> - name: "executor.workerpool.size"
>>   value: 60
>>
>> ...
>>
>> Along with the above design, we are going to introduce new annotation
>> model to the configuration beans. So each component will have its
>> configuration defined as one or more Java beans annotated with the
>> following annotations
>>
>>- *org.wso2.carbon.kernel.annotations.Configuration* - *Class level
>>annotation, This corresponds to a configuration bean to be used by a
>>component*
>>- *org.wso2.carbon.kernel.annotations.Element* - *Field level
>>annotation, This corresponds to a fields of the class*
>>- *org.wso2.carbon.kernel.annotations.Ignore* - *Field level
>>annotation, This is to specify that field needs to be ignored when the
>>configuration is generated*
>>
>> Sample bean class looks like,
>>
>> @Configuration(namespace = "wso2.carbon", description = "Carbon 
>> Configuration Parameters")
>> public class CarbonConfiguration {
>>
>> @Element(description = "value to uniquely identify a server", required = 
>> true)
>> private String id = "carbon-kernel";
>>
>> @Element(description = "server name")
>> private String name = "WSO2 Carbon Kernel";
>>
>> @Element(description = "server version")
>> private String version = "5.2.0";
>>
>> @Ignore
>> private String tenant = Constants.DEFAULT_TENANT;
>>
>> ...
>>
>> }
>>
>> So we are going to generate the relevant segment of the configuration
>> file(for documentation purposes) automatically at compile time by reading
>> above annotations and default values.
>>
>> If anyone needs to override the default value, He needs to copy the
>> particular segment of the configuration to the deployment.yaml and change
>> the value.
>>
>> Appreciate your input on this.
>>
>> Thanks
&g

Re: [Architecture] test message

[Afkham Azeez]

Ack

On Sun, Oct 16, 2016 at 12:49 AM, <j...@corpsrv.eu> wrote:

> test message
>
> ___
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Configuration files in C5

[Afkham Azeez]

I think Imesh's suggestion merges all the config files and complicates
stuff a lot. With the deployment.properties file we are including only the
bits that most users will be concerned about and will provide a simple way
to configure such stuff.

On Fri, Oct 14, 2016 at 9:50 AM, Isuru Perera <isu...@wso2.com> wrote:

> +1 for using a YAML file instead of a properties file.
>
> On Fri, Oct 14, 2016 at 8:45 AM, Imesh Gunaratne <im...@wso2.com> wrote:
>
>> I would like to propose to use a single YAML file for each distribution
>> (product/profile) to make the configuration process easier.
>>
>> I understand that we are trying to do something similar using a
>> properties file (by overriding configurations in separate files), however
>> IMO a properties file might not suite well for this purpose. A YAML file or
>> any other type of a file which is more readable and designed for managing
>> hierarchical data structures would work well. More importantly having a
>> single configuration file would make the configuration process more simpler
>> and clean. WDYT?
>>
>> Thanks
>>
>>
>> On Thursday, October 13, 2016, Sidath Weerasinghe <sid...@wso2.com>
>> wrote:
>>
>>> Hi Jayanga,
>>>
>>> What are the most frequently changing configurations in C5 which are
>>> going to store in the deployment.properties" file ?
>>>
>>> On Thu, Oct 13, 2016 at 5:07 PM, Jayanga Dissanayake <jaya...@wso2.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> With C5, we introduced "ConfigResolver" which enhances the user
>>>> experience in changing configuration values. With the previous C4x
>>>> approach, users had to know where the configuration files are and to,
>>>> change several configuration files to get the product working in some
>>>> scenarios.
>>>>
>>>> With "ConfigResolver" it allows us to have more frequently changing
>>>> configurations in one location "deployment.properties" file.
>>>>
>>>> A product has set of configurations that are needed to be changed in
>>>> the deployments and there are some other configurations that we don't
>>>> change unless there is a complex situation. Hence, ideally,
>>>> deployment.properties file should contain only the configurations that are
>>>> frequently used and can add more entries if a requirement arise.
>>>>
>>>> But with the requirements coming in with the "profile" support [1]. we
>>>> have to rethink the way config resolver handle the configuration files.
>>>>
>>>> eg:
>>>> 1. We need to enable indexing in API store and publisher, not in other
>>>> profiles.
>>>> 2. Enabling certain handlers in particular profiles.
>>>>
>>>> At present, there is no configuration to enable/disable these features.
>>>> We have to rethink the way we define configurations in features in future.
>>>> We have to have a way to enable/disable certain features so that those
>>>> could be disabled in certain profiles.
>>>>
>>>> Any idea/questions/clarifications are highly appreciated as it will
>>>> help to model the new configurations story in C5.
>>>>
>>>> [1] "Multiple profile support for C5 based products."
>>>>
>>>> Thanks,
>>>> *Jayanga Dissanayake*
>>>> Associate Technical Lead
>>>> WSO2 Inc. - http://wso2.com/
>>>> lean . enterprise . middleware
>>>> email: jaya...@wso2.com
>>>> mobile: +94772207259
>>>> <http://wso2.com/signature>
>>>>
>>>> ___
>>>> Architecture mailing list
>>>> Architecture@wso2.org
>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>
>>>>
>>>
>>>
>>> --
>>> Thank You,
>>> Best Regards,
>>>
>>> Sidath Weerasinghe
>>>
>>>
>>> *Intern*
>>>
>>> *WSO2, Inc. *
>>>
>>> *lean . enterprise . middleware *
>>>
>>>
>>> *Mobile: +94719802550 <%2B94719802550>*
>>>
>>> *Email: *sid...@wso2.com
>>>
>>> Blog: https://medium.com/@sidath
>>>
>>> Linkedin: https://lk.linkedin.com/in/sidathweerasinghe
>>>
>>
>>
>> --
>> *Imesh Gunaratne*
&

Re: [Architecture] Multiple profile support for C5 based products.

[Afkham Azeez]

Please note that during a discussion this week, we decided that instead of
all carbon servers running on the same ports, each of the 5 products will
have their own well known ports. For example, APIM GW port will be 8084
(just an example). So while we would still have portOffset support, we
won't need to specify the offset when running multiple products on the same
machine and they will be able to discover each other on these well known
ports.

On Wed, Oct 12, 2016 at 6:29 PM, Nuwan Dias <nuw...@wso2.com> wrote:

> Do we really need multi-profile support (at an osgi level) on C5? What if
> we have separate dedicated runtimes per profile? Which means that we have a
> gateway runtime, store runtime, etc. within the same distribution? Each
> will run on its own port, maybe using offsets by default (no need to worry
> about that if each profile is a container).
>
> Also if we have profiles, do we also have a global profile which runs all
> components in one? I doubt we'll be able to achieve the target of 1 to 2
> sec startup time if we do that.
>
> Thanks,
> NuwanD.
>
> On Wed, Oct 12, 2016 at 3:40 PM, Muhammed Shariq <sha...@wso2.com> wrote:
>
>> With regards to the discussion on improving some of the limitations in
>> the our current product profiles support [1], we had a discussion to
>> rethink how we can improve the support for running different profiles in C5.
>>
>> Participants - Lakmal, Azeez, Imesh, Kishanthan, Jayanga, Chandana, Rohan
>>
>> *Limitations with the current approach*
>>
>> - We only consider the bundles.info and load the necessary bundles,
>> however there might be certain configs and deployment artifacts that's not
>> required
>> - Some functionalities that aren't required for certain profiles are
>> enabled (e.g. transports)
>> - Distribution size is not optimal for container native architecture
>>
>> *Suggestion for profile support in C5*
>>
>> Considering the above limitation, we decided on building a profile
>> specific distribution from the base distribution. The base distribution
>> will pack all artifacts and also profile specific descriptors
>>
>> If there are any configs specific to a profile, those configs can be
>> changed using the deployment.properties file. The base distribution will
>> have deployment.properties file specific to the profiles, that way we don't
>> have to duplicate any config files.
>>
>> During the runtime, each profile will run as an independent process - for
>> this, we'll be moving away from default 9763/9443 ports, instead each
>> product will have an assigned port range.
>>
>> We will provide a tool that will look into the profile descriptors and
>> build the bare minimum runtime corresponding to the profile. This tool will
>> copy the required bundles, config files etc by looking into the meta info.
>> We can use the same tool to start the runtimes as well.
>>
>> The directory structure of the base distribution will be something like;
>>
>> wso2-am
>> |-- bin
>> |-- osgi
>>  |-- plugins
>> |-- conf
>> |-- profiles
>>  |-- gateway
>>   |-- metadata.yaml
>>   |-- deployment.properties
>>  |-- km
>>   |-- metadata.yaml
>>   |-- deployment.properties
>> |-- *runtime* --> wiill be created by the tool
>>  |-- *wso2-am-gateway*
>>  |-- *wso2-am-km*
>>
>> As shown in the folder structure, the tool will create the profile
>> specific runtime by reading the bundles.info, metadata file etc. In the
>> metadata.yaml file, we can specify what are the config files and other
>> resources needed for that particular profile. So a given runtime will only
>> have the minimum required jars and configs.
>>
>> Additionally, the tool provides the option to create docker-compose yaml
>> that would take the created runtimes and deploy it in containers.
>>
>> We have started implementing the tool, please share your thoughts /
>> suggestions.
>>
>> [1] - [Architecture] How can we improve our profiles story?
>>
>> --
>> Thanks,
>> Shariq
>> Associate Technical Lead
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
> Nuwan Dias
>
> Software Architect - WSO2, Inc. http://wso2.com
> email : nuw...@wso2.com
> Phone : +94 777 775 729
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] How can we improve our profiles story?

[Afkham Azeez]

distribution (like we do now) and
>>>> provide a script / tool to create the bear minimum pack at runtime.
>>>>
>>>> If we go ahead with Option-1, then we will be creating multiple
>>>> distributions for a single product so a product like API-M will have many
>>>> different distributions. This I feel will make the simple case too
>>>> complicated, specially for a user trying to get started.
>>>>
>>>> With Option-2, we can still have the default profile as it is for the
>>>> simplest case, but provide users the ability to create profile specific
>>>> distributions for larger deployments. Users can then use these profile
>>>> specific distributions to create their images.
>>>>
>>>> In both cases, I feel we are moving away from using profiles as we used
>>>> to use them since we are creating a pack with only the required jars and
>>>> artifacts.
>>>>
>>>> Considering these factors, should we look to creating only the
>>>> container friendly bear minimum distribution (Option-1) or provide the
>>>> ability to a create profile specific pack from the default distribution?
>>>>
>>>>
>>>> On Wed, Oct 5, 2016 at 9:03 AM, Lakmal Warusawithana <lak...@wso2.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, Oct 4, 2016 at 5:39 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>
>>>>>> At runtime, there should be profile specific packs shouldn't have
>>>>>> anything extra other than the bear minimum. This is to make it container
>>>>>> friendly as well.
>>>>>>
>>>>>
>>>>> Yes, reducing image size is critical to support container native
>>>>> architecture.
>>>>>
>>>>>
>>>>>>
>>>>>> On Tue, Oct 4, 2016 at 5:27 PM, Muhammed Shariq <sha...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi folks,
>>>>>>>
>>>>>>> I had a chat with Sameera and Jayanga on how we can improve support
>>>>>>> for managing configurations for a particular profile.
>>>>>>>
>>>>>>> We were discussing the possibility of extending the ConfigResolver
>>>>>>> [1] concept to manage profile specific configurations. With 
>>>>>>> ConfigResolver,
>>>>>>> we have the ability to override certain configurations using the
>>>>>>> deployment.properties file, this way we only need modify a single file 
>>>>>>> to
>>>>>>> override configurations scattered in different files.
>>>>>>>
>>>>>>> We are looking into the possibility of using the ConfigResolver
>>>>>>> approach to handle configs related to a specific profile as well. The 
>>>>>>> idea
>>>>>>> is to have a profile specific deployment.properties file where we can
>>>>>>> specify the configs related to that particular profile. This way we can
>>>>>>> avoid duplicating the same config file and parameteriz values using the
>>>>>>> properties file.
>>>>>>>
>>>>>>> If we build a pack specific to a particular profile, this might
>>>>>>> create complication for the simplest scenario where a user wants to 
>>>>>>> simply
>>>>>>> extract the distribution and try it out. With profile specific
>>>>>>> distributions, users will have to deal with multiple distributions
>>>>>>> (configuring offsets etc) even to try out a product which is cumbersome.
>>>>>>>
>>>>>>> So I feel, having a per profile properties is a feasible mechanism
>>>>>>> to define profile specify configs. Shall we go ahead with this approach?
>>>>>>>
>>>>>>> Any concerns, thoughts?
>>>>>>>
>>>>>>> [1] - [Architecture] [C5] Adding deployment.properties file to
>>>>>>> override configurations in components
>>>>>>> [2] - [C5] Less configuration elements with meaningful defaults
>>>>>>>
>>>>>>> On Mon, Oct 3, 2016 at 5:11 PM, Sameera Jayasoma <same...@wso2.com>
>>>>>>> wrote:
>>>>

Re: [Architecture] How can we improve our profiles story?

[Afkham Azeez]

At runtime, there should be profile specific packs shouldn't have anything
extra other than the bear minimum. This is to make it container friendly as
well.

On Tue, Oct 4, 2016 at 5:27 PM, Muhammed Shariq <sha...@wso2.com> wrote:

> Hi folks,
>
> I had a chat with Sameera and Jayanga on how we can improve support for
> managing configurations for a particular profile.
>
> We were discussing the possibility of extending the ConfigResolver [1]
> concept to manage profile specific configurations. With ConfigResolver, we
> have the ability to override certain configurations using the
> deployment.properties file, this way we only need modify a single file to
> override configurations scattered in different files.
>
> We are looking into the possibility of using the ConfigResolver approach
> to handle configs related to a specific profile as well. The idea is to
> have a profile specific deployment.properties file where we can specify the
> configs related to that particular profile. This way we can avoid
> duplicating the same config file and parameteriz values using the
> properties file.
>
> If we build a pack specific to a particular profile, this might create
> complication for the simplest scenario where a user wants to simply extract
> the distribution and try it out. With profile specific distributions, users
> will have to deal with multiple distributions (configuring offsets etc)
> even to try out a product which is cumbersome.
>
> So I feel, having a per profile properties is a feasible mechanism to
> define profile specify configs. Shall we go ahead with this approach?
>
> Any concerns, thoughts?
>
> [1] - [Architecture] [C5] Adding deployment.properties file to override
> configurations in components
> [2] - [C5] Less configuration elements with meaningful defaults
>
> On Mon, Oct 3, 2016 at 5:11 PM, Sameera Jayasoma <same...@wso2.com> wrote:
>
>> Hi All,
>>
>> We can categorize all the files which need to be handled by profiles into
>> three groups.
>>
>>
>> *1) OSGi repository *
>>
>>- *Location:* repository/components
>>- *Description:* Contains all the OSGi bundles, dropins artifacts and
>>other required files. This repository is already organized into multiple
>>profiles(if any).
>>
>>
>> *2) Config repository*
>>
>>- *Location:* repository/conf
>>- *Description: Contains configuration files of the products. We need
>>to figure out a way handle profile specific configuration files.*
>>
>>
>> *3) Artifact repository*
>>
>>- *Location*: repository/deployment
>>- *Description*: Contains deployment artifacts of the product. We
>>need to figure out a way to handle profile specific deployment artifacts.
>>
>>
>> Shariq from the platform team will work on this.
>>
>> Thanks,
>> Sameera.
>>
>> On Fri, Sep 23, 2016 at 3:18 PM, Nuwan Dias <nuw...@wso2.com> wrote:
>>
>>>
>>>
>>> On Fri, Sep 23, 2016 at 2:38 PM, Srinath Perera <srin...@wso2.com>
>>> wrote:
>>>
>>>> I think this happen with ESB NIO transport and Servelt transport for
>>>> webapps. ( Nuwan, is there other examples?).
>>>>
>>>
>>> In the regsitry.xml, we configure the indexers and handlers. These again
>>> are only required in certain profiles. There are also some configs in the
>>> api-manager.xml which only makes sense to enable in certain profiles.
>>>
>>>>
>>>> On Fri, Sep 23, 2016 at 9:42 AM, Kishanthan Thangarajah <
>>>> kishant...@wso2.com> wrote:
>>>>
>>>>> Current issue is that all bundles and artifacts (conf files, webapps)
>>>>> are common to the server which are shared among all the profiles. We don't
>>>>> have a way to delete and modify them when starting up a profile.
>>>>>
>>>>> One other option is we could pack everything (profile specific
>>>>> artifacts) in the base distribution and provide a build script (ant) which
>>>>> create profile specific runtime a.
>>>>>
>>>>> We will check for the other alternatives along with this PoC and see.
>>>>>
>>>>> On Thu, Sep 22, 2016 at 12:27 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>
>>>>>> We proposed an idea to build a pack based on a profile. That will
>>>>>> contain only the essential stuff. So rather than starting up a runtime 
>>>>>> and
>>>>>> then loading a profil

Re: [Architecture] Store framework for C5 based Products

[Afkham Azeez]

On Wed, Sep 28, 2016 at 3:27 PM, Chathura Ekanayake 
wrote:

> If we are not storing trivial artifacts, it is very difficult to use a
> common UI for store and especially for publisher. If a store framework is
> used, we end up extending it and in many situations it can become more
> complex than developing a new UI. Therefore, I agree with Nuwan and
> SajithAR that we should focus on reusable UIs only for parts that can be
> used as it is, or with minimal modifications such as login, user
> management, etc.
>
>
+1
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] How can we improve our profiles story?

[Afkham Azeez]

We proposed an idea to build a pack based on a profile. That will contain
only the essential stuff. So rather than starting up a runtime and then
loading a profile, you build a pack that contains the bare minimum stuff
required. Perhaps we can have a descriptor which describes what non-OSGi
stuff are required for a profile and we can combine that with the OSGi
bundles.info to figure out exactly what is needed. Can someone in the
kernel team do a quick PoC?

On Thu, Sep 22, 2016 at 11:26 AM, Srinath Perera <srin...@wso2.com> wrote:

> Smaeera, are these things we can fix?
>
> --Srinath
>
> On Thu, Sep 22, 2016 at 11:23 AM, Nuwan Dias <nuw...@wso2.com> wrote:
>
>> Hi,
>>
>> This is to raise some concerns over the current server profiles. Although
>> we are able to control the bundles which are loaded to the runtime based on
>> the -Dprofile parameter, we still lack the ability of removing files and
>> modifying configuration files when the server starts on a profile. And this
>> is forcing us to start unnecessary bundles at startup. Let me explain...
>>
>> API Manager has both webapps and a gateway in its distribution. The
>> synapse bundles are only required in the Gateway profiles. However since
>> the axis2.xml file of API Manager defines the http transport senders and
>> receivers based on the Synapse passthrough senders and receivers, the axis2
>> engine will try to load the synapse classes on startup. Ideally if we were
>> able to modify the axis2.xml on the Publisher, Store and Key Manager
>> profiles and replace the passthrough senders and receivers with our
>> standard http senders and receivers, we could avoid loading the synapse
>> bundles on the Publisher, Store and Key Manager.
>>
>> The same problem occurs for registry indexers and handlers. Since the
>> registry indexers and handlers are configured on the registry.xml, even
>> though these are only required in the publisher and store profiles, these
>> bundles will be activated and running even on the Gateway, Key Manager and
>> Traffic Manager. So unless we modify the registry.xml on those nodes
>> manually, we can't stop those bundles from running.
>>
>> Another problem we're facing is the inability to remove webapps. Since
>> all webapps in the repository/deployment/server/webapps and
>> repository/deployment/server/jaggeryapps are deployed into the runtime,
>> unless we remove these webapps manually there is no other way to stop them
>> from being deployed in unrelated profiles.
>>
>> I heard there is a discussion to bind a profile to a container. Which
>> would solve these problems. However it still won't help the "non-container"
>> deployments. Are there ways to overcome the above mentioned limitations and
>> enhance the efficiency of our profiles?
>>
>> Thanks,
>> NuwanD.
>>
>> --
>> Nuwan Dias
>>
>> Software Architect - WSO2, Inc. http://wso2.com
>> email : nuw...@wso2.com
>> Phone : +94 777 775 729
>>
>
>
>
> --
> 
> Srinath Perera, Ph.D.
>http://people.apache.org/~hemapani/
>http://srinathsview.blogspot.com/
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] How do we get DAS server location?

[Afkham Azeez]

>>> one or more them is accepting Thrift connections. This is not desirable due
>>> to obvious reasons.
>>>
>>> How shall we proceed?
>>>
>>> Thanks,
>>>
>>> On 30 June 2016 at 08:52, Srinath Perera <srin...@wso2.com> wrote:
>>>
>>>> Thanks, sound good. Please do ASAP. We are at beta, so too late even
>>>> now.
>>>>
>>>> --Srinath
>>>>
>>>> On Thu, Jun 30, 2016 at 8:09 AM, Gokul Balakrishnan <go...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Srinath,
>>>>>
>>>>> As per our offline chat earlier, the initial plan is to locate
>>>>> the Thrift endpoint  through mDNS service discovery, considering the host
>>>>> and port first.
>>>>>
>>>>> I have used the JmDNS library pointed by Nirmal to do a PoC on this
>>>>> scenario, and I've also already incorporated the logic into the databridge
>>>>> Thrift server to enable service registration through a system property the
>>>>> users could set (-DenableDiscovery). The corresponding client code goes
>>>>> into the publisher OSGi service initialisation. This too is controllable 
>>>>> by
>>>>> the same system property the user could set on the Thrift client (which
>>>>> will be the product talking to DAS/CEP).
>>>>>
>>>>> I'm doing some testing on the entire scenario, and once completed,
>>>>> I'll commit the changes into the relevant repos and send an update to this
>>>>> thread.
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> On Thursday, 30 June 2016, Srinath Perera <srin...@wso2.com> wrote:
>>>>>
>>>>>> Resending as it hits a filter rule.
>>>>>>
>>>>>> Gokul, please give an update on this?
>>>>>>
>>>>>> --Srinath
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Gokul Balakrishnan
>>>>> Senior Software Engineer,
>>>>> WSO2, Inc. http://wso2.com
>>>>> M +94 77 5935 789 | +44 7563 570502
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> 
>>>> Srinath Perera, Ph.D.
>>>>http://people.apache.org/~hemapani/
>>>>http://srinathsview.blogspot.com/
>>>>
>>>
>>>
>>>
>>> --
>>> Gokul Balakrishnan
>>> Senior Software Engineer,
>>> WSO2, Inc. http://wso2.com
>>> M +94 77 5935 789 | +44 7563 570502
>>>
>>>
>>
>>
>> --
>> Gokul Balakrishnan
>> Senior Software Engineer,
>> WSO2, Inc. http://wso2.com
>> M +94 77 5935 789 | +44 7563 570502
>>
>>
>
>
> --
> *Anjana Fernando*
> Senior Technical Lead
> WSO2 Inc. | http://wso2.com
> lean . enterprise . middleware
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] MSF4J Helloworld sample size > 20 MB ?

[Afkham Azeez]

I have tried removing these dependencies but it is not possible because
these are transitive dependencies of some of the msf4j-core's dependencies
and are required at runtime.

On Fri, Jun 17, 2016 at 2:47 PM, Sagara Gunathunga <sag...@wso2.com> wrote:

> Got update and reduced file size to 12 MB.
>
> BTW can't we reduce dependencies further ?  As an example we have bundled
> 3 JSON libraries (Gson, Json-smart, Jackson ) and 2 logging APIs
> ( commons-logging and SLF4J ) with above sample.
>
> Thanks !
>
> On Fri, Jun 17, 2016 at 1:55 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> Yes;
>>
>> ls -al target/helloworld-1.1.0-SNAPSHOT.jar
>>
>> -rw-r--r--  1 azeez  staff  *1143* Jun 16 17:23
>> target/helloworld-1.1.0-SNAPSHOT.jar
>>
>>
>>
>> On Fri, Jun 17, 2016 at 1:52 PM, Thusitha Thilina Dayaratne <
>> thusit...@wso2.com> wrote:
>>
>>> Hi Sagara,
>>>
>>> In the latest pack the helloworld sample is 12.2 MB. AFAIR Azeeze has
>>> removed most of the unwanted OSGi stuffs that get bundled.
>>> Will check on this further.
>>>
>>> Thanks
>>> Thusitha
>>>
>>> On Fri, Jun 17, 2016 at 1:43 PM, Sagara Gunathunga <sag...@wso2.com>
>>> wrote:
>>>
>>>>
>>>> In MSF4J 1.1.0-SNAPSHOT version helloworld sample size ( same go for
>>>> other samples as well)  became 20.1 MB, I don't think this is acceptable
>>>> size for a such simple sample. By looking at shade plugin I found number of
>>>> unwanted OSGi Jar files get bundled with the sample's uber Jar file, we
>>>> have to exclude these unwanted dependencies from parent POM files or need
>>>> to refractor Carbon-transport not to import those OSGi dependencies
>>>> automatically WDYT ?
>>>>
>>>> [INFO] Including
>>>> org.wso2.carbon:org.wso2.carbon.kernel.feature:zip:5.1.0 in the shaded jar.
>>>>  ( I don't think we need Zip files here)
>>>>
>>>> [INFO] Including org.osgi:org.osgi.core:jar:6.0.0 in the shaded jar.
>>>> [INFO] Including org.wso2.carbon:org.wso2.carbon.tools.core:jar:5.1.0
>>>> in the shaded jar.
>>>> [INFO] Including
>>>> org.wso2.carbon:org.wso2.carbon.runtime.feature:zip:5.1.0 in the shaded 
>>>> jar.
>>>> [INFO] Including
>>>> org.wso2.eclipse.equinox:org.eclipse.equinox.launcher.gtk.linux.x86:jar:1.1.200.v20150204-1316
>>>> in the shaded jar.
>>>> [INFO] Including
>>>> org.wso2.eclipse.equinox:org.eclipse.equinox.common:jar:3.6.200.v20130402-1505
>>>> in the shaded jar.
>>>> [INFO] Including
>>>> org.wso2.eclipse.equinox:org.eclipse.equinox.simpleconfigurator:jar:1.1.0.v20131217-1203
>>>> in the shaded jar.
>>>> [INFO] Including
>>>> org.wso2.eclipse.equinox:org.eclipse.equinox.util:jar:1.0.500.v20130404-1337
>>>> in the shaded jar.
>>>> [INFO] Including
>>>> org.wso2.eclipse.equinox:org.eclipse.equinox.ds:jar:1.4.200.v20131126-2331
>>>> in the shaded jar.
>>>> [INFO] Including
>>>> org.wso2.eclipse.equinox:org.eclipse.equinox.launcher:jar:1.3.0.v20140415-2008
>>>> in the shaded jar.
>>>> [INFO] Including
>>>> org.wso2.eclipse.equinox:org.eclipse.equinox.concurrent:jar:1.1.0.v20130327-1442
>>>> in the shaded jar.
>>>> [INFO] Including
>>>> org.wso2.eclipse.equinox:org.eclipse.equinox.frameworkadmin:jar:2.0.100.v20131209-2144
>>>> in the shaded jar.
>>>> [INFO] Including
>>>> org.wso2.eclipse.equinox:org.eclipse.equinox.frameworkadmin.equinox:jar:1.0.500.v20131211-1531
>>>> in the shaded jar.
>>>> [INFO] Including
>>>> org.wso2.eclipse.equinox:org.eclipse.equinox.console:jar:1.1.0.v20140131-1639
>>>> in the shaded jar.
>>>> [INFO] Including
>>>> org.apache.felix:org.apache.felix.gogo.command:jar:0.10.0.v201209301215 in
>>>> the shaded jar.
>>>> [INFO] Including
>>>> org.apache.felix:org.apache.felix.gogo.shell:jar:0.10.0.v201212101605 in
>>>> the shaded jar.
>>>> [INFO] Including
>>>> org.apache.felix:org.apache.felix.gogo.runtime:jar:0.10.0.v201209301036 in
>>>> the shaded jar.
>>>> [INFO] Including org.ops4j.pax.logging:pax-logging-api:jar:1.8.5 in the
>>>> shaded jar.
>>>> [INFO] Including org.ops4j.pax.logging:pax-logging-log4j2:jar:1.8.5 in
>>>> the shaded jar.
>>>> [INFO] In

Re: [Architecture] Initial Implementation of content-aware support in Carbon-Gateway

[Afkham Azeez]

>>>>>
>>>>>Need to specify the portion of the message that needs to be  read
>>>>>via XPath or JSONPath according to  the  content type.
>>>>>-
>>>>>
>>>>> When message hits the content aware mediator it checks weather
>>>>>message is already read if so then it takes MessageDataSource which is 
>>>>> data
>>>>>holder for already read message inputstream according contentType.Else 
>>>>> it
>>>>>gets the matching reader from reader registry and read the input 
>>>>> stream and
>>>>>load the inputstream into MessageDataSource.
>>>>>-
>>>>>
>>>>>XPath and JSONPath are evaluated using MessageDataSource
>>>>>-
>>>>>
>>>>>Serialize data from MessageDataSource to CarbonMessage before
>>>>>sending to the transport level after mediation.
>>>>>
>>>>>
>>>>>
>>>>> XML Reading
>>>>>
>>>>>-
>>>>>
>>>>>Axiom is used for represent XML messages as OMElements
>>>>>-
>>>>>
>>>>>Axiom uses StAX API for read and write XML messages which is
>>>>>inherently supports deferred building concept.
>>>>>-
>>>>>
>>>>>AxiomXpath is used for evaluate XPath and it used Jaxen as
>>>>>underlying XPath library.
>>>>>-
>>>>>
>>>>>XPath libraries are pluggable.
>>>>>
>>>>>
>>>>>
>>>>> JSON Reading
>>>>>
>>>>>-
>>>>>
>>>>>Jayway library is used for represent JSONPath.
>>>>>-
>>>>>
>>>>>Underlying JSON library is Jackson.
>>>>>-
>>>>>
>>>>>Jackson has JSONParser and JSONGenerator which are similar to
>>>>>StreamingXMLReader and Writer in StAX API and can read,  write events 
>>>>> in
>>>>>streaming manner.
>>>>>- Jackson supports data binding as well.
>>>>>
>>>>>
>>>>> Thanks
>>>>> --
>>>>> Best Regards
>>>>> Isuru Ranawaka
>>>>> M: +94714629880
>>>>> Blog : http://isurur.blogspot.com/
>>>>> ___
>>>>> Architecture mailing list
>>>>> Architecture@wso2.org
>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>
>>>>
>>>> ___
>>>> Architecture mailing list
>>>> Architecture@wso2.org
>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>
>>>>
>>>
>>>
>>> --
>>> Best Regards
>>> Isuru Ranawaka
>>> M: +94714629880
>>> Blog : http://isurur.blogspot.com/
>>>
>>> ___
>>> Architecture mailing list
>>> Architecture@wso2.org
>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>
>>>
>>
>>
>> --
>> Nandika Jayawardana
>> WSO2 Inc ; http://wso2.com
>> lean.enterprise.middleware
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
> Best Regards
> Isuru Ranawaka
> M: +94714629880
> Blog : http://isurur.blogspot.com/
>
> ___
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] MSF4J Helloworld sample size > 20 MB ?

[Afkham Azeez]

Yes;

ls -al target/helloworld-1.1.0-SNAPSHOT.jar

-rw-r--r--  1 azeez  staff  *1143* Jun 16 17:23
target/helloworld-1.1.0-SNAPSHOT.jar



On Fri, Jun 17, 2016 at 1:52 PM, Thusitha Thilina Dayaratne <
thusit...@wso2.com> wrote:

> Hi Sagara,
>
> In the latest pack the helloworld sample is 12.2 MB. AFAIR Azeeze has
> removed most of the unwanted OSGi stuffs that get bundled.
> Will check on this further.
>
> Thanks
> Thusitha
>
> On Fri, Jun 17, 2016 at 1:43 PM, Sagara Gunathunga <sag...@wso2.com>
> wrote:
>
>>
>> In MSF4J 1.1.0-SNAPSHOT version helloworld sample size ( same go for
>> other samples as well)  became 20.1 MB, I don't think this is acceptable
>> size for a such simple sample. By looking at shade plugin I found number of
>> unwanted OSGi Jar files get bundled with the sample's uber Jar file, we
>> have to exclude these unwanted dependencies from parent POM files or need
>> to refractor Carbon-transport not to import those OSGi dependencies
>> automatically WDYT ?
>>
>> [INFO] Including org.wso2.carbon:org.wso2.carbon.kernel.feature:zip:5.1.0
>> in the shaded jar.  ( I don't think we need Zip files here)
>>
>> [INFO] Including org.osgi:org.osgi.core:jar:6.0.0 in the shaded jar.
>> [INFO] Including org.wso2.carbon:org.wso2.carbon.tools.core:jar:5.1.0 in
>> the shaded jar.
>> [INFO] Including
>> org.wso2.carbon:org.wso2.carbon.runtime.feature:zip:5.1.0 in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.launcher.gtk.linux.x86:jar:1.1.200.v20150204-1316
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.common:jar:3.6.200.v20130402-1505
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.simpleconfigurator:jar:1.1.0.v20131217-1203
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.util:jar:1.0.500.v20130404-1337
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.ds:jar:1.4.200.v20131126-2331
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.launcher:jar:1.3.0.v20140415-2008
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.concurrent:jar:1.1.0.v20130327-1442
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.frameworkadmin:jar:2.0.100.v20131209-2144
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.frameworkadmin.equinox:jar:1.0.500.v20131211-1531
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.console:jar:1.1.0.v20140131-1639
>> in the shaded jar.
>> [INFO] Including
>> org.apache.felix:org.apache.felix.gogo.command:jar:0.10.0.v201209301215 in
>> the shaded jar.
>> [INFO] Including
>> org.apache.felix:org.apache.felix.gogo.shell:jar:0.10.0.v201212101605 in
>> the shaded jar.
>> [INFO] Including
>> org.apache.felix:org.apache.felix.gogo.runtime:jar:0.10.0.v201209301036 in
>> the shaded jar.
>> [INFO] Including org.ops4j.pax.logging:pax-logging-api:jar:1.8.5 in the
>> shaded jar.
>> [INFO] Including org.ops4j.pax.logging:pax-logging-log4j2:jar:1.8.5 in
>> the shaded jar.
>> [INFO] Including 
>> org.wso2.eclipse.equinox:org.eclipse.equinox.cm:jar:1.1.0.v20131021-1936
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.preferences:jar:3.5.200.v20140224-1527
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.equinox:org.eclipse.equinox.simpleconfigurator.manipulator:jar:2.0.0.v20131217-1203
>> in the shaded jar.
>> [INFO] Including
>> org.wso2.eclipse.osgi:org.eclipse.osgi.compatibility.state:jar:1.0.1.v20140709-1414
>> in the shaded jar.
>>
>>
>>
>>
>> Thanks !
>> --
>> Sagara Gunathunga
>>
>> Architect; WSO2, Inc.;  http://wso2.com
>> V.P Apache Web Services;http://ws.apache.org/
>> Linkedin; http://www.linkedin.com/in/ssagara
>> Blog ;  http://ssagara.blogspot.com
>>
>>
>
>
> --
> Thusitha Dayaratne
> Software Engineer
> WSO2 Inc. - lean . enterprise . middleware |  wso2.com
>
> Mobile  +94712756809
> Blog  alokayasoya.blogspot.com
> Abouthttp://about.me/thusithathilina
>
>


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Securing communication between Hazelcast and WSO2 servers

[Afkham Azeez]

Yes securing the cluster channel is available only in enterprise Hazelcast
& for 4.4 based releases, it requires a kernel patch we have released as
well.
On Jun 14, 2016 12:22 PM, "Srinath Perera"  wrote:

> I think it is only for enterprise version.
>
> Right deployment architecture is to secure it through a firewall.
> Basically, only designated nodes should be able to connect to the hazelcast
> cluster.
>
> --srinath
>
>
>
> On Mon, Jun 13, 2016 at 10:36 PM, Hasitha Hiranya 
> wrote:
>
>> Hi,
>>
>> When a cluster is setup out of WSO2 servers Hazelcast is configured to
>>
>> 1. Make cluster calls across nodes (push notifications)
>> 2. Use as a distributed cache among the nodes
>>
>> Is there any possibility for a third party Hazelcast client to come and
>> consume this data or push un-relevant notifications to the nodes? Also, are
>> we storing sensitive data in HZ cache?
>>
>> Hazelcast communication is happening inside MZ. But yet some companies
>> would like to make it secured.
>>
>> SSL can be configured at HZ client level as per [1], but not sure if it
>> only for enterprise.
>>
>> [1]. http://docs.hazelcast.org/docs/3.5/manual/html/ssl.html
>>
>> Thanks
>>
>> --
>> *Hasitha Abeykoon*
>> Senior Software Engineer; WSO2, Inc.; http://wso2.com
>> *cell:* *+94 719363063*
>> *blog: **abeykoon.blogspot.com* 
>>
>>
>
>
> --
> 
> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera
> Site: http://home.apache.org/~hemapani/
> Photos: http://www.flickr.com/photos/hemapani/
> Phone: 0772360902
>
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Do we need a House Keeping Task for C5 Based Products

[Afkham Azeez]

I think we should rely on the components that create garbage to clear out
their own garbage without having a central task in the kernel.

On Tue, Jun 7, 2016 at 3:29 PM, Thusitha Thilina Dayaratne <
thusit...@wso2.com> wrote:

> Hi All,
>
> I'm trying to implement file upload support for msf4j with FormParam. In
> the none streaming mode, we need to create temp files in some location and
> clean them after a particular time period.
>
> For that purpose at the moment I'm using apache commons-io provided
> FileCleaningTracker[1, 2]. There we can give the set of file objects that
> we need to track. This will be running in a separate thread. When those
> objects are eligible for GC, it will delete the those files. In this way we
> can be sure that the temp objects will not get clear before the actual
> service consumes them.
>
> IMHO it would be much easier if we can provide a similar support from the
> kernel level, since products will require similar functionality in the
> future.
>
> AFAIU rather than tracking the file object references, if we run this as a
> periodic task (like in c4) we have to assume that the temp files are been
> consumed after a pre-configured time. Is it safe to assume so?
>
> WDYT?
>
> [1] -
> https://commons.apache.org/proper/commons-io/javadocs/api-2.2/org/apache/commons/io/FileCleaningTracker.html
> [2] -
> https://github.com/apache/commons-io/blob/trunk/src/main/java/org/apache/commons/io/FileCleaningTracker.java
>
> Thanks
> --
> Thusitha Dayaratne
> Software Engineer
> WSO2 Inc. - lean . enterprise . middleware |  wso2.com
>
> Mobile  +94712756809
> Blog  alokayasoya.blogspot.com
> Abouthttp://about.me/thusithathilina
>
>


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [C5] Java Util Logging vs Pax Logging for C5

[Afkham Azeez]

Your stress test is a purely theoretical one. There will be no instance
where a server writes 100k logs or 1m logs simultaneously. If that happens,
we will have other problems like quickly running out of disk space, for
example. Hence I think for all practical purposes, JUL is good.

On Tue, May 24, 2016 at 3:36 PM, KasunG Gajasinghe <kas...@wso2.com> wrote:

> Hi,
>
> We have done an initial POC on the affect of logging framework for the C5
> server startup between SLF4J (via pax logging) and JUL. Below are the
> initial findings.
>
> *TL;DR* The c5 kernel starts in *0.7 seconds* with JUL, a 350ms decrease
> compared to pax-logging. But the time taken to log under a load test is
> quiet high for JUL.
>
> Product tested - wso2carbon-kernel-5.1.0-SNAPSHOT
> JDK - 1.8.0_60
> OS  - OS X El Capitan
>
> Pax-Logging (SLF4J) Java Util Logging
> startup 1.059s 0.705s
> 100k logs 4.5s 7.9s
> 1m logs 23.482s 57.723s
>
>
> Current C5 kernel uses SLF4J API provided Pax-Logging. Pax-Logging uses
> Log4J 2 as the back-end. With that, the server startup is around 1.059s.
> The server startup decreased by around 350 ms after removing pax-logging
> and replacing the slf4j logs with java util logging.
>
> It seems that log4j2 loads ~750 classes to find classes with @Plugin
> annotation which causes this delay in the startup. This seems to be
> essential for its function.
>
> [image: Inline image 2]
>
> As the number of logs increases, the time taken to log all the log entries
> increased considerably with java util logging.
>
> [image: Inline image 3]
>
> With these numbers, not sure whether we want to go with JUL.
>
>
>
> --
>
> *Kasun Gajasinghe*Senior Software Engineer, WSO2 Inc.
> email: kasung AT spamfree wso2.com
> linked-in: http://lk.linkedin.com/in/gajasinghe
> blog: http://kasunbg.org
>
>
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] MSF4J 1.1.0 and API-M 1.10.0 using Swagger

[Afkham Azeez]

README has been fixed

On Tue, May 24, 2016 at 1:28 AM, Senaka Fernando <sen...@wso2.com> wrote:

>
>
> On Mon, May 23, 2016 at 8:57 PM, Senaka Fernando <sen...@wso2.com> wrote:
>
>> Hi Azeez,
>>
>> Thanks for getting back. Sorry I was in a hurry putting a demo together
>> for Thursday - reading your replies, I see that I haven't explained a few
>> things.
>>
>> On Mon, May 23, 2016 at 7:42 PM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>>
>>>
>>> On Tue, May 24, 2016 at 12:10 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>>
>>>>
>>>> On Mon, May 23, 2016 at 9:42 PM, Senaka Fernando <sen...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I'm working on pulling a demo together based on $subject. So, good
>>>>> news is that it works nearly perfectly. But, then there were a few things
>>>>> that I noted.
>>>>>
>>>>>1. API-M is unable to process the Swagger URL presented by MSF4J
>>>>>(due to inverted commas, AFAIU).
>>>>>
>>>>> What inverted commas? Swagger URL is like this:
>>>>
>>>> http://localhost:8080/swagger?path=/stockquote
>>>>
>>>
>> That works. Sorry - I should have tried that. But, we have to fix the
>> README(s), [1].
>>
>
> Sorry, [1]
> https://github.com/wso2/msf4j/blob/master/samples/stockquote/fatjar/README.md
>
>
>> The URL points to the right place, but the text is wrong.
>>
>> I copied that and didn't think much since the file import worked well.
>> Also, the microservice responds with and without inverted commas (i.e. curl
>> http://localhost:8080/swagger?path=/stockquote > 1.out; curl
>> http://localhost:8080/swagger?path="/stockquote; > 2.out; diff 1.out
>> 2.out).
>>
>>>
>>>>
>>>>>
>>>>>1. It parses the Swagger file well, but there were two issues:
>>>>>   1. It uses the Title and the default one presented by MSF4J has
>>>>>   spaces and won't work - (so had to fix the sample - but that's 
>>>>> fine).
>>>>>
>>>>>
>>>> Can you post an example?
>>>>
>>>
>>> Did you mean this: "title" : "Stockquote Swagger Definition"
>>>
>>
>> Yes, that is what I meant. I fixed this in the
>> src/main/java/org/wso2/msf4j/example/StockQuoteService.java class to
>> "StockQuote". May be it is not an issue of MSF4J, but API-M. But, if you
>> try to create a new API with
>> http://localhost:8080/swagger?path=/stockquote as the Swagger URL, API-M
>> fails.
>>
>>>
>>>>
>>>>>
>>>>>1.
>>>>>   2. It doesn't extract the context from the Swagger file
>>>>>   appropriately, but appends that into the resources on the API 
>>>>> definition.
>>>>>   So, if you stick to the defaults, the URL looks a little ugly, "
>>>>>   http://localhost:8280/*stockquote*/1.0/*stockquote*/IBM;
>>>>>
>>>>> For the point above of course, I didn't find any workaround.
>>
>> Thanks,
>> Senaka.
>>
>>>
>>>>>1.
>>>>>
>>>>> Are these things for which we have fixes/workarounds?
>>>>>
>>>>> Thanks,
>>>>> Senaka.
>>>>> --
>>>>>
>>>>>
>>>>> *[image: http://wso2.com] <http://wso2.com>Senaka Fernando*
>>>>> Solutions Architect; WSO2 Inc.; http://wso2.com
>>>>>
>>>>>
>>>>>
>>>>> *Member; Apache Software Foundation; http://apache.org
>>>>> <http://apache.org>E-mail: senaka AT wso2.com <http://wso2.com>**P:
>>>>> +44 203 318 6025 <%2B44%20203%20318%206025>;*
>>>>>
>>>>>
>>>>> *M: +44 782 741 1966 <%2B44%20782%20741%201966>Linked-In:
>>>>> http://linkedin.com/in/senakafernando
>>>>> <http://linkedin.com/in/senakafernando>*Lean . Enterprise . Middleware
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Afkham Azeez*
>>>> Director of Architecture; WSO2, Inc.; http://wso2.com
>>>> Member; Apache Software Foundation; http://www.apache.org/
>>>> * <ht

Re: [Architecture] MSF4J 1.1.0 and API-M 1.10.0 using Swagger

[Afkham Azeez]

On Tue, May 24, 2016 at 12:10 AM, Afkham Azeez <az...@wso2.com> wrote:

>
>
> On Mon, May 23, 2016 at 9:42 PM, Senaka Fernando <sen...@wso2.com> wrote:
>
>> Hi all,
>>
>> I'm working on pulling a demo together based on $subject. So, good news
>> is that it works nearly perfectly. But, then there were a few things that I
>> noted.
>>
>>1. API-M is unable to process the Swagger URL presented by MSF4J (due
>>to inverted commas, AFAIU).
>>
>> What inverted commas? Swagger URL is like this:
>
> http://localhost:8080/swagger?path=/stockquote
>
>
>>
>>1. It parses the Swagger file well, but there were two issues:
>>   1. It uses the Title and the default one presented by MSF4J has
>>   spaces and won't work - (so had to fix the sample - but that's fine).
>>
>>
> Can you post an example?
>

Did you mean this: "title" : "Stockquote Swagger Definition"

>
>
>>
>>1.
>>   2. It doesn't extract the context from the Swagger file
>>   appropriately, but appends that into the resources on the API 
>> definition.
>>   So, if you stick to the defaults, the URL looks a little ugly, "
>>   http://localhost:8280/*stockquote*/1.0/*stockquote*/IBM;
>>
>> Are these things for which we have fixes/workarounds?
>>
>> Thanks,
>> Senaka.
>> --
>>
>>
>> *[image: http://wso2.com] <http://wso2.com>Senaka Fernando*
>> Solutions Architect; WSO2 Inc.; http://wso2.com
>>
>>
>>
>> *Member; Apache Software Foundation; http://apache.org
>> <http://apache.org>E-mail: senaka AT wso2.com <http://wso2.com>**P: +44
>> 203 318 6025 <%2B44%20203%20318%206025>;*
>>
>>
>> *M: +44 782 741 1966 <%2B44%20782%20741%201966>Linked-In:
>> http://linkedin.com/in/senakafernando
>> <http://linkedin.com/in/senakafernando>*Lean . Enterprise . Middleware
>>
>
>
>
> --
> *Afkham Azeez*
> Director of Architecture; WSO2, Inc.; http://wso2.com
> Member; Apache Software Foundation; http://www.apache.org/
> * <http://www.apache.org/>*
> *email: **az...@wso2.com* <az...@wso2.com>
> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
> *http://blog.afkham.org* <http://blog.afkham.org>
> *twitter: **http://twitter.com/afkham_azeez*
> <http://twitter.com/afkham_azeez>
> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
> <http://lk.linkedin.com/in/afkhamazeez>*
>
> *Lean . Enterprise . Middleware*
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] MSF4J 1.1.0 and API-M 1.10.0 using Swagger

[Afkham Azeez]

On Mon, May 23, 2016 at 9:42 PM, Senaka Fernando <sen...@wso2.com> wrote:

> Hi all,
>
> I'm working on pulling a demo together based on $subject. So, good news is
> that it works nearly perfectly. But, then there were a few things that I
> noted.
>
>1. API-M is unable to process the Swagger URL presented by MSF4J (due
>to inverted commas, AFAIU).
>
> What inverted commas? Swagger URL is like this:

http://localhost:8080/swagger?path=/stockquote


>
>1. It parses the Swagger file well, but there were two issues:
>   1. It uses the Title and the default one presented by MSF4J has
>   spaces and won't work - (so had to fix the sample - but that's fine).
>
>
Can you post an example?


>
>1.
>   2. It doesn't extract the context from the Swagger file
>   appropriately, but appends that into the resources on the API 
> definition.
>   So, if you stick to the defaults, the URL looks a little ugly, "
>   http://localhost:8280/*stockquote*/1.0/*stockquote*/IBM;
>
> Are these things for which we have fixes/workarounds?
>
> Thanks,
> Senaka.
> --
>
>
> *[image: http://wso2.com] <http://wso2.com>Senaka Fernando*
> Solutions Architect; WSO2 Inc.; http://wso2.com
>
>
>
> *Member; Apache Software Foundation; http://apache.org
> <http://apache.org>E-mail: senaka AT wso2.com <http://wso2.com>**P: +44
> 203 318 6025 <%2B44%20203%20318%206025>;*
>
>
> *M: +44 782 741 1966 <%2B44%20782%20741%201966>Linked-In:
> http://linkedin.com/in/senakafernando
> <http://linkedin.com/in/senakafernando>*Lean . Enterprise . Middleware
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [Dev] OOM Issues and HazelCast behaviour

[Afkham Azeez]

System.exit(121) triggers a restart in Carbon. This was there in Carbon 4,
but we can add it to Carbon 5 as well.

On Tue, May 10, 2016 at 2:18 PM, Srinath Perera <srin...@wso2.com> wrote:

> +1, when server hits an issue it cannot recover and continue to work
> meaningfully, it is best to shut down or restart. That approximate crash
> failure behaviour ( which is a good thing).
>
> On Tue, May 10, 2016 at 11:40 AM, Ramith Jayasinghe <ram...@wso2.com>
> wrote:
>
>> HI Azeez/Sameera,
>>
>>  Having encountered some OOM issues (relating to HZ) I looked at what
>> Hazelcast does in face of GC errors [1]. It will drop all connections,
>> threads, and shuts it self down.
>>
>> Now, Think about one of our server's ( say MB) face a OOM and embedded HZ
>> instance shuts it self down. I propose when this happens entire server
>> should stop its processing intelligently ( if not shutting down).
>> Rational is one of the sub-system in the server shutdown while other
>> parts of the server don't care, which is wrong. As for MB, it tries to stop
>> processing in a meaningful manner and I argue this should be something any
>> server should do if they use HZ clustering.
>>
>> thoughts?
>>
>>
>> [1]
>> https://github.com/hazelcast/hazelcast/blob/master/hazelcast/src/main/java/com/hazelcast/instance/DefaultOutOfMemoryHandler.java
>>
>> --
>> Ramith Jayasinghe
>> Technical Lead
>> WSO2 Inc., http://wso2.com
>> lean.enterprise.middleware
>>
>> E: ram...@wso2.com
>> P: +94 772534930
>>
>> ___
>> Dev mailing list
>> d...@wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
>
> --
> 
> Srinath Perera, Ph.D.
>http://people.apache.org/~hemapani/
>http://srinathsview.blogspot.com/
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Ports to use for Admin Services in C5

[Afkham Azeez]

There is a way to do this. At the point of deploying the service, you have
to specify on which transports that service is exposed. This is similar to
the concept of exposing services on selected transports in Axis2.

On Fri, May 6, 2016 at 2:26 PM, Sagara Gunathunga <sag...@wso2.com> wrote:

>
>
> On Thu, May 5, 2016 at 2:32 PM, Kishanthan Thangarajah <
> kishant...@wso2.com> wrote:
>
>> Another thing is, should we also work on exposing admin services on one
>> listener (probably over https) and other user api's on different listener?
>> May be we need to bring in some changes to MSF4J core to support this via
>> OSGi level service properties and listener id's.
>>
>
> Usually it uses separate port for admin services so that that port can be
> protected with high level of security, +1 explore this option.
>
> Thanks !
>
>>
>>
>> On Thu, May 5, 2016 at 7:39 AM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> Will you run admin stuff & user stuff on the same instances? At least
>>> shouldn't our recommendation be that admin & user stuff have to be
>>> separate, as a best practice?
>>>
>>> On Wed, May 4, 2016 at 9:12 PM, Hasitha Aravinda <hasi...@wso2.com>
>>> wrote:
>>>
>>>> Hi Manu,
>>>>
>>>> In my point of view, we have to decide it based on what API does and
>>>> who are the actual users involve.
>>>>
>>>> In BPS, we have two sets of users: workflow participants and admin
>>>> user/devOps of the BPS. Based on these users we can categorized BPS APIs
>>>> into two sets.
>>>>
>>>>- Admin APIs : There are few APIs like artifact deployer API,
>>>>accessed only by administrators of the server or devOps.
>>>>
>>>>
>>>>- User APIs : BPMN Rest API and HumanTask API are user APIs,
>>>>because these APIs only accessed by participants of processes and user
>>>>tasks. But we can argue some of the operations are admin operations, but
>>>>those are business admin operations. These resources/operations need to
>>>>be authorized using an ACL, based on current user and his role in 
>>>> workflow
>>>>or user-task.
>>>>
>>>> For example in HumanTask [1], we have several roles i.e. Business
>>>> Administrator, Potential Owners, Excluded Owners, Stakeholders etc. Based
>>>> on current user and his role in defined task, user are authorized to
>>>> perform an operation.
>>>>
>>>> ​IMO having clear separations between User API and Admin API may
>>>> important when securing these APIs separately.
>>>>
>>>> [1] -
>>>> http://docs.oasis-open.org/bpel4people/ws-humantask-1.1-spec-cs-01.html#_Toc261430341
>>>>
>>>> Thanks,
>>>> Hasitha.
>>>>
>>>> On Wed, May 4, 2016 at 7:55 PM, Manuranga Perera <m...@wso2.com> wrote:
>>>>
>>>>> How do we define an admin vs non-admin API?
>>>>> Is getting list of users different from getting the list of processes?
>>>>>
>>>>> A customer written UI may have to call both. We can argue that some
>>>>> things are 100% admin eg: shutdown server. But to me this seems like an
>>>>> arbitrary decision.
>>>>>
>>>>>
>>>>> On Wed, May 4, 2016 at 12:14 AM, Hasitha Aravinda <hasi...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Another thing, we need to consider exposing different ports for user
>>>>>> APIs and Admin APIs to have a clear separation. In C4 all user and admin
>>>>>> APIs exposed in 9443 and 9763. AFAIK this is not supported in current 
>>>>>> MSF4J
>>>>>> OSGi version.
>>>>>>
>>>>>> Thanks,
>>>>>> Hasitha.
>>>>>>
>>>>>> On Wed, May 4, 2016 at 9:26 AM, Nandika Jayawardana <nand...@wso2.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> In all the carbon platform versions up to now, we used 9443, and
>>>>>>> 9763 ports for admin services for all server products. Are we going to 
>>>>>>> use
>>>>>>> the same ports for C5.
>>>>>>>
>>>>>>> Regards
>>>>>>> Nandika
>>>>>>

Re: [Architecture] Handling product versions in microservices

[Afkham Azeez]

In what cases would you require to maintain multiple versions of a
microservice in a product? Or is this related to different versions of the
same product? If it is the second case, it is the responsibility of the
gateway to route to the correct instances and there is no need to maintain
version information on the product runtime side.

On Thu, May 5, 2016 at 2:12 PM, Sameera Jayasoma <same...@wso2.com> wrote:

>
>
> On Thu, May 5, 2016 at 12:42 PM, Hasitha Aravinda <hasi...@wso2.com>
> wrote:
>
>> ​Some more questions
>>
>> On Thu, May 5, 2016 at 12:30 PM, Sameera Jayasoma <same...@wso2.com>
>> wrote:
>>
>>> I believe its wrong to use the product version to version your micro
>>> services.
>>>
>>> You need to have different version for your microservices. e.g. starting
>>> from 1.0.0. If you don't change any of your microservices you shouldn't be
>>> changing these versions.
>>>
>>
>> ​According to Rest API guideline, it should be product version. ​isn't it
>> ?
>>
>
> We cannot change API versions for each and every product release.  You
> should change the API version only if you have introduced a change.  We are
> following the same approach to version export packages from bundles.
>
>
>>
>> Let's say we are using version from 1.0.0.
>> ​If we are going to do a modification​
>>
>> ​to a microservice ( ​let's say foo 1.0) and new version is foo 1.1 Do we
>> need to maintain both versions to provide backward compatibility ? If so we
>> have to maintain two microservices: Foo 1.0 and Foo 1.1, because version is
>> a part of the path in microservice.
>>
>
> I am not sure whether need to maintain the previous versions of micro
> services. Thats something we need to discuss. Also it adds complexity.
>
> If you have introduced any additions to your micro services, then you can
> bump up the minor version number. If you've introduced incompatible
> changes, then you have bump up the major version number.
>
>
>
>>
>> eg:
>> @Path("
>> /bps/bpmn/v
>> ​1​
>> .0/
>> ​foo
>> ")
>> ​ ,
>> @Path("
>> /bps/bpmn/v
>> ​1​
>> .
>> ​1​
>> /
>> ​foo
>> ")
>> ​
>> Thanks,
>> Hasitha.
>>
>>
>>> On Thu, May 5, 2016 at 11:02 AM, Himasha Guruge <himas...@wso2.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> We have refactored our REST APIs by implementing Microservice
>>>> interface. According to the new REST guideline , we need to maintain the
>>>> URL format like below. For example,
>>>>
>>>> http://localhost:/bps/bpmn/v4.0/repository/deployments
>>>>
>>>> How are we going to maintain the product version updates with
>>>> microservices? If we are to maintain the product version with a  parameter
>>>> been set from a config file, how can we achieve this?
>>>>
>>>>
>>>> Thanks,
>>>> Himasha Guruge
>>>> *Software Engineer*
>>>> WS*O2* *Inc.*
>>>> Mobile: +94 777459299
>>>> himas...@wso2.com
>>>>
>>>> ___
>>>> Architecture mailing list
>>>> Architecture@wso2.org
>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>
>>>>
>>>
>>>
>>> --
>>> Sameera Jayasoma,
>>> Software Architect,
>>>
>>> WSO2, Inc. (http://wso2.com)
>>> email: same...@wso2.com
>>> blog: http://blog.sameera.org
>>> twitter: https://twitter.com/sameerajayasoma
>>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections
>>> Mobile: 0094776364456
>>>
>>> Lean . Enterprise . Middleware
>>>
>>>
>>> ___
>>> Architecture mailing list
>>> Architecture@wso2.org
>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>
>>>
>>
>>
>> --
>> --
>> Hasitha Aravinda,
>> Senior Software Engineer,
>> WSO2 Inc.
>> Email: hasi...@wso2.com
>> Mobile : +94 718 210 200
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
> Sameera Jayasoma,
> Software Architect,
>
> WSO2, Inc. (http://wso2.com)
> email: same...@wso2.com
> blog: http://blog.sameera.org
> twitter: https://twitter.com/sameerajayasoma
> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections
> Mobile: 0094776364456
>
> Lean . Enterprise . Middleware
>
>


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Ports to use for Admin Services in C5

[Afkham Azeez]

Will you run admin stuff & user stuff on the same instances? At least
shouldn't our recommendation be that admin & user stuff have to be
separate, as a best practice?

On Wed, May 4, 2016 at 9:12 PM, Hasitha Aravinda <hasi...@wso2.com> wrote:

> Hi Manu,
>
> In my point of view, we have to decide it based on what API does and who
> are the actual users involve.
>
> In BPS, we have two sets of users: workflow participants and admin
> user/devOps of the BPS. Based on these users we can categorized BPS APIs
> into two sets.
>
>- Admin APIs : There are few APIs like artifact deployer API, accessed
>only by administrators of the server or devOps.
>
>
>- User APIs : BPMN Rest API and HumanTask API are user APIs, because
>these APIs only accessed by participants of processes and user tasks. But
>we can argue some of the operations are admin operations, but those are
>business admin operations. These resources/operations need to
>be authorized using an ACL, based on current user and his role in workflow
>or user-task.
>
> For example in HumanTask [1], we have several roles i.e. Business
> Administrator, Potential Owners, Excluded Owners, Stakeholders etc. Based
> on current user and his role in defined task, user are authorized to
> perform an operation.
>
> ​IMO having clear separations between User API and Admin API may important
> when securing these APIs separately.
>
> [1] -
> http://docs.oasis-open.org/bpel4people/ws-humantask-1.1-spec-cs-01.html#_Toc261430341
>
> Thanks,
> Hasitha.
>
> On Wed, May 4, 2016 at 7:55 PM, Manuranga Perera <m...@wso2.com> wrote:
>
>> How do we define an admin vs non-admin API?
>> Is getting list of users different from getting the list of processes?
>>
>> A customer written UI may have to call both. We can argue that some
>> things are 100% admin eg: shutdown server. But to me this seems like an
>> arbitrary decision.
>>
>>
>> On Wed, May 4, 2016 at 12:14 AM, Hasitha Aravinda <hasi...@wso2.com>
>> wrote:
>>
>>> Another thing, we need to consider exposing different ports for user
>>> APIs and Admin APIs to have a clear separation. In C4 all user and admin
>>> APIs exposed in 9443 and 9763. AFAIK this is not supported in current MSF4J
>>> OSGi version.
>>>
>>> Thanks,
>>> Hasitha.
>>>
>>> On Wed, May 4, 2016 at 9:26 AM, Nandika Jayawardana <nand...@wso2.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> In all the carbon platform versions up to now, we used 9443, and 9763
>>>> ports for admin services for all server products. Are we going to use the
>>>> same ports for C5.
>>>>
>>>> Regards
>>>> Nandika
>>>>
>>>> --
>>>> Nandika Jayawardana
>>>> WSO2 Inc ; http://wso2.com
>>>> lean.enterprise.middleware
>>>>
>>>> ___
>>>> Architecture mailing list
>>>> Architecture@wso2.org
>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>
>>>>
>>>
>>>
>>> --
>>> --
>>> Hasitha Aravinda,
>>> Senior Software Engineer,
>>> WSO2 Inc.
>>> Email: hasi...@wso2.com
>>> Mobile : +94 718 210 200
>>>
>>> ___
>>> Architecture mailing list
>>> Architecture@wso2.org
>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>
>>>
>>
>>
>> --
>> With regards,
>> *Manu*ranga Perera.
>>
>> phone : 071 7 70 20 50
>> mail : m...@wso2.com
>>
>
>
>
> --
> --
> Hasitha Aravinda,
> Senior Software Engineer,
> WSO2 Inc.
> Email: hasi...@wso2.com
> Mobile : +94 718 210 200
>
> ___
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [C5] Carbon Permission Model - Meeting Notes

[Afkham Azeez]

Model 2 is basically saying every operation has to be represented as a
resource and then use resource permissions. IMO, that is not a very good
model because representing every operation as a resource is not natural. In
addition, if you take Java Security, they have a model for securing code
blocks. That is at the lowest level, IMO. At the next level, I think we
need a model to specify who can do what at the business logic level (Users
with role user-admin-foo, can add users to the user group foo). The next
level would be to specify who could invoke operations. e.g. users with
add-user permission can call the addUsersToGroup operation. So, if a
someone needs to add a user to group foo, he should have the permissions to
call the addUsersToGroup operation, as well as permissions to add the user
to the foo group (if applicable). So I prefer model 1.

Azeez

On Tue, Apr 19, 2016 at 11:24 AM, Ramith Jayasinghe <ram...@wso2.com> wrote:

> Solution 2 seems too rigid. Why? my argument is there could be scenarios
> more than one functionality ( / method) in a product could require same
> functionality.
> if we use class names, things become complicated and cluttered isn't it?
> thoughts?
>
> On Tue, Apr 19, 2016 at 10:02 AM, Sameera Jayasoma <same...@wso2.com>
> wrote:
>
>> The purpose of this meeting was to brainstorm the new permission model
>> for Carbon 5. This mail contain the summary of the meeting. We haven’t
>> finalized anything yet.
>>
>> During the meeting we discussed on service permissions and resource
>> permissions. Service permissions are applied to micro services. These
>> services could be admin services or product specific services. Resource
>> permissions applies to all the resources in our platform. E.g Registry
>> resource, Topic, Queues etc.
>>
>> Let me try to explain the difference using an example. Consider the
>> operation “addUserToGrop” in “UserMgtService”. Here we can define
>> permission at different levels. A user Bob can invoke this method, but Bob
>> cannot add users to group Foo. Granting privileges to invoke this method
>> can be done via service permissions. Restricting Bob from adding user to
>> group Foo can be done via resource permissions.
>>
>> A permission has a name and optional list of actions. There are
>> permissions with just the name and they are usually called named
>> permission. Either we give users named permissions or we don’t. Above
>> mentioned service permissions are named permissions. E.g. Either we allow
>> user to invoke the addUserToGroup method or we don’t. There are no actions
>> associated with this permission. But for resource permissions , there are
>> associated actions. E.g. Registry resources can be added, deleted, updated
>> etc.
>>
>>
>> *Declaring Permissions.*We discussed two ways to declare required
>> permissions in services.
>>
>> *Solution 01*
>>
>> @RequirePermission(“manage/users”)
>> public class UserMgtService {
>>   @RequirePermission(“add-user”)
>>   public  void addUserToGropu(User user, String groupName) {
>>   }
>> }
>>
>> This is exactly same as Carbon 4 model. Permissions required invoke a
>> service is declared in the services.xml. With microservices based services
>> we can declare required permission with annotations.
>>
>> *Solution 02*
>>
>> @Secure
>> public class UserMgtService {
>>public  void addUserToGropu(User user, String groupName) {
>>}
>> }
>>
>> Here we calculate the permission string using the classname and the
>> method name. E.g.
>> org.wso2.carbon.identity.mgt.UserMgtService.addUser. These service
>> permission can be checked with an microservice interceptor.
>>
>> There are pros and cons of both of these methods. We need to discuss and
>> come to a conclusion soon.
>>
>> Also we need to figure out how to declare resource permissions.
>>
>> Thanks,
>> Sameera.
>>
>> --
>> Sameera Jayasoma,
>> Software Architect,
>>
>> WSO2, Inc. (http://wso2.com)
>> email: same...@wso2.com
>> blog: http://blog.sameera.org
>> twitter: https://twitter.com/sameerajayasoma
>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections
>> Mobile: 0094776364456
>>
>> Lean . Enterprise . Middleware
>>
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
> Ramith Jayasinghe
> Technical Lead
> WSO2 Inc., http://wso

Re: [Architecture] Supporting Carbon Metrics on Carbon 5 based products

[Afkham Azeez]

Name is ok. Yes, I think for advanced metrics config stuff, we will need to
provide a file.

On Thu, Apr 7, 2016 at 2:02 PM, Isuru Perera <isu...@wso2.com> wrote:

> Hi Azeez,
>
> How about carbon-metrics-extensions for new repo name? This new repo will
> have the MSF4J REST API to get data from JDBC.
>
> I have another question regarding JDBC Reporter. Since all reporters are
> now a part of metrics core, the JDBC reporter will also be available for
> MSF4J. Earlier, MSF4J supported only Console, JMX and DAS reporters. In a
> future MSF4J version, when metrics wants to use the JDBC reporter we need
> to figure out a way to initialize the JDBC Data Source.
>
> In OSGi environment, I'm planning to use carbon-datasources [1]. However
> we cannot use that in MSF4J. We need to be able to directly configure a
> datasource for metrics. So, I thought of giving an option to initialize a
> HikariCP [2] datasource directly from metrics. I can use a properties file
> to configure HikariCP [3]. (I chose HikariCP as it's already used in
> carbon-datasources).
>
> This means, when MSF4J wants to use JDBC Reporter, we need following
> config files.
>
>- metrics.yml (To enable Metrics and JDBC Reporter)
>- metrics-datasource.properties (To configure HikariCP datasource)
>
> WDYT?
>
> Thanks!
>
> Best Regards,
> [1] https://github.com/wso2/carbon-datasources
> [2] http://brettwooldridge.github.io/HikariCP/
> [3] https://github.com/brettwooldridge/HikariCP#initialization
>
>
> On Thu, Mar 31, 2016 at 11:54 AM, Isuru Perera <isu...@wso2.com> wrote:
>
>> Hi Azeez,
>>
>> I meant the same thing as you. I should have explained it more clearly. :)
>>
>> So, I'll break metrics into 2 repositories. One repository will be having
>> the core Metrics features, which *will be used* by MSF4J.
>>
>> Other metrics related repository will have the REST API, written with
>> MSF4J. So, it depends on MSF4J. (I'm thinking of a name for the new repo)
>>
>> Thanks!
>>
>> Best Regards,
>>
>>
>> On Thu, Mar 31, 2016 at 11:17 AM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>>
>>>
>>> On Thu, Mar 31, 2016 at 11:03 AM, Isuru Perera <isu...@wso2.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I have another question.
>>>>
>>>> The MSF4J depends on Metrics and now Metrics needs MSF4J to implement
>>>> REST services. So, there is a cyclic dependency. I think it's wrong that
>>>> carbon-metrics repo [1] depending on msf4j repo [2].
>>>>
>>>> One solution to this is to have separate repos for metrics.
>>>>
>>>> One repo will include only the metrics core features and it'll not
>>>> depend on the MSF4J. Then MSF4J can depend on metrics core. We can create
>>>> another repository to keep the REST services and gadgets to view the
>>>> metrics.
>>>>
>>>
>>> We are targeting MSF4J at external developers. Unlike our other
>>> components & products, if the relevant code is in several places, it could
>>> become cumbersome for external developers. I think what should be done is
>>> have metrics-core, on which MSF4J depends, and then have metrics REST
>>> services somewhere else. That can take a dependency on both metrics-core
>>> and MSF4J.
>>>
>>>
>>>>
>>>> WDYT?
>>>>
>>>> Thanks!
>>>>
>>>> Best Regards,
>>>>
>>>> [1] https://github.com/wso2/carbon-metrics
>>>> [2] https://github.com/wso2/msf4j
>>>>
>>>> On Thu, Mar 17, 2016 at 10:53 PM, Kishanthan Thangarajah <
>>>> kishant...@wso2.com> wrote:
>>>>
>>>>> Like with generic HTTP Monitoring Dashboard (gadgets), we could have
>>>>> generic gadgets for Metrics here so that products can reuse them with the
>>>>> analytics dashboard.
>>>>>
>>>>> I think you can do a carbon 5 based release of the metrics
>>>>> dependencies (a carbon feature release) and also expose those services 
>>>>> used
>>>>> with metrics data fetching as microservices as you mentioned.
>>>>>
>>>>> On Mon, Mar 14, 2016 at 5:01 PM, Isuru Perera <isu...@wso2.com> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Thanks for the feedback. So, we agree that the Carbon Metrics should
>>>>>> continue supporting JDBC reporter and visualize the M

Re: [Architecture] [AppCloud] Publishing AppCloud specific data from MSF4J and AS HTTP Monitoring Data Publishers

[Afkham Azeez]

Yes, looks like "arbitrary" is a concept in DAS but it doesn't make sense
to name env vars like that.

On Mon, Apr 4, 2016 at 2:59 PM, Kalpa Welivitigoda <kal...@wso2.com> wrote:

> Hi Pirinthapan,
>
> As per the offline chat we had, this is being implemented and tracked with
> [1]. About the prefix, how about we using "WSO2_" as the prefix.
> WSO2_TENANT_ID seems to make more sense. WDYT?
>
> [1] https://wso2.org/jira/browse/WSAS-2225
>
> On Mon, Apr 4, 2016 at 1:17 PM, Pirinthapan Mahendran <
> pirintha...@wso2.com> wrote:
>
>> Hi all,
>>
>> In AppCloud we will be having the HTTP Monitoring dashboards to showing
>> the statistics of the application usages. To accomplish this, we will be
>> using the data publishers shiped with msf4j and AS-6.0.0. But still we need
>> to publish some AppCloud specific data (i.e : Application Name, Application
>> Version, etc.)  and tenant id from these publishers to DAS.
>>
>> To publish these custom data we have come with the following solution,
>> which requires some modification in the msf4j and AS publishers.
>>
>> *When creating the containers during the application deployment, AppCloud
>> will set these custom data as Environment Variables. These environment
>> variables are prefixed with the keyword 'arbitrary'. *
>>
>> *e.g : If we are setting application name as an environment variable, the
>> key of this environment variable will looks like
>> 'arbitrary_applicationName'.*
>>
>> *Then during the publishing time, the msf4j or AS will read all the
>> environment variables with the prefix 'arbitrary' and publish them as
>> arbitrary attributes to DAS.*
>>
>> *The stream definition in the DAS will be configured to capture these
>> arbitrary attributes and use them for its internal processing.*
>>
>> In order to implement this solution we are planning to add these changes
>> to the next releases of msf4j and AS with the coordination of msf4j and As
>> teams.
>>
>> If you have any comments on the above solution, we are kindly
>> appreciating them.
>>
>> Thanks & Regards,
>> Mahendran Pirinthapan
>> Software Engineer | WSO2 Inc.
>> Mobile +94772378732.
>>
>
>
>
> --
> Best Regards,
>
> Kalpa Welivitigoda
> Software Engineer, WSO2 Inc. http://wso2.com
> Email: kal...@wso2.com
> Mobile: +94776509215
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

On Sun, Apr 3, 2016 at 6:48 AM, Sanjiva Weerawarana <sanj...@wso2.com>
wrote:

> :-). I think there are two things we should show:
> - one can do circuit breaker (CB) with MSF4J
> - when should you use it
>
> Wrapping of DB calls in Hystrix may indeed be appropriate in some cases,
> but not commonly. Second, at that level its really not about MSF4J at all.
>

Yes, that is correct, MSF4J after all is just a Java library, but people
ask how to do do xyz with MSF4J. So in my recent blogposts, I have been
trying to make it easy for people to find the answers through Googling for
some of the questions I have received from people whom I have spoken to.

http://techblog.netflix.com/2011/12/making-netflix-api-more-resilient.html
is a good read, and for one of the use cases, they wrap a DB select query
in a circuit breaker since if that fails, it is possible to fallback to the
cache.


>
> If we show a sample that's a service invoking another service or some
> random Java code invoking a service using CB then that's better IMO. We
> should mention when it is indeed appropriate to use as a fancy try catch.
>
> For example, in the registry code we have scenarios where there can be
> transient DB issues because of concurrent access. Should we use CB there to
> make that code more resilient? Almost always the transaction "failure" in
> those cases is a temporary issue and a non-issue ... just bad timing.
>

For transient issues, the common way to try to recover is retrying, and
after a few retries (we can use binary backoff with timeout), it could be
possible to have some sort of fallback, perhaps send a notification about
the failure & serve from a cache until it times out.


>
> If we can make our code better with CB we absolutely MUST do it. The key
> is to provide guidance on when its appropriate to do it!
>

Yes, agreed.

>
> Sanjiva.
>
> On Sat, Apr 2, 2016 at 8:24 AM, Afkham Azeez <az...@wso2.com> wrote:
>
>> Well one of your responses indicated that the the blogpost suggests
>> wrapping all DB calls in circuit breakers, and Frank's response indicated
>> that it is suggesting wrapping all calls to external systems in circuit
>> breakers.  If it can confuse such brilliant minds, I guess it could mislead
>> an average developer. That blogpost was on how to implement the pattern in
>> the context of MSF4J using Hystrix, and I think it should have been
>> preceded with a blogpost introducing the pattern & explaining where it is
>> applicable. Sorry for the confusion again. I will work on a separate post.
>>
>> Thanks
>> Azeez
>>
>> On Fri, Apr 1, 2016 at 11:59 PM, Sanjiva Weerawarana <sanj...@wso2.com>
>> wrote:
>>
>>> Azeez, asking questions and asking to understand what the purpose of
>>> some code is not a criticism.
>>>
>>> Of course there should be a sample! I only asked about it because to me
>>> a client side sample made more sense - and then it went into a discussion
>>> of when this should be used etc..
>>>
>>> I would not only put the blog back but also use the opportunity to
>>> explain to "lazy developers" when and where one should use a
>>> circuitbreaker. That's 1000x more valuable than just the code on how to do
>>> it.
>>>
>>> Sanjiva.
>>>
>>> On Thu, Mar 31, 2016 at 8:00 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> The blog post has been removed. Sorry for all the confusion. This was
>>>> only done as part of the agreement we had during last week's meeting to
>>>> demonstrate certain features such as Spring support, JPA support, support
>>>> for patterns etc. in order to help developers understand how to implement
>>>> certain stuff with MSF4J. Our target community of MSF4J is primarily
>>>> developers, and developers like to refer to sample code segments in order
>>>> to proceed with implementing their solutions. We lazy developers love to
>>>> Google search for code segments, e.g. JDBC connection example, and then
>>>> copy, paste and modify those segments. What I have been trying to do with
>>>> the series of blogposts is to make available such code segments developers
>>>> could readily use. Since this post and mail thread has generated a lot of
>>>> negative feeling and confusion, I think it is better to get rid of this
>>>> controversial blog post.
>>>>
>>>> Thanks
>>>> Azeez
>>>>
>>>> On Thu, Mar 31, 2016 at 6:54 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>
>>>>>
>>>>

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

Well one of your responses indicated that the the blogpost suggests
wrapping all DB calls in circuit breakers, and Frank's response indicated
that it is suggesting wrapping all calls to external systems in circuit
breakers.  If it can confuse such brilliant minds, I guess it could mislead
an average developer. That blogpost was on how to implement the pattern in
the context of MSF4J using Hystrix, and I think it should have been
preceded with a blogpost introducing the pattern & explaining where it is
applicable. Sorry for the confusion again. I will work on a separate post.

Thanks
Azeez

On Fri, Apr 1, 2016 at 11:59 PM, Sanjiva Weerawarana <sanj...@wso2.com>
wrote:

> Azeez, asking questions and asking to understand what the purpose of some
> code is not a criticism.
>
> Of course there should be a sample! I only asked about it because to me a
> client side sample made more sense - and then it went into a discussion of
> when this should be used etc..
>
> I would not only put the blog back but also use the opportunity to explain
> to "lazy developers" when and where one should use a circuitbreaker. That's
> 1000x more valuable than just the code on how to do it.
>
> Sanjiva.
>
> On Thu, Mar 31, 2016 at 8:00 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> The blog post has been removed. Sorry for all the confusion. This was
>> only done as part of the agreement we had during last week's meeting to
>> demonstrate certain features such as Spring support, JPA support, support
>> for patterns etc. in order to help developers understand how to implement
>> certain stuff with MSF4J. Our target community of MSF4J is primarily
>> developers, and developers like to refer to sample code segments in order
>> to proceed with implementing their solutions. We lazy developers love to
>> Google search for code segments, e.g. JDBC connection example, and then
>> copy, paste and modify those segments. What I have been trying to do with
>> the series of blogposts is to make available such code segments developers
>> could readily use. Since this post and mail thread has generated a lot of
>> negative feeling and confusion, I think it is better to get rid of this
>> controversial blog post.
>>
>> Thanks
>> Azeez
>>
>> On Thu, Mar 31, 2016 at 6:54 PM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>>
>>>
>>> On Thu, Mar 31, 2016 at 5:03 PM, Frank Leymann <fr...@wso2.com> wrote:
>>>
>>>> Yes, all the stability patterns (that Nygard describes, the circuit
>>>> breaker being just one of them)
>>>> is not associated with microservices, but applies to all distributed
>>>> applications. In fact, Nygard's
>>>> book has been published in 2007, lng before the microservice
>>>> discussion came up ;-)
>>>>
>>>> Yes Frank, agreed. With the hype about microservices, people have
>>> started talking about these a lot and during the evaluation phase, people
>>> look at features available in frameworks. I don't understand the excitement
>>> here. We are not saying CircuitBreaker etc have to be used. That is as
>>> stupid as saying every object instantiation has to be done via a Factory.
>>>
>>>
>>>> Applying these patterns to each and any invocation would be a complete
>>>> misuse.
>>>>
>>>
>>> Yes, it would very stupid for someone to design a system like that or to
>>> suggest something like that, like I said it would be like asking to
>>> instantiate all objects using the Factory pattern!
>>>
>>> Patterns are just part of the toolkit of architects & developers.
>>> Knowing to use the appropriate one at the appropriate place requires proper
>>> judgment. This sample nor this mail thread is not suggesting to use these
>>> everywhere, and I don't understand what gave the impression that we are
>>> suggesting such a thing.
>>>
>>>
>>>> And it will very likely
>>>> result in performance hits...  The circuit breaker pattern, for
>>>> example, is recommended to be applied
>>>> to "out-of-spec errors", i.e. errors that you don't cover in the spec
>>>> (because the errors are too unlikely ;-))
>>>> of the invoked function or in the spec of the program making the call
>>>> (aka client). Often, these are errors
>>>> that never happen during testing unless you really set up a badly
>>>> behaving test environment. And it has
>>>> impact on the design/implementation of the circuit breaker i

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

The blog post has been removed. Sorry for all the confusion. This was only
done as part of the agreement we had during last week's meeting to
demonstrate certain features such as Spring support, JPA support, support
for patterns etc. in order to help developers understand how to implement
certain stuff with MSF4J. Our target community of MSF4J is primarily
developers, and developers like to refer to sample code segments in order
to proceed with implementing their solutions. We lazy developers love to
Google search for code segments, e.g. JDBC connection example, and then
copy, paste and modify those segments. What I have been trying to do with
the series of blogposts is to make available such code segments developers
could readily use. Since this post and mail thread has generated a lot of
negative feeling and confusion, I think it is better to get rid of this
controversial blog post.

Thanks
Azeez

On Thu, Mar 31, 2016 at 6:54 PM, Afkham Azeez <az...@wso2.com> wrote:

>
>
> On Thu, Mar 31, 2016 at 5:03 PM, Frank Leymann <fr...@wso2.com> wrote:
>
>> Yes, all the stability patterns (that Nygard describes, the circuit
>> breaker being just one of them)
>> is not associated with microservices, but applies to all distributed
>> applications. In fact, Nygard's
>> book has been published in 2007, lng before the microservice
>> discussion came up ;-)
>>
>> Yes Frank, agreed. With the hype about microservices, people have started
> talking about these a lot and during the evaluation phase, people look at
> features available in frameworks. I don't understand the excitement here.
> We are not saying CircuitBreaker etc have to be used. That is as stupid as
> saying every object instantiation has to be done via a Factory.
>
>
>> Applying these patterns to each and any invocation would be a complete
>> misuse.
>>
>
> Yes, it would very stupid for someone to design a system like that or to
> suggest something like that, like I said it would be like asking to
> instantiate all objects using the Factory pattern!
>
> Patterns are just part of the toolkit of architects & developers. Knowing
> to use the appropriate one at the appropriate place requires proper
> judgment. This sample nor this mail thread is not suggesting to use these
> everywhere, and I don't understand what gave the impression that we are
> suggesting such a thing.
>
>
>> And it will very likely
>> result in performance hits...  The circuit breaker pattern, for example,
>> is recommended to be applied
>> to "out-of-spec errors", i.e. errors that you don't cover in the spec
>> (because the errors are too unlikely ;-))
>> of the invoked function or in the spec of the program making the call
>> (aka client). Often, these are errors
>> that never happen during testing unless you really set up a badly
>> behaving test environment. And it has
>> impact on the design/implementation of the circuit breaker itself (or
>> clients), for example "critical work"
>> not accepted by the circuit breaker has be queued (by the client? by the
>> circuit breaker?) for later use
>> (automatic replay?).
>>
>> Thus, using one of the stability patterns is a (architecture/design)
>> decision with implications on other
>> components architecture/design.
>>
>> Documenting a sample use of the circuit breaker pattern should also
>> discuss these ramifications.
>>
>>
> Thanks. We will include these recommendations in our documentation.
>
>
>>
>> Best regards,
>> Frank
>>
>> 2016-03-31 9:12 GMT+02:00 Sanjiva Weerawarana <sanj...@wso2.com>:
>>
>>> Agreed. However I had understood that the circuit breaker pattern was
>>> advocated primarily for service clients in MSA (and of course it has
>>> nothing do with being micro).
>>>
>>> The general story of better failure handling applies to all code and is
>>> of course not MSA specific.
>>>
>>> Anyway .. Sample is fine.
>>> On Mar 31, 2016 9:19 AM, "Afkham Azeez" <az...@wso2.com> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Mar 31, 2016 at 9:04 AM, Sanjiva Weerawarana <sanj...@wso2.com>
>>>> wrote:
>>>>
>>>>> That's why I said "fancy try catch" :-).
>>>>>
>>>>> However, are you SERIOUSLY saying that we for example should be
>>>>> wrapping all our DB access code in this stuff? If not who exactly should 
>>>>> be
>>>>> doing this? What are the perf implications?
>>>>>
>>>>
>>>>

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

On Thu, Mar 31, 2016 at 5:03 PM, Frank Leymann <fr...@wso2.com> wrote:

> Yes, all the stability patterns (that Nygard describes, the circuit
> breaker being just one of them)
> is not associated with microservices, but applies to all distributed
> applications. In fact, Nygard's
> book has been published in 2007, lng before the microservice
> discussion came up ;-)
>
> Yes Frank, agreed. With the hype about microservices, people have started
talking about these a lot and during the evaluation phase, people look at
features available in frameworks. I don't understand the excitement here.
We are not saying CircuitBreaker etc have to be used. That is as stupid as
saying every object instantiation has to be done via a Factory.


> Applying these patterns to each and any invocation would be a complete
> misuse.
>

Yes, it would very stupid for someone to design a system like that or to
suggest something like that, like I said it would be like asking to
instantiate all objects using the Factory pattern!

Patterns are just part of the toolkit of architects & developers. Knowing
to use the appropriate one at the appropriate place requires proper
judgment. This sample nor this mail thread is not suggesting to use these
everywhere, and I don't understand what gave the impression that we are
suggesting such a thing.


> And it will very likely
> result in performance hits...  The circuit breaker pattern, for example,
> is recommended to be applied
> to "out-of-spec errors", i.e. errors that you don't cover in the spec
> (because the errors are too unlikely ;-))
> of the invoked function or in the spec of the program making the call (aka
> client). Often, these are errors
> that never happen during testing unless you really set up a badly behaving
> test environment. And it has
> impact on the design/implementation of the circuit breaker itself (or
> clients), for example "critical work"
> not accepted by the circuit breaker has be queued (by the client? by the
> circuit breaker?) for later use
> (automatic replay?).
>
> Thus, using one of the stability patterns is a (architecture/design)
> decision with implications on other
> components architecture/design.
>
> Documenting a sample use of the circuit breaker pattern should also
> discuss these ramifications.
>
>
Thanks. We will include these recommendations in our documentation.


>
> Best regards,
> Frank
>
> 2016-03-31 9:12 GMT+02:00 Sanjiva Weerawarana <sanj...@wso2.com>:
>
>> Agreed. However I had understood that the circuit breaker pattern was
>> advocated primarily for service clients in MSA (and of course it has
>> nothing do with being micro).
>>
>> The general story of better failure handling applies to all code and is
>> of course not MSA specific.
>>
>> Anyway .. Sample is fine.
>> On Mar 31, 2016 9:19 AM, "Afkham Azeez" <az...@wso2.com> wrote:
>>
>>>
>>>
>>> On Thu, Mar 31, 2016 at 9:04 AM, Sanjiva Weerawarana <sanj...@wso2.com>
>>> wrote:
>>>
>>>> That's why I said "fancy try catch" :-).
>>>>
>>>> However, are you SERIOUSLY saying that we for example should be
>>>> wrapping all our DB access code in this stuff? If not who exactly should be
>>>> doing this? What are the perf implications?
>>>>
>>>
>>> No I am not saying that. However, there will be use cases where people
>>> want to use this pattern and this is a simplified sample that demonstrates
>>> how to use this pattern. In Nygards book about how an SQL statement
>>> execution failure resulted in an entire checking in system in an airline
>>> failing because the failure propagated is a good example of uncontrolled
>>> failure propagation (Release It, Chapter 2: Case study: The exception that
>>> grounded an airline, for those of you who have the book). So my example was
>>> somewhat inspired by that case study and is highly simplified.
>>>
>>> If a sample is too complicated, people get lost in the implementation
>>> details rather than seeing how the core concept or pattern is implemented.
>>> I certainly can implement another sample which demonstrates client->service
>>> or service->service calls, it certainly would add more code but the core
>>> concept demonstrated would be the same.
>>>
>>>
>>>
>>>>
>>>> Of course wrapping remote service calls in this stuff makes sense -
>>>> great way to adjust to transient issues. In that case the overhead is
>>>> heavily masked by the latency - I'm not so convinced that is the 

Re: [Architecture] Supporting Carbon Metrics on Carbon 5 based products

[Afkham Azeez]

gt;> >> >> > metrics.
>>>> >> >> > What
>>>> >> >> > will happen to that in Carbon 5?
>>>> >> >> >
>>>> >> >> > How can we implement REST services in Carbon 5? (We may not
>>>> need this
>>>> >> >> > if
>>>> >> >> > we
>>>> >> >> > are not developing any UIs)
>>>> >> >> >
>>>> >> >> > By default, Metrics data are stored in local H2 database. Can
>>>> we use
>>>> >> >> > the
>>>> >> >> > same approach in Carbon 5? (This is not needed if we stop using
>>>> the
>>>> >> >> > JDBC
>>>> >> >> > reporter).
>>>> >> >> >
>>>> >> >> > There is also a properties file to configure Metric Levels. I
>>>> hope
>>>> >> >> > that's
>>>> >> >> > not a problem.
>>>> >> >> >
>>>> >> >> > The plan is to use v2.0.0 for Metrics release with Carbon 5. We
>>>> can
>>>> >> >> > maintain
>>>> >> >> > v1.x.x branch for Carbon 4.x.x products.
>>>> >> >> >
>>>> >> >> > Currently the WSO2 Gateway and MSF4J uses Carbon Metrics as
>>>> Maven
>>>> >> >> > Dependencies. We need a proper Carbon 5 supported release to
>>>> >> >> > integrate
>>>> >> >> > Carbon Metrics to these products.
>>>> >> >> >
>>>> >> >> > I really appreciate your feedback on this.
>>>> >> >> >
>>>> >> >> > Thanks!
>>>> >> >> >
>>>> >> >> > Best Regards,
>>>> >> >> >
>>>> >> >> > --
>>>> >> >> > Isuru Perera
>>>> >> >> > Associate Technical Lead | WSO2, Inc. | http://wso2.com/
>>>> >> >> > Lean . Enterprise . Middleware
>>>> >> >> >
>>>> >> >> > about.me/chrishantha
>>>> >> >> > Contact: +IsuruPereraWSO2
>>>> >> >>
>>>> >> >>
>>>> >> >>
>>>> >> >> --
>>>> >> >> Ramith Jayasinghe
>>>> >> >> Technical Lead
>>>> >> >> WSO2 Inc., http://wso2.com
>>>> >> >> lean.enterprise.middleware
>>>> >> >>
>>>> >> >> E: ram...@wso2.com
>>>> >> >> P: +94 777542851
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > --
>>>> >> > S. Suhothayan
>>>> >> > Technical Lead & Team Lead of WSO2 Complex Event Processor
>>>> >> > WSO2 Inc. http://wso2.com
>>>> >> > lean . enterprise . middleware
>>>> >> >
>>>> >> > cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
>>>> >> > twitter: http://twitter.com/suhothayan | linked-in:
>>>> >> > http://lk.linkedin.com/in/suhothayan
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Ramith Jayasinghe
>>>> >> Technical Lead
>>>> >> WSO2 Inc., http://wso2.com
>>>> >> lean.enterprise.middleware
>>>> >>
>>>> >> E: ram...@wso2.com
>>>> >> P: +94 777542851
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > S. Suhothayan
>>>> > Technical Lead & Team Lead of WSO2 Complex Event Processor
>>>> > WSO2 Inc. http://wso2.com
>>>> > lean . enterprise . middleware
>>>> >
>>>> > cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
>>>> > twitter: http://twitter.com/suhothayan | linked-in:
>>>> > http://lk.linkedin.com/in/suhothayan
>>>>
>>>>
>>>>
>>>> --
>>>> Ramith Jayasinghe
>>>> Technical Lead
>>>> WSO2 Inc., http://wso2.com
>>>> lean.enterprise.middleware
>>>>
>>>> E: ram...@wso2.com
>>>> P: +94 777542851
>>>>
>>>
>>>
>>>
>>> --
>>> Isuru Perera
>>> Associate Technical Lead | WSO2, Inc. | http://wso2.com/
>>> Lean . Enterprise . Middleware
>>>
>>> about.me/chrishantha
>>> Contact: +IsuruPereraWSO2
>>> <https://www.google.com/+IsuruPereraWSO2/about>
>>>
>>> ___
>>> Architecture mailing list
>>> Architecture@wso2.org
>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>
>>>
>>
>>
>> --
>> *Kishanthan Thangarajah*
>> Associate Technical Lead,
>> Platform Technologies Team,
>> WSO2, Inc.
>> lean.enterprise.middleware
>>
>> Mobile - +94773426635
>> Blog - *http://kishanthan.wordpress.com
>> <http://kishanthan.wordpress.com>*
>> Twitter - *http://twitter.com/kishanthan <http://twitter.com/kishanthan>*
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
> Isuru Perera
> Associate Technical Lead | WSO2, Inc. | http://wso2.com/
> Lean . Enterprise . Middleware
>
> about.me/chrishantha
> Contact: +IsuruPereraWSO2 <https://www.google.com/+IsuruPereraWSO2/about>
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

On Thu, Mar 31, 2016 at 9:04 AM, Sanjiva Weerawarana <sanj...@wso2.com>
wrote:

> That's why I said "fancy try catch" :-).
>
> However, are you SERIOUSLY saying that we for example should be wrapping
> all our DB access code in this stuff? If not who exactly should be doing
> this? What are the perf implications?
>

No I am not saying that. However, there will be use cases where people want
to use this pattern and this is a simplified sample that demonstrates how
to use this pattern. In Nygards book about how an SQL statement execution
failure resulted in an entire checking in system in an airline failing
because the failure propagated is a good example of uncontrolled failure
propagation (Release It, Chapter 2: Case study: The exception that grounded
an airline, for those of you who have the book). So my example was somewhat
inspired by that case study and is highly simplified.

If a sample is too complicated, people get lost in the implementation
details rather than seeing how the core concept or pattern is implemented.
I certainly can implement another sample which demonstrates client->service
or service->service calls, it certainly would add more code but the core
concept demonstrated would be the same.



>
> Of course wrapping remote service calls in this stuff makes sense - great
> way to adjust to transient issues. In that case the overhead is heavily
> masked by the latency - I'm not so convinced that is the case for
> transactional JDBC calls but maybe it is. In that case WE must use it
> internally.
>
> Sanjiva.
>
> On Thu, Mar 31, 2016 at 8:53 AM, Afkham Azeez <az...@wso2.com> wrote:
>
>> Equating these fault tolerance patterns to Java 8 Optional or try-catch,
>> is a highly oversimplified view. What Hystrix and these patterns provides
>> is a framework for building fault tolerant systems. Something that is
>> useful in the toolkit of an architect & developer.
>>
>> On Thu, Mar 31, 2016 at 8:36 AM, Sanjiva Weerawarana <sanj...@wso2.com>
>> wrote:
>>
>>> This is almost kinda like that stupid new Java8 thing of "we removed
>>> null by wrapping it in a fancy object" ;-).
>>>
>>> On Thu, Mar 31, 2016 at 8:32 AM, Sanjiva Weerawarana <sanj...@wso2.com>
>>> wrote:
>>>
>>>> So this is not what I expected the real use case to be ... this is
>>>> basically a fancy try catch.
>>>>
>>>> Don't we want to show a client side example?
>>>>
>>>> On Thu, Mar 31, 2016 at 6:28 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>>
>>>>> Timeout is related to the actual operation taking more time than
>>>>> anticipated. In such a case, without waiting indefinitely, the operation
>>>>> times out and the fallback of the Hystrix command will be invoked. The
>>>>> circuit will be open for a fixed period of time configured by
>>>>> https://github.com/Netflix/Hystrix/wiki/Configuration#circuitBreaker.sleepWindowInMilliseconds
>>>>>
>>>>> On Thu, Mar 31, 2016 at 2:53 AM, Harshan Liyanage <hars...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Azeez,
>>>>>>
>>>>>> Does this timeout in open state occurs in exponentially (first
>>>>>> timeout in 10 secs, next in 20 secs etc) or linearly when transitioning
>>>>>> back to half-open state? For example if the state is in "Open" and now 
>>>>>> the
>>>>>> timeout (lets say 10secs timeout) occurs. Then the state is moved to
>>>>>> "half-open" state. But the next request is also a failure and breaker 
>>>>>> state
>>>>>> is moved back to "open". In this occasion the what will be the timeout
>>>>>> value? Is it 10 secs or 20 secs?
>>>>>>
>>>>>> Having an exponential timeout might be beneficiary here as it might
>>>>>> save lot of resources if the service is continuously failing. But I think
>>>>>> it would be better if we can provide both options in a configurable 
>>>>>> manner.
>>>>>> So it is up to the developer to decide which method to use.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Harshan Liyanage
>>>>>> Software Engineer
>>>>>> Mobile: *+94724423048*
>>>>>> Email: hars...@wso2.com
>>>>>> Blog : http://harshanliyanage.blogspot.com/
>>>>>> *WSO2, Inc. :** wso2.com <http://wso2

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

Equating these fault tolerance patterns to Java 8 Optional or try-catch, is
a highly oversimplified view. What Hystrix and these patterns provides is a
framework for building fault tolerant systems. Something that is useful in
the toolkit of an architect & developer.

On Thu, Mar 31, 2016 at 8:36 AM, Sanjiva Weerawarana <sanj...@wso2.com>
wrote:

> This is almost kinda like that stupid new Java8 thing of "we removed null
> by wrapping it in a fancy object" ;-).
>
> On Thu, Mar 31, 2016 at 8:32 AM, Sanjiva Weerawarana <sanj...@wso2.com>
> wrote:
>
>> So this is not what I expected the real use case to be ... this is
>> basically a fancy try catch.
>>
>> Don't we want to show a client side example?
>>
>> On Thu, Mar 31, 2016 at 6:28 AM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> Timeout is related to the actual operation taking more time than
>>> anticipated. In such a case, without waiting indefinitely, the operation
>>> times out and the fallback of the Hystrix command will be invoked. The
>>> circuit will be open for a fixed period of time configured by
>>> https://github.com/Netflix/Hystrix/wiki/Configuration#circuitBreaker.sleepWindowInMilliseconds
>>>
>>> On Thu, Mar 31, 2016 at 2:53 AM, Harshan Liyanage <hars...@wso2.com>
>>> wrote:
>>>
>>>> Hi Azeez,
>>>>
>>>> Does this timeout in open state occurs in exponentially (first timeout
>>>> in 10 secs, next in 20 secs etc) or linearly when transitioning back to
>>>> half-open state? For example if the state is in "Open" and now the timeout
>>>> (lets say 10secs timeout) occurs. Then the state is moved to "half-open"
>>>> state. But the next request is also a failure and breaker state is moved
>>>> back to "open". In this occasion the what will be the timeout value? Is it
>>>> 10 secs or 20 secs?
>>>>
>>>> Having an exponential timeout might be beneficiary here as it might
>>>> save lot of resources if the service is continuously failing. But I think
>>>> it would be better if we can provide both options in a configurable manner.
>>>> So it is up to the developer to decide which method to use.
>>>>
>>>> Thanks,
>>>>
>>>> Harshan Liyanage
>>>> Software Engineer
>>>> Mobile: *+94724423048*
>>>> Email: hars...@wso2.com
>>>> Blog : http://harshanliyanage.blogspot.com/
>>>> *WSO2, Inc. :** wso2.com <http://wso2.com/>*
>>>> lean.enterprise.middleware.
>>>>
>>>> On Wed, Mar 30, 2016 at 5:05 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>>
>>>>> I have written a sample which demonstrates circuit breaker in action;
>>>>> http://blog.afkham.org/2016/03/microservices-circuit-breaker.html
>>>>>
>>>>> On Sat, Mar 12, 2016 at 6:09 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>
>>>>>> This is a feature supported by some microservices frameworks. On the
>>>>>> server side, in this case MSF4J runtime, failure counts are kept track of
>>>>>> and then if the failures exceed certain thresholds, the circuit trips and
>>>>>> rather than dispatch to the service, it returns service unavailable.
>>>>>>
>>>>>> Can you explain why this is not needed in a container environment?
>>>>>>
>>>>>> On Sat, Mar 12, 2016 at 12:56 PM, Sanjiva Weerawarana <
>>>>>> sanj...@wso2.com> wrote:
>>>>>>
>>>>>>> I don't understand what server side circuit breaker means. How does
>>>>>>> the server adjust itself? Where's that bit of logic running?
>>>>>>>
>>>>>>> IMO this is not needed in a container world.
>>>>>>>
>>>>>>> On Fri, Mar 11, 2016 at 4:38 PM, Afkham Azeez <az...@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Yes, that is client side circuit breaker. What Aruna is
>>>>>>>> implementing is server side circuit breaker. Yes, we need both.
>>>>>>>>
>>>>>>>> On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana <
>>>>>>>> lak...@wso2.com> wrote:
>>>>>>>>
>>>>>>>>> Did you looked at [1]
>>>>>>>>>
>>>>>>>>&g

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

The same concept applies to the client side as well. We could modify this
sample to show interaction between two microservices, where one service
calls the other, and if that call keeps failing, the circuit would trip,
which would be a sample for client side circuit breaker.

 In the real world, DB calls could fail for sometime (too many calls to the
DB at a given time), and there the circuit would trip and fallback to
serving from a cache, which is what this simplified example demonstrates.


On Thu, Mar 31, 2016 at 8:32 AM, Sanjiva Weerawarana <sanj...@wso2.com>
wrote:

> So this is not what I expected the real use case to be ... this is
> basically a fancy try catch.
>
> Don't we want to show a client side example?
>
> On Thu, Mar 31, 2016 at 6:28 AM, Afkham Azeez <az...@wso2.com> wrote:
>
>> Timeout is related to the actual operation taking more time than
>> anticipated. In such a case, without waiting indefinitely, the operation
>> times out and the fallback of the Hystrix command will be invoked. The
>> circuit will be open for a fixed period of time configured by
>> https://github.com/Netflix/Hystrix/wiki/Configuration#circuitBreaker.sleepWindowInMilliseconds
>>
>> On Thu, Mar 31, 2016 at 2:53 AM, Harshan Liyanage <hars...@wso2.com>
>> wrote:
>>
>>> Hi Azeez,
>>>
>>> Does this timeout in open state occurs in exponentially (first timeout
>>> in 10 secs, next in 20 secs etc) or linearly when transitioning back to
>>> half-open state? For example if the state is in "Open" and now the timeout
>>> (lets say 10secs timeout) occurs. Then the state is moved to "half-open"
>>> state. But the next request is also a failure and breaker state is moved
>>> back to "open". In this occasion the what will be the timeout value? Is it
>>> 10 secs or 20 secs?
>>>
>>> Having an exponential timeout might be beneficiary here as it might save
>>> lot of resources if the service is continuously failing. But I think it
>>> would be better if we can provide both options in a configurable manner. So
>>> it is up to the developer to decide which method to use.
>>>
>>> Thanks,
>>>
>>> Harshan Liyanage
>>> Software Engineer
>>> Mobile: *+94724423048*
>>> Email: hars...@wso2.com
>>> Blog : http://harshanliyanage.blogspot.com/
>>> *WSO2, Inc. :** wso2.com <http://wso2.com/>*
>>> lean.enterprise.middleware.
>>>
>>> On Wed, Mar 30, 2016 at 5:05 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> I have written a sample which demonstrates circuit breaker in action;
>>>> http://blog.afkham.org/2016/03/microservices-circuit-breaker.html
>>>>
>>>> On Sat, Mar 12, 2016 at 6:09 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>
>>>>> This is a feature supported by some microservices frameworks. On the
>>>>> server side, in this case MSF4J runtime, failure counts are kept track of
>>>>> and then if the failures exceed certain thresholds, the circuit trips and
>>>>> rather than dispatch to the service, it returns service unavailable.
>>>>>
>>>>> Can you explain why this is not needed in a container environment?
>>>>>
>>>>> On Sat, Mar 12, 2016 at 12:56 PM, Sanjiva Weerawarana <
>>>>> sanj...@wso2.com> wrote:
>>>>>
>>>>>> I don't understand what server side circuit breaker means. How does
>>>>>> the server adjust itself? Where's that bit of logic running?
>>>>>>
>>>>>> IMO this is not needed in a container world.
>>>>>>
>>>>>> On Fri, Mar 11, 2016 at 4:38 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>>
>>>>>>> Yes, that is client side circuit breaker. What Aruna is implementing
>>>>>>> is server side circuit breaker. Yes, we need both.
>>>>>>>
>>>>>>> On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana <
>>>>>>> lak...@wso2.com> wrote:
>>>>>>>
>>>>>>>> Did you looked at [1]
>>>>>>>>
>>>>>>>> Netflix Hystrix <https://github.com/Netflix/Hystrix> is an
>>>>>>>> incredibly useful library for writing code that invokes remote 
>>>>>>>> services.
>>>>>>>> Hystrix times out calls that exceed the specified threshold. It 
>>>>>>>&g

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

Timeout is related to the actual operation taking more time than
anticipated. In such a case, without waiting indefinitely, the operation
times out and the fallback of the Hystrix command will be invoked. The
circuit will be open for a fixed period of time configured by
https://github.com/Netflix/Hystrix/wiki/Configuration#circuitBreaker.sleepWindowInMilliseconds

On Thu, Mar 31, 2016 at 2:53 AM, Harshan Liyanage <hars...@wso2.com> wrote:

> Hi Azeez,
>
> Does this timeout in open state occurs in exponentially (first timeout in
> 10 secs, next in 20 secs etc) or linearly when transitioning back to
> half-open state? For example if the state is in "Open" and now the timeout
> (lets say 10secs timeout) occurs. Then the state is moved to "half-open"
> state. But the next request is also a failure and breaker state is moved
> back to "open". In this occasion the what will be the timeout value? Is it
> 10 secs or 20 secs?
>
> Having an exponential timeout might be beneficiary here as it might save
> lot of resources if the service is continuously failing. But I think it
> would be better if we can provide both options in a configurable manner. So
> it is up to the developer to decide which method to use.
>
> Thanks,
>
> Harshan Liyanage
> Software Engineer
> Mobile: *+94724423048*
> Email: hars...@wso2.com
> Blog : http://harshanliyanage.blogspot.com/
> *WSO2, Inc. :** wso2.com <http://wso2.com/>*
> lean.enterprise.middleware.
>
> On Wed, Mar 30, 2016 at 5:05 AM, Afkham Azeez <az...@wso2.com> wrote:
>
>> I have written a sample which demonstrates circuit breaker in action;
>> http://blog.afkham.org/2016/03/microservices-circuit-breaker.html
>>
>> On Sat, Mar 12, 2016 at 6:09 PM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> This is a feature supported by some microservices frameworks. On the
>>> server side, in this case MSF4J runtime, failure counts are kept track of
>>> and then if the failures exceed certain thresholds, the circuit trips and
>>> rather than dispatch to the service, it returns service unavailable.
>>>
>>> Can you explain why this is not needed in a container environment?
>>>
>>> On Sat, Mar 12, 2016 at 12:56 PM, Sanjiva Weerawarana <sanj...@wso2.com>
>>> wrote:
>>>
>>>> I don't understand what server side circuit breaker means. How does the
>>>> server adjust itself? Where's that bit of logic running?
>>>>
>>>> IMO this is not needed in a container world.
>>>>
>>>> On Fri, Mar 11, 2016 at 4:38 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>
>>>>> Yes, that is client side circuit breaker. What Aruna is implementing
>>>>> is server side circuit breaker. Yes, we need both.
>>>>>
>>>>> On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana <lak...@wso2.com
>>>>> > wrote:
>>>>>
>>>>>> Did you looked at [1]
>>>>>>
>>>>>> Netflix Hystrix <https://github.com/Netflix/Hystrix> is an
>>>>>> incredibly useful library for writing code that invokes remote services.
>>>>>> Hystrix times out calls that exceed the specified threshold. It 
>>>>>> implements
>>>>>> a *circuit breaker* pattern, which stops the client from waiting
>>>>>> needlessly for an unresponsive service. If the error rate for a service
>>>>>> exceeds a specified threshold, Hystrix trips the circuit breaker and all
>>>>>> requests will fail immediately for a specified period of time. Hystrix 
>>>>>> lets
>>>>>> you define a fallback action when a request fails, such as reading from a
>>>>>> cache or returning a default value. If you are using the JVM you should
>>>>>> definitely consider using Hystrix.
>>>>>>
>>>>>> [1] https://github.com/Netflix/Hystrix
>>>>>>
>>>>>> On Fri, Mar 11, 2016 at 2:42 PM, Aruna Karunarathna <ar...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Devs,
>>>>>>>
>>>>>>> *Scenario*
>>>>>>>
>>>>>>> The deployed services in a MSF4J may fail to serve the requests due
>>>>>>> to various factors. e.g,
>>>>>>> 1. Less resources in the server.
>>>>>>> 2. High Load in the server
>>>>>>> 3, Some services take more time to respond etc.
>>

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

I have written a sample which demonstrates circuit breaker in action;
http://blog.afkham.org/2016/03/microservices-circuit-breaker.html

On Sat, Mar 12, 2016 at 6:09 PM, Afkham Azeez <az...@wso2.com> wrote:

> This is a feature supported by some microservices frameworks. On the
> server side, in this case MSF4J runtime, failure counts are kept track of
> and then if the failures exceed certain thresholds, the circuit trips and
> rather than dispatch to the service, it returns service unavailable.
>
> Can you explain why this is not needed in a container environment?
>
> On Sat, Mar 12, 2016 at 12:56 PM, Sanjiva Weerawarana <sanj...@wso2.com>
> wrote:
>
>> I don't understand what server side circuit breaker means. How does the
>> server adjust itself? Where's that bit of logic running?
>>
>> IMO this is not needed in a container world.
>>
>> On Fri, Mar 11, 2016 at 4:38 PM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> Yes, that is client side circuit breaker. What Aruna is implementing is
>>> server side circuit breaker. Yes, we need both.
>>>
>>> On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana <lak...@wso2.com>
>>> wrote:
>>>
>>>> Did you looked at [1]
>>>>
>>>> Netflix Hystrix <https://github.com/Netflix/Hystrix> is an incredibly
>>>> useful library for writing code that invokes remote services. Hystrix times
>>>> out calls that exceed the specified threshold. It implements a *circuit
>>>> breaker* pattern, which stops the client from waiting needlessly for
>>>> an unresponsive service. If the error rate for a service exceeds a
>>>> specified threshold, Hystrix trips the circuit breaker and all requests
>>>> will fail immediately for a specified period of time. Hystrix lets you
>>>> define a fallback action when a request fails, such as reading from a cache
>>>> or returning a default value. If you are using the JVM you should
>>>> definitely consider using Hystrix.
>>>>
>>>> [1] https://github.com/Netflix/Hystrix
>>>>
>>>> On Fri, Mar 11, 2016 at 2:42 PM, Aruna Karunarathna <ar...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Devs,
>>>>>
>>>>> *Scenario*
>>>>>
>>>>> The deployed services in a MSF4J may fail to serve the requests due to
>>>>> various factors. e.g,
>>>>> 1. Less resources in the server.
>>>>> 2. High Load in the server
>>>>> 3, Some services take more time to respond etc.
>>>>>
>>>>> In this kind of a situation, if the server is getting requests though
>>>>> there is no resources to serve those requests, and eventually the server
>>>>> will get unusable.
>>>>>
>>>>> *Solution*
>>>>>
>>>>> The Circuit Breaker design pattern can save the server from above
>>>>> scenarios, The typical design can be illustrated as in the following
>>>>> diagram.
>>>>>
>>>>>
>>>>> So as in the above diagram, when number of failures of a particular
>>>>> resource exceeds the Max Failure Count, then the state of that resource is
>>>>> moved to the open state with a timeout value (Trip Breaker). At this point
>>>>> the requests coming to the server is routed back without passing the
>>>>> internal to process further.
>>>>>
>>>>> After the timeout has reached, the state is moved to Half-Open state,
>>>>> and if the consecutive request pass to the server to process (Attempt
>>>>> Reset), if success then close the circuit (Reset Breaker), If fail then
>>>>> again move the state to the Open with a timeout value (Trip Breaker).
>>>>>
>>>>> Any thoughts, suggestions regarding the above approach?
>>>>>
>>>>> References
>>>>> [1].
>>>>> http://www.javaworld.com/article/2824163/application-performance/stability-patterns-applied-in-a-restful-architecture.html?page=2
>>>>> [2].
>>>>> http://ssagara.blogspot.com/2015/05/timeout-and-circuit-breaker-pattern-in.html
>>>>> [3]. https://pragprog.com/book/mnee/release-it
>>>>>
>>>>>
>>>>> Regards,
>>>>> Aruna
>>>>> --
>>>>>
>>>>> *Aruna Sujith Karunarathna *
>>>>> WSO2, Inc | lean. enterprise. middle

Re: [Architecture] [bpmn] Rewrite REST API with msf4j

[Afkham Azeez]

One more thing, we are going to rename javax.rs.* annotations to
org.wso2.msf4j.annotations.* to avoid confusing MSF4J annotations with
JAXRS annotations.

On Tue, Mar 29, 2016 at 11:25 AM, Afkham Azeez <az...@wso2.com> wrote:

> Why don't you pass the relevant information in the HTTP payload? You can
> define a Java bean that contains all the information you need to pass and
> then use that as a method parameter.
>
> On Tue, Mar 29, 2016 at 10:49 AM, Himasha Guruge <himas...@wso2.com>
> wrote:
>
>> Hi Hasitha,
>>
>> Thanks for the provided suggestions. I tested the suggested approaches
>> and we can go ahead with HttpRequest injection and the usage of @QueryParam.
>>
>>
>> Regards,
>> Himasha
>>
>> On Fri, Mar 25, 2016 at 11:02 AM, Hasitha Aravinda <hasi...@wso2.com>
>> wrote:
>>
>>> Hi team,
>>>
>>> Please find my comments inline.
>>>
>>> On Fri, Mar 25, 2016 at 10:27 AM, Himasha Guruge <himas...@wso2.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I'm in the process of rewriting our existing BPMN REST API for C5, with
>>>> msf4j in place. While checking the used annotations in the code,  I came
>>>> across the use of @Context annotation in order to inject UserInfo [1]
>>>>  interface which provides access to application and request URI
>>>> information. We have used following two methods of this interface
>>>> throughout the code.
>>>>
>>>> (1) QueryParameters()
>>>> (2) getBaseUri()
>>>>
>>>
>>> Do we have an equlent for
>>> ​
>>> uriInfo.getBaseUri in MSF4J supported annotation ? Can't we use
>>> io.netty.handler.codec.http.HttpRequest.getUri()
>>> ​ ​
>>>
>>> ​?​
>>>
>>>
>>> I tested if it's possible to inject UriInfo interface in msf4j by
>>>> deploying a simple microservice sample (which uses @Context UriInfo info)
>>>>  which failed with an error in executing the request, which means the
>>>> injection of it is not possible with current msf4j implementation. [2]
>>>>
>>>> In [2] it is mentioned that there is @QueryParam annotation support in
>>>> msf4j which we can use alternatively where we would have to pass the query
>>>> parameter to the method in the same way we use @PathParam. See usage in 
>>>> [3].
>>>>
>>>> However, issues arise in following scenarios.
>>>>
>>>>  For example, In ModelService[4] we have a large  range of allowed
>>>>  query params (name,nameLike,category,categoryLike etc) for a method, and
>>>> based on the query params that the user has  passed (by using
>>>> uriInfo.getQueryParameters()) we process the request accordingly.   In such
>>>> a case if we are to use @
>>>> ​​
>>>> QueryParam it would mean we would have to pass all possible query
>>>> parameters (Around 15) to the method.
>>>>
>>>
>>> I think this is should be ok.
>>>
>>> ​
>>>>
>>>>
>>> Also, we have used
>>>> ​​
>>>> uriInfo.getBaseUri() method throughout as well.
>>>>
>>>> How are we going to proceed with this? Appreciate your suggestions on
>>>> this.
>>>>
>>>> [1]
>>>> https://jsr311.java.net/nonav/releases/1.1/javax/ws/rs/core/UriInfo.html
>>>>
>>>>
>>>> [2]
>>>> https://docs.wso2.com/display/MSF4J100/Key+Concepts#KeyConcepts-Supportedannotations
>>>>
>>>> [3] http://www.mkyong.com/webservices/jax-rs/jax-rs-queryparam-example/
>>>>
>>>> [4]
>>>> https://github.com/wso2/carbon-business-process/blob/de4d3ca5a2c3d670dcbf07ee6b58f0afd9489e95/components/bpmn/org.wso2.carbon.bpmn.rest/src/main/java/org/wso2/carbon/bpmn/rest/service/repository/ModelService.java#L98
>>>>
>>>> Regards,
>>>>
>>>> Himasha Guruge
>>>> *Software Engineer*
>>>> WS*O2* *Inc.*
>>>> Mobile: +94 777459299
>>>> himas...@wso2.com
>>>>
>>>
>>> ​Thanks,
>>> Hasitha. ​
>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Hasitha Aravinda,
>>> Senior Software Engineer,
>>> WSO2 Inc.
>>> Email: hasi...@wso2.com
>>> Mobile : +94 718 210 200
>>>
>>
>>
>>
>> --
>> Himasha Guruge
>

Re: [Architecture] [bpmn] Rewrite REST API with msf4j

[Afkham Azeez]

Why don't you pass the relevant information in the HTTP payload? You can
define a Java bean that contains all the information you need to pass and
then use that as a method parameter.

On Tue, Mar 29, 2016 at 10:49 AM, Himasha Guruge <himas...@wso2.com> wrote:

> Hi Hasitha,
>
> Thanks for the provided suggestions. I tested the suggested approaches and
> we can go ahead with HttpRequest injection and the usage of @QueryParam.
>
>
> Regards,
> Himasha
>
> On Fri, Mar 25, 2016 at 11:02 AM, Hasitha Aravinda <hasi...@wso2.com>
> wrote:
>
>> Hi team,
>>
>> Please find my comments inline.
>>
>> On Fri, Mar 25, 2016 at 10:27 AM, Himasha Guruge <himas...@wso2.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> I'm in the process of rewriting our existing BPMN REST API for C5, with
>>> msf4j in place. While checking the used annotations in the code,  I came
>>> across the use of @Context annotation in order to inject UserInfo [1]
>>>  interface which provides access to application and request URI
>>> information. We have used following two methods of this interface
>>> throughout the code.
>>>
>>> (1) QueryParameters()
>>> (2) getBaseUri()
>>>
>>
>> Do we have an equlent for
>> ​
>> uriInfo.getBaseUri in MSF4J supported annotation ? Can't we use
>> io.netty.handler.codec.http.HttpRequest.getUri()
>> ​ ​
>>
>> ​?​
>>
>>
>> I tested if it's possible to inject UriInfo interface in msf4j by
>>> deploying a simple microservice sample (which uses @Context UriInfo info)
>>>  which failed with an error in executing the request, which means the
>>> injection of it is not possible with current msf4j implementation. [2]
>>>
>>> In [2] it is mentioned that there is @QueryParam annotation support in
>>> msf4j which we can use alternatively where we would have to pass the query
>>> parameter to the method in the same way we use @PathParam. See usage in [3].
>>>
>>> However, issues arise in following scenarios.
>>>
>>>  For example, In ModelService[4] we have a large  range of allowed
>>>  query params (name,nameLike,category,categoryLike etc) for a method, and
>>> based on the query params that the user has  passed (by using
>>> uriInfo.getQueryParameters()) we process the request accordingly.   In such
>>> a case if we are to use @
>>> ​​
>>> QueryParam it would mean we would have to pass all possible query
>>> parameters (Around 15) to the method.
>>>
>>
>> I think this is should be ok.
>>
>> ​
>>>
>>>
>> Also, we have used
>>> ​​
>>> uriInfo.getBaseUri() method throughout as well.
>>>
>>> How are we going to proceed with this? Appreciate your suggestions on
>>> this.
>>>
>>> [1]
>>> https://jsr311.java.net/nonav/releases/1.1/javax/ws/rs/core/UriInfo.html
>>>
>>>
>>> [2]
>>> https://docs.wso2.com/display/MSF4J100/Key+Concepts#KeyConcepts-Supportedannotations
>>>
>>> [3] http://www.mkyong.com/webservices/jax-rs/jax-rs-queryparam-example/
>>>
>>> [4]
>>> https://github.com/wso2/carbon-business-process/blob/de4d3ca5a2c3d670dcbf07ee6b58f0afd9489e95/components/bpmn/org.wso2.carbon.bpmn.rest/src/main/java/org/wso2/carbon/bpmn/rest/service/repository/ModelService.java#L98
>>>
>>> Regards,
>>>
>>> Himasha Guruge
>>> *Software Engineer*
>>> WS*O2* *Inc.*
>>> Mobile: +94 777459299
>>> himas...@wso2.com
>>>
>>
>> ​Thanks,
>> Hasitha. ​
>>
>>
>>
>>
>> --
>> --
>> Hasitha Aravinda,
>> Senior Software Engineer,
>> WSO2 Inc.
>> Email: hasi...@wso2.com
>> Mobile : +94 718 210 200
>>
>
>
>
> --
> Himasha Guruge
> *Software Engineer*
> WS*O2* *Inc.*
> Mobile: +94 777459299
> himas...@wso2.com
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Session Affinity Alternatives

[Afkham Azeez]

On Fri, Mar 18, 2016 at 11:38 AM, Srinath Perera <srin...@wso2.com> wrote:

> Hi Frank,
>
> Proposal is when the session parameter has changed, framework (e.g.
> Servelt runtime, or MSS framework) will write the update to the disk
> asynchronsly.
>
> Since we are a middleware platform, impact of losing a session depends on
> the kind of the application end user is running.
>
> However, AFAIK session replication or pressitance that is in WSO2 AS was
> rarely used. ( Azeez, please correct me if I am wrong).
>

Yes, that is correct. In traditional deployments, you don't expect servers
to fail often, and try to keep the unplanned downtime to a minimum as
possible. However, with the new move towards containers, the game has
changed. Containers come and go often. So, we need to have a different
mindset when dealing with containers, and session affinity might not be the
best option in the container world. So for container based deployments, we
need to look at the application and then make recommendations on the best
session management option.

I think we agreed on the following:

1. Default in WSO2 products will be session affinity based. This keeps
things simple.
2. For HA deployments, we will need session persistence.


>
>
> --Srinath
>
> On Thu, Mar 17, 2016 at 11:42 PM, Frank Leymann <fr...@wso2.com> wrote:
>
>> Sorry for jumping in late in the discussion
>>
>> Session affinity in general is bad (scalability, HA) - I guess that's
>> what we all agree on.  But in certain scenarios, sticky sessions may be
>> fine.  Thus, what is the underlying scenario in the case we discuss?
>>
>> As far as I understand, persisting session state requires the application
>> to be aware of this. Or is the proposal that provide an environment that
>> reconstructs the session state on behalf of the application?  As Srinath
>> points out we are loosing data if a session aborts and the application is
>> not a transaction: is that critical in our scenario?
>>
>>
>> Best regards,
>> Frank
>>
>> 2016-03-17 10:21 GMT+01:00 Srinath Perera <srin...@wso2.com>:
>>
>>> Let's not do session replication. It is very hard to make it work IMO.
>>>
>>> I would like to propose a variation to Azeez's version.
>>>
>>> We can do local session + session affinity + asynchronously save the
>>> session state to a DB
>>>
>>> If a node cannot find the session, it will go and reload session from
>>> the DB. ( This should happen if the node has failed, or we have moved
>>> session away due to high load).
>>>
>>> With this model, there is a chance that you might loose last update to
>>> the session. However, that will be very rare. I would keep "asynchronously
>>> save the session state to a DB" off by default, so user will enable it for
>>> complex scenarios.
>>>
>>> --Srinath
>>>
>>>
>>>
>>> On Sat, Mar 12, 2016 at 6:25 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> Of course the simplest solution is similar to what Tomcat does, local
>>>> sessions (no replication) & in a cluster, have session affinity configured
>>>> at the load balancer, so that should be the default. If the node that had
>>>> the session dies, the clients connected to that instance would get errors
>>>> or have to login again. For HA deployments, we would need session
>>>> replication or session persistence.
>>>>
>>>> On Sat, Mar 12, 2016 at 12:58 PM, Sanjiva Weerawarana <sanj...@wso2.com
>>>> > wrote:
>>>>
>>>>> Azeez we cannot have a model where every simple server (cluster)
>>>>> deployment requires Redis.
>>>>>
>>>>> Please indicate what you think the right solution is .. its not clear
>>>>> to me.
>>>>>
>>>>> On Thu, Mar 10, 2016 at 7:34 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>
>>>>>> Storing everything as cookies may not work always  and there could be
>>>>>> sensitive runtime data that you don't want to save on the browser. In
>>>>>> addition, when it comes to Java programming models, using the HTTP 
>>>>>> session
>>>>>> to store serializable objects is the natural way of programming. Yes, 
>>>>>> your
>>>>>> solution would work for certain cases, but it doesn't cover all cases.
>>>>>>
>>>>>> On Thu, Mar 10, 2016 at 6:48 PM, Joseph Fonseka <jos...@wso2.com>
>&

Re: [Architecture] The new disruptor based Netty transport is not working well for MSF4J

[Afkham Azeez]

Our initial assumptions were wrong and we were able to narrow down this
issue to a wrong configuration parameter name. Due to that, we were always
running with a single disruptor event handler thread instead of a thread
pool. Now we are seeing better perf values with disruptor as opposed to the
Netty executor thread pool, with some tuning of disruptor configuration
parameters.

Azeez

On Mon, Mar 14, 2016 at 1:55 PM, Sagara Gunathunga <sag...@wso2.com> wrote:

>
>
> On Mon, Mar 14, 2016 at 1:53 PM, Srinath Perera <srin...@wso2.com> wrote:
>
>> I talked to Ranawaka and Isuru in detail.
>>
>> Disrupter helps a lot when tasks are CPU bound. In such cases, in can
>> works with very few threads and reduce the overhead of communication
>> between threads.
>>
>> However, when threads  block for I/O this advantage is reduced a lot. In
>> those cases, we need to have multiple disrupter workers ( batch
>> processors). We are doing that.
>>
>> However, doing  test with 500ms sleep is not fair IMO. Sleep often waits
>> more than the given value. With 200 threads, it can only do 400TPS with
>> 500ms sleep. I think we should compare against a DB backend.
>>
>> Shell we test disrupter and java work pool model against a DB backend?
>>
>
> +1 I also think we should use more realistic backend such as DB.
>
> Thanks !
>
>>
>> --Srinath
>>
>>
>>
>> On Mon, Mar 14, 2016 at 10:26 AM, Kasun Indrasiri <ka...@wso2.com> wrote:
>>
>>> Hi,
>>>
>>> Let me try to clarify few things here.
>>>
>>> - Initially we implemented Netty HTTP transport with conventional thread
>>> model (workers) and at the same time we also tested the Disruptor based
>>> model for the same Gateway/Header based routing use case. Disruptor based
>>> approach gave us around  ~20k of perf gain on perf testing environments.
>>> - MSF4J 1.0 GA didn't use GW's HTTP transport code as it is. It reused
>>> basic transport runtime but with a custom Netty handler that is used to
>>> dispatch the requests. So, MSFJ 1.0 GA, didn't have disruptor and
>>> performance/latency of MSF4J 1.0 GA has nothing to do with Disruprtor.
>>>
>>> - Now we are trying to migrate MSF4J into the exact same transport code
>>> as it with the GW core (carbon-transport's HTTP transport). And that's
>>> where we came across the above perf issue.
>>> - So, unlike GW scenario, for MSF4J and even for any other content-aware
>>> ESB scenario, the above approach is not the optimum it seems. Hence we are
>>> now looking into how such scenarios are handled with Disruptor.
>>>
>>> In that context, if we look at the original LMAX use case[1] is also
>>> quite similar to what we are trying with content aware scenarios. In their
>>> use case they had heavy CPU intensive components(such as Business Logic
>>> component) as well as IO bound components (such as Un-marshaller,
>>> Replicator). And they get better performance for the same use case with
>>> Disruptor over a conventional worker-thread model.
>>>
>>>
>>> [image: Inline image 1]
>>>
>>> So, we need to have a close look into that how we can implement a
>>> dependent consumer scenario[2] (one consumer is IO bound and the other is
>>> CPU bound) and check whether we can get more perf gain compared to the
>>> conventional worker thread model.
>>>
>>> Ranawaka is current looking into this.
>>>
>>> [1] http://martinfowler.com/articles/lmax.html
>>> [2]
>>> http://mechanitis.blogspot.com/2011/07/dissecting-disruptor-wiring-up.html
>>>
>>> On Sun, Mar 13, 2016 at 8:11 AM, Isuru Ranawaka <isu...@wso2.com> wrote:
>>>
>>>> Hi Azeez,
>>>>
>>>> In GW Disruptor threads are not used for make calls for backends.
>>>> Backends are called by Netty worker pool and those calls are non blocking
>>>> calls. So if backend responds after a delay it won't be a problem.
>>>>
>>>>
>>>> In MSF4J  if it includes IO operations or delayed operations then it
>>>> causes problems because processing happens through Disruptor threads and
>>>> by occupying all the limited disruptor threads. But this should be solved
>>>> by operating Disruptor Event Handlers through workerpool and now we are
>>>> looking in to that why it does not provide expected results.
>>>>
>>>> thanks
>>>> IsuruR
>>>>
>>>> On Sat, Mar 12, 2016 at 6:46

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

On Mon, Mar 14, 2016 at 7:18 AM, Lakmal Warusawithana <lak...@wso2.com>
wrote:

> Hi Azeez,
>
> Without server side circuit breaker, can't we achieve this simply in load
> balancers? See nginx "Passive Health Monitoring". It has implemented this
> into some level.
>

Yes, it can be achieved to a certain degree, but concepts like half open
cct, variable time windows (e.g. 5 min, 10 min, 15 min failure rates) etc.
offer a lot of flexibility, and we don't have to do a lot to achieve that
because we already have metrics support in MSF4J. Also, we don't have to
implement this for multiple load balancers.


>
> My point is all these micro services going to be fronted by a load
> balancers and if clients not implementing client side circuit breaker,
>  best place to support circuit breakers is in the load balancers.
>
> On Sat, Mar 12, 2016 at 6:16 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> Isabelle & Sanjiva,
>>
>> http://www.javaworld.com/article/2824163/application-performance/stability-patterns-applied-in-a-restful-architecture.html?page=2
>> which Aruna had given as a reference explains circuit breaker on the server
>> side. IMO, circuit breaker on both client side & server side are useful
>> features in a microservices framework. In fact these are features people
>> look for when evaluating microservices frameworks.
>>
>> Thanks
>> Azeez
>>
>> On Sat, Mar 12, 2016 at 5:11 PM, Isabelle Mauny <isabe...@wso2.com>
>> wrote:
>>
>>> If an API GW is used to access the microservices , then the circuit
>>> breaker pattern would apply at that level, right ?  If the client is a web
>>> app directly calling MS (not that this would be a good pattern at all !)
>>> then the client is the web app. In any case, they are all clients calling
>>> the microservices. So I am not sure about server side either..
>>>
>>>
>>> -
>>> *Isabelle Mauny*
>>> VP, Product Management - WSO2, Inc. - http://wso2.com/
>>>
>>> On Sat, Mar 12, 2016 at 8:26 AM, Sanjiva Weerawarana <sanj...@wso2.com>
>>> wrote:
>>>
>>>> I don't understand what server side circuit breaker means. How does the
>>>> server adjust itself? Where's that bit of logic running?
>>>>
>>>> IMO this is not needed in a container world.
>>>>
>>>> On Fri, Mar 11, 2016 at 4:38 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>
>>>>> Yes, that is client side circuit breaker. What Aruna is implementing
>>>>> is server side circuit breaker. Yes, we need both.
>>>>>
>>>>> On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana <lak...@wso2.com
>>>>> > wrote:
>>>>>
>>>>>> Did you looked at [1]
>>>>>>
>>>>>> Netflix Hystrix <https://github.com/Netflix/Hystrix> is an
>>>>>> incredibly useful library for writing code that invokes remote services.
>>>>>> Hystrix times out calls that exceed the specified threshold. It 
>>>>>> implements
>>>>>> a *circuit breaker* pattern, which stops the client from waiting
>>>>>> needlessly for an unresponsive service. If the error rate for a service
>>>>>> exceeds a specified threshold, Hystrix trips the circuit breaker and all
>>>>>> requests will fail immediately for a specified period of time. Hystrix 
>>>>>> lets
>>>>>> you define a fallback action when a request fails, such as reading from a
>>>>>> cache or returning a default value. If you are using the JVM you should
>>>>>> definitely consider using Hystrix.
>>>>>>
>>>>>> [1] https://github.com/Netflix/Hystrix
>>>>>>
>>>>>> On Fri, Mar 11, 2016 at 2:42 PM, Aruna Karunarathna <ar...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Devs,
>>>>>>>
>>>>>>> *Scenario*
>>>>>>>
>>>>>>> The deployed services in a MSF4J may fail to serve the requests due
>>>>>>> to various factors. e.g,
>>>>>>> 1. Less resources in the server.
>>>>>>> 2. High Load in the server
>>>>>>> 3, Some services take more time to respond etc.
>>>>>>>
>>>>>>> In this kind of a situation,

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

On Mon, Mar 14, 2016 at 7:41 AM, Lakmal Warusawithana <lak...@wso2.com>
wrote:

> Actually even client side circuit breakers are not going to work, if we
> fronted micro services via LB.
>

Cct breakers work within time time windows, and within a time window, if
many calls failed, the client side cct breaker should trip, and that would
be the correct behavior.


>
> On Mon, Mar 14, 2016 at 7:18 AM, Lakmal Warusawithana <lak...@wso2.com>
> wrote:
>
>> Hi Azeez,
>>
>> Without server side circuit breaker, can't we achieve this simply in
>> load balancers? See nginx "Passive Health Monitoring". It has implemented
>> this into some level.
>>
>> My point is all these micro services going to be fronted by a load
>> balancers and if clients not implementing client side circuit breaker,
>>  best place to support circuit breakers is in the load balancers.
>>
>> On Sat, Mar 12, 2016 at 6:16 PM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> Isabelle & Sanjiva,
>>>
>>> http://www.javaworld.com/article/2824163/application-performance/stability-patterns-applied-in-a-restful-architecture.html?page=2
>>> which Aruna had given as a reference explains circuit breaker on the server
>>> side. IMO, circuit breaker on both client side & server side are useful
>>> features in a microservices framework. In fact these are features people
>>> look for when evaluating microservices frameworks.
>>>
>>> Thanks
>>> Azeez
>>>
>>> On Sat, Mar 12, 2016 at 5:11 PM, Isabelle Mauny <isabe...@wso2.com>
>>> wrote:
>>>
>>>> If an API GW is used to access the microservices , then the circuit
>>>> breaker pattern would apply at that level, right ?  If the client is a web
>>>> app directly calling MS (not that this would be a good pattern at all !)
>>>> then the client is the web app. In any case, they are all clients calling
>>>> the microservices. So I am not sure about server side either..
>>>>
>>>>
>>>> -
>>>> *Isabelle Mauny*
>>>> VP, Product Management - WSO2, Inc. - http://wso2.com/
>>>>
>>>> On Sat, Mar 12, 2016 at 8:26 AM, Sanjiva Weerawarana <sanj...@wso2.com>
>>>> wrote:
>>>>
>>>>> I don't understand what server side circuit breaker means. How does
>>>>> the server adjust itself? Where's that bit of logic running?
>>>>>
>>>>> IMO this is not needed in a container world.
>>>>>
>>>>> On Fri, Mar 11, 2016 at 4:38 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>
>>>>>> Yes, that is client side circuit breaker. What Aruna is implementing
>>>>>> is server side circuit breaker. Yes, we need both.
>>>>>>
>>>>>> On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana <
>>>>>> lak...@wso2.com> wrote:
>>>>>>
>>>>>>> Did you looked at [1]
>>>>>>>
>>>>>>> Netflix Hystrix <https://github.com/Netflix/Hystrix> is an
>>>>>>> incredibly useful library for writing code that invokes remote services.
>>>>>>> Hystrix times out calls that exceed the specified threshold. It 
>>>>>>> implements
>>>>>>> a *circuit breaker* pattern, which stops the client from waiting
>>>>>>> needlessly for an unresponsive service. If the error rate for a service
>>>>>>> exceeds a specified threshold, Hystrix trips the circuit breaker and all
>>>>>>> requests will fail immediately for a specified period of time. Hystrix 
>>>>>>> lets
>>>>>>> you define a fallback action when a request fails, such as reading from 
>>>>>>> a
>>>>>>> cache or returning a default value. If you are using the JVM you should
>>>>>>> definitely consider using Hystrix.
>>>>>>>
>>>>>>> [1] https://github.com/Netflix/Hystrix
>>>>>>>
>>>>>>> On Fri, Mar 11, 2016 at 2:42 PM, Aruna Karunarathna <ar...@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Devs,
>>>>>>>>
>>>>>>>> *Scenario*
>>>>>>>>
>>>>>>>> The deployed services in 

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

We are not leaving out anything. We are in the process of implementing
circuit breaker, bulkhead, timeout. Basically, core features people look
for when evaluating microservices frameworks. We have started with circuit
breaker.

Thanks
Azeez

On Sun, Mar 13, 2016 at 6:57 PM, Frank Leymann <fr...@wso2.com> wrote:

> I do have a more general question: what justifies the focus on the
> "circuit breaker" pattern at all? It is just one patter to solve recurring
> problems with stability, i.e. other patterns are there too that are
> important (e.g. "timeout" - see Michael Nygard's nice book).
>
> Thus, what are the criteria that guide our decisions in selecting some
> patterns for implementations, some others for leaving out?
>
>
> Best regards,
> Frank
>
> 2016-03-12 8:26 GMT+01:00 Sanjiva Weerawarana <sanj...@wso2.com>:
>
>> I don't understand what server side circuit breaker means. How does the
>> server adjust itself? Where's that bit of logic running?
>>
>> IMO this is not needed in a container world.
>>
>> On Fri, Mar 11, 2016 at 4:38 PM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> Yes, that is client side circuit breaker. What Aruna is implementing is
>>> server side circuit breaker. Yes, we need both.
>>>
>>> On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana <lak...@wso2.com>
>>> wrote:
>>>
>>>> Did you looked at [1]
>>>>
>>>> Netflix Hystrix <https://github.com/Netflix/Hystrix> is an incredibly
>>>> useful library for writing code that invokes remote services. Hystrix times
>>>> out calls that exceed the specified threshold. It implements a *circuit
>>>> breaker* pattern, which stops the client from waiting needlessly for
>>>> an unresponsive service. If the error rate for a service exceeds a
>>>> specified threshold, Hystrix trips the circuit breaker and all requests
>>>> will fail immediately for a specified period of time. Hystrix lets you
>>>> define a fallback action when a request fails, such as reading from a cache
>>>> or returning a default value. If you are using the JVM you should
>>>> definitely consider using Hystrix.
>>>>
>>>> [1] https://github.com/Netflix/Hystrix
>>>>
>>>> On Fri, Mar 11, 2016 at 2:42 PM, Aruna Karunarathna <ar...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Devs,
>>>>>
>>>>> *Scenario*
>>>>>
>>>>> The deployed services in a MSF4J may fail to serve the requests due to
>>>>> various factors. e.g,
>>>>> 1. Less resources in the server.
>>>>> 2. High Load in the server
>>>>> 3, Some services take more time to respond etc.
>>>>>
>>>>> In this kind of a situation, if the server is getting requests though
>>>>> there is no resources to serve those requests, and eventually the server
>>>>> will get unusable.
>>>>>
>>>>> *Solution*
>>>>>
>>>>> The Circuit Breaker design pattern can save the server from above
>>>>> scenarios, The typical design can be illustrated as in the following
>>>>> diagram.
>>>>>
>>>>>
>>>>> So as in the above diagram, when number of failures of a particular
>>>>> resource exceeds the Max Failure Count, then the state of that resource is
>>>>> moved to the open state with a timeout value (Trip Breaker). At this point
>>>>> the requests coming to the server is routed back without passing the
>>>>> internal to process further.
>>>>>
>>>>> After the timeout has reached, the state is moved to Half-Open state,
>>>>> and if the consecutive request pass to the server to process (Attempt
>>>>> Reset), if success then close the circuit (Reset Breaker), If fail then
>>>>> again move the state to the Open with a timeout value (Trip Breaker).
>>>>>
>>>>> Any thoughts, suggestions regarding the above approach?
>>>>>
>>>>> References
>>>>> [1].
>>>>> http://www.javaworld.com/article/2824163/application-performance/stability-patterns-applied-in-a-restful-architecture.html?page=2
>>>>> [2].
>>>>> http://ssagara.blogspot.com/2015/05/timeout-and-circuit-breaker-pattern-in.html
>>>>> [3]. https://pragprog.com/book/mnee/release-it
>>>>>
>>>>>
>>

Re: [Architecture] The new disruptor based Netty transport is not working well for MSF4J

[Afkham Azeez]

Kasun et al,
In MSF4J, the threads from the disruptor pool itself are used for
processing in the MSF4J operation. In the case of the GW passthrough & HBR
scenarios, are those disruptor threads used to make the calls to the actual
backends? Is that a blocking call? What if the backend service is changed
so that it responds after a delay rather than instantaneously?

On Sat, Mar 12, 2016 at 6:21 PM, Afkham Azeez <az...@wso2.com> wrote:

>
>
> On Sat, Mar 12, 2016 at 1:40 PM, Sanjiva Weerawarana <sanj...@wso2.com>
> wrote:
>
>> On Thu, Mar 10, 2016 at 6:20 PM, Sagara Gunathunga <sag...@wso2.com>
>> wrote:
>>
>>> On Thu, Mar 10, 2016 at 10:26 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> No from day 1, we have decided that GW & MSF4J will use the same Netty
>>>> transport component so that the config file will be the same as well as
>>>> improvements made to that transport will be automatically available for
>>>> both products. So now at least for MSF4J, we have issues in using the Netty
>>>> transport in its current state, so we have to fix those issues.
>>>>
>>>
>>> Reuse of same config files and components provide an advantage to us as
>>> the F/W developers/maintainers but my question was what are the benefits
>>> grant to end users of MSF4J through Carbon transport ?
>>>
>>
>> We are writing MSF4J and the rest of the platform. Not someone else. As
>> such we have to keep them consistent.
>>
>> For end users our target has to be to give the best performance possible.
>>
>>
>>>   I don't think we can compromise performance numbers for a reason that
>>> is more important for F/W maintainers than end users, IMHO if we continue
>>> to use Carbon transport at least it should perform as same level as vanilla
>>> Netty.
>>>
>>
>> There's no reason why that cannot be the case.
>>
>> Can't we keep Disruptor while improve performance of Carbon transport ?
>>>
>>
>> Disruptor is a technique to make things more performant not less
>> performant. We have to figure out what's wrong and fix it - not throw the
>> baby out with the bathwater.
>>
>
> Yes, we are in the process of trying to figure out why disruptor as
> opposed to the Netty executor threadpool gives better performance for the
> gateway (dispatching to a zero delay backend), while for an MSF4J service
> which has a sleep in it, it is the other way around.
>
>
>>
>> Sanjiva.
>> --
>> Sanjiva Weerawarana, Ph.D.
>> Founder, CEO & Chief Architect; WSO2, Inc.;  http://wso2.com/
>> email: sanj...@wso2.com; office: (+1 650 745 4499 | +94  11 214 5345)
>> x5700; cell: +94 77 787 6880 | +1 408 466 5099; voip: +1 650 265 8311
>> blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva
>> Lean . Enterprise . Middleware
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
> *Afkham Azeez*
> Director of Architecture; WSO2, Inc.; http://wso2.com
> Member; Apache Software Foundation; http://www.apache.org/
> * <http://www.apache.org/>*
> *email: **az...@wso2.com* <az...@wso2.com>
> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
> *http://blog.afkham.org* <http://blog.afkham.org>
> *twitter: **http://twitter.com/afkham_azeez*
> <http://twitter.com/afkham_azeez>
> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
> <http://lk.linkedin.com/in/afkhamazeez>*
>
> *Lean . Enterprise . Middleware*
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Session Affinity Alternatives

[Afkham Azeez]

Of course the simplest solution is similar to what Tomcat does, local
sessions (no replication) & in a cluster, have session affinity configured
at the load balancer, so that should be the default. If the node that had
the session dies, the clients connected to that instance would get errors
or have to login again. For HA deployments, we would need session
replication or session persistence.

On Sat, Mar 12, 2016 at 12:58 PM, Sanjiva Weerawarana <sanj...@wso2.com>
wrote:

> Azeez we cannot have a model where every simple server (cluster)
> deployment requires Redis.
>
> Please indicate what you think the right solution is .. its not clear to
> me.
>
> On Thu, Mar 10, 2016 at 7:34 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> Storing everything as cookies may not work always  and there could be
>> sensitive runtime data that you don't want to save on the browser. In
>> addition, when it comes to Java programming models, using the HTTP session
>> to store serializable objects is the natural way of programming. Yes, your
>> solution would work for certain cases, but it doesn't cover all cases.
>>
>> On Thu, Mar 10, 2016 at 6:48 PM, Joseph Fonseka <jos...@wso2.com> wrote:
>>
>>> I think we should go with 3 to keep things simple.
>>>
>>> To solve HA problem ( without session persistence or replication ).
>>>
>>> 1. Use SSO to authenticate the user.
>>> 2. Use the session to store the claims return from IdP. ( Ex user_id,
>>> roles )
>>> 3. DO NOT store app specific data on the session instead use cookies,
>>> local storage in the browser.
>>> 4. In case the container get terminated and user get redirected to
>>> another container it will initiate a SSO flow and repopulate a new session
>>> with user claims. Then the app can continue as normal.
>>>
>>> WDYT?
>>>
>>> Regards
>>> Jo
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Mar 10, 2016 at 2:21 PM, Lakmal Warusawithana <lak...@wso2.com>
>>> wrote:
>>>
>>>> My order of preference - 3, 2.
>>>>
>>>> For simple deployment, session affinity work fine. But if we want to
>>>> deploy large distributed deployment with HA, we need to go for option 2.
>>>>
>>>> On Thu, Mar 10, 2016 at 10:41 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>>
>>>>> I forgot to add earlier our design decision about using Redis to store
>>>>> sessions; the Kubernetes scheduler may decide to kill & container & start
>>>>> up different instance if its health checks detects problems. So in such a
>>>>> case, if we had used affinity, the clients connected to that instance 
>>>>> which
>>>>> was killed will lose their session data. So as a best practice they
>>>>> recommend using an external service with session persistence. Of course
>>>>> this is not the simplest case, so yes, the default should be local 
>>>>> sessions
>>>>> with affinity.
>>>>>
>>>>> On Thu, Mar 10, 2016 at 10:23 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>
>>>>>> Petstore is #2. We use the Redis service to store the session. For an
>>>>>> HA deployment such a model is required, but yes, for the simplest case, 
>>>>>> we
>>>>>> can have local sessions and then use session affinity capabilities of the
>>>>>> LB.
>>>>>>
>>>>>> On Thu, Mar 10, 2016 at 10:17 AM, Sanjiva Weerawarana <
>>>>>> sanj...@wso2.com> wrote:
>>>>>>
>>>>>>> Manu, #1 is not a no-session story. What Azeez has done for the
>>>>>>> petstore is a model where session state is in a DB.
>>>>>>>
>>>>>>> Session as a service is the same thing ... basically a data service
>>>>>>> in front of a DB.
>>>>>>>
>>>>>>> So really the basic question is can you do without a session? My
>>>>>>> answer is no, not practical. If you go full HATEOS you can do without
>>>>>>> sessions but even then you have to re-authenticate every call which is 
>>>>>>> not
>>>>>>> practical.
>>>>>>>
>>>>>>> So its #3 :-).
>>>>>>>
>>>>>>> On Wed, Mar 9, 2016 at 8:01 PM, Manuranga Perera <m...@wso2.com>
>&g

Re: [Architecture] The new disruptor based Netty transport is not working well for MSF4J

[Afkham Azeez]

On Sat, Mar 12, 2016 at 1:40 PM, Sanjiva Weerawarana <sanj...@wso2.com>
wrote:

> On Thu, Mar 10, 2016 at 6:20 PM, Sagara Gunathunga <sag...@wso2.com>
> wrote:
>
>> On Thu, Mar 10, 2016 at 10:26 AM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> No from day 1, we have decided that GW & MSF4J will use the same Netty
>>> transport component so that the config file will be the same as well as
>>> improvements made to that transport will be automatically available for
>>> both products. So now at least for MSF4J, we have issues in using the Netty
>>> transport in its current state, so we have to fix those issues.
>>>
>>
>> Reuse of same config files and components provide an advantage to us as
>> the F/W developers/maintainers but my question was what are the benefits
>> grant to end users of MSF4J through Carbon transport ?
>>
>
> We are writing MSF4J and the rest of the platform. Not someone else. As
> such we have to keep them consistent.
>
> For end users our target has to be to give the best performance possible.
>
>
>>   I don't think we can compromise performance numbers for a reason that
>> is more important for F/W maintainers than end users, IMHO if we continue
>> to use Carbon transport at least it should perform as same level as vanilla
>> Netty.
>>
>
> There's no reason why that cannot be the case.
>
> Can't we keep Disruptor while improve performance of Carbon transport ?
>>
>
> Disruptor is a technique to make things more performant not less
> performant. We have to figure out what's wrong and fix it - not throw the
> baby out with the bathwater.
>

Yes, we are in the process of trying to figure out why disruptor as opposed
to the Netty executor threadpool gives better performance for the gateway
(dispatching to a zero delay backend), while for an MSF4J service which has
a sleep in it, it is the other way around.


>
> Sanjiva.
> --
> Sanjiva Weerawarana, Ph.D.
> Founder, CEO & Chief Architect; WSO2, Inc.;  http://wso2.com/
> email: sanj...@wso2.com; office: (+1 650 745 4499 | +94  11 214 5345)
> x5700; cell: +94 77 787 6880 | +1 408 466 5099; voip: +1 650 265 8311
> blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva
> Lean . Enterprise . Middleware
>
> ___
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

Isabelle & Sanjiva,
http://www.javaworld.com/article/2824163/application-performance/stability-patterns-applied-in-a-restful-architecture.html?page=2
which Aruna had given as a reference explains circuit breaker on the server
side. IMO, circuit breaker on both client side & server side are useful
features in a microservices framework. In fact these are features people
look for when evaluating microservices frameworks.

Thanks
Azeez

On Sat, Mar 12, 2016 at 5:11 PM, Isabelle Mauny <isabe...@wso2.com> wrote:

> If an API GW is used to access the microservices , then the circuit
> breaker pattern would apply at that level, right ?  If the client is a web
> app directly calling MS (not that this would be a good pattern at all !)
> then the client is the web app. In any case, they are all clients calling
> the microservices. So I am not sure about server side either..
>
>
> -
> *Isabelle Mauny*
> VP, Product Management - WSO2, Inc. - http://wso2.com/
>
> On Sat, Mar 12, 2016 at 8:26 AM, Sanjiva Weerawarana <sanj...@wso2.com>
> wrote:
>
>> I don't understand what server side circuit breaker means. How does the
>> server adjust itself? Where's that bit of logic running?
>>
>> IMO this is not needed in a container world.
>>
>> On Fri, Mar 11, 2016 at 4:38 PM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> Yes, that is client side circuit breaker. What Aruna is implementing is
>>> server side circuit breaker. Yes, we need both.
>>>
>>> On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana <lak...@wso2.com>
>>> wrote:
>>>
>>>> Did you looked at [1]
>>>>
>>>> Netflix Hystrix <https://github.com/Netflix/Hystrix> is an incredibly
>>>> useful library for writing code that invokes remote services. Hystrix times
>>>> out calls that exceed the specified threshold. It implements a *circuit
>>>> breaker* pattern, which stops the client from waiting needlessly for
>>>> an unresponsive service. If the error rate for a service exceeds a
>>>> specified threshold, Hystrix trips the circuit breaker and all requests
>>>> will fail immediately for a specified period of time. Hystrix lets you
>>>> define a fallback action when a request fails, such as reading from a cache
>>>> or returning a default value. If you are using the JVM you should
>>>> definitely consider using Hystrix.
>>>>
>>>> [1] https://github.com/Netflix/Hystrix
>>>>
>>>> On Fri, Mar 11, 2016 at 2:42 PM, Aruna Karunarathna <ar...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Devs,
>>>>>
>>>>> *Scenario*
>>>>>
>>>>> The deployed services in a MSF4J may fail to serve the requests due to
>>>>> various factors. e.g,
>>>>> 1. Less resources in the server.
>>>>> 2. High Load in the server
>>>>> 3, Some services take more time to respond etc.
>>>>>
>>>>> In this kind of a situation, if the server is getting requests though
>>>>> there is no resources to serve those requests, and eventually the server
>>>>> will get unusable.
>>>>>
>>>>> *Solution*
>>>>>
>>>>> The Circuit Breaker design pattern can save the server from above
>>>>> scenarios, The typical design can be illustrated as in the following
>>>>> diagram.
>>>>>
>>>>>
>>>>> So as in the above diagram, when number of failures of a particular
>>>>> resource exceeds the Max Failure Count, then the state of that resource is
>>>>> moved to the open state with a timeout value (Trip Breaker). At this point
>>>>> the requests coming to the server is routed back without passing the
>>>>> internal to process further.
>>>>>
>>>>> After the timeout has reached, the state is moved to Half-Open state,
>>>>> and if the consecutive request pass to the server to process (Attempt
>>>>> Reset), if success then close the circuit (Reset Breaker), If fail then
>>>>> again move the state to the Open with a timeout value (Trip Breaker).
>>>>>
>>>>> Any thoughts, suggestions regarding the above approach?
>>>>>
>>>>> References
>>>>> [1].
>>>>> http://www.javaworld.com/article/2824163/application-performance/stability-patterns-applied-in

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

This is a feature supported by some microservices frameworks. On the server
side, in this case MSF4J runtime, failure counts are kept track of and then
if the failures exceed certain thresholds, the circuit trips and rather
than dispatch to the service, it returns service unavailable.

Can you explain why this is not needed in a container environment?

On Sat, Mar 12, 2016 at 12:56 PM, Sanjiva Weerawarana <sanj...@wso2.com>
wrote:

> I don't understand what server side circuit breaker means. How does the
> server adjust itself? Where's that bit of logic running?
>
> IMO this is not needed in a container world.
>
> On Fri, Mar 11, 2016 at 4:38 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> Yes, that is client side circuit breaker. What Aruna is implementing is
>> server side circuit breaker. Yes, we need both.
>>
>> On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana <lak...@wso2.com>
>> wrote:
>>
>>> Did you looked at [1]
>>>
>>> Netflix Hystrix <https://github.com/Netflix/Hystrix> is an incredibly
>>> useful library for writing code that invokes remote services. Hystrix times
>>> out calls that exceed the specified threshold. It implements a *circuit
>>> breaker* pattern, which stops the client from waiting needlessly for an
>>> unresponsive service. If the error rate for a service exceeds a specified
>>> threshold, Hystrix trips the circuit breaker and all requests will fail
>>> immediately for a specified period of time. Hystrix lets you define a
>>> fallback action when a request fails, such as reading from a cache or
>>> returning a default value. If you are using the JVM you should definitely
>>> consider using Hystrix.
>>>
>>> [1] https://github.com/Netflix/Hystrix
>>>
>>> On Fri, Mar 11, 2016 at 2:42 PM, Aruna Karunarathna <ar...@wso2.com>
>>> wrote:
>>>
>>>> Hi Devs,
>>>>
>>>> *Scenario*
>>>>
>>>> The deployed services in a MSF4J may fail to serve the requests due to
>>>> various factors. e.g,
>>>> 1. Less resources in the server.
>>>> 2. High Load in the server
>>>> 3, Some services take more time to respond etc.
>>>>
>>>> In this kind of a situation, if the server is getting requests though
>>>> there is no resources to serve those requests, and eventually the server
>>>> will get unusable.
>>>>
>>>> *Solution*
>>>>
>>>> The Circuit Breaker design pattern can save the server from above
>>>> scenarios, The typical design can be illustrated as in the following
>>>> diagram.
>>>>
>>>>
>>>> So as in the above diagram, when number of failures of a particular
>>>> resource exceeds the Max Failure Count, then the state of that resource is
>>>> moved to the open state with a timeout value (Trip Breaker). At this point
>>>> the requests coming to the server is routed back without passing the
>>>> internal to process further.
>>>>
>>>> After the timeout has reached, the state is moved to Half-Open state,
>>>> and if the consecutive request pass to the server to process (Attempt
>>>> Reset), if success then close the circuit (Reset Breaker), If fail then
>>>> again move the state to the Open with a timeout value (Trip Breaker).
>>>>
>>>> Any thoughts, suggestions regarding the above approach?
>>>>
>>>> References
>>>> [1].
>>>> http://www.javaworld.com/article/2824163/application-performance/stability-patterns-applied-in-a-restful-architecture.html?page=2
>>>> [2].
>>>> http://ssagara.blogspot.com/2015/05/timeout-and-circuit-breaker-pattern-in.html
>>>> [3]. https://pragprog.com/book/mnee/release-it
>>>>
>>>>
>>>> Regards,
>>>> Aruna
>>>> --
>>>>
>>>> *Aruna Sujith Karunarathna *
>>>> WSO2, Inc | lean. enterprise. middleware.
>>>> #20, Palm Grove, Colombo 03, Sri Lanka
>>>> Mobile: +94 71 9040362 | Work: +94 112145345
>>>> Email: ar...@wso2.com | Web: www.wso2.com
>>>>
>>>>
>>>> ___
>>>> Architecture mailing list
>>>> Architecture@wso2.org
>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>
>>>>
>>>
>>>
>>> --
>>> Lakmal Warusawithana
>>> Director - Cloud Ar

Re: [Architecture] [MSF4J][Improvement] Intercceptors as an annotation

[Afkham Azeez]

In the annotation approach, the class level interceptors will be invoked
first in the specified order, and then the method level interceptors will
be invoked in the specified order.

On Fri, Mar 11, 2016 at 3:22 PM, Samiyuru Senarathne <samiy...@wso2.com>
wrote:

> +1 This provides more control.
> We should also define a way of specifying the order of interceptor
> execution.
>
> If we are gonna promote lambda handlers we can provide something like:
>
> new MSF4JRunner()
> .addInterceptor("/path/with/wild/cards/and/path/params", new
> Interceptor())
> .deploy(lambdaHandlers)
> .start()
>
>
>
> On Fri, Mar 11, 2016 at 2:29 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> It would be elegant to specify the interceptors as annotations rather
>> than programatically adding them.
>>
>> e.g. 1
>> Apply interceptors globally to all operations in a service
>>
>> @Path("/foo")
>> @Interceptor(classes = {"org.wso2.msf4j.OAuthInterceptor",
>> "org.wso2.msf4j.LogInterceptor"})
>> public class FooService {
>>
>> }
>>
>> e.g.2
>> Apply interceptors selectively to operations
>>
>> @Path("/foo")
>> @Interceptor(classes = {"org.wso2.msf4j.LogInterceptor"})
>> public class FooService {
>>
>>@Path("secure")
>>@Interceptor(classes={"org.wso2.msf4j.OAuthInterceptor"})
>>public void secureOp(){}
>>
>>@Path("unsecure")
>>public void unsecureOp{}
>> }
>>
>> Thoughts welcome. This would simplify the programming model for users of
>> MSF4J.
>>
>>
>> --
>> *Afkham Azeez*
>> Director of Architecture; WSO2, Inc.; http://wso2.com
>> Member; Apache Software Foundation; http://www.apache.org/
>> * <http://www.apache.org/>*
>> *email: **az...@wso2.com* <az...@wso2.com>
>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>> *http://blog.afkham.org* <http://blog.afkham.org>
>> *twitter: **http://twitter.com/afkham_azeez*
>> <http://twitter.com/afkham_azeez>
>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>> <http://lk.linkedin.com/in/afkhamazeez>*
>>
>> *Lean . Enterprise . Middleware*
>>
>
>
>
> --
> Samiyuru Senarathne
> *Software Engineer*
> Mobile : +94 (0) 71 134 6087
> samiy...@wso2.com
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Circuit Breaker Pattern for MSF4J

[Afkham Azeez]

Yes, that is client side circuit breaker. What Aruna is implementing is
server side circuit breaker. Yes, we need both.

On Fri, Mar 11, 2016 at 4:04 PM, Lakmal Warusawithana <lak...@wso2.com>
wrote:

> Did you looked at [1]
>
> Netflix Hystrix <https://github.com/Netflix/Hystrix> is an incredibly
> useful library for writing code that invokes remote services. Hystrix times
> out calls that exceed the specified threshold. It implements a *circuit
> breaker* pattern, which stops the client from waiting needlessly for an
> unresponsive service. If the error rate for a service exceeds a specified
> threshold, Hystrix trips the circuit breaker and all requests will fail
> immediately for a specified period of time. Hystrix lets you define a
> fallback action when a request fails, such as reading from a cache or
> returning a default value. If you are using the JVM you should definitely
> consider using Hystrix.
>
> [1] https://github.com/Netflix/Hystrix
>
> On Fri, Mar 11, 2016 at 2:42 PM, Aruna Karunarathna <ar...@wso2.com>
> wrote:
>
>> Hi Devs,
>>
>> *Scenario*
>>
>> The deployed services in a MSF4J may fail to serve the requests due to
>> various factors. e.g,
>> 1. Less resources in the server.
>> 2. High Load in the server
>> 3, Some services take more time to respond etc.
>>
>> In this kind of a situation, if the server is getting requests though
>> there is no resources to serve those requests, and eventually the server
>> will get unusable.
>>
>> *Solution*
>>
>> The Circuit Breaker design pattern can save the server from above
>> scenarios, The typical design can be illustrated as in the following
>> diagram.
>>
>>
>> So as in the above diagram, when number of failures of a particular
>> resource exceeds the Max Failure Count, then the state of that resource is
>> moved to the open state with a timeout value (Trip Breaker). At this point
>> the requests coming to the server is routed back without passing the
>> internal to process further.
>>
>> After the timeout has reached, the state is moved to Half-Open state, and
>> if the consecutive request pass to the server to process (Attempt Reset),
>> if success then close the circuit (Reset Breaker), If fail then again move
>> the state to the Open with a timeout value (Trip Breaker).
>>
>> Any thoughts, suggestions regarding the above approach?
>>
>> References
>> [1].
>> http://www.javaworld.com/article/2824163/application-performance/stability-patterns-applied-in-a-restful-architecture.html?page=2
>> [2].
>> http://ssagara.blogspot.com/2015/05/timeout-and-circuit-breaker-pattern-in.html
>> [3]. https://pragprog.com/book/mnee/release-it
>>
>>
>> Regards,
>> Aruna
>> --
>>
>> *Aruna Sujith Karunarathna *
>> WSO2, Inc | lean. enterprise. middleware.
>> #20, Palm Grove, Colombo 03, Sri Lanka
>> Mobile: +94 71 9040362 | Work: +94 112145345
>> Email: ar...@wso2.com | Web: www.wso2.com
>>
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

[Architecture] [MSF4J][Improvement] Intercceptors as an annotation

[Afkham Azeez]

It would be elegant to specify the interceptors as annotations rather than
programatically adding them.

e.g. 1
Apply interceptors globally to all operations in a service

@Path("/foo")
@Interceptor(classes = {"org.wso2.msf4j.OAuthInterceptor",
"org.wso2.msf4j.LogInterceptor"})
public class FooService {

}

e.g.2
Apply interceptors selectively to operations

@Path("/foo")
@Interceptor(classes = {"org.wso2.msf4j.LogInterceptor"})
public class FooService {

   @Path("secure")
   @Interceptor(classes={"org.wso2.msf4j.OAuthInterceptor"})
   public void secureOp(){}

   @Path("unsecure")
   public void unsecureOp{}
}

Thoughts welcome. This would simplify the programming model for users of
MSF4J.


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] The new disruptor based Netty transport is not working well for MSF4J

[Afkham Azeez]

Shall we aim to get to the bottom of this by EoD today?

On Fri, Mar 11, 2016 at 11:44 AM, Samiyuru Senarathne <samiy...@wso2.com>
wrote:

> Transport config used for the above JFR:
>  -
>   id: "jaxrs-http"
>   host: "0.0.0.0"
>   port: 8080
>   bossThreadPoolSize: 1
>   workerThreadPoolSize: 16
>   parameters:
> #   -
> #name: "execThreadPoolSize"
> #value: 2048
>-
> name: "disruptor.buffer.size"
> value: 1024
>-
> name: "disruptor.count"
> value: 5
>-
> name: "disruptor.eventhandler.count"
> value: 256
> #   -
> #name: "disruptor.wait.strategy"
> #value: "SLEEP_WAITING"
>-
> name: "share.disruptor.with.outbound"
> value: false
>-
> name: "disruptor.consumer.external.worker.pool.size"
> value: 256
>
>
> On Fri, Mar 11, 2016 at 11:41 AM, Samiyuru Senarathne <samiy...@wso2.com>
> wrote:
>
>> Please find the attached JFR dump of MSF4J-carbon-message-disruptor test
>> performed with following 'ab' config.
>> An MSF4J service that 'consumes a 1KB and sleeps for 50ms before
>> responding the same 1KB' is used as the service.
>>
>> ab -k -p 1kb_rand_data.txt -s 999 -c 400 -n 5000 -H "Accept:text/plain"
>> http://204.13.85.2:8080/EchoService/echo
>>
>> AB Results Summary:
>> Concurrency Level:  400
>> Time taken for tests:   30.224 seconds
>> Complete requests:  3000
>> Failed requests:0
>> Keep-Alive requests:3000
>> Total transferred:  3345000 bytes
>> Total body sent:3609000
>> HTML transferred:   3072000 bytes
>> Requests per second:99.26 [#/sec] (mean)
>> Time per request:   4029.882 [ms] (mean)
>> Time per request:   10.075 [ms] (mean, across all concurrent requests)
>> Transfer rate:  108.08 [Kbytes/sec] received
>> 116.61 kb/s sent
>> 224.69 kb/s total
>>
>>
>>
>> On Fri, Mar 11, 2016 at 10:39 AM, Samiyuru Senarathne <samiy...@wso2.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Please find results of the tests I have done so far.
>>>
>>> https://docs.google.com/a/wso2.com/spreadsheets/d/16TXeXU022b5ILqkRsY3zZdnu2OiA3yWEN42xVL8eXa4/edit?usp=sharing
>>>
>>> MSF4J 1.0.0 section of this tests gives a good insight into the netty
>>> thread behaviour. I focused a bit more on that because confirming the
>>> practical behaviour of netty thread model is one of the first things we
>>> should do.
>>>
>>> According to these results,
>>>
>>>- Increasing the netty boss pool does not make any difference.
>>>- For the netty worker pool, it's enough to have a number of threads
>>>equal to the number of cpus.
>>>- Increasing the number of threads of the pool of the netty handler
>>>that does the actual work increases the performance significantly for 
>>> high
>>>    concurrency levels.
>>>
>>> Regarding the tests that include disruptor,
>>> I applied the optimal configurations according to [1]. But I could not
>>> get results even close to the results of GW and the results I got are bit
>>> strange (99 TPS regardless of concurrency level). I think we should try
>>> more variations of disruptor settings before coming to a conclusion.
>>>
>>> [1] -
>>> https://docs.google.com/a/wso2.com/spreadsheets/d/1ck2O7eMkswJSQCgW8ciOkIyZkXGiZhIxRWNN-R9vgMk/edit?usp=sharing
>>>
>>> Best Regards,
>>> Samiyuru
>>>
>>> On Fri, Mar 11, 2016 at 10:06 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> I think Samiyuru tested with that nrw worker pool & still the
>>>> performance is unacceptable.
>>>> On Mar 11, 2016 9:45 AM, "Isuru Ranawaka" <isu...@wso2.com> wrote:
>>>>
>>>>> Hi ,
>>>>>
>>>>> We have already added  native worker pool for Disruptor and Samiyuru
>>>>> is doing testing on that. We will make disruptor optional as well.
>>>>>
>>>>> thanks
>>>>>
>>>>> On Thu, Mar 10, 2016 at 9:29 AM, Kasun Indrasiri <ka...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Yes, we can make the disruptor optional. Also, we should try using
>>>>>> the native worker pool for Event Handler[1

Re: [Architecture] The new disruptor based Netty transport is not working well for MSF4J

[Afkham Azeez]

I think Samiyuru tested with that nrw worker pool & still the performance
is unacceptable.
On Mar 11, 2016 9:45 AM, "Isuru Ranawaka" <isu...@wso2.com> wrote:

> Hi ,
>
> We have already added  native worker pool for Disruptor and Samiyuru is
> doing testing on that. We will make disruptor optional as well.
>
> thanks
>
> On Thu, Mar 10, 2016 at 9:29 AM, Kasun Indrasiri <ka...@wso2.com> wrote:
>
>> Yes, we can make the disruptor optional. Also, we should try using the
>> native worker pool for Event Handler[1], so that the Disruptor itself runs
>> the event handler on a worker pool. We'll implement both approaches and do
>> a comparison.
>>
>> [1]
>> https://lmax-exchange.github.io/disruptor/docs/com/lmax/disruptor/dsl/Disruptor.html#handleEventsWithWorkerPool(com.lmax.disruptor.WorkHandler..
>> .)
>>
>> On Thu, Mar 10, 2016 at 8:48 AM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> After upgrading to the new transport, we are seeing a significant drop
>>> in performance for any service that take some time to execute. We have
>>> tried with the configuration used for the gateway which gave the best
>>> figures on the same hardware. We have also noted that using a separate
>>> dedicated executor thread pool, which is supported by Netty, gave much
>>> better performance than the disruptor based implementation. Even in theory,
>>> disruptor cannot give better performance when used with a real service that
>>> does some real work, rather than doing passthrough, for example. Can we
>>> improve the Netty transport to make going through disruptor optional?
>>>
>>> --
>>> *Afkham Azeez*
>>> Director of Architecture; WSO2, Inc.; http://wso2.com
>>> Member; Apache Software Foundation; http://www.apache.org/
>>> * <http://www.apache.org/>*
>>> *email: **az...@wso2.com* <az...@wso2.com>
>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>>> *http://blog.afkham.org* <http://blog.afkham.org>
>>> *twitter: **http://twitter.com/afkham_azeez*
>>> <http://twitter.com/afkham_azeez>
>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>>> <http://lk.linkedin.com/in/afkhamazeez>*
>>>
>>> *Lean . Enterprise . Middleware*
>>>
>>
>>
>>
>> --
>> Kasun Indrasiri
>> Software Architect
>> WSO2, Inc.; http://wso2.com
>> lean.enterprise.middleware
>>
>> cell: +94 77 556 5206
>> Blog : http://kasunpanorama.blogspot.com/
>>
>
>
>
> --
> Best Regards
> Isuru Ranawaka
> M: +94714629880
> Blog : http://isurur.blogspot.com/
>
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Session Affinity Alternatives

[Afkham Azeez]

Storing everything as cookies may not work always  and there could be
sensitive runtime data that you don't want to save on the browser. In
addition, when it comes to Java programming models, using the HTTP session
to store serializable objects is the natural way of programming. Yes, your
solution would work for certain cases, but it doesn't cover all cases.

On Thu, Mar 10, 2016 at 6:48 PM, Joseph Fonseka <jos...@wso2.com> wrote:

> I think we should go with 3 to keep things simple.
>
> To solve HA problem ( without session persistence or replication ).
>
> 1. Use SSO to authenticate the user.
> 2. Use the session to store the claims return from IdP. ( Ex user_id,
> roles )
> 3. DO NOT store app specific data on the session instead use cookies,
> local storage in the browser.
> 4. In case the container get terminated and user get redirected to another
> container it will initiate a SSO flow and repopulate a new session with
> user claims. Then the app can continue as normal.
>
> WDYT?
>
> Regards
> Jo
>
>
>
>
>
>
> On Thu, Mar 10, 2016 at 2:21 PM, Lakmal Warusawithana <lak...@wso2.com>
> wrote:
>
>> My order of preference - 3, 2.
>>
>> For simple deployment, session affinity work fine. But if we want to
>> deploy large distributed deployment with HA, we need to go for option 2.
>>
>> On Thu, Mar 10, 2016 at 10:41 AM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> I forgot to add earlier our design decision about using Redis to store
>>> sessions; the Kubernetes scheduler may decide to kill & container & start
>>> up different instance if its health checks detects problems. So in such a
>>> case, if we had used affinity, the clients connected to that instance which
>>> was killed will lose their session data. So as a best practice they
>>> recommend using an external service with session persistence. Of course
>>> this is not the simplest case, so yes, the default should be local sessions
>>> with affinity.
>>>
>>> On Thu, Mar 10, 2016 at 10:23 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> Petstore is #2. We use the Redis service to store the session. For an
>>>> HA deployment such a model is required, but yes, for the simplest case, we
>>>> can have local sessions and then use session affinity capabilities of the
>>>> LB.
>>>>
>>>> On Thu, Mar 10, 2016 at 10:17 AM, Sanjiva Weerawarana <sanj...@wso2.com
>>>> > wrote:
>>>>
>>>>> Manu, #1 is not a no-session story. What Azeez has done for the
>>>>> petstore is a model where session state is in a DB.
>>>>>
>>>>> Session as a service is the same thing ... basically a data service in
>>>>> front of a DB.
>>>>>
>>>>> So really the basic question is can you do without a session? My
>>>>> answer is no, not practical. If you go full HATEOS you can do without
>>>>> sessions but even then you have to re-authenticate every call which is not
>>>>> practical.
>>>>>
>>>>> So its #3 :-).
>>>>>
>>>>> On Wed, Mar 9, 2016 at 8:01 PM, Manuranga Perera <m...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Options
>>>>>> 1) No session. Everything is in DB or Window.localStorage.
>>>>>> Authentication via a token validation endpoint. (We keep the token in a
>>>>>> front end cookie)
>>>>>> 2) Session as a service
>>>>>> 3) The session is local, works with session affinity
>>>>>> 4) The session is distributed
>>>>>>
>>>>>> My personal order of preference - 1, 2, 3, 4
>>>>>> Azeez says 2 (or 1? )
>>>>>> Sanjiva says 3, with 4 being plug-able
>>>>>>
>>>>>> I think 1 is doable.
>>>>>> Yes, there will be some development overhead.
>>>>>> But it'll be scalable/simpler at run time.
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 9, 2016 at 6:59 PM, Sanjiva Weerawarana <sanj...@wso2.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Not practical Azeez - you're massively complicating the deployment
>>>>>>> and second its far less performant than replication. Earlier we did 
>>>>>>> global
>>>>>>> replication which we really shouldn't do.
>>>>>>>
>>>>>>>

Re: [Architecture] Session Affinity Alternatives

[Afkham Azeez]

I forgot to add earlier our design decision about using Redis to store
sessions; the Kubernetes scheduler may decide to kill & container & start
up different instance if its health checks detects problems. So in such a
case, if we had used affinity, the clients connected to that instance which
was killed will lose their session data. So as a best practice they
recommend using an external service with session persistence. Of course
this is not the simplest case, so yes, the default should be local sessions
with affinity.

On Thu, Mar 10, 2016 at 10:23 AM, Afkham Azeez <az...@wso2.com> wrote:

> Petstore is #2. We use the Redis service to store the session. For an HA
> deployment such a model is required, but yes, for the simplest case, we can
> have local sessions and then use session affinity capabilities of the LB.
>
> On Thu, Mar 10, 2016 at 10:17 AM, Sanjiva Weerawarana <sanj...@wso2.com>
> wrote:
>
>> Manu, #1 is not a no-session story. What Azeez has done for the petstore
>> is a model where session state is in a DB.
>>
>> Session as a service is the same thing ... basically a data service in
>> front of a DB.
>>
>> So really the basic question is can you do without a session? My answer
>> is no, not practical. If you go full HATEOS you can do without sessions but
>> even then you have to re-authenticate every call which is not practical.
>>
>> So its #3 :-).
>>
>> On Wed, Mar 9, 2016 at 8:01 PM, Manuranga Perera <m...@wso2.com> wrote:
>>
>>> Options
>>> 1) No session. Everything is in DB or Window.localStorage.
>>> Authentication via a token validation endpoint. (We keep the token in a
>>> front end cookie)
>>> 2) Session as a service
>>> 3) The session is local, works with session affinity
>>> 4) The session is distributed
>>>
>>> My personal order of preference - 1, 2, 3, 4
>>> Azeez says 2 (or 1? )
>>> Sanjiva says 3, with 4 being plug-able
>>>
>>> I think 1 is doable.
>>> Yes, there will be some development overhead.
>>> But it'll be scalable/simpler at run time.
>>>
>>>
>>> On Wed, Mar 9, 2016 at 6:59 PM, Sanjiva Weerawarana <sanj...@wso2.com>
>>> wrote:
>>>
>>>> Not practical Azeez - you're massively complicating the deployment and
>>>> second its far less performant than replication. Earlier we did global
>>>> replication which we really shouldn't do.
>>>>
>>>> I'm not suggesting replication .. I'm saying we support non-HA sessions
>>>> by default but make that part pluggable so we can plug in a replicating
>>>> model (or even a DB model) if needed.
>>>>
>>>> On Wed, Mar 9, 2016 at 6:43 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>
>>>>> What if we follow an approach of persisting the session to a
>>>>> datastore, like we've done in the petstore sample, that way you don't need
>>>>> to worry about affinity or the node having the session failing. In memory
>>>>> session replication is costly & leads to a whole lot of other issues, like
>>>>> the ones we've seen with replicated caches, so it best to avoid it.
>>>>>
>>>>> On Wed, Mar 9, 2016 at 6:32 PM, Sanjiva Weerawarana <sanj...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Manu's question is in the context of the reusable UI framework stuff
>>>>>> we're working on.
>>>>>>
>>>>>> Fundamentally, is it necessary to have sessions to write a UI? Can we
>>>>>> use HATEOS for some stuff, browser local storage for some stuff etc. and
>>>>>> not have sessions at all??
>>>>>>
>>>>>> I feel we need sessions as a lot of simple things become hard without
>>>>>> them.
>>>>>>
>>>>>> Then comes the question of how do we do sessions and whether we do
>>>>>> some kind of replication or rely on affinity based routing.
>>>>>>
>>>>>> On Wed, Mar 9, 2016 at 5:23 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>>
>>>>>>> With such a model, you don't have to worry about things like session
>>>>>>> replication in order to achieve HA.
>>>>>>>
>>>>>>> On Wed, Mar 9, 2016 at 3:32 PM, Manuranga Perera <m...@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Should we 

Re: [Architecture] The new disruptor based Netty transport is not working well for MSF4J

[Afkham Azeez]

No from day 1, we have decided that GW & MSF4J will use the same Netty
transport component so that the config file will be the same as well as
improvements made to that transport will be automatically available for
both products. So now at least for MSF4J, we have issues in using the Netty
transport in its current state, so we have to fix those issues.

On Thu, Mar 10, 2016 at 10:22 AM, Sagara Gunathunga <sag...@wso2.com> wrote:

>
> When we discuss last week about Carbon transports for MSF4J the main
> rational we identified was moving to Carbon transport will decouple
> transport threads from worker threads through Disruptor and provide lot of
> flexibility and manageability. If disruptor can't give better performance
> we should skip it no argument about that, but once we skip the disruptor
> the rational we made last week about thread pool decoupling become invalid
> if so why MSF4J should depend on Carbon transport ? If Carbon transport
> can't provide real advantages shouldn't MSF4J depend on vanilla Netty
> transports and make thing more lightweight ?
>
> Thanks !
>
> On Thu, Mar 10, 2016 at 9:50 AM, Afkham Azeez <az...@wso2.com> wrote:
>
>> We need to do this fast because this task has taken close to 3 weeks now.
>>
>> On Thu, Mar 10, 2016 at 9:29 AM, Kasun Indrasiri <ka...@wso2.com> wrote:
>>
>>> Yes, we can make the disruptor optional. Also, we should try using the
>>> native worker pool for Event Handler[1], so that the Disruptor itself runs
>>> the event handler on a worker pool. We'll implement both approaches and do
>>> a comparison.
>>>
>>> [1]
>>> https://lmax-exchange.github.io/disruptor/docs/com/lmax/disruptor/dsl/Disruptor.html#handleEventsWithWorkerPool(com.lmax.disruptor.WorkHandler..
>>> .)
>>>
>>> On Thu, Mar 10, 2016 at 8:48 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> After upgrading to the new transport, we are seeing a significant drop
>>>> in performance for any service that take some time to execute. We have
>>>> tried with the configuration used for the gateway which gave the best
>>>> figures on the same hardware. We have also noted that using a separate
>>>> dedicated executor thread pool, which is supported by Netty, gave much
>>>> better performance than the disruptor based implementation. Even in theory,
>>>> disruptor cannot give better performance when used with a real service that
>>>> does some real work, rather than doing passthrough, for example. Can we
>>>> improve the Netty transport to make going through disruptor optional?
>>>>
>>>> --
>>>> *Afkham Azeez*
>>>> Director of Architecture; WSO2, Inc.; http://wso2.com
>>>> Member; Apache Software Foundation; http://www.apache.org/
>>>> * <http://www.apache.org/>*
>>>> *email: **az...@wso2.com* <az...@wso2.com>
>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>>>> *http://blog.afkham.org* <http://blog.afkham.org>
>>>> *twitter: **http://twitter.com/afkham_azeez*
>>>> <http://twitter.com/afkham_azeez>
>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>>>> <http://lk.linkedin.com/in/afkhamazeez>*
>>>>
>>>> *Lean . Enterprise . Middleware*
>>>>
>>>
>>>
>>>
>>> --
>>> Kasun Indrasiri
>>> Software Architect
>>> WSO2, Inc.; http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> cell: +94 77 556 5206
>>> Blog : http://kasunpanorama.blogspot.com/
>>>
>>
>>
>>
>> --
>> *Afkham Azeez*
>> Director of Architecture; WSO2, Inc.; http://wso2.com
>> Member; Apache Software Foundation; http://www.apache.org/
>> * <http://www.apache.org/>*
>> *email: **az...@wso2.com* <az...@wso2.com>
>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>> *http://blog.afkham.org* <http://blog.afkham.org>
>> *twitter: **http://twitter.com/afkham_azeez*
>> <http://twitter.com/afkham_azeez>
>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>> <http://lk.linkedin.com/in/afkhamazeez>*
>>
>> *Lean . Enterprise . Middleware*
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
> Sagara Gunathunga
>
> Architect; WSO2, Inc.;  http://wso2.com
> V.P Apache Web Services;h

Re: [Architecture] Session Affinity Alternatives

[Afkham Azeez]

Petstore is #2. We use the Redis service to store the session. For an HA
deployment such a model is required, but yes, for the simplest case, we can
have local sessions and then use session affinity capabilities of the LB.

On Thu, Mar 10, 2016 at 10:17 AM, Sanjiva Weerawarana <sanj...@wso2.com>
wrote:

> Manu, #1 is not a no-session story. What Azeez has done for the petstore
> is a model where session state is in a DB.
>
> Session as a service is the same thing ... basically a data service in
> front of a DB.
>
> So really the basic question is can you do without a session? My answer is
> no, not practical. If you go full HATEOS you can do without sessions but
> even then you have to re-authenticate every call which is not practical.
>
> So its #3 :-).
>
> On Wed, Mar 9, 2016 at 8:01 PM, Manuranga Perera <m...@wso2.com> wrote:
>
>> Options
>> 1) No session. Everything is in DB or Window.localStorage. Authentication
>> via a token validation endpoint. (We keep the token in a front end cookie)
>> 2) Session as a service
>> 3) The session is local, works with session affinity
>> 4) The session is distributed
>>
>> My personal order of preference - 1, 2, 3, 4
>> Azeez says 2 (or 1? )
>> Sanjiva says 3, with 4 being plug-able
>>
>> I think 1 is doable.
>> Yes, there will be some development overhead.
>> But it'll be scalable/simpler at run time.
>>
>>
>> On Wed, Mar 9, 2016 at 6:59 PM, Sanjiva Weerawarana <sanj...@wso2.com>
>> wrote:
>>
>>> Not practical Azeez - you're massively complicating the deployment and
>>> second its far less performant than replication. Earlier we did global
>>> replication which we really shouldn't do.
>>>
>>> I'm not suggesting replication .. I'm saying we support non-HA sessions
>>> by default but make that part pluggable so we can plug in a replicating
>>> model (or even a DB model) if needed.
>>>
>>> On Wed, Mar 9, 2016 at 6:43 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> What if we follow an approach of persisting the session to a datastore,
>>>> like we've done in the petstore sample, that way you don't need to worry
>>>> about affinity or the node having the session failing. In memory session
>>>> replication is costly & leads to a whole lot of other issues, like the ones
>>>> we've seen with replicated caches, so it best to avoid it.
>>>>
>>>> On Wed, Mar 9, 2016 at 6:32 PM, Sanjiva Weerawarana <sanj...@wso2.com>
>>>> wrote:
>>>>
>>>>> Manu's question is in the context of the reusable UI framework stuff
>>>>> we're working on.
>>>>>
>>>>> Fundamentally, is it necessary to have sessions to write a UI? Can we
>>>>> use HATEOS for some stuff, browser local storage for some stuff etc. and
>>>>> not have sessions at all??
>>>>>
>>>>> I feel we need sessions as a lot of simple things become hard without
>>>>> them.
>>>>>
>>>>> Then comes the question of how do we do sessions and whether we do
>>>>> some kind of replication or rely on affinity based routing.
>>>>>
>>>>> On Wed, Mar 9, 2016 at 5:23 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>
>>>>>> With such a model, you don't have to worry about things like session
>>>>>> replication in order to achieve HA.
>>>>>>
>>>>>> On Wed, Mar 9, 2016 at 3:32 PM, Manuranga Perera <m...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Should we aim to do the same in the UIs we ship, such as products ES?
>>>>>>> There will be some extra effort.
>>>>>>>
>>>>>>> On Wed, Mar 9, 2016 at 2:12 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>>>
>>>>>>>> In the petstore sample, the sessions of the frontend apps are
>>>>>>>> stored in Redis.
>>>>>>>>
>>>>>>>> On Wed, Mar 9, 2016 at 1:57 PM, Imesh Gunaratne <im...@wso2.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Manuranga,
>>>>>>>>>
>>>>>>>>> Yes, what you are saying it true! We should only use session aware
>>>>>>>>> load balancing for existing applications which has session management
>>>>>>

[Architecture] The new disruptor based Netty transport is not working well for MSF4J

[Afkham Azeez]

After upgrading to the new transport, we are seeing a significant drop in
performance for any service that take some time to execute. We have tried
with the configuration used for the gateway which gave the best figures on
the same hardware. We have also noted that using a separate dedicated
executor thread pool, which is supported by Netty, gave much better
performance than the disruptor based implementation. Even in theory,
disruptor cannot give better performance when used with a real service that
does some real work, rather than doing passthrough, for example. Can we
improve the Netty transport to make going through disruptor optional?

-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Session Affinity Alternatives

[Afkham Azeez]

What if we follow an approach of persisting the session to a datastore,
like we've done in the petstore sample, that way you don't need to worry
about affinity or the node having the session failing. In memory session
replication is costly & leads to a whole lot of other issues, like the ones
we've seen with replicated caches, so it best to avoid it.

On Wed, Mar 9, 2016 at 6:32 PM, Sanjiva Weerawarana <sanj...@wso2.com>
wrote:

> Manu's question is in the context of the reusable UI framework stuff we're
> working on.
>
> Fundamentally, is it necessary to have sessions to write a UI? Can we use
> HATEOS for some stuff, browser local storage for some stuff etc. and not
> have sessions at all??
>
> I feel we need sessions as a lot of simple things become hard without them.
>
> Then comes the question of how do we do sessions and whether we do some
> kind of replication or rely on affinity based routing.
>
> On Wed, Mar 9, 2016 at 5:23 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> With such a model, you don't have to worry about things like session
>> replication in order to achieve HA.
>>
>> On Wed, Mar 9, 2016 at 3:32 PM, Manuranga Perera <m...@wso2.com> wrote:
>>
>>> Should we aim to do the same in the UIs we ship, such as products ES?
>>> There will be some extra effort.
>>>
>>> On Wed, Mar 9, 2016 at 2:12 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> In the petstore sample, the sessions of the frontend apps are stored in
>>>> Redis.
>>>>
>>>> On Wed, Mar 9, 2016 at 1:57 PM, Imesh Gunaratne <im...@wso2.com> wrote:
>>>>
>>>>> Hi Manuranga,
>>>>>
>>>>> Yes, what you are saying it true! We should only use session aware
>>>>> load balancing for existing applications which has session management
>>>>> features built into them.
>>>>>
>>>>> Ideally when implementing new applications those should be designed in
>>>>> a way to store their sessions outside the application (irrespective of 
>>>>> they
>>>>> run on containers or not). This can be done with either using a database 
>>>>> or
>>>>> a service (ex: Redis). In that way we can scale the application and 
>>>>> session
>>>>> management service separately and also route request without handling
>>>>> sessions at the load balancer level.
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Wed, Mar 9, 2016 at 1:12 PM, Manuranga Perera <m...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> We are currently using sessions and session affinity in our apps. But
>>>>>> going forward, especially in Micro Services/Docker model does it make 
>>>>>> scene?
>>>>>>
>>>>>> Eg: If we bring up a new container due to high load, requests will
>>>>>> still route to old continents due to the session. If we kill a container
>>>>>> that is associated with some session where should the request go?
>>>>>>
>>>>>> We have written (I think) a session aware router for Docker. It's ok
>>>>>> for external apps, but I think it defeats the purpose of 
>>>>>> containerization,
>>>>>> due to about reasons.
>>>>>>
>>>>>> I think the correct way to do this in our apps is to, have
>>>>>> authentication as a service. A micro service will translate the 
>>>>>> session-id
>>>>>> to a token. App depends fully on the token.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>> --
>>>>>> With regards,
>>>>>> *Manu*ranga Perera.
>>>>>>
>>>>>> phone : 071 7 70 20 50
>>>>>> mail : m...@wso2.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Imesh Gunaratne*
>>>>> Senior Technical Lead
>>>>> WSO2 Inc: http://wso2.com
>>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>>> W: http://imesh.io
>>>>> Lean . Enterprise . Middleware
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Afkham Azeez*
>>>> Director of Architecture; WSO2, Inc.; http://wso2.com
>>>> Member; Apache Software Foundation; http://www.apache.org/
>

Re: [Architecture] Session Affinity Alternatives

[Afkham Azeez]

With such a model, you don't have to worry about things like session
replication in order to achieve HA.

On Wed, Mar 9, 2016 at 3:32 PM, Manuranga Perera <m...@wso2.com> wrote:

> Should we aim to do the same in the UIs we ship, such as products ES?
> There will be some extra effort.
>
> On Wed, Mar 9, 2016 at 2:12 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> In the petstore sample, the sessions of the frontend apps are stored in
>> Redis.
>>
>> On Wed, Mar 9, 2016 at 1:57 PM, Imesh Gunaratne <im...@wso2.com> wrote:
>>
>>> Hi Manuranga,
>>>
>>> Yes, what you are saying it true! We should only use session aware load
>>> balancing for existing applications which has session management features
>>> built into them.
>>>
>>> Ideally when implementing new applications those should be designed in a
>>> way to store their sessions outside the application (irrespective of they
>>> run on containers or not). This can be done with either using a database or
>>> a service (ex: Redis). In that way we can scale the application and session
>>> management service separately and also route request without handling
>>> sessions at the load balancer level.
>>>
>>> Thanks
>>>
>>> On Wed, Mar 9, 2016 at 1:12 PM, Manuranga Perera <m...@wso2.com> wrote:
>>>
>>>> We are currently using sessions and session affinity in our apps. But
>>>> going forward, especially in Micro Services/Docker model does it make 
>>>> scene?
>>>>
>>>> Eg: If we bring up a new container due to high load, requests will
>>>> still route to old continents due to the session. If we kill a container
>>>> that is associated with some session where should the request go?
>>>>
>>>> We have written (I think) a session aware router for Docker. It's ok
>>>> for external apps, but I think it defeats the purpose of containerization,
>>>> due to about reasons.
>>>>
>>>> I think the correct way to do this in our apps is to, have
>>>> authentication as a service. A micro service will translate the session-id
>>>> to a token. App depends fully on the token.
>>>>
>>>> What do you think?
>>>>
>>>> --
>>>> With regards,
>>>> *Manu*ranga Perera.
>>>>
>>>> phone : 071 7 70 20 50
>>>> mail : m...@wso2.com
>>>>
>>>
>>>
>>>
>>> --
>>> *Imesh Gunaratne*
>>> Senior Technical Lead
>>> WSO2 Inc: http://wso2.com
>>> T: +94 11 214 5345 M: +94 77 374 2057
>>> W: http://imesh.io
>>> Lean . Enterprise . Middleware
>>>
>>>
>>
>>
>> --
>> *Afkham Azeez*
>> Director of Architecture; WSO2, Inc.; http://wso2.com
>> Member; Apache Software Foundation; http://www.apache.org/
>> * <http://www.apache.org/>*
>> *email: **az...@wso2.com* <az...@wso2.com>
>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>> *http://blog.afkham.org* <http://blog.afkham.org>
>> *twitter: **http://twitter.com/afkham_azeez*
>> <http://twitter.com/afkham_azeez>
>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>> <http://lk.linkedin.com/in/afkhamazeez>*
>>
>> *Lean . Enterprise . Middleware*
>>
>
>
>
> --
> With regards,
> *Manu*ranga Perera.
>
> phone : 071 7 70 20 50
> mail : m...@wso2.com
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Session Affinity Alternatives

[Afkham Azeez]

In the petstore sample, the sessions of the frontend apps are stored in
Redis.

On Wed, Mar 9, 2016 at 1:57 PM, Imesh Gunaratne <im...@wso2.com> wrote:

> Hi Manuranga,
>
> Yes, what you are saying it true! We should only use session aware load
> balancing for existing applications which has session management features
> built into them.
>
> Ideally when implementing new applications those should be designed in a
> way to store their sessions outside the application (irrespective of they
> run on containers or not). This can be done with either using a database or
> a service (ex: Redis). In that way we can scale the application and session
> management service separately and also route request without handling
> sessions at the load balancer level.
>
> Thanks
>
> On Wed, Mar 9, 2016 at 1:12 PM, Manuranga Perera <m...@wso2.com> wrote:
>
>> We are currently using sessions and session affinity in our apps. But
>> going forward, especially in Micro Services/Docker model does it make scene?
>>
>> Eg: If we bring up a new container due to high load, requests will still
>> route to old continents due to the session. If we kill a container that is
>> associated with some session where should the request go?
>>
>> We have written (I think) a session aware router for Docker. It's ok for
>> external apps, but I think it defeats the purpose of containerization, due
>> to about reasons.
>>
>> I think the correct way to do this in our apps is to, have authentication
>> as a service. A micro service will translate the session-id to a token. App
>> depends fully on the token.
>>
>> What do you think?
>>
>> --
>> With regards,
>> *Manu*ranga Perera.
>>
>> phone : 071 7 70 20 50
>> mail : m...@wso2.com
>>
>
>
>
> --
> *Imesh Gunaratne*
> Senior Technical Lead
> WSO2 Inc: http://wso2.com
> T: +94 11 214 5345 M: +94 77 374 2057
> W: http://imesh.io
> Lean . Enterprise . Middleware
>
>


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Web Profile support on AS 6.0.0

[Afkham Azeez]

Better to have one distribution.

On Wed, Feb 10, 2016 at 11:31 AM, Manoj Kumara <ma...@wso2.com> wrote:

> Hi All,
>
> On Carbon 4.x.x era to support Web profiles we have implemented a
> ServerListener on top of Carbonized Tomcat. Further when introducing Tomcat
> based features to Carbon servers we had to do lot of work in order to get
> the support and this added overhead on maintaining as well.
>
> On future releases we are planing to use vanilla Tomcat servers with added
> extensions to support additional features required by Carbon servers (ex:
> SSO, Class loading, analytics etc.) to simplify the internals and to be
> align with the latest Tomcat releases.
>
> I'm currently working on $Subject task and when it comes to Web Profile
> support we can choose few approaches as I can see,
>
>1. Include TomEE based libs on Tomcat server to get the WebProfile
>support (IMO approach is not suitable as that is what TomEE project is
>trying to address the overhead of adding customized features on top of
>Tomcat)
>2. Use TomEE Jax-RS distribution instead of Tomcat such that with
>single distribution we'll have features of both Tomcat and TomEE
>3. Create two flavours of WSO2 AS 6.0.0 one with Tomcat and another
>with TomEE Jax-RS. So depending on the requirement customers can choose
>which distribution to use.
>
> As per my understanding I prefer the 3nd approach as that way use can
> choose which variance he want depending on the requirements.
>
> Appreciate your feedback on this.
>
> Regards,
> Manoj
>
>
>


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [MSF4j] Creating an archetype for a microservice

[Afkham Azeez]

This doc has to go to the README.md file in the archetype dir

On Thu, Feb 4, 2016 at 9:13 PM, Manuri Amaya Perera <manu...@wso2.com>
wrote:

> Hi Azeez,
>
> I have sent the PR[1]. And please find the documentation here[2].
>
> [1] https://github.com/wso2/msf4j/pull/124
> [2]
> https://docs.google.com/a/wso2.com/document/d/1A5iS6JxbqVuazFwe_EVKeuaXiwCVzOoYOUlA7LSPTPY/edit?usp=sharing
>
> Thank you.
>
> On Thu, Feb 4, 2016 at 8:40 AM, Manuri Amaya Perera <manu...@wso2.com>
> wrote:
>
>> Hi Imesh,
>>
>> Aruna has explained the answers to your questions.
>> Also you can find the previous discussion on creating maven archetypes
>> for a generic osgi bundle and a carbon component can be found here[1].
>>
>> As we did for carbon-bundle-archetype and carbon-component-archetype[2],
>> this also can be published to maven central so that anybody can use it
>> without having to locally build it.
>>
>>
>> [1] mail subject: [CARBON] Creating an archetype for a simple carbon
>> component
>> [2] https://repo1.maven.org/maven2/org/wso2/carbon
>>
>> Thank you.
>>
>>
>>
>> On Wednesday, 3 February 2016, Imesh Gunaratne <im...@wso2.com> wrote:
>>
>>> Hi Manuri,
>>>
>>> Would you mind explaining the purpose of this feature?
>>> - What is an archetype?
>>> - Why do we need it?
>>> - What are we trying to achieve with it in MSF4J?
>>>
>>> Thanks
>>>
>>> On Wed, Feb 3, 2016 at 9:29 PM, Manuri Amaya Perera <manu...@wso2.com>
>>> wrote:
>>>
>>>> Hi Azeez,
>>>>
>>>> To place archetypes I have added a module "archetypes" under msf4j. The
>>>> parent of the pom of this module is msf4j-parent. Under "archetypes" module
>>>> I have created msf4j-microservice-archetype module which contains the
>>>> archetype.
>>>>
>>>> The structure is as follows
>>>> msf4j
>>>> ├── archetypes
>>>> │   ├── msf4j-microservice-archetype
>>>> │   │   ├── pom.xml
>>>> │   │   ├── src
>>>> │   │   │   └── main
>>>> │   │   │   └── resources
>>>> │   │   │   ├── archetype-resources
>>>> │   │   │   │   ├── pom.xml
>>>> │   │   │   │   └── src
>>>> │   │   │   │   └── main
>>>> │   │   │   │   └── java
>>>> │   │   │   │ ├── Application.java
>>>> │   │   │   │ └── MicroService.java
>>>> │   │   │   └── META-INF
>>>> │   │   │   └── maven
>>>> │   │   │   └── archetype-metadata.xml
>>>> │   ├── pom.xml
>>>>
>>>> For the project generated from the archetype the default values I have
>>>> given at the moment are as follows,
>>>>
>>>> groupId = org.wso2.msf4j
>>>> artifactId = org.wso2.msf4j.microservice
>>>> version = 1.0.0-SNAPSHOT
>>>> package = org.wso2.msf4j.microservice
>>>>
>>>> The structure of a project created from this archetype with the default
>>>> values, is as follows,
>>>> ├── pom.xml
>>>> ├── src
>>>> │   └── main
>>>> │   └── java
>>>> │   └── org
>>>> │   └── wso2
>>>> │   └── msf4j
>>>> │   └── microservice
>>>> │   ├── Application.java
>>>> │   └── MicroService.java
>>>>
>>>>
>>>> Application.java contains the main method. The class which contains
>>>> methods for http CRUD operations is named as "MicroService.java" because
>>>> that name should be a generic one.
>>>>
>>>> Please suggest if any changes are needed to be done.
>>>>
>>>> And for GET, POST, PUT, DELETE operations should there be any
>>>> implementation?
>>>> We can do something like, keeping some information in a datastructure
>>>> and perform CRUD on it.  For example, create a POJO class Student and keep
>>>> a map of students inside MicroService class.
>>>> Any suggestions on this?
>>>>
>>>>
>>>> Thank you.
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Manuri Amaya Perera*
>>>>
>&g

Re: [Architecture] Netty Servlet Bridge - Servlet environment on top of netty

[Afkham Azeez]

If we can use this, then we don't need to bring in Tomcat as a bundle for
products requiring servlet API support.

On Thu, Jan 7, 2016 at 11:42 AM, Samiyuru Senarathne <samiy...@wso2.com>
wrote:

> Hi,
>
> According to the requirements of C5 platform, if it is helpful to create a
> servlet environment on top of netty to reuse existing servlet based
> projects on top of netty the *netty-servlet-bridge project [1]
> <https://github.com/bigpuritz/netty-servlet-bridge>* may be useful.
>
> [1] - https://github.com/bigpuritz/netty-servlet-bridge
>
> Best Regards,
> Samiyuru
> --
> Samiyuru Senarathne
> *Software Engineer*
> Mobile : +94 (0) 71 134 6087
> samiy...@wso2.com
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] PAX exam based OSGi testing seems inefficient & suboptimal

[Afkham Azeez]

Yes, in the case of kernel, you have used PAX exam to install all the
kernel bundles and then test the relevant features. Ideally, you would add
mock bundles and then test just one or a few real bundles. So this is a
unit test for OSGi bundles. That should have been the correct approach.

Also, does PAX exam have a mode to support bundles.info? If so it will be
easy because providing all the bundles, including the kernel bundles, when
you are testing a particular feature in a different bundle is cumbersome.

Arun had mentioned that if the PAX exam tests run successfully, you can
assume that the product will start successfully. That will not be the case
because the way in which the PAX exam runtime starts up is different from
how a product runtime starts up.

On Thu, Dec 17, 2015 at 11:10 AM, Sameera Jayasoma <same...@wso2.com> wrote:

> Hi Azeez,
>
> Pax Exam is not a replacement for our integration tests. It is a way to
> run unit tests which require an OSGi environment.
>
> Pax Exam is the most suitable solution to test Carbon Kernel and our
> components (repos). We need to evaluate Pax Exam for testing our middleware
> products. Carbon Kernel does not have transports, hence its bit hard to
> execute integration tests. Carbon Kernel provides different OSGI utilities
> for products. Therefore we need to run unit tests in an OSGI environment.
> This is a must for Carbon kernel.
>
> On Thu, Dec 17, 2015 at 12:12 AM, Afkham Azeez <az...@wso2.com> wrote:
>
>> 1. PAX exam tests being successful does not imply that the product pack
>> will properly start
>> This is because the way in which bundles are added to the PAX exam
>> runtime is completely different from how the features are installed into
>> product packs
>>
>
> I don't agree with this point. :)  Let me explain.
>
> We load all the bundles from the bundles.info file into a clean OSGI
> environments . In PAX Exam we load all the bundles programmatically into a
>  clean environment plus Pax Exam bundles. Only difference is the Pax Exam
> bundles.
>
>
>>
>> 2. Installing 100s of bundles programatically to test a product is not
>> practical and a very cumbersome error prone approach
>> I don't believe PAX exam was designed to do this. IMO, PAX exam should be
>> used to test a bundle or may be just a few bundles, and also instead of a
>> fully fledged runtime like a product, it is designed to have mock OSGi
>> services & bundles along with the bundles under test.
>>
>
> One option is to load all bundles from the bundles.info file. This will
> make sure Pax Exam tests are up to date with product changes..
>
>
>
>> 3. Correct approach for product integration testing is to use the product
>> pack
>> This is because of 1 & 2 above. In Carbon 4, we use the pack, extract it
>> and run tests against it. I still believe that is the correct approach
>> which needs to be followed in Carbon 5 based products as well.
>>
>
> Yes this is true for products. We have to go with integration test
> approach for our products. Pax Exam can be used to test functionalities
> which requires OSGi in Carbon kernel and components.
>
>
>>
>> 4. PAX exam along with mock bundles & mock OSGi services for testing OSGi
>> bundles
>> I think PAX exam will be useful for unit testing individual bundles
>>
>
> This is true.
>
>
> Thanks,
> Sameera.
>
>
>
>>
>> Thanks
>> Azeez
>>
>>
>> On Wed, Dec 16, 2015 at 9:49 PM, Aruna Karunarathna <ar...@wso2.com>
>> wrote:
>>
>>>
>>>
>>> On Wed, Dec 16, 2015 at 8:22 PM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> I have been trying to develop PAX exam based tests for MSS and it seems
>>>> very hard to use because there is little help for troubleshooting. MSS has
>>>> just one feature and it has been a struggle to get the test setup. With
>>>> much more complex products, it would be a total nightmare. A lot of time is
>>>> wasted due to trial and error method of troubleshooting.
>>>>
>>>> Hi Azeez,
>>>
>>> Yes, agree on you that, it's not very easy to configure the test setup
>>> at the beginning, when we were integrating it to kernel it took some time
>>> to understand the anatomy of PAX-Exam test cases and the first test case to
>>> up and running.
>>> First what we did was, make the base test case solid, and make sure the
>>> server is up and running, For example, when the test container starts it
>>> does not start as we run the carbon.sh file, since the PAX-Test container
>>> doesn't use 

Re: [Architecture] PAX exam based OSGi testing seems inefficient & suboptimal

[Afkham Azeez]

1. PAX exam tests being successful does not imply that the product pack
will properly start
This is because the way in which bundles are added to the PAX exam runtime
is completely different from how the features are installed into product
packs

2. Installing 100s of bundles programatically to test a product is not
practical and a very cumbersome error prone approach
I don't believe PAX exam was designed to do this. IMO, PAX exam should be
used to test a bundle or may be just a few bundles, and also instead of a
fully fledged runtime like a product, it is designed to have mock OSGi
services & bundles along with the bundles under test.

3. Correct approach for product integration testing is to use the product
pack
This is because of 1 & 2 above. In Carbon 4, we use the pack, extract it
and run tests against it. I still believe that is the correct approach
which needs to be followed in Carbon 5 based products as well.

4. PAX exam along with mock bundles & mock OSGi services for testing OSGi
bundles
I think PAX exam will be useful for unit testing individual bundles

Thanks
Azeez


On Wed, Dec 16, 2015 at 9:49 PM, Aruna Karunarathna <ar...@wso2.com> wrote:

>
>
> On Wed, Dec 16, 2015 at 8:22 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> I have been trying to develop PAX exam based tests for MSS and it seems
>> very hard to use because there is little help for troubleshooting. MSS has
>> just one feature and it has been a struggle to get the test setup. With
>> much more complex products, it would be a total nightmare. A lot of time is
>> wasted due to trial and error method of troubleshooting.
>>
>> Hi Azeez,
>
> Yes, agree on you that, it's not very easy to configure the test setup at
> the beginning, when we were integrating it to kernel it took some time to
> understand the anatomy of PAX-Exam test cases and the first test case to up
> and running.
> First what we did was, make the base test case solid, and make sure the
> server is up and running, For example, when the test container starts it
> does not start as we run the carbon.sh file, since the PAX-Test container
> doesn't use the org.wso2.carbon.launcher.jar to boot up the server, so we
> had to simulate that before starting the PAX-Exam container [1]. But after
> that
> people wrote test cases without much hassle.
>
> After those efforts, I think we make use of Pax to solve most of our in
> container testing scenarios, We were able to make sure each OSGi services
> are running solidly. So after code changes, it made very easy to identify
> the broken OSGi services etc, without manual testing. also it helped to
> simulate complex cases such as start up coordination.
>
>
>> Either we are using PAX exam incorrectly or PAX exam is not best
>> framework to be adopted.
>>
>> I'm not sure that PAX exam complicates things when it comes to products,
> but for kernel I think we need in container OSGi test cases. So we can
> verify the services are working accordingly.
>
> From what I have experienced so far, in most of our current products,
> missing OSGi service or some service component issues can only be
> identified after starting the product. We need to identify these issues at
> the build time. Either using Pax or some other alternative.
>
> [1].
> https://github.com/wso2/carbon-kernel/blob/master/tests/osgi-tests/src/test/java/org/wso2/carbon/osgi/utils/OSGiTestUtils.java#L43
>
> Regards,
> Aruna
>
>> Thoughts welcome.
>>
>> --
>> *Afkham Azeez*
>> Director of Architecture; WSO2, Inc.; http://wso2.com
>> Member; Apache Software Foundation; http://www.apache.org/
>> * <http://www.apache.org/>*
>> *email: **az...@wso2.com* <az...@wso2.com>
>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>> *http://blog.afkham.org* <http://blog.afkham.org>
>> *twitter: **http://twitter.com/afkham_azeez*
>> <http://twitter.com/afkham_azeez>
>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>> <http://lk.linkedin.com/in/afkhamazeez>*
>>
>> *Lean . Enterprise . Middleware*
>>
>
>
>
> --
>
> *Aruna Sujith Karunarathna *| Software Engineer
> WSO2, Inc | lean. enterprise. middleware.
> #20, Palm Grove, Colombo 03, Sri Lanka
> Mobile: +94 71 9040362 | Work: +94 112145345
> Email: ar...@wso2.com | Web: www.wso2.com
>
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] PAX exam based OSGi testing seems inefficient & suboptimal

[Afkham Azeez]

Check the attached class which programatically adds all the bundles
necessary just to get MSS, our smallest product running & its complexity.
It is not practical to do this for all products. The correct way to
integration test products is to use the product packs itself, not an full
product environment created using PAX exam.

On Thu, Dec 17, 2015 at 12:12 AM, Afkham Azeez <az...@wso2.com> wrote:

> 1. PAX exam tests being successful does not imply that the product pack
> will properly start
> This is because the way in which bundles are added to the PAX exam runtime
> is completely different from how the features are installed into product
> packs
>
> 2. Installing 100s of bundles programatically to test a product is not
> practical and a very cumbersome error prone approach
> I don't believe PAX exam was designed to do this. IMO, PAX exam should be
> used to test a bundle or may be just a few bundles, and also instead of a
> fully fledged runtime like a product, it is designed to have mock OSGi
> services & bundles along with the bundles under test.
>
> 3. Correct approach for product integration testing is to use the product
> pack
> This is because of 1 & 2 above. In Carbon 4, we use the pack, extract it
> and run tests against it. I still believe that is the correct approach
> which needs to be followed in Carbon 5 based products as well.
>
> 4. PAX exam along with mock bundles & mock OSGi services for testing OSGi
> bundles
> I think PAX exam will be useful for unit testing individual bundles
>
> Thanks
> Azeez
>
>
> On Wed, Dec 16, 2015 at 9:49 PM, Aruna Karunarathna <ar...@wso2.com>
> wrote:
>
>>
>>
>> On Wed, Dec 16, 2015 at 8:22 PM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> I have been trying to develop PAX exam based tests for MSS and it seems
>>> very hard to use because there is little help for troubleshooting. MSS has
>>> just one feature and it has been a struggle to get the test setup. With
>>> much more complex products, it would be a total nightmare. A lot of time is
>>> wasted due to trial and error method of troubleshooting.
>>>
>>> Hi Azeez,
>>
>> Yes, agree on you that, it's not very easy to configure the test setup at
>> the beginning, when we were integrating it to kernel it took some time to
>> understand the anatomy of PAX-Exam test cases and the first test case to up
>> and running.
>> First what we did was, make the base test case solid, and make sure the
>> server is up and running, For example, when the test container starts it
>> does not start as we run the carbon.sh file, since the PAX-Test container
>> doesn't use the org.wso2.carbon.launcher.jar to boot up the server, so we
>> had to simulate that before starting the PAX-Exam container [1]. But after
>> that
>> people wrote test cases without much hassle.
>>
>> After those efforts, I think we make use of Pax to solve most of our in
>> container testing scenarios, We were able to make sure each OSGi services
>> are running solidly. So after code changes, it made very easy to identify
>> the broken OSGi services etc, without manual testing. also it helped to
>> simulate complex cases such as start up coordination.
>>
>>
>>> Either we are using PAX exam incorrectly or PAX exam is not best
>>> framework to be adopted.
>>>
>>> I'm not sure that PAX exam complicates things when it comes to products,
>> but for kernel I think we need in container OSGi test cases. So we can
>> verify the services are working accordingly.
>>
>> From what I have experienced so far, in most of our current products,
>> missing OSGi service or some service component issues can only be
>> identified after starting the product. We need to identify these issues at
>> the build time. Either using Pax or some other alternative.
>>
>> [1].
>> https://github.com/wso2/carbon-kernel/blob/master/tests/osgi-tests/src/test/java/org/wso2/carbon/osgi/utils/OSGiTestUtils.java#L43
>>
>> Regards,
>> Aruna
>>
>>> Thoughts welcome.
>>>
>>> --
>>> *Afkham Azeez*
>>> Director of Architecture; WSO2, Inc.; http://wso2.com
>>> Member; Apache Software Foundation; http://www.apache.org/
>>> * <http://www.apache.org/>*
>>> *email: **az...@wso2.com* <az...@wso2.com>
>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>>> *http://blog.afkham.org* <http://blog.afkham.org>
>>> *twitter: **http://twitter.com/afkham_azeez*
>>> <http://twitter.com/afkham_azeez>
>>> *linked-in: **htt

[Architecture] PAX exam based OSGi testing seems inefficient & suboptimal

[Afkham Azeez]

I have been trying to develop PAX exam based tests for MSS and it seems
very hard to use because there is little help for troubleshooting. MSS has
just one feature and it has been a struggle to get the test setup. With
much more complex products, it would be a total nightmare. A lot of time is
wasted due to trial and error method of troubleshooting.

Either we are using PAX exam incorrectly or PAX exam is not best framework
to be adopted.

Thoughts welcome.

-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] [MSS] Return javax.ws.rs.core.Response while running OSGi mode

[Afkham Azeez]

Yes it is working. I have tested it with both OSGi bundle mode & deployable
jar mode in the MSS server runtime.

On Wed, Dec 16, 2015 at 11:53 AM, Sagara Gunathunga <sag...@wso2.com> wrote:

>
>
> On Wed, Dec 2, 2015 at 4:49 PM, Aruna Karunarathna <ar...@wso2.com> wrote:
>
>>
>>
>> On Mon, Nov 30, 2015 at 12:50 PM, Aruna Karunarathna <ar...@wso2.com>
>> wrote:
>>
>>>
>>>
>>> On Mon, Nov 30, 2015 at 11:59 AM, Sameera Jayasoma <same...@wso2.com>
>>> wrote:
>>>
>>>> I guess this a bug. You can fix this and contribute back to Aries.
>>>>
>>>
>>> Actually I have filed a bug regarding this for Aries project [1]
>>> Seems like it is in in-progress state.
>>>
>>> [1]. https://issues.apache.org/jira/browse/ARIES-1461
>>>
>>
>> The SPI-fly guys have provided a possible fix. I tested with the fragment
>> bundle for the javax.ws.rs-api_2.0.0.jar and seems it's working
>> perfectly.
>>
>> We should be able to use that once it's released.
>>
>
>  Have we fixed this issue in trunk ? Can I use Response as return type now
> ?
>
> Thanks !
>
>
>>
>> Regards,
>> Aruna
>>
>>>
>>>> On Thu, Nov 26, 2015 at 12:11 PM, Aruna Karunarathna <ar...@wso2.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 24, 2015 at 10:02 PM, Sagara Gunathunga <sag...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Nov 23, 2015 at 2:15 PM, Aruna Karunarathna <ar...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Nov 16, 2015 at 12:39 PM, Samiyuru Senarathne <
>>>>>>> samiy...@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi Sagara,
>>>>>>>>
>>>>>>>> This issue is about loading our implementation
>>>>>>>> of javax.ws.rs.core.Response [1] in OSGi mode using
>>>>>>>> java java.util.ServiceLoader [2] by the relevant bundle. This issue is 
>>>>>>>> yet
>>>>>>>> to be fixed and according to the discussions we had, the solution is 
>>>>>>>> to use
>>>>>>>> SPI Fly [3] to fix the ServiceLoader issue.
>>>>>>>>
>>>>>>>
>>>>>>> I tried using SPI-fly approach to overcome this problem. Seems like
>>>>>>> we cant use SPI Fly approach directly, since in the javax.ws.rs-api they
>>>>>>> are loading the implementation class using the class.forName().
>>>>>>> We have to investigate the possibilities of loading the
>>>>>>> implementation class using a fragment bundle approach, and load the 
>>>>>>> class
>>>>>>> using the
>>>>>>> ServiceLoader.load().
>>>>>>>
>>>>>>
>>>>>> One quick suggestion, shall we change JAX-RS API sepc implementation
>>>>>> from Oracle to Geronimo and check with SPI-fly ? BTW I haven't check how
>>>>>> Geronimo spec load Response class.
>>>>>>
>>>>>>
>>>>> Hi Sagara,
>>>>>
>>>>> Was able to make it working using the SPI-Fly. But we need the
>>>>> following attributes in the respective bundles.
>>>>>
>>>>> *javax.ws.rs.ext.RuntimeDelegate* in
>>>>> the org.wso2.carbon.mss_1.0.0.SNAPSHOT.jar
>>>>>
>>>>>
>>>>> *javax.ws.rs.ext.RuntimeDelegate#findDelegate*
>>>>> in the javax.ws.rs-api_2.0.0.jar
>>>>>
>>>>> I had to tested this by creating a orbit bundle from
>>>>> the javax.ws.rs-api_2.0.0.jar.
>>>>>
>>>>> @Sameera
>>>>> I tried to create a fragment bundle for the javax.ws.rs-api_2.0.0.jar
>>>>> to add the  SPI-Consumer entry. Seems like spi-fly not picking up the 
>>>>> entry
>>>>> is for the javax.ws.rs-api_2.0.0.jar
>>>>>
>>>>> I guess we have to create an orbit. Any thoughts?
>>>>>
>>>>> Regards,
>>>>> Aruna
>>>>> --
>>>>>
>>>>> *Aruna Sujith Karunarathna

Re: [Architecture] We no longer need $CARBON_HOME/repository/deployment/server

[Afkham Azeez]

It becomes difficult when multiple deployers deploy artifacts that have the
same extension and would complicate things.

On Thu, Dec 10, 2015 at 9:22 PM, Chanaka Fernando <chana...@wso2.com> wrote:

> Can't we combine the components having the same deployment directory?
> Let's say if we are combining GW and AS. Then we can have the same
> deployment directory and under that, we can have folders for webapps(AS),
> carbonapps(Common), gw-routes(GW), etc.
>
> On Thu, Dec 10, 2015 at 3:43 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> We need that because we can combine components.
>>
>> On Thu, Dec 10, 2015 at 8:52 PM, Chanaka Fernando <chana...@wso2.com>
>> wrote:
>>
>>> Do we really need "mss" at the end? If we take the MSS server, then we
>>> are already inside it. Wouldn't that be enough having
>>> "$CARBON_HOME/repository/deployment" ?
>>>
>>> On Thu, Dec 10, 2015 at 6:33 AM, Aruna Karunarathna <ar...@wso2.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Dec 10, 2015 at 11:57 AM, Sameera Jayasoma <same...@wso2.com>
>>>> wrote:
>>>>
>>>>> +1. We need to do this change in C5 :)
>>>>>
>>>>
>>>> Noted. Created a jira to track this [1]
>>>>
>>>> [1]. https://wso2.org/jira/browse/CARBON-15723
>>>>
>>>>>
>>>>> Thanks,
>>>>> Sameera
>>>>>
>>>>> On Thu, Dec 10, 2015 at 11:37 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>
>>>>>> For example, the MSS deployment dir is
>>>>>> $CARBON_HOME/repository/deployment/server/mss
>>>>>>
>>>>>> We no longer need the server part because this was a concept to do
>>>>>> with Axis2 server & client ConfigContext creation, which we no longer 
>>>>>> have
>>>>>> in Carbon 5.
>>>>>>
>>>>>> So the MSS deployment directory should be;
>>>>>> $CARBON_HOME/repository/deployment/mss
>>>>>>
>>>>>> Same should be noted for Gateway.
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Afkham Azeez*
>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com
>>>>>> Member; Apache Software Foundation; http://www.apache.org/
>>>>>> * <http://www.apache.org/>*
>>>>>> *email: **az...@wso2.com* <az...@wso2.com>
>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>>>>>> *http://blog.afkham.org* <http://blog.afkham.org>
>>>>>> *twitter: **http://twitter.com/afkham_azeez*
>>>>>> <http://twitter.com/afkham_azeez>
>>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>>>>>> <http://lk.linkedin.com/in/afkhamazeez>*
>>>>>>
>>>>>> *Lean . Enterprise . Middleware*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sameera Jayasoma,
>>>>> Software Architect,
>>>>>
>>>>> WSO2, Inc. (http://wso2.com)
>>>>> email: same...@wso2.com
>>>>> blog: http://blog.sameera.org
>>>>> twitter: https://twitter.com/sameerajayasoma
>>>>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections
>>>>> Mobile: 0094776364456
>>>>>
>>>>> Lean . Enterprise . Middleware
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Aruna Sujith Karunarathna *| Software Engineer
>>>> WSO2, Inc | lean. enterprise. middleware.
>>>> #20, Palm Grove, Colombo 03, Sri Lanka
>>>> Mobile: +94 71 9040362 | Work: +94 112145345
>>>> Email: ar...@wso2.com | Web: www.wso2.com
>>>>
>>>>
>>>> ___
>>>> Architecture mailing list
>>>> Architecture@wso2.org
>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>
>>>>
>>>
>>>
>>> --
>>> Thank you and Best Regards,
>>> Chanaka Fernando
>>> Senior Technical Lead
>>> WSO2, Inc.; http://wso2.com
>>> lean.enterprise.middleware
>>>
>>> mobile: +94 773337238
>>> Blog : http://soatutoria

Re: [Architecture] We no longer need $CARBON_HOME/repository/deployment/server

[Afkham Azeez]

The directory should be;

$CARBON_HOME/deployment/microservices

On Fri, Dec 11, 2015 at 1:38 PM, Afkham Azeez <az...@wso2.com> wrote:

> It becomes difficult when multiple deployers deploy artifacts that have
> the same extension and would complicate things.
>
> On Thu, Dec 10, 2015 at 9:22 PM, Chanaka Fernando <chana...@wso2.com>
> wrote:
>
>> Can't we combine the components having the same deployment directory?
>> Let's say if we are combining GW and AS. Then we can have the same
>> deployment directory and under that, we can have folders for webapps(AS),
>> carbonapps(Common), gw-routes(GW), etc.
>>
>> On Thu, Dec 10, 2015 at 3:43 PM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> We need that because we can combine components.
>>>
>>> On Thu, Dec 10, 2015 at 8:52 PM, Chanaka Fernando <chana...@wso2.com>
>>> wrote:
>>>
>>>> Do we really need "mss" at the end? If we take the MSS server, then we
>>>> are already inside it. Wouldn't that be enough having
>>>> "$CARBON_HOME/repository/deployment" ?
>>>>
>>>> On Thu, Dec 10, 2015 at 6:33 AM, Aruna Karunarathna <ar...@wso2.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Dec 10, 2015 at 11:57 AM, Sameera Jayasoma <same...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> +1. We need to do this change in C5 :)
>>>>>>
>>>>>
>>>>> Noted. Created a jira to track this [1]
>>>>>
>>>>> [1]. https://wso2.org/jira/browse/CARBON-15723
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Sameera
>>>>>>
>>>>>> On Thu, Dec 10, 2015 at 11:37 AM, Afkham Azeez <az...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> For example, the MSS deployment dir is
>>>>>>> $CARBON_HOME/repository/deployment/server/mss
>>>>>>>
>>>>>>> We no longer need the server part because this was a concept to do
>>>>>>> with Axis2 server & client ConfigContext creation, which we no longer 
>>>>>>> have
>>>>>>> in Carbon 5.
>>>>>>>
>>>>>>> So the MSS deployment directory should be;
>>>>>>> $CARBON_HOME/repository/deployment/mss
>>>>>>>
>>>>>>> Same should be noted for Gateway.
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Afkham Azeez*
>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com
>>>>>>> Member; Apache Software Foundation; http://www.apache.org/
>>>>>>> * <http://www.apache.org/>*
>>>>>>> *email: **az...@wso2.com* <az...@wso2.com>
>>>>>>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>>>>>>> *http://blog.afkham.org* <http://blog.afkham.org>
>>>>>>> *twitter: **http://twitter.com/afkham_azeez*
>>>>>>> <http://twitter.com/afkham_azeez>
>>>>>>> *linked-in: **http://lk.linkedin.com/in/afkhamazeez
>>>>>>> <http://lk.linkedin.com/in/afkhamazeez>*
>>>>>>>
>>>>>>> *Lean . Enterprise . Middleware*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sameera Jayasoma,
>>>>>> Software Architect,
>>>>>>
>>>>>> WSO2, Inc. (http://wso2.com)
>>>>>> email: same...@wso2.com
>>>>>> blog: http://blog.sameera.org
>>>>>> twitter: https://twitter.com/sameerajayasoma
>>>>>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections
>>>>>> Mobile: 0094776364456
>>>>>>
>>>>>> Lean . Enterprise . Middleware
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *Aruna Sujith Karunarathna *| Software Engineer
>>>>> WSO2, Inc | lean. enterprise. middleware.
>>>>> #20, Palm Grove, Colombo 03, Sri Lanka
>>>>> Mobile: +94 71 9040362 | Work: +94 112145345
>>>>> Email: ar...@wso2.com | Web: www.wso2.com
>>>>>
>>>>>
&

Re: [Architecture] Issue when you start up MSS in server mode

[Afkham Azeez]

The deployment engine should call the deployer only for valid artifact
types. So yes, it looks like the fixes have to be done in the kernel.

On Thu, Dec 10, 2015 at 1:47 PM, Samiyuru Senarathne <samiy...@wso2.com>
wrote:

> First part of this issue is trying to deploy invalid files. The second
> part of the issue is deployment engine tries to deploy the artifact again
> and again even though an exception is thrown for that particular artifact
> by the deployer. Because of this the exception is thrown again and again.
>
> The second part of the problem will be solved with the kernel GA since the
> GA's deployment engine records the faulty artifacts.
>
> IMO the first part of issue also should be fixed in deployment engine. The
> reason for that is as follows.
>
> *org.wso2.carbon.mss.internal.deployer.MSSDeployer* implements
> *org.wso2.carbon.kernel.deployment.Deployer* interface that is provided
> by the deployment engine of carbon kernel.
>
> org.wso2.carbon.kernel.deployment.Deployer interface has the following
> functions.
>
>- Object deploy(Artifact var1) throws CarbonDeploymentException;
>- void undeploy(Object var1) throws CarbonDeploymentException;
>- Object update(Artifact var1) throws CarbonDeploymentException;
>- URL getLocation();
>- ArtifactType getArtifactType();
>
> Here deploy(Artifact var1) function is called with a
> *org.wso2.carbon.kernel.deployment.Artifact* parameter by the kernel's
> deployment engine whenever a new artifact is found. So, deploy() function
> is called with both valid and invalid artifact types.
>
> Here, getArtifactType() is meant to get the artifact type that is
> supported by a particular deployer implementation. IMO since this function
> is there to identify the supported artifact types of a Deployer, deployment
> engine should filter the found artifacts using the ArtifactType before
> calling the deploy() function of a Deployer implementation. Then, the
> deployment engine will call deploy() function only with valid artifacts for
> that specific deployer. Therefore, each deployer implementation will not
> have to do duplicate implementations to filt the artifact types they
> support. IMO unless getting an ArtifactType from the deployer
> implementation is less useful.
>
> WDYT?
>
> Best Regards,
> Samiyuru
>
> On Thu, Dec 10, 2015 at 11:35 AM, Afkham Azeez <az...@wso2.com> wrote:
>
>> [2015-12-10 11:34:09,012]  INFO
>> {org.wso2.carbon.mss.internal.deployer.MSSDeployer} - Deploying artifact:
>> /Users/azeez/projects/github/wso2/product-mss/product/target/wso2mss-1.0.0-SNAPSHOT/repository/deployment/server/mss/README.txt
>>
>> [2015-12-10 11:34:09,013] ERROR
>> {org.wso2.carbon.kernel.internal.deployment.DeploymentEngine} - Error while
>> deploying artifacts
>> org.wso2.carbon.kernel.deployment.exception.CarbonDeploymentException:
>> Error while processing the artifact:
>> /Users/azeez/projects/github/wso2/product-mss/product/target/wso2mss-1.0.0-SNAPSHOT/repository/deployment/server/mss/README.txt
>>
>> at
>> org.wso2.carbon.mss.internal.deployer.MSSDeployer.deploy(MSSDeployer.java:78)
>>
>> at
>> org.wso2.carbon.kernel.internal.deployment.DeploymentEngine.lambda$deployArtifacts$18(DeploymentEngine.java:229)
>>
>> at
>> org.wso2.carbon.kernel.internal.deployment.DeploymentEngine$$Lambda$45/1996363093.accept(Unknown
>> Source)
>>
>> at java.util.ArrayList.forEach(ArrayList.java:1249)
>>
>> at
>> org.wso2.carbon.kernel.internal.deployment.DeploymentEngine.deployArtifacts(DeploymentEngine.java:225)
>>
>> at
>> org.wso2.carbon.kernel.internal.deployment.RepositoryScanner.sweep(RepositoryScanner.java:110)
>>
>> at
>> org.wso2.carbon.kernel.internal.deployment.RepositoryScanner.scan(RepositoryScanner.java:68)
>>
>> at
>> org.wso2.carbon.kernel.internal.deployment.SchedulerTask.run(SchedulerTask.java:43)
>>
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>>
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>>
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>
>> at java.lang.Thread.run(Thread.java:745)
>>
>

Re: [Architecture] Removing $CARBON_HOME/repository/ directory from C5 onwards

[Afkham Azeez]

Yes, we can move them to the CARBON_HOME. Repository directory has been
there for historic reasons since 2007 or so. We don't need it anymore.

On Thu, Dec 10, 2015 at 9:32 PM, Sameera Jayasoma <same...@wso2.com> wrote:

> Hi Folks,
>
> How about the @Subject? We've introduced repository directory a long time
> back to support deploying our product in other Application servers. Now we
> don't support that, therefore there is no need to maintain that directory
> structure.
>
> Thanks,
> Sameera.
>
> --
> Sameera Jayasoma,
> Software Architect,
>
> WSO2, Inc. (http://wso2.com)
> email: same...@wso2.com
> blog: http://blog.sameera.org
> twitter: https://twitter.com/sameerajayasoma
> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections
> Mobile: 0094776364456
>
> Lean . Enterprise . Middleware
>
>


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

[Architecture] We no longer need $CARBON_HOME/repository/deployment/server

[Afkham Azeez]

For example, the MSS deployment dir is
$CARBON_HOME/repository/deployment/server/mss

We no longer need the server part because this was a concept to do with
Axis2 server & client ConfigContext creation, which we no longer have in
Carbon 5.

So the MSS deployment directory should be;
$CARBON_HOME/repository/deployment/mss

Same should be noted for Gateway.


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Possible Issue related to transport state in CarbonTransport in Carbon 5

[Afkham Azeez]

Change the code as follows:

void stopTransport() { <- one of the template methods
stop(); <- abstract method is called
if (state.equals(State.STARTED)) {
state = State.STOPPED; <- state is changed without knowing the
status of stop() call
} else {
throw new IllegalStateException("Cannot stop transport " + id +
". Current state: " + state);
}

}

>From the stop() method above, if something fails, throw a RuntimeException.

On Tue, Dec 8, 2015 at 3:50 PM, Samiyuru Senarathne <samiy...@wso2.com>
wrote:

> Hi,
>
> CarbonTransport [1] class in carbon-kernel implements template pattern and
> it has the following abstract methods.
>
>- protected abstract void start();
>- protected abstract void stop();
>- protected abstract void beginMaintenance();
>- protected abstract void endMaintenance();
>
> These abstract methods should be implemented by any class that implements
> a carbon transport. (For example NettyListener [2] does this). These
> abstract methods are called by the template methods of CarbonTransport
> class. These template methods maintain the state of the carbon transport as
> shown in the following code block.
>
> void stopTransport() { <- one of the template methods
> if (state.equals(State.STARTED)) {
> state = State.STOPPED; <- state is changed without knowing
> the status of stop() call
> } else {
> throw new IllegalStateException("Cannot stop transport " + id
> + ". Current state: " + state);
> }
> stop(); <- abstract method is called
> }
>
> As shown in the above code block CarbonTransport's template functions
> change the state of the transport without knowing the actual state of the
> transport after invoking the abstract function (assuming the abstract
> method call succeeded).
>
> IMO this behaviour can produce invalid states when the operations of the
> abstract method implementations do not complete as expected. IMO in order
> to fix this we can introduce an exception type to the abstract method's
> signature for the implementations to notify any failure or add a state
> return to the signature.
>
>- protected abstract void start() throws CarbonTransportException;
>- protected abstract void stop() throws CarbonTransportException;
>- protected abstract void beginMaintenance()
> throws CarbonTransportException;
>- protected abstract void endMaintenance()
> throws CarbonTransportException;
>
> Or
>
>- protected abstract State start();
>- protected abstract State stop();
>- protected abstract State beginMaintenance();
>- protected abstract State endMaintenance();
>
>
> WDYT?
>
> [1] -
> https://github.com/wso2/carbon-kernel/blob/master/core/src/main/java/org/wso2/carbon/kernel/transports/CarbonTransport.java
> [2] -
> https://github.com/wso2/carbon-transports/blob/master/http/netty/component/src/main/java/org/wso2/carbon/transport/http/netty/listener/NettyListener.java
>
> Thank you.
>
> Best Regards,
> Samiyuru
>
> --
> Samiyuru Senarathne
> *Software Engineer*
> Mobile : +94 (0) 71 134 6087
> samiy...@wso2.com
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Possible Issue related to transport state in CarbonTransport in Carbon 5

[Afkham Azeez]

Yes, you can't recover from transport initialization errors except by
fixing the errors and restarting the server.

On Tue, Dec 8, 2015 at 5:17 PM, Samiyuru Senarathne <samiy...@wso2.com>
wrote:

> That means, are we going to consider the failures in the above abstract
> methods as *unrecoverable* by throwing RuntimeException?
>
> On Tue, Dec 8, 2015 at 3:54 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> Change the code as follows:
>>
>> void stopTransport() { <- one of the template methods
>> stop(); <- abstract method is called
>> if (state.equals(State.STARTED)) {
>> state = State.STOPPED; <- state is changed without knowing
>> the status of stop() call
>> } else {
>> throw new IllegalStateException("Cannot stop transport " + id
>> + ". Current state: " + state);
>> }
>>
>> }
>>
>> From the stop() method above, if something fails, throw a
>> RuntimeException.
>>
>> On Tue, Dec 8, 2015 at 3:50 PM, Samiyuru Senarathne <samiy...@wso2.com>
>> wrote:
>>
>>> Hi,
>>>
>>> CarbonTransport [1] class in carbon-kernel implements template pattern
>>> and it has the following abstract methods.
>>>
>>>- protected abstract void start();
>>>- protected abstract void stop();
>>>- protected abstract void beginMaintenance();
>>>- protected abstract void endMaintenance();
>>>
>>> These abstract methods should be implemented by any class that
>>> implements a carbon transport. (For example NettyListener [2] does this).
>>> These abstract methods are called by the template methods of
>>> CarbonTransport class. These template methods maintain the state of the
>>> carbon transport as shown in the following code block.
>>>
>>> void stopTransport() { <- one of the template methods
>>> if (state.equals(State.STARTED)) {
>>> state = State.STOPPED; <- state is changed without knowing
>>> the status of stop() call
>>> } else {
>>> throw new IllegalStateException("Cannot stop transport " +
>>> id + ". Current state: " + state);
>>> }
>>> stop(); <- abstract method is called
>>> }
>>>
>>> As shown in the above code block CarbonTransport's template functions
>>> change the state of the transport without knowing the actual state of the
>>> transport after invoking the abstract function (assuming the abstract
>>> method call succeeded).
>>>
>>> IMO this behaviour can produce invalid states when the operations of the
>>> abstract method implementations do not complete as expected. IMO in order
>>> to fix this we can introduce an exception type to the abstract method's
>>> signature for the implementations to notify any failure or add a state
>>> return to the signature.
>>>
>>>- protected abstract void start() throws CarbonTransportException;
>>>- protected abstract void stop() throws CarbonTransportException;
>>>- protected abstract void beginMaintenance()
>>> throws CarbonTransportException;
>>>- protected abstract void endMaintenance()
>>> throws CarbonTransportException;
>>>
>>> Or
>>>
>>>- protected abstract State start();
>>>- protected abstract State stop();
>>>- protected abstract State beginMaintenance();
>>>- protected abstract State endMaintenance();
>>>
>>>
>>> WDYT?
>>>
>>> [1] -
>>> https://github.com/wso2/carbon-kernel/blob/master/core/src/main/java/org/wso2/carbon/kernel/transports/CarbonTransport.java
>>> [2] -
>>> https://github.com/wso2/carbon-transports/blob/master/http/netty/component/src/main/java/org/wso2/carbon/transport/http/netty/listener/NettyListener.java
>>>
>>> Thank you.
>>>
>>> Best Regards,
>>> Samiyuru
>>>
>>> --
>>> Samiyuru Senarathne
>>> *Software Engineer*
>>> Mobile : +94 (0) 71 134 6087
>>> samiy...@wso2.com
>>>
>>
>>
>>
>> --
>> *Afkham Azeez*
>> Director of Architecture; WSO2, Inc.; http://wso2.com
>> Member; Apache Software Foundation; http://www.apache.org/
>> * <http://www.apache.org/>*
>> *email: **az...@wso2.com* <az...@wso2.com>
>> * cell: +94 77 3320919 <%2B94%2077%203320919>blog: *
>> *http://blog.a

Re: [Architecture] Usage of OSGi based server mode in WSO2 MSS

[Afkham Azeez]

We need it for WSO2 features which expose MSS services.

On Thu, Dec 3, 2015 at 9:53 AM, Samiyuru Senarathne <samiy...@wso2.com>
wrote:

> Hi,
>
> What are the use cases of running WSO2 MSS in OSGi mode as a usual carbon
> based product?
>
> If it is only the usages of microservices deployer, (assuming there is no
> any need of installing features on MSS), we can just make MSS run as a
> server with the deployer that works like a usual carbon based product but
> without OSGi. With this approach we can achieve same lightweightness and
> performance as in the standalone lightweight uber jar mode of MSS [1].
> Further, if we can take this approach, the complexities such as the need of
> SPI-fly will also go away. Unless with OSGi, MSS startup time, complexity
> and the bulkiness get increased.
>
> WDYT?
>
> [1] - https://github.com/wso2/product-mss (Performance in README.md)
>
> Thank you.
>
> Best Regards,
> Samiyuru
> --
> Samiyuru Senarathne
> *Software Engineer*
> Mobile : +94 (0) 71 134 6087
> samiy...@wso2.com
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Specifying the tenant domain in the request URI for the cross-tenant scenarois in API Manager REST API

[Afkham Azeez]

>>> let me try to understand:
>>>>>
>>>>> A user (us...@wso2.com) wants to access a resource created by a
>>>>> certain tenant (cloud.com) - correct?  The tenant created the
>>>>> resource and we decided to make the tenant-id part of the context of the
>>>>> resource's URL.
>>>>>
>>>>> Thus, the user who wants to access the tenant's resource is a
>>>>> parameter of some sort of (new?) API that requests access to the tenant's
>>>>> resource.
>>>>>
>>>>> This (new?) API would look something like this:
>>>>>
>>>>> //{tenant}//=us...@wso2.com
>>>>>
>>>>> Does it make sense or did I miss the discussion?
>>>>>
>>>>>
>>>>>
>>>>> Best regards,
>>>>> Frank
>>>>>
>>>>> 2015-11-17 16:16 GMT+01:00 Sanjeewa Malalgoda <sanje...@wso2.com>:
>>>>>
>>>>>> If we think this very carefully requested tenant domain should be
>>>>>> part of the API.
>>>>>> Let me explain it,
>>>>>> We have API and that behave in some manner according to provided
>>>>>> parameters and inputs.
>>>>>> In this particular scenario, we are passing completely different data
>>>>>> set and get different results.
>>>>>> So cant we take this as user input for API? If we added that as
>>>>>> parameter then it will be part of API(non mandatory parameter).
>>>>>>
>>>>>> Or else we may need to have ability of setting custom fields in
>>>>>> carbon context as that can be accessible across all places of started 
>>>>>> flow.
>>>>>> But according to current implementation we cannot place anything in
>>>>>> carbon context.
>>>>>> With tenant app sharing concept we need to have logged in tenant and
>>>>>> requested tenant both in message context to give proper results.
>>>>>> Shouldn't we have that capability in carbon context?
>>>>>>
>>>>>> Thanks,
>>>>>> sanjeewa.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Nov 16, 2015 at 2:20 PM, Malintha Amarasinghe <
>>>>>> malint...@wso2.com> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> In API Manager REST API, we will be supporting cross tenant
>>>>>>> scenarios. Few examples would be like below.
>>>>>>>
>>>>>>> 1. us...@wso2.com wants to see the APIs that are available in the
>>>>>>> tenant store; cloud.com
>>>>>>> 2. us...@wso2.com wants to see the documents of an API weatherAPI
>>>>>>> that are available in the tenant store; cloud.com
>>>>>>>
>>>>>>> For this requirement we need to pass the "required tenant domain"
>>>>>>> parameter with the request as the logged in user belongs to is a 
>>>>>>> different
>>>>>>> tenant. There are several possible ways to specify this.
>>>>>>>
>>>>>>> 1. Specify this as a query parameter for every resource this is
>>>>>>> required
>>>>>>>
>>>>>>> ex: GET 
>>>>>>> http://127.0.0.1:9763/api/am/publisher/v1/apis?*tenantDomain=cloud.com
>>>>>>> <http://cloud.com>*
>>>>>>>
>>>>>>>
>>>>>>> 2. Specify this as a header parameter
>>>>>>>
>>>>>>> 3. Specify this within the URI.
>>>>>>>
>>>>>>> a. before the Jax-RS web app context
>>>>>>>
>>>>>>> ex: GET http://127.0.0.1:9763/*t/cloud.com <http://cloud.com>*
>>>>>>> /api/am/publisher/v1/apis
>>>>>>>
>>>>>>> b. after the Jax-RS web app context
>>>>>>>
>>>>>>> ex: GET http://127.0.0.1:9763/api/am/publisher/v1/*t/cloud.com
>>>>>>> <http://cloud.com>*/apis
>>>>>>>
>>>>>>>
>>>>>>> Regarding these three options, option (3.a) would require the web
>>>>>>> app to be deployed on each and every tenant. Or otherwise we should
>>>>>>> manually register contexts in tomcat run-time to match those contexts. 
>>>>>>> Even
>>>>>>> if we register context manually, this would not be a good solution as 
>>>>>>> the
>>>>>>> user might expect the web app to be in the tenant space (when he sees 
>>>>>>> the
>>>>>>> /t/tenant part in the URI) but it is not there actually. Option (3.b) 
>>>>>>> would
>>>>>>> require the web app to handle the tenant as a resource and every other
>>>>>>> resource would need to be a sub-resource. And we need to specifically
>>>>>>> handle for the super tenant which we don't specify the tenant in the 
>>>>>>> URL.
>>>>>>> For options (1) and (2), we don't need to face such difficulties.
>>>>>>>
>>>>>>> Please share your thoughts.
>>>>>>>
>>>>>>> Thanks.
>>>>>>> Malintha
>>>>>>>
>>>>>>> --
>>>>>>> Malintha Amarasinghe
>>>>>>> Software Engineer
>>>>>>> *WSO2, Inc. - lean | enterprise | middleware*
>>>>>>> http://wso2.com/
>>>>>>>
>>>>>>> Mobile : +94 712383306
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *Sanjeewa Malalgoda*
>>>>>> WSO2 Inc.
>>>>>> Mobile : +94713068779
>>>>>>
>>>>>> <http://sanjeewamalalgoda.blogspot.com/>blog
>>>>>> :http://sanjeewamalalgoda.blogspot.com/
>>>>>> <http://sanjeewamalalgoda.blogspot.com/>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>>
>>> --
>>> Malintha Amarasinghe
>>> Software Engineer
>>> *WSO2, Inc. - lean | enterprise | middleware*
>>> http://wso2.com/
>>>
>>> Mobile : +94 712383306
>>>
>>
>>
>>
>> --
>> Nuwan Dias
>>
>> Technical Lead - WSO2, Inc. http://wso2.com
>> email : nuw...@wso2.com
>> Phone : +94 777 775 729
>>
>
>


-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

Re: [Architecture] Move away from XML to YAML config files

[Afkham Azeez]

Moving from XML to YAML is not simply fashion motivated. From time to time
we have heard complaints from admins who setup & configure products in
production using simple editors such as vi. Working with XML can be a real
pain for such users. These config changes will be introduced in the Carbon
5 based products, which will anyway have a significant migration cost. For
the Carbon 4.x based family of products, we will retain the XML based
configurations. Also like you said, we will rely more on sensible defaults,
which is what we have done with the Carbon 5 based WSO2 MSS (Microservices
Server), where providing the config file is optional.

Thanks
Azeez

On Thu, Nov 26, 2015 at 7:52 PM, Cyril Rognon <cyril.rog...@emoxa.fr> wrote:

> Hi all
>
> I do not like or dislike xml nor yaml but I do not trust fashion only
> motivated changes .
>
> If the main motivation to move to yaml is an outdated feeling with xml it
> seems a high price to pay to handle migration and the loss of validation.
>
> Could we offer a simple way to modify conf or lower the conf requirements
> without changing the implementation?
>
> My .2 cents.
>
> Cheers,
> Cyril
> Le 26 nov. 2015 15:01, "Frank Leymann" <fr...@wso2.com> a écrit :
>
>> +1
>>
>> I love to use YAML because of its simplicity, although I hate to give up
>> schema validation that XSD brings to the table.  But (as far as I know) you
>> can't have both.
>>
>>
>>
>> Best regards,
>> Frank
>>
>> 2015-11-26 7:14 GMT+01:00 Srinath Perera <srin...@wso2.com>:
>>
>>> Hi Azeez,
>>>
>>> Way I see it we should eventually move everything. IMO, Text based
>>> config files are much easier to deal with than XML.
>>>
>>> However, at the same time we should try to keep the advantages from
>>> previous solutions ( like detecting errors).
>>>
>>> --Srinath
>>>
>>> On Thu, Nov 26, 2015 at 11:35 AM, Ramith Jayasinghe <ram...@wso2.com>
>>> wrote:
>>>
>>>> " The Java developers will continue to work with those configurations."
>>>>  -im not clear on this. does that mean configurations file such has
>>>> identity.xml , apimanager.xml will continue to exist? or there will be a
>>>> yaml version of it too?
>>>>
>>>> On Thu, Nov 26, 2015 at 11:29 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>>
>>>>> I think you are still thinking like a typical Java developer who is
>>>>> happy to write web.xml and other XML files. We are not going to change 
>>>>> that
>>>>> experience. The Java developers will continue to work with those
>>>>> configurations. However, the YAML experience is for people who run the
>>>>> systems in production. They are not developers. They typically hate XML
>>>>> config because it is too verbose and makes life difficult when they are
>>>>> working with editors such as vi. They are very happy with simple text 
>>>>> based
>>>>> configurations. Also there scripting languages that rely on indentation. 
>>>>> So
>>>>> we need to start thinking beyond a typical Java development experience.
>>>>>
>>>>> On Thu, Nov 26, 2015 at 11:00 AM, Ramith Jayasinghe <ram...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> My point is that, wouldn't that mess up the user-experience to a new
>>>>>> level!. Do we have an idea on how comfortable the average user is with 
>>>>>> YAML
>>>>>> ( - when they are suppose to configure fairly complicated 
>>>>>> configurations).
>>>>>> Yes we can validate (- in fact we need to when we can regardless
>>>>>> weather its xml or yaml being used) then the user experience would be
>>>>>> 'server didn't start up because I couldn't indent a space correctly - to 
>>>>>> me
>>>>>> that's bad user experience).
>>>>>>
>>>>>>
>>>>>> On Thu, Nov 26, 2015 at 10:42 AM, Srinath Perera <srin...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I also think that if we use YAML, we should do some work to validate
>>>>>>> the configurations and complain. That will fix the case of single space
>>>>>>> mess up everything.
>>>>>>>
>>>>>>> Thanks
>>&

Re: [Architecture] Move away from XML to YAML config files

[Afkham Azeez]

Who configures identity.xml & apimanager.xml in production systems? The
DevOps or Systems personnel, so yes, they will have to be in YAML or some
other simple text format.

On Thu, Nov 26, 2015 at 11:35 AM, Ramith Jayasinghe <ram...@wso2.com> wrote:

> " The Java developers will continue to work with those configurations."
>  -im not clear on this. does that mean configurations file such has
> identity.xml , apimanager.xml will continue to exist? or there will be a
> yaml version of it too?
>
> On Thu, Nov 26, 2015 at 11:29 AM, Afkham Azeez <az...@wso2.com> wrote:
>
>> I think you are still thinking like a typical Java developer who is happy
>> to write web.xml and other XML files. We are not going to change that
>> experience. The Java developers will continue to work with those
>> configurations. However, the YAML experience is for people who run the
>> systems in production. They are not developers. They typically hate XML
>> config because it is too verbose and makes life difficult when they are
>> working with editors such as vi. They are very happy with simple text based
>> configurations. Also there scripting languages that rely on indentation. So
>> we need to start thinking beyond a typical Java development experience.
>>
>> On Thu, Nov 26, 2015 at 11:00 AM, Ramith Jayasinghe <ram...@wso2.com>
>> wrote:
>>
>>> My point is that, wouldn't that mess up the user-experience to a new
>>> level!. Do we have an idea on how comfortable the average user is with YAML
>>> ( - when they are suppose to configure fairly complicated configurations).
>>> Yes we can validate (- in fact we need to when we can regardless weather
>>> its xml or yaml being used) then the user experience would be 'server
>>> didn't start up because I couldn't indent a space correctly - to me that's
>>> bad user experience).
>>>
>>>
>>> On Thu, Nov 26, 2015 at 10:42 AM, Srinath Perera <srin...@wso2.com>
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I also think that if we use YAML, we should do some work to validate
>>>> the configurations and complain. That will fix the case of single space
>>>> mess up everything.
>>>>
>>>> Thanks
>>>> Srinath
>>>>
>>>> On Mon, Nov 16, 2015 at 10:18 PM, Harsha Thirimanna <hars...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Azeez,
>>>>> Since we are normally writing XSD for each xml config files and then
>>>>> we could validate it against when the relevant bundle getting activated. 
>>>>> As
>>>>> in my understand, this is valid case for each product.
>>>>> Are there any way to do this with YAML, JSON or do we have any other
>>>>> aspect like doing well document about the config files and its values ?
>>>>>
>>>>>
>>>>> *Harsha Thirimanna*
>>>>> Senior Software Engineer; WSO2, Inc.; http://wso2.com
>>>>> * <http://www.apache.org/>*
>>>>> *email: **hars...@wso2.com* <az...@wso2.com>* cell: +94 71 5186770 *
>>>>> *twitter: **http://twitter.com/ <http://twitter.com/afkham_azeez>*
>>>>> *harshathirimannlinked-in: **http:
>>>>> <http://lk.linkedin.com/in/afkhamazeez>**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122
>>>>> <http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122>*
>>>>>
>>>>> *Lean . Enterprise . Middleware*
>>>>>
>>>>>
>>>>> On Thu, Nov 12, 2015 at 2:11 PM, Maninda Edirisooriya <
>>>>> mani...@wso2.com> wrote:
>>>>>
>>>>>> From this major release we can think about managing configurations
>>>>>> from a single component. At the moment when someone wants to add a config
>>>>>> file it is just added and reading these configs are done with a 
>>>>>> boilerplate
>>>>>> code. Some issues we get are due to unavailability of config files and
>>>>>> config contents in them. If we can add a component that reads
>>>>>> configurations in the conf directory and validate at the server startup
>>>>>> most of these issues will not be coming. I think it is okay to keed
>>>>>> different formats like YAML, XML, and JSON as configs but we should
>>>>>> validate them in a dedicated component at server startup. Each component
>>>>>> that adds config 

Re: [Architecture] Move away from XML to YAML config files

[Afkham Azeez]

I think you are still thinking like a typical Java developer who is happy
to write web.xml and other XML files. We are not going to change that
experience. The Java developers will continue to work with those
configurations. However, the YAML experience is for people who run the
systems in production. They are not developers. They typically hate XML
config because it is too verbose and makes life difficult when they are
working with editors such as vi. They are very happy with simple text based
configurations. Also there scripting languages that rely on indentation. So
we need to start thinking beyond a typical Java development experience.

On Thu, Nov 26, 2015 at 11:00 AM, Ramith Jayasinghe <ram...@wso2.com> wrote:

> My point is that, wouldn't that mess up the user-experience to a new
> level!. Do we have an idea on how comfortable the average user is with YAML
> ( - when they are suppose to configure fairly complicated configurations).
> Yes we can validate (- in fact we need to when we can regardless weather
> its xml or yaml being used) then the user experience would be 'server
> didn't start up because I couldn't indent a space correctly - to me that's
> bad user experience).
>
>
> On Thu, Nov 26, 2015 at 10:42 AM, Srinath Perera <srin...@wso2.com> wrote:
>
>> Hi All,
>>
>> I also think that if we use YAML, we should do some work to validate the
>> configurations and complain. That will fix the case of single space mess up
>> everything.
>>
>> Thanks
>> Srinath
>>
>> On Mon, Nov 16, 2015 at 10:18 PM, Harsha Thirimanna <hars...@wso2.com>
>> wrote:
>>
>>> Hi Azeez,
>>> Since we are normally writing XSD for each xml config files and then we
>>> could validate it against when the relevant bundle getting activated. As in
>>> my understand, this is valid case for each product.
>>> Are there any way to do this with YAML, JSON or do we have any other
>>> aspect like doing well document about the config files and its values ?
>>>
>>>
>>> *Harsha Thirimanna*
>>> Senior Software Engineer; WSO2, Inc.; http://wso2.com
>>> * <http://www.apache.org/>*
>>> *email: **hars...@wso2.com* <az...@wso2.com>* cell: +94 71 5186770 *
>>> *twitter: **http://twitter.com/ <http://twitter.com/afkham_azeez>*
>>> *harshathirimannlinked-in: **http:
>>> <http://lk.linkedin.com/in/afkhamazeez>**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122
>>> <http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122>*
>>>
>>> *Lean . Enterprise . Middleware*
>>>
>>>
>>> On Thu, Nov 12, 2015 at 2:11 PM, Maninda Edirisooriya <mani...@wso2.com>
>>> wrote:
>>>
>>>> From this major release we can think about managing configurations from
>>>> a single component. At the moment when someone wants to add a config file
>>>> it is just added and reading these configs are done with a boilerplate
>>>> code. Some issues we get are due to unavailability of config files and
>>>> config contents in them. If we can add a component that reads
>>>> configurations in the conf directory and validate at the server startup
>>>> most of these issues will not be coming. I think it is okay to keed
>>>> different formats like YAML, XML, and JSON as configs but we should
>>>> validate them in a dedicated component at server startup. Each component
>>>> that adds config fies should implement the validation interface exposed by
>>>> that component and that componet can provide utility classes to support
>>>> parsing each format easily from the implented classes.
>>>>
>>>>
>>>> On Thu, Nov 12, 2015 at 10:45 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>>
>>>>> The product/component configs have to be rethought as part of
>>>>> rewriting them or improving them. The target audience of product config
>>>>> files as devops or admin folks. That community prefers simple text 
>>>>> formats,
>>>>> and given an alternative to XML, they will take it.
>>>>>
>>>>> On Wed, Nov 11, 2015 at 9:02 PM, Ramith Jayasinghe <ram...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> I'm not against Yaml or jason.
>>>>>>
>>>>>> @Sagara,
>>>>>>  your point also holds for XML ( we don't need to educate people on
>>>>>> how to use XML). In my view it boils down to number of configs we would
&g

Re: [Architecture] Move away from XML to YAML config files

[Afkham Azeez]

http://lmgtfy.com/?q=frameworks+using+YAML+config

On Thu, Nov 26, 2015 at 12:43 PM, Ramith Jayasinghe <ram...@wso2.com> wrote:

> examples?
>
> On Thu, Nov 26, 2015 at 12:22 PM, Afkham Azeez <az...@wso2.com> wrote:
>
>> The world has already moved. Even developers are very comfortable with
>> less verbose textual configurations. Check out any new framework. Everybody
>> prefers less verbose textual formats.
>>
>> On Thu, Nov 26, 2015 at 12:09 PM, Ramith Jayasinghe <ram...@wso2.com>
>> wrote:
>>
>>> Who will spend more time (interms of iterations and hours) changing
>>> configuration files? developer or devops guy?
>>> Who will more likely to make a mistake (due to different skill levels
>>> etc ) while changing a value ? a developer ( on his laptop) or a devOp ( in
>>> a production system)
>>> Who are more likely to evaluate our products (on functionality) ? a
>>> developer or a devop guy? (I think most of the time its a developer who
>>> will evaluate and recommend and devOps guy get to install our product in
>>> production system.).
>>>
>>> My point is, you are trying to cater to expert (and subset of users) and
>>> alienate all other users (which is likely bigger in numbers than number of
>>> devops involved in a project).
>>> thoughts?
>>> ,
>>>
>>>
>>> On Thu, Nov 26, 2015 at 11:49 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> Who configures identity.xml & apimanager.xml in production systems? The
>>>> DevOps or Systems personnel, so yes, they will have to be in YAML or some
>>>> other simple text format.
>>>>
>>>> On Thu, Nov 26, 2015 at 11:35 AM, Ramith Jayasinghe <ram...@wso2.com>
>>>> wrote:
>>>>
>>>>> " The Java developers will continue to work with those
>>>>> configurations."
>>>>>  -im not clear on this. does that mean configurations file such has
>>>>> identity.xml , apimanager.xml will continue to exist? or there will be a
>>>>> yaml version of it too?
>>>>>
>>>>> On Thu, Nov 26, 2015 at 11:29 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>>>
>>>>>> I think you are still thinking like a typical Java developer who is
>>>>>> happy to write web.xml and other XML files. We are not going to change 
>>>>>> that
>>>>>> experience. The Java developers will continue to work with those
>>>>>> configurations. However, the YAML experience is for people who run the
>>>>>> systems in production. They are not developers. They typically hate XML
>>>>>> config because it is too verbose and makes life difficult when they are
>>>>>> working with editors such as vi. They are very happy with simple text 
>>>>>> based
>>>>>> configurations. Also there scripting languages that rely on indentation. 
>>>>>> So
>>>>>> we need to start thinking beyond a typical Java development experience.
>>>>>>
>>>>>> On Thu, Nov 26, 2015 at 11:00 AM, Ramith Jayasinghe <ram...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> My point is that, wouldn't that mess up the user-experience to a new
>>>>>>> level!. Do we have an idea on how comfortable the average user is with 
>>>>>>> YAML
>>>>>>> ( - when they are suppose to configure fairly complicated 
>>>>>>> configurations).
>>>>>>> Yes we can validate (- in fact we need to when we can regardless
>>>>>>> weather its xml or yaml being used) then the user experience would be
>>>>>>> 'server didn't start up because I couldn't indent a space correctly - 
>>>>>>> to me
>>>>>>> that's bad user experience).
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Nov 26, 2015 at 10:42 AM, Srinath Perera <srin...@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I also think that if we use YAML, we should do some work to
>>>>>>>> validate the configurations and complain. That will fix the case of 
>>>>>>>> single
>>>>>>>> space mess up everything.
>>>>>>>>
>>

Re: [Architecture] Move away from XML to YAML config files

[Afkham Azeez]

The world has already moved. Even developers are very comfortable with less
verbose textual configurations. Check out any new framework. Everybody
prefers less verbose textual formats.

On Thu, Nov 26, 2015 at 12:09 PM, Ramith Jayasinghe <ram...@wso2.com> wrote:

> Who will spend more time (interms of iterations and hours) changing
> configuration files? developer or devops guy?
> Who will more likely to make a mistake (due to different skill levels etc
> ) while changing a value ? a developer ( on his laptop) or a devOp ( in a
> production system)
> Who are more likely to evaluate our products (on functionality) ? a
> developer or a devop guy? (I think most of the time its a developer who
> will evaluate and recommend and devOps guy get to install our product in
> production system.).
>
> My point is, you are trying to cater to expert (and subset of users) and
> alienate all other users (which is likely bigger in numbers than number of
> devops involved in a project).
> thoughts?
> ,
>
>
> On Thu, Nov 26, 2015 at 11:49 AM, Afkham Azeez <az...@wso2.com> wrote:
>
>> Who configures identity.xml & apimanager.xml in production systems? The
>> DevOps or Systems personnel, so yes, they will have to be in YAML or some
>> other simple text format.
>>
>> On Thu, Nov 26, 2015 at 11:35 AM, Ramith Jayasinghe <ram...@wso2.com>
>> wrote:
>>
>>> " The Java developers will continue to work with those configurations."
>>>  -im not clear on this. does that mean configurations file such has
>>> identity.xml , apimanager.xml will continue to exist? or there will be a
>>> yaml version of it too?
>>>
>>> On Thu, Nov 26, 2015 at 11:29 AM, Afkham Azeez <az...@wso2.com> wrote:
>>>
>>>> I think you are still thinking like a typical Java developer who is
>>>> happy to write web.xml and other XML files. We are not going to change that
>>>> experience. The Java developers will continue to work with those
>>>> configurations. However, the YAML experience is for people who run the
>>>> systems in production. They are not developers. They typically hate XML
>>>> config because it is too verbose and makes life difficult when they are
>>>> working with editors such as vi. They are very happy with simple text based
>>>> configurations. Also there scripting languages that rely on indentation. So
>>>> we need to start thinking beyond a typical Java development experience.
>>>>
>>>> On Thu, Nov 26, 2015 at 11:00 AM, Ramith Jayasinghe <ram...@wso2.com>
>>>> wrote:
>>>>
>>>>> My point is that, wouldn't that mess up the user-experience to a new
>>>>> level!. Do we have an idea on how comfortable the average user is with 
>>>>> YAML
>>>>> ( - when they are suppose to configure fairly complicated configurations).
>>>>> Yes we can validate (- in fact we need to when we can regardless
>>>>> weather its xml or yaml being used) then the user experience would be
>>>>> 'server didn't start up because I couldn't indent a space correctly - to 
>>>>> me
>>>>> that's bad user experience).
>>>>>
>>>>>
>>>>> On Thu, Nov 26, 2015 at 10:42 AM, Srinath Perera <srin...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I also think that if we use YAML, we should do some work to validate
>>>>>> the configurations and complain. That will fix the case of single space
>>>>>> mess up everything.
>>>>>>
>>>>>> Thanks
>>>>>> Srinath
>>>>>>
>>>>>> On Mon, Nov 16, 2015 at 10:18 PM, Harsha Thirimanna <hars...@wso2.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hi Azeez,
>>>>>>> Since we are normally writing XSD for each xml config files and then
>>>>>>> we could validate it against when the relevant bundle getting 
>>>>>>> activated. As
>>>>>>> in my understand, this is valid case for each product.
>>>>>>> Are there any way to do this with YAML, JSON or do we have any other
>>>>>>> aspect like doing well document about the config files and its values ?
>>>>>>>
>>>>>>>
>>>>>>> *Harsha Thirimanna*
>>>>>>> Senior Software Engineer; WSO2, Inc.; http://wso2.com

Re: [Architecture] Move away from XML to YAML config files

[Afkham Azeez]

Yes I agree. I was saying there are traditional files like web.xml which
will continue to be in XML.

On Thu, Nov 26, 2015 at 11:44 AM, Srinath Perera <srin...@wso2.com> wrote:

> Hi Azeez,
>
> Way I see it we should eventually move everything. IMO, Text based config
> files are much easier to deal with than XML.
>
> However, at the same time we should try to keep the advantages from
> previous solutions ( like detecting errors).
>
> --Srinath
>
> On Thu, Nov 26, 2015 at 11:35 AM, Ramith Jayasinghe <ram...@wso2.com>
> wrote:
>
>> " The Java developers will continue to work with those configurations."
>>  -im not clear on this. does that mean configurations file such has
>> identity.xml , apimanager.xml will continue to exist? or there will be a
>> yaml version of it too?
>>
>> On Thu, Nov 26, 2015 at 11:29 AM, Afkham Azeez <az...@wso2.com> wrote:
>>
>>> I think you are still thinking like a typical Java developer who is
>>> happy to write web.xml and other XML files. We are not going to change that
>>> experience. The Java developers will continue to work with those
>>> configurations. However, the YAML experience is for people who run the
>>> systems in production. They are not developers. They typically hate XML
>>> config because it is too verbose and makes life difficult when they are
>>> working with editors such as vi. They are very happy with simple text based
>>> configurations. Also there scripting languages that rely on indentation. So
>>> we need to start thinking beyond a typical Java development experience.
>>>
>>> On Thu, Nov 26, 2015 at 11:00 AM, Ramith Jayasinghe <ram...@wso2.com>
>>> wrote:
>>>
>>>> My point is that, wouldn't that mess up the user-experience to a new
>>>> level!. Do we have an idea on how comfortable the average user is with YAML
>>>> ( - when they are suppose to configure fairly complicated configurations).
>>>> Yes we can validate (- in fact we need to when we can regardless
>>>> weather its xml or yaml being used) then the user experience would be
>>>> 'server didn't start up because I couldn't indent a space correctly - to me
>>>> that's bad user experience).
>>>>
>>>>
>>>> On Thu, Nov 26, 2015 at 10:42 AM, Srinath Perera <srin...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> I also think that if we use YAML, we should do some work to validate
>>>>> the configurations and complain. That will fix the case of single space
>>>>> mess up everything.
>>>>>
>>>>> Thanks
>>>>> Srinath
>>>>>
>>>>> On Mon, Nov 16, 2015 at 10:18 PM, Harsha Thirimanna <hars...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Azeez,
>>>>>> Since we are normally writing XSD for each xml config files and then
>>>>>> we could validate it against when the relevant bundle getting activated. 
>>>>>> As
>>>>>> in my understand, this is valid case for each product.
>>>>>> Are there any way to do this with YAML, JSON or do we have any other
>>>>>> aspect like doing well document about the config files and its values ?
>>>>>>
>>>>>>
>>>>>> *Harsha Thirimanna*
>>>>>> Senior Software Engineer; WSO2, Inc.; http://wso2.com
>>>>>> * <http://www.apache.org/>*
>>>>>> *email: **hars...@wso2.com* <az...@wso2.com>* cell: +94 71 5186770 *
>>>>>> *twitter: **http://twitter.com/ <http://twitter.com/afkham_azeez>*
>>>>>> *harshathirimannlinked-in: **http:
>>>>>> <http://lk.linkedin.com/in/afkhamazeez>**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122
>>>>>> <http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122>*
>>>>>>
>>>>>> *Lean . Enterprise . Middleware*
>>>>>>
>>>>>>
>>>>>> On Thu, Nov 12, 2015 at 2:11 PM, Maninda Edirisooriya <
>>>>>> mani...@wso2.com> wrote:
>>>>>>
>>>>>>> From this major release we can think about managing configurations
>>>>>>> from a single component. At the moment when someone wants to add a 
>>>>>>> config
>>>>>>> file it is just added and reading these configs are done with a 
>>>

Re: [Architecture] Monitoring Microservices in WSO2 MSS

[Afkham Azeez]

TTP request headers from Netty in order to render some parts of
>>>>> above dashboard, IMHO we need to find a way to exact all HTTP headers from
>>>>> Netty.
>>>>>
>>>>> Can you list down list of required HTTP headers that we can't extract
>>>>> from current API ?  May be Kasun can help out here.
>>>>>
>>>>> Thanks !
>>>>>
>>>>> On Fri, Nov 20, 2015 at 4:05 PM, Isuru Perera <isu...@wso2.com> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> For the MSS alpha release, we used the HTTP Monitoring Dashboard used
>>>>>> in WSO2 AS product and we implemented an HTTP Monitoring Interceptor for
>>>>>> MSS [1].
>>>>>>
>>>>>> However, with MSS we don't get all data required by the standard HTTP
>>>>>> Monitoring Dashboard. For example, in the Tomcat Valve for AS [2], we 
>>>>>> have
>>>>>> used the User-Agent header to get client details (Browser, OS & Device).
>>>>>> The Tomcat Request object has all other necessary details (Remote IP,
>>>>>> Locale etc)
>>>>>>
>>>>>> In our interceptor [3], we use the Netty HTTP Request object and the
>>>>>> Java Reflection details to get details for the monitoring event.
>>>>>>
>>>>>> Since these details are not enough for the HTTP Monitoring Dashboard,
>>>>>> we need to think of using an MSS specific dashboard.
>>>>>>
>>>>>> WDYT?
>>>>>>
>>>>>> With regarding to Metrics annotations, we have to create some gadgets
>>>>>> in DAS. We need to have a discussion on this.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> Best Regards,
>>>>>>
>>>>>> [1] https://github.com/wso2/product-mss/pull/31
>>>>>> [2]
>>>>>> https://github.com/wso2/carbon-deployment/blob/master/components/monitoring/org.wso2.carbon.as.monitoring/src/main/java/org/wso2/carbon/as/monitoring/collector/http/WebAppMonitoringPublisherValve.java
>>>>>> [3]
>>>>>> https://github.com/wso2/product-mss/blob/v1.0.0-alpha/carbon-mss/components/org.wso2.carbon.mss/src/main/java/org/wso2/carbon/mss/httpmonitoring/HTTPMonitoringInterceptor.java
>>>>>>
>>>>>> --
>>>>>> Isuru Perera
>>>>>> Associate Technical Lead | WSO2, Inc. | http://wso2.com/
>>>>>> Lean . Enterprise . Middleware
>>>>>>
>>>>>> about.me/chrishantha
>>>>>> Contact: +IsuruPereraWSO2
>>>>>> <https://www.google.com/+IsuruPereraWSO2/about>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sagara Gunathunga
>>>>>
>>>>> Architect; WSO2, Inc.;  http://wso2.com
>>>>> V.P Apache Web Services;http://ws.apache.org/
>>>>> Linkedin; http://www.linkedin.com/in/ssagara
>>>>> Blog ;  http://ssagara.blogspot.com
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Isuru Perera
>>>> Associate Technical Lead | WSO2, Inc. | http://wso2.com/
>>>> Lean . Enterprise . Middleware
>>>>
>>>> about.me/chrishantha
>>>> Contact: +IsuruPereraWSO2
>>>> <https://www.google.com/+IsuruPereraWSO2/about>
>>>>
>>>
>>>
>>>
>>> --
>>> /sumedha
>>> m: +94 773017743
>>> b :  bit.ly/sumedha
>>>
>>
>>
>>
>> --
>> Sagara Gunathunga
>>
>> Architect; WSO2, Inc.;  http://wso2.com
>> V.P Apache Web Services;http://ws.apache.org/
>> Linkedin; http://www.linkedin.com/in/ssagara
>> Blog ;  http://ssagara.blogspot.com
>>
>>
>
>
> --
> Isuru Perera
> Associate Technical Lead | WSO2, Inc. | http://wso2.com/
> Lean . Enterprise . Middleware
>
> about.me/chrishantha
> Contact: +IsuruPereraWSO2 <https://www.google.com/+IsuruPereraWSO2/about>
>



-- 
*Afkham Azeez*
Director of Architecture; WSO2, Inc.; http://wso2.com
Member; Apache Software Foundation; http://www.apache.org/
* <http://www.apache.org/>*
*email: **az...@wso2.com* <az...@wso2.com>
* cell: +94 77 3320919blog: **http://blog.afkham.org*
<http://blog.afkham.org>
*twitter: **http://twitter.com/afkham_azeez*
<http://twitter.com/afkham_azeez>
*linked-in: **http://lk.linkedin.com/in/afkhamazeez
<http://lk.linkedin.com/in/afkhamazeez>*

*Lean . Enterprise . Middleware*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture