Re: [Architecture] [Iam-dev] [VOTE] Release WSO2 Identity Server 5.10.0 RC2
Hi All, Tested followings, 1. OIDC scope REST APIs 2. OAuth2 scope REST APIs. No Blockers found. [+] Stable - go ahead and release. On Sun, Mar 8, 2020 at 11:26 PM Janak Amarasena wrote: > Hi all, > > We are pleased to announce the second release candidate of WSO2 Identity > Server 5.10.0. > > > *New Features:* > >1. Passwordless authentication support >2. An improved User Portal >3. New RESTful APIs for user self-services and server management >4. Scope based authorization for internal REST APIs >5. Unique User ID support >6. Tenant wise email-sender configuration > > > > *Fixes:* > This release includes the following issue fixes and improvements: > >- 5.10.0-M1 <https://github.com/wso2/product-is/milestone/95?closed=1> >- 5.10.0-M2 <https://github.com/wso2/product-is/milestone/96?closed=1> >- 5.10.0-M3 <https://github.com/wso2/product-is/milestone/97?closed=1> >- 5.10.0-M4 <https://github.com/wso2/product-is/milestone/98?closed=1> >- 5.10.0-M5 <https://github.com/wso2/product-is/milestone/99?closed=1> >- 5.10.0-M6 <https://github.com/wso2/product-is/milestone/100?closed=1> >- 5.10.0-M7 <https://github.com/wso2/product-is/milestone/101?closed=1> >- 5.10.0-M8 <https://github.com/wso2/product-is/milestone/102?closed=1> >- 5.10.0-M9 <https://github.com/wso2/product-is/milestone/103?closed=1> >- 5.10.0-Alpha ><https://github.com/wso2/product-is/milestone/104?closed=1> >- 5.10.0-Alpha2 ><https://github.com/wso2/product-is/milestone/105?closed=1> >- 5.10.0-Alpha3 ><https://github.com/wso2/product-is/milestone/106?closed=1> >- 5.10.0-Beta ><https://github.com/wso2/product-is/milestone/107?closed=1> >- 5.10.0-Beta2 ><https://github.com/wso2/product-is/milestone/108?closed=1> >- 5.10.0-Beta3 ><https://github.com/wso2/product-is/milestone/109?closed=1> >- 5.10.0-GA <https://github.com/wso2/product-is/milestone/92?closed=1> > > > *Source and Distribution* > The source and distribution > <https://github.com/wso2/product-is/releases/download/v5.10.0-rc2/wso2is-5.10.0-rc2.zip> > are > available at https://github.com/wso2/product-is/releases/tag/v5.10.0-rc2 > > > Please download the product, test it, and vote using the following > convention. > [+] Stable - go ahead and release > [-] Broken - do not release (explain why) > > > Thank you, > WSO2 Identity and Access Management Team > > -- > *Janak Amarasena* | Senior Software Engineer | WSO2 Inc. > (m) +9464144 | (w) +94112145345 | (e) ja...@wso2.com > > > <https://wso2.com/signature> > ___ > Iam-dev mailing list > iam-...@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/iam-dev > -- *Sarubi Thillainathan* | Senior Software Engineer | WSO2 Inc. (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com *[image: https://wso2.com/signature] <https://wso2.com/signature>* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Dropping Unregistered Scope(s) from OAuth Request in IS.
Hi Johann/All, We had a couple of discussions with the team, there we decided to *not** to drop the unregistered scopes from OAuth Request in IS*. But as mentioned earlier, from IS 5.10.0, we'll be more descriptive and show the display name of the scope and it's the description as well when we are getting the consent from the user. Also, if the scope is not registered under the OAuth2 scope or OIDC scope in the IS, then we will display with the provided scope name in the consent page. Please find the corresponding improvement of PR [1]. Note in such case, scopes which are not registered will display with the provided scope name and scopes which are registered will displayed with their corresponding display name and description in the consent page. [1] https://github.com/wso2/identity-apps/pull/521 On Tue, Mar 10, 2020 at 1:54 PM Johann Nallathamby wrote: > Hi Sarubi, > > As Asela pointed out there are use cases for differentiating the access > token not just based on client or user or registered scopes but based on > other environmental attributes. The easiest way of representing these > environmental attributes in OAuth2 and getting unique access tokens in WSO2 > IS is using scopes. This is the reason why WSO2 API Manager also uses > whitelabeled scope prefixes. > > For example APIM customers using this feature to get unique access tokens > per device. They might be not able to register the devices before hand. @Nuwan > Dias and @Sanjeewa Malalgoda may be > able to comment more on this. > > Regards, > Johann. > > On Thu, Feb 13, 2020 at 5:32 PM Asela Pathberiya wrote: > >> >> >> On Thu, Feb 13, 2020 at 11:15 AM Sarubi Thillainathan >> wrote: >> >>> >>> >>> On Thu, Feb 13, 2020 at 10:50 AM Asela Pathberiya >>> wrote: >>> >>>> >>>> >>>> On Thu, Feb 13, 2020 at 10:48 AM Sarubi Thillainathan >>>> wrote: >>>> >>>>> Hi Asela, >>>>> >>>>> Just to be clear, Can we register scope values as regex patterns ? >>>>>> In APIM there is scope white listing capabilities which can be sent >>>>>> any scope value related to the given regex, "device_*" such scope. >>>>>> >>>>> Nope, in IS we don't have this capability. >>>>> The only thing that we enforce is can't have space in the scope name. >>>>> >>>> >>>> There are cases in which application needs to send some random scope to >>>> identify the devices. Can't we handle such cases by default ? >>>> >>> Yes, we can't handle such cases default. I would like to know why those >>> needs to be random? If it is for identifying the device then can't we >>> register those beforehand? >>> >> >> Just thought of similar to this [1] as we are not supporting multiple >> access token for given user/application >> >> [1] >> https://apim.docs.wso2.com/en/3.0.0/Learn/APISecurity/OAuth2/OAuth2Scopes/scope-whitelisting/ >> >> >>> >>> >>>> >>>> >>> Thanks, >>>> Asela. >>>> >>>> >>>>> Thanks, >>>>> Sarubi. >>>>> >>>>> On Wed, Feb 12, 2020 at 6:06 PM Asela Pathberiya >>>>> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Wed, Feb 12, 2020 at 5:44 PM Sarubi Thillainathan >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Feb 12, 2020 at 5:38 PM Sarubi Thillainathan < >>>>>>> sar...@wso2.com> wrote: >>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> Currently in IS, whenever a token request comes with a list of >>>>>>>> scopes we'll be showing all the scopes and get the consent from the >>>>>>>> user >>>>>>>> regardless of that scopes are requested or not in the Identity Server. >>>>>>>> But by going forward with IS 5.10.0, we'll be more descriptive and >>>>>>>> decided to show the display name of the scope and it's the description >>>>>>>> as >>>>>>>> well when we are getting the consent from the user. Also, if the scope >>>>>>>> is >>>>>>>> not registered under the OAuth2 scope or OIDC scope in the IS, then we >>>>>>>
Re: [Architecture] Dropping Unregistered Scope(s) from OAuth Request in IS.
On Thu, Feb 13, 2020 at 10:50 AM Asela Pathberiya wrote: > > > On Thu, Feb 13, 2020 at 10:48 AM Sarubi Thillainathan > wrote: > >> Hi Asela, >> >> Just to be clear, Can we register scope values as regex patterns ? >>> In APIM there is scope white listing capabilities which can be sent any >>> scope value related to the given regex, "device_*" such scope. >>> >> Nope, in IS we don't have this capability. >> The only thing that we enforce is can't have space in the scope name. >> > > There are cases in which application needs to send some random scope to > identify the devices. Can't we handle such cases by default ? > Yes, we can't handle such cases default. I would like to know why those needs to be random? If it is for identifying the device then can't we register those beforehand? > > Thanks, > Asela. > > >> Thanks, >> Sarubi. >> >> On Wed, Feb 12, 2020 at 6:06 PM Asela Pathberiya wrote: >> >>> >>> >>> On Wed, Feb 12, 2020 at 5:44 PM Sarubi Thillainathan >>> wrote: >>> >>>> >>>> >>>> >>>> On Wed, Feb 12, 2020 at 5:38 PM Sarubi Thillainathan >>>> wrote: >>>> >>>>> Hi All, >>>>> >>>>> Currently in IS, whenever a token request comes with a list of scopes >>>>> we'll be showing all the scopes and get the consent from the user >>>>> regardless of that scopes are requested or not in the Identity Server. >>>>> But by going forward with IS 5.10.0, we'll be more descriptive and >>>>> decided to show the display name of the scope and it's the description as >>>>> well when we are getting the consent from the user. Also, if the scope is >>>>> not registered under the OAuth2 scope or OIDC scope in the IS, then we >>>>> decided to skip that particular scope from the consent page also in the >>>>> response as a default behaviour. >>>>> >>>> >>> Just to be clear, Can we register scope values as regex patterns ? >>> In APIM there is scope white listing capabilities which can be sent any >>> scope value related to the given regex, "device_*" such scope. >>> >>> Thanks, >>> Asela. >>> >>> >>>> >>>>> In order to keep the backward compatibility, we'll keep a flag so that >>>>> we can enable it if we want to list the scope which is not registered. >>>>> Note >>>>> that in that case scopes which are not registered will display with the >>>>> provided scope name and scopes which are registered will displayed with >>>>> their corresponding display name and description in the consent page. >>>>> >>>>> Highly appreciate your ideas and suggestion on this. >>>>> >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> Sarubi. >>>>> -- >>>>> *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. >>>>> (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com >>>>> >>>>> *[image: https://wso2.com/signature] <https://wso2.com/signature>* >>>>> >>>> >>>> >>>> -- >>>> *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. >>>> (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com >>>> >>>> *[image: https://wso2.com/signature] <https://wso2.com/signature>* >>>> ___ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>> >>> >>> -- >>> Thanks & Regards, >>> Asela >>> >>> Mobile : +94 777 625 933 >>> >>> http://soasecurity.org/ >>> http://xacmlinfo.org/ >>> ___ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >> >> >> -- >> *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. >> (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com >> >> *[image: https://wso2.com/signature] <https://wso2.com/signature>* >> > > > -- > Thanks & Regards, > Asela > > Mobile : +94 777 625 933 > > http://soasecurity.org/ > http://xacmlinfo.org/ > -- *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com *[image: https://wso2.com/signature] <https://wso2.com/signature>* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Dropping Unregistered Scope(s) from OAuth Request in IS.
Hi Asela, Just to be clear, Can we register scope values as regex patterns ? > In APIM there is scope white listing capabilities which can be sent any > scope value related to the given regex, "device_*" such scope. > Nope, in IS we don't have this capability. The only thing that we enforce is can't have space in the scope name. Thanks, Sarubi. On Wed, Feb 12, 2020 at 6:06 PM Asela Pathberiya wrote: > > > On Wed, Feb 12, 2020 at 5:44 PM Sarubi Thillainathan > wrote: > >> >> >> >> On Wed, Feb 12, 2020 at 5:38 PM Sarubi Thillainathan >> wrote: >> >>> Hi All, >>> >>> Currently in IS, whenever a token request comes with a list of scopes >>> we'll be showing all the scopes and get the consent from the user >>> regardless of that scopes are requested or not in the Identity Server. >>> But by going forward with IS 5.10.0, we'll be more descriptive and >>> decided to show the display name of the scope and it's the description as >>> well when we are getting the consent from the user. Also, if the scope is >>> not registered under the OAuth2 scope or OIDC scope in the IS, then we >>> decided to skip that particular scope from the consent page also in the >>> response as a default behaviour. >>> >> > Just to be clear, Can we register scope values as regex patterns ? > In APIM there is scope white listing capabilities which can be sent any > scope value related to the given regex, "device_*" such scope. > > Thanks, > Asela. > > >> >>> In order to keep the backward compatibility, we'll keep a flag so that >>> we can enable it if we want to list the scope which is not registered. Note >>> that in that case scopes which are not registered will display with the >>> provided scope name and scopes which are registered will displayed with >>> their corresponding display name and description in the consent page. >>> >>> Highly appreciate your ideas and suggestion on this. >>> >>> >>> >>> >>> Thanks, >>> Sarubi. >>> -- >>> *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. >>> (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com >>> >>> *[image: https://wso2.com/signature] <https://wso2.com/signature>* >>> >> >> >> -- >> *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. >> (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com >> >> *[image: https://wso2.com/signature] <https://wso2.com/signature>* >> _______ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> > > > -- > Thanks & Regards, > Asela > > Mobile : +94 777 625 933 > > http://soasecurity.org/ > http://xacmlinfo.org/ > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > -- *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com *[image: https://wso2.com/signature] <https://wso2.com/signature>* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Dropping Unregistered Scope(s) from OAuth Request in IS.
On Wed, Feb 12, 2020 at 5:38 PM Sarubi Thillainathan wrote: > Hi All, > > Currently in IS, whenever a token request comes with a list of scopes > we'll be showing all the scopes and get the consent from the user > regardless of that scopes are requested or not in the Identity Server. > But by going forward with IS 5.10.0, we'll be more descriptive and decided > to show the display name of the scope and it's the description as well > when we are getting the consent from the user. Also, if the scope is not > registered under the OAuth2 scope or OIDC scope in the IS, then we decided > to skip that particular scope from the consent page also in the response as > a default behaviour. > > In order to keep the backward compatibility, we'll keep a flag so that we > can enable it if we want to list the scope which is not registered. Note > that in that case scopes which are not registered will display with the > provided scope name and scopes which are registered will displayed with > their corresponding display name and description in the consent page. > > Highly appreciate your ideas and suggestion on this. > > > > > Thanks, > Sarubi. > -- > *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. > (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com > > *[image: https://wso2.com/signature] <https://wso2.com/signature>* > -- *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com *[image: https://wso2.com/signature] <https://wso2.com/signature>* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] Dropping Unregistered Scope(s) from OAuth Request in IS.
Hi All, Currently in IS, whenever a token request comes with a list of scopes we'll be showing all the scopes and get the consent from the user regardless of that scopes are requested or not in the Identity Server. But by going forward with IS 5.10.0, we'll be more descriptive and decided to show the display name of the scope and it's the description as well when we are getting the consent from the user. Also, if the scope is not registered under the OAuth2 scope or OIDC scope in the IS, then we decided to skip that particular scope from the consent page also in the response as a default behaviour. In order to keep the backward compatibility, we'll keep a flag so that we can enable it if we want to list the scope which is not registered. Note that in that case scopes which are not registered will display with the provided scope name and scopes which are registered will displayed with their corresponding display name and description in the consent page. Highly appreciate your ideas and suggestion on this. Thanks, Sarubi. -- *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com *[image: https://wso2.com/signature] <https://wso2.com/signature>* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] Integrate Augmented BNF parser to Validate Input Path Value Given in SCIM2 PATCH Operation
Hi All, According to the SCIM specification for Patch Operations [1] (such as "add", "replace" & "remove"), "path" attribute is MANDATORY for "remove" and OPTIONAL for "add" and "replace". The "path" attribute value is a string which contains an attribute path describing the target of the operation. As example: "path":"name.familyName" "path":"addresses[type eq \"work\"]" This "path" attribute is described using the ABNF syntax rule as follows, PATH = attrPath / valuePath [subAttr] attrPath, valuePath and subAttr are defined under filtering Section 3.4.2.2 <https://tools.ietf.org/html/rfc7644#section-3.4.2.2>. In order to validate the path syntax and use it, we decided to use ABNF parser. So that we can easily validate the syntax without performing complex operations. I have done research on supported java ABNF parsers and found two parsers, Please find my findings below. 1. parse2 [2] This parse2 will produce the "aParse" parser generator that reads Augmented BNF grammars and produces Java, C++ or C# classes that can build parse trees for valid instances of those grammars. This is free to download and use without any obligations or limitations. I downloaded <http://www.parse2.com/download.shtml> the latest aparse-2.5.jar and generated the Java Classes using the specified ABNF rules. Please find the generated classes in [4] and find the created set of ABNF rules which use to build the parser in [5]. 2. *APG* - ABNF Parser Generator [3] This *APG* Parser is originally designed to generate recursive-descent parsers directly from the ABNF grammar. This ABNF Parser Generator is written completely in the Java language. It is released under the GNU General Public License v2.0. By using the same ABNF rules set [5] I have generated parsers in Java. Please find the parsers in [6]. By comparing both, *we decided to go with parse2 *generated parsers due to the following concerns, 1. parse2 generated java class doesn't contain any embedded parse2 class, just depends on pure java, where apg-java generated parsers need to embedded with apg-java grammar package. 2. *APG *java license (GNU General Public License v2.0.) is not compatible with our Apache License Version 2.0 license according to [7]. Due to these limitations, we selected parse2 to integrate with our implementation. Highly appreciate your feedback and input around this. [1] https://tools.ietf.org/html/rfc7644#section-3.5.2 [2] http://www.parse2.com/index.shtml [3] http://www.coasttocoastresearch.com/ [4] https://github.com/sarubi/scim-filter-parse2-parser/tree/master/src [5] https://github.com/sarubi/scim-filter-parse2-parser/blob/master/src/scim-filter-rules.abnf [6] https://github.com/sarubi/scim-filter-apg-parser/tree/master/src/scim [7] https://www.apache.org/licenses/GPL-compatibility.html Thanks, Sarubi. -- *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com *[image: https://wso2.com/signature] <https://wso2.com/signature>* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Dev] [VOTE] Release WSO2 Identity Server 5.8.0 RC3
?closed=1> >>>>>>>- 5.8.0-RC1 fixes >>>>>>><https://github.com/wso2/product-is/milestone/78?closed=1> >>>>>>>- 5.8.0-Beta5 fixes >>>>>>><https://github.com/wso2/product-is/milestone/80?closed=1> >>>>>>>- 5.8.0-Beta4 fixes >>>>>>><https://github.com/wso2/product-is/milestone/79?closed=1> >>>>>>>- 5.8.0-Beta3 fixes >>>>>>><https://github.com/wso2/product-is/milestone/77?closed=1> >>>>>>>- 5.8.0-Beta fixes >>>>>>><https://github.com/wso2/product-is/milestone/75?closed=1> >>>>>>>- 5.8.0-Alpha5 fixes >>>>>>><https://github.com/wso2/product-is/milestone/74?closed=1> >>>>>>>- 5.8.0-Alpha4 fixes >>>>>>><https://github.com/wso2/product-is/milestone/73?closed=1> >>>>>>>- 5.8.0-Alpha3 fixes >>>>>>><https://github.com/wso2/product-is/milestone/72?closed=1> >>>>>>>- 5.8.0-Alpha2 fixes >>>>>>><https://github.com/wso2/product-is/milestone/71?closed=1> >>>>>>>- 5.8.0-Alpha fixes >>>>>>><https://github.com/wso2/product-is/milestone/70?closed=1> >>>>>>>- 5.8.0-M26 fixes >>>>>>><https://github.com/wso2/product-is/milestone/69?closed=1> >>>>>>>- 5.8.0-M25 fixes >>>>>>><https://github.com/wso2/product-is/milestone/68?closed=1> >>>>>>>- 5.8.0-M24 fixes >>>>>>><https://github.com/wso2/product-is/milestone/67?closed=1> >>>>>>>- 5.8.0-M6 fixes >>>>>>><https://github.com/wso2/product-is/milestone/64?closed=1> >>>>>>>- 5.8.0-M5 fixes >>>>>>><https://github.com/wso2/product-is/milestone/63?closed=1> >>>>>>>- 5.8.0-M4 fixes >>>>>>><https://github.com/wso2/product-is/milestone/62?closed=1> >>>>>>>- 5.8.0-M3 fixes >>>>>>><https://github.com/wso2/product-is/milestone/61?closed=1> >>>>>>>- 5.8.0-M2 fixes >>>>>>><https://github.com/wso2/product-is/milestone/60?closed=1> >>>>>>>- 5.8.0-M1 fixes >>>>>>><https://github.com/wso2/product-is/milestone/59?closed=1> >>>>>>> >>>>>>> >>>>>>> Source and distribution >>>>>>> >>>>>>> Runtime - https://github.com/wso2/product-is/releases/tag/v >>>>>>> <https://github.com/wso2/product-is/releases/download/v5.8.0-rc3/wso2is-5.8.0-rc3.zip> >>>>>>> 5.8.0-rc3 >>>>>>> <https://github.com/wso2/product-is/releases/download/v5.8.0-rc3/wso2is-5.8.0-rc3.zip> >>>>>>> Analytics - >>>>>>> https://github.com/wso2/analytics-is/releases/tag/v5.8.0-rc3 >>>>>>> <https://github.com/wso2/analytics-is/releases/download/v5.8.0-rc3/wso2is-analytics-5.8.0-rc3.zip> >>>>>>> >>>>>>> >>>>>>> Please download, test the product and vote. >>>>>>> >>>>>>> [+] Stable - go ahead and release >>>>>>> [-] Broken - do not release (explain why) >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> - WSO2 Identity and Access Management Team - >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Hasanthi Dissanayake >>>>>>> >>>>>>> Senior Software Engineer | WSO2 >>>>>>> >>>>>>> E: hasan...@wso2.com >>>>>>> M :0718407133| http://wso2.com <http://wso2.com/> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Hasanthi Dissanayake >>>>>> >>>>>> Senior Software Engineer | WSO2 >>>>>> >>>>>> E: hasan...@wso2.com >>>>>> M :0718407133| http://wso2.com <http://wso2.com/> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Shanika Wickramasinghe* >>>>> Software Engineer - QA Team >>>>> >>>>> Email: shani...@wso2.com >>>>> Mobile : +94713503563 >>>>> Web : http://wso2.com >>>>> >>>>> <http://wso2.com/signature> >>>>> >>>> >>>> >>>> -- >>>> *Isuranga Perera* | Software Engineer | WSO2 Inc. >>>> +94 71 735 7034 | isura...@wso2.com >>>> >>>> ___ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>> >>> >>> -- >>> >>> Hasanthi Dissanayake | Senior Software Engineer | WSO2 Inc. >>> (m) +94718407133 | (w) +94112145345 | Email: hasan...@wso2.com >>> >>> >> >> -- >> >> Hasanthi Dissanayake | Senior Software Engineer | WSO2 Inc. >> (m) +94718407133 | (w) +94112145345 | Email: hasan...@wso2.com >> >> ___ >> Dev mailing list >> d...@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > -- *Sarubi Thillainathan* | Software Engineer | WSO2 Inc. (m) +94 (0) 76 684 9101 | (e) sar...@wso2.com,stsa...@gmail.com *[image: https://wso2.com/signature] <https://wso2.com/signature>* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [IS] Adding Tenant Base Account Recovery ReCaptcha
Hi all, PR for this feature is get merged and will be available with next IS release 5.8 onward. PRs can be found in [1], [2], [3]. [1] https://github.com/wso2/carbon-identity-framework/pull/1926 [2] https://github.com/wso2/carbon-identity-framework/pull/1926 [3] https://github.com/wso2-extensions/identity-governance/pull/255 Thanks, Sarubi. On Mon, Oct 8, 2018 at 10:25 AM Sarubi Thillainathan wrote: > Hi all, > > After the offline discussion with Isura and Hasintha for this $Subject > issue, > The proposed solution has an intermediate page that is able to grab the > tenant domain name. This intermediate page can be enabled via the > context-parameter “EnableMultiTenancy” to true in > “accountrecoveryendpoint.war” web.xml. > > Modified flow of implementation is as follows: > > [image: xx.png] > > > Thanks, > Sarubi. > > On Mon, Oct 1, 2018 at 7:57 AM Hasintha Indrajee > wrote: > >> >> >> On Mon, Oct 1, 2018 at 5:20 AM Isura Karunaratne wrote: >> >>> >>> >>> On Sun, Sep 30, 2018 at 6:07 PM Sarubi Thillainathan >>> wrote: >>> >>>> >>>> >>>> >>>> On Sun, Sep 30, 2018 at 6:03 PM Sarubi Thillainathan >>>> wrote: >>>> >>>>> Hi all, >>>>> >>>>> I'm working on adding ReCaptcha for account recovery that is, username >>>>> recovery and password recovery. I was able to integrate this feature with >>>>> WSO2 Identity Server. But there is a limitation found that from the login >>>>> dashboard we can not able to obtain the user's tenant domain prior to >>>>> loading the username/ password recovery page with reCaptcha. Since this >>>>> account recovery configurations under resident Idp are the tenant >>>>> dependant. Currently, I'm reading from carbon super configurations. >>>>> >>>>> For this issue, the proposed solution is to have an intermediate page >>>>> that is able to grab the tenant domain name. This intermediate page can be >>>>> enable via the configuration file (identity XML). >>>>> >>>> I assume that the intermediate page must be enabled in a multi tenancy >>> deployment. We can make this configuraiton to false by default and clearly >>> document to enable it in multi tenancy deployments. >>> >> >> >>> +1 >>> >> >> >>> Thanks >>> Isura. >>> >>>> >>>>> >>>>> Planned flow of implementation is as follows: >>>>> >>>>> [image: Untitled Diagram (1).png] >>>>> >>>>> Please provide your thoughts and feedback on this. >>>>> >>>>> Thanks, >>>>> Sarubi. >>>>> -- >>>>> *Sarubi Thillainathan * >>>>> *Software Engineer - WSO2 Inc.* >>>>> >>>>> *Mobile : +94 (0) 76 68 49 101* >>>>> >>>> >>>> >>>> -- >>>> *Sarubi Thillainathan * >>>> *Software Engineer - WSO2 Inc.* >>>> >>>> *Mobile : +94 (0) 76 68 49 101* >>>> >>> >>> >>> -- >>> >>> *Isura Dilhara Karunaratne* >>> Associate Technical Lead | WSO2 <http://wso2.com/> >>> *lean.enterprise.middleware* >>> Email: is...@wso2.com >>> Mob : +94 772 254 810 >>> Blog : http://isurad.blogspot.com/ >>> >>> >>> >>> >> >> -- >> Hasintha Indrajee >> WSO2, Inc. >> Mobile:+94 771892453 >> >> > > -- > *Sarubi Thillainathan * > *Software Engineer - WSO2 Inc.* > > *Mobile : +94 (0) 76 68 49 101* > -- *Sarubi Thillainathan * *Software Engineer - WSO2 Inc.* *Mobile : +94 (0) 76 68 49 101* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [IS] Adding Tenant Base Account Recovery ReCaptcha
Hi all, After the offline discussion with Isura and Hasintha for this $Subject issue, The proposed solution has an intermediate page that is able to grab the tenant domain name. This intermediate page can be enabled via the context-parameter “EnableMultiTenancy” to true in “accountrecoveryendpoint.war” web.xml. Modified flow of implementation is as follows: [image: xx.png] Thanks, Sarubi. On Mon, Oct 1, 2018 at 7:57 AM Hasintha Indrajee wrote: > > > On Mon, Oct 1, 2018 at 5:20 AM Isura Karunaratne wrote: > >> >> >> On Sun, Sep 30, 2018 at 6:07 PM Sarubi Thillainathan >> wrote: >> >>> >>> >>> >>> On Sun, Sep 30, 2018 at 6:03 PM Sarubi Thillainathan >>> wrote: >>> >>>> Hi all, >>>> >>>> I'm working on adding ReCaptcha for account recovery that is, username >>>> recovery and password recovery. I was able to integrate this feature with >>>> WSO2 Identity Server. But there is a limitation found that from the login >>>> dashboard we can not able to obtain the user's tenant domain prior to >>>> loading the username/ password recovery page with reCaptcha. Since this >>>> account recovery configurations under resident Idp are the tenant >>>> dependant. Currently, I'm reading from carbon super configurations. >>>> >>>> For this issue, the proposed solution is to have an intermediate page >>>> that is able to grab the tenant domain name. This intermediate page can be >>>> enable via the configuration file (identity XML). >>>> >>> I assume that the intermediate page must be enabled in a multi tenancy >> deployment. We can make this configuraiton to false by default and clearly >> document to enable it in multi tenancy deployments. >> > > >> +1 >> > > >> Thanks >> Isura. >> >>> >>>> >>>> Planned flow of implementation is as follows: >>>> >>>> [image: Untitled Diagram (1).png] >>>> >>>> Please provide your thoughts and feedback on this. >>>> >>>> Thanks, >>>> Sarubi. >>>> -- >>>> *Sarubi Thillainathan * >>>> *Software Engineer - WSO2 Inc.* >>>> >>>> *Mobile : +94 (0) 76 68 49 101* >>>> >>> >>> >>> -- >>> *Sarubi Thillainathan * >>> *Software Engineer - WSO2 Inc.* >>> >>> *Mobile : +94 (0) 76 68 49 101* >>> >> >> >> -- >> >> *Isura Dilhara Karunaratne* >> Associate Technical Lead | WSO2 <http://wso2.com/> >> *lean.enterprise.middleware* >> Email: is...@wso2.com >> Mob : +94 772 254 810 >> Blog : http://isurad.blogspot.com/ >> >> >> >> > > -- > Hasintha Indrajee > WSO2, Inc. > Mobile:+94 771892453 > > -- *Sarubi Thillainathan * *Software Engineer - WSO2 Inc.* *Mobile : +94 (0) 76 68 49 101* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [IS] Adding Tenant Base Account Recovery ReCaptcha
On Sun, Sep 30, 2018 at 6:03 PM Sarubi Thillainathan wrote: > Hi all, > > I'm working on adding ReCaptcha for account recovery that is, username > recovery and password recovery. I was able to integrate this feature with > WSO2 Identity Server. But there is a limitation found that from the login > dashboard we can not able to obtain the user's tenant domain prior to > loading the username/ password recovery page with reCaptcha. Since this > account recovery configurations under resident Idp are the tenant > dependant. Currently, I'm reading from carbon super configurations. > > For this issue, the proposed solution is to have an intermediate page that > is able to grab the tenant domain name. This intermediate page can be > enable via the configuration file (identity XML). > > Planned flow of implementation is as follows: > > [image: Untitled Diagram (1).png] > > Please provide your thoughts and feedback on this. > > Thanks, > Sarubi. > -- > *Sarubi Thillainathan * > *Software Engineer - WSO2 Inc.* > > *Mobile : +94 (0) 76 68 49 101* > -- *Sarubi Thillainathan * *Software Engineer - WSO2 Inc.* *Mobile : +94 (0) 76 68 49 101* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Dev] [VOTE] Release WSO2 Identity Server 5.7.0 RC3
Hi All, I have tested the following on both LDAP and JDBC user stores and no issues were found. * Multi-attribute filter search with and without pagination * All the available SCIM2 endpoints which are given in doc [1] [+] Stable - go ahead and release. [1] https://docs.wso2.com/display/IS570/apidocs/SCIM2-endpoints/ On Sat, Sep 15, 2018 at 2:24 AM Mathuriga Thavarajah wrote: > Hi All, > > I have tested the following and no issues were found. > > * Settip up MySQL 5.7 > * Configuring a Read-write Active Directory User Store as a > secondary user store > * Configuring Multi-factor Authentication (Basic and Google as a > federated authenticator) > * Configuring LDAP Active Directory as a primary store in WSO2 > Identity Server 5.7.0 RC3 on windows instance. > > [+] Stable - go ahead and release. > > Regards, > Mathuriga. > > > On Fri, Sep 14, 2018 at 5:23 PM Thanuja Jayasinghe > wrote: > >> Hi All, >> >> I have tested the following and no issues were found. >> >>- User account association >>- Workflow management >>- Adaptive authentication >>- Role-based >> - User age based >> >> [+] Stable - go ahead and release >> >> Thanks, >> Thanuja >> ___ >> Dev mailing list >> d...@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> > > > -- > > *Mathuriga Thavarajah* > Software Engineer > WSO2 Inc. - http ://wso2.com > > Email : mathur...@wso2.com > Mobile : +94778191300 > > > > *[image: http://wso2.com/signature] <http://wso2.com/signature>* > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > -- *Sarubi Thillainathan * *Software Engineer - WSO2 Inc.* *Mobile : +94 (0) 76 68 49 101* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] Introducing Logical Operators for SCIM2 Filters in WSO2 Identity Server
Hi all, I am working on a project to introducing logical operators for SCIM2 filters in WSO2 Identity Server. We will initially support 'AND', 'OR' logical operators for following SCIM2 filters such as eq, sw, ew, co. Please find the milestone plan in [1] for the project $Subject. Comments or Suggestions are highly appreciated. [1] https://docs.google.com/spreadsheets/d/1rw_MB-5N4BuGCSPYvd2sm8sHX9QS22opZ_T4k9LJ5ic/edit?usp=sharing Thanks, Sarubi -- *Sarubi Thillainathan * *Software Engineer - WSO2 Inc.* *Mobile : +94 (0) 76 68 49 101* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Tool to configure changes while starting an API-M profile
Hi Nuwan, I'll include that as well. Thanks, Sarubi On Wed, Jun 13, 2018 at 2:19 PM, Nuwan Dias wrote: > In the doc, it would be better to put the logs that are being printed > while the optimization happens as an example. > > On Wed, Jun 13, 2018 at 12:06 PM, Sarubi Thillainathan > wrote: > >> Hi all, >> >> PR for $Subject can be found in [1]. >> Documentation for this can be found in [2] >> >> [1] https://github.com/wso2/product-apim/pull/3350 >> [2] https://docs.google.com/document/d/1nB112HRslaqXLQA50lXdHblf >> 42QgMTIzTCAPVaOFjrA/edit?usp=sharing >> >> >> On Fri, Jun 8, 2018 at 9:14 AM, Chamila De Alwis >> wrote: >> >>> Hi Nuwan, >>> >>> Thanks! I understand the context now and I completely agree that some >>> kind of convenience is required to prune the base distribution to profile >>> specific ones. It could very well be automated better than being documented >>> instructions. >>> >>> My idea was that if some kind of automation could be done, it would be >>> better suited on top of what we have already built out (ex: our knowledge >>> on Puppet, what WUM client does with pack building). On the other hand, >>> automation which considers details after a certain point could have to >>> cater for a wider spectrum of requirements, which could make the tool >>> complicated. >>> >>> +1 to make the tool to automate the prunning part of the process. >>> >>> Startup time getting affected by the `-Dprofile` property might not be >>> that much of a concern at all (other than for Containerized scenarios, >>> which could make use of pre-baked optimized packs), because instance >>> downtimes are almost always pre-planned and will be done with back up >>> nodes. So few additional seconds would not be that much of a difference. >>> >>> >>> Regards, >>> Chamila de Alwis >>> Committer and PMC Member - Apache Stratos >>> Associate Technical Lead | WSO2 >>> +94 77 220 7163 >>> Blog: https://medium.com/@chamilad >>> >>> >>> >>> >>> On Thu, Jun 7, 2018 at 1:18 AM Nuwan Dias wrote: >>> >>>> While these concerns are valid, I think we need to step back and look >>>> at this. If you look at are current documentation related to profiles, you >>>> will see that we have lots of manual instructions to optimize the profiles. >>>> What this tool/command does is to simply automate all those manual >>>> instructions. So our "produce" at the end of the day will be exactly the >>>> same as it is today. >>>> >>>> This tool does modify some configs, but I wouldn't consider that as a >>>> config automation. Config automation is supposed to do configs to make the >>>> product work according to a particular deployment architecture. The configs >>>> we're modifying in this case do nothing to do with deployment architecture. >>>> All it does is to remove unnecessary processes and stuff based on the >>>> profile. I would rather consider the outcome/produce of this tool to be >>>> used as the base image on which config automation tools should run on. >>>> >>>> I don't think WUM is a problem because WUM always gives you a fresh >>>> distribution. It does not work/modify your current running instance. >>>> Therefore a WUM user has to run the tool/process on the WUM updated >>>> distribution as they do today. So the pipeline would be >>>> >>>> wum_update --> optimize --> puppet --> run >>>> >>>> As Sinthuja has pointed it would perhaps be good to create a new distro >>>> (.zip) after this process so that one can use it as the base distro. We >>>> have currently focused on making the optimization work seamlessly with the >>>> -Dprofile=x startup command, which is how a majority of of customers using >>>> profiles use the product. >>>> >>>> Thanks, >>>> NuwanD. >>>> >>>> On Thu, Jun 7, 2018 at 12:19 PM, Chamila De Alwis >>>> wrote: >>>> >>>>> Thanks Sarubi for the detailed explanation! I had understood the >>>>> context wrong and this cleared up some of the doubts I had. However, I >>>>> still have a hard time grasping the whole user story. >>>>> >>>>> 1. I assume the changes you mention are the ones at [1]. If that is >>>>&
Re: [Architecture] Tool to configure changes while starting an API-M profile
Hi all, PR for $Subject can be found in [1]. Documentation for this can be found in [2] [1] https://github.com/wso2/product-apim/pull/3350 [2] https://docs.google.com/document/d/1nB112HRslaqXLQA50lXdHblf42QgMTIzTCAPVaOFjrA/edit?usp=sharing On Fri, Jun 8, 2018 at 9:14 AM, Chamila De Alwis wrote: > Hi Nuwan, > > Thanks! I understand the context now and I completely agree that some kind > of convenience is required to prune the base distribution to profile > specific ones. It could very well be automated better than being documented > instructions. > > My idea was that if some kind of automation could be done, it would be > better suited on top of what we have already built out (ex: our knowledge > on Puppet, what WUM client does with pack building). On the other hand, > automation which considers details after a certain point could have to > cater for a wider spectrum of requirements, which could make the tool > complicated. > > +1 to make the tool to automate the prunning part of the process. > > Startup time getting affected by the `-Dprofile` property might not be > that much of a concern at all (other than for Containerized scenarios, > which could make use of pre-baked optimized packs), because instance > downtimes are almost always pre-planned and will be done with back up > nodes. So few additional seconds would not be that much of a difference. > > > Regards, > Chamila de Alwis > Committer and PMC Member - Apache Stratos > Associate Technical Lead | WSO2 > +94 77 220 7163 > Blog: https://medium.com/@chamilad > > > > > On Thu, Jun 7, 2018 at 1:18 AM Nuwan Dias wrote: > >> While these concerns are valid, I think we need to step back and look at >> this. If you look at are current documentation related to profiles, you >> will see that we have lots of manual instructions to optimize the profiles. >> What this tool/command does is to simply automate all those manual >> instructions. So our "produce" at the end of the day will be exactly the >> same as it is today. >> >> This tool does modify some configs, but I wouldn't consider that as a >> config automation. Config automation is supposed to do configs to make the >> product work according to a particular deployment architecture. The configs >> we're modifying in this case do nothing to do with deployment architecture. >> All it does is to remove unnecessary processes and stuff based on the >> profile. I would rather consider the outcome/produce of this tool to be >> used as the base image on which config automation tools should run on. >> >> I don't think WUM is a problem because WUM always gives you a fresh >> distribution. It does not work/modify your current running instance. >> Therefore a WUM user has to run the tool/process on the WUM updated >> distribution as they do today. So the pipeline would be >> >> wum_update --> optimize --> puppet --> run >> >> As Sinthuja has pointed it would perhaps be good to create a new distro >> (.zip) after this process so that one can use it as the base distro. We >> have currently focused on making the optimization work seamlessly with the >> -Dprofile=x startup command, which is how a majority of of customers using >> profiles use the product. >> >> Thanks, >> NuwanD. >> >> On Thu, Jun 7, 2018 at 12:19 PM, Chamila De Alwis >> wrote: >> >>> Thanks Sarubi for the detailed explanation! I had understood the context >>> wrong and this cleared up some of the doubts I had. However, I still have a >>> hard time grasping the whole user story. >>> >>> 1. I assume the changes you mention are the ones at [1]. If that is the >>> case, there are indeed configuration changes involved. These are typically >>> handled by a configuration automation tool. This is the overlap I'm >>> concerned with. With the limited information I have, I guess any config >>> automation will have to consider both scenarios where this script is >>> present and absent. In other words, a set of scripts that work with a pack >>> edited by this script would not work with the shipped pack. >>> >>> 2. What would be the effect of WUM updates? If the script considers the >>> config changes in #1, and if any WUM updates affect those configurations >>> (any changes would be additions), would the script also have to be updated? >>> >>> 3. Going along with what Sinthuja has raised above, if new packs are to >>> be created, then most of the operations would be what a config automation >>> tool does. >>> >>> 4. What would be overlap/effect
Re: [Architecture] Tool to configure changes while starting an API-M profile
Hi Chamin,
We are not going to maintain a separate repository. We will include it into
the startup-scripts folder under product apim.
Thanks,
Sarubi
On Fri, Jun 1, 2018 at 5:40 PM, Chamin Dias wrote:
> Hi,
>
> How are we going to maintain the code for this? Are we going to maintain
> this like the import-export tool
> <https://github.com/wso2/product-apim/tree/v2.2.0/modules/api-import-export>
> or are we going to maintain it separately (another repo maybe)?
>
> Thanks.
>
> On Fri, Jun 1, 2018 at 11:38 AM, Sarubi Thillainathan
> wrote:
>
>> Adding architecture
>>
>> On Thu, May 31, 2018 at 10:00 AM, Sarubi Thillainathan
>> wrote:
>>
>>> Hi Naduni,
>>>
>>> Currently, I'm handling port offsets as passing an environment variable
>>> when starting the server only if the user wants to change the port.
>>>
>>>
>>> Thanks
>>>
>>> On Wed, May 30, 2018 at 12:26 PM, Dinusha Dissanayake >> > wrote:
>>>
>>>>
>>>>
>>>> On Wed, May 30, 2018 at 12:13 PM, Naduni Pamudika
>>>> wrote:
>>>>
>>>>> Hi Sarubi,
>>>>>
>>>>> How are you going to handle port offsets for the profiles? I think
>>>>> rather than reading the offset from the carbon.xml file, it would be easy
>>>>> to give it as a parameter when running the script. We can make this
>>>>> parameter optional. Otherwise the user has to update carbon.xml file
>>>>> before
>>>>> running the script and I do not see any reason for doing it as we are
>>>>> automating the process here.
>>>>>
>>>> Port offset can be passed as a environment variable like we do *sh
>>>> wso2server.sh -DportOffset={offset}*. I am not sure if it is the best
>>>> way to do this.
>>>>
>>>>> Thanks,
>>>>> Naduni
>>>>>
>>>>> On Wed, May 30, 2018 at 11:45 AM, Dinusha Dissanayake <
>>>>> dinus...@wso2.com> wrote:
>>>>>
>>>>>> Hi Krishan/All,
>>>>>>
>>>>>> On Wed, May 30, 2018 at 11:35 AM, Krishan Wijesena >>>>> > wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, May 30, 2018 at 10:51 AM, Sarubi Thillainathan <
>>>>>>> sar...@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi Fazlan,
>>>>>>>>
>>>>>>>> Yes, it's a shell script. As per the discussion with Nuwan, we have
>>>>>>>> planned that user have to manually run the script file according to
>>>>>>>> his/her
>>>>>>>> profile preference before starting the server.
>>>>>>>>
>>>>>>>how about starting the server using the same script. If some one
>>>>>>> want to run KM profile, user just want to execute the script with
>>>>>>> prefered
>>>>>>> configurations for KM.
>>>>>>>
>>>>>> AFAIR, this is going to remove unnecessary web apps and do the
>>>>>> necessary configuration. If there are using tools like "chef" or "yajsw"
>>>>>> for there further configurations , then starting the server using the
>>>>>> same
>>>>>> script might not be ideal.
>>>>>>
>>>>>> Thanks,
>>>>>> DinushaD.
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, May 30, 2018 at 10:39 AM, Fazlan Nazeem
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Sarubi,
>>>>>>>>>
>>>>>>>>> Is this going to be a shell script?
>>>>>>>>>
>>>>>>>>> If so how are you planning to execute this? Does the user have to
>>>>>>>>> manually run the script before starting the server with the profile
>>>>>>>>> or will
>>>>>>>>> the script execution be automatically executed when we start the
>>>>>>>>> server
>>>>>>>>> with the profile?
>>>>>>>>>
>>>>>>>>> On Wed, May 30, 2018 at 10:34 AM, Sarubi Thillainathan <
>>>&
Re: [Architecture] Tool to configure changes while starting an API-M profile
Hi Chamila, According to the offline discussion with Nuwan, Here what we are doing is, automate configuration changes to optimize API-M profiles. Up to now, it has been manually done by users [1] when they wanted to run a selected profile. We provide two options to accomplish this, 1. The user can do the optimization separately before the server starts up by run the profileSetup.sh along with a particular profile flag. (ex: profileSetup.sh -Dprofile=api-store ) Once it has done, can use this APIM package as the root image for CM tools. 2. The user can start the wso2server.sh with a particular profile along with --optimize flag. (i.e) wso2server.sh --optimize -Dprofile=api-store. if and only -optimize flag found we do the optimization for that particular profile then start the server automatically, otherwise not. Please find inline comments for further details: On Tue, Jun 5, 2018 at 4:11 AM, Chamila De Alwis wrote: > Hi Sarubi, > > I have a couple of questions regarding the fit between the changes that > this script does and the config automation tools. > > Is this a tool focused on evaluation part of the product or the deployment > part? IMO both would require a re-evaluation of the approach. > This is focused on deployment part of product. > > IIUC, this script would alter the startup pack, with file additions and > removals done to config files, deployable artifacts, and other library > files. This could be a problematic case for continuous config automation > story. > > Consider a scenario where Puppet (or any other tool) configures a set of > instances, using the APIM pack as the root. The tool might be configured to > continuously run (say, with a 30-minute interval). Is there a chance for > the server to be reconfigured every time the tool is run, because the tool > detects the changes that this script does at the startup, as changes that > should be rolled back? > According to the scenario that you have mentioned, Puppet (or any other tool) can configures a set of instances, using optimized APIM pack as the root. > In that case, it might be better to have this tool as a separate script > rather than one which starts up the server as well. > Yes, now we maintain a seperate script which will get executed based on the user specification for server start-up. > > Another point against such a tool bound to server startup would be the > mixing of concerns. AFAIU, what this script does falls under what the > config automation tools do. IMO WSO2 server startup scripts should only > work on the bootstrapping of the server runtime, rather than the config > automation part. Also, we would essentially be repeating what the config > automation tools should do, but in Bash which sometimes is a lot harder. > Obtaining config values, managing them meaningfully, and applying them to > files is essentially what a config automation tool does. We may be > re-inventing the wheel here. For an example, should this tool be concerned > about the port offset at all since even dev deployments will be done using > separate VMs/Containers? Or should it be worried about different values for > the config options in different environments? > > Also, this script don't concerned about the port offset things since it designed only to configure the changes for corresponding profile specified. > Additionally, users who manage config automation usually adhere to one set > of scripts. Puppet users almost always go for Puppet, Chef users for Chef > etc. What we would be delivering might step on what they already have, or > are comfortable working with, and in turn, the adaption rate might take a > hit. > > User can go with first option, run the profile setup script seperatly before server start up and get optimized API-M pack as the base image > IMO you might benefit from reviewing your approach with the Installation > Experience effort, as the responsibilities of this tool fall close to, if > not overlap with, what the Installation Experience scripts deliver/would > deliver with different providers. > > WDYT? > > > Regards, > Chamila de Alwis > Committer and PMC Member - Apache Stratos > Associate Technical Lead | WSO2 > +94 77 220 7163 > Blog: https://medium.com/@chamilad > > Commnets or suggestions are highly appreciated. [1]https://docs.wso2.com/display/AM220/Product+Profiles -- *Sarubi Thillainathan * *Software Engineer - WSO2 Inc.* *Mobile : +94 (0) 76 68 49 101* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Tool to configure changes while starting an API-M profile
Hi Prasanna/All,
These are the points and improvements that we discussed at the review
meeting (prepared by Thilini).
- Print logs for each action and failures
- Alter the script in order to handle already completed actions. For
example, if the script was executed for multiple times, print a log and
skip already completed actions like webapp copying/removing, print a log
- Do a server startup time comparison with and without the optimizations
- Test throttling feature with optimization
- Provide the option to invoke the profile optimization shell script
during startup also
- Change the logic and separate out each profile optimization
- Remove authenticationendpoint.war from gateway, publisher and store
profiles
- Remove calculator webapp from all profiles
- Remove micro gateway from the build and the publisher profile
- Avoid registry indexing from gateway and traffic manager. Add a note
in the documentation on how to enable indexing, in a case whether the
indexing is required in the gateway.
- Remove synapse configs from km, publisher and traffic manager
- Set optimized = true option in the script.
Mail thread can be found in [1].
[1]Updated invitation: [APIM] [REVIEW] Tool to configure changes while
starting an API-M profile
On Sun, Jun 3, 2018 at 12:17 AM, Prasanna Dangalla
wrote:
> Hi Sarubi,
>
> Shall we add the points and improvements that we discussed recently ti
> this mail thread.
>
> Further make sure the tool will not fail if you run it multiple times for
> the same profile.
>
> Thanks
> Prasanna
>
> On Fri, Jun 1, 2018 at 5:42 PM Chamin Dias wrote:
>
>> Hi,
>>
>> How are we going to maintain the code for this? Are we going to maintain
>> this like the import-export tool
>> <https://github.com/wso2/product-apim/tree/v2.2.0/modules/api-import-export>
>> or are we going to maintain it separately (another repo maybe)?
>>
>> Thanks.
>>
>> On Fri, Jun 1, 2018 at 11:38 AM, Sarubi Thillainathan
>> wrote:
>>
>>> Adding architecture
>>>
>>> On Thu, May 31, 2018 at 10:00 AM, Sarubi Thillainathan
>>> wrote:
>>>
>>>> Hi Naduni,
>>>>
>>>> Currently, I'm handling port offsets as passing an environment variable
>>>> when starting the server only if the user wants to change the port.
>>>>
>>>>
>>>> Thanks
>>>>
>>>> On Wed, May 30, 2018 at 12:26 PM, Dinusha Dissanayake <
>>>> dinus...@wso2.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, May 30, 2018 at 12:13 PM, Naduni Pamudika
>>>>> wrote:
>>>>>
>>>>>> Hi Sarubi,
>>>>>>
>>>>>> How are you going to handle port offsets for the profiles? I think
>>>>>> rather than reading the offset from the carbon.xml file, it would be easy
>>>>>> to give it as a parameter when running the script. We can make this
>>>>>> parameter optional. Otherwise the user has to update carbon.xml file
>>>>>> before
>>>>>> running the script and I do not see any reason for doing it as we are
>>>>>> automating the process here.
>>>>>>
>>>>> Port offset can be passed as a environment variable like we do *sh
>>>>> wso2server.sh -DportOffset={offset}*. I am not sure if it is the best
>>>>> way to do this.
>>>>>
>>>>>> Thanks,
>>>>>> Naduni
>>>>>>
>>>>>> On Wed, May 30, 2018 at 11:45 AM, Dinusha Dissanayake <
>>>>>> dinus...@wso2.com> wrote:
>>>>>>
>>>>>>> Hi Krishan/All,
>>>>>>>
>>>>>>> On Wed, May 30, 2018 at 11:35 AM, Krishan Wijesena <
>>>>>>> krish...@wso2.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, May 30, 2018 at 10:51 AM, Sarubi Thillainathan <
>>>>>>>> sar...@wso2.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Fazlan,
>>>>>>>>>
>>>>>>>>> Yes, it's a shell script. As per the discussion with Nuwan, we
>>>>>>>>> have planned that user have to manually run the script file according
>>>>>>>>> to
>>>>>>>>> his/her profile preference before starting the server.
>>>>>>>>>
>>>>>>&
Re: [Architecture] Tool to configure changes while starting an API-M profile
Adding architecture
On Thu, May 31, 2018 at 10:00 AM, Sarubi Thillainathan
wrote:
> Hi Naduni,
>
> Currently, I'm handling port offsets as passing an environment variable
> when starting the server only if the user wants to change the port.
>
>
> Thanks
>
> On Wed, May 30, 2018 at 12:26 PM, Dinusha Dissanayake
> wrote:
>
>>
>>
>> On Wed, May 30, 2018 at 12:13 PM, Naduni Pamudika
>> wrote:
>>
>>> Hi Sarubi,
>>>
>>> How are you going to handle port offsets for the profiles? I think
>>> rather than reading the offset from the carbon.xml file, it would be easy
>>> to give it as a parameter when running the script. We can make this
>>> parameter optional. Otherwise the user has to update carbon.xml file before
>>> running the script and I do not see any reason for doing it as we are
>>> automating the process here.
>>>
>> Port offset can be passed as a environment variable like we do *sh
>> wso2server.sh -DportOffset={offset}*. I am not sure if it is the best
>> way to do this.
>>
>>> Thanks,
>>> Naduni
>>>
>>> On Wed, May 30, 2018 at 11:45 AM, Dinusha Dissanayake >> > wrote:
>>>
>>>> Hi Krishan/All,
>>>>
>>>> On Wed, May 30, 2018 at 11:35 AM, Krishan Wijesena
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, May 30, 2018 at 10:51 AM, Sarubi Thillainathan <
>>>>> sar...@wso2.com> wrote:
>>>>>
>>>>>> Hi Fazlan,
>>>>>>
>>>>>> Yes, it's a shell script. As per the discussion with Nuwan, we have
>>>>>> planned that user have to manually run the script file according to
>>>>>> his/her
>>>>>> profile preference before starting the server.
>>>>>>
>>>>>how about starting the server using the same script. If some one
>>>>> want to run KM profile, user just want to execute the script with prefered
>>>>> configurations for KM.
>>>>>
>>>> AFAIR, this is going to remove unnecessary web apps and do the
>>>> necessary configuration. If there are using tools like "chef" or "yajsw"
>>>> for there further configurations , then starting the server using the same
>>>> script might not be ideal.
>>>>
>>>> Thanks,
>>>> DinushaD.
>>>>
>>>>>
>>>>>>
>>>>>> On Wed, May 30, 2018 at 10:39 AM, Fazlan Nazeem
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Sarubi,
>>>>>>>
>>>>>>> Is this going to be a shell script?
>>>>>>>
>>>>>>> If so how are you planning to execute this? Does the user have to
>>>>>>> manually run the script before starting the server with the profile or
>>>>>>> will
>>>>>>> the script execution be automatically executed when we start the server
>>>>>>> with the profile?
>>>>>>>
>>>>>>> On Wed, May 30, 2018 at 10:34 AM, Sarubi Thillainathan <
>>>>>>> sar...@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>> I am working on a project to optimize the API-M profiles by
>>>>>>>> automating the configuration changes while starting an API-M profile.
>>>>>>>> According to the user input, perform corresponding profile set up for
>>>>>>>> the
>>>>>>>> wso2am-2.5.0-SNAPSHOT on both Unix and Windows environments.
>>>>>>>>
>>>>>>>> Please find the milestone plan in [1] for the project $Subject.
>>>>>>>>
>>>>>>>> Comments or Suggestions are highly appreciated.
>>>>>>>>
>>>>>>>> [1] https://docs.google.com/spreadsheets/d/1C7UyD1e3JvPM17ZpivwD
>>>>>>>> jboL8GX2CD5ZF1B3QTBhFaU/edit?usp=sharing
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> --
>>>>>>>> *Sarubi Thillainathan *
>>>>>>>> *Software Engineer - WSO2 Inc.*
>>>>>>>>
>>>>>>>> *Mobile : +94 (0) 76 68 49 101*
>>>>>>>> --
>>>>>>>> You received this message because you a
