Re: [Architecture] [APIM] Workflow for API State change
;>> >>>>> Hi Chamila, >>>>> >>>>> I think its better to draw a diagram including API-M and BPS and >>>>> describe the parameters passed in-between :) >>>>> >>>>> Thanks, >>>>> NuwanD. >>>>> >>>>> On Mon, Aug 22, 2016 at 10:52 AM, Chamila Adhikarinayake < >>>>> chami...@wso2.com> wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> This feature is to add workflow extension support to API lifecycle >>>>>> state changes. This feature will be similar to workflow extension support >>>>>> in API store. >>>>>> >>>>>> There will be two workflow extensions provided with this. WS Workflow >>>>>> Executor and Simple Workflow Executor. Simple Workflow Executor carries >>>>>> out >>>>>> an operation without any intervention by a workflow admin. WS Workflow >>>>>> Executor will call external service (In this case service in BPS) for >>>>>> workflow tasks. >>>>>> >>>>>> For this we are planning to introduce BPMN process instead of using >>>>>> BPEL. A new rest api will be added to publisher rest apis for the BPMN >>>>>> process to use as the callback service. This new rest api will be >>>>>> protected >>>>>> with a scope for workflow. >>>>>> >>>>>> Since the REST api is protected with OAuth tokens, we need a method >>>>>> to provide Oauth application information to BPS service. Current plan is >>>>>> to >>>>>> generate oauth application from the APIM side using dcr endpoint and pass >>>>>> the client id and secret to BPS service when invoking the workflow's >>>>>> execute method. BPMN process will then use these application information >>>>>> to >>>>>> generate token and using that it will invoke the callback rest api to >>>>>> continue the workflow. >>>>>> >>>>>> Lifecycle executor will be called based on the status of the workflow. >>>>>> >>>>>> A new UI will be added to Admin Portal to handle human tasks (to >>>>>> approve or reject) >>>>>> >>>>>> Any suggestion is highly appreciated. >>>>>> >>>>>> Thanks, >>>>>> Chamila >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Chamila Adhikarinayake >>>>>> Software Engineer >>>>>> WSO2, Inc. >>>>>> Mobile - +94712346437 >>>>>> Email - chami...@wso2.com >>>>>> Blog - http://helpfromadhi.blogspot.com/ >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Nuwan Dias >>>>> >>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>> email : nuw...@wso2.com >>>>> Phone : +94 777 775 729 >>>>> >>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Chamila Adhikarinayake >>>> Software Engineer >>>> WSO2, Inc. >>>> Mobile - +94712346437 >>>> Email - chami...@wso2.com >>>> Blog - http://helpfromadhi.blogspot.com/ >>>> >>>> ___ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Jenananthan Yogendran >>> *Senior Software Engineer,* >>> *WSO2 inc., http://wso2.com <http://wso2.com>* >>> >>> >>> ___ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Nandika Jayawardana >> WSO2 Inc ; http://wso2.com >> lean.enterprise.middleware >> > > > > -- > Regards, > Chamila Adhikarinayake > Software Engineer > WSO2, Inc. > Mobile - +94712346437 > Email - chami...@wso2.com > Blog - http://helpfromadhi.blogspot.com/ > > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Janaka Ranabahu* Associate Technical Lead, WSO2 Inc. http://wso2.com *E-mail: jan...@wso2.com <http://wso2.com>**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Bulk export of APIs via APIM
Hi Thilini, On Wed, Aug 3, 2016 at 11:40 AM, Janaka Ranabahu <jan...@wso2.com> wrote: > Hi, > > On Wed, Aug 3, 2016 at 9:54 AM, Thilini Cooray <thili...@wso2.com> wrote: > >> Hi Janaka, >> >> >> On Tue, Aug 2, 2016 at 5:43 PM, Janaka Ranabahu <jan...@wso2.com> wrote: >> >>> Hi Malintha, Kaveesha, >>> >>> On Fri, Jul 22, 2016 at 3:29 PM, Malintha Amarasinghe < >>> malint...@wso2.com> wrote: >>> >>>> >>>> >>>> On Fri, Jul 22, 2016 at 3:25 PM, Malintha Amarasinghe < >>>> malint...@wso2.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> +1 for this improvement as it will make the developer's life easy when >>>>> exporting/importing multiple APIs. >>>>> >>>>> The option [2] is better in situations like there are many APIs to >>>>> bulk export so that the it might be difficult for a user to specify each >>>>> and every APIs to export: what he wants is simply export all the APIs >>>>> available. The overhead of the API call is not significant IMHO; there >>>>> will >>>>> be only one (or very few) additional calls to getAllAPIs. >>>>> >>>>> But as we discussed offline, we can start from step [1], and then >>>>> improve it by adding step [2]. >>>>> >>>> Step 1: CSV file with list of API name/version/provider --> Bulk export >>>> Step 2: call getAllApis() and get the list of APIs --> Generates the >>>> CSV file (we can implement this as a different operation and then provide >>>> the generated CSV file as an input to Step1) >>>> >>> Do we really have a use-case where a customer has to import/export all >>> the APis in a environment. It is highly unlikely that this would happen. >>> IMO, we should only support the file based approach. That would be >>> sufficient for the bulk import/export purposes. >>> >> >> What if a user decides to move from one environment to another (from Dev >> to QA or on-premise to API Cloud etc.) and wants to move all his deployed >> APIs to the new environment? >> If he has a huge number of APIs, IMO it would not be easy to name them >> one by one on a file. Rather it would be easy if we have an option to >> export and import as a whole. >> > Quick clarification. In this scenario, we are only considering the APIs published by that user right? Not all the APIs in the system is it? Or do we have to be a admin user who has access to all the APIs? > >> I too agree that this is not a first priority use case for the tool that >> we need to address, however IMO this is something better to have. >> > Agreed. Then we do not have to call getAllApis() and create a CSV out of > that. Just sending a query parameter saying all APIs would be sufficient > right? > > Thanks, > Janaka > > >> >> WDYT? >> >> Thanks, >> ThiliniC. >> >> >> >>> >>> Thanks, >>> Janaka >>> >>>> >>>> >>>> Regarding the file format I would prefer CSV as it would make it easy >>>>> to edit if need using existing tools. WDYT? >>>>> >>>>> Thanks! >>>>> >>>>> On Fri, Jul 22, 2016 at 10:58 AM, Kaveesha Perera <kavee...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> I'm planing to improve APIM import/export tool to support bulk import >>>>>> and bulk export of APIs. >>>>>> >>>>>> When considering the bulk export, there it came up to a discussion >>>>>> point on how to get the API list to be export. Considering several facts, >>>>>> came up with two suggestions as follows. >>>>>> >>>>>> [1]. Asking user to submit a file with explicitly written list of >>>>>> APIs. (API detail may include name, version and provider) >>>>>> >>>>>> [2]. Getting the prevailing list of APIs using getAllAPIs REST API, >>>>>> save the usable content of resulting string in a separate file, then >>>>>> allow >>>>>> user to edit it such that he can remove the unwanted APIs from the list, >>>>>> use it as a input for the bulk export. >>>>>> >>>>>> Currently we prefer procedure [1], since t
Re: [Architecture] Bulk export of APIs via APIM
Hi, On Wed, Aug 3, 2016 at 9:54 AM, Thilini Cooray <thili...@wso2.com> wrote: > Hi Janaka, > > > On Tue, Aug 2, 2016 at 5:43 PM, Janaka Ranabahu <jan...@wso2.com> wrote: > >> Hi Malintha, Kaveesha, >> >> On Fri, Jul 22, 2016 at 3:29 PM, Malintha Amarasinghe <malint...@wso2.com >> > wrote: >> >>> >>> >>> On Fri, Jul 22, 2016 at 3:25 PM, Malintha Amarasinghe < >>> malint...@wso2.com> wrote: >>> >>>> Hi, >>>> >>>> +1 for this improvement as it will make the developer's life easy when >>>> exporting/importing multiple APIs. >>>> >>>> The option [2] is better in situations like there are many APIs to bulk >>>> export so that the it might be difficult for a user to specify each and >>>> every APIs to export: what he wants is simply export all the APIs >>>> available. The overhead of the API call is not significant IMHO; there will >>>> be only one (or very few) additional calls to getAllAPIs. >>>> >>>> But as we discussed offline, we can start from step [1], and then >>>> improve it by adding step [2]. >>>> >>> Step 1: CSV file with list of API name/version/provider --> Bulk export >>> Step 2: call getAllApis() and get the list of APIs --> Generates the CSV >>> file (we can implement this as a different operation and then provide the >>> generated CSV file as an input to Step1) >>> >> Do we really have a use-case where a customer has to import/export all >> the APis in a environment. It is highly unlikely that this would happen. >> IMO, we should only support the file based approach. That would be >> sufficient for the bulk import/export purposes. >> > > What if a user decides to move from one environment to another (from Dev > to QA or on-premise to API Cloud etc.) and wants to move all his deployed > APIs to the new environment? > If he has a huge number of APIs, IMO it would not be easy to name them one > by one on a file. Rather it would be easy if we have an option to export > and import as a whole. > > I too agree that this is not a first priority use case for the tool that > we need to address, however IMO this is something better to have. > Agreed. Then we do not have to call getAllApis() and create a CSV out of that. Just sending a query parameter saying all APIs would be sufficient right? Thanks, Janaka > > WDYT? > > Thanks, > ThiliniC. > > > >> >> Thanks, >> Janaka >> >>> >>> >>> Regarding the file format I would prefer CSV as it would make it easy to >>>> edit if need using existing tools. WDYT? >>>> >>>> Thanks! >>>> >>>> On Fri, Jul 22, 2016 at 10:58 AM, Kaveesha Perera <kavee...@wso2.com> >>>> wrote: >>>> >>>>> Hi All, >>>>> >>>>> I'm planing to improve APIM import/export tool to support bulk import >>>>> and bulk export of APIs. >>>>> >>>>> When considering the bulk export, there it came up to a discussion >>>>> point on how to get the API list to be export. Considering several facts, >>>>> came up with two suggestions as follows. >>>>> >>>>> [1]. Asking user to submit a file with explicitly written list of >>>>> APIs. (API detail may include name, version and provider) >>>>> >>>>> [2]. Getting the prevailing list of APIs using getAllAPIs REST API, >>>>> save the usable content of resulting string in a separate file, then allow >>>>> user to edit it such that he can remove the unwanted APIs from the list, >>>>> use it as a input for the bulk export. >>>>> >>>>> Currently we prefer procedure [1], since the process [2] creates >>>>> additional overhead due to the REST API call. Still there can be are pros >>>>> and cons in both the processors. If any opinions on above and any options >>>>> please do reply. >>>>> >>>>> Regards. >>>>> -- >>>>> Kaveesha Perera >>>>> Intern - Software Engineering >>>>> >>>>> mobile: 0716130471 >>>>> >>>>> ___ >>>>> Architecture mailing list >>>>> Architecture@wso2.org >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >
Re: [Architecture] Facilitating Updating API with import/export tool in APIM
On Tue, Aug 2, 2016 at 8:38 PM, Nuwan Dias <nuw...@wso2.com> wrote: > Yes, +1. I too believe we should continue developing this as a library. > That would bring out a lot more possibilities on how this library can be > used instead of limiting it to an API and hence would serve a wider > audience. > This would not bring more possibilities but limit the possibilities we have. Import/export should be 2 core functionalists of an API. These operations are heavy ones. Doing that in a client side would add lots of overhead. We should do this on the server side and get a zip file as in the previous versions. That way we can minimize the overhead and which is a more cleaner approach. > > The reason we had the import/export functionality as a REST API earlier > was because we didn't have a nice clean REST API for the publisher and > store. Now that we have it IMO we should utilize that instead of > introducing yet another API proxying the publisher API. > > No. What I've asked here is not to add another layer. If I understand correctly, developing this as a separate library would be like adding a abstraction layer for something we do not have OOTB. Instead what I propose is we add 2 methods to the REST API itself called import and export. I believe that import and export should be 2 core functionalists of an API instead of using multiple calls to do that operation. If you look at other products like G-Reg, their core API has methods called "dump" and "restore" which does exactly the same. Thanks, Janaka > > > > > On Tue, Aug 2, 2016 at 7:53 PM, Joseph Fonseka <jos...@wso2.com> wrote: > >> Hi Janaka >> >> On Tue, Aug 2, 2016 at 4:45 PM, Janaka Ranabahu <jan...@wso2.com> wrote: >> >>> >>>- Creating the archive in the client machine would require multiple >>>HTTP calls between the client and the server. >>>- If we are going to support bulk import or export, then the number >>>of HTTP calls would increase depending on the number of APIs. >>> >>> Given the frequency of import/export and the number of APIs in >> a typical system I do not think we have to worry about number of HTTP calls >> required to import/export from client side. >> >>> >>>- There should be a native support for Import/Export from the REST >>>API. Otherwise clients who prefer to write their own tool would be forced >>>to either use the library we give, or have to use multiple API calls to >>> get >>>the required information. >>>- Shipping a different library would be problematic for customers >>>who are trying to automate the import/export(using some tool like Jenkins >>>or Puppet) >>>- Shipping a client library would also mean that we are bound to use >>>Java(or the language that supports that). What if a user want to write a >>> C# >>>tool for this. Then our client library would not help them a lot. >>>- If some users write tools of their own using the library we give, >>>if we release a new version on the import/export library, then those >>> users >>>have to rewrite their tools with the new version. That would be a lot of >>>overhead for the users. >>> >>> Therefore my suggestion is that we expose a OAuth2 protected >>> resource/method to import/export APIs using the REST API. This is exactly >>> what Sanjeewa has mentioned in his previous reply. >>> >> >> I agree with the above giving a HTTP API will be a more generic solution >> which can be used in many scenarios. But if we protect it with OAuth using >> it for day to day will be cumbersome since you have to deal with generating >> tokens. So if we develop a cli tool which utilize that API we can hide this >> complexity from the user. >> >> IMO we should continue on developing the import/export library which >> utilize the REST API. Then we have the flexibility of developing the >> import/export as an API, CLI tool or some other plugin if required. >> >> Thanks & Regards >> Jo >> >> >>> >>> >>>> >>>>>>> Thanks, >>>>>>> NuwanD. >>>>>>> >>>>>>> On Thu, Jul 21, 2016 at 11:31 AM, Sanjeewa Malalgoda < >>>>>>> sanje...@wso2.com> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jul 20, 2016 at 12:33 PM, Rasika Perera <rasi...@wso2.com> >>>>>>>> wrote: >
Re: [Architecture] API Manager Config Changes in version 2.0.0
>>>>>> Old Config New Config Description
>>>>>> The section that
>>>>>> contains the JWT configurations
>>>>>> Specifies the name of the JWT
>>>>>> header
>>>>>> Specifies the name of the
>>>>>> class that generates the JWT
>>>>>>
>>>>>>
>>>>>> 3. Changes to Analytics configurations.
>>>>>>
>>>>>> Old Config New Config Description
>>>>>> The section that contains the
>>>>>> Analytics configurations
>>>>>>
>>>>>> In addition to the above changes, a new section was introduced to
>>>>>> hold OAuth2.0 related API Manager specific configurations. This new
>>>>>> section
>>>>>> brings together OAuth related configs which were earlier scattered over
>>>>>> the
>>>>>> api-manager.xml file. This is what it contains now.
>>>>>>
>>>>>>
>>>>>>
>>>>>> true
>>>>>> am_application_scope
>>>>>>
>>>>>> ^device_.*
>>>>>> openid
>>>>>>
>>>>>> /oauth2/token
>>>>>>
>>>>>> https://${carbon.local.ip}:${https.nio.port}/revoke
>>>>>> false
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> NuwanD.
>>>>>>
>>>>>> --
>>>>>> Nuwan Dias
>>>>>>
>>>>>> Technical Lead - WSO2, Inc. http://wso2.com
>>>>>> email : nuw...@wso2.com
>>>>>> Phone : +94 777 775 729
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Bhathiya Jayasekara*
>>>>> *Senior Software Engineer,*
>>>>> *WSO2 inc., http://wso2.com <http://wso2.com>*
>>>>>
>>>>> *Phone: +94715478185 <%2B94715478185>*
>>>>> *LinkedIn: http://www.linkedin.com/in/bhathiyaj
>>>>> <http://www.linkedin.com/in/bhathiyaj>*
>>>>> *Twitter: https://twitter.com/bhathiyax
>>>>> <https://twitter.com/bhathiyax>*
>>>>> *Blog: http://movingaheadblog.blogspot.com
>>>>> <http://movingaheadblog.blogspot.com/>*
>>>>>
>>>>> ___
>>>>> Architecture mailing list
>>>>> Architecture@wso2.org
>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Malintha Amarasinghe
>>>> Software Engineer
>>>> *WSO2, Inc. - lean | enterprise | middleware*
>>>> http://wso2.com/
>>>>
>>>> Mobile : +94 712383306
>>>>
>>>> ___
>>>> Architecture mailing list
>>>> Architecture@wso2.org
>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks
>>> Abimaran Kugathasan
>>> Senior Software Engineer
>>>
>>> Email : abima...@wso2.com
>>> Mobile : +94 773922820
>>>
>>> <http://stackoverflow.com/users/515034>
>>> <http://lk.linkedin.com/in/abimaran>
>>> <http://www.lkabimaran.blogspot.com/> <https://github.com/abimarank>
>>> <https://twitter.com/abimaran>
>>>
>>>
>>> ___
>>> Architecture mailing list
>>> Architecture@wso2.org
>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>
>>>
>>
>>
>> --
>> *Susinda Perera*
>> Software Engineer
>> B.Sc.(Eng), M.Sc(Computer Science), AMIE(SL)
>> Mobile:(+94)716049075
>> Blog: susinda.blogspot.com
>> WSO2 Inc. http://wso2.com/
>> Tel : 94 11 214 5345 Fax :94 11 2145300
>>
>>
>> ___
>> Architecture mailing list
>> Architecture@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
> Nuwan Dias
>
> Technical Lead - WSO2, Inc. http://wso2.com
> email : nuw...@wso2.com
> Phone : +94 777 775 729
>
> ___
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>
--
*Janaka Ranabahu*
Associate Technical Lead, WSO2 Inc.
http://wso2.com
*E-mail: jan...@wso2.com <http://wso2.com>**M: **+94 718370861*
Lean . Enterprise . Middleware
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Generating a single Event per Message from our Products
Hi Srinath, On Fri, Mar 25, 2016 at 11:26 AM, Srinath Perera <srin...@wso2.com> wrote: > As per meeting ( Paricipants: Sanjiva, Shankar, Sumedha, Anjana, Miyuru, > Seshika, Suho, Nirmal, Nuwan) > > Currently we generate several events per message from our products. For > example, when a message hits APIM, following events will be generated. > > >1. One from HTTP level >2. 1-2 from authentication and authorization logic >3. 1 from Throttling >4. 1 for ESB level stats >5. 2 for request and response > > If APIM is handling 10K TPS, that means DAS is receiving events in about > 80K TPS. Although data bridge that transfers events are fast, writing to > Disk ( via RDBMS or Hbase) is a problem. We can scale Hbase. However, that > will run to a scenario where APIM deployment will need a very large > deployment of DAS. > > We decided to figure out a way to collect all the events and send a single > event to DAS. Basically idea is to extend the data publisher library such > that user can keep adding readings to the library, and it will collect the > readings and send them over as a single event to the server. > > However, some flows might terminated in the middle due to failures. There > are two solutions. > > >1. Get the product to call a flush from a finally block >2. Get the library to auto flush collected reading every few seconds > > I feel #2 is simpler. > > Do we have any concerns about going to this model? > > Suho, Anjana we need to think how to do this with our stream definition as > we force you to define the streams before hand. > Can't we write something similar to JDBC batch processing where the code would only do a publisher.addBatch() or something similar. The data publisher can be configured to flush the batched requests to DAS when they hit a certain threshold. Ex:- We define the batch size as 10(using code or config xml). Then if we have 5 streams, the publisher would send 5 requests to DAS(for each stream) instead of 50. IMO, this would allow us to keep the existing stream definitions and reduce the number of calls from a server to DAS. WDYT? Thanks, Janaka > > --Srinath > > > > > > -- > > Blog: http://srinathsview.blogspot.com twitter:@srinath_perera > Site: http://home.apache.org/~hemapani/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > > ___ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Janaka Ranabahu* Associate Technical Lead, WSO2 Inc. http://wso2.com *E-mail: jan...@wso2.com <http://wso2.com>**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Analytics][APIM] - Implement Geo location graph in Analytics
espect to the long >>>>> value for the given ip which between this above long values. Hope you will >>>>> got this idea on our approach. Is there any way to do bit wise operation >>>>> in >>>>> order to get the network_cidr value ? . >>>>> >>>> >>> Can't we do it by keeping the network IP and the subnet as two columns >>> and the geoname_id as the third. Say for example, if 192.168.0.0/20 is >>> the cidr, for IPv4 routing what is usually done is we get the IP as int, >>> then do a bitwise AND with the subnet mask (e.g. if subnet mask is 20, that >>> would mean 20 bits with value 1 and remaining 12 bits of value 0, i.e. >>> 0) and check whether that returns the >>> network IP. >>> >>> You might find more info here [1]. I think there should be libraries >>> that wrap this operation. But if performance is a concern and we need to >>> keep the cache search implementation very lean, we can implement it >>> ourselves. >>> >>> WDYT? >>> >>> [1] >>> http://stackoverflow.com/questions/4209760/validate-an-ip-address-with-mask >>> >>> Thanks, >>> Lasantha >>> >>> >>>>> Thanks >>>>> Tharindu >>>>> >>>>> On Mon, Mar 7, 2016 at 12:05 AM, Lasantha Fernando <lasan...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> I think what Sachith suggests also makes sense. But am also rooting >>>>>> for the in-memory cache implementation suggested by Sanjeewa with >>>>>> ip-netmask approach. >>>>>> >>>>>> Please find my comments inline. >>>>>> >>>>>> On 5 March 2016 at 23:50, Sachith Withana <sach...@wso2.com> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> From what I understand/was told, this happens once a day ( or >>>>>>> relatively infrequently), and you wanna avoid searching through all the >>>>>>> geo >>>>>>> data per ip ( since you are grouping the requests by IP). >>>>>>> >>>>>>> IF that's the case, it would be better to use a separate DB table to >>>>>>> cache these data ( IP, geoID ..etc) with the IP being the primary key ( >>>>>>> which would improve the lookup time), and even though there will be >>>>>>> cache >>>>>>> misses, it would eventually reduce the (#cacheMisses/ Hits). >>>>>>> >>>>>>> Having a DB cache would be better since you do want to persist these >>>>>>> data to be used over time. >>>>>>> >>>>>>> BTW in a cache miss, if we can figure out a way to limit the search >>>>>>> range on the original table or at least stop the search once a match is >>>>>>> found, it would greatly improve the cache miss time as well. >>>>>>> >>>>>>> That's my two cents. >>>>>>> >>>>>>> Cheers, >>>>>>> Sachith >>>>>>> >>>>>>> On Sun, Mar 6, 2016 at 8:24 AM, Janaka Ranabahu <jan...@wso2.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Sanjeewa, >>>>>>>> >>>>>>>> On Sun, Mar 6, 2016 at 7:25 AM, Sanjeewa Malalgoda < >>>>>>>> sanje...@wso2.com> wrote: >>>>>>>> >>>>>>>>> Implementing cache is better than having another table mapping >>>>>>>>> IMO. What if we query database and keep IP range and network name in >>>>>>>>> memory. >>>>>>>>> Then we may do quick search on network name and then based on that >>>>>>>>> rest can load some other way. >>>>>>>>> WDYT? >>>>>>>>> >>>>>>>> We thought of having an in memory cache but we faced several >>>>>>>> issues along the way. Let me explain the situation as it is per now. >>>>>>>> >>>>>>>> The Max-Mind DB has the IP addresses with the IP and the netmask. >>>>>>>> Ex: 192.168.0.0/20 >>>>
[Architecture] [AS 6.0.0] HTTP Monitoring dashboard - Why are we parsing the User-Agent header in the event builder?
Hi App Server team, According to the code in [1], the user-agent string is parsed and some of the information are extracted from the user-agent at event publishing time. Could you guys please clarify why you guys haven't published the whole user-agent string to DAS and use a UDF to extract the corresponding data at data summarization time? There are several concerns I see in the current approach. 1. This will add additional overhead to the server when processing each request as it has to process the user-agent string to filter out these data. 2. We are currently limiting the information that can be extracted from the user-agent at the data publishing time. If we publish the whole user-agent string, then the users have the option of coming up with a new analytics script to extract any data from the user-agent. 3. If we encounter a bug/limitation or upgrade/replace in the user-agent processing library, then we have to change/update the event publisher code. Having a user defined function in DAS to extract the information from the user-agent would address this scenario as we do not have to do any changes to the data publishers. 4. We need to parse the user-agent from all the places where we publish the HTTP data. Based on the current plans, if we are going to integrate the HTTP Monitoring dashboard to API Manager, then from the API Manager side, we also have to parse the user-agent and extract the data from the gateway nodes before publishing the data. Therefore I see that the better approach would be to publish the whole user-agent string and extract data from DAS data summarization time. WDYT? Thanks, Janaka [1] https://github.com/wso2/product-as/blob/wso2as-6.0.0/modules/http-statistics-monitoring/src/main/java/org/wso2/appserver/monitoring/utils/EventBuilder.java -- *Janaka Ranabahu* Associate Technical Lead, WSO2 Inc. http://wso2.com *E-mail: jan...@wso2.com <http://wso2.com>**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Analytics][APIM] - Implement Geo location graph in Analytics
to give us the most suitable way of doing this solution?. >> >> [1] - Implementing Geographical based Analytics in API Manager mail >> thread. >> >> [2] - http://dev.maxmind.com/geoip/geoip2/geolite2/ >> >> >> *Thanks* >> >> *Tharindu Dharmarathna* >> Associate Software Engineer >> WSO2 Inc.; http://wso2.com >> lean.enterprise.middleware >> >> mobile: *+94779109091 <%2B94779109091>* >> > > > > -- > > *Sanjeewa Malalgoda* > WSO2 Inc. > Mobile : +94713068779 > > <http://sanjeewamalalgoda.blogspot.com/>blog > :http://sanjeewamalalgoda.blogspot.com/ > <http://sanjeewamalalgoda.blogspot.com/> > > > -- *Janaka Ranabahu* Associate Technical Lead, WSO2 Inc. http://wso2.com *E-mail: jan...@wso2.com <http://wso2.com>**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [API Analytics] Health Monitoring
Hi Maheshakya, On Thu, Mar 3, 2016 at 4:30 PM, Maheshakya Wijewardena <mahesha...@wso2.com> wrote: > Hi, > > Due to the potential security issues and complexity issues arose in the > discussions among APIM analytics team members, we have decided to get rid > of the separate Java client for API health monitoring. Instead, this will > be implemented as CEP execution plans like other APIM analytics tasks. The > following information will be extracted from the APIM event streams. > I believe we agreed not to call this API Health monitoring as we can not guarantee the health of the API using only these values. Instead we opted to call this as something like API Availability. Lets finalize a new name for this as well. IMO, we should be calling this as "API Availability". WDYT? Thanks, Janaka > >1. API request count frequency. >2. API response time >3. API response error codes > > The criteria to categorize healthy/unhealthy scenario is as follows: > >1. If the API request count frequency gets lower than the 90th >percentile of the overall, that API is not healthy within the given period >of time. >2. If the API response times get higher than 90th percentile of the >overall, that API is not healthy within the given period of time. >3. If the API response contains error codes (given in configuration, >eg: 5xx, 4xx, that API is not healthy within the given period of time. > > If an API is stated as no healthy, the reason will be provided (one or > more of the above three). > > Best regards. > > On Tue, Mar 1, 2016 at 10:07 AM, Maheshakya Wijewardena < > mahesha...@wso2.com> wrote: > >> Hi, >> >> We have decided not to include the HTTP methods that affects the backends >> of APIs, such as POST, DELETE since the health checker should not cause any >> differences in the backend. >> >> Best regards. >> >> On Tue, Feb 23, 2016 at 10:53 AM, Maheshakya Wijewardena < >> mahesha...@wso2.com> wrote: >> >>> Hi, >>> >>> We have discussed (APIM team and analytics team) and came to a decision >>> to implement the health monitor for APIM as a separate Java client. The >>> following diagram depicts the architecture of the health monitor. >>> >>> >>> >>> >>>- Health checker incorporates the full life cycle of an API, >>>therefore it is located outside the firewall and can be viewed by the >>> admin >>>of the APIM. >>>- This client consumes quota from the whole API, therefore the >>>client needs to manage it. >>>- This Java client will ping (call API methods from the client) the >>>APIs periodically (hourly, daily, weekly) and tracks the status codes of >>>the responses. >>>- For each API, there will be a separate thread spawned and that >>>thread will keep track of that API. >>>- This information will be published to DAS inside the firewall >>>using HTTPS protocal. >>>- Based on the status code, a status will be reported as one of the >>>following (These are to be decided): >>> 1. Healthy - 2xx >>> 2. Some problems occurred during the time period - 4xx >>> 3. Failures reported - 5xx >>>- Following configuration information will be given the the health >>>monitor using a yaml file: >>> - APIM credentials: >>> - APIM username >>> - APIM password >>> - APIM configuration info: >>> - APIM host >>> - APIM port >>> - Application name >>> - Application tier >>> - Application key type >>> - Application key validity time (default -1, infinitely valid) >>> - Information of APIs: >>> - API name >>> - API url >>> - API version >>> - API provider >>> - API tier >>> - API parameters (if available) >>> - API payload (if available) >>> - DAS creadentials: >>> - DAS username >>> - DAS password >>> - DAS configuration info: >>> - DAS receiver url >>> - An email notification can be configured to be sent when a >>>failure is reported for an API. >>> >>> WDYT? >>> >>> Best regards. >>> >>> References: >>> >>> [1] API metrics: http://apimetrics.io/check-api-health/ >>> -- >>> Pruthuvi Maheshakya Wijewardena >>> mahesha...@wso2.com >>> +94711228855 >>> >>> >>> >> >> >> -- >> Pruthuvi Maheshakya Wijewardena >> mahesha...@wso2.com >> +94711228855 >> >> >> > > > -- > Pruthuvi Maheshakya Wijewardena > mahesha...@wso2.com > +94711228855 > > > -- *Janaka Ranabahu* Associate Technical Lead, WSO2 Inc. http://wso2.com *E-mail: jan...@wso2.com <http://wso2.com>**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] APIM Subscriber Notification
Hi Jo, On Wed, Feb 10, 2016 at 8:34 AM, Joseph Fonseka <jos...@wso2.com> wrote: > Hi Sam > > Few thought related to notification. > > 1. As I see notification can be a more generic use case not specific to > notifying new API versions. > Ex. Notify when a new API is added. > Is this really needed? IMO, we should not do this. The reason is that, IMO, all the users will not have a need to know about the new APIs that are added. Even if we allow them to choose whether to get notifications on new APIs, it is not useful practically. Users will subscribe and use APIs based on their use-case and for an end user, having a new API will not make them change their use-case and re-implement what they have done. If a user needs a new API and a set of new features, he/she would come to the store and search for a particular API that matches their needs. Notify when subscription is blocked. > IMO, this is also not practical. If we are to do this, then we need to have a mechanism of determining when to send an alert to the and to whom should we send the alert saying that their subscription is blocked. Currently if the subscription is blocked, then we anyhow send that in the response message. Thanks, Janaka > Notify throttling limit reach. > Notify when action related to workflow is performed. > etc.. > Thus when developing the notification component we should keep provision > to support above. > > 2. As Amila pointed out we should provide an option to enable / disable > notifications based on subscriber preference. > > Thanks & Regards > Jo > > > > > > On Wed, Feb 10, 2016 at 2:00 AM, Amila De Silva <ami...@wso2.com> wrote: > >> Hi Sam, >> >> Will the notification be pushed to all the Subscribers of an API, or does >> the Subscriber have an option to select for which APIs they should be >> receiving notifications for? >> >> On Tue, Feb 9, 2016 at 7:26 PM, Sam Sivayogam <s...@wso2.com> wrote: >> >>> Hi All, >>> >>> Currently in APIM when a publisher makes a new version of an existing >>> api there is no OTB solution to notify the existing subscibers. >>> >>> Since there is requirement for this, we planned to introuduce a >>> Notification Feature in the coming APIM release. This Feature will help the >>> publishers to send a notification message to the existing subscribers; when >>> they are making a new copy of an existing API. since most users prefer >>> notifications to be sent through email, we will provide a default >>> implementation which will notify the subscribers through email. >>> >>> What we are planning to do >>> >>> 1. Create a EmailNotifier class which will send notification emails to >>> the existing subscribers. >>> 2. Provide a Notifier interface which will help the publishers to create >>> their own Notification implementations according to their requirements such >>> as SMSNotification. >>> 3. Create a admin dashboard page to add email configurations, these >>> configurations will be saved in a registry and will be used by EmailNotifier >>> 4. Add an option to select the Notify subscribers in the API copying page >>> 5. Planning to Implement the EmailNotifier class using JAVA Mail API[1] >>> >>> Any suggestions and thoughts are highly appreciated. >>> >>> [1] http://www.oracle.com/technetwork/java/faq-135477.html#1 >>> >>> -- >>> *Sam Sivayogam* >>> >>> Software Engineer >>> Mobile : +94 772 906 439 >>> Office : +94 112 145 345 >>> *WSO2, Inc. :** wso2.com <http://wso2.com/>* >>> lean.enterprise.middleware. >>> >> >> >> >> -- >> *Amila De Silva* >> >> WSO2 Inc. >> mobile :(+94) 775119302 >> >> > > > -- > > -- > *Joseph Fonseka* > WSO2 Inc.; http://wso2.com > lean.enterprise.middleware > > mobile: +94 772 512 430 > skype: jpfonseka > > * <http://lk.linkedin.com/in/rumeshbandara>* > > -- *Janaka Ranabahu* Associate Technical Lead, WSO2 Inc. http://wso2.com *E-mail: jan...@wso2.com <http://wso2.com>**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Meeting Note : Implement a Pluggable Version Strategy feature in APIManager
attribute to RXT to store context template (This is
used to display the context template in UI when editing the API)
I think that the second option is better than adding new columns to the
tables.
Please share your suggestions or thoughts on this.
Thanks
*Dinesh J. Weerakkody*
Software Engineer
WSO2 Inc.
lean | enterprise | middleware
M : +94 727 361788 | E : dine...@wso2.com | W : www.wso2.com
On Tue, Dec 23, 2014 at 5:43 PM, Dinesh J Weerakkody dine...@wso2.com
wrote:
*Participants*
APIM Team : Sumedha, Nuwan, Sanjeewa, Jo, Roshan, Lakshman, Dinesh
ESB Team : Kasun
*Redmine*
https://redmine.wso2.com/issues/153
We have discussed the following options as solutions to implement a
pluggable version strategy for APIM manager product.
*Option 1 : include version to context path*
We allow users to define API context as a template in API create UI.
Context = /api/{uri.var.version}/anytext
Version = 1.0.0
Then we generate the context path based on a the template when
creating the synapse xml and use default version-type (empty).
*Current version*
api xmlns=http://ws.apache.org/ns/synapse;
name=admin-API
context=/api”
version=1.0.0
*version-type=url*
*Proposed version*
api xmlns=http://ws.apache.org/ns/synapse;
name=admin-API
*context=/api/1.0.0*
version=1.0.0
Please note that version appended to context while version tag keeping
as it is. If user does not include the {uri.var.version}, current process
will be used.
Ex:
context=/1.0.0/api
context=/gov/1.0.0/api
context=/gov/1.0.0/api2
*Option 2 : Match APIs based on configured context pattern*
api xmlns=http://ws.apache.org/ns/synapse;
name=admin-API
context-pattern=”{uri.var.context}/{uri.var.version}”
context=/api”
version=1.0.0
version-type=url
Ex:
{uri.var.version}/{uri.var.context}= http://localhost:8280/1.0.0/api
{uri.var.context}/{uri.var.version}= http://localhost:8280/api/1.0.0/
In synapse level we have to uniquely identify a API with the context
and version. With second option, it will be difficult and can be lead to
conflicts If user define an API with context=”1.0.0” version=”api” while
having an API with context=”api” version=”1.0.0”. So we have decided to
drop the second option and throughly test the first option.
Please share your suggestions or thoughts on this.
Thanks
*Dinesh J. Weerakkody*
Software Engineer
WSO2 Inc.
lean | enterprise | middleware
M : +94 727 361788 | E : dine...@wso2.com | W : www.wso2.com
--
Nuwan Dias
Associate Tech Lead - WSO2, Inc. http://wso2.com
email : nuw...@wso2.com
Phone : +94 777 775 729
--
*Janaka Ranabahu*
Senior Software Engineer; WSO2 Inc.; http://wso2.com
*E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861
%2B94%20718370861*
Lean . Enterprise . Middleware
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Dev][G-Reg]Copy to new version functionality in G-Reg
/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com *E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available
. If it is one to many there will be same deployer mentioned in several app types. Should we focus on that feature here. If it is to implement many to many relationship then we are doing that feature now. You are correct in saying we need to concentrate on this current feature only. But since we are going to be doing refactoring anyway I propose to separate out Runtime concept gradually. There is no harm in adding a new meta data object saying Runtime. It can only do us good. Less refactoring in the future. Could you tell me why we need to add cartridge.xml to *.apptype archive itself? I prefer to keep it separate as a part of PaaS. And use Runtime object to refer to it. I don't want CartridgeInfo objects being used all over in our code. Rather I prefer to see Runtime being used because AppFactory and Stratos can evolve fairly independently. thanks, dimuthu Right now it is 1 to 1 relationship. In future it is 1 to n relationship. So if we keep it in a separate XML the work we have to do is simply externalise the deployment xmls to support the below scenario. For example consider app creation page. There is a Application Type drop down. In future we are going to have Runtimes drop down. So if we have deployment as a separate concept in the architecture it is going to be much better. thanks, dimuthu Look forward your views in this. Thanks Regards, S.A.Rajeevan Software Engineer WSO2 Inc E-Mail: rajeev...@wso2.com | Mobile : +94776411636 -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware Thanks Regards Danushka Fernando Software Engineer WSO2 inc. http://wso2.com/ Mobile : +94716332729 -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com *E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Moving deployment section from configuration xml and Facilitate AppFactory users to deploy apps with their custom app types if its not already available
Hi Dimuthu,
On Thu, Dec 4, 2014 at 10:42 AM, Dimuthu Leelarathne dimut...@wso2.com
wrote:
Hi Guys,
Please start with why. Lets do minimal to achieve why with most
healthiest way. Why we need to do this is to external parties to add
interpreter based languages and their cartridges. We do not need a new
deployer type right now. I proposed a new Runtime object only because we
need to minimise the refactoring in the future. I don't think we need a
deployer IMO.
Maybe I raised my question in a wrong manner.
I'm not talking about introducing a new deployer. Please see my comments
below.
Also please let me know whether I'm raising a invalid question.
thanks,
dimuthu
On Thu, Dec 4, 2014 at 10:06 AM, Janaka Ranabahu jan...@wso2.com wrote:
Hi Rajeevan,
Could you explain a bit more on how we are going to relate the lifecycle
stage with the runtime? If you look at the appfactory.xml, you might have
noticed that the Deployer information is defined for each lifecycle stage.
So with this new runtime.xml, how are we going to address that? Does the
runtime.xml contains all configurations or are we going to have different
sub directories and have a runtime.xml in each of them for each lifecycle
environment?
Thanks,
Janaka
On Wed, Dec 3, 2014 at 2:17 PM, Danushka Fernando danush...@wso2.com
wrote:
Hi Dimuthu and All
We decided to go with a separate file for this runtime configs. So we
will deploy this file to a different location with a different axis2
deployer. And we will mention in the apptype.xml which runtime to be used.
WDYT?
Thanks Regards
Danushka Fernando
Software Engineer
WSO2 inc. http://wso2.com/
Mobile : +94716332729
On Wed, Dec 3, 2014 at 11:49 AM, Aiyadurai Rajeevan rajeev...@wso2.com
wrote:
Hi Dimuthu,
Thanks for the suggestion. So, As a conclusion I will go ahead with the
implementation as having a runtime.xml for the whole below peroperties and
populate a map from there.
The following section maps directly with the existing Deployer
section in the appfactory.xml which we already defines for each stage.
Runtime
Runtimeappserver/Runtime
ClassNameorg.wso2.carbon.appfactory.jenkins.deploy.JenkinsArtifactDeployer/ClassName
Endpointhttps://sc.s2.AF_HOST:9463/services//Endpoint
This endpoint
is different from one environment to another even for a single runtime.
RepositoryProvider
ProviderClassorg.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider/ProviderClass
BaseURLhttps://gitblit.s2.wso2.com:8444//BaseURL
URLPattern{@stage}/as/URLPattern
This URL pattern also different from one environment to another even
for a single runtime.
Previously we had 3 such configurations which defines these changing
properties of a runtime environment. My question is, if we are defining a
runtime of a new apptype how are we going to map the lifecycle stages of
that apptype with the runtime?
Thanks,
Janaka
AdminUserNameadmin/AdminUserName
AdminPasswordadmin/AdminPassword
/RepositoryProvider
AliasPrefixas/AliasPrefix
CartridgeTypePrefixas/CartridgeTypePrefix
DeploymentPolicyaf-deployment/DeploymentPolicy
AutoscalePolicyeconomy/AutoscalePolicy
RepoURL/RepoURL
DataCartridgeType/DataCartridgeType
DataCartridgeAlias/DataCartridgeAlias
SubscribeOnDeploymentfalse/SubscribeOnDeployment
/Runtime
Thanks Regards,
S.A.Rajeevan
Software Engineer WSO2 Inc
E-Mail: rajeev...@wso2.com | Mobile : +94776411636
On Wed, Dec 3, 2014 at 11:03 AM, Dimuthu Leelarathne dimut...@wso2.com
wrote:
HI Rajeevan,
No GUI please. We are changing the whole user story here.
thanks,
dimuthu
On Wed, Dec 3, 2014 at 10:54 AM, Aiyadurai Rajeevan
rajeev...@wso2.com wrote:
Hi Dimuthu/All,
In addition to this mail conversation we had discussed this in an
internal forum, Here is the update of thatdiscussion
As of today We are using appfactory.xml file for the runtime
configurations the below fraction is the the configuration properties.
ApplicationType name=*
ClassName
org.wso2.carbon.appfactory.jenkins.deploy.JenkinsArtifactDeployer
/ClassName
Endpointhttps://sc.s2.AF_HOST:9463/services//Endpoint
RepositoryProvider
Property name=Class
org.wso2.carbon.appfactory.s4.integration.GITBlitBasedGITRepositoryProvider\
/Property
Property name=BaseURLhttps://gitblit.s2.wso2.com:8444/
/Property
Property name=URLPattern{@stage}/as/Property
Property name=AdminUserNameadmin/Property
Property name=AdminPasswordadmin/Property
/RepositoryProvider
Properties
Property name=aliasasdev/Property
Property name=cartridgeTypeasdev/Property
Property name=deploymentPolicyaf-deployment
/Property
Property name=autoscalePolicyeconomy/Property
Property name=repoURL
Re: [Architecture] Remove multiple backend calls that happens when loading resource overview page
Hi Amalka, Previously we had database users and database template creation capability from the databases/datasources page. Where are these operations now? Are they with the newly added 'Databases' page? I'm unable to locate them in the attached screenshots. Thanks, Janaka On Wed, Oct 15, 2014 at 8:46 AM, Amalka Subasinghe ama...@wso2.com wrote: Hi, Currently when loading App Factory resources overview page, it do multiple backend calls to load the various information; to reduce backend calls we are planning to do the following as an initial step. 1. Move Resources - databases section to new tab called 'Databases' and change the layout of it 2. Load Resources page only from App Factory database Thanks Amalka ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com *E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Remove multiple backend calls that happens when loading resource overview page
Hi Amalka, On Tue, Nov 4, 2014 at 8:13 AM, Amalka Subasinghe ama...@wso2.com wrote: Hi Janaka, According to the new changes, when creating a database, it will create default user and template. If user want to create his own user and template, he can go for 'advanced options'. So that mean that we do not have a way to add a user/template before adding a database right? Also with this changes, can I edit/remove any of my existing templates/users? I see no way of doing that with the new set of UIs. Thanks, Janaka Thanks Amalka On Tue, Nov 4, 2014 at 8:03 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Amalka, Previously we had database users and database template creation capability from the databases/datasources page. Where are these operations now? Are they with the newly added 'Databases' page? I'm unable to locate them in the attached screenshots. Thanks, Janaka On Wed, Oct 15, 2014 at 8:46 AM, Amalka Subasinghe ama...@wso2.com wrote: Hi, Currently when loading App Factory resources overview page, it do multiple backend calls to load the various information; to reduce backend calls we are planning to do the following as an initial step. 1. Move Resources - databases section to new tab called 'Databases' and change the layout of it 2. Load Resources page only from App Factory database Thanks Amalka ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com *E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Amalka Subasinghe WSO2 Inc. Mobile: +94 77 9401267 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com *E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] G-Reg vs ES for artefacts
Hi Guys, Will these limitations get fixed in the upcoming release? If so do we have a timeline for that? Thanks, Janaka On Sat, Mar 8, 2014 at 10:43 AM, Chandana Napagoda chand...@wso2.comwrote: Hi Samisa In addition to above issue, ES is still not supporting all the RXT configuration elements(ex: unbounded table option) which are currently supported by G-Reg. We have reported JIRAs to get those added to coming up releases. Regards, Chandana On Mar 8, 2014 7:09 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Samisa, IMO, it should be ES. But the current ES architecture does not pick new RXT types automatically and some configurations to jaggery config files are required to show the new asset in both Store and Publisher[1]. Until we have a more flexible way of generating the Store and Publisher UI, I believe we have to stick with G-Reg. Thanks, Janaka [1] https://docs.wso2.org/display/ES100/Adding+a+New+Asset+Type On Fri, Mar 7, 2014 at 8:08 PM, Samisa Abeysinghe sam...@wso2.comwrote: We used to position G-Reg as a store of anything with RXT. Now what ES is there, and is more capable, do we propose the users use ES or G-Reg RXT like use cases? Thanks, Samisa... Samisa Abeysinghe Vice President Developer Evangelism WSO2 Inc. http://wso2.com ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Improve G-Reg default lifecycle to Enterprise level
Hi Isabelle, This is something that should be part of Governance 2.0. I was not aware that Governance 2.0 was in route before C5 time frame and sorry for the confusion. Also this requirement is not about adding more states to the lifecycle. I haven't done a full analysis of the real world scenarios but it seems to me that we need to add validations(and may be transitions UIs and Executors) depending on the model we go with. One such example is that, if our lifecycle model has a check item called Architecture document attached, then we should have a validation step to verify that. Therefore to fulfill this requirement we might need to introduce new asset types and lifecycle extensions. Thanks, Janaka On Mon, Mar 3, 2014 at 5:26 AM, Isabelle Mauny isabe...@wso2.com wrote: How is that related to C5 ?? C5 is a research project at this point.. Coming up with a more representative LC as default can be done now, based on our extensive experience with customers... Isabelle. -- Isabelle Mauny Director, Product Management; WSO2, Inc.; http://wso2.com/ email: isabe...@wso2.com isabe...@wso2.com - mobile: +34 616050684 On Mon, Mar 3, 2014 at 5:07 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi G-Reg team, The default G-Reg lifecycle needs to be improved to Enterprise level. The current one we have only has 3 lifecycle states which does not have cover the full SDLC. Also we do not have a good set of check items in each state. Could we take some of the real world samples which are been widely used and model our default lifecyle based on those. That would improve the user experience a lot. Also do you guys have any plans to improve this with C5? WDYT? Thanks, Janaka -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [G-Reg] What is the purpose of allowing users to modify a lifecycle that is already in use
On Sun, Mar 2, 2014 at 11:26 PM, Eranda Sooriyabandara era...@wso2.comwrote: Hi Janaka, On Mon, Feb 24, 2014 at 7:19 PM, Janaka Ranabahu jan...@wso2.com wrote: Hi Subash, On Fri, Feb 21, 2014 at 5:43 PM, Subash Chaturanga sub...@wso2.comwrote: HI Janaka, In a Prod or non Prod env ithere can be a use case that at a given point of time, governance master needs to refactor the LC config (some config change as Ajith mentioned) and reflect the changes on all associated 100s of services. So I believe this feature is a useful one (may be in a new form like refactor). But the current behavior of G-Reg is not to update the existing artifacts. Is that a design decision? WDYM by not allow to update? If you modify a check item name in a state, none of the existing resources in that state does not reflect that change. Thanks, Janaka thanks Eranda Thanks, Janaka Vendors like WSRR have the capability to maintain versions of the same LC, which is a good feature to have for GReg as well, where it will allows users to create versions of LC for those major/minor changes and it will avoid those inconsistencies. On Sat, Feb 22, 2014 at 3:51 AM, Janaka Ranabahu jan...@wso2.comwrote: Hi Ajith, Please see my concerns inline. On Fri, Feb 21, 2014 at 4:33 PM, Ajith Vitharana aji...@wso2.comwrote: Hi Janaka, First ***ServiceLifeCycle is already in use and your changes may affect existing usage * is sufficient warning. Because we can't add lengthy descriptions in small UI box. Practically, do we change the *LC state id* having large number of artifacts in a production system ?. What happens if some user do so? This may not happen in a production instance. But in a developer environment this is something very likely to happen. Renaming the existing *LC state id* is only one use case. yes it is only one possibility. It is the first one I tried and it created couple of inconsistent behaviors. Also we can not limit users to not to change the state id. It is one possibility that we have allowed. Following are are the valid points of allowing to update the existing LC configuration. 1. Add new state to existing configurations. 2. Add/update executors, validators, transition UI ..etc. These 2 are not likely scenarios for a already defined LC. If there is a requirement for a new state, that indicates that the organization process has changed. Then what we need is a new version of the LC and not to modify the current one. 3. Add/Update the parameters. 4 Add/update the check Items ..etc This will not work for the existing resources. I tried this and it was not reflected immediately. So that is why I raised the concern that by allowing this, there can be resources in the same state but with different check items. So IMO we should not allow to modify a LC if it is already in use. By not allowing, we can enforce the users to have a naming conversion for the LC, which is version aware and there is very limited opportunity for inconsistent behavior. WDYT? Thanks, Janaka Thanks. Ajith. On Sat, Feb 22, 2014 at 2:16 AM, Janaka Ranabahu jan...@wso2.comwrote: Hi, I have noticed that we allow users to modify a lifecycle that is already assigned to resources. Did we do a complete analysis of the situations that could come by allowing this behavior? I have noticed the following inconsistent behavior due to this. When saving the lifecycle configuration, it only gives a warning saying* ServiceLifeCycle is already in use and your changes may affect existing usage. Are you sure you want to save these Lifecycle changes?* There is no descriptive explanation about what is the impact of the change. It happens to be that, if you rename a state, and if there were resources on that state, the lifecycle portlet disappear from the UI. There is no way to remove, add or do any action on the lifecycle for that resource and that resource is completely unusable(from LC point of view) from that point onward. Also we do not update the existing resources with the change. If we add a new check item or change a check item of an existing state, that change is not reflected for the resources that are already associated with the lifecycle. Because of that, G-Reg began to have resources with the same lifecycle, but with different behaviors. Therefore, I'm wondering what is the purpose of modifying a lifecycle which is already in use. IMO, this had introduced a inconsistent behavior and had not solve anything. The real problem comes when there is a high number of resources that is associated with the lifecycle. Thanks, Janaka -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware -- Ajith Vitharana. WSO2 Inc. - http://wso2.org Email : aji...@wso2.com Mobile : +94772217350
Re: [Architecture] [G-Reg] What is the purpose of allowing users to modify a lifecycle that is already in use
Hi Subash, On Fri, Feb 21, 2014 at 5:43 PM, Subash Chaturanga sub...@wso2.com wrote: HI Janaka, In a Prod or non Prod env ithere can be a use case that at a given point of time, governance master needs to refactor the LC config (some config change as Ajith mentioned) and reflect the changes on all associated 100s of services. So I believe this feature is a useful one (may be in a new form like refactor). But the current behavior of G-Reg is not to update the existing artifacts. Is that a design decision? Thanks, Janaka Vendors like WSRR have the capability to maintain versions of the same LC, which is a good feature to have for GReg as well, where it will allows users to create versions of LC for those major/minor changes and it will avoid those inconsistencies. On Sat, Feb 22, 2014 at 3:51 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Ajith, Please see my concerns inline. On Fri, Feb 21, 2014 at 4:33 PM, Ajith Vitharana aji...@wso2.com wrote: Hi Janaka, First ***ServiceLifeCycle is already in use and your changes may affect existing usage * is sufficient warning. Because we can't add lengthy descriptions in small UI box. Practically, do we change the *LC state id* having large number of artifacts in a production system ?. What happens if some user do so? This may not happen in a production instance. But in a developer environment this is something very likely to happen. Renaming the existing *LC state id* is only one use case. yes it is only one possibility. It is the first one I tried and it created couple of inconsistent behaviors. Also we can not limit users to not to change the state id. It is one possibility that we have allowed. Following are are the valid points of allowing to update the existing LC configuration. 1. Add new state to existing configurations. 2. Add/update executors, validators, transition UI ..etc. These 2 are not likely scenarios for a already defined LC. If there is a requirement for a new state, that indicates that the organization process has changed. Then what we need is a new version of the LC and not to modify the current one. 3. Add/Update the parameters. 4 Add/update the check Items ..etc This will not work for the existing resources. I tried this and it was not reflected immediately. So that is why I raised the concern that by allowing this, there can be resources in the same state but with different check items. So IMO we should not allow to modify a LC if it is already in use. By not allowing, we can enforce the users to have a naming conversion for the LC, which is version aware and there is very limited opportunity for inconsistent behavior. WDYT? Thanks, Janaka Thanks. Ajith. On Sat, Feb 22, 2014 at 2:16 AM, Janaka Ranabahu jan...@wso2.comwrote: Hi, I have noticed that we allow users to modify a lifecycle that is already assigned to resources. Did we do a complete analysis of the situations that could come by allowing this behavior? I have noticed the following inconsistent behavior due to this. When saving the lifecycle configuration, it only gives a warning saying* ServiceLifeCycle is already in use and your changes may affect existing usage. Are you sure you want to save these Lifecycle changes?* There is no descriptive explanation about what is the impact of the change. It happens to be that, if you rename a state, and if there were resources on that state, the lifecycle portlet disappear from the UI. There is no way to remove, add or do any action on the lifecycle for that resource and that resource is completely unusable(from LC point of view) from that point onward. Also we do not update the existing resources with the change. If we add a new check item or change a check item of an existing state, that change is not reflected for the resources that are already associated with the lifecycle. Because of that, G-Reg began to have resources with the same lifecycle, but with different behaviors. Therefore, I'm wondering what is the purpose of modifying a lifecycle which is already in use. IMO, this had introduced a inconsistent behavior and had not solve anything. The real problem comes when there is a high number of resources that is associated with the lifecycle. Thanks, Janaka -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware -- Ajith Vitharana. WSO2 Inc. - http://wso2.org Email : aji...@wso2.com Mobile : +94772217350 -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861 %2B94%20718370861* Lean . Enterprise . Middleware -- Thanks /subash *Subash Chaturanga* Senior Software Engineer :Integration TG; WSO2 Inc. http://wso2.com email: sub...@wso2.com blog: http://subashsdm.blogspot.com
[Architecture] [G-Reg] What is the purpose of allowing users to modify a lifecycle that is already in use
Hi, I have noticed that we allow users to modify a lifecycle that is already assigned to resources. Did we do a complete analysis of the situations that could come by allowing this behavior? I have noticed the following inconsistent behavior due to this. When saving the lifecycle configuration, it only gives a warning saying* ServiceLifeCycle is already in use and your changes may affect existing usage. Are you sure you want to save these Lifecycle changes?* There is no descriptive explanation about what is the impact of the change. It happens to be that, if you rename a state, and if there were resources on that state, the lifecycle portlet disappear from the UI. There is no way to remove, add or do any action on the lifecycle for that resource and that resource is completely unusable(from LC point of view) from that point onward. Also we do not update the existing resources with the change. If we add a new check item or change a check item of an existing state, that change is not reflected for the resources that are already associated with the lifecycle. Because of that, G-Reg began to have resources with the same lifecycle, but with different behaviors. Therefore, I'm wondering what is the purpose of modifying a lifecycle which is already in use. IMO, this had introduced a inconsistent behavior and had not solve anything. The real problem comes when there is a high number of resources that is associated with the lifecycle. Thanks, Janaka -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [AppFactory] Create AppFactory applications from already existing application binaries
Hi Shamika, On Mon, Jan 6, 2014 at 4:43 PM, Shamika Ariyawansa sham...@wso2.com wrote: Hi All, New feature that is going to be introduced to AppFactory is creating a new application by uploading exiting binary file of an application. e.g WAR *User Scenario* 1. User logs on to the system, goes to the application creation page. 2. In there user provides basic information related to the application, such as name, key, description then he/she would be able to create the application by choosing one of the following options, a. Create the application from the scratch by selecting the repository type and application type which maps with existing functionality. *OR* b. Create the application by uploading the binary file and selecting the binary file type. By doing so the application will be created as non build-able application. Can we improve this so that a user can create an application pointing to a existing source code so that AF can clone that instead of the default template? Thanks, Janaka 3. In Repos and Builds page user will be able to see the uploaded application and he/she will be able to do following operations from there, a. Delete the existing application. b. Upload new version of the same application. - Provides a way to upload new binary file. c. Test the application by deploying to Dev cloud. Note that for applications created like this, source repository paths, build options and not shown to the users. 4. From Life Cycle Management page user will be able to Promote and Demote the application through different life cycles. *Solution* So far in AppFactory we maintain two logical types of application flows. Buildable and non-Buildable. Buildabale applications are mainly handled and deployed by the buildserver (Jenkins) whereas non-Buildable are maintained and deployed by the AppFactory itself. uploading existing application functionality will be implemented considering Non-Buildable application flow as follows. [image: Inline image 2] Further App Creation, Build and Repos and other UIs will be changed accordingly. Regards, -- Shamika Ariyawansa Senior Software Engineer WSO2, Inc.; http://wso2.com LK - +94 7639629 Ext 5999 US - +1 408 754 7388 Ext 51732 Mob:+ 94 772929486 *twitter: **https://twitter.com/Amila_Shamika*https://twitter.com/Amila_Shamika * linked-in: *http://www.linkedin.com/pub/dir/Shamika/Ariyawansa *Lean . Enterprise . Middleware* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [AppFactory] Create AppFactory applications from already existing application binaries
Hi Manjula, On Tue, Jan 7, 2014 at 11:05 AM, Manjula Rathnayake manju...@wso2.comwrote: Hi Janaka, On Tue, Jan 7, 2014 at 10:41 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Manjula, While I completely agree that this feature is something different from what I've stated, I'm trying to understand the usability of this feature. From what I've gathered so far from this discussion is that, we do not allow code modifications, versions and build for these kinds of applications. So basically this feature is only there to deploy, test and govern a pre-built artifact. But then, we loose many powerful development features that are provided by AppFactory. With this feature, the source code management, versioning and build(which are three important features of AF) is happening out of AppFactory and they have to manage them separately. But instead, what we need to do is to promote AppFactory for developers so that developers would be attracted to it. IMO, having a feature to create applications from an existing repo is more important in that context. I understand your point, but uploading a war file(or any deployable artifact) is the feature we implement here. There might be situations where users do not need to share their source code into our source repository. They just need to maintain binary artifacts within appfactory. But the feature you mentioned is a valid feature we can provide and can be implemented separately. Understood. I have few more points to clarify. Now when a user creates such application project do we allow him to upload the same binary to the same version of that application. Say that the user found a bug in the uploaded application. What is the procedure of fixing the bug and testing it using AF. Does he need to create a new version to upload the new artifact or can he simply replace the existing artifact in current version? How can this help to enforce the governance best practices of an application? Eg:- when an application version in promoted to the production environment, how do we guarantee that there will be no commits and builds on that version? Even though that we can stop deployment from AF side, there is no guarantee that the original source is in the same state. Do we rename the binary to AppFactory default format when creating these types of applications? Otherwise default applications in AF will have one format(applicaiton_key-version) where this would have something else. Also If not, how does features like log download(which needs to find the application name to download logs) will work? Do we allow developers to be invited into these types of applications? If so what are the functionality available to them? Is it only deployment and developer testing or do we allow a developer to upload a new binary/new version of the artifact? Thanks, Janaka thank you. WDYT? On Mon, Jan 6, 2014 at 7:24 PM, Manjula Rathnayake manju...@wso2.comwrote: Hi all, On Mon, Jan 6, 2014 at 6:48 PM, Ashansa Perera asha...@wso2.com wrote: As I feel creating an application pointing to an existing code base is different than allowing to upload an artifact and create an application, since this will not include any source code management. So we should consider those as two different use cases. IMO we can relate the use case that Janaka has mentioned to our main flow of application creation where we can provide the option of - creating an application pointing to an existing source code ( so AF can clone that and create application) - creating an application with default template +1, this feature is about uploading a war file and no source code is involved. Also, is this only about war files or do we support all the applications types of AF and are we going to support other app types like aar and car? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 5:45 PM, Harsha Thirimanna hars...@wso2.comwrote: Hi Janaka, +1 for that. There will some work to do this. Because 1. we should process the pom file of that external repository and change artifactId, groupId to our one. 2. we have to have credentials to access external repository. thanks *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/* * email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770* * twitter: **http://twitter.com/ http://twitter.com/afkham_azeez* *harshathirimann linked-in: **http: http://lk.linkedin.com/in/afkhamazeez**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122 http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* *Lean . Enterprise . Middleware* On Mon, Jan 6, 2014 at 5:28 PM, Janaka Ranabahu jan...@wso2.comwrote: Hi Shamika, On Mon, Jan 6, 2014 at 4:43 PM, Shamika Ariyawansa sham...@wso2.comwrote: Hi All, New feature that is going to be introduced to AppFactory is creating a new application by uploading exiting binary file of an application. e.g WAR
Re: [Architecture] [AppFactory] Create AppFactory applications from already existing application binaries
Hi Manjula, Thanks for the clarifications. Seems good. Thanks, Janaka On Tue, Jan 7, 2014 at 11:54 AM, Manjula Rathnayake manju...@wso2.comwrote: Hi Janaka, On Tue, Jan 7, 2014 at 11:33 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Manjula, On Tue, Jan 7, 2014 at 11:05 AM, Manjula Rathnayake manju...@wso2.comwrote: Hi Janaka, On Tue, Jan 7, 2014 at 10:41 AM, Janaka Ranabahu jan...@wso2.comwrote: Hi Manjula, While I completely agree that this feature is something different from what I've stated, I'm trying to understand the usability of this feature. From what I've gathered so far from this discussion is that, we do not allow code modifications, versions and build for these kinds of applications. So basically this feature is only there to deploy, test and govern a pre-built artifact. But then, we loose many powerful development features that are provided by AppFactory. With this feature, the source code management, versioning and build(which are three important features of AF) is happening out of AppFactory and they have to manage them separately. But instead, what we need to do is to promote AppFactory for developers so that developers would be attracted to it. IMO, having a feature to create applications from an existing repo is more important in that context. I understand your point, but uploading a war file(or any deployable artifact) is the feature we implement here. There might be situations where users do not need to share their source code into our source repository. They just need to maintain binary artifacts within appfactory. But the feature you mentioned is a valid feature we can provide and can be implemented separately. Understood. I have few more points to clarify. Now when a user creates such application project do we allow him to upload the same binary to the same version of that application. Say that the user found a bug in the uploaded application. What is the procedure of fixing the bug and testing it using AF. Does he need to create a new version to upload the new artifact or can he simply replace the existing artifact in current version? If it is development stage, it can be replaced with new binary file. But if it is in other stages, it has to follow application life cycle. They can use issue tracker to report bugs and demote on testing stage or even in production stage. As per the diagram, initial development stage artifacts are stored in gitblit. How can this help to enforce the governance best practices of an application? Eg:- when an application version in promoted to the production environment, how do we guarantee that there will be no commits and builds on that version? Even though that we can stop deployment from AF side, there is no guarantee that the original source is in the same state. Yes, If someone manage their source code outside of AF, we can not do anything on that. But still they can use application life cycle flow, issue tracking, user management, log viewers like features. Do we rename the binary to AppFactory default format when creating these types of applications? Otherwise default applications in AF will have one format(applicaiton_key-version) where this would have something else. Also If not, how does features like log download(which needs to find the application name to download logs) will work? This is a limitation in appfactory. However we have decided to enforce this as artifact name matters in many places of appfactory(security, logs, deployment,etc). So we have to rename the binary. Do we allow developers to be invited into these types of applications? If so what are the functionality available to them? Is it only deployment and developer testing or do we allow a developer to upload a new binary/new version of the artifact? Application upload is done by app owners but we can allow developers as well in any case if appfactory users need to do so, as it is configurable. This is a feature to bring current existing applications into AppFactory and manage application life cycle. When they need to bring the source code, AF can provide a link to add source. In that case, we can commit the initial code and change the application type to default source code based application types. As you mentioned, some part of governance story is missing due to the fact that source code control is not done via AppFactory. thank you. Thanks, Janaka thank you. WDYT? On Mon, Jan 6, 2014 at 7:24 PM, Manjula Rathnayake manju...@wso2.comwrote: Hi all, On Mon, Jan 6, 2014 at 6:48 PM, Ashansa Perera asha...@wso2.comwrote: As I feel creating an application pointing to an existing code base is different than allowing to upload an artifact and create an application, since this will not include any source code management. So we should consider those as two different use cases. IMO we can relate the use case that Janaka has mentioned to our main flow
Re: [Architecture] Resilient application creation process
Hi Manjula, On Mon, Jan 6, 2014 at 10:24 AM, Manjula Rathnayake manju...@wso2.comwrote: Hi all, Another option is to retry to create the application even after failed. There we create the application again and again until it get created. If it fails, users should be able to role back. In this process, for example, if application rxt adding process succeeds but creating git repo fails, we should be able to create git repo and continue without trying to add the rxt. If I understood you correctly, then what we need to do is to retry the failed process/action a number of times. But that can not guarantee whether the app creation would complete successfully. Say that the git repo creation(or any other task) failed due to a network error or some other serious issue. Then retrying will not solve the issue. IMO, what we need in the first place is the rollback functionality and as an improvement we can do the retry logic. WDYT? Thanks, Janaka thank you. On Mon, Jan 6, 2014 at 10:06 AM, Ashansa Perera asha...@wso2.com wrote: Hi all, To make the application creation process resilient we discussed to implement a rollback mechanism so that if any resource/infrastructure creation failed while creating the application we can rollback the app creation. With this we would be able to reuse the same application key and utilize the resources. Another suggestion was to ping the external servers before starting application creation process, but since pinging servers cannot guarantee anything as server can fail in between, we will be going ahead with the rollback application creation. -- Thanks Regards, Ashansa Perera Software Engineer WSO2, Inc ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Manjula Rathnayaka Software Engineer WSO2, Inc. Mobile:+94 77 743 1987 ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [App Factory] Per Developer Repos
*Manisha Eleperuma* Software Engineer WSO2, Inc.: http://wso2.com lean.enterprise.middleware *blog: http://manisha-eleperuma.blogspot.com/ http://manisha-eleperuma.blogspot.com/* *mobile: +94 71 8279777 %2B94%2071%208279777* -- You received this message because you are subscribed to the Google Groups WSO2 Engineering Group group. To unsubscribe from this group and stop receiving emails from it, send an email to engineering-group+unsubscr...@wso2.com. For more options, visit https://groups.google.com/a/wso2.com/groups/opt_out. -- S.Uthaiyashankar VP Engineering WSO2 Inc. http://wso2.com/ - lean . enterprise . middleware Phone: +94 714897591 -- You received this message because you are subscribed to the Google Groups WSO2 Engineering Group group. To unsubscribe from this group and stop receiving emails from it, send an email to engineering-group+unsubscr...@wso2.com. For more options, visit https://groups.google.com/a/wso2.com/groups/opt_out. -- Shiroshica Kulatilake Architect, WSO2, Inc. http://wso2.com/ Phone: +94 776523867 -- Dimuthu Leelarathne Architect Product Lead of App Factory WSO2, Inc. (http://wso2.com) email: dimut...@wso2.com Mobile : 0773661935 Lean . Enterprise . Middleware -- ~Regards *Manisha Eleperuma* Software Engineer WSO2, Inc.: http://wso2.com lean.enterprise.middleware *blog: http://manisha-eleperuma.blogspot.com/ http://manisha-eleperuma.blogspot.com/* *mobile: +94 71 8279777 %2B94%2071%208279777* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Fwd: [AppFactory] We should have a XSD to the appfactory configuration file.
Hi Harsha, I would not like to take this as an immediate action item that we must do. IMO, we should give more priority to cleaning up the Appfactory.xml and making it simpler and cleaner as possible. That is when we can focus on the XSD to validate the config. WDYT? Thanks, Janaka On Mon, Dec 16, 2013 at 7:30 AM, Harsha Thirimanna hars...@wso2.com wrote: Hi Eranda, We were rapidly changing with this config things. So we didn't care about that previously. But now it is stable and should be validate using XSD with jaxb. Thanks On Dec 16, 2013 12:24 AM, Eranda Sooriyabandara era...@wso2.com wrote: How are you validating the app-factory configuration now? What is the end result of misconfiguration? thanks Eranda On Fri, Dec 6, 2013 at 8:40 AM, Harsha Thirimanna hars...@wso2.comwrote: -- Forwarded message -- From: Harsha Thirimanna hars...@wso2.com Date: Dec 6, 2013 12:23 AM Subject: Fwd: [AppFactory] We should have a XSD to the appfactory configuration file. To: Dimuthu Leelarathne dimut...@wso2.com, Janaka Ranabahu jan...@wso2.com, Ramith Jayasinghe ram...@wso2.com, Ajanthan Balachandran ajant...@wso2.com, Ranga Siriwardena ra...@wso2.com, Danushka Fernando danush...@wso2.com Cc: *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/* * email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770* * twitter: **http://twitter.com/ http://twitter.com/afkham_azeez* *harshathirimann linked-in: **http: http://lk.linkedin.com/in/afkhamazeez**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122 http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* *Lean . Enterprise . Middleware* -- Forwarded message -- From: Harsha Thirimanna hars...@wso2.com Date: Thu, Dec 5, 2013 at 11:12 PM Subject: [AppFactory] We should have a XSD to the appfactory configuration file. To: strategy-group strategy-gr...@wso2.com $subject. WDYT ? *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/* * email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770* * twitter: **http://twitter.com/ http://twitter.com/afkham_azeez* *harshathirimann linked-in: **http: http://lk.linkedin.com/in/afkhamazeez**//www.linkedin.com/pub/harsha-thirimanna/10/ab8/122 http://www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* *Lean . Enterprise . Middleware* ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Eranda Sooriyabandara*Senior Software Engineer; Integration Technologies Team; WSO2 Inc.; http://wso2.com Lean . Enterprise . Middleware E-mail: eranda AT wso2.com Mobile: +94 716 472 816 Linked-In: http://www.linkedin.com/in/erandasooriyabandara Blog: http://emsooriyabandara.blogspot.com/ ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com * E-mail: jan...@wso2.com http://wso2.com**M: **+94 718370861* Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation in Jenkins
Hi Shamika, On Wed, Jul 24, 2013 at 5:20 PM, Ajanthan Balachandran ajant...@wso2.comwrote: On Wed, Jul 24, 2013 at 4:34 PM, Shamika Ariyawansa sham...@wso2.comwrote: HI, As we discussed so fa,r we tried/trying following approaches for the $subject. 1. Deploying Jenkins web app in AS per tenant. - Solution was not scalable due to the size of the Jenkins Web-app (61MB - without plugins) and its not practicable to deploy this as the tenant count gets increased. If the content of the war duplicated for tenants you can put the common libs into $CARBON_HOME/repository/components/lib(in parent classloader) and make minimal war file that contains tenant specific stuffs. How will this handle the load of a build job? Say that each of these Jenkins server instances can run a number of build jobs(more than 1 build per tenant). How can we handle the concurrent build load? 2. Use one Jenkins server and make it possible to make it multi-tenant by introducing a role-based plugin (an extension to Role-Strategy Plugin). Here all the tenants related jobs are stored in one space (no operation between tenant) and the multi-tenancy is achieved by having a filtering mechanism based on the logged users tenant. Problem here is everything will be done in one workspace so it will be difficult to manage when the the tenant count gets increased with the job count. Jenkins has lots of extension points. Can we have a concept of folders/collections in Jenkins jobs? If so we can group jobs by tenant. This could be done as a part of the Role-Strategy plugin. 3. Patch the Jenkins to set the JENKINS_HOME directory on the fly so that separate HOME directory will be used for the different tenants. By looking at the Jenkins code we found that the Jenkins Home is set to a singleton class (jenkins.model.Jenkins) and the whole system uses that class to obtain JENKINS HOME. As a solution we can update this class to return JENKINS_HOME based on logged users tenant. Main risk for this is that in the in above class has a public variable to store the JENKINS_HOME (variable - root). Also there is also an encapsulated method to get this too.( getRootDir() ). We are not sure the how the other plugins have referred this. I am trying to do an hard-coded test whether this works or not? This will not work unless you reload all the configurations from disk after returning the JENKINS_HOME.In jenkins on start up all the config files are loaded from disk(job configs also).We change JENKINS_HOME at the middle but still in the memory there are configs(job configs) from previous JENKINS_HOME. We should be able to work with an existing Jenkins deployment without doing any modifications to the code. If this can be done through a plugin, then I guess it is fine. Otherwise I'm -1 for this. Thanks, Janaka WDYT? -- Shamika Ariyawansa Senior Software Engineer Mob:+ 94 772929486 -- ajanthan -- Ajanthan Balachandiran Senior Software Engineer; Solutions Technologies Team ;WSO2, Inc.; http://wso2.com/ email: ajanthan http://goog_595075977@wso2.com; cell: +94775581497 blog: http://bkayts.blogspot.com/ Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com* E-mail: jan...@wso2.com **M: **+94 718370861** *Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] [Appfactory] Tenant Isolation in Jenkins
Hi, On Mon, Jul 29, 2013 at 10:53 AM, Harsha Thirimanna hars...@wso2.comwrote: *Harsha Thirimanna* Senior Software Engineer; WSO2, Inc.; http://wso2.com * http://www.apache.org/** email: **hars...@wso2.com* az...@wso2.com* cell: +94 71 5186770** twitter: **http://twitter.com/ http://twitter.com/afkham_azeez** harshathirimann linked-in: **http: http://lk.linkedin.com/in/afkhamazeez**// www.linkedin.com/pub/harsha-thirimanna/10/ab8/122* * * *Lean . Enterprise . Middleware* * * On Mon, Jul 29, 2013 at 1:07 AM, Janaka Ranabahu jan...@wso2.com wrote: Hi Shamika, On Wed, Jul 24, 2013 at 5:20 PM, Ajanthan Balachandran ajant...@wso2.com wrote: On Wed, Jul 24, 2013 at 4:34 PM, Shamika Ariyawansa sham...@wso2.comwrote: HI, As we discussed so fa,r we tried/trying following approaches for the $subject. 1. Deploying Jenkins web app in AS per tenant. - Solution was not scalable due to the size of the Jenkins Web-app (61MB - without plugins) and its not practicable to deploy this as the tenant count gets increased. If the content of the war duplicated for tenants you can put the common libs into $CARBON_HOME/repository/components/lib(in parent classloader) and make minimal war file that contains tenant specific stuffs. How will this handle the load of a build job? Say that each of these Jenkins server instances can run a number of build jobs(more than 1 build per tenant). How can we handle the concurrent build load? As ajanthan said , if we can deploy jenkins instance for every tenant then we can maintain multiple slave node to every master node to handle the load. This point was mentioned that with previous email thread by ajanthan. Does this means that we have multiple slave nodes that is been used by every master(of each tenant)? Or do we spawn a number of slaves for every tenant? Ideally what we need to do is to spawn a Jenkins slave on demand. Thanks, Janaka Subject : [Architecture] [Appfactory] Tenant Isolation for Jenkins. 2. Use one Jenkins server and make it possible to make it multi-tenant by introducing a role-based plugin (an extension to Role-Strategy Plugin). Here all the tenants related jobs are stored in one space (no operation between tenant) and the multi-tenancy is achieved by having a filtering mechanism based on the logged users tenant. Problem here is everything will be done in one workspace so it will be difficult to manage when the the tenant count gets increased with the job count. Jenkins has lots of extension points. Can we have a concept of folders/collections in Jenkins jobs? If so we can group jobs by tenant. This could be done as a part of the Role-Strategy plugin. 3. Patch the Jenkins to set the JENKINS_HOME directory on the fly so that separate HOME directory will be used for the different tenants. By looking at the Jenkins code we found that the Jenkins Home is set to a singleton class (jenkins.model.Jenkins) and the whole system uses that class to obtain JENKINS HOME. As a solution we can update this class to return JENKINS_HOME based on logged users tenant. Main risk for this is that in the in above class has a public variable to store the JENKINS_HOME (variable - root). Also there is also an encapsulated method to get this too.( getRootDir() ). We are not sure the how the other plugins have referred this. I am trying to do an hard-coded test whether this works or not? This will not work unless you reload all the configurations from disk after returning the JENKINS_HOME.In jenkins on start up all the config files are loaded from disk(job configs also).We change JENKINS_HOME at the middle but still in the memory there are configs(job configs) from previous JENKINS_HOME. We should be able to work with an existing Jenkins deployment without doing any modifications to the code. If this can be done through a plugin, then I guess it is fine. Otherwise I'm -1 for this. Thanks, Janaka WDYT? -- Shamika Ariyawansa Senior Software Engineer Mob:+ 94 772929486 -- ajanthan -- Ajanthan Balachandiran Senior Software Engineer; Solutions Technologies Team ;WSO2, Inc.; http://wso2.com/ email: ajanthan http://goog_595075977@wso2.com; cell: +94775581497 blog: http://bkayts.blogspot.com/ Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka Ranabahu* Senior Software Engineer; WSO2 Inc.; http://wso2.com* E-mail: jan...@wso2.com **M: **+94 718370861* * *Lean . Enterprise . Middleware ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- *Janaka
Re: [Architecture] G-Reg Lifecycle, BPEL Integration
Hi Pulasthi, Senaka,
Why are we saying BPEL Integration? BPEL process is also exposed as a
service. So shouldn't this be generic to any service call? From what I see,
what you should have is a executor that can do a web service call. The
parameters of the service call can be taken from either a G-Reg resource or
from the LC configuration or can be passed from the UI it self. I don't see
a point in limiting that to only BPEL.
Please correct me if I'm wrong.
Thanks,
Janaka
On Fri, May 31, 2013 at 4:16 PM, Eranda Sooriyabandara era...@wso2.comwrote:
Hi Pulasthi,
Please find the comments inline.
On Fri, May 31, 2013 at 10:00 AM, Pulasthi Supun pulas...@wso2.comwrote:
Hi All
I am working on the $subject. The aim of this feature is make it possible
to invoke BPEL processes during Lifecycle state transition in G-Reg for
example with this feature it will be possible to call an BPEL process when
a Promote is done.
This is achieved via a transitions executor and the BPEL process will be
invoked through an web web-service call. following is an sample Lifecycle
configuration
data name=transitionExecution
execution forEvent=Promote
class=org.wso2.carbon.governance.registry.extensions.executors.BpelExecutor
parameter name=bpel.epr
value=
http://10.200.3.107:9763/services/AdderProcess.AdderProcesshttpAdderProcessBindingEndpoint/;
/
parameter name=bpel.payload
p:AdderProcessRequest xmlns:p=http://wso2.org/bps/sample;
x xmlns=http://wso2.org/bps/sample
/_system/governance/{@resourcePath}/{@xvalue}/x
y xmlns=http://wso2.org/bps/sample
/_system/governance/{@resourcePath}/{@yvalue}/y
/p:AdderProcessRequest
/parameter
/execution
/data
+1 for this approach.
The transition execution needs to be provided with the following details
- End Point Reference of the BPEL
The End point can either be given directly or trough an
property or artifact attribute
Ex. parameter name=bpel.epr
value=/_system/governance/{@resourcePath}/{@epr}/
- The Payload that needed to invoke the BPEL process.
The values of the payload can be--
Nuwan Dias
Member, Management Committee - Solutions Technology Group
Senior Software Engineer - WSO2, Inc. http://wso2.com
email : nuw...@wso2.com
Phone : +94 777 775 729 defined either directly or trough an property or
artifact attribute
Ex.
data name=transitionExecution
execution forEvent=Promote
class=org.wso2.carbon.governance.registry.extensions.executors.BpelExecutor
parameter name=bpel.epr
value=/_system/governance/{@resourcePath}/{@epr}/
parameter name=bpel.payload
p:AdderProcessRequest xmlns:p=http://wso2.org/bps/sample;
x xmlns=http://wso2.org/bps/sample
/_system/governance/{@resourcePath}/{@xvalue}/x
y xmlns=http://wso2.org/bps/sample;6/y
/p:AdderProcessRequest
/parameter
/execution
/data
- Optional - Define whether the BPEL process to be called in
an synchronous or asynchronous manner by default synchronous approach is
used
In the asynchronous mode no response will be saved
- Optional - Define the location to store the response
- Optional - Define whether the response should be saved as an
property or artifact attribute
- Optional - Define the name of the property or attribute to store
the response
What you mean by a response of a business process? Can you please give an
example use case for a process sending a response.
Also if we can go further, we may able to show the current Activity name
executing in the process. When we fully support business process
visualization in G-Reg we may able to use that information for visualizing
the current state of the process instance in G-Reg.
Any feedback and comments are welcome
thanks
Eranda
--
*Eranda Sooriyabandara
*Software Engineer;
Integration Technologies Team;
WSO2 Inc.; http://wso2.com
Lean . Enterprise . Middleware
E-mail: eranda AT wso2.com
Mobile: +94 716 472 816
Linked-In: http://www.linkedin.com/in/erandasooriyabandara
Blog: http://emsooriyabandara.blogspot.com/
*
*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
*Janaka Ranabahu*
Member - Solutions Management Committee;
Senior Software Engineer; WSO2 Inc.; http://wso2.com*
E-mail: jan...@wso2.com
**M: **+94 718370861**
*Lean . Enterprise . Middleware
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
