Cool. @Upul can you work with Mohan on this? On Fri, Feb 19, 2016 at 5:38 PM, Srinath Perera <[email protected]> wrote:
> We should be easily extend CEP's Execution Manager ( IMO better called > template manager) to cover Spark queries as well. I have talked to both > Anjana and Suho, and they both think it can be done. > > If we are doing this, Upul work with Mohan. > > --Srinath > > On Thu, Feb 18, 2016 at 2:48 PM, Thilini Cooray <[email protected]> wrote: > >> Hi, >> >> On Thu, Feb 18, 2016 at 2:32 PM, Nirmal Fernando <[email protected]> wrote: >> >>> Hi Thilini, >>> >>> On Thursday, February 18, 2016, Thilini Cooray <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> I would like to have a little clarification regarding the name of the >>>> feature. >>>> >>>> As I can understand, our intention is to filter out the APIs whose >>>> usage is displaying an abnormal behaviour. (As you have mentioned, the >>>> benchmark for the abnormal detection will be emphasized by a policy). >>>> What is the reason for selecting feature name as abnormal *tier usage >>>> *instead >>>> of *API usage*? >>>> >>>> Tier is a concept we use in API Manager for throttling. We use tiers in >>>> API level, Application level and resource level. >>>> Since the end result of the feature is a set of APIs not tiers that we >>>> need alert its creator, It is a bit unclear to me the reason for using >>>> name *tier >>>> *for this. >>>> >>> >>> Isn't tier is also the unit which we subscribed for? This alert is >>> basically help you remind that you are not utilizing the tier properly. >>> >> >> Yes, we are subscribing to a particular tier in order to access an API. >> Thank you for the explanation. >> >> >>> >>>> Thanks. >>>> >>>> >>>> On Thu, Feb 18, 2016 at 7:33 AM, Upul Bandara <[email protected]> wrote: >>>> >>>>> >>>>> Hi All, >>>>> >>>>> We are in the process of implementing following APIM analytics feature: >>>>> >>>>> *Abnormal tier usage *- if we haven't got any API hits or less than *X >>>>> *number of API hits continuously for* Y* days, we should send an >>>>> alert. This could be an indication of a weird state (eg: the application >>>>> is >>>>> not functioning properly). >>>>> >>>>> This feature will enable API creators to identify faulty APIs and >>>>> hence to take necessary actions. >>>>> >>>>> We have identified this as a batch analytics task and hence, planning >>>>> to use Spark SQL queries. All API requests are recorded in >>>>> “ORG_WSO2_APIMGT_STATISTICS_REQUEST” table. Therefore, by grouping above >>>>> table using “api, api_version and applicationId” we can get request counts >>>>> per “api, api_version and applicationId” basis. Next, by filtering out >>>>> using the policy enforced by api developer/admin ( i.e. if we haven't got >>>>> any API hits or less than X number of API hits continuously for Y days, we >>>>> should send an alert and we will be keeping those policies in a >>>>> configuration file.) we can identify what are the APIs for which we need >>>>> to >>>>> send alters. >>>>> >>>>> I have managed to write a Spark SQL query to extract APIs which show >>>>> abnormal tier usage. Now, I’m working on following tasks. >>>>> >>>>> >>>>> >>>>> 1. >>>>> >>>>> We need to think how we could configure this alert as it's not >>>>> using a Siddhi execution plan. >>>>> 2. >>>>> >>>>> Since we are using Spark SQL for identifying abnormal tire usage, >>>>> we have to find a proper way of sending alerts from DAS. >>>>> >>>>> >>>>> Your suggestions regarding our approach and tasks currently I'm >>>>> working on are welcome. >>>>> >>>>> Thanks, >>>>> Upul >>>>> -- >>>>> Upul Bandara, >>>>> Mob: +94 715 468 345. >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> Best Regards, >>>> >>>> *Thilini Cooray* >>>> Software Engineer >>>> Mobile : +94 (0) 774 570 112 <%2B94%20%280%29%20773%20451194> >>>> E-mail : [email protected] >>>> >>>> WSO2 Inc. www.wso2.com >>>> lean.enterprise.middleware >>>> >>> >>> >>> -- >>> >>> Thanks & regards, >>> Nirmal >>> >>> Team Lead - WSO2 Machine Learner >>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>> Mobile: +94715779733 >>> Blog: http://nirmalfdo.blogspot.com/ >>> >>> >>> >>> >> >> >> -- >> Best Regards, >> >> *Thilini Cooray* >> Software Engineer >> Mobile : +94 (0) 774 570 112 <%2B94%20%280%29%20773%20451194> >> E-mail : [email protected] >> >> WSO2 Inc. www.wso2.com >> lean.enterprise.middleware >> > > > > -- > ============================ > Blog: http://srinathsview.blogspot.com twitter:@srinath_perera > Site: http://people.apache.org/~hemapani/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > -- Thanks & regards, Nirmal Team Lead - WSO2 Machine Learner Associate Technical Lead - Data Technologies Team, WSO2 Inc. Mobile: +94715779733 Blog: http://nirmalfdo.blogspot.com/
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
