We should be easily extend CEP's Execution Manager ( IMO better called template manager) to cover Spark queries as well. I have talked to both Anjana and Suho, and they both think it can be done.
If we are doing this, Upul work with Mohan. --Srinath On Thu, Feb 18, 2016 at 2:48 PM, Thilini Cooray <[email protected]> wrote: > Hi, > > On Thu, Feb 18, 2016 at 2:32 PM, Nirmal Fernando <[email protected]> wrote: > >> Hi Thilini, >> >> On Thursday, February 18, 2016, Thilini Cooray <[email protected]> wrote: >> >>> Hi, >>> >>> I would like to have a little clarification regarding the name of the >>> feature. >>> >>> As I can understand, our intention is to filter out the APIs whose usage >>> is displaying an abnormal behaviour. (As you have mentioned, the benchmark >>> for the abnormal detection will be emphasized by a policy). >>> What is the reason for selecting feature name as abnormal *tier usage >>> *instead >>> of *API usage*? >>> >>> Tier is a concept we use in API Manager for throttling. We use tiers in >>> API level, Application level and resource level. >>> Since the end result of the feature is a set of APIs not tiers that we >>> need alert its creator, It is a bit unclear to me the reason for using name >>> *tier >>> *for this. >>> >> >> Isn't tier is also the unit which we subscribed for? This alert is >> basically help you remind that you are not utilizing the tier properly. >> > > Yes, we are subscribing to a particular tier in order to access an API. > Thank you for the explanation. > > >> >>> Thanks. >>> >>> >>> On Thu, Feb 18, 2016 at 7:33 AM, Upul Bandara <[email protected]> wrote: >>> >>>> >>>> Hi All, >>>> >>>> We are in the process of implementing following APIM analytics feature: >>>> >>>> *Abnormal tier usage *- if we haven't got any API hits or less than *X >>>> *number of API hits continuously for* Y* days, we should send an >>>> alert. This could be an indication of a weird state (eg: the application is >>>> not functioning properly). >>>> >>>> This feature will enable API creators to identify faulty APIs and hence >>>> to take necessary actions. >>>> >>>> We have identified this as a batch analytics task and hence, planning >>>> to use Spark SQL queries. All API requests are recorded in >>>> “ORG_WSO2_APIMGT_STATISTICS_REQUEST” table. Therefore, by grouping above >>>> table using “api, api_version and applicationId” we can get request counts >>>> per “api, api_version and applicationId” basis. Next, by filtering out >>>> using the policy enforced by api developer/admin ( i.e. if we haven't got >>>> any API hits or less than X number of API hits continuously for Y days, we >>>> should send an alert and we will be keeping those policies in a >>>> configuration file.) we can identify what are the APIs for which we need to >>>> send alters. >>>> >>>> I have managed to write a Spark SQL query to extract APIs which show >>>> abnormal tier usage. Now, I’m working on following tasks. >>>> >>>> >>>> >>>> 1. >>>> >>>> We need to think how we could configure this alert as it's not >>>> using a Siddhi execution plan. >>>> 2. >>>> >>>> Since we are using Spark SQL for identifying abnormal tire usage, >>>> we have to find a proper way of sending alerts from DAS. >>>> >>>> >>>> Your suggestions regarding our approach and tasks currently I'm working >>>> on are welcome. >>>> >>>> Thanks, >>>> Upul >>>> -- >>>> Upul Bandara, >>>> Mob: +94 715 468 345. >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Best Regards, >>> >>> *Thilini Cooray* >>> Software Engineer >>> Mobile : +94 (0) 774 570 112 <%2B94%20%280%29%20773%20451194> >>> E-mail : [email protected] >>> >>> WSO2 Inc. www.wso2.com >>> lean.enterprise.middleware >>> >> >> >> -- >> >> Thanks & regards, >> Nirmal >> >> Team Lead - WSO2 Machine Learner >> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >> Mobile: +94715779733 >> Blog: http://nirmalfdo.blogspot.com/ >> >> >> >> > > > -- > Best Regards, > > *Thilini Cooray* > Software Engineer > Mobile : +94 (0) 774 570 112 <%2B94%20%280%29%20773%20451194> > E-mail : [email protected] > > WSO2 Inc. www.wso2.com > lean.enterprise.middleware > -- ============================ Blog: http://srinathsview.blogspot.com twitter:@srinath_perera Site: http://people.apache.org/~hemapani/ Photos: http://www.flickr.com/photos/hemapani/ Phone: 0772360902
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
