On Thu, Jul 21, 2016 at 11:46 AM, Ashen Weerathunga <[email protected]> wrote:
> Hi all, > > I'm writing a siddhi extension for Markov models. It can be used to detect > abnormal user behaviors of many real world applications such as detecting > abnormal API request patterns, detecting fraudulent bank transactions etc. > There are different variations in Markov models. Therefore this > implementation will be done using Markov chain[1] which is a basic Markov > model. > > Markov chain consists of following key features [2]. > > - Set of states > - Transition between states > - Future depends on the present > - Future does not depend on the past > > Transition probabilities between states will be updated in real time with > new input events and abnormal state transition notifications will be sent > as for the user defined probability threshold. > Could you explain a bit more about how you are detecting an abnormal state transition? For example, it is done based on the transition matrix of the markov chain? If so how/where do we define this matrix? > This will be implemented as a stream processor and it will have following > input and output parameters. > > *Input parameters* > > Parameter Type Required/Optional Description > id String required id of the user > state String required current state of the user > duration int | long | time required max duration that will be considered > as a continuation of the previous state of the particular user > trainingBatchSize int | long required no of events required to train the > model initially. Notifications will not be given until the no of input > events reach this limit > abnormalTransitionProbability double required transisiiton probability > threshold that should be used to identify abnormal state transitions > > *Output Parameters* > > Parameter Type Name Description > id String user id id of the user > startState String start state start satate of the user > endState String end state end state of the user > transitionProbability double transition probability transition > probability from start state to end state > notify boolean notify notification whether it is a abnormal transition or > not > > As an example following will return notification as true if a user has > done a state transition which has a probability less than or equal to 0.01, > > > from inputStream#markovModels:markovChain(id, state, 60 min, 500, 0.01) > select * > insert into outputStream; > > > Please let me know if you have any suggestions on this. > > [1]https://en.wikipedia.org/wiki/Markov_chain > [2]http://bit-player.org/wp-content/extras/markov/#/ > > Thanks and Regards, > Ashen > -- > *Ashen Weerathunga* > Software Engineer > WSO2 Inc.: http://wso2.com > lean.enterprise.middleware > > Email: [email protected] > Mobile: +94 716042995 <94716042995> > LinkedIn: *http://lk.linkedin.com/in/ashenweerathunga > <http://lk.linkedin.com/in/ashenweerathunga>* > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Malith Jayasinghe WSO2, Inc. (http://wso2.com) Email : [email protected] Mobile : 0770704040 Lean . Enterprise . Middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
