Re: [Architecture] Nominal attributes in Siddhi
On Fri, Feb 27, 2015 at 7:26 AM, Srinath Perera srin...@wso2.com wrote:
Mohan, could you check when you can?
Sure, will do..
Thanks,
Mohan
On Fri, Feb 27, 2015 at 6:46 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Srinath,
I have wrapped up the implementation and sent a pull request. Please
let me know if there are any issues with merging the pull request.
I have attached the draft version of the report I wrote based on the
current implementation, this contains more improvements we should do to
support fully for the Hoeffding algorithm.
As per our discussion happened in Bloomington, I had a look in to
regression analysis for data streams and there are lots of interesting
things we can do with Siddhi to do multi-dimensional regressions.
Mostly people tried implementing one parse algorithms to find patterns
with sensor data and dynamically modify the model. I hope to send a detail
email on this.
Thanks
Lahiru
On Sep 23, 2014, at 3:39 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Suho,
I have modified Attribute class which will keep all the possible values
in attribute and can create the numeric value 2 for “rainy”. In the
incoming event we can have the possible values for each attribute. I have
the code committed to my local github[1] and once I have a working copy
with some good tests I can send a pull request.
Once we have the tree build for these events, Siddhi should be able to
take a given event and do a prediction and determine the value of the last
attribute and based on the last attribute values we
can move event in to different event streams.
Ex: We can send 10 events with proper value for play (yes,no) after you
have the tree built you can just send the rest of the attributes in the
event and Decision tree will predict the value for play (yes or no), so the
query can be written to the play event. In the Window implementation we can
specify minimum events required to build the tree and always keep the last
attribute as the class attribute (During the initial implementation).
So query will be, from
weather#siddhiht(minimum_events_for_tree_build,give_desired_class_values)
select * insert into play for all-events;
For a simple scenario we can assume there is only a single class we
need to check in this case we can say play=“yes”, so query can look like
below for 10 minimum events.
weather#siddhiht(10,play=yes)
select * insert into play for all-events;
If events come empty play value before the tree is built we can throw
an exception or just emit the decision but it will be with low accuracy.
Regards
Lahiru
[1]https://github.com/glahiru/siddhi
On Sep 23, 2014, at 1:56 AM, Sriskandarajah Suhothayan s...@wso2.com
wrote:
Looks OK.
Can you please clarify how would you write queries in this case? (e.g
If its rainy how would you check?)
and internally how all of these will be processed?
Regards
Suho
On Tue, Sep 23, 2014 at 10:30 AM, Srinath Perera srin...@wso2.com
wrote:
I think this is OK. Suho, WDYT?
On Mon, Sep 22, 2014 at 12:15 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Devs,
I am developing a classification algorithm for siddhi and I have a
requirement of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the
definition we have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a
particular event. I can see in the code Siddhi doesn’t support this (Please
correct me if I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast,
rainy}, temperature nominal{hot, mild, cool}, humidity nominal{high,
normal},windy nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which
maps to these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the
calculation using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy to
implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Srinath Perera, Ph.D.
http://people.apache.org/~hemapani/
http://srinathsview.blogspot.com/
--
*S. Suhothayan *
Technical Lead Team Lead of WSO2 Complex Event Processor
*WSO2 Inc. *http://wso2.com
* http://wso2.com/*
lean . enterprise .
Re: [Architecture] Nominal attributes in Siddhi
Mohan, could you check when you can?
On Fri, Feb 27, 2015 at 6:46 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Srinath,
I have wrapped up the implementation and sent a pull request. Please let
me know if there are any issues with merging the pull request.
I have attached the draft version of the report I wrote based on the
current implementation, this contains more improvements we should do to
support fully for the Hoeffding algorithm.
As per our discussion happened in Bloomington, I had a look in to
regression analysis for data streams and there are lots of interesting
things we can do with Siddhi to do multi-dimensional regressions.
Mostly people tried implementing one parse algorithms to find patterns
with sensor data and dynamically modify the model. I hope to send a detail
email on this.
Thanks
Lahiru
On Sep 23, 2014, at 3:39 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Suho,
I have modified Attribute class which will keep all the possible values
in attribute and can create the numeric value 2 for “rainy”. In the
incoming event we can have the possible values for each attribute. I have
the code committed to my local github[1] and once I have a working copy
with some good tests I can send a pull request.
Once we have the tree build for these events, Siddhi should be able to
take a given event and do a prediction and determine the value of the last
attribute and based on the last attribute values we
can move event in to different event streams.
Ex: We can send 10 events with proper value for play (yes,no) after you
have the tree built you can just send the rest of the attributes in the
event and Decision tree will predict the value for play (yes or no), so the
query can be written to the play event. In the Window implementation we can
specify minimum events required to build the tree and always keep the last
attribute as the class attribute (During the initial implementation).
So query will be, from
weather#siddhiht(minimum_events_for_tree_build,give_desired_class_values)
select * insert into play for all-events;
For a simple scenario we can assume there is only a single class we need
to check in this case we can say play=“yes”, so query can look like below
for 10 minimum events.
weather#siddhiht(10,play=yes)
select * insert into play for all-events;
If events come empty play value before the tree is built we can throw an
exception or just emit the decision but it will be with low accuracy.
Regards
Lahiru
[1]https://github.com/glahiru/siddhi
On Sep 23, 2014, at 1:56 AM, Sriskandarajah Suhothayan s...@wso2.com
wrote:
Looks OK.
Can you please clarify how would you write queries in this case? (e.g If
its rainy how would you check?)
and internally how all of these will be processed?
Regards
Suho
On Tue, Sep 23, 2014 at 10:30 AM, Srinath Perera srin...@wso2.com wrote:
I think this is OK. Suho, WDYT?
On Mon, Sep 22, 2014 at 12:15 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Devs,
I am developing a classification algorithm for siddhi and I have a
requirement of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the
definition we have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a
particular event. I can see in the code Siddhi doesn’t support this (Please
correct me if I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast,
rainy}, temperature nominal{hot, mild, cool}, humidity nominal{high,
normal},windy nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which
maps to these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the
calculation using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy to
implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Srinath Perera, Ph.D.
http://people.apache.org/~hemapani/
http://srinathsview.blogspot.com/
--
*S. Suhothayan *
Technical Lead Team Lead of WSO2 Complex Event Processor
*WSO2 Inc. *http://wso2.com
* http://wso2.com/*
lean . enterprise . middleware
*cell: (+94) 779 756 757 %28%2B94%29%20779%20756%20757 | blog:
http://suhothayan.blogspot.com/
Re: [Architecture] Nominal attributes in Siddhi
Hi All,
I have finished implementing a Window (ClassifyHtWindowProcessor) to do the
classification based on Hoeffding Tree. So if you define the event stream as
below
you can send initial set of events to build the classification model then after
the building the tree with given number of events WindowProcess will start
doing the predictions for the defined class (With current implementation only
one class is supported and it has to be the last attribute in the stream
definition). Window processor arguments are number of training events required
to build the tree and the desired class value comes true with the prediction
(In this case event will emit when the decision comes as not to play ).
siddhiManager.defineStream(define stream weather (outlook optional(sunny,
overcast, rainy), temperature optional(hot, mild, cool),humidity optional(high,
normal),windy optional(TRUE, FALSE),play optional(yes, no)));
siddhiManager.addQuery(from weather#window.classifyHt(14,no) +
select play +
insert into Results for all-events ;”);
Current implementation has following limitations.
Do not support numeric values to build the distribution.
Only a single class can be predicted.
To improve the performance we can do Tie Breaking Hoeffding tree implementation
and do more refining for the algorithm.
If you have more feedback or real dataset I can test that would be great.
Regards
Lahiru
On Sep 23, 2014, at 4:52 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edumailto:lginn...@indiana.edu wrote:
Hi Paul,
On Sep 23, 2014, at 4:35 AM, Paul Fremantle
p...@wso2.commailto:p...@wso2.com wrote:
I think this is really useful. I'm not convinced the word nominal is well-known
enough to be completely clear in this context.
what about
define stream weather( outlook options={sunny, overcast, rainy} )
+1
Lahiru
Paul
On 21 September 2014 19:45, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edumailto:lginn...@indiana.edu wrote:
Hi Devs,
I am developing a classification algorithm for siddhi and I have a requirement
of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the definition we
have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a particular
event. I can see in the code Siddhi doesn’t support this (Please correct me if
I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast, rainy},
temperature nominal{hot, mild, cool}, humidity nominal{high, normal},windy
nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which maps to
these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the calculation
using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy to
implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.orgmailto:Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Paul Fremantle
CTO and Co-Founder, WSO2
OASIS WS-RX TC Co-chair, Apache Member
UK: +44 207 096 0336tel:%2B44%20207%20096%200336
blog: http://pzf.fremantle.orghttp://pzf.fremantle.org/
twitter.com/pzfreohttp://twitter.com/pzfreo
p...@wso2.commailto:p...@wso2.com
wso2.comhttp://wso2.com/ Lean Enterprise Middleware
Disclaimer: This communication may contain privileged or other confidential
information and is intended exclusively for the addressee/s. If you are not the
intended recipient/s, or believe that you may have received this communication
in error, please reply to the sender indicating that fact and delete the copy
you received and in addition, you should not print, copy, retransmit,
disseminate, or otherwise use the information contained in this communication.
Internet communications cannot be guaranteed to be timely, secure, error or
virus-free. The sender does not accept liability for any errors or omissions.
___
Architecture mailing list
Architecture@wso2.orgmailto:Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
___
Architecture mailing list
Architecture@wso2.orgmailto:Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Nominal attributes in Siddhi
Can I suggest s/optional/options/
Optional means that you can leave this out. Options means you must choose
one of the following.
Paul
On 8 October 2014 08:31, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi All,
I have finished implementing a Window (ClassifyHtWindowProcessor) to do
the classification based on Hoeffding Tree. So if you define the event
stream as below
you can send initial set of events to build the classification model then
after the building the tree with given number of events WindowProcess will
start doing the predictions for the defined class (With current
implementation only one class is supported and it has to be the last
attribute in the stream definition). Window processor arguments are number
of training events required to build the tree and the desired class value
comes true with the prediction (In this case event will emit when the
decision comes as not to play ).
siddhiManager.defineStream(define stream weather (outlook
optional(sunny, overcast, rainy), temperature optional(hot, mild,
cool),humidity optional(high, normal),windy optional(TRUE,
FALSE),play optional(yes, no)));
siddhiManager.addQuery(from weather#window.classifyHt(14,no) +
select play +
insert into Results for all-events ;”);
Current implementation has following limitations.
Do not support numeric values to build the distribution.
Only a single class can be predicted.
To improve the performance we can do Tie Breaking Hoeffding tree
implementation and do more refining for the algorithm.
If you have more feedback or real dataset I can test that would be great.
Regards
Lahiru
On Sep 23, 2014, at 4:52 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Paul,
On Sep 23, 2014, at 4:35 AM, Paul Fremantle p...@wso2.com wrote:
I think this is really useful. I'm not convinced the word nominal is
well-known enough to be completely clear in this context.
what about
define stream weather( outlook options={sunny, overcast, rainy} )
+1
Lahiru
Paul
On 21 September 2014 19:45, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Devs,
I am developing a classification algorithm for siddhi and I have a
requirement of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the
definition we have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a
particular event. I can see in the code Siddhi doesn’t support this (Please
correct me if I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast,
rainy}, temperature nominal{hot, mild, cool}, humidity nominal{high,
normal},windy nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which
maps to these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the
calculation using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy to
implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Paul Fremantle
CTO and Co-Founder, WSO2
OASIS WS-RX TC Co-chair, Apache Member
UK: +44 207 096 0336
blog: http://pzf.fremantle.org
twitter.com/pzfreo
p...@wso2.com
wso2.com Lean Enterprise Middleware
Disclaimer: This communication may contain privileged or other
confidential information and is intended exclusively for the addressee/s.
If you are not the intended recipient/s, or believe that you may have
received this communication in error, please reply to the sender indicating
that fact and delete the copy you received and in addition, you should not
print, copy, retransmit, disseminate, or otherwise use the information
contained in this communication. Internet communications cannot be
guaranteed to be timely, secure, error or virus-free. The sender does not
accept liability for any errors or omissions.
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
___
Architecture mailing list
Architecture@wso2.org
Re: [Architecture] Nominal attributes in Siddhi
Sorry to be picky, but if Ht stands for Hoeffding Tree then it should be HT
(all caps).
Paul
On 8 October 2014 09:40, Paul Fremantle p...@wso2.com wrote:
Can I suggest s/optional/options/
Optional means that you can leave this out. Options means you must choose
one of the following.
Paul
On 8 October 2014 08:31, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi All,
I have finished implementing a Window (ClassifyHtWindowProcessor) to do
the classification based on Hoeffding Tree. So if you define the event
stream as below
you can send initial set of events to build the classification model then
after the building the tree with given number of events WindowProcess will
start doing the predictions for the defined class (With current
implementation only one class is supported and it has to be the last
attribute in the stream definition). Window processor arguments are number
of training events required to build the tree and the desired class value
comes true with the prediction (In this case event will emit when the
decision comes as not to play ).
siddhiManager.defineStream(define stream weather (outlook
optional(sunny, overcast, rainy), temperature optional(hot, mild,
cool),humidity optional(high, normal),windy optional(TRUE,
FALSE),play optional(yes, no)));
siddhiManager.addQuery(from weather#window.classifyHt(14,no) +
select play +
insert into Results for all-events ;”);
Current implementation has following limitations.
Do not support numeric values to build the distribution.
Only a single class can be predicted.
To improve the performance we can do Tie Breaking Hoeffding tree
implementation and do more refining for the algorithm.
If you have more feedback or real dataset I can test that would be
great.
Regards
Lahiru
On Sep 23, 2014, at 4:52 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Paul,
On Sep 23, 2014, at 4:35 AM, Paul Fremantle p...@wso2.com wrote:
I think this is really useful. I'm not convinced the word nominal is
well-known enough to be completely clear in this context.
what about
define stream weather( outlook options={sunny, overcast, rainy} )
+1
Lahiru
Paul
On 21 September 2014 19:45, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Devs,
I am developing a classification algorithm for siddhi and I have a
requirement of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the
definition we have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a
particular event. I can see in the code Siddhi doesn’t support this (Please
correct me if I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast,
rainy}, temperature nominal{hot, mild, cool}, humidity nominal{high,
normal},windy nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which
maps to these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the
calculation using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy to
implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Paul Fremantle
CTO and Co-Founder, WSO2
OASIS WS-RX TC Co-chair, Apache Member
UK: +44 207 096 0336
blog: http://pzf.fremantle.org
twitter.com/pzfreo
p...@wso2.com
wso2.com Lean Enterprise Middleware
Disclaimer: This communication may contain privileged or other
confidential information and is intended exclusively for the addressee/s.
If you are not the intended recipient/s, or believe that you may have
received this communication in error, please reply to the sender indicating
that fact and delete the copy you received and in addition, you should not
print, copy, retransmit, disseminate, or otherwise use the information
contained in this communication. Internet communications cannot be
guaranteed to be timely, secure, error or virus-free. The sender does not
accept liability for any errors or omissions.
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
___
Architecture mailing list
Architecture@wso2.org
Re: [Architecture] Nominal attributes in Siddhi
Also it seems like mostly this is known as VFDT in the literature (Very
Fast Decision Tree). So that might be a better name (e.g. ClassifyVFDT).
PS this is really cool.
Paul
On 8 October 2014 09:41, Paul Fremantle p...@wso2.com wrote:
Sorry to be picky, but if Ht stands for Hoeffding Tree then it should be
HT (all caps).
Paul
On 8 October 2014 09:40, Paul Fremantle p...@wso2.com wrote:
Can I suggest s/optional/options/
Optional means that you can leave this out. Options means you must choose
one of the following.
Paul
On 8 October 2014 08:31, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi All,
I have finished implementing a Window (ClassifyHtWindowProcessor) to
do the classification based on Hoeffding Tree. So if you define the event
stream as below
you can send initial set of events to build the classification model
then after the building the tree with given number of events WindowProcess
will start doing the predictions for the defined class (With current
implementation only one class is supported and it has to be the last
attribute in the stream definition). Window processor arguments are number
of training events required to build the tree and the desired class value
comes true with the prediction (In this case event will emit when the
decision comes as not to play ).
siddhiManager.defineStream(define stream weather (outlook
optional(sunny, overcast, rainy), temperature optional(hot, mild,
cool),humidity optional(high, normal),windy optional(TRUE,
FALSE),play optional(yes, no)));
siddhiManager.addQuery(from weather#window.classifyHt(14,no) +
select play +
insert into Results for all-events ;”);
Current implementation has following limitations.
Do not support numeric values to build the distribution.
Only a single class can be predicted.
To improve the performance we can do Tie Breaking Hoeffding tree
implementation and do more refining for the algorithm.
If you have more feedback or real dataset I can test that would be
great.
Regards
Lahiru
On Sep 23, 2014, at 4:52 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Paul,
On Sep 23, 2014, at 4:35 AM, Paul Fremantle p...@wso2.com wrote:
I think this is really useful. I'm not convinced the word nominal is
well-known enough to be completely clear in this context.
what about
define stream weather( outlook options={sunny, overcast, rainy} )
+1
Lahiru
Paul
On 21 September 2014 19:45, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Devs,
I am developing a classification algorithm for siddhi and I have a
requirement of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the
definition we have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a
particular event. I can see in the code Siddhi doesn’t support this (Please
correct me if I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast,
rainy}, temperature nominal{hot, mild, cool}, humidity nominal{high,
normal},windy nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which
maps to these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the
calculation using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy
to implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Paul Fremantle
CTO and Co-Founder, WSO2
OASIS WS-RX TC Co-chair, Apache Member
UK: +44 207 096 0336
blog: http://pzf.fremantle.org
twitter.com/pzfreo
p...@wso2.com
wso2.com Lean Enterprise Middleware
Disclaimer: This communication may contain privileged or other
confidential information and is intended exclusively for the addressee/s.
If you are not the intended recipient/s, or believe that you may have
received this communication in error, please reply to the sender indicating
that fact and delete the copy you received and in addition, you should not
print, copy, retransmit, disseminate, or otherwise use the information
contained in this communication. Internet communications cannot be
guaranteed to be timely, secure, error or virus-free. The sender does not
accept liability for any errors or omissions.
___
Re: [Architecture] Nominal attributes in Siddhi
Hi Paul, Sorry for the typo with option and I will fix it. But for VFDT there are multiple algorithms that you can, do you think we shouldn’t show it in the window name ? Thanks Lahiru On Oct 8, 2014, at 4:45 AM, Paul Fremantle p...@wso2.commailto:p...@wso2.com wrote: ClassifyVFDT ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Nominal attributes in Siddhi
In that case, I'd just call it ClassifyHT Paul On 8 October 2014 16:22, Ginnaliya Gamathige, Lahiru Manananda lginn...@indiana.edu wrote: Hi Paul, Sorry for the typo with option and I will fix it. But for VFDT there are multiple algorithms that you can, do you think we shouldn’t show it in the window name ? Thanks Lahiru On Oct 8, 2014, at 4:45 AM, Paul Fremantle p...@wso2.com wrote: ClassifyVFDT ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture -- Paul Fremantle CTO and Co-Founder, WSO2 OASIS WS-RX TC Co-chair, Apache Member UK: +44 207 096 0336 blog: http://pzf.fremantle.org twitter.com/pzfreo p...@wso2.com wso2.com Lean Enterprise Middleware Disclaimer: This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, retransmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. ___ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Nominal attributes in Siddhi
Hi Suho,
I have modified Attribute class which will keep all the possible values in
attribute and can create the numeric value 2 for “rainy”. In the incoming event
we can have the possible values for each attribute. I have the code committed
to my local github[1] and once I have a working copy with some good tests I can
send a pull request.
Once we have the tree build for these events, Siddhi should be able to take a
given event and do a prediction and determine the value of the last attribute
and based on the last attribute values we
can move event in to different event streams.
Ex: We can send 10 events with proper value for play (yes,no) after you have
the tree built you can just send the rest of the attributes in the event and
Decision tree will predict the value for play (yes or no), so the query can be
written to the play event. In the Window implementation we can specify minimum
events required to build the tree and always keep the last attribute as the
class attribute (During the initial implementation).
So query will be, from
weather#siddhiht(minimum_events_for_tree_build,give_desired_class_values)
select * insert into play for all-events;
For a simple scenario we can assume there is only a single class we need to
check in this case we can say play=“yes”, so query can look like below for 10
minimum events.
weather#siddhiht(10,play=yes)
select * insert into play for all-events;
If events come empty play value before the tree is built we can throw an
exception or just emit the decision but it will be with low accuracy.
Regards
Lahiru
[1]https://github.com/glahiru/siddhi
On Sep 23, 2014, at 1:56 AM, Sriskandarajah Suhothayan
s...@wso2.commailto:s...@wso2.com wrote:
Looks OK.
Can you please clarify how would you write queries in this case? (e.g If its
rainy how would you check?)
and internally how all of these will be processed?
Regards
Suho
On Tue, Sep 23, 2014 at 10:30 AM, Srinath Perera
srin...@wso2.commailto:srin...@wso2.com wrote:
I think this is OK. Suho, WDYT?
On Mon, Sep 22, 2014 at 12:15 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edumailto:lginn...@indiana.edu wrote:
Hi Devs,
I am developing a classification algorithm for siddhi and I have a requirement
of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the definition we
have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a particular
event. I can see in the code Siddhi doesn’t support this (Please correct me if
I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast, rainy},
temperature nominal{hot, mild, cool}, humidity nominal{high, normal},windy
nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which maps to
these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the calculation
using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy to
implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.orgmailto:Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Srinath Perera, Ph.D.
http://people.apache.org/~hemapani/
http://srinathsview.blogspot.com/
--
S. Suhothayan
Technical Lead Team Lead of WSO2 Complex Event Processor
WSO2 Inc. http://wso2.com http://wso2.com/
http://wso2.com/
lean . enterprise . middleware
cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in:
http://lk.linkedin.com/in/suhothayan
___
Architecture mailing list
Architecture@wso2.orgmailto:Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Nominal attributes in Siddhi
I think this is really useful. I'm not convinced the word nominal is
well-known enough to be completely clear in this context.
what about
define stream weather( outlook options={sunny, overcast, rainy} )
Paul
On 21 September 2014 19:45, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Devs,
I am developing a classification algorithm for siddhi and I have a
requirement of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the
definition we have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a
particular event. I can see in the code Siddhi doesn’t support this (Please
correct me if I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast,
rainy}, temperature nominal{hot, mild, cool}, humidity nominal{high,
normal},windy nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which
maps to these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the
calculation using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy to
implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Paul Fremantle
CTO and Co-Founder, WSO2
OASIS WS-RX TC Co-chair, Apache Member
UK: +44 207 096 0336
blog: http://pzf.fremantle.org
twitter.com/pzfreo
p...@wso2.com
wso2.com Lean Enterprise Middleware
Disclaimer: This communication may contain privileged or other confidential
information and is intended exclusively for the addressee/s. If you are not
the intended recipient/s, or believe that you may have received this
communication in error, please reply to the sender indicating that fact and
delete the copy you received and in addition, you should not print, copy,
retransmit, disseminate, or otherwise use the information contained in this
communication. Internet communications cannot be guaranteed to be timely,
secure, error or virus-free. The sender does not accept liability for any
errors or omissions.
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Nominal attributes in Siddhi
Hi Paul,
On Sep 23, 2014, at 4:35 AM, Paul Fremantle
p...@wso2.commailto:p...@wso2.com wrote:
I think this is really useful. I'm not convinced the word nominal is well-known
enough to be completely clear in this context.
what about
define stream weather( outlook options={sunny, overcast, rainy} )
+1
Lahiru
Paul
On 21 September 2014 19:45, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edumailto:lginn...@indiana.edu wrote:
Hi Devs,
I am developing a classification algorithm for siddhi and I have a requirement
of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the definition we
have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a particular
event. I can see in the code Siddhi doesn’t support this (Please correct me if
I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast, rainy},
temperature nominal{hot, mild, cool}, humidity nominal{high, normal},windy
nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which maps to
these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the calculation
using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy to
implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.orgmailto:Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Paul Fremantle
CTO and Co-Founder, WSO2
OASIS WS-RX TC Co-chair, Apache Member
UK: +44 207 096 0336tel:%2B44%20207%20096%200336
blog: http://pzf.fremantle.orghttp://pzf.fremantle.org/
twitter.com/pzfreohttp://twitter.com/pzfreo
p...@wso2.commailto:p...@wso2.com
wso2.comhttp://wso2.com/ Lean Enterprise Middleware
Disclaimer: This communication may contain privileged or other confidential
information and is intended exclusively for the addressee/s. If you are not the
intended recipient/s, or believe that you may have received this communication
in error, please reply to the sender indicating that fact and delete the copy
you received and in addition, you should not print, copy, retransmit,
disseminate, or otherwise use the information contained in this communication.
Internet communications cannot be guaranteed to be timely, secure, error or
virus-free. The sender does not accept liability for any errors or omissions.
___
Architecture mailing list
Architecture@wso2.orgmailto:Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Nominal attributes in Siddhi
I think this is OK. Suho, WDYT?
On Mon, Sep 22, 2014 at 12:15 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Devs,
I am developing a classification algorithm for siddhi and I have a
requirement of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the
definition we have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a
particular event. I can see in the code Siddhi doesn’t support this (Please
correct me if I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast,
rainy}, temperature nominal{hot, mild, cool}, humidity nominal{high,
normal},windy nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which
maps to these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the
calculation using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy to
implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Srinath Perera, Ph.D.
http://people.apache.org/~hemapani/
http://srinathsview.blogspot.com/
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Re: [Architecture] Nominal attributes in Siddhi
Looks OK.
Can you please clarify how would you write queries in this case? (e.g If
its rainy how would you check?)
and internally how all of these will be processed?
Regards
Suho
On Tue, Sep 23, 2014 at 10:30 AM, Srinath Perera srin...@wso2.com wrote:
I think this is OK. Suho, WDYT?
On Mon, Sep 22, 2014 at 12:15 AM, Ginnaliya Gamathige, Lahiru Manananda
lginn...@indiana.edu wrote:
Hi Devs,
I am developing a classification algorithm for siddhi and I have a
requirement of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the
definition we have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a
particular event. I can see in the code Siddhi doesn’t support this (Please
correct me if I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast,
rainy}, temperature nominal{hot, mild, cool}, humidity nominal{high,
normal},windy nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which
maps to these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the
calculation using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy to
implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
--
Srinath Perera, Ph.D.
http://people.apache.org/~hemapani/
http://srinathsview.blogspot.com/
--
*S. Suhothayan*
Technical Lead Team Lead of WSO2 Complex Event Processor
*WSO2 Inc. *http://wso2.com
* http://wso2.com/*
lean . enterprise . middleware
*cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
http://suhothayan.blogspot.com/twitter: http://twitter.com/suhothayan
http://twitter.com/suhothayan | linked-in:
http://lk.linkedin.com/in/suhothayan http://lk.linkedin.com/in/suhothayan*
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
[Architecture] Nominal attributes in Siddhi
Hi Devs,
I am developing a classification algorithm for siddhi and I have a requirement
of defining attributes as a nominal attribute.
Nominal attribute is similar to a string attribute but during the definition we
have to give the possible values for that attribute.
One of the value from that set can come as the actual value in a particular
event. I can see in the code Siddhi doesn’t support this (Please correct me if
I am wrong), so
I am thinking of changing the Siddhi language grammar to parse nominal
attribute as below.
Example query: define stream weather(outlook nominal{sunny, overcast, rainy},
temperature nominal{hot, mild, cool}, humidity nominal{high, normal},windy
nominal{TRUE, FALSE},play nominal{yes, no}).
If there is a better way to do this, please provide me some feedback.
During the classification algorithm it will use numeric values which maps to
these possible values(Ex: sunny=0,overcase=1,rainy=2) and do the calculation
using numeric values. Initial phase of the algorithm
I am implementing[1], I hope to process nominal values (which is easy to
implement), then algorithm should support processing numeric data too.
If you have any feedback please let me know.
[1]http://homes.cs.washington.edu/~pedrod/papers/kdd00.pdf
Regards
Lahiru
___
Architecture mailing list
Architecture@wso2.org
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
