[
https://issues.apache.org/jira/browse/MADLIB-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Frank McQuillan updated MADLIB-909:
-----------------------------------
Description:
Story
As a data scientist, I want to perform time oriented session reconstruction on
my data set, so that I can prepare for input to predictive analytics algorithms.
Details
1) How to sessionize. Common to use a time-based session reconstruction that
defines a session as a sequence of events by a particular user where no more
than n seconds has elapsed between successive events. That is, if we don’t
see an event from a user for n seconds, we start a new session. The
requirement for MADlib is to do the same thing, but also to add a minimum
number of seconds that must elapse between events for the session to be valid.
As stated in [1], black bots may send requests at excessive rates so we want a
way to be be able to filter them out.
2) Interface could be:
session_split (
source_table,
output_table,
id,
time_stamp,
time_out,
min_time,
OUTPUTNULL)
where the last 2 params are:
* min_time = minimum number of seconds that must elapse between clicks in order
for this session to be considered a real (human) session (default=0)
* OUTPUTNULL = If TRUE, write the session to the output table with NULL values
for appropriate columns. If FALSE, do not write out the NULL session to output
table. Default is FALSE.
References
[1] Blog on bot signatures from Akamai
https://blogs.akamai.com/2013/06/identifying-and-mitigating-unwanted-bot-traffic.html
[2] General information on sessionization
https://en.wikipedia.org/wiki/Session_(web_analytics)
[3] PDL tools implementation
http://pivotalsoftware.github.io/PDLTools/group__grp__sessionization.html
was:
Story
As a data scientist, I want to perform time oriented session reconstruction on
my data set, so that I can prepare for input to predictive analytics algorithms.
Details
1) How to sessionize. Common to use a time-based session reconstruction that
defines a session as a sequence of events by a particular user where no more
than n seconds has elapsed between successive events. That is, if we don’t
see an event from a user for n seconds, we start a new session. The
requirement for MADlib is to do the same thing, but also to add a minimum
number of seconds that must elapse between events for the session to be valid.
As stated in [1], black bots may send requests at excessive rates so we want a
way to be be able to filter them out.
2) Interface could be:
session_split (
source_table,
output_table,
id,
time_stamp,
time_out,
min_time,
OUTPUTNULL)
where the last 2 params are:
* min_time = minimum number of seconds that must elapse between clicks in order
for this session to be considered a real (human) session (default=0)
* OUTPUTNULL = If TRUE, write the session to the output table with NULL values
for appropriate columns. If FALSE, do not write out the NULL session to output
table. Default is FALSE.
References
[1] Blog on bot signatures from Akamai
https://blogs.akamai.com/2013/06/identifying-and-mitigating-unwanted-bot-traffic.html
[2] General information on sessionization
https://en.wikipedia.org/wiki/Session_(web_analytics)
> Sessionization
> --------------
>
> Key: MADLIB-909
> URL: https://issues.apache.org/jira/browse/MADLIB-909
> Project: Apache MADlib
> Issue Type: New Feature
> Components: Module: Utilities
> Reporter: Frank McQuillan
> Labels: gsoc, starter
> Fix For: v1.9.1
>
>
> Story
> As a data scientist, I want to perform time oriented session reconstruction
> on my data set, so that I can prepare for input to predictive analytics
> algorithms.
> Details
> 1) How to sessionize. Common to use a time-based session reconstruction that
> defines a session as a sequence of events by a particular user where no more
> than n seconds has elapsed between successive events. That is, if we don’t
> see an event from a user for n seconds, we start a new session. The
> requirement for MADlib is to do the same thing, but also to add a minimum
> number of seconds that must elapse between events for the session to be valid.
> As stated in [1], black bots may send requests at excessive rates so we want
> a way to be be able to filter them out.
> 2) Interface could be:
> session_split (
> source_table,
> output_table,
> id,
> time_stamp,
> time_out,
> min_time,
> OUTPUTNULL)
> where the last 2 params are:
> * min_time = minimum number of seconds that must elapse between clicks in
> order for this session to be considered a real (human) session (default=0)
> * OUTPUTNULL = If TRUE, write the session to the output table with NULL
> values for appropriate columns. If FALSE, do not write out the NULL session
> to output table. Default is FALSE.
> References
> [1] Blog on bot signatures from Akamai
> https://blogs.akamai.com/2013/06/identifying-and-mitigating-unwanted-bot-traffic.html
> [2] General information on sessionization
> https://en.wikipedia.org/wiki/Session_(web_analytics)
> [3] PDL tools implementation
> http://pivotalsoftware.github.io/PDLTools/group__grp__sessionization.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
