Re: [Dev] Adding drill down capabilities to the Area Chart in IS Analytics Server Dashboard

[Inosh Goonewardena]

Hi Hasintha,

Currently, we are not doing any region based summarizations for sessions.
So at the moment we don't need this field in session stream. However if
there is such requirement we can add it to the stream. Not only the region
if there any other information that is useful related to sessions we can
add them to the stream

On Thu, Jun 30, 2016 at 11:51 AM, Hasintha Indrajee 
wrote:

> Hi Inosh,
>
> Do we need this new field in session stream as well ?
>
> On Wed, Jun 29, 2016 at 3:13 PM, Nuwandi Wickramasinghe  > wrote:
>
>> Please find the public jira in [1]
>>
>> [1] https://wso2.org/jira/browse/IDENTITY-4739
>>
>> On Wed, Jun 29, 2016 at 2:51 PM, Hasintha Indrajee 
>> wrote:
>>
>>> Hi all,
>>>
>>> Had an offline discussion with Nuwandi and following are some points.
>>>
>>> - We can filter region from IP and provide ability to drill down, but if
>>> we follow this approach, region is not shown in the tables. This leads to a
>>> different user experience.
>>>
>>> Since we don't have any other possible solutions as suggested by Mohan,
>>> let's go with the first approach. Please create a jira for this and state
>>> required changes to the stream definitions.
>>>
>>> On Wed, Jun 29, 2016 at 12:38 PM, Mohanadarshan Vivekanandalingam <
>>> mo...@wso2.com> wrote:
>>>
 HI Johann & Hasintha,

 Appreciate any feedback on above, then we can implement accordingly and
 do some perf tests.. If drill capability is required for Map then we need
 to follow above option.. Here, we can

 1) Change the stream in Identity level and publish
 2) Change at DAS level by adding another attributes in Siddhi scope..

 But, I like to avoid option [2] to avoid unnecessary event conversions
 (which impact the TPS).. Then, option [1] is the only solution in this
 case..

 Thanks,
 Mohan


 On Tue, Jun 28, 2016 at 4:16 PM, Nuwandi Wickramasinghe <
 nuwan...@wso2.com> wrote:

> Hi all,
>
> IS Analytics dashboard contains a map in which we have marked the
> regions of users who have authenticated through Identity Server. Our
> concern is whether we should allow to drill down from this map.
>
> Normally when drilling down from a gadget, in the dashboard we only
> show the data related to that selected field. And there's a data table
> below which shows the raw data corresponding to the selected field. This
> data table is populated using the raw stream which is published from
> Identity Server side.
>
> When implementing drill down capabilities from the area chart (map),
> we are facing an issue in populating the data table since the raw data
> stream published from IS contains only the remoteIp (which is used to
> determine the region) but not the region field itself.
>
> Regions and IP addresses have one to many relationship. Which means a
> large range of IPs corresponds to one region. Therefore we can get the
> region from IP but not vise versa. This leaves the issue that we cannot
> search in the raw stream using the region selected from the map.
>
> Suggested solution is to change the raw data stream to contain an
> additional 'region' attribute which could be used to search for the data
> related to a particular region. The value of region for the raw data will
> be sent as 'NOT_AVAILABLE'. We are planning to update this value to the
> correct region according to the remoteIp value using a spark query. We
> considered updating the records from the execution plan but that could
> delay the real time process.
>
> If we are going to implement this, Hasintha will have to introduce the
> new region attribute to the stream with a default value. Also this might
> introduce some performance degrade in the db level.
>
> We are still considering the need of drilling down from the region
> map. Appreciate other opinions regarding the subject.
>
> Thanks
> Nuwandi
>
> --
>
> Best Regards,
>
> Nuwandi Wickramasinghe
>
> Software Engineer
>
> WSO2 Inc.
>
> Web : http://wso2.com
>
> Mobile : 0719214873
>



 --
 *V. Mohanadarshan*
 *Associate Tech Lead,*
 *Data Technologies Team,*
 *WSO2, Inc. http://wso2.com  *
 *lean.enterprise.middleware.*

 email: mo...@wso2.com
 phone:(+94) 771117673

>>>
>>>
>>>
>>> --
>>> Hasintha Indrajee
>>> WSO2, Inc.
>>> Mobile:+94 771892453
>>>
>>>
>>
>>
>> --
>>
>> Best Regards,
>>
>> Nuwandi Wickramasinghe
>>
>> Software Engineer
>>
>> WSO2 Inc.
>>
>> Web : http://wso2.com
>>
>> Mobile : 0719214873
>>
>
>
>
> --
> Hasintha Indrajee
> WSO2, Inc.
> Mobile:+94 771892453
>
>


-- 
Thanks & Regards,

Inosh Goonewardena
Associate Technical Lead- WSO2 Inc.
Mobile: +94779966317
___
Dev mailing list
Dev@wso2.org
htt

Re: [Dev] Adding drill down capabilities to the Area Chart in IS Analytics Server Dashboard

[Hasintha Indrajee]

Hi Inosh,

Do we need this new field in session stream as well ?

On Wed, Jun 29, 2016 at 3:13 PM, Nuwandi Wickramasinghe 
wrote:

> Please find the public jira in [1]
>
> [1] https://wso2.org/jira/browse/IDENTITY-4739
>
> On Wed, Jun 29, 2016 at 2:51 PM, Hasintha Indrajee 
> wrote:
>
>> Hi all,
>>
>> Had an offline discussion with Nuwandi and following are some points.
>>
>> - We can filter region from IP and provide ability to drill down, but if
>> we follow this approach, region is not shown in the tables. This leads to a
>> different user experience.
>>
>> Since we don't have any other possible solutions as suggested by Mohan,
>> let's go with the first approach. Please create a jira for this and state
>> required changes to the stream definitions.
>>
>> On Wed, Jun 29, 2016 at 12:38 PM, Mohanadarshan Vivekanandalingam <
>> mo...@wso2.com> wrote:
>>
>>> HI Johann & Hasintha,
>>>
>>> Appreciate any feedback on above, then we can implement accordingly and
>>> do some perf tests.. If drill capability is required for Map then we need
>>> to follow above option.. Here, we can
>>>
>>> 1) Change the stream in Identity level and publish
>>> 2) Change at DAS level by adding another attributes in Siddhi scope..
>>>
>>> But, I like to avoid option [2] to avoid unnecessary event conversions
>>> (which impact the TPS).. Then, option [1] is the only solution in this
>>> case..
>>>
>>> Thanks,
>>> Mohan
>>>
>>>
>>> On Tue, Jun 28, 2016 at 4:16 PM, Nuwandi Wickramasinghe <
>>> nuwan...@wso2.com> wrote:
>>>
 Hi all,

 IS Analytics dashboard contains a map in which we have marked the
 regions of users who have authenticated through Identity Server. Our
 concern is whether we should allow to drill down from this map.

 Normally when drilling down from a gadget, in the dashboard we only
 show the data related to that selected field. And there's a data table
 below which shows the raw data corresponding to the selected field. This
 data table is populated using the raw stream which is published from
 Identity Server side.

 When implementing drill down capabilities from the area chart (map), we
 are facing an issue in populating the data table since the raw data stream
 published from IS contains only the remoteIp (which is used to determine
 the region) but not the region field itself.

 Regions and IP addresses have one to many relationship. Which means a
 large range of IPs corresponds to one region. Therefore we can get the
 region from IP but not vise versa. This leaves the issue that we cannot
 search in the raw stream using the region selected from the map.

 Suggested solution is to change the raw data stream to contain an
 additional 'region' attribute which could be used to search for the data
 related to a particular region. The value of region for the raw data will
 be sent as 'NOT_AVAILABLE'. We are planning to update this value to the
 correct region according to the remoteIp value using a spark query. We
 considered updating the records from the execution plan but that could
 delay the real time process.

 If we are going to implement this, Hasintha will have to introduce the
 new region attribute to the stream with a default value. Also this might
 introduce some performance degrade in the db level.

 We are still considering the need of drilling down from the region map.
 Appreciate other opinions regarding the subject.

 Thanks
 Nuwandi

 --

 Best Regards,

 Nuwandi Wickramasinghe

 Software Engineer

 WSO2 Inc.

 Web : http://wso2.com

 Mobile : 0719214873

>>>
>>>
>>>
>>> --
>>> *V. Mohanadarshan*
>>> *Associate Tech Lead,*
>>> *Data Technologies Team,*
>>> *WSO2, Inc. http://wso2.com  *
>>> *lean.enterprise.middleware.*
>>>
>>> email: mo...@wso2.com
>>> phone:(+94) 771117673
>>>
>>
>>
>>
>> --
>> Hasintha Indrajee
>> WSO2, Inc.
>> Mobile:+94 771892453
>>
>>
>
>
> --
>
> Best Regards,
>
> Nuwandi Wickramasinghe
>
> Software Engineer
>
> WSO2 Inc.
>
> Web : http://wso2.com
>
> Mobile : 0719214873
>



-- 
Hasintha Indrajee
WSO2, Inc.
Mobile:+94 771892453
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Adding drill down capabilities to the Area Chart in IS Analytics Server Dashboard

[Nuwandi Wickramasinghe]

Please find the public jira in [1]

[1] https://wso2.org/jira/browse/IDENTITY-4739

On Wed, Jun 29, 2016 at 2:51 PM, Hasintha Indrajee 
wrote:

> Hi all,
>
> Had an offline discussion with Nuwandi and following are some points.
>
> - We can filter region from IP and provide ability to drill down, but if
> we follow this approach, region is not shown in the tables. This leads to a
> different user experience.
>
> Since we don't have any other possible solutions as suggested by Mohan,
> let's go with the first approach. Please create a jira for this and state
> required changes to the stream definitions.
>
> On Wed, Jun 29, 2016 at 12:38 PM, Mohanadarshan Vivekanandalingam <
> mo...@wso2.com> wrote:
>
>> HI Johann & Hasintha,
>>
>> Appreciate any feedback on above, then we can implement accordingly and
>> do some perf tests.. If drill capability is required for Map then we need
>> to follow above option.. Here, we can
>>
>> 1) Change the stream in Identity level and publish
>> 2) Change at DAS level by adding another attributes in Siddhi scope..
>>
>> But, I like to avoid option [2] to avoid unnecessary event conversions
>> (which impact the TPS).. Then, option [1] is the only solution in this
>> case..
>>
>> Thanks,
>> Mohan
>>
>>
>> On Tue, Jun 28, 2016 at 4:16 PM, Nuwandi Wickramasinghe <
>> nuwan...@wso2.com> wrote:
>>
>>> Hi all,
>>>
>>> IS Analytics dashboard contains a map in which we have marked the
>>> regions of users who have authenticated through Identity Server. Our
>>> concern is whether we should allow to drill down from this map.
>>>
>>> Normally when drilling down from a gadget, in the dashboard we only show
>>> the data related to that selected field. And there's a data table below
>>> which shows the raw data corresponding to the selected field. This data
>>> table is populated using the raw stream which is published from Identity
>>> Server side.
>>>
>>> When implementing drill down capabilities from the area chart (map), we
>>> are facing an issue in populating the data table since the raw data stream
>>> published from IS contains only the remoteIp (which is used to determine
>>> the region) but not the region field itself.
>>>
>>> Regions and IP addresses have one to many relationship. Which means a
>>> large range of IPs corresponds to one region. Therefore we can get the
>>> region from IP but not vise versa. This leaves the issue that we cannot
>>> search in the raw stream using the region selected from the map.
>>>
>>> Suggested solution is to change the raw data stream to contain an
>>> additional 'region' attribute which could be used to search for the data
>>> related to a particular region. The value of region for the raw data will
>>> be sent as 'NOT_AVAILABLE'. We are planning to update this value to the
>>> correct region according to the remoteIp value using a spark query. We
>>> considered updating the records from the execution plan but that could
>>> delay the real time process.
>>>
>>> If we are going to implement this, Hasintha will have to introduce the
>>> new region attribute to the stream with a default value. Also this might
>>> introduce some performance degrade in the db level.
>>>
>>> We are still considering the need of drilling down from the region map.
>>> Appreciate other opinions regarding the subject.
>>>
>>> Thanks
>>> Nuwandi
>>>
>>> --
>>>
>>> Best Regards,
>>>
>>> Nuwandi Wickramasinghe
>>>
>>> Software Engineer
>>>
>>> WSO2 Inc.
>>>
>>> Web : http://wso2.com
>>>
>>> Mobile : 0719214873
>>>
>>
>>
>>
>> --
>> *V. Mohanadarshan*
>> *Associate Tech Lead,*
>> *Data Technologies Team,*
>> *WSO2, Inc. http://wso2.com  *
>> *lean.enterprise.middleware.*
>>
>> email: mo...@wso2.com
>> phone:(+94) 771117673
>>
>
>
>
> --
> Hasintha Indrajee
> WSO2, Inc.
> Mobile:+94 771892453
>
>


-- 

Best Regards,

Nuwandi Wickramasinghe

Software Engineer

WSO2 Inc.

Web : http://wso2.com

Mobile : 0719214873
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Adding drill down capabilities to the Area Chart in IS Analytics Server Dashboard

[Hasintha Indrajee]

Hi all,

Had an offline discussion with Nuwandi and following are some points.

- We can filter region from IP and provide ability to drill down, but if we
follow this approach, region is not shown in the tables. This leads to a
different user experience.

Since we don't have any other possible solutions as suggested by Mohan,
let's go with the first approach. Please create a jira for this and state
required changes to the stream definitions.

On Wed, Jun 29, 2016 at 12:38 PM, Mohanadarshan Vivekanandalingam <
mo...@wso2.com> wrote:

> HI Johann & Hasintha,
>
> Appreciate any feedback on above, then we can implement accordingly and do
> some perf tests.. If drill capability is required for Map then we need to
> follow above option.. Here, we can
>
> 1) Change the stream in Identity level and publish
> 2) Change at DAS level by adding another attributes in Siddhi scope..
>
> But, I like to avoid option [2] to avoid unnecessary event conversions
> (which impact the TPS).. Then, option [1] is the only solution in this
> case..
>
> Thanks,
> Mohan
>
>
> On Tue, Jun 28, 2016 at 4:16 PM, Nuwandi Wickramasinghe  > wrote:
>
>> Hi all,
>>
>> IS Analytics dashboard contains a map in which we have marked the regions
>> of users who have authenticated through Identity Server. Our concern is
>> whether we should allow to drill down from this map.
>>
>> Normally when drilling down from a gadget, in the dashboard we only show
>> the data related to that selected field. And there's a data table below
>> which shows the raw data corresponding to the selected field. This data
>> table is populated using the raw stream which is published from Identity
>> Server side.
>>
>> When implementing drill down capabilities from the area chart (map), we
>> are facing an issue in populating the data table since the raw data stream
>> published from IS contains only the remoteIp (which is used to determine
>> the region) but not the region field itself.
>>
>> Regions and IP addresses have one to many relationship. Which means a
>> large range of IPs corresponds to one region. Therefore we can get the
>> region from IP but not vise versa. This leaves the issue that we cannot
>> search in the raw stream using the region selected from the map.
>>
>> Suggested solution is to change the raw data stream to contain an
>> additional 'region' attribute which could be used to search for the data
>> related to a particular region. The value of region for the raw data will
>> be sent as 'NOT_AVAILABLE'. We are planning to update this value to the
>> correct region according to the remoteIp value using a spark query. We
>> considered updating the records from the execution plan but that could
>> delay the real time process.
>>
>> If we are going to implement this, Hasintha will have to introduce the
>> new region attribute to the stream with a default value. Also this might
>> introduce some performance degrade in the db level.
>>
>> We are still considering the need of drilling down from the region map.
>> Appreciate other opinions regarding the subject.
>>
>> Thanks
>> Nuwandi
>>
>> --
>>
>> Best Regards,
>>
>> Nuwandi Wickramasinghe
>>
>> Software Engineer
>>
>> WSO2 Inc.
>>
>> Web : http://wso2.com
>>
>> Mobile : 0719214873
>>
>
>
>
> --
> *V. Mohanadarshan*
> *Associate Tech Lead,*
> *Data Technologies Team,*
> *WSO2, Inc. http://wso2.com  *
> *lean.enterprise.middleware.*
>
> email: mo...@wso2.com
> phone:(+94) 771117673
>



-- 
Hasintha Indrajee
WSO2, Inc.
Mobile:+94 771892453
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] Adding drill down capabilities to the Area Chart in IS Analytics Server Dashboard

[Mohanadarshan Vivekanandalingam]

HI Johann & Hasintha,

Appreciate any feedback on above, then we can implement accordingly and do
some perf tests.. If drill capability is required for Map then we need to
follow above option.. Here, we can

1) Change the stream in Identity level and publish
2) Change at DAS level by adding another attributes in Siddhi scope..

But, I like to avoid option [2] to avoid unnecessary event conversions
(which impact the TPS).. Then, option [1] is the only solution in this
case..

Thanks,
Mohan


On Tue, Jun 28, 2016 at 4:16 PM, Nuwandi Wickramasinghe 
wrote:

> Hi all,
>
> IS Analytics dashboard contains a map in which we have marked the regions
> of users who have authenticated through Identity Server. Our concern is
> whether we should allow to drill down from this map.
>
> Normally when drilling down from a gadget, in the dashboard we only show
> the data related to that selected field. And there's a data table below
> which shows the raw data corresponding to the selected field. This data
> table is populated using the raw stream which is published from Identity
> Server side.
>
> When implementing drill down capabilities from the area chart (map), we
> are facing an issue in populating the data table since the raw data stream
> published from IS contains only the remoteIp (which is used to determine
> the region) but not the region field itself.
>
> Regions and IP addresses have one to many relationship. Which means a
> large range of IPs corresponds to one region. Therefore we can get the
> region from IP but not vise versa. This leaves the issue that we cannot
> search in the raw stream using the region selected from the map.
>
> Suggested solution is to change the raw data stream to contain an
> additional 'region' attribute which could be used to search for the data
> related to a particular region. The value of region for the raw data will
> be sent as 'NOT_AVAILABLE'. We are planning to update this value to the
> correct region according to the remoteIp value using a spark query. We
> considered updating the records from the execution plan but that could
> delay the real time process.
>
> If we are going to implement this, Hasintha will have to introduce the new
> region attribute to the stream with a default value. Also this might
> introduce some performance degrade in the db level.
>
> We are still considering the need of drilling down from the region map.
> Appreciate other opinions regarding the subject.
>
> Thanks
> Nuwandi
>
> --
>
> Best Regards,
>
> Nuwandi Wickramasinghe
>
> Software Engineer
>
> WSO2 Inc.
>
> Web : http://wso2.com
>
> Mobile : 0719214873
>



-- 
*V. Mohanadarshan*
*Associate Tech Lead,*
*Data Technologies Team,*
*WSO2, Inc. http://wso2.com  *
*lean.enterprise.middleware.*

email: mo...@wso2.com
phone:(+94) 771117673
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev