Re: [Dev] [Architecture] APIM Statistics introduce new field to the Analytics events

2018-01-03 Thread Hasitha Hiranya 
Hi,

One scenario I faced with a customer is as follows.

1. A Custom HTTP header is coming with each request specifying the region
request was originated.
2. At gateway, this needs to be extracted as an field and published to
Analyser.
3. Analyser summarises the data and put to a DB (hourly/daily/monthly stats)
4. A dashboard can pull that data and show.

As per current implementation, we need to

1. Write a custom handler to extract values
2. Create a new stream to publish new field
3. Learn apache spark queries to perform summarisation
4. Learn Jaggery and create the dashboard

Are we discussing a customisable way to provide above functionality here?
If so, yes, it will be a great value addition.

Thanks

On Tue, Jan 2, 2018 at 9:26 AM, Fazlan Nazeem  wrote:

> +1 to add those extra information. This will strengthen our analytics
> story.
>
> @Rukshan,
>
> 1) Any specific reason to move those attributes from meta to value fields?
> 2) How are we planning to populate the data center ID? Environment
> variable?
>
> Please create a git issue if we haven't done it yet.
>
> On Mon, Dec 25, 2017 at 10:14 AM, Harsha Kumara  wrote:
>
>> Adding Fazlan. Shall we create a JIRA and prioritize this.
>>
>> On Mon, Dec 25, 2017 at 10:14 AM, Harsha Kumara  wrote:
>>
>>> +1 to include this. It's a basic requirement to differentiate the prod
>>> and sandbox traffic in the system. Also, correlation Id will help to track
>>> the request and response if needed.
>>>
>>> On Sat, Dec 23, 2017 at 12:25 PM, Rukshan Premathunga 
>>> wrote:
>>>
 Hi All,

 Can we check the possibility of the $Subject. In the latest
 APIM(wso2am-2.1.0-update4), correlation ID and the key type(sand or prod)
 is include in the event as JSON attribute[1]. I think it is better to moved
 those attributes as a single value attribute and passed to the Analyzer.
 So we can upgrade the stream versioning and do these changes.

 Also it is better to persist key type, data center ID in the STAT_DB
 and we can filter production and Sandbox analytics future. So when
 migrating we need to alter tables with key type. But we can assume that
 existing data are from the production key and insert those key type when
 migrating.

 So Any idea about this?


 [1]  meta_clientType:{"keyType":"PRODUCTION","correlationID",
 "19c4350a-2079-406a-bece-808954b4f2d7"}
 Thanks and Regards

 --
 Rukshan Chathuranga.
 Software Engineer.
 WSO2, Inc.
 +94711822074 <+94%2071%20182%202074>

 ___
 Architecture mailing list
 architect...@wso2.org
 https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture


>>>
>>>
>>> --
>>> Harsha Kumara
>>> Software Engineer, WSO2 Inc.
>>> Mobile: +94775505618 <+94%2077%20550%205618>
>>> Blog:harshcreationz.blogspot.com
>>>
>>
>>
>>
>> --
>> Harsha Kumara
>> Software Engineer, WSO2 Inc.
>> Mobile: +94775505618 <+94%2077%20550%205618>
>> Blog:harshcreationz.blogspot.com
>>
>
>
>
> --
> Thanks & Regards,
>
> *Fazlan Nazeem*
> Senior Software Engineer
> WSO2 Inc
> Mobile : +94772338839
> <%2B94%20%280%29%20773%20451194>
> fazl...@wso2.com
>
> ___
> Architecture mailing list
> architect...@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
*Hasitha Abeykoon*
Associate Technical Lead; WSO2, Inc.; http://wso2.com
*cell:* *+94 719363063*
*blog: **abeykoon.blogspot.com* 
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [Architecture] APIM Statistics introduce new field to the Analytics events

2018-01-01 Thread Fazlan Nazeem 
+1 to add those extra information. This will strengthen our analytics
story.

@Rukshan,

1) Any specific reason to move those attributes from meta to value fields?
2) How are we planning to populate the data center ID? Environment
variable?

Please create a git issue if we haven't done it yet.

On Mon, Dec 25, 2017 at 10:14 AM, Harsha Kumara  wrote:

> Adding Fazlan. Shall we create a JIRA and prioritize this.
>
> On Mon, Dec 25, 2017 at 10:14 AM, Harsha Kumara  wrote:
>
>> +1 to include this. It's a basic requirement to differentiate the prod
>> and sandbox traffic in the system. Also, correlation Id will help to track
>> the request and response if needed.
>>
>> On Sat, Dec 23, 2017 at 12:25 PM, Rukshan Premathunga 
>> wrote:
>>
>>> Hi All,
>>>
>>> Can we check the possibility of the $Subject. In the latest
>>> APIM(wso2am-2.1.0-update4), correlation ID and the key type(sand or prod)
>>> is include in the event as JSON attribute[1]. I think it is better to moved
>>> those attributes as a single value attribute and passed to the Analyzer.
>>> So we can upgrade the stream versioning and do these changes.
>>>
>>> Also it is better to persist key type, data center ID in the STAT_DB and
>>> we can filter production and Sandbox analytics future. So when migrating we
>>> need to alter tables with key type. But we can assume that existing data
>>> are from the production key and insert those key type when migrating.
>>>
>>> So Any idea about this?
>>>
>>>
>>> [1]  meta_clientType:{"keyType":"PRODUCTION","correlationID",
>>> "19c4350a-2079-406a-bece-808954b4f2d7"}
>>> Thanks and Regards
>>>
>>> --
>>> Rukshan Chathuranga.
>>> Software Engineer.
>>> WSO2, Inc.
>>> +94711822074 <+94%2071%20182%202074>
>>>
>>> ___
>>> Architecture mailing list
>>> architect...@wso2.org
>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>>
>>>
>>
>>
>> --
>> Harsha Kumara
>> Software Engineer, WSO2 Inc.
>> Mobile: +94775505618 <+94%2077%20550%205618>
>> Blog:harshcreationz.blogspot.com
>>
>
>
>
> --
> Harsha Kumara
> Software Engineer, WSO2 Inc.
> Mobile: +94775505618 <+94%2077%20550%205618>
> Blog:harshcreationz.blogspot.com
>



-- 
Thanks & Regards,

*Fazlan Nazeem*
Senior Software Engineer
WSO2 Inc
Mobile : +94772338839
<%2B94%20%280%29%20773%20451194>
fazl...@wso2.com
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [Architecture] APIM Statistics introduce new field to the Analytics events

2017-12-24 Thread Harsha Kumara 
Adding Fazlan. Shall we create a JIRA and prioritize this.

On Mon, Dec 25, 2017 at 10:14 AM, Harsha Kumara  wrote:

> +1 to include this. It's a basic requirement to differentiate the prod and
> sandbox traffic in the system. Also, correlation Id will help to track the
> request and response if needed.
>
> On Sat, Dec 23, 2017 at 12:25 PM, Rukshan Premathunga 
> wrote:
>
>> Hi All,
>>
>> Can we check the possibility of the $Subject. In the latest
>> APIM(wso2am-2.1.0-update4), correlation ID and the key type(sand or prod)
>> is include in the event as JSON attribute[1]. I think it is better to moved
>> those attributes as a single value attribute and passed to the Analyzer.
>> So we can upgrade the stream versioning and do these changes.
>>
>> Also it is better to persist key type, data center ID in the STAT_DB and
>> we can filter production and Sandbox analytics future. So when migrating we
>> need to alter tables with key type. But we can assume that existing data
>> are from the production key and insert those key type when migrating.
>>
>> So Any idea about this?
>>
>>
>> [1]  meta_clientType:{"keyType":"PRODUCTION","correlationID",
>> "19c4350a-2079-406a-bece-808954b4f2d7"}
>> Thanks and Regards
>>
>> --
>> Rukshan Chathuranga.
>> Software Engineer.
>> WSO2, Inc.
>> +94711822074 <+94%2071%20182%202074>
>>
>> ___
>> Architecture mailing list
>> architect...@wso2.org
>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>>
>>
>
>
> --
> Harsha Kumara
> Software Engineer, WSO2 Inc.
> Mobile: +94775505618 <+94%2077%20550%205618>
> Blog:harshcreationz.blogspot.com
>



-- 
Harsha Kumara
Software Engineer, WSO2 Inc.
Mobile: +94775505618
Blog:harshcreationz.blogspot.com
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev

Re: [Dev] [Architecture] APIM Statistics introduce new field to the Analytics events

2017-12-24 Thread Harsha Kumara 
+1 to include this. It's a basic requirement to differentiate the prod and
sandbox traffic in the system. Also, correlation Id will help to track the
request and response if needed.

On Sat, Dec 23, 2017 at 12:25 PM, Rukshan Premathunga 
wrote:

> Hi All,
>
> Can we check the possibility of the $Subject. In the latest
> APIM(wso2am-2.1.0-update4), correlation ID and the key type(sand or prod)
> is include in the event as JSON attribute[1]. I think it is better to moved
> those attributes as a single value attribute and passed to the Analyzer.
> So we can upgrade the stream versioning and do these changes.
>
> Also it is better to persist key type, data center ID in the STAT_DB and
> we can filter production and Sandbox analytics future. So when migrating we
> need to alter tables with key type. But we can assume that existing data
> are from the production key and insert those key type when migrating.
>
> So Any idea about this?
>
>
> [1]  meta_clientType:{"keyType":"PRODUCTION","correlationID",
> "19c4350a-2079-406a-bece-808954b4f2d7"}
> Thanks and Regards
>
> --
> Rukshan Chathuranga.
> Software Engineer.
> WSO2, Inc.
> +94711822074 <+94%2071%20182%202074>
>
> ___
> Architecture mailing list
> architect...@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>


-- 
Harsha Kumara
Software Engineer, WSO2 Inc.
Mobile: +94775505618
Blog:harshcreationz.blogspot.com
___
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev