Hi all,
I think we cannot use primary keys since this table has row data and same
eventId can have multiple roles associated with the same username. We need
the role information. If we define primary keys we will probably have only
one record per eventId, because the last record that comes with the same
eventId and username will place the older record.
What we can do is, as Rukshan said, create a separate one column with with
username and eventId (comma separated), index it as a facet and then
implement a custom aggregate function to get the unique eventId count per
user. At runtime, we cannot combine random fields and create facets. a
single field should contain comma separated values ( In this scenario,
"username, eventId").
On Wed, Mar 9, 2016 at 8:39 PM, Rukshan Premathunga <ruks...@wso2.com>
wrote:
> Hi all,
>
> we need this two level of grouping is due to duplicated rows right? in
> that case can we define the userName, authSuccess as primary key when
> persist. Then this duplication will not occur and unique rows will persist.
> If it worked we can use the query damith mention earlier, right?
>
> If it not worked, i'm sure that we need extra attribute with composition
> of "eventId" and "userName", if we need multi level grouping.
>
> Thanks and regards.
>
>
> On Wed, Mar 9, 2016 at 8:09 PM, Seshika Fernando <sesh...@wso2.com> wrote:
>
>> Hi Rukshan,
>>
>> Could you clarify whether we need to create a seperate row for this? My
>> understanding of facets was that you can combine fields and create facets,
>> without creating a column (that combines other fields) explicitly.
>>
>> seshi
>>
>> On Wed, Mar 9, 2016 at 8:03 PM, Damith Wickramasinghe <dami...@wso2.com>
>> wrote:
>>
>>> Hi Rukshan,
>>>
>>> Thanks for the response. For above approach we need to have another
>>> column which has comma separated values for the facet values which is not
>>> having in my row data table at the moment. I was thinking of finding a way
>>> to achieve it without creating this additional field.
>>>
>>> @seshika if this is the case we will have to run a spark query to create
>>> this additional column. Either we can send this as another attribute from
>>> IS side but I don't think its a correct approach since we have to handle
>>> these data customizations in DAS side. WDYT ?
>>>
>>> Regards,
>>> Damith.
>>>
>>> On Wed, Mar 9, 2016 at 7:29 PM, Rukshan Premathunga <ruks...@wso2.com>
>>> wrote:
>>>
>>>> Hi Damith,
>>>>
>>>> you can define a facet attribute (ex: eventId_userName_facet) with
>>>> composition of eventId and userName. Then you can request the query as
>>>> below.
>>>>
>>>> {
>>>> tableName : "AUTHENTICATIONANALYTICSSTREAM",
>>>> groupByField:"eventId_userName_facet",
>>>> aggregateLevel: 1,
>>>> query : "timestamp:[" + startTimeStamp + " TO " +
>>>> endTimeStamp + "] AND authenticationSuccess:True",
>>>> NoOfRecords : 1000000,
>>>> aggregateFields : [
>>>> {
>>>> fieldName:"eventId",
>>>> aggregate:"COUNT",
>>>> alias:"AuthSuccess_Sum"
>>>> }
>>>> ]
>>>> }
>>>>
>>>>
>>>> Thanks and Regards.
>>>>
>>>> On Wed, Mar 9, 2016 at 5:25 PM, Damith Wickramasinghe <dami...@wso2.com
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Assume following row data set is persisted in DAS for stream
>>>>> definition of eventId , userName, authSuccess .
>>>>>
>>>>> 1,user1,true
>>>>> 1,user1,true
>>>>> 2,user1,true
>>>>> 3,user2,true
>>>>> 3,user2,true
>>>>> 3,user2,true
>>>>> 4,user3,true
>>>>> 4,user3,true
>>>>> 5,user3,true
>>>>> 6,user3,true
>>>>>
>>>>>
>>>>> I need to query above data set and get per user success count for a
>>>>> given time period. Since we have duplicate event ids we need to group by
>>>>> eventid first and then group by username.Assume above data range is for a
>>>>> specific time period. Final result I need is below which has two fields
>>>>> username and authSuccessCount.
>>>>>
>>>>> user1 , 2
>>>>> user2 , 1
>>>>> user3 , 3
>>>>>
>>>>> I can follow below type of query to group by using one field by making
>>>>> the eventId a Facet.
>>>>>
>>>>> {
>>>>> tableName : "AUTHENTICATIONANALYTICSSTREAM",
>>>>> groupByField:"eventId",
>>>>> query : "timestamp:[" + startTimeStamp + " TO " +
>>>>> endTimeStamp + "] AND authenticationSuccess:True",
>>>>> NoOfRecords : 1000000,
>>>>> aggregateFields : [
>>>>> {
>>>>> fieldName:"eventId",
>>>>> aggregate:"COUNT",
>>>>> alias:"AuthSuccess_Sum"
>>>>> }
>>>>> ]
>>>>> }
>>>>>
>>>>> How can we have multiple grouping functionality to achieve above ?
>>>>>
>>>>> Thanks,
>>>>> Damith.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Software Engineer
>>>>> WSO2 Inc.; http://wso2.com
>>>>> <http://www.google.com/url?q=http%3A%2F%2Fwso2.com&sa=D&sntz=1&usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg>
>>>>> lean.enterprise.middleware
>>>>>
>>>>> mobile: *+94728671315 <%2B94728671315>*
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Dev mailing list
>>>>> Dev@wso2.org
>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Rukshan Chathuranga.
>>>> Software Engineer.
>>>> WSO2, Inc.
>>>>
>>>
>>>
>>>
>>> --
>>> Software Engineer
>>> WSO2 Inc.; http://wso2.com
>>> <http://www.google.com/url?q=http%3A%2F%2Fwso2.com&sa=D&sntz=1&usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg>
>>> lean.enterprise.middleware
>>>
>>> mobile: *+94728671315 <%2B94728671315>*
>>>
>>>
>>
>
>
> --
> Rukshan Chathuranga.
> Software Engineer.
> WSO2, Inc.
>
> _______________________________________________
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>
--
Gimantha Bandara
Software Engineer
WSO2. Inc : http://wso2.com
Mobile : +94714961919
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
