Only thing is then we will have to create multiple columns for the different dimensions - eg:- username/eventid, role/eventid, IDP/eventId etc;
On Wed, Mar 9, 2016 at 9:25 PM, Gimantha Bandara <[email protected]> wrote: > Hi all, > > I think we cannot use primary keys since this table has row data and same > eventId can have multiple roles associated with the same username. We need > the role information. If we define primary keys we will probably have only > one record per eventId, because the last record that comes with the same > eventId and username will place the older record. > > What we can do is, as Rukshan said, create a separate one column with with > username and eventId (comma separated), index it as a facet and then > implement a custom aggregate function to get the unique eventId count per > user. At runtime, we cannot combine random fields and create facets. a > single field should contain comma separated values ( In this scenario, > "username, eventId"). > > On Wed, Mar 9, 2016 at 8:39 PM, Rukshan Premathunga <[email protected]> > wrote: > >> Hi all, >> >> we need this two level of grouping is due to duplicated rows right? in >> that case can we define the userName, authSuccess as primary key when >> persist. Then this duplication will not occur and unique rows will persist. >> If it worked we can use the query damith mention earlier, right? >> >> If it not worked, i'm sure that we need extra attribute with composition >> of "eventId" and "userName", if we need multi level grouping. >> >> Thanks and regards. >> >> >> On Wed, Mar 9, 2016 at 8:09 PM, Seshika Fernando <[email protected]> >> wrote: >> >>> Hi Rukshan, >>> >>> Could you clarify whether we need to create a seperate row for this? My >>> understanding of facets was that you can combine fields and create facets, >>> without creating a column (that combines other fields) explicitly. >>> >>> seshi >>> >>> On Wed, Mar 9, 2016 at 8:03 PM, Damith Wickramasinghe <[email protected]> >>> wrote: >>> >>>> Hi Rukshan, >>>> >>>> Thanks for the response. For above approach we need to have another >>>> column which has comma separated values for the facet values which is not >>>> having in my row data table at the moment. I was thinking of finding a way >>>> to achieve it without creating this additional field. >>>> >>>> @seshika if this is the case we will have to run a spark query to >>>> create this additional column. Either we can send this as another attribute >>>> from IS side but I don't think its a correct approach since we have to >>>> handle these data customizations in DAS side. WDYT ? >>>> >>>> Regards, >>>> Damith. >>>> >>>> On Wed, Mar 9, 2016 at 7:29 PM, Rukshan Premathunga <[email protected]> >>>> wrote: >>>> >>>>> Hi Damith, >>>>> >>>>> you can define a facet attribute (ex: eventId_userName_facet) with >>>>> composition of eventId and userName. Then you can request the query >>>>> as below. >>>>> >>>>> { >>>>> tableName : "AUTHENTICATIONANALYTICSSTREAM", >>>>> groupByField:"eventId_userName_facet", >>>>> aggregateLevel: 1, >>>>> query : "timestamp:[" + startTimeStamp + " TO " + >>>>> endTimeStamp + "] AND authenticationSuccess:True", >>>>> NoOfRecords : 1000000, >>>>> aggregateFields : [ >>>>> { >>>>> fieldName:"eventId", >>>>> aggregate:"COUNT", >>>>> alias:"AuthSuccess_Sum" >>>>> } >>>>> ] >>>>> } >>>>> >>>>> >>>>> Thanks and Regards. >>>>> >>>>> On Wed, Mar 9, 2016 at 5:25 PM, Damith Wickramasinghe < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Assume following row data set is persisted in DAS for stream >>>>>> definition of eventId , userName, authSuccess . >>>>>> >>>>>> 1,user1,true >>>>>> 1,user1,true >>>>>> 2,user1,true >>>>>> 3,user2,true >>>>>> 3,user2,true >>>>>> 3,user2,true >>>>>> 4,user3,true >>>>>> 4,user3,true >>>>>> 5,user3,true >>>>>> 6,user3,true >>>>>> >>>>>> >>>>>> I need to query above data set and get per user success count for a >>>>>> given time period. Since we have duplicate event ids we need to group by >>>>>> eventid first and then group by username.Assume above data range is for a >>>>>> specific time period. Final result I need is below which has two fields >>>>>> username and authSuccessCount. >>>>>> >>>>>> user1 , 2 >>>>>> user2 , 1 >>>>>> user3 , 3 >>>>>> >>>>>> I can follow below type of query to group by using one field by >>>>>> making the eventId a Facet. >>>>>> >>>>>> { >>>>>> tableName : "AUTHENTICATIONANALYTICSSTREAM", >>>>>> groupByField:"eventId", >>>>>> query : "timestamp:[" + startTimeStamp + " TO " + >>>>>> endTimeStamp + "] AND authenticationSuccess:True", >>>>>> NoOfRecords : 1000000, >>>>>> aggregateFields : [ >>>>>> { >>>>>> fieldName:"eventId", >>>>>> aggregate:"COUNT", >>>>>> alias:"AuthSuccess_Sum" >>>>>> } >>>>>> ] >>>>>> } >>>>>> >>>>>> How can we have multiple grouping functionality to achieve above ? >>>>>> >>>>>> Thanks, >>>>>> Damith. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Software Engineer >>>>>> WSO2 Inc.; http://wso2.com >>>>>> <http://www.google.com/url?q=http%3A%2F%2Fwso2.com&sa=D&sntz=1&usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg> >>>>>> lean.enterprise.middleware >>>>>> >>>>>> mobile: *+94728671315 <%2B94728671315>* >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> [email protected] >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Rukshan Chathuranga. >>>>> Software Engineer. >>>>> WSO2, Inc. >>>>> >>>> >>>> >>>> >>>> -- >>>> Software Engineer >>>> WSO2 Inc.; http://wso2.com >>>> <http://www.google.com/url?q=http%3A%2F%2Fwso2.com&sa=D&sntz=1&usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg> >>>> lean.enterprise.middleware >>>> >>>> mobile: *+94728671315 <%2B94728671315>* >>>> >>>> >>> >> >> >> -- >> Rukshan Chathuranga. >> Software Engineer. >> WSO2, Inc. >> >> _______________________________________________ >> Dev mailing list >> [email protected] >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Gimantha Bandara > Software Engineer > WSO2. Inc : http://wso2.com > Mobile : +94714961919 >
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
