Hi all, Now I have changed [1] based on the information given in this thread about the time period. The examples given were also changed to show how the processing is done based on the minutes and hours instead of seconds and minutes.
@Sachith and Shazni, can you please check the changes and let me know whether they are correct? [1] https://docs.wso2.com/display/DAS310/Incremental+Processing Best Regards, Rukshani. On Thu, Jan 26, 2017 at 9:11 AM, Rukshani Weerasinha <[email protected]> wrote: > Thank you Shavi. > > I will change this and let you know once it is done. > > On Wed, Jan 25, 2017 at 5:15 PM, Shavindri Dissanayake <[email protected] > > wrote: > >> Adding Ruk. >> >> Thanks & Regards >> Shavindri Dissanayake >> Technical Writer >> >> WSO2 Inc. >> lean.enterprise.middleware >> >> On Wed, Jan 25, 2017 at 5:03 PM, Shazni Nazeer <[email protected]> wrote: >> >>> There's also a change in the second param of the incrementalParams as >>> DAY, HOUR instead of number of seconds as mentioned in the documentation. >>> >>> @Documentation - could this page be corrected? >>> >>> On Wed, Jan 25, 2017 at 4:52 PM, Shazni Nazeer <[email protected]> wrote: >>> >>>> Thanks Fazlan. >>>> >>>> That'll be useful. Only difference I see is primary key comes prior to >>>> incrementalParams. I'll try this out. >>>> >>>> On Wed, Jan 25, 2017 at 4:44 PM, Fazlan Nazeem <[email protected]> >>>> wrote: >>>> >>>>> Hi Shazni, >>>>> >>>>> Please check [1] for how we use incrementalParams in APIM-Analytics >>>>> scripts. >>>>> >>>>> [1] https://git.io/vMxJq >>>>> >>>>> On Wed, Jan 25, 2017 at 1:29 PM, Shazni Nazeer <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I have a spark table as shown below. >>>>>> >>>>>> CREATE TEMPORARY TABLE APIHourlyTrafficSummary using CarbonAnalytics >>>>>> options (tableName "org_wso2telco_analytics_hub_s >>>>>> tream_traffic_summary_per_hour", >>>>>> schema " >>>>>> year INT -i, >>>>>> month INT -i, >>>>>> day INT -i, >>>>>> hour INT -i, >>>>>> eventTimeStamp STRING -i, >>>>>> api STRING -i, >>>>>> serviceProvider STRING -i, >>>>>> apiPublisher STRING -i, >>>>>> applicationName STRING -i, >>>>>> operatorId STRING -i, >>>>>> isSuccess BOOLEAN -i, >>>>>> msisdn STRING -i, >>>>>> direction STRING -i, >>>>>> totalCount INTEGER -i, >>>>>> serviceProviderId STRING -i, >>>>>> operatorName STRING -i, >>>>>> apiPublisherID STRING -i, >>>>>> apiID STRING -i, >>>>>> department STRING -i, >>>>>> applicationId STRING -i", >>>>>> primaryKeys "year,month,day,hour,eventTime >>>>>> Stamp,api,serviceProvider,apiPublisher,applicationName,opera >>>>>> torId,isSuccess,msisdn,direction" >>>>>> ); >>>>>> >>>>>> I want to add incrementalParams as mentioned in the following [1]. >>>>>> >>>>>> [1] https://docs.wso2.com/display/DAS310/Incremental+Processing >>>>>> >>>>>> Therefore I have added the line as below. >>>>>> >>>>>> CREATE TEMPORARY TABLE APIHourlyTrafficSummary using >>>>>> CarbonAnalytics options (tableName "org_wso2telco_analytics_hub_s >>>>>> tream_traffic_summary_per_hour", >>>>>> schema " >>>>>> year INT -i, >>>>>> month INT -i, >>>>>> day INT -i, >>>>>> hour INT -i, >>>>>> eventTimeStamp STRING -i, >>>>>> api STRING -i, >>>>>> serviceProvider STRING -i, >>>>>> apiPublisher STRING -i, >>>>>> applicationName STRING -i, >>>>>> operatorId STRING -i, >>>>>> isSuccess BOOLEAN -i, >>>>>> msisdn STRING -i, >>>>>> direction STRING -i, >>>>>> totalCount INTEGER -i, >>>>>> serviceProviderId STRING -i, >>>>>> operatorName STRING -i, >>>>>> apiPublisherID STRING -i, >>>>>> apiID STRING -i, >>>>>> department STRING -i, >>>>>> applicationId STRING -i", >>>>>> * incrementalParams "APIHourlyTrafficSummary, 0",* >>>>>> primaryKeys "year,month,day,hour,eventTime >>>>>> Stamp,api,serviceProvider,apiPublisher,applicationName,opera >>>>>> torId,isSuccess,msisdn,direction" >>>>>> ); >>>>>> >>>>>> But executing the script throws an error. looks like a syntax error. >>>>>> What am I doing wrong here? >>>>>> >>>>>> -- >>>>>> Shazni Nazeer >>>>>> Associate Technical Lead | WSO2 >>>>>> >>>>>> Mob : +94 777737331 >>>>>> LinkedIn : http://lk.linkedin.com/in/shazninazeer >>>>>> Blog : http://shazninazeer.blogspot.com >>>>>> >>>>>> <http://wso2.com/signature> >>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> [email protected] >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Thanks & Regards, >>>>> >>>>> Fazlan Nazeem >>>>> >>>>> *Software Engineer* >>>>> >>>>> *WSO2 Inc* >>>>> Mobile : +94772338839 >>>>> <%2B94%20%280%29%20773%20451194> >>>>> [email protected] >>>>> >>>> >>>> >>>> >>>> -- >>>> Shazni Nazeer >>>> Associate Technical Lead | WSO2 >>>> >>>> Mob : +94 777737331 >>>> LinkedIn : http://lk.linkedin.com/in/shazninazeer >>>> Blog : http://shazninazeer.blogspot.com >>>> >>>> <http://wso2.com/signature> >>>> >>> >>> >>> >>> -- >>> Shazni Nazeer >>> Associate Technical Lead | WSO2 >>> >>> Mob : +94 777737331 >>> LinkedIn : http://lk.linkedin.com/in/shazninazeer >>> Blog : http://shazninazeer.blogspot.com >>> >>> <http://wso2.com/signature> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "WSO2 Documentation Group" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/a/wso2.com/d/optout. >>> >> >> > > > -- > Rukshani Weerasinha > > WSO2 Inc. > Web:http://wso2.com > Mobile: 0777 683 738 > > -- Rukshani Weerasinha WSO2 Inc. Web:http://wso2.com Mobile: 0777 683 738
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
