Hi Gihan, I will try that way. thank you so much for the support you have given so far
Thanks On Wed, Oct 22, 2014 at 6:49 PM, Gihan Anuruddha <[email protected]> wrote: > Hi Lakshman, > > That's is a good observation. So I believe this event should be either > first one or the last one. Can you please do a another small test? > Shutdown the BAM server. > Delete the Cassandra data folder completely (You can find that in > repository/databases/cassandra) > Truncate the APIM related RDBMS tables. > Run your script with 100 requests. > > Please try to identify the message that going to miss. > > I think we need to check data sending part in APIM side? Also add a debug > log message to publishEvent method with static counter and print that > value. So it will be easy to identify the issue. > > Regards, > Gihan > > On Wed, Oct 22, 2014 at 6:37 PM, Lakshman Udayakantha <[email protected]> > wrote: > >> Hi Gihan, >> >> I sent 10000 requests with the preconditions you have mentioned. only one >> record is missed in cassendra database for every time I sent 10000 requests >> via APIM. >> >> Thanks >> >> On Wed, Oct 22, 2014 at 9:28 AM, Lakshman Udayakantha <[email protected] >> > wrote: >> >>> Hi Gihan, >>> >>> I will do that test and get back to you. >>> >>> Thanks >>> >>> On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> On Wednesday, October 22, 2014, Lakshman Udayakantha < >>>> [email protected]> wrote: >>>> >>>>> Hi Inosh, >>>>> >>>>> data in cassandra database are summized to a relational database. >>>>> records in cassandra database and this relational table is same. so I >>>>> think >>>>> there is no problem with cassandra explorer. >>>>> >>>> >>>> Ok understood. I have misread your previous email. >>>> >>>>> >>>>> Thanks >>>>> >>>>> On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Lakshman, >>>>>> >>>>>> You mean data is therw in cassandra and cassandra explorer doesn't >>>>>> show these data? Did you compare by using cassandra CLI tool? >>>>>> >>>>>> On Wednesday, October 22, 2014, Lakshman Udayakantha < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> HI Gihan , >>>>>>> >>>>>>> They are lost randomly. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Lakshman, >>>>>>>> >>>>>>>> How many records did you send to BAM and how many events are lost? >>>>>>>> >>>>>>>> Regards, >>>>>>>> Gihan >>>>>>>> >>>>>>>> On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Gihan, >>>>>>>>> >>>>>>>>> These data in cassandra database are summarised by a hive script >>>>>>>>> and insert into a relational database. I checked that relational >>>>>>>>> database >>>>>>>>> table also. Number of records in cassandra database and relational >>>>>>>>> database >>>>>>>>> table are same. Then it should not be a problem with cassandra >>>>>>>>> explorer. >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> >>>>>>>>> On Wed, Oct 22, 2014 at 7:53 AM, Manoj Gunawardena < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> This can be an issue for other products as well. ES, AS and EMM >>>>>>>>>> also publishing data to BAM. In ES we are using load balancing data >>>>>>>>>> publisher. I also notice if we call stop method, sometimes data not >>>>>>>>>> get >>>>>>>>>> publish to BAM. We changed this to initiate data publisher at the >>>>>>>>>> first >>>>>>>>>> event and call data publishers stop method in the bundle activator's >>>>>>>>>> stop >>>>>>>>>> method. >>>>>>>>>> Need to test with load. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Oct 21, 2014 at 10:07 PM, Gihan Anuruddha <[email protected] >>>>>>>>>> > wrote: >>>>>>>>>> >>>>>>>>>>> Hi Lakshman, >>>>>>>>>>> >>>>>>>>>>> It should be in Cassandra if publishEvent method executes. How >>>>>>>>>>> do you validate the data loss in Cassandra side? BAM embedded >>>>>>>>>>> Cassandra explorer is a not good tool for that. Can you please >>>>>>>>>>> verify that with tool like CQL or cassandra-cli? >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Gihan >>>>>>>>>>> >>>>>>>>>>> On Tue, Oct 21, 2014 at 9:18 PM, Lakshman Udayakantha < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi All, >>>>>>>>>>>> >>>>>>>>>>>> I am working on https://wso2.org/jira/browse/APIMANAGER-2064. >>>>>>>>>>>> >>>>>>>>>>>> By looking at the Cassandra database on BAM, I could figure out >>>>>>>>>>>> some requests publish to Cassandra database and some do not when I >>>>>>>>>>>> invoke >>>>>>>>>>>> the api. That is the reason for the missing requests in statistics >>>>>>>>>>>> dashboard. >>>>>>>>>>>> >>>>>>>>>>>> I tried debugging the APIMgtUsageDataBridgeDataPublisher class >>>>>>>>>>>> since it has publishEvent method to publish data. >>>>>>>>>>>> APIMgtUsageDataBridgeDataPublisher class uses >>>>>>>>>>>> LoadBalancingDataPublisher as >>>>>>>>>>>> the dataPublisher in this case. >>>>>>>>>>>> >>>>>>>>>>>> I found that every time when I invoke the api via rest client, >>>>>>>>>>>> this method will execute. but randomly some requests are not >>>>>>>>>>>> publishing in >>>>>>>>>>>> Cassandra database. I used the >>>>>>>>>>>> latest org.wso2.carbon.databridge.agent.thrift jar. but the >>>>>>>>>>>> problem is >>>>>>>>>>>> still there. >>>>>>>>>>>> >>>>>>>>>>>> if requests come to publishEvent method, should they publish in >>>>>>>>>>>> Cassandra database?and is there any known reason for data get not >>>>>>>>>>>> published >>>>>>>>>>>> in cassendra database. if not where I can find the data publisher >>>>>>>>>>>> to >>>>>>>>>>>> cassendra in BAM from API Manager. >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> -- >>>>>>>>>>>> Lakshman Udayakantha >>>>>>>>>>>> WSO2 Inc. www.wso2.com >>>>>>>>>>>> lean.enterprise.middleware >>>>>>>>>>>> Mobile: *0711241005 <0711241005>* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Dev mailing list >>>>>>>>>>>> [email protected] >>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> W.G. Gihan Anuruddha >>>>>>>>>>> Senior Software Engineer | WSO2, Inc. >>>>>>>>>>> M: +94772272595 >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Dev mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Manoj Gunawardena >>>>>>>>>> Tech Lead >>>>>>>>>> WSO2, Inc.: http://wso2.com >>>>>>>>>> lean.enterprise.middleware >>>>>>>>>> Mobile : +94 77 2291643 >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Lakshman Udayakantha >>>>>>>>> WSO2 Inc. www.wso2.com >>>>>>>>> lean.enterprise.middleware >>>>>>>>> Mobile: *0711241005 <0711241005>* >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> W.G. Gihan Anuruddha >>>>>>>> Senior Software Engineer | WSO2, Inc. >>>>>>>> M: +94772272595 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Lakshman Udayakantha >>>>>>> WSO2 Inc. www.wso2.com >>>>>>> lean.enterprise.middleware >>>>>>> Mobile: *0711241005 <0711241005>* >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> >>>>>> Inosh Goonewardena >>>>>> Associate Technical Lead- WSO2 Inc. >>>>>> Mobile: +94779966317 >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Lakshman Udayakantha >>>>> WSO2 Inc. www.wso2.com >>>>> lean.enterprise.middleware >>>>> Mobile: *0711241005 <0711241005>* >>>>> >>>>> >>>> >>>> -- >>>> Regards, >>>> >>>> Inosh Goonewardena >>>> Associate Technical Lead- WSO2 Inc. >>>> Mobile: +94779966317 >>>> >>>> >>> >>> >>> -- >>> Lakshman Udayakantha >>> WSO2 Inc. www.wso2.com >>> lean.enterprise.middleware >>> Mobile: *0711241005 <0711241005>* >>> >>> >> >> >> -- >> Lakshman Udayakantha >> WSO2 Inc. www.wso2.com >> lean.enterprise.middleware >> Mobile: *0711241005 <0711241005>* >> >> > > > -- > W.G. Gihan Anuruddha > Senior Software Engineer | WSO2, Inc. > M: +94772272595 > -- Lakshman Udayakantha WSO2 Inc. www.wso2.com lean.enterprise.middleware Mobile: *0711241005*
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
