Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-28 Thread Dilini Muthumala
Hi Ravindra, Thanks for your suggestion. Hi all, Seems like the following is the root cause for this issue: When an endpoint is unavailable, we noticed that, whatever is given after the send mediator (in the insequence) doesn't get executed. As a result, the BAM mediator, which is placed after

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-27 Thread Ravindra Ranwala
Hi Lakshman, AFAIK this looks like a configuration issue. You have given a name to endpoints which were defined inline. Can you please change the configuration as given in article [1] and simplify this. Then try that out again. [1]

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-26 Thread Lakshman Udayakantha
Hi All, As per the offline discussion with Lakmali, The reason for this issue is when one end point is down, It will fail to go to the bam mediator(which is responsible for publishing event to destination statistics table in BAM cassandra database)which is added after send mediator in synapse

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-22 Thread Maninda Edirisooriya
Hi Lakshman, First you should make sure all the required statistics are published to BAM with the data publisher. In order to do it you can, 1. Add a debug log to the data publisher publishing method to log all the messages passed through it. 2. Then you can use Cassandra Explorer or another tool

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-22 Thread Lakshman Udayakantha
Hi Gihan, I sent 1 requests with the preconditions you have mentioned. only one record is missed in cassendra database for every time I sent 1 requests via APIM. Thanks On Wed, Oct 22, 2014 at 9:28 AM, Lakshman Udayakantha lakshm...@wso2.com wrote: Hi Gihan, I will do that test and

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-22 Thread Gihan Anuruddha
Hi Lakshman, That's is a good observation. So I believe this event should be either first one or the last one. Can you please do a another small test? Shutdown the BAM server. Delete the Cassandra data folder completely (You can find that in repository/databases/cassandra) Truncate the APIM

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-22 Thread Lakshman Udayakantha
Hi Gihan, I will try that way. thank you so much for the support you have given so far Thanks On Wed, Oct 22, 2014 at 6:49 PM, Gihan Anuruddha gi...@wso2.com wrote: Hi Lakshman, That's is a good observation. So I believe this event should be either first one or the last one. Can you please

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-22 Thread Manoj Gunawardena
Hi Lakshman, This is very useful results, Thanks for sharing. We will do the same test for ES as well. Is this test run for single BAM instance, or multiple? (load balanced environment) Thanks On Wed, Oct 22, 2014 at 6:37 PM, Lakshman Udayakantha lakshm...@wso2.com wrote: Hi Gihan, I sent

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-22 Thread Lakshman Udayakantha
Hi Manoj, I tested this with single BAM instance. As Gihan mentioned ,Still need to check there is a problem in APIM side. I will update this thread if problem is in APIM Thanks On Wed, Oct 22, 2014 at 7:49 PM, Manoj Gunawardena man...@wso2.com wrote: Hi Lakshman, This is very useful

[Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-21 Thread Lakshman Udayakantha
Hi All, I am working on https://wso2.org/jira/browse/APIMANAGER-2064. By looking at the Cassandra database on BAM, I could figure out some requests publish to Cassandra database and some do not when I invoke the api. That is the reason for the missing requests in statistics dashboard. I tried

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-21 Thread Gihan Anuruddha
Hi Lakshman, It should be in Cassandra if publishEvent method executes. How do you validate the data loss in Cassandra side? BAM embedded Cassandra explorer is a not good tool for that. Can you please verify that with tool like CQL or cassandra-cli? Regards, Gihan On Tue, Oct 21, 2014 at 9:18

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-21 Thread Manoj Gunawardena
Hi, This can be an issue for other products as well. ES, AS and EMM also publishing data to BAM. In ES we are using load balancing data publisher. I also notice if we call stop method, sometimes data not get publish to BAM. We changed this to initiate data publisher at the first event and call

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-21 Thread Lakshman Udayakantha
Hi Gihan, These data in cassandra database are summarised by a hive script and insert into a relational database. I checked that relational database table also. Number of records in cassandra database and relational database table are same. Then it should not be a problem with cassandra explorer.

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-21 Thread Gihan Anuruddha
Hi Lakshman, How many records did you send to BAM and how many events are lost? Regards, Gihan On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha lakshm...@wso2.com wrote: Hi Gihan, These data in cassandra database are summarised by a hive script and insert into a relational database. I

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-21 Thread Lakshman Udayakantha
HI Gihan , They are lost randomly. Thanks On Wed, Oct 22, 2014 at 9:00 AM, Gihan Anuruddha gi...@wso2.com wrote: Hi Lakshman, How many records did you send to BAM and how many events are lost? Regards, Gihan On Wed, Oct 22, 2014 at 8:18 AM, Lakshman Udayakantha lakshm...@wso2.com

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-21 Thread Inosh Goonewardena
Hi Lakshman, You mean data is therw in cassandra and cassandra explorer doesn't show these data? Did you compare by using cassandra CLI tool? On Wednesday, October 22, 2014, Lakshman Udayakantha lakshm...@wso2.com wrote: HI Gihan , They are lost randomly. Thanks On Wed, Oct 22, 2014 at

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-21 Thread Lakshman Udayakantha
Hi Inosh, data in cassandra database are summized to a relational database. records in cassandra database and this relational table is same. so I think there is no problem with cassandra explorer. Thanks On Wed, Oct 22, 2014 at 9:14 AM, Inosh Goonewardena in...@wso2.com wrote: Hi Lakshman,

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-21 Thread Inosh Goonewardena
Hi, On Wednesday, October 22, 2014, Lakshman Udayakantha lakshm...@wso2.com wrote: Hi Inosh, data in cassandra database are summized to a relational database. records in cassandra database and this relational table is same. so I think there is no problem with cassandra explorer. Ok

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-21 Thread Gihan Anuruddha
Hi Lakshman, Can you please do this test. I believe currently you are using BAM 2.4.1. Please you this 2.5.0 RC [1]. Clean your RDBMS as well. Then using a tool like JMeter send 10,000 requests to APIM and see you are getting 10,000 events in BAM. Please run these tests without any debugging

Re: [Dev] [BAM][API-Manager] issue in updating API Manager statistics dashboard with BAM

2014-10-21 Thread Lakshman Udayakantha
Hi Gihan, I will do that test and get back to you. Thanks On Wed, Oct 22, 2014 at 9:23 AM, Inosh Goonewardena in...@wso2.com wrote: Hi, On Wednesday, October 22, 2014, Lakshman Udayakantha lakshm...@wso2.com wrote: Hi Inosh, data in cassandra database are summized to a relational