@thaya, Can you please specify the x & y axes of these graphs. Isn't x axis is a timestamp in both the graphs, or is it the time taken for the process (hrs:mins:secs)?
Rgds On Wed, Sep 3, 2014 at 9:03 AM, Srinath Perera <[email protected]> wrote: > Hi Thayalan, > > 1. 19K is good > 2. Please draw X axis from 0, then only u can see overall trends and > effect > 3. We need to do Cassandra and BAM related tunning (if any) and rerun the > test. > 4. Please start a GDoc and start to write down the setup and configs > results etc > 5. Please finished tunning and direct publish and publish with analytics > numbers before change Cassandra cluster as Anjana mentioned. > > Look good. > > --Srinath > > > On Tue, Sep 2, 2014 at 9:48 PM, Thayalan <[email protected]> wrote: > >> Hi Anjana, >> >> Yes, we have seperate partitions for commitlog and data in all 3 >> cassandra nodes. The data has been sent to single node using the client. >> I'll stop one cassandra node while sending the data and see how the TPS >> will get affected and share the results with you. The replication factor >> used in this set-up is 3 with RW consistentcy level set to QUORUM. Please >> note we haven't done any performance optimization in any of the cassandra >> nodes. >> >> Thanks, >> Thayalan >> >> >> >> On Tue, Sep 2, 2014 at 7:41 PM, Anjana Fernando <[email protected]> wrote: >> >>> Looks good! .. so with this we basically gets like 19K TPS average I >>> guess .. In this setup, do we use separate partitions for the commit log >>> and for the data? .. that is what basically recommended for a Cassandra >>> setup. And also, how are we sending data, is it in a client load balanced >>> manner? .. or through a single node? .. please test those scenarios >>> separately, to check how it will affect the performance. And also, can you >>> like reduce the number of Cassandra nodes also and check how it can affect >>> the performance. Basically wants to see, if we can get higher TPS values if >>> we add more nodes, or by any chance, if the opposite effect happens. And >>> also, do mention the replication factor used for this. >>> >>> Cheers, >>> Anjana. >>> >>> >>> On Tue, Sep 2, 2014 at 4:26 PM, Thayalan <[email protected]> wrote: >>> >>>> Hi All, >>>> >>>> Please find below the initial receiver perf test results depicted in a >>>> graph. I'm in the process of capturing the throughput with analyzer script >>>> running as well. I'll share the results soon. >>>> >>>> [image: Inline image 2] >>>> >>>> [image: Inline image 3] >>>> >>>> Notes: >>>> 1. The perf test performed using the DEBS data contains 69008700 events >>>> (records) >>>> 2. Throughput captured for every 1000000 events >>>> 3. Environment Details: (Perf Cloud, openstack) >>>> Deployment Pattern: As per BAM Cluster guide >>>> <https://docs.wso2.com/display/CLUSTER420/Fully-Distributed%2C+High-Availability+BAM+Setup#Fully-Distributed,High-AvailabilityBAMSetup-Hadoopcluster> >>>> Machine Configuration: >>>> Node1 & Node2: 8 Core, 16GB Mem, 160GB HDD >>>> Node3, 4 & 5: 8 Core, 16GB Mem, 160GB HDD & 1TB Volume seperately >>>> attached & mounted for Cassandra Data partition >>>> >>>> Node1 & Node2: BAM Receiver & Analzer Noders, Hadoop Master & Secondary >>>> respect >>>> >>>> >>>> On Fri, Aug 15, 2014 at 6:59 PM, Sinthuja Ragendran <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I have completed upto puppetizing BAM Receiver configurations, and due >>>>> to support priorities I wasn't able to work full time on this. Hence >>>>> cassandra and analyzer confgurations are still pending. Anyhow since I >>>>> have >>>>> done with the base confgurations, I should be able to provide the complete >>>>> puppet scripts by next week. >>>>> >>>>> Thanks, >>>>> Sinthuja. >>>>> >>>>> >>>>> On Fri, Aug 15, 2014 at 6:22 PM, Sanjiva Weerawarana <[email protected] >>>>> > wrote: >>>>> >>>>>> Is the setup all automated with Puppet? >>>>>> >>>>>> >>>>>> On Fri, Aug 15, 2014 at 11:26 AM, Thayalan <[email protected]> wrote: >>>>>> >>>>>>> Hi Srinath, >>>>>>> >>>>>>> Due to support priority this has not been started yet. However I've >>>>>>> already done the environment set-up in performance cloud. Probably I can >>>>>>> share the initial findings by next week. >>>>>>> >>>>>>> Thanks, >>>>>>> Thayalan >>>>>>> >>>>>>> >>>>>>> On Fri, Aug 15, 2014 at 8:58 AM, Srinath Perera <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> How are we doing with the subject? >>>>>>>> -- >>>>>>>> ============================ >>>>>>>> Director, Research, WSO2 Inc. >>>>>>>> Visiting Faculty, University of Moratuwa >>>>>>>> Member, Apache Software Foundation >>>>>>>> Research Scientist, Lanka Software Foundation >>>>>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>>>>> Site: http://people.apache.org/~hemapani/ >>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>> Phone: 0772360902 >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Sanjiva Weerawarana, Ph.D. >>>>>> Founder, Chairman & CEO; WSO2, Inc.; http://wso2.com/ >>>>>> email: [email protected]; office: (+1 650 745 4499 | +94 11 214 >>>>>> 5345) x5700; cell: +94 77 787 6880 | +1 408 466 5099; voip: +1 650 >>>>>> 265 8311 >>>>>> blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva >>>>>> Lean . Enterprise . Middleware >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Sinthuja Rajendran* >>>>> Senior Software Engineer <http://wso2.com/> >>>>> WSO2, Inc.:http://wso2.com >>>>> >>>>> Blog: http://sinthu-rajan.blogspot.com/ >>>>> Mobile: +94774273955 >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Thayalan Sivapaleswararajah >>>> Associate Technical Lead - QA >>>> Mob: +94(0)777872485 >>>> Tel : +94(0)(11)2145345 >>>> Fax : +94(0)(11)2145300 >>>> Email: [email protected] >>>> >>>> >>>> *Disclaimer*: *This communication may contain privileged or other >>>> confidential information and is intended exclusively for the addressee/s. >>>> If you are not the intended recipient/s, or believe that you may have >>>> received this communication in error, please reply to the sender indicating >>>> that fact and delete the copy you received and in addition, you should not >>>> print, copy, retransmit, disseminate, or otherwise use the information >>>> contained in this communication. Internet communications cannot be >>>> guaranteed to be timely, secure, error or virus-free. The sender does not >>>> accept liability for any errors or omissions.* >>>> >>>> >>> >>> >>> -- >>> *Anjana Fernando* >>> Senior Technical Lead >>> WSO2 Inc. | http://wso2.com >>> lean . enterprise . middleware >>> >> >> >> >> -- >> Regards, >> Thayalan Sivapaleswararajah >> Associate Technical Lead - QA >> Mob: +94(0)777872485 >> Tel : +94(0)(11)2145345 >> Fax : +94(0)(11)2145300 >> Email: [email protected] >> >> >> *Disclaimer*: *This communication may contain privileged or other >> confidential information and is intended exclusively for the addressee/s. >> If you are not the intended recipient/s, or believe that you may have >> received this communication in error, please reply to the sender indicating >> that fact and delete the copy you received and in addition, you should not >> print, copy, retransmit, disseminate, or otherwise use the information >> contained in this communication. Internet communications cannot be >> guaranteed to be timely, secure, error or virus-free. The sender does not >> accept liability for any errors or omissions.* >> >> > > > -- > ============================ > Srinath Perera, Ph.D. > http://people.apache.org/~hemapani/ > http://srinathsview.blogspot.com/ > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Niranda Perera* Software Engineer, WSO2 Inc. Mobile: +94-71-554-8430 Twitter: @n1r44 <https://twitter.com/N1R44>
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
