Hi Thayalan, 1. 19K is good 2. Please draw X axis from 0, then only u can see overall trends and effect 3. We need to do Cassandra and BAM related tunning (if any) and rerun the test. 4. Please start a GDoc and start to write down the setup and configs results etc 5. Please finished tunning and direct publish and publish with analytics numbers before change Cassandra cluster as Anjana mentioned.
Look good. --Srinath On Tue, Sep 2, 2014 at 9:48 PM, Thayalan <[email protected]> wrote: > Hi Anjana, > > Yes, we have seperate partitions for commitlog and data in all 3 cassandra > nodes. The data has been sent to single node using the client. I'll stop > one cassandra node while sending the data and see how the TPS will get > affected and share the results with you. The replication factor used in > this set-up is 3 with RW consistentcy level set to QUORUM. Please note we > haven't done any performance optimization in any of the cassandra nodes. > > Thanks, > Thayalan > > > > On Tue, Sep 2, 2014 at 7:41 PM, Anjana Fernando <[email protected]> wrote: > >> Looks good! .. so with this we basically gets like 19K TPS average I >> guess .. In this setup, do we use separate partitions for the commit log >> and for the data? .. that is what basically recommended for a Cassandra >> setup. And also, how are we sending data, is it in a client load balanced >> manner? .. or through a single node? .. please test those scenarios >> separately, to check how it will affect the performance. And also, can you >> like reduce the number of Cassandra nodes also and check how it can affect >> the performance. Basically wants to see, if we can get higher TPS values if >> we add more nodes, or by any chance, if the opposite effect happens. And >> also, do mention the replication factor used for this. >> >> Cheers, >> Anjana. >> >> >> On Tue, Sep 2, 2014 at 4:26 PM, Thayalan <[email protected]> wrote: >> >>> Hi All, >>> >>> Please find below the initial receiver perf test results depicted in a >>> graph. I'm in the process of capturing the throughput with analyzer script >>> running as well. I'll share the results soon. >>> >>> [image: Inline image 2] >>> >>> [image: Inline image 3] >>> >>> Notes: >>> 1. The perf test performed using the DEBS data contains 69008700 events >>> (records) >>> 2. Throughput captured for every 1000000 events >>> 3. Environment Details: (Perf Cloud, openstack) >>> Deployment Pattern: As per BAM Cluster guide >>> <https://docs.wso2.com/display/CLUSTER420/Fully-Distributed%2C+High-Availability+BAM+Setup#Fully-Distributed,High-AvailabilityBAMSetup-Hadoopcluster> >>> Machine Configuration: >>> Node1 & Node2: 8 Core, 16GB Mem, 160GB HDD >>> Node3, 4 & 5: 8 Core, 16GB Mem, 160GB HDD & 1TB Volume seperately >>> attached & mounted for Cassandra Data partition >>> >>> Node1 & Node2: BAM Receiver & Analzer Noders, Hadoop Master & Secondary >>> respect >>> >>> >>> On Fri, Aug 15, 2014 at 6:59 PM, Sinthuja Ragendran <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> I have completed upto puppetizing BAM Receiver configurations, and due >>>> to support priorities I wasn't able to work full time on this. Hence >>>> cassandra and analyzer confgurations are still pending. Anyhow since I have >>>> done with the base confgurations, I should be able to provide the complete >>>> puppet scripts by next week. >>>> >>>> Thanks, >>>> Sinthuja. >>>> >>>> >>>> On Fri, Aug 15, 2014 at 6:22 PM, Sanjiva Weerawarana <[email protected]> >>>> wrote: >>>> >>>>> Is the setup all automated with Puppet? >>>>> >>>>> >>>>> On Fri, Aug 15, 2014 at 11:26 AM, Thayalan <[email protected]> wrote: >>>>> >>>>>> Hi Srinath, >>>>>> >>>>>> Due to support priority this has not been started yet. However I've >>>>>> already done the environment set-up in performance cloud. Probably I can >>>>>> share the initial findings by next week. >>>>>> >>>>>> Thanks, >>>>>> Thayalan >>>>>> >>>>>> >>>>>> On Fri, Aug 15, 2014 at 8:58 AM, Srinath Perera <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> How are we doing with the subject? >>>>>>> -- >>>>>>> ============================ >>>>>>> Director, Research, WSO2 Inc. >>>>>>> Visiting Faculty, University of Moratuwa >>>>>>> Member, Apache Software Foundation >>>>>>> Research Scientist, Lanka Software Foundation >>>>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>>>> Site: http://people.apache.org/~hemapani/ >>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>> Phone: 0772360902 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> [email protected] >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Sanjiva Weerawarana, Ph.D. >>>>> Founder, Chairman & CEO; WSO2, Inc.; http://wso2.com/ >>>>> email: [email protected]; office: (+1 650 745 4499 | +94 11 214 5345) >>>>> x5700; cell: +94 77 787 6880 | +1 408 466 5099; voip: +1 650 265 8311 >>>>> blog: http://sanjiva.weerawarana.org/; twitter: @sanjiva >>>>> Lean . Enterprise . Middleware >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Sinthuja Rajendran* >>>> Senior Software Engineer <http://wso2.com/> >>>> WSO2, Inc.:http://wso2.com >>>> >>>> Blog: http://sinthu-rajan.blogspot.com/ >>>> Mobile: +94774273955 >>>> >>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Regards, >>> Thayalan Sivapaleswararajah >>> Associate Technical Lead - QA >>> Mob: +94(0)777872485 >>> Tel : +94(0)(11)2145345 >>> Fax : +94(0)(11)2145300 >>> Email: [email protected] >>> >>> >>> *Disclaimer*: *This communication may contain privileged or other >>> confidential information and is intended exclusively for the addressee/s. >>> If you are not the intended recipient/s, or believe that you may have >>> received this communication in error, please reply to the sender indicating >>> that fact and delete the copy you received and in addition, you should not >>> print, copy, retransmit, disseminate, or otherwise use the information >>> contained in this communication. Internet communications cannot be >>> guaranteed to be timely, secure, error or virus-free. The sender does not >>> accept liability for any errors or omissions.* >>> >>> >> >> >> -- >> *Anjana Fernando* >> Senior Technical Lead >> WSO2 Inc. | http://wso2.com >> lean . enterprise . middleware >> > > > > -- > Regards, > Thayalan Sivapaleswararajah > Associate Technical Lead - QA > Mob: +94(0)777872485 > Tel : +94(0)(11)2145345 > Fax : +94(0)(11)2145300 > Email: [email protected] > > > *Disclaimer*: *This communication may contain privileged or other > confidential information and is intended exclusively for the addressee/s. > If you are not the intended recipient/s, or believe that you may have > received this communication in error, please reply to the sender indicating > that fact and delete the copy you received and in addition, you should not > print, copy, retransmit, disseminate, or otherwise use the information > contained in this communication. Internet communications cannot be > guaranteed to be timely, secure, error or virus-free. The sender does not > accept liability for any errors or omissions.* > > -- ============================ Srinath Perera, Ph.D. http://people.apache.org/~hemapani/ http://srinathsview.blogspot.com/
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
