Re: Difference between yum and git
Thanks Jonathan, Joaquin, Sorry, I found logback.xml caused the difference. I changed logging level TRACE and the maxFileSize of debug.log in conf/logback.xml. logback.xml of C* by yum (changed): logginglevel: TRACE maxFileSize of debug.log: 500MB logback.xml of C* by git (default): logginglevel: DEBUG maxFileSize of debug.log: 20MB I have another question. When the size of debug.log becomes the maxFileSize, Cassandra stops temporarily? Some operations failed at that time due to Read Timeout. According to vmstat, the process didn't run as below. Is it normal?? $ vmstat 1 procs ---memory-- ---swap-- -io --system-- -cpu- r b swpd free buff cache si sobibo in cs us sy id wa st 22 1 0 1645528 477624 343747600 0 1460 11908 23589 81 12 4 4 0 8 1 0 1642520 477624 344023200 0 1396 12080 24284 82 7 6 5 1 21 1 0 1639732 477624 344305200 0 1404 11331 22967 82 10 4 3 0 1 1 0 1636848 477624 344566400 0 1440 12085 24791 81 9 5 5 0 0 0 0 1635144 477624 344753200 0 1044 12982 24702 57 5 27 10 1 < debug.log became maxFileSize 0 0 0 1635176 477624 344770000 016 628 1128 0 0 100 0 0 0 0 0 1635176 477624 344770400 0 0 612 1108 1 0 100 0 0 0 0 0 1635176 477624 344770800 0 0 603 1103 0 0 100 0 0 0 0 0 1635176 477624 344770800 0 0 636 1136 1 0 99 0 0 0 0 0 1635176 477624 344771600 0 0 582 1088 0 0 100 0 0 15 1 0 1634492 477624 344816000 0 1024 8069 12479 16 2 79 3 0 27 1 0 1631732 477624 345077200 0 1620 12144 28098 81 10 7 3 0 9 1 0 1629036 477624 345370000 0 1416 12003 23268 79 10 6 5 0 8 1 0 1626524 477624 345612400 0 1372 14024 22991 78 8 7 6 1 Thanks, Yuji On Thu, May 11, 2017 at 12:23 PM, Jonathan Haddad wrote: > Where are you getting Cassandra 2.2 built from yum? > On Wed, May 10, 2017 at 9:54 PM Yuji Ito wrote: > >> Hi Joaquin, >> >> > Were both tests run from the same machine at close the same time? >> Yes. I run the both tests within 30 min. >> I retried them today. The result was the same as yesterday. >> >> The test run on the same instances and the same Java. >> >> Thanks, >> Yuji >> >> >> On Thu, May 11, 2017 at 3:27 AM, Joaquin Casares < >> joaq...@thelastpickle.com> wrote: >> >>> Hi Yuji, >>> >>> Were both tests run from the same machine at close the same time? If >>> not, noisy neighbors may be affecting your performance on different AWS >>> instances. >>> >>> You should verify that you're using the same version of Java during both >>> tests. >>> >>> Also, ensure that you're using the same test instance (that is not >>> running Cassandra) to connect to both Cassandra clusters. >>> >>> Cheers, >>> >>> Joaquin >>> >>> Joaquin Casares >>> Consultant >>> Austin, TX >>> >>> Apache Cassandra Consulting >>> http://www.thelastpickle.com >>> >>> On Wed, May 10, 2017 at 5:01 AM, Yuji Ito wrote: >>> Hi all, I'm trying a simple performance test. The test requests select operations (CL.SERIAL or CL.QUORUM) by increasing the number of threads. There is the difference of the performance between C* installed by yum and C* which I built by myself. What causes the difference? I use C* 2.2.8. One of them was installed by yum (# yum install cassandra22). Another was acquired by git from https://github.com/apache/ cassandra/tree/cassandra-2.2.8 and built it by myself. I changed cassandra.yaml to set `commitlog_sync: batch` and `commitlog_sync_batch_window_in_ms: 2`. My environment: - a cluster has 3 nodes - node: AWS EC2 m4.large with 200 IOPS EBS volume - Replication Factor: 3 - 1 rows Result: ** yum select (CL.SERIAL) threads operations/sec 1 188 2 156 4 434 8 396 16 837 32 1176 64 2206 128 4115 256 7272 ** git select (CL.SERIAL) threads operations/sec 1 192 2 162 4 264 8 446 16 733 32 1114 64 1715 128 2776 256 3920 ** yum select (CL.QUORUM) threads operations/sec 1 434 2 909 4 1481 8 1904 16 2666 32 3106 64 3555 128 5000 256 9014 ** git select (CL.QUORUM) threads operations/sec 1 666 2 1538 4 2500 8 16 4210 32 5333 64 6597 128 7356 256 8075 Thanks, Yuji >>> >>
Re: Difference between yum and git
Where are you getting Cassandra 2.2 built from yum? On Wed, May 10, 2017 at 9:54 PM Yuji Ito wrote: > Hi Joaquin, > > > Were both tests run from the same machine at close the same time? > Yes. I run the both tests within 30 min. > I retried them today. The result was the same as yesterday. > > The test run on the same instances and the same Java. > > Thanks, > Yuji > > > On Thu, May 11, 2017 at 3:27 AM, Joaquin Casares < > joaq...@thelastpickle.com> wrote: > >> Hi Yuji, >> >> Were both tests run from the same machine at close the same time? If not, >> noisy neighbors may be affecting your performance on different AWS >> instances. >> >> You should verify that you're using the same version of Java during both >> tests. >> >> Also, ensure that you're using the same test instance (that is not >> running Cassandra) to connect to both Cassandra clusters. >> >> Cheers, >> >> Joaquin >> >> Joaquin Casares >> Consultant >> Austin, TX >> >> Apache Cassandra Consulting >> http://www.thelastpickle.com >> >> On Wed, May 10, 2017 at 5:01 AM, Yuji Ito wrote: >> >>> Hi all, >>> >>> I'm trying a simple performance test. >>> The test requests select operations (CL.SERIAL or CL.QUORUM) by >>> increasing the number of threads. >>> There is the difference of the performance between C* installed by yum >>> and C* which I built by myself. >>> What causes the difference? >>> >>> I use C* 2.2.8. >>> One of them was installed by yum (# yum install cassandra22). >>> Another was acquired by git from >>> https://github.com/apache/cassandra/tree/cassandra-2.2.8 and built it >>> by myself. >>> I changed cassandra.yaml to set `commitlog_sync: batch` and >>> `commitlog_sync_batch_window_in_ms: 2`. >>> >>> My environment: >>> - a cluster has 3 nodes >>> - node: AWS EC2 m4.large with 200 IOPS EBS volume >>> - Replication Factor: 3 >>> - 1 rows >>> >>> Result: >>> ** yum >>> select (CL.SERIAL) >>> threads operations/sec >>> 1 188 >>> 2 156 >>> 4 434 >>> 8 396 >>> 16 837 >>> 32 1176 >>> 64 2206 >>> 128 4115 >>> 256 7272 >>> >>> ** git >>> select (CL.SERIAL) >>> threads operations/sec >>> 1 192 >>> 2 162 >>> 4 264 >>> 8 446 >>> 16 733 >>> 32 1114 >>> 64 1715 >>> 128 2776 >>> 256 3920 >>> >>> ** yum >>> select (CL.QUORUM) >>> threads operations/sec >>> 1 434 >>> 2 909 >>> 4 1481 >>> 8 1904 >>> 16 2666 >>> 32 3106 >>> 64 3555 >>> 128 5000 >>> 256 9014 >>> >>> ** git >>> select (CL.QUORUM) >>> threads operations/sec >>> 1 666 >>> 2 1538 >>> 4 2500 >>> 8 >>> 16 4210 >>> 32 5333 >>> 64 6597 >>> 128 7356 >>> 256 8075 >>> >>> Thanks, >>> Yuji >>> >>> >> >
Re: Difference between yum and git
Hi Joaquin, > Were both tests run from the same machine at close the same time? Yes. I run the both tests within 30 min. I retried them today. The result was the same as yesterday. The test run on the same instances and the same Java. Thanks, Yuji On Thu, May 11, 2017 at 3:27 AM, Joaquin Casares wrote: > Hi Yuji, > > Were both tests run from the same machine at close the same time? If not, > noisy neighbors may be affecting your performance on different AWS > instances. > > You should verify that you're using the same version of Java during both > tests. > > Also, ensure that you're using the same test instance (that is not running > Cassandra) to connect to both Cassandra clusters. > > Cheers, > > Joaquin > > Joaquin Casares > Consultant > Austin, TX > > Apache Cassandra Consulting > http://www.thelastpickle.com > > On Wed, May 10, 2017 at 5:01 AM, Yuji Ito wrote: > >> Hi all, >> >> I'm trying a simple performance test. >> The test requests select operations (CL.SERIAL or CL.QUORUM) by >> increasing the number of threads. >> There is the difference of the performance between C* installed by yum >> and C* which I built by myself. >> What causes the difference? >> >> I use C* 2.2.8. >> One of them was installed by yum (# yum install cassandra22). >> Another was acquired by git from https://github.com/apache/cass >> andra/tree/cassandra-2.2.8 and built it by myself. >> I changed cassandra.yaml to set `commitlog_sync: batch` and >> `commitlog_sync_batch_window_in_ms: 2`. >> >> My environment: >> - a cluster has 3 nodes >> - node: AWS EC2 m4.large with 200 IOPS EBS volume >> - Replication Factor: 3 >> - 1 rows >> >> Result: >> ** yum >> select (CL.SERIAL) >> threads operations/sec >> 1 188 >> 2 156 >> 4 434 >> 8 396 >> 16 837 >> 32 1176 >> 64 2206 >> 128 4115 >> 256 7272 >> >> ** git >> select (CL.SERIAL) >> threads operations/sec >> 1 192 >> 2 162 >> 4 264 >> 8 446 >> 16 733 >> 32 1114 >> 64 1715 >> 128 2776 >> 256 3920 >> >> ** yum >> select (CL.QUORUM) >> threads operations/sec >> 1 434 >> 2 909 >> 4 1481 >> 8 1904 >> 16 2666 >> 32 3106 >> 64 3555 >> 128 5000 >> 256 9014 >> >> ** git >> select (CL.QUORUM) >> threads operations/sec >> 1 666 >> 2 1538 >> 4 2500 >> 8 >> 16 4210 >> 32 5333 >> 64 6597 >> 128 7356 >> 256 8075 >> >> Thanks, >> Yuji >> >> >
Re: Difference between yum and git
Hi Yuji, Were both tests run from the same machine at close the same time? If not, noisy neighbors may be affecting your performance on different AWS instances. You should verify that you're using the same version of Java during both tests. Also, ensure that you're using the same test instance (that is not running Cassandra) to connect to both Cassandra clusters. Cheers, Joaquin Joaquin Casares Consultant Austin, TX Apache Cassandra Consulting http://www.thelastpickle.com On Wed, May 10, 2017 at 5:01 AM, Yuji Ito wrote: > Hi all, > > I'm trying a simple performance test. > The test requests select operations (CL.SERIAL or CL.QUORUM) by increasing > the number of threads. > There is the difference of the performance between C* installed by yum and > C* which I built by myself. > What causes the difference? > > I use C* 2.2.8. > One of them was installed by yum (# yum install cassandra22). > Another was acquired by git from https://github.com/apache/cass > andra/tree/cassandra-2.2.8 and built it by myself. > I changed cassandra.yaml to set `commitlog_sync: batch` and > `commitlog_sync_batch_window_in_ms: 2`. > > My environment: > - a cluster has 3 nodes > - node: AWS EC2 m4.large with 200 IOPS EBS volume > - Replication Factor: 3 > - 1 rows > > Result: > ** yum > select (CL.SERIAL) > threads operations/sec > 1 188 > 2 156 > 4 434 > 8 396 > 16 837 > 32 1176 > 64 2206 > 128 4115 > 256 7272 > > ** git > select (CL.SERIAL) > threads operations/sec > 1 192 > 2 162 > 4 264 > 8 446 > 16 733 > 32 1114 > 64 1715 > 128 2776 > 256 3920 > > ** yum > select (CL.QUORUM) > threads operations/sec > 1 434 > 2 909 > 4 1481 > 8 1904 > 16 2666 > 32 3106 > 64 3555 > 128 5000 > 256 9014 > > ** git > select (CL.QUORUM) > threads operations/sec > 1 666 > 2 1538 > 4 2500 > 8 > 16 4210 > 32 5333 > 64 6597 > 128 7356 > 256 8075 > > Thanks, > Yuji > >
Difference between yum and git
Hi all, I'm trying a simple performance test. The test requests select operations (CL.SERIAL or CL.QUORUM) by increasing the number of threads. There is the difference of the performance between C* installed by yum and C* which I built by myself. What causes the difference? I use C* 2.2.8. One of them was installed by yum (# yum install cassandra22). Another was acquired by git from https://github.com/apache/ cassandra/tree/cassandra-2.2.8 and built it by myself. I changed cassandra.yaml to set `commitlog_sync: batch` and `commitlog_sync_batch_window_in_ms: 2`. My environment: - a cluster has 3 nodes - node: AWS EC2 m4.large with 200 IOPS EBS volume - Replication Factor: 3 - 1 rows Result: ** yum select (CL.SERIAL) threads operations/sec 1 188 2 156 4 434 8 396 16 837 32 1176 64 2206 128 4115 256 7272 ** git select (CL.SERIAL) threads operations/sec 1 192 2 162 4 264 8 446 16 733 32 1114 64 1715 128 2776 256 3920 ** yum select (CL.QUORUM) threads operations/sec 1 434 2 909 4 1481 8 1904 16 2666 32 3106 64 3555 128 5000 256 9014 ** git select (CL.QUORUM) threads operations/sec 1 666 2 1538 4 2500 8 16 4210 32 5333 64 6597 128 7356 256 8075 Thanks, Yuji