it was good idea to have a look at StorageProxy :-)
1.0.10 Performance Tests StorageProxy RangeOperations: 546 ReadOperations: 694563 TotalHints: 0 TotalRangeLatencyMicros: 4469484 TotalReadLatencyMicros:245669679 TotalWriteLatencyMicros: 57819722 WriteOperations:208741 0.7.10 Performance Tests StorageProxy RangeOperations: 520 ReadOperations: 671476 TotalRangeLatencyMicros: 2208902 TotalReadLatencyMicros: 162186009 TotalWriteLatencyMicros: 33911222 WriteOperations: 204806 2012/9/3 aaron morton <aa...@thelastpickle.com> > The whole test run is taking longer ? So it could be slower queries or > slower test setup / tear down? > > If you are creating and truncate the KS for each of the 500 tests is that > taking longer ? (Schema code has changed a lot 0.7 > 1.0) > Can you log the execution time for tests and find ones that are taking > longer ? > > There are full request metrics available on the StorageProxy JMX object. > > Cheers > > ----------------- > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 31/08/2012, at 4:45 PM, Илья Шипицин <chipits...@gmail.com> wrote: > > we are using functional tests ( ~500 tests in time). > it is hard to tell which query is slower, it is "slower in general". > > same hardware. 1 node, 32Gb RAM, 8Gb heap. default cassandra settings. > as we are talking about functional tests, so we recreate KS just before > tests are run. > > I do not know how to record queries (there are a lot of them), if you are > interested, I can set up a special stand for you. > > 2012/8/31 aaron morton <aa...@thelastpickle.com> > >> we are running somewhat queue-like with aggressive write-read patterns. >> >> We'll need some more details... >> >> How much data ? >> How many machines ? >> What is the machine spec ? >> How many clients ? >> Is there an example of a slow request ? >> How are you measuring that it's slow ? >> Is there anything unusual in the log ? >> >> Cheers >> >> ----------------- >> Aaron Morton >> Freelance Developer >> @aaronmorton >> http://www.thelastpickle.com >> >> On 31/08/2012, at 3:30 AM, Edward Capriolo <edlinuxg...@gmail.com> wrote: >> >> If you move from 7.X to 0.8X or 1.0X you have to rebuild sstables as >> soon as possible. If you have large bloomfilters you can hit a bug >> where the bloom filters will not work properly. >> >> >> On Thu, Aug 30, 2012 at 9:44 AM, Илья Шипицин <chipits...@gmail.com> >> wrote: >> >> we are running somewhat queue-like with aggressive write-read patterns. >> I was looking for scripting queries from live Cassandra installation, but >> I >> didn't find any. >> >> is there something like thrift-proxy or other query logging/scripting >> engine >> ? >> >> 2012/8/30 aaron morton <aa...@thelastpickle.com> >> >> >> in terms of our high-rate write load cassandra1.0.11 is about 3 (three!!) >> times slower than cassandra-0.7.8 >> >> We've not had any reports of a performance drop off. All tests so far have >> show improvements in both read and write performance. >> >> I agree, such digests save some network IO, but they seem to be very bad >> in terms of CPU and disk IO. >> >> The sha1 is created so we can diagnose corruptions in the -Data component >> of the SSTables. They are not used to save network IO. >> It is calculated while streaming the Memtable to disk so has no impact on >> disk IO. While not the fasted algorithm I would assume it's CPU overhead >> in >> this case is minimal. >> >> there's already relatively small Bloom filter file, which can be used for >> saving network traffic instead of sha1 digest. >> >> Bloom filters are used to test if a row key may exist in an SSTable. >> >> any explanation ? >> >> If you can provide some more information on your use case we may be able >> to help. >> >> Cheers >> >> >> ----------------- >> Aaron Morton >> Freelance Developer >> @aaronmorton >> http://www.thelastpickle.com >> >> On 30/08/2012, at 5:18 AM, Илья Шипицин <chipits...@gmail.com> wrote: >> >> in terms of our high-rate write load cassandra1.0.11 is about 3 (three!!) >> times slower than cassandra-0.7.8 >> after some investigation carried out I noticed files with "sha1" extension >> (which are missing for Cassandra-0.7.8) >> >> in maybeWriteDigest() function I see no option fot switching sha1 digests >> off. >> >> I agree, such digests save some network IO, but they seem to be very bad >> in terms of CPU and disk IO. >> why to use one more digest (which have to be calculated), there's already >> relatively small Bloom filter file, which can be used for saving network >> traffic instead of sha1 digest. >> >> any explanation ? >> >> Ilya Shipitsin >> >> >> >> >> > >
