[
https://issues.apache.org/jira/browse/CASSANDRA-7088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13981113#comment-13981113
]
Jacek Furmankiewicz commented on CASSANDRA-7088:
------------------------------------------------
Here is what I can see.
The error occurs mostly after test cases where do a lot of multi-threaded mass
inserts.
We have loaders for reference data that can accept a CSV file (let's say with
10,000 records). We parse it line by line and parallelize the writes to various
Cassandra tables (so just lots of runnable in an Executor all dumping data to
Cassandra across many concurrent threads).
Then as part of the validation we fetch a single column from a row (which has
data as a byte array document). After 2-3 queries like that that are invoked
from these test cases I can see the BigInteger exception pop up...and then the
entire test suite just freezes for a few second waiting for Cassandra to
proceed.
Just by manually tailing system.log while watching the test suite execute I can
see that this errors occurs every time we execute a test that has massive
concurrent writes.
> Massive performance degradation for TRUNCATE when migrating to 2.0
> ------------------------------------------------------------------
>
> Key: CASSANDRA-7088
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7088
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Linux Mint 16
> Reporter: Jacek Furmankiewicz
> Attachments: cassandra.yaml
>
>
> We attempted to migrate our developers to Cassandra 2.0.7 from 1.2.
> Everything worked perfectly, but we have experienced a massive drop in
> developer velocity.
> We run integration tests with Cucumber BDD and 1000 BDDs went from 7 minutes
> (Cassandra 1.2) to 15 minutes (2.0.7),
> This is when we run Cassandra of the ramdisk (/dev/shm) to make it run faster
> on dev boxes.
> When we tried pointed to actual drives the difference was dramatic: the
> entire suite took over 70 minutes (!) vs 15 in Cassandra 1.2.
> After investigation, we found that most of the time is spent in the
> truncation logic between every scenario, where we truncate all the column
> families and start with a clean DB for the next test case.
> This used to be super fast in 1.2, is now very slow in 2.0.
> It may not seem important, but upgrading to 2.0 has basically cut down
> developer velocity by 100%, just by more than doubling the time it takes to
> run our BDD suite.
> We truncate the CFs using the Ruby driver:
> $cassandra.column_families.each do |column_family|
> name = column_family[0].to_s
> $cassandra.truncate! name
> end
> I am attaching our cassandra.yaml. Please note we already switched off
> auto_compaction before truncate, just as we did in 1.2 for dev boxes, Made no
> difference.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
