Re: Hi Memory consumption with Copy command

2016-04-23 Thread Stefania Alborghetti
That's really excellent! Thank you so much for sharing the results. Regarding sstableloader, I am not familiar with its performance so I cannot make any recommendation as I've never compared it with COPY FROM. I have however compared COPY FROM with another bulk import tool, cassandra-loader,

Re: Hi Memory consumption with Copy command

2016-04-23 Thread Bhuvan Rawal
I built cython and disabled bundled driver, the performance has been impressive. Memory issue is resolved and Im currently getting around 100,000 rows per second, its stressing both the client CPU as well as cassandra nodes. Thats the fastest I have ever seen it perform. With 60 Million rows

Re: Hi Memory consumption with Copy command

2016-04-23 Thread Bhuvan Rawal
Thanks Stefania for the informative answer. The next blog was pretty useful as well: http://www.datastax.com/dev/blog/how-we-optimized-cassandra-cqlsh-copy-from . Ill upgrade to 3.0.5 and test with C extensions enabled and report on this thread. On Sat, Apr 23, 2016 at 8:54 AM, Stefania

Re: Hi Memory consumption with Copy command

2016-04-22 Thread Stefania Alborghetti
Hi Bhuvan Support for large datasets in COPY FROM was added by CASSANDRA-11053 , which is available in 2.1.14, 2.2.6, 3.0.5 and 3.5. Your scenario is valid with this patch applied. The 3.0.x and 3.x releases are already available, whilst the