Re: scylladb

On Tue, Mar 14, 2017 at 7:43 AM, Eric Evans wrote: > On Sun, Mar 12, 2017 at 4:01 PM, James Carman > wrote: > > Does all of this Scylla talk really even belong on the Cassandra user > > mailing list in the first place? > > I personally found it interesting, informative, and on-topic when it > wa

Re: scylladb

On Sun, Mar 12, 2017 at 4:01 PM, James Carman wrote: > Does all of this Scylla talk really even belong on the Cassandra user > mailing list in the first place? I personally found it interesting, informative, and on-topic when it was about justification of the 10x performance claim, numa, scheduli

Re: scylladb

On Mon, Mar 13, 2017 at 12:17 AM, benjamin roth wrote: > @Dor,Jeff: > > I think Jeff pointed out an important fact: You cannot stop CS, swap > binaries and start Scylla. To be honest that was AFAIR the only "Oooh :(" I > had when reading the Scylla "marketing material". > If you're on 2.1.x you

Re: scylladb

Hi, We did indeed consider support for a mixed cluster, but in the end decided against it, for many reasons: - the internode protocol is underdocumented and keeps changing, so it would be hard to support it, and hard to test it - it would limit the kind of optimizations we can do by redu

Re: scylladb

We came to the thread to provide technical answers about whether the difference in performance arise from C++ only or beyond. When the discussion included numa, we even dove deep into the weeds. I think we provided enough answers and I respect all of the opinions here and thus if someone has furthe

Re: scylladb

@Dor,Jeff: I think Jeff pointed out an important fact: You cannot stop CS, swap binaries and start Scylla. To be honest that was AFAIR the only "Oooh :(" I had when reading the Scylla "marketing material". If that worked it would be very valuable from both Scylla's and a users' point of view. As

Re: scylladb

I don't think ScyallDB guys started this conversation in the first place to suggest or promote "drop-in replacement". It was something that is brought up by one of the Cassandra users and ScyallDB guys just clarified it. They are gracious enough to share the internals in detail. honestly, I find i

Re: scylladb

On 2017-03-12 14:29 (-0700), James Carman wrote: > Well, looking back, it appears this thread is from 2015, so apparently > everyone is okay with it. > > Promoting a value-add product that makes using Cassandra easier/more > efficient/etc would be cool, but coming to the Cassandra mailing list

Re: scylladb

Well, looking back, it appears this thread is from 2015, so apparently everyone is okay with it. Promoting a value-add product that makes using Cassandra easier/more efficient/etc would be cool, but coming to the Cassandra mailing list to promote a "drop-in replacement" (use us, not Cassandra) isn

Re: scylladb

yes. On Sun, Mar 12, 2017 at 2:01 PM, James Carman wrote: > Does all of this Scylla talk really even belong on the Cassandra user > mailing list in the first place? > > > > > On Sun, Mar 12, 2017 at 4:07 PM Jeff Jirsa wrote: > > > > On 2017-03-11 22:33 (-0700), Dor Laor wrote: > > On Sat, Mar

Re: scylladb

Does all of this Scylla talk really even belong on the Cassandra user mailing list in the first place? On Sun, Mar 12, 2017 at 4:07 PM Jeff Jirsa wrote: On 2017-03-11 22:33 (-0700), Dor Laor wrote: > On Sat, Mar 11, 2017 at 10:02 PM, Jeff Jirsa wrote: > > On 2017-03-10 09:57 (-0800), Rake

Re: scylladb

ne in real workload, we just shot ourselves in the foot. This was the > case around our > beta but it was just a beta. Today we think we're in a very solid > position. We still > have lots to complete around repair (which is ok but not great). There > is a work > i

Re: scylladb

On 2017-03-11 22:33 (-0700), Dor Laor wrote: > On Sat, Mar 11, 2017 at 10:02 PM, Jeff Jirsa wrote: > > On 2017-03-10 09:57 (-0800), Rakesh Kumar wrote: > > > Cassanda vs Scylla is a valid comparison because they both are > > compatible. Scylla is a drop-in replacement for Cassandra. > > > > No

Re: scylladb

e mixed feelings about anti-compaction for incremental repair but we're likely to go through this route too > > > > On Sun, Mar 12, 2017 at 2:15 PM, Jonathan Haddad > wrote: > >> I don't think Jeff comes across as angry. He's simply pointing out that

Re: scylladb

So again. Know what you don't know. On Sun, Mar 12, 2017 at 2:15 PM, Jonathan Haddad wrote: > I don't think Jeff comes across as angry. He's simply pointing out that > ScyllaDB isn't a drop in replacement for Cassandra. Saying that it is is > very misleading. Th

Re: scylladb

On Sun, Mar 12, 2017 at 11:15 AM, Jonathan Haddad wrote: > I don't think Jeff comes across as angry. He's simply pointing out that > ScyllaDB isn't a drop in > Agree, I take it back, it's wasn't due to this. > replacement for Cassandra. Saying that it is

Re: scylladb

On Sun, Mar 12, 2017 at 6:40 AM, Stefan Podkowinski wrote: > If someone would create a benchmark showing that Cassandra is 10x faster > than Aerospike, would that mean Cassandra is 100x faster than ScyllaDB? > > Joking aside, I personally don't pay a lot of attention

Re: scylladb

I don't think Jeff comes across as angry. He's simply pointing out that ScyllaDB isn't a drop in replacement for Cassandra. Saying that it is is very misleading. The marketing material should really say something like "drop in replacement for some workloads" or "ai

RE: scylladb

Will you support custom secondary indexes, triggers and UDF? I checked index code but it’s just a couple of files with commented Java code. I’m curious to test Scylladb but our application uses LWT and custom secondary indexes, I understand LWT is coming (soon?). -- Jacques-Henri Berthemet

Re: scylladb

On Sat, Mar 11, 2017 at 1:52 AM Avi Kivity wrote: > > > Lastly, why don't you test Scylla yourself? It's pretty easy to set up, > there's nothing to tune. > > Avi > I'll look seriously at Scylla when it is 3.0.12 compatible.

Re: scylladb

On Sun, Mar 12, 2017 at 11:40 AM, Edward Capriolo wrote: > > > On Sun, Mar 12, 2017 at 1:38 AM, benjamin roth wrote: > >> There is no reason to be angry. This is progress. This is the circle of >> live. >> >> It happens anywhere at any time. >> >> Am 12.03.2017 07:34 schrieb "Dor Laor" : >> >>>

Re: scylladb

On Sun, Mar 12, 2017 at 1:38 AM, benjamin roth wrote: > There is no reason to be angry. This is progress. This is the circle of > live. > > It happens anywhere at any time. > > Am 12.03.2017 07:34 schrieb "Dor Laor" : > >> On Sat, Mar 11, 2017 at 10:02 PM, Jeff Jirsa wrote: >> >>> >>> >>> On 201

Re: scylladb

If someone would create a benchmark showing that Cassandra is 10x faster than Aerospike, would that mean Cassandra is 100x faster than ScyllaDB? Joking aside, I personally don't pay a lot of attention to any published benchmarks and look at them as pure marketing material. What I'm int

Re: scylladb

One more thing. Pretty much every database that is written in C++ or Java uses native kernel threads for non-blocking I/O as well. They didn't use Seaster or Quasar but anyways I am going to read up on Seaster and see what it really does. On Sun, Mar 12, 2017 at 3:48 AM, Kant Kodali wrote: > > I

Re: scylladb

am *sure* that the difference is substantial. >>> >>> Can you please elaborate on your workload and the type of machines you >>> use? >>> Schema and row length plus access pattern are also welcomed. >>> >>> Cheers, >>> Dor >>> >

Re: scylladb

> If you have thread-per-core and N (logical) cores, and have M tasks > running concurrently where M > N, then you need a scheduler to decide which > of those M tasks gets to run on those N kernel threads. Whether those M > tasks are user-level threads, or callbacks, or a mix of the two is > immat

Re: scylladb

nsive applications. But for these I/O intensive applications thread-per-core is the right choice, regardless of the points you raise. I encourage you to study the seastar code base [1] and documentation [2] to see how we handled those problems. I'll also comment a bit below.

Re: scylladb

code base [1] and documentation [2] > to see how we handled those problems. I'll also comment a bit below. > > > [1] https://github.com/scylladb/seastar > > [2] http://www.seastar-project.org/ > > On 03/12/2017 11:07 AM, Kant Kodali wrote: > > @Avi > > "U

Re: scylladb

is the right choice, regardless of the points you raise. I encourage you to study the seastar code base [1] and documentation [2] to see how we handled those problems. I'll also comment a bit below. [1] https://github.com/scylladb/seastar [2] http://www.seastar-project.org/ On 03/12/2017 11

Re: scylladb

On Sun, Mar 12, 2017 at 12:23 AM, Avi Kivity wrote: > > > On 03/12/2017 12:19 AM, Kant Kodali wrote: > > My response is inline. > > On Sat, Mar 11, 2017 at 1:43 PM, Avi Kivity wrote: > >> There are several issues at play here. >> >> First, a database runs a large number of concurrent operations,

Re: scylladb

like below : > https://issues.apache.org/jira/browse/CASSANDRA-10989 > Possibly going forward the number of cores per node is only going to > increase as it has been seen for last 5-6 years. In a way thats suggesting > a significant change in design and possibly thats what scylladb is upto

Re: scylladb

. In a way thats suggesting a significant change in design and possibly thats what scylladb is upto. "We found that we need a cpu scheduler which takes into account the priority of different tasks, such as repair, compaction, streaming, read operations and write operations." >From my und

Re: scylladb

@Avi "User-level scheduling is great for high performance I/O intensive applications like databases and file systems." This is generally a claim made by people you want to use user-level threads but I rarely had seen any significant performance gain. Since you are claiming that you do. It would be

Re: scylladb

btw, for an example of how user-level tasks can be scheduled in a way that cannot be done with kernel threads, see this pair of blog posts: http://www.scylladb.com/2016/04/14/io-scheduler-1/ http://www.scylladb.com/2016/04/29/io-scheduler-2/ There's simply no way to get this kind of cont

Re: scylladb

On 03/12/2017 12:19 AM, Kant Kodali wrote: My response is inline. On Sat, Mar 11, 2017 at 1:43 PM, Avi Kivity > wrote: There are several issues at play here. First, a database runs a large number of concurrent operations, each of which only consumes a s

Re: scylladb

On Sat, Mar 11, 2017 at 2:19 PM, Kant Kodali wrote: > My response is inline. > > On Sat, Mar 11, 2017 at 1:43 PM, Avi Kivity wrote: > >> There are several issues at play here. >> >> First, a database runs a large number of concurrent operations, each of >> which only consumes a small amount of C

Re: scylladb

There is no reason to be angry. This is progress. This is the circle of live. It happens anywhere at any time. Am 12.03.2017 07:34 schrieb "Dor Laor" : > On Sat, Mar 11, 2017 at 10:02 PM, Jeff Jirsa wrote: > >> >> >> On 2017-03-10 09:57 (-0800), Rakesh Kumar wrote: >> > Cassanda vs Scylla is a

Re: scylladb

On Sat, Mar 11, 2017 at 10:02 PM, Jeff Jirsa wrote: > > > On 2017-03-10 09:57 (-0800), Rakesh Kumar wrote: > > Cassanda vs Scylla is a valid comparison because they both are > compatible. Scylla is a drop-in replacement for Cassandra. > > No, they aren't, and no, it isn't > Jeff is angry with us

Re: scylladb

Why? Am 12.03.2017 07:02 schrieb "Jeff Jirsa" : > > > On 2017-03-10 09:57 (-0800), Rakesh Kumar wrote: > > Cassanda vs Scylla is a valid comparison because they both are > compatible. Scylla is a drop-in replacement for Cassandra. > > No, they aren't, and no, it isn't > > > > >

Re: scylladb

On 2017-03-10 09:57 (-0800), Rakesh Kumar wrote: > Cassanda vs Scylla is a valid comparison because they both are compatible. > Scylla is a drop-in replacement for Cassandra. No, they aren't, and no, it isn't

Re: scylladb

better efficiency, it sound good but can a pice of code written in C++ give >> 1000% throughput than a Java app? Is TPC design 10X more performant than >> SEDA arch? >> >> And if C/C++ is indeed that fast how can Aerospike (which is itself >> written in C) c

Re: scylladb

nt than > SEDA arch? > > And if C/C++ is indeed that fast how can Aerospike (which is itself > written in C) claim to be 10X faster than Scylla here > http://www.aerospike.com/benchmarks/scylladb-initial/ ? (Combining your's > and aerospike's benchmarks it appears that Ae

Re: scylladb

My response is inline. On Sat, Mar 11, 2017 at 1:43 PM, Avi Kivity wrote: > There are several issues at play here. > > First, a database runs a large number of concurrent operations, each of > which only consumes a small amount of CPU. The high concurrency is need to > hide latency: disk latency

Re: scylladb

There are several issues at play here. First, a database runs a large number of concurrent operations, each of which only consumes a small amount of CPU. The high concurrency is need to hide latency: disk latency, or the latency of contacting a remote node. This means that the scheduler will n

Re: scylladb

Here is the Java version http://docs.paralleluniverse.co/quasar/ but I still don't see how user level scheduling can be beneficial (This is a well debated problem)? How can this add to the performance? or say why is user level scheduling necessary Given the Thread per core design and the callback m

Re: scylladb

Scylla uses a the seastar framework, which provides for both user-level thread scheduling and simple run-to-completion tasks. Huge pages are limited to 2MB (and 1GB, but these aren't available as transparent hugepages). On 03/11/2017 10:26 PM, Kant Kodali wrote: @Dor 1) You guys have a CPU

Re: scylladb

@Dor 1) You guys have a CPU scheduler? you mean user level thread Scheduler that maps user level threads to kernel level threads? I thought C++ by default creates native kernel threads but sure nothing will stop someone to create a user level scheduling library if that's what you are talking about

Re: scylladb

Agreed, I'd recommend to treat benchmarks as a rough guide to see where there is potential, and follow through with your own tests. On 03/11/2017 09:37 PM, Edward Capriolo wrote: Benchmarks are great for FUDly blog posts. Real world work loads matter more. Every NoSQL vendor wins their benchm

Re: scylladb

Here's a test (by Samsung MSL) comparing Scylla to Cassandra 3.9: http://www.scylladb.com/2017/02/15/scylladb-vs-cassandra-performance-benchmark-samsung/ there's a link at the end to the original report. On 03/11/2017 09:08 PM, Bhuvan Rawal wrote: "Lastly, why don't you

Re: scylladb

hem in detail. And no, you can't multiply results like that unless they >> were done with very similar configurations and test harnesses. >> >> Lastly, why don't you test Scylla yourself? It's pretty easy to set up, >> there's nothing to tune. >> >&g

Re: scylladb

ng to comment about the Aerospike numbers, I haven't studied > them in detail. And no, you can't multiply results like that unless they > were done with very similar configurations and test harnesses. > > Lastly, why don't you test Scylla yourself? It's pretty easy t

Re: scylladb

unless they were done with very similar configurations and test harnesses. Lastly, why don't you test Scylla yourself? It's pretty easy to set up, there's nothing to tune. Avi [1] https://www.infoq.com/presentations/scylladb [2] http://www.scylladb.com/technology/cassandra-

Re: scylladb

Thanks a lot for your detailed explanation! I am very curious about the future development of Scylladb! Especially about mvs and lwt! Am 11.03.2017 02:05 schrieb "Dor Laor" : > On Fri, Mar 10, 2017 at 4:45 PM, Kant Kodali wrote: > >> http://performanceterracotta.blo

Re: scylladb

to the user and are per-core (hyper thread). Such a shard access RAM only within its numa node. Memory is bonded to each thread/numa node. We have our own malloc allocator built for this scheme. > If scyllaDB has efficient Secondary indexes, LWT and MV's then that is > something.

Re: scylladb

http://performanceterracotta.blogspot.com/2012/09/numa-java.html http://docs.oracle.com/javase/7/docs/technotes/guides/vm/performance-enhancements-7.html http://openjdk.java.net/jeps/163 If scyllaDB has efficient Secondary indexes, LWT and MV's then that is something. I would be glad to se

Re: scylladb

I don’t see how C++ > changes anything in this regard unless you can cache all your data in > memory. > > > > I’d be curious to know how ScyllaDB performs with a 100+ nodes cluster > with PBs of data compared to Cassandra. > > *--* > > *Jacques-Henri Berthemet* > &g

RE: scylladb

limiting factor is your disk write speed and latency, I don’t see how C++ changes anything in this regard unless you can cache all your data in memory. I’d be curious to know how ScyllaDB performs with a 100+ nodes cluster with PBs of data compared to Cassandra. -- Jacques-Henri Berthemet From

Re: scylladb

To: user@cassandra.apache.org Sent: Friday, March 10, 2017 11:59 AM Subject: Re: scylladb Agreed C++ gives an added advantage to talk to underlying hardware with better efficiency, it sound good but can a pice of code written in C++ give 1000% throughput than a Java app? Is TPC design 10X more

Re: scylladb

itself written in C) claim to be 10X faster than Scylla here http://www.aerospike.com/benchmarks/scylladb-initial/ ? (Combining your's and aerospike's benchmarks it appears that Aerospike is 100X performant than C* - I highly doubt that!! ) For a moment lets forget about evaluating 2

Re: scylladb

ScyllaDB engineer here. C++ is really an enabling technology here. It is directly responsible for a small fraction of the gain by executing faster than Java. But it is indirectly responsible for the gain by allowing us direct control over memory and threading. Just as an example, Scylla

Re: scylladb

I'd say the benchmark would be complete only when at the point of inflexion the necessary system benchmarks are provided. Looking at scylladb report it is unclear as to what system parameter was being the bottleneck. Also an observation - its mentioned in the report that they are using 1KB r

Re: scylladb

I dont think ScyllaDB performance is because of C++. The design decisions in scylladb are indeed different from Cassandra such as getting rid of SEDA and moving to TPC and so on. If someone thinks it is because of C++ then just show the benchmarks that proves it is indeed the C++ which gave 10X

Re: scylladb

They spend an enormous amount of time focusing on performance. You can expect them to continue on with their optimization and keep crushing it. P.S., I don't work for ScyllaDB. On Thu, Mar 9, 2017 at 6:02 PM, Rakesh Kumar wrote: > In all of their presentation they keep harping on the f

Re: scylladb

In all of their presentation they keep harping on the fact that scylladb is written in C++ and does not carry the overhead of Java. Still the difference looks staggering. From: daemeon reiydelle Sent: Thursday, March 9, 2017 14:21 To: user

Re: scylladb

I just did a Twitter search on scylladb and did not see any tweets about actual use, so far. -- Jack Krupansky On Wed, Nov 11, 2015 at 10:54 AM, Carlos Alonso wrote: > Any update about this? > > @Carlos Rolo, did you tried it? Thoughts? > > Carlos Alonso | Software Engineer |

Re: scylladb

Sure! I have a lot of blog post on backlog to blog asap about this, otherwise I would only share results mid 2016 :P Regards, Carlos Juzarte Rolo Cassandra Consultant Pythian - Love your data rolo@pythian | Twitter: @cjrolo | Linkedin: *linkedin.com/in/carlosjuzarterolo

Re: scylladb

Killer, @cjrolo. Will you update via this thread? On Wed, Nov 11, 2015 at 7:57 AM, Carlos Rolo wrote: > Not yet, but not far from doing it. No rain here yet! :) > > On a more serious tone, should be done before end of the Month. > > -- > > > > -- [image: datastax_logo.png]

Re: scylladb

Not yet, but not far from doing it. No rain here yet! :) On a more serious tone, should be done before end of the Month. -- --

Re: scylladb

www.pythian.com > > On Thu, Nov 5, 2015 at 12:07 PM, Dani Traphagen < > dani.trapha...@datastax.com> wrote: > >> As of two days ago, they say they've got it @cjrolo. >> >> https://github.com/scylladb/scylla/wiki/RELEASE-Scylla-0.11-Beta >> >> &

Re: scylladb

891 81 00 | Tel: +1 613 565 8696 x1649 www.pythian.com On Thu, Nov 5, 2015 at 12:07 PM, Dani Traphagen wrote: > As of two days ago, they say they've got it @cjrolo. > > https://github.com/scylladb/scylla/wiki/RELEASE-Scylla-0.11-Beta > > > On Thursday, November 5, 2015, Ca

Re: scylladb

As of two days ago, they say they've got it @cjrolo. https://github.com/scylladb/scylla/wiki/RELEASE-Scylla-0.11-Beta On Thursday, November 5, 2015, Carlos Rolo wrote: > I will not try until multi-DC is implemented. More than an month has > passed since I looked for it, so it could

Re: scylladb

k. > > > On Nov 5, 2015, at 9:22 AM, tommaso barbugli > wrote: > > > > Hi guys, > > > > did anyone already try Scylladb (yet another fastest NoSQL database in > town) and has some thoughts/hands-on experience to share? > > > > Cheers, > > Tommaso > > -- --

Re: scylladb

Nope, no one I know. Let me know if you try it I'd love to hear your feedback. > On Nov 5, 2015, at 9:22 AM, tommaso barbugli wrote: > > Hi guys, > > did anyone already try Scylladb (yet another fastest NoSQL database in town) > and has some thoughts/hands-on experienc

scylladb

Hi guys, did anyone already try Scylladb (yet another fastest NoSQL database in town) and has some thoughts/hands-on experience to share? Cheers, Tommaso

Re: ScyllaDB, a new open source, Cassandra-compatible NoSQL

I think there is a very important point in Scylladb - latency. Performance can be an important requirement, but the fact scylladb is written in C and uses lock free algorithms inside means it should have lower latency than Cassandra, which enables it's use for a wider range of applications

Re: ScyllaDB, a new open source, Cassandra-compatible NoSQL

Looking at the architecture and what scylladb does, I'm not surprised they got 10x improvement. SeaStar skips a lot of the overhead of copying stuff and it gives them CPU core affinity. Anyone that's listened to Clif Click talk about cache misses, locks and other low level stuff would

Re: ScyllaDB, a new open source, Cassandra-compatible NoSQL

On Wed, Sep 23, 2015 at 12:20 AM, Minh Do wrote: > First glance at their github, it looks like they re-implemented Cassandra > in C++. 90% components in Cassandra are > in scylladb, i.e. compaction, repair, CQL, gossip, SStable. > True > > > With C++, I believe this help

Re: ScyllaDB, a new open source, Cassandra-compatible NoSQL

very interesting. I'm glad to see someone building a drop in replacement for Cassandra. On Tue, Sep 22, 2015 at 5:40 PM, Tzach Livyatan wrote: > Hi Sachin > > On Tue, Sep 22, 2015 at 11:40 PM, Sachin Nikam wrote: > >> Tzach, >> Can you point to any documentation

Re: ScyllaDB, a new open source, Cassandra-compatible NoSQL

Hi Sachin On Tue, Sep 22, 2015 at 11:40 PM, Sachin Nikam wrote: > Tzach, > Can you point to any documentation on scylladb site which talks about > how/why scylla db performs better than Cassandra while using the same > architecture? > see here http://www.scylladb.com/technolo

Re: ScyllaDB, a new open source, Cassandra-compatible NoSQL

On Wed, Sep 23, 2015 at 12:13 AM, Venkatesh Arivazhagan < venkey.a...@gmail.com> wrote: > I came across this article: > zdnet.com/article/kvm-creators-open-source-fast-cassandra-drop-in-replacement-scylla/ > > Tzach, I would love to know/understand moree about ScyllaDB too. Al

Re: ScyllaDB, a new open source, Cassandra-compatible NoSQL

First glance at their github, it looks like they re-implemented Cassandra in C++. 90% components in Cassandra are in scylladb, i.e. compaction, repair, CQL, gossip, SStable. With C++, I believe this helps performance to some extent up to a point when compaction has not run yet. Then, it will be

Re: ScyllaDB, a new open source, Cassandra-compatible NoSQL

I came across this article: zdnet.com/article/kvm-creators-open-source-fast-cassandra-drop-in-replacement-scylla/ Tzach, I would love to know/understand moree about ScyllaDB too. Also the benchmark seems to have only 1 DB Server. Do you have benchmark numbers where more than 1 DB servers were

Re: ScyllaDB, a new open source, Cassandra-compatible NoSQL

Tzach, Can you point to any documentation on scylladb site which talks about how/why scylla db performs better than Cassandra while using the same architecture? Regards Sachin On Tue, Sep 22, 2015 at 9:18 AM, Tzach Livyatan wrote: > Hello Cassandra users, > > We are pleased to annou

ScyllaDB, a new open source, Cassandra-compatible NoSQL

Hello Cassandra users, We are pleased to announce a new member of the Cassandra Ecosystem - ScyllaDB ScyllaDB is a new, open source, Cassandra-compatible NoSQL data store, written with the goal of delivering superior performance and consistent low latency. Today, ScyllaDB runs 1M tps per server