Re: 2.0
- world Hi Jonathan, This topic may have been discussed elsewhere, or my memory is worse off than I thought, but what is our long term vision for thrift support? Admittedly, I need to learn much more about the binary CQL protocol, and I understand Ed's concerns, as well (more acutely now) about existing installations, but we probably wouldn't have dreamt up a new client interface/protocol if we went planning, at some point, on retiring the old one. And, also, I missed the Avro debate from the past, so I'm not sure how much that affects current and future thinking. After raising the issue here on the dev list, it certainly seems like 2.0 is premature for a full-on switch over, and Ed raised some interesting metrics to consider when we could declare the CQL protocol as 'accepted'. I'm curious as to how you are seeing it roll out. Thanks for your time, -Jason On Fri, Nov 30, 2012 at 2:49 PM, Jonathan Ellis jbel...@gmail.com wrote: As attractive as it would be to clean house, I think we owe it to our users to keep Thrift around for the forseeable future rather than orphan all Thrift-using applications (which is virtually everyone) on 1.2. On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com wrote: Hi Jonathan, I'm in favor of paying off the technical debt, as well, and I wonder if there is value in removing support for thrift with 2.0? We're currently in 'do as little as possible' mode with thrift, so should we aggressively cast it off and push the binary CQL protocol? Seems like a jump to '2.0', along with the other initiatives, would be a reasonable time/milestone to do so. Thanks, -Jason On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote: The more I think about it, the more I think we should call 1.2-next, 2.0. I'd like to spend some time paying off our technical debt: - replace supercolumns with composites (CASSANDRA-3237) - rewrite counters (CASSANDRA-4775) - improve storage engine support for wide rows - better stage management to improve latency (disruptor? lightweight threads? custom executor + queue?) - improved repair (CASSANDRA-3362, 2699) Of course, we're planning some new features as well: - triggers (CASSANDRA-1311) - improved query fault tolerance (CASSANDRA-4705) - row size limits (CASSANDRA-3929) - cql3 integration for hadoop (CASSANDRA-4421) - improved caching (CASSANDRA-1956, 2864) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced
Re: 2.0
Oops, meant to address this specifically to Jonathan, but since I've confused 'reply' with 'forward'. my apologies for any extra noise on this topic. On Mon, Dec 3, 2012 at 9:25 AM, Jason Brown jasedbr...@gmail.com wrote: - world Hi Jonathan, This topic may have been discussed elsewhere, or my memory is worse off than I thought, but what is our long term vision for thrift support? Admittedly, I need to learn much more about the binary CQL protocol, and I understand Ed's concerns, as well (more acutely now) about existing installations, but we probably wouldn't have dreamt up a new client interface/protocol if we went planning, at some point, on retiring the old one. And, also, I missed the Avro debate from the past, so I'm not sure how much that affects current and future thinking. After raising the issue here on the dev list, it certainly seems like 2.0 is premature for a full-on switch over, and Ed raised some interesting metrics to consider when we could declare the CQL protocol as 'accepted'. I'm curious as to how you are seeing it roll out. Thanks for your time, -Jason On Fri, Nov 30, 2012 at 2:49 PM, Jonathan Ellis jbel...@gmail.com wrote: As attractive as it would be to clean house, I think we owe it to our users to keep Thrift around for the forseeable future rather than orphan all Thrift-using applications (which is virtually everyone) on 1.2. On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com wrote: Hi Jonathan, I'm in favor of paying off the technical debt, as well, and I wonder if there is value in removing support for thrift with 2.0? We're currently in 'do as little as possible' mode with thrift, so should we aggressively cast it off and push the binary CQL protocol? Seems like a jump to '2.0', along with the other initiatives, would be a reasonable time/milestone to do so. Thanks, -Jason On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote: The more I think about it, the more I think we should call 1.2-next, 2.0. I'd like to spend some time paying off our technical debt: - replace supercolumns with composites (CASSANDRA-3237) - rewrite counters (CASSANDRA-4775) - improve storage engine support for wide rows - better stage management to improve latency (disruptor? lightweight threads? custom executor + queue?) - improved repair (CASSANDRA-3362, 2699) Of course, we're planning some new features as well: - triggers (CASSANDRA-1311) - improved query fault tolerance (CASSANDRA-4705) - row size limits (CASSANDRA-3929) - cql3 integration for hadoop (CASSANDRA-4421) - improved caching (CASSANDRA-1956, 2864) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced
Re: 2.0
I agree with Edward here. We use Thrift too and we haven't really found a good enough reason to move to CQL3. -- Drew On Dec 1, 2012, at 10:24 AM, Edward Capriolo edlinuxg...@gmail.com wrote: I do not understand why everyone wants to force this issue on removing thrift. If cql, cql sparse tables and the new transport are better people will naturally begin to use them, but as it stands now I see the it this way: Thrift still has more clients for more languages, thrift has more higher level clients for more languages. Thrift has Hadoop support hive support and pig support in the wild. Thrift has third party tools like Orm tools, support for tools like flume. Most of cql3 features like collections do not work with compact tables, and compact tables are much more space efficient then their cql3 sparse counterparts, composite rows with UTf column names, blank rows, etc. Cql3 binary client is only available for in beta stage for a few languages. So the project can easily remove thrift today but until a majority of the tooling by the community adopts the transport and for the most part cqls sparse tables it is not going to mean anything. Many people already have code live in production working fine with the old toolset and will be unwilling to convert something just because Think about it like this a company like mine that already has something in production. Even if you could convince me us that cql native transport was better, which by the way no one has showed me a vast performance reason to this point, they still may not want to invest the resources to convert their app. Many companies endured the painful transition from Cassandra 0.6 to Cassandra 0.7 conversion and they are not eagerly going to entertain another change which is mostly cosmetic. Also I find issues like this extremely frustrating. https://issues.apache.org/jira/browse/CASSANDRA-4924 It seems like the project is drawing a hard line in the sand dividing people. Is it the case that cql3's sparse tables can't be accessed by thrift, or is it the case that no one wants to make this happen? Like is it technically impossible? It seems not to me in Cassandra Row key, column, and value are all still byte arrays right? So I do not see why thrift users need to be locked out of them. Just like composites we will figure out how to pack the bytes. I hope that we can stop talking about removing thrift until there is some consensus between active users that it is not in use anymore. This consensus is not as simple as n committers saying that something is technically not needed anymore. It has to look at the users, the number of clients, the number of languages, the number of high level tools available. In the mean time when issues like 4924 pop up it would be better if people tried to find solutions for maximum forward and backward compatibility instead of drawing a line and trying to shut thrift users out of things. Avro was much the same way . I had a spirited debate on irc and got basicallly insulted because i belived thrift was not dead. The glory of avro never came true because it really did not work for clients outside a few languages. Cql and the binary transport has to pass this same litmus test. Let it gain momentum and have rock solid clients for 5 languages and have higher level tools written on top of it then its easy to say thrift is not needed anymore. On Saturday, December 1, 2012, Sylvain Lebresne wrote: I agree on 2.0. For the thrift part, we've said clearly that we wouldn't remove it any time soon so let's stick to that. Besides, I would agree it's too soon anyway. What we can do however in the relatively short term on that front, is to pull thrift in it's own jar (we've almost removed all internal dependencies on thrift, and the few remaining ones will be easy to kill) and make that jar optional if you don't want to use it. -- Sylvain On Sat, Dec 1, 2012 at 2:52 AM, Ray Slakinski ray.slakin...@gmail.comjavascript:; wrote: I agree, I don't think its a great idea to drop thrift until the back end tools are 100% compatible and have some level of agreement from the major users of Cassandra. Paying off technical dept though I'm all for, and I think its key to the long term success of the application. Right now Supercolumns to someone new coming to the system might think Hey, these things look great. Lets use them and in a few months time hate all things that are cassandra. Ray Slakinski On 12/01, Jonathan Ellis wrote: As attractive as it would be to clean house, I think we owe it to our users to keep Thrift around for the forseeable future rather than orphan all Thrift-using applications (which is virtually everyone) on 1.2. On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.comjavascript:; wrote: Hi Jonathan, I'm in favor of paying off the technical debt, as well, and I wonder if there is value in
Re: 2.0
On Sat, Dec 1, 2012 at 12:24 PM, Edward Capriolo edlinuxg...@gmail.com wrote: I do not understand why everyone wants to force this issue on removing thrift. I'm -1 on removing thrift, and by my count, that would put us at -3 binding if it ever came to vote, so let's consider this proposition closed and move on to other more constructive points. -Brandon
Re: 2.0
I agree on 2.0. For the thrift part, we've said clearly that we wouldn't remove it any time soon so let's stick to that. Besides, I would agree it's too soon anyway. What we can do however in the relatively short term on that front, is to pull thrift in it's own jar (we've almost removed all internal dependencies on thrift, and the few remaining ones will be easy to kill) and make that jar optional if you don't want to use it. -- Sylvain On Sat, Dec 1, 2012 at 2:52 AM, Ray Slakinski ray.slakin...@gmail.comwrote: I agree, I don't think its a great idea to drop thrift until the back end tools are 100% compatible and have some level of agreement from the major users of Cassandra. Paying off technical dept though I'm all for, and I think its key to the long term success of the application. Right now Supercolumns to someone new coming to the system might think Hey, these things look great. Lets use them and in a few months time hate all things that are cassandra. Ray Slakinski On 12/01, Jonathan Ellis wrote: As attractive as it would be to clean house, I think we owe it to our users to keep Thrift around for the forseeable future rather than orphan all Thrift-using applications (which is virtually everyone) on 1.2. On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com wrote: Hi Jonathan, I'm in favor of paying off the technical debt, as well, and I wonder if there is value in removing support for thrift with 2.0? We're currently in 'do as little as possible' mode with thrift, so should we aggressively cast it off and push the binary CQL protocol? Seems like a jump to '2.0', along with the other initiatives, would be a reasonable time/milestone to do so. Thanks, -Jason On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote: The more I think about it, the more I think we should call 1.2-next, 2.0. I'd like to spend some time paying off our technical debt: - replace supercolumns with composites (CASSANDRA-3237) - rewrite counters (CASSANDRA-4775) - improve storage engine support for wide rows - better stage management to improve latency (disruptor? lightweight threads? custom executor + queue?) - improved repair (CASSANDRA-3362, 2699) Of course, we're planning some new features as well: - triggers (CASSANDRA-1311) - improved query fault tolerance (CASSANDRA-4705) - row size limits (CASSANDRA-3929) - cql3 integration for hadoop (CASSANDRA-4421) - improved caching (CASSANDRA-1956, 2864) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced
Re: 2.0
Hi Jonathan, I'm in favor of paying off the technical debt, as well, and I wonder if there is value in removing support for thrift with 2.0? We're currently in 'do as little as possible' mode with thrift, so should we aggressively cast it off and push the binary CQL protocol? Seems like a jump to '2.0', along with the other initiatives, would be a reasonable time/milestone to do so. Thanks, -Jason On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote: The more I think about it, the more I think we should call 1.2-next, 2.0. I'd like to spend some time paying off our technical debt: - replace supercolumns with composites (CASSANDRA-3237) - rewrite counters (CASSANDRA-4775) - improve storage engine support for wide rows - better stage management to improve latency (disruptor? lightweight threads? custom executor + queue?) - improved repair (CASSANDRA-3362, 2699) Of course, we're planning some new features as well: - triggers (CASSANDRA-1311) - improved query fault tolerance (CASSANDRA-4705) - row size limits (CASSANDRA-3929) - cql3 integration for hadoop (CASSANDRA-4421) - improved caching (CASSANDRA-1956, 2864) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced
Re: 2.0
Good idea. Lets remove thrift, CQL3 is still beta, but I am willing to upgrade to a version that removes thrift. Then when all our clients can not connect they will be forced to get with the program. On Fri, Nov 30, 2012 at 5:33 PM, Jason Brown jasedbr...@gmail.com wrote: Hi Jonathan, I'm in favor of paying off the technical debt, as well, and I wonder if there is value in removing support for thrift with 2.0? We're currently in 'do as little as possible' mode with thrift, so should we aggressively cast it off and push the binary CQL protocol? Seems like a jump to '2.0', along with the other initiatives, would be a reasonable time/milestone to do so. Thanks, -Jason On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote: The more I think about it, the more I think we should call 1.2-next, 2.0. I'd like to spend some time paying off our technical debt: - replace supercolumns with composites (CASSANDRA-3237) - rewrite counters (CASSANDRA-4775) - improve storage engine support for wide rows - better stage management to improve latency (disruptor? lightweight threads? custom executor + queue?) - improved repair (CASSANDRA-3362, 2699) Of course, we're planning some new features as well: - triggers (CASSANDRA-1311) - improved query fault tolerance (CASSANDRA-4705) - row size limits (CASSANDRA-3929) - cql3 integration for hadoop (CASSANDRA-4421) - improved caching (CASSANDRA-1956, 2864) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced
Re: 2.0
As attractive as it would be to clean house, I think we owe it to our users to keep Thrift around for the forseeable future rather than orphan all Thrift-using applications (which is virtually everyone) on 1.2. On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com wrote: Hi Jonathan, I'm in favor of paying off the technical debt, as well, and I wonder if there is value in removing support for thrift with 2.0? We're currently in 'do as little as possible' mode with thrift, so should we aggressively cast it off and push the binary CQL protocol? Seems like a jump to '2.0', along with the other initiatives, would be a reasonable time/milestone to do so. Thanks, -Jason On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote: The more I think about it, the more I think we should call 1.2-next, 2.0. I'd like to spend some time paying off our technical debt: - replace supercolumns with composites (CASSANDRA-3237) - rewrite counters (CASSANDRA-4775) - improve storage engine support for wide rows - better stage management to improve latency (disruptor? lightweight threads? custom executor + queue?) - improved repair (CASSANDRA-3362, 2699) Of course, we're planning some new features as well: - triggers (CASSANDRA-1311) - improved query fault tolerance (CASSANDRA-4705) - row size limits (CASSANDRA-3929) - cql3 integration for hadoop (CASSANDRA-4421) - improved caching (CASSANDRA-1956, 2864) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced
Re: 2.0
My hope is that after 1.2 (i.e. by the time we're 2.0'ing), the binary CQL protocol is out of beta :). On Fri, Nov 30, 2012 at 2:39 PM, Edward Capriolo edlinuxg...@gmail.comwrote: Good idea. Lets remove thrift, CQL3 is still beta, but I am willing to upgrade to a version that removes thrift. Then when all our clients can not connect they will be forced to get with the program
Re: 2.0
+1 Regards, /VJ On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote: The more I think about it, the more I think we should call 1.2-next, 2.0. I'd like to spend some time paying off our technical debt: - replace supercolumns with composites (CASSANDRA-3237) - rewrite counters (CASSANDRA-4775) - improve storage engine support for wide rows - better stage management to improve latency (disruptor? lightweight threads? custom executor + queue?) - improved repair (CASSANDRA-3362, 2699) Of course, we're planning some new features as well: - triggers (CASSANDRA-1311) - improved query fault tolerance (CASSANDRA-4705) - row size limits (CASSANDRA-3929) - cql3 integration for hadoop (CASSANDRA-4421) - improved caching (CASSANDRA-1956, 2864) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced
Re: 2.0
I'm not thrilled with Thrift, but I'd like to see and hear more real world use of CQL first (Avro all the things is not that long ago). That said, a major rev /could/ do this - not start the thrift server by default. It's then a hoop jump to enable it via nodetool/yaml, and signals to the client communities and user base, where the project intends to go. Bill On 30/11/12 22:49, Jonathan Ellis wrote: As attractive as it would be to clean house, I think we owe it to our users to keep Thrift around for the forseeable future rather than orphan all Thrift-using applications (which is virtually everyone) on 1.2. On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com wrote: Hi Jonathan, I'm in favor of paying off the technical debt, as well, and I wonder if there is value in removing support for thrift with 2.0? We're currently in 'do as little as possible' mode with thrift, so should we aggressively cast it off and push the binary CQL protocol? Seems like a jump to '2.0', along with the other initiatives, would be a reasonable time/milestone to do so. Thanks, -Jason On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote: The more I think about it, the more I think we should call 1.2-next, 2.0. I'd like to spend some time paying off our technical debt: - replace supercolumns with composites (CASSANDRA-3237) - rewrite counters (CASSANDRA-4775) - improve storage engine support for wide rows - better stage management to improve latency (disruptor? lightweight threads? custom executor + queue?) - improved repair (CASSANDRA-3362, 2699) Of course, we're planning some new features as well: - triggers (CASSANDRA-1311) - improved query fault tolerance (CASSANDRA-4705) - row size limits (CASSANDRA-3929) - cql3 integration for hadoop (CASSANDRA-4421) - improved caching (CASSANDRA-1956, 2864) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced
Re: 2.0
I agree, I don't think its a great idea to drop thrift until the back end tools are 100% compatible and have some level of agreement from the major users of Cassandra. Paying off technical dept though I'm all for, and I think its key to the long term success of the application. Right now Supercolumns to someone new coming to the system might think Hey, these things look great. Lets use them and in a few months time hate all things that are cassandra. Ray Slakinski On 12/01, Jonathan Ellis wrote: As attractive as it would be to clean house, I think we owe it to our users to keep Thrift around for the forseeable future rather than orphan all Thrift-using applications (which is virtually everyone) on 1.2. On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown jasedbr...@gmail.com wrote: Hi Jonathan, I'm in favor of paying off the technical debt, as well, and I wonder if there is value in removing support for thrift with 2.0? We're currently in 'do as little as possible' mode with thrift, so should we aggressively cast it off and push the binary CQL protocol? Seems like a jump to '2.0', along with the other initiatives, would be a reasonable time/milestone to do so. Thanks, -Jason On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis jbel...@gmail.com wrote: The more I think about it, the more I think we should call 1.2-next, 2.0. I'd like to spend some time paying off our technical debt: - replace supercolumns with composites (CASSANDRA-3237) - rewrite counters (CASSANDRA-4775) - improve storage engine support for wide rows - better stage management to improve latency (disruptor? lightweight threads? custom executor + queue?) - improved repair (CASSANDRA-3362, 2699) Of course, we're planning some new features as well: - triggers (CASSANDRA-1311) - improved query fault tolerance (CASSANDRA-4705) - row size limits (CASSANDRA-3929) - cql3 integration for hadoop (CASSANDRA-4421) - improved caching (CASSANDRA-1956, 2864) -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced -- Jonathan Ellis Project Chair, Apache Cassandra co-founder, http://www.datastax.com @spyced
