Re: Rough roadmap for 4.0
List looks really good. I will let you know if there is something else we plan to add to this list. On Thu, Nov 3, 2016 at 7:47 PM, Nate McCall wrote: > It was brought up recently at the PMC level that our goals as a > project are not terribly clear. > > This is a pretty good point as outside of Jira 'Fix Version' labelling > (which we actually suck less at compared to a lot of other ASF > projects) this really isnt tracked anywhere outside of general tribal > knowledge about who is working on what. > > I would like to see us change this for two reasons: > - it's important we are clear with our community about where we are going > - we need to start working more closely together > > To that end, i've put together a list (in no particular order) of the > *major* features in which I know folks are interested, have patches > coming, are awaiting design review, etc.: > > - CASSANDRA-9425 Immutable node-local schema > - CASSANDRA-10699 Strongly consistent schema alterations > - CASSANDRA-12229 NIO streaming > - CASSANDRA-8457 NIO messaging > - CASSANDRA-12345 Gossip 2.0 > - CASSANDRA-9754 Birch trees > > What did I miss? What else would folks like to see? Specifically, this > should be "new stuff that could/will break things" given we are upping > the major version. > > To be clear, it's not my intention to set this in stone and then beat > people about the head with it. More to have it there to point it at a > high level and foster better communication with our users from the > perspective of an open source project. > > Please keep in mind that given everything else going on, I think it's > a fantastic idea to keep this list small and spend some time focusing > on stability particularly as we transition to a new release process. > > -Nate >
Rough roadmap for 4.0
It was brought up recently at the PMC level that our goals as a project are not terribly clear. This is a pretty good point as outside of Jira 'Fix Version' labelling (which we actually suck less at compared to a lot of other ASF projects) this really isnt tracked anywhere outside of general tribal knowledge about who is working on what. I would like to see us change this for two reasons: - it's important we are clear with our community about where we are going - we need to start working more closely together To that end, i've put together a list (in no particular order) of the *major* features in which I know folks are interested, have patches coming, are awaiting design review, etc.: - CASSANDRA-9425 Immutable node-local schema - CASSANDRA-10699 Strongly consistent schema alterations - CASSANDRA-12229 NIO streaming - CASSANDRA-8457 NIO messaging - CASSANDRA-12345 Gossip 2.0 - CASSANDRA-9754 Birch trees What did I miss? What else would folks like to see? Specifically, this should be "new stuff that could/will break things" given we are upping the major version. To be clear, it's not my intention to set this in stone and then beat people about the head with it. More to have it there to point it at a high level and foster better communication with our users from the perspective of an open source project. Please keep in mind that given everything else going on, I think it's a fantastic idea to keep this list small and spend some time focusing on stability particularly as we transition to a new release process. -Nate
Re: Bug in paging reads and limit reads (short reads!)
Sure! I will move further questions to JIRA. Thanks On Thu, Nov 3, 2016 at 4:34 AM, Benjamin Lerer wrote: > I really have to look at the code of the different versions first to answer > your questions. > I will put as much as information as I can in the ticket but feel free to > ask more questions in the ticket if you need to. > > Benjamin > > On Wed, Nov 2, 2016 at 10:58 PM, Bhaskar Muppana > wrote: > > > Thanks Benjamin! I am also hoping to ask couple of questions on > difference > > in this code base between 2.0, 2.1 and 3.0. I understand 2.0 is closed > from > > community point of view. But, we are still using in some cases, so we > need > > to fix it internally. > > > > With my limited understanding of the code, it looks like to me there is > no > > protection against short reads in 2.0. But in 3.0, we seem to have some > > short read transformation. At least, from comments it feels like that > code > > should have taken care of cases like this. So this must be a bug in 3.0 > > code, where as in 2.0 the complete feature for short read protection is > > missing. Is my understanding right? Can you please shed more light on > this? > > > > I have created a JIRA as you asked - > > https://issues.apache.org/jira/browse/CASSANDRA-12872 > > > > This is my first JIRA, so I don't know conventions to set priority. As > its > > data inconsistency issue, I believe this is critical. Feel free to change > > the priority according to community conventions. > > > > Thanks, > > Bhaskar > > > > On Wed, Nov 2, 2016 at 1:26 AM, Benjamin Lerer < > > benjamin.le...@datastax.com> > > wrote: > > > > > Hi Bhaskar, > > > > > > Thanks for reporting that problem. It is a nice catch :-) > > > > > > Could you open a JIRA ticket with all the information that you > provided? > > > > > > I will try to fix that problem. > > > > > > Benjamin > > > > > > > > > On Wed, Nov 2, 2016 at 12:00 AM, Bhaskar Muppana > > > > wrote: > > > > > > > Hi Guys, > > > > > > > > We are seeing an issue with paging reads missing some small number of > > > > columns when we do paging/limit reads. We get this on a single DC > > cluster > > > > itself when both reads and writes are happening with QUORUM. > > Paging/limit > > > > reads see this issue. I have attached the ccm based script which > > > reproduces > > > > the problem. > > > > > > > > * Keyspace RF - 2 > > > > * Table (id int, course text, marks int, primary key(id, course)) > > > > * replicas for partition key 1 - r1, r2 and r3 > > > > * insert (1, '1', 1) , (1, '2', 2), (1, '3', 3), (1, '4', 4), (1, > > > '5', > > > > 5) - succeeded on all 3 replicas > > > > * insert (1, '6', 6) succeeded on r1 and r3, failed on r2 > > > > * delete (1, '2'), (1, '3'), (1, '4'), (1, '5') succeeded on r1 and > r2, > > > > failed on r3 > > > > * insert (1, '7', 7) succeeded on r1 and r2, failed on r3 > > > > > > > > Local data on 3 nodes looks like as below now > > > > > > > > r1: (1, '1', 1), tombstone(2-5 records), (1, '6', 6), (1, '7', 7) > > > > r2: (1, '1', 1), tombstone(2-5 records), (1, '7', 7) > > > > r3: (1, '1', 1), (1, '2', 2), (1, '3', 3), (1, '4', 4), (1, '5', > > > > 5), (1, '6', 6) > > > > > > > > If we do a paging read with page_size 2, and if it gets data from r2 > > and > > > > r3, then it will only get the data (1, '1', 1) and (1, '7', 7) > skipping > > > > record 6. This problem would happen if the same query is not doing > > paging > > > > but limit set to 2 records. > > > > > > > > Resolution code for reads works same for paging queries and normal > > > > queries. Co-ordinator shouldn't respond back to client with > > > records/columns > > > > that it didn't have complete visibility on all required replicas (in > > this > > > > case 2 replicas). In above case, it is sending back record (1, '7', > 7) > > > back > > > > to client, but its visibility on r3 is limited up to (1, '2', 2) and > it > > > is > > > > relying on just r2 data to assume (1, '6', 6) doesn't exist, which is > > > > wrong. End of the resolution all it can conclusively say any thing > > about > > > is > > > > (1, '1', 1), which exists and (1, '2', 2), which is deleted. > > > > > > > > Ideally we should have different resolution implementation for > > > > paging/limit queries. > > > > > > > > We could reproduce this on 2.0.17, 2.1.16 and 3.0.9. > > > > > > > > Seems like 3.0.9 we have ShortReadProtection transformation on list > > > > queries. I assume that is to protect against the cases like above. > But, > > > we > > > > can reproduce the issue in 3.0.9 as well. > > > > > > > > Thanks, > > > > Bhaskar > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [VOTE FAILED] Release Apache Cassandra 3.10
I am changing my vote to -1 and closing this vote for the regression reported in CASSANDRA-12867 by Kurt Greaves. For the record, I count 2 binding (1 non-binding) +1 votes and 3 binding (1 non-binding, counting Kurt) -1 votes. I've marked CASSANDRA-12867 as Critical with fixver=3.10,3.0.10, so help getting that reviewed and committed would be great! I'll rebuild another release set when that is completed. -- Kind regards, Michael On 10/31/2016 10:18 AM, Michael Shuler wrote: > I propose the following artifacts for release as 3.10. > > sha1: a3828ca8b755fc98799867baf07039f7ff53be05 > Git: > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.10-tentative > Artifacts: > https://repository.apache.org/content/repositories/orgapachecassandra-1130/org/apache/cassandra/apache-cassandra/3.10/ > Staging repository: > https://repository.apache.org/content/repositories/orgapachecassandra-1130/ > > The Debian packages are available here: http://people.apache.org/~mshuler > > The vote will be open for 72 hours (longer if needed). > > [1]: (CHANGES.txt) https://goo.gl/8uMiBt > [2]: (NEWS.txt) https://goo.gl/prKdlZ >
Re: [VOTE FAILED] Release Apache Cassandra 3.0.10
I am changing my vote to -1 and closing this vote for the regression reported in CASSANDRA-12867 by Kurt Greaves, which also affects 3.0.10. For the record, I count 3 binding +1 votes and 3 binding -1 votes. -- Kind regards, Michael On 10/31/2016 11:12 AM, Michael Shuler wrote: > I propose the following artifacts for release as 3.0.10. > > sha1: 817ba038783212b716f6981b26c8348ffdc92f59 > Git: > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.0.10-tentative > Artifacts: > https://repository.apache.org/content/repositories/orgapachecassandra-1131/org/apache/cassandra/apache-cassandra/3.0.10/ > Staging repository: > https://repository.apache.org/content/repositories/orgapachecassandra-1131/ > > The Debian packages are available here: http://people.apache.org/~mshuler > > The vote will be open for 72 hours (longer if needed). > > [1]: (CHANGES.txt) https://goo.gl/4e1P3e > [2]: (NEWS.txt) https://goo.gl/AwLFr1 >
Re: [VOTE] Release Apache Cassandra 3.10
-1, regression On 2016-10-31 08:18 (-0700), Michael Shuler wrote: > I propose the following artifacts for release as 3.10. > > sha1: a3828ca8b755fc98799867baf07039f7ff53be05 > Git: > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.10-tentative > Artifacts: > https://repository.apache.org/content/repositories/orgapachecassandra-1130/org/apache/cassandra/apache-cassandra/3.10/ > Staging repository: > https://repository.apache.org/content/repositories/orgapachecassandra-1130/ > > The Debian packages are available here: http://people.apache.org/~mshuler > > The vote will be open for 72 hours (longer if needed). > > [1]: (CHANGES.txt) https://goo.gl/8uMiBt > [2]: (NEWS.txt) https://goo.gl/prKdlZ >
Re: [VOTE] Release Apache Cassandra 3.0.10
-1 , regression On 11/2/16, 1:52 PM, "Nate McCall" wrote: >[Copied from 3.10 thread] >Sorry all. Changing my vote to -1 per my comment: >https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_CASSANDRA-2D12867-3FfocusedCommentId-3D15630442-26page-3Dcom.atlassian.jira.plugin.system.issuetabpanels-3Acomment-2Dtabpanel-23comment-2D15630442&d=DQIBaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=5qTCPBUoKAfdyok_l_vNigJdlnQ7nGFEUFa3tD1vpZ8&s=FYE_fyQNYhT49ZmXtG7uD8OdZu0jK4QmGKd7ekXYHN0&e= > > >This pattern is more common in the wild than you might think. > >On Thu, Nov 3, 2016 at 2:03 AM, Sylvain Lebresne wrote: >> +1 >> >> On Mon, Oct 31, 2016 at 6:29 PM, Nate McCall wrote: >> >>> +1 >>> >>> On Tue, Nov 1, 2016 at 3:12 AM, Michael Shuler >>> wrote: >>> > I propose the following artifacts for release as 3.0.10. >>> > >>> > sha1: 817ba038783212b716f6981b26c8348ffdc92f59 >>> > Git: >>> > https://urldefense.proofpoint.com/v2/url?u=http-3A__git-2Dwip-2Dus.apache.org_repos_asf-3Fp-3Dcassandra.git-3Ba-3D&d=DQIBaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=5qTCPBUoKAfdyok_l_vNigJdlnQ7nGFEUFa3tD1vpZ8&s=dOHUsh-e39QqTZf4VqN62_A5sZP6Kveg4uU2hyC9HEA&e= >>> > >>> shortlog;h=refs/tags/3.0.10-tentative >>> > Artifacts: >>> > https://urldefense.proofpoint.com/v2/url?u=https-3A__repository.apache.org_content_repositories_&d=DQIBaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=5qTCPBUoKAfdyok_l_vNigJdlnQ7nGFEUFa3tD1vpZ8&s=Jl2Iqfyc-VgJCBmYZr94CgwwKt1Br6IVB43CzxNvrdo&e= >>> > >>> orgapachecassandra-1131/org/apache/cassandra/apache-cassandra/3.0.10/ >>> > Staging repository: >>> > https://urldefense.proofpoint.com/v2/url?u=https-3A__repository.apache.org_content_repositories_&d=DQIBaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=5qTCPBUoKAfdyok_l_vNigJdlnQ7nGFEUFa3tD1vpZ8&s=Jl2Iqfyc-VgJCBmYZr94CgwwKt1Br6IVB43CzxNvrdo&e= >>> > >>> orgapachecassandra-1131/ >>> > >>> > The Debian packages are available here: >>> > https://urldefense.proofpoint.com/v2/url?u=http-3A__people.apache.org_-7E&d=DQIBaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=5qTCPBUoKAfdyok_l_vNigJdlnQ7nGFEUFa3tD1vpZ8&s=Q_LY2aLvKArYUAlc6t1YVSp2G34RXq0idaNcR-pHy5o&e= >>> > >>> mshuler >>> > >>> > The vote will be open for 72 hours (longer if needed). >>> > >>> > [1]: (CHANGES.txt) >>> > https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_4e1P3e&d=DQIBaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=5qTCPBUoKAfdyok_l_vNigJdlnQ7nGFEUFa3tD1vpZ8&s=J6csojRJTc7AASTIGG86riwmy8YwfeOrh1_u9azW_6s&e= >>> > >>> > [2]: (NEWS.txt) >>> > https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_AwLFr1&d=DQIBaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=5qTCPBUoKAfdyok_l_vNigJdlnQ7nGFEUFa3tD1vpZ8&s=FfpburGStOOVl85ypvF104oCDC6tcr_xTpTkfrgBMEE&e= >>> > >>> smime.p7s Description: S/MIME cryptographic signature
Re: Bug in paging reads and limit reads (short reads!)
I really have to look at the code of the different versions first to answer your questions. I will put as much as information as I can in the ticket but feel free to ask more questions in the ticket if you need to. Benjamin On Wed, Nov 2, 2016 at 10:58 PM, Bhaskar Muppana wrote: > Thanks Benjamin! I am also hoping to ask couple of questions on difference > in this code base between 2.0, 2.1 and 3.0. I understand 2.0 is closed from > community point of view. But, we are still using in some cases, so we need > to fix it internally. > > With my limited understanding of the code, it looks like to me there is no > protection against short reads in 2.0. But in 3.0, we seem to have some > short read transformation. At least, from comments it feels like that code > should have taken care of cases like this. So this must be a bug in 3.0 > code, where as in 2.0 the complete feature for short read protection is > missing. Is my understanding right? Can you please shed more light on this? > > I have created a JIRA as you asked - > https://issues.apache.org/jira/browse/CASSANDRA-12872 > > This is my first JIRA, so I don't know conventions to set priority. As its > data inconsistency issue, I believe this is critical. Feel free to change > the priority according to community conventions. > > Thanks, > Bhaskar > > On Wed, Nov 2, 2016 at 1:26 AM, Benjamin Lerer < > benjamin.le...@datastax.com> > wrote: > > > Hi Bhaskar, > > > > Thanks for reporting that problem. It is a nice catch :-) > > > > Could you open a JIRA ticket with all the information that you provided? > > > > I will try to fix that problem. > > > > Benjamin > > > > > > On Wed, Nov 2, 2016 at 12:00 AM, Bhaskar Muppana > > wrote: > > > > > Hi Guys, > > > > > > We are seeing an issue with paging reads missing some small number of > > > columns when we do paging/limit reads. We get this on a single DC > cluster > > > itself when both reads and writes are happening with QUORUM. > Paging/limit > > > reads see this issue. I have attached the ccm based script which > > reproduces > > > the problem. > > > > > > * Keyspace RF - 2 > > > * Table (id int, course text, marks int, primary key(id, course)) > > > * replicas for partition key 1 - r1, r2 and r3 > > > * insert (1, '1', 1) , (1, '2', 2), (1, '3', 3), (1, '4', 4), (1, > > '5', > > > 5) - succeeded on all 3 replicas > > > * insert (1, '6', 6) succeeded on r1 and r3, failed on r2 > > > * delete (1, '2'), (1, '3'), (1, '4'), (1, '5') succeeded on r1 and r2, > > > failed on r3 > > > * insert (1, '7', 7) succeeded on r1 and r2, failed on r3 > > > > > > Local data on 3 nodes looks like as below now > > > > > > r1: (1, '1', 1), tombstone(2-5 records), (1, '6', 6), (1, '7', 7) > > > r2: (1, '1', 1), tombstone(2-5 records), (1, '7', 7) > > > r3: (1, '1', 1), (1, '2', 2), (1, '3', 3), (1, '4', 4), (1, '5', > > > 5), (1, '6', 6) > > > > > > If we do a paging read with page_size 2, and if it gets data from r2 > and > > > r3, then it will only get the data (1, '1', 1) and (1, '7', 7) skipping > > > record 6. This problem would happen if the same query is not doing > paging > > > but limit set to 2 records. > > > > > > Resolution code for reads works same for paging queries and normal > > > queries. Co-ordinator shouldn't respond back to client with > > records/columns > > > that it didn't have complete visibility on all required replicas (in > this > > > case 2 replicas). In above case, it is sending back record (1, '7', 7) > > back > > > to client, but its visibility on r3 is limited up to (1, '2', 2) and it > > is > > > relying on just r2 data to assume (1, '6', 6) doesn't exist, which is > > > wrong. End of the resolution all it can conclusively say any thing > about > > is > > > (1, '1', 1), which exists and (1, '2', 2), which is deleted. > > > > > > Ideally we should have different resolution implementation for > > > paging/limit queries. > > > > > > We could reproduce this on 2.0.17, 2.1.16 and 3.0.9. > > > > > > Seems like 3.0.9 we have ShortReadProtection transformation on list > > > queries. I assume that is to protect against the cases like above. But, > > we > > > can reproduce the issue in 3.0.9 as well. > > > > > > Thanks, > > > Bhaskar > > > > > > > > > > > > > > > > > > > > > > > >
[GitHub] cassandra pull request #80: CASSANDRA-12869: Don't load mx4j beans twice
Github user nastra closed the pull request at: https://github.com/apache/cassandra/pull/80 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] cassandra issue #81: 12869 trunk
Github user nastra commented on the issue: https://github.com/apache/cassandra/pull/81 merged already --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] cassandra issue #80: CASSANDRA-12869: Don't load mx4j beans twice
Github user nastra commented on the issue: https://github.com/apache/cassandra/pull/80 merged already --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] cassandra pull request #81: 12869 trunk
Github user nastra closed the pull request at: https://github.com/apache/cassandra/pull/81 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
