3.10 fixver, CHANGES.txt, and the cassandra-3.11 branch

[Michael Shuler]

Sam and I fixed up CHANGES.txt and fix versions in JIRA that had 3.11.

Since 3.10 has not been released yet,and the cassandra-3.11 branch was
created from the 3.10-tentative tag for fixed on top of 3.10, the
cassandra-3.11 branch will be used for the 3.10 release, as well as for
fixed bugs on top of 3.10 that will become 3.11. There should be no JIRA
tickets or CHANGES entries for 3.11, until such time that 3.10 is
actually released :)

Any new features / improvements targeted for 3.12 should be committed to
the cassandra-3.X branch. This should hopefully help the cassandra-3.11
branch settle down.

-- 
Kind regards,
Michael

Re: PgConf US 2017

[Jonathan Ellis]

Hi Josh,

What kind of cross pollination are you hoping to see?  What would the PG
community be interested in hearing about Cassandra?

On Mon, Nov 28, 2016 at 11:01 AM, Joshua D. Drake 
wrote:

> Hello folks,
>
> This is the last week of the PGConf US 2017 CFP. It would be great to see
> some integrated content from other communities.
>
> http://www.pgconf.us/2017/submit/
>
> Sincerely,
>
> JD
> --
> Command Prompt, Inc.  http://the.postgres.company/
> +1-503-667-4564
> PostgreSQL Centered full stack support, consulting and development.
> Everyone appreciates your honesty, until you are honest with them.
> Unless otherwise stated, opinions are my own.
>



-- 
Jonathan Ellis
co-founder, http://www.datastax.com
@spyced

Streaming and MVs

[Benjamin Roth]

I don't know where else to discuss this issue, so I post it here.

I am trying to get CS to run stable with MVs since the beginning of july.
Normal reads + write do work as expected but when it comes to repairs or
bootstrapping it still feels far far away from what I would call fast and
stable. The other day I just wanted to bootstrap a new node. I tried it 2
times.
First time the bootstrap failed due to WTEs. I fixed this issue by not
timing out in streams but then it turned out that the bootstrap (load
roughly 250-300 GB) didn't even finish in 24h. What if I really had a
problem and had to get up some nodes fast? No way!

I think the root cause of it all is the way streams are handled on tables
with MVs.
Sending them to the regular write path implies so many bottlenecks and
sometimes also redundant writes. Let me explain:

1. Bootstrap
During a bootstrap, all ranges from all KS and all CFs that will belong to
the new node will be streamed. MVs are treated like all other CFs and all
ranges that will move to the new node will also be streamed during
bootstrap.
Sending streams of the base tables through the write path will have the
following negative impacts:

   - Writes are sent to the commit log. Not necessary. When node is stopped
   during bootstrap, bootstrap will simply start over. No need to recover from
   commit logs. Non-MV tables won't have a CL anyway
   - MV mutations will not be applied instantly but send to the batch log.
   This is of course necessary during the range movement (if PK of MV differs
   from base table) but what happens: The batchlog will be completely flooded.
   This leads to ridiculously large batchlogs (I observerd BLs with 60GB
   size), zillions of compactions and quadrillions of tombstones. This is a
   pure resource killer, especially because BL uses a CF as a queue.
   - Applying every mutation separately causes read-before-writes during MV
   mutation. This is of course an order of magnitude slower than simply
   streaming down an SSTable. This effect becomes even worse while bootstrap
   progresses and creates more and more (uncompacted) SSTables. Many of them
   wont ever be compacted because the batchlog eats all the resources
   available for compaction
   - Streaming down the MV tables AND applying the mutations of the
   basetables leads to redundant writes. Redundant writes are local if PK of
   the MV == PK of the base table and - even worse - remote if not. Remote MV
   updates will impact nodes that aren't even part of the bootstrap.
   - CDC should also not be necessary during bootstrap, should it? TBD

2. Repair
Negative impact is similar to bootstrap but, ...

   - Sending repairs through write path will not mark the streamed tables
   as repaired. See CASSANDRA-12888. Doing NOT so will instantly solve that
   issue. Much simpler with any other solution
   - It will change the "repair design" a bit. Repairing a base table will
   not automatically repair the MV. But is this bad at all? To be honest as a
   newbie it was very hard for me to understand what I had to do to be sure
   that everything is repaired correctly. Recently I was told NOT to repair MV
   CFs but only to repair the base tables. This means one cannot just call
   "nodetool repair $keyspace" - this is complicated, not transparent and it
   sucks. I changed the behaviour in my own branch and let run the dtests for
   MVs. 2 tests failed:
  - base_replica_repair_test of course failes due to the design change
  - really_complex_repair_test fails because it intentionally times out
  the batch log. IMHO this is a bearable situation. It is comparable to
  resurrected tombstones when running a repair after GCGS expired. You also
  would not expect this to be magically fixed. gcgs default is 10
days and I
  can expect that anybody also repairs its MVs during that period, not only
  the base table

3. Rebuild
Same like bootstrap, isn't it?

Did I forget any cases?
What do you think?

-- 
Benjamin Roth
Prokurist

Jaumo GmbH · www.jaumo.com
Wehrstraße 46 · 73035 Göppingen · Germany
Phone +49 7161 304880-6 · Fax +49 7161 304880-1
AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: CASSANDRA-8596

[Benjamin Lerer]

One of the problem is that we have to deal with a huge number of tickets.
We could let them open forever or close the ones that we think are not real
problems or that we know we will never have time to do.
CASSANDRA-8954 has been opened for a year before Aleksey closed it.
When those things happen you can feel free to reopen the ticket and ask for
some explanations.

I think most of us have more work assigned than we can deal in the next 2-3
coming months. Some patch might wait for a review for months sometime and I
ping people constantly.

If you are waiting for some answers do not hesitate to ping the person with
who you are in contact with (simply put its name on JIRA). Beware that some
people take time for responding (sometime a week).

On Tue, Nov 29, 2016 at 9:56 AM, Vladimir Yudovin 
wrote:

> Hi guys, thanks for answer!
>
>
>
> Actually the last activity on this ticket was my question
>
> Does this issue exist with other snitches? How can they benefit from my
> solution?
>
>
>
>
> Also before working on ticked or prepare patch it's good to know if it's
> needed at all. ;-)
>
> Look at https://issues.apache.org/jira/browse/CASSANDRA-8945 It was
> Resolved to "Not A Problem" without any discussion.
>
>
>
> We made this and other fixes in our own Cassandra build, but if things are
> silently denied obviously it's not stimulate donation and participation, as
> it takes more time and energy than just to fix it for us only.
>
>
>
> Best regards, Vladimir Yudovin,
>
> Winguzone - Cloud Cassandra Hosting
>
>
>
>
>
>
>  On Tue, 29 Nov 2016 03:45:59 -0500 Oleksandr Petrov &
> lt;oleksandr.pet...@gmail.com> wrote 
>
>
>
>
> I would recommend several things that could help this (and probably many
>
> other patches that accidentally went under radar) to get accepted. It's
>
> hard to keep an enormous amount of issues in memory, so we try to rely on
>
> simple rules that help everyone to see current status and understand what
>
> to do next with the ticket:
>
>
>
>  * if you are planning to work on ticket, assign it to yourself.
>
> Sometimes it's also good to comment on it (if there was previous activity),
>
> as someone else might be already working on it but forgot to mention it or
>
> assign it. Doesn't happen often though.
>
>  * If you're working on it, put it into "work in progress" status.
>
>  * If you have finished working on the patch and think it's ready for
>
> review, put it into "ready for review" state.
>
>  * (bonus) ping someone on cassandra-dev IRC channel to get your patch
>
> submitted for CI. Most of time however, this will be done by reviewer
>
> anyways. But having a passing build is always reassuring.
>
>
>
> I know that committers (and active contributors) are frequently reviewing
>
> "ready for review" tickets and try to provide feedback. Relying only on
>
> ticket comments is a bit hard, since it's too easy to lose track of them.
>
>
>
> Sometimes it's possible that the next person who can review the ticket will
>
> only have time in a couple of weeks. Everyone does their best. But if you
>
> feel that it went forgotten, just leave another comment.
>
>
>
> Hope this helps!
>
>
>
> On Tue, Nov 29, 2016 at 9:19 AM Vladimir Yudovin  >
>
> wrote:
>
>
>
> > Hi,
>
> >
>
> >
>
> >
>
> > in the light of broader community involvement I would like to bring
>
> > attentions to the https://issues.apache.org/
> jira/browse/CASSANDRA-8596 I
>
> > open with proposed patch.
>
> >
>
> > It's stuck for almost 1.5 years.
>
> >
>
> >
>
> >
>
> > I would like to help with Cassandra development and improvement, and I
>
> > have several other fixes, some has JIRA, some not.
>
> >
>
> >
>
> >
>
> > Best regards, Vladimir Yudovin,
>
> >
>
> > Winguzone - Cloud Cassandra Hosting
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > --
>
> Alex Petrov
>
>
>
>
>
>
>

Re: CASSANDRA-8596

[Vladimir Yudovin]

 On Tue, 29 Nov 2016 03:55:15 -0500 Benjamin Lerer 
 wrote 

>It also seems that the ticket has never been marked as patch available

Yes, as it still was under discussion that never was finished. 





Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting






 On Tue, 29 Nov 2016 03:55:15 -0500 Benjamin Lerer 
 wrote 




Hi Vladimir, 

 

It also seems that the ticket has never been marked as patch available and 

that you did not assign yourself to it. Most of us look at those to 

determine in which state is a ticket. 

 

On Tue, Nov 29, 2016 at 9:42 AM, Jeff Jirsa  wrote: 

 

> Hey Vladimir 

> 

> Always nice to see patches, and pinging this list is a good way to draw 

> attention to them if they go idle 

> 

> Looks like it stalled because Brandon recommended two changes and you 

> addressed one but not the other - Brandon may have more feedback, he's 
very 

> familiar with that part of the codebase. 

> 

> These days, it'd also be nice to see a unit test (or dtest) to demonstrate 

> that the patch corrects the issue properly - that not only proves 

> correctness, but better protects us in the future in case someone tries to 

> alter the schema of system.peers in an incompatible way. 

> 

> 

> 

> -- 

> Jeff Jirsa 

> 

> 

> > On Nov 29, 2016, at 12:19 AM, Vladimir Yudovin 
 

> wrote: 

> > 

> > Hi, 

> > 

> > 

> > 

> > in the light of broader community involvement I would like to bring 

> attentions to the https://issues.apache.org/jira/browse/CASSANDRA-8596 I 

> open with proposed patch. 

> > 

> > It's stuck for almost 1.5 years. 

> > 

> > 

> > 

> > I would like to help with Cassandra development and improvement, and 
I 

> have several other fixes, some has JIRA, some not. 

> > 

> > 

> > 

> > Best regards, Vladimir Yudovin, 

> > 

> > Winguzone - Cloud Cassandra Hosting 

> > 

> > 

> > 

> > 

> > 

> 

Re: CASSANDRA-8596

[Vladimir Yudovin]

Hi guys, thanks for answer!



Actually the last activity on this ticket was my question 

Does this issue exist with other snitches? How can they benefit from my 
solution?




Also before working on ticked or prepare patch it's good to know if it's needed 
at all. ;-)

Look at https://issues.apache.org/jira/browse/CASSANDRA-8945 It was Resolved to 
"Not A Problem" without any discussion.



We made this and other fixes in our own Cassandra build, but if things are 
silently denied obviously it's not stimulate donation and participation, as it 
takes more time and energy than just to fix it for us only.



Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting






 On Tue, 29 Nov 2016 03:45:59 -0500 Oleksandr Petrov 
 wrote 




I would recommend several things that could help this (and probably many 

other patches that accidentally went under radar) to get accepted. It's 

hard to keep an enormous amount of issues in memory, so we try to rely on 

simple rules that help everyone to see current status and understand what 

to do next with the ticket: 

 

 * if you are planning to work on ticket, assign it to yourself. 

Sometimes it's also good to comment on it (if there was previous activity), 

as someone else might be already working on it but forgot to mention it or 

assign it. Doesn't happen often though. 

 * If you're working on it, put it into "work in progress" status. 

 * If you have finished working on the patch and think it's ready for 

review, put it into "ready for review" state. 

 * (bonus) ping someone on cassandra-dev IRC channel to get your patch 

submitted for CI. Most of time however, this will be done by reviewer 

anyways. But having a passing build is always reassuring. 

 

I know that committers (and active contributors) are frequently reviewing 

"ready for review" tickets and try to provide feedback. Relying only on 

ticket comments is a bit hard, since it's too easy to lose track of them. 

 

Sometimes it's possible that the next person who can review the ticket will 

only have time in a couple of weeks. Everyone does their best. But if you 

feel that it went forgotten, just leave another comment. 

 

Hope this helps! 

 

On Tue, Nov 29, 2016 at 9:19 AM Vladimir Yudovin  

wrote: 

 

> Hi, 

> 

> 

> 

> in the light of broader community involvement I would like to bring 

> attentions to the https://issues.apache.org/jira/browse/CASSANDRA-8596 I 

> open with proposed patch. 

> 

> It's stuck for almost 1.5 years. 

> 

> 

> 

> I would like to help with Cassandra development and improvement, and I 

> have several other fixes, some has JIRA, some not. 

> 

> 

> 

> Best regards, Vladimir Yudovin, 

> 

> Winguzone - Cloud Cassandra Hosting 

> 

> 

> 

> 

> 

> -- 

Alex Petrov 

Re: CASSANDRA-8596

[Benjamin Lerer]

Hi Vladimir,

It also seems that the ticket has never been marked as patch available and
that you did not assign yourself to it. Most of us look at those to
determine in which state is a ticket.

On Tue, Nov 29, 2016 at 9:42 AM, Jeff Jirsa  wrote:

> Hey Vladimir
>
> Always nice to see patches, and pinging this list is a good way to draw
> attention to them if they go idle
>
> Looks like it stalled because Brandon recommended two changes and you
> addressed one but not the other - Brandon may have more feedback, he's very
> familiar with that part of the codebase.
>
> These days, it'd also be nice to see a unit test (or dtest) to demonstrate
> that the patch corrects the issue properly - that not only proves
> correctness, but better protects us in the future in case someone tries to
> alter the schema of system.peers in an incompatible way.
>
>
>
> --
> Jeff Jirsa
>
>
> > On Nov 29, 2016, at 12:19 AM, Vladimir Yudovin 
> wrote:
> >
> > Hi,
> >
> >
> >
> > in the light of broader community involvement  I would like to bring
> attentions to the https://issues.apache.org/jira/browse/CASSANDRA-8596 I
> open with proposed patch.
> >
> > It's stuck for almost 1.5 years.
> >
> >
> >
> > I would like to help with Cassandra development and improvement, and I
> have several other fixes, some has JIRA, some not.
> >
> >
> >
> > Best regards, Vladimir Yudovin,
> >
> > Winguzone - Cloud Cassandra Hosting
> >
> >
> >
> >
> >
>

Re: CASSANDRA-8596

[Oleksandr Petrov]

I would recommend several things that could help this (and probably many
other patches that accidentally went under radar) to get accepted. It's
hard to keep an enormous amount of issues in memory, so we try to rely on
simple rules that help everyone to see current status and understand what
to do next with the ticket:

   * if you are planning to work on ticket, assign it to yourself.
Sometimes it's also good to comment on it (if there was previous activity),
as someone else might be already working on it but forgot to mention it or
assign it. Doesn't happen often though.
  * If you're working on it, put it into "work in progress" status.
  * If you have finished working on the patch and think it's ready for
review, put it into "ready for review" state.
  * (bonus) ping someone on cassandra-dev IRC channel to get your patch
submitted for CI. Most of time however, this will be done by reviewer
anyways. But having a passing build is always reassuring.

I know that committers (and active contributors) are frequently reviewing
"ready for review" tickets and try to provide feedback. Relying only on
ticket comments is a bit hard, since it's too easy to lose track of them.

Sometimes it's possible that the next person who can review the ticket will
only have time in a couple of weeks. Everyone does their best. But if you
feel that it went forgotten, just leave another comment.

Hope this helps!

On Tue, Nov 29, 2016 at 9:19 AM Vladimir Yudovin 
wrote:

> Hi,
>
>
>
> in the light of broader community involvement  I would like to bring
> attentions to the https://issues.apache.org/jira/browse/CASSANDRA-8596 I
> open with proposed patch.
>
> It's stuck for almost 1.5 years.
>
>
>
> I would like to help with Cassandra development and improvement, and I
> have several other fixes, some has JIRA, some not.
>
>
>
> Best regards, Vladimir Yudovin,
>
> Winguzone - Cloud Cassandra Hosting
>
>
>
>
>
> --
Alex Petrov

Re: CASSANDRA-8596

[Jeff Jirsa]

Hey Vladimir

Always nice to see patches, and pinging this list is a good way to draw 
attention to them if they go idle

Looks like it stalled because Brandon recommended two changes and you addressed 
one but not the other - Brandon may have more feedback, he's very familiar with 
that part of the codebase.

These days, it'd also be nice to see a unit test (or dtest) to demonstrate that 
the patch corrects the issue properly - that not only proves correctness, but 
better protects us in the future in case someone tries to alter the schema of 
system.peers in an incompatible way. 



-- 
Jeff Jirsa


> On Nov 29, 2016, at 12:19 AM, Vladimir Yudovin  wrote:
> 
> Hi,
> 
> 
> 
> in the light of broader community involvement  I would like to bring 
> attentions to the https://issues.apache.org/jira/browse/CASSANDRA-8596 I open 
> with proposed patch.
> 
> It's stuck for almost 1.5 years. 
> 
> 
> 
> I would like to help with Cassandra development and improvement, and I have 
> several other fixes, some has JIRA, some not.
> 
> 
> 
> Best regards, Vladimir Yudovin, 
> 
> Winguzone - Cloud Cassandra Hosting
> 
> 
> 
> 
> 

CASSANDRA-8596

[Vladimir Yudovin]

Hi,



in the light of broader community involvement  I would like to bring attentions 
to the https://issues.apache.org/jira/browse/CASSANDRA-8596 I open with 
proposed patch.

It's stuck for almost 1.5 years. 



I would like to help with Cassandra development and improvement, and I have 
several other fixes, some has JIRA, some not.



Best regards, Vladimir Yudovin, 

Winguzone - Cloud Cassandra Hosting