Re: Git Repo Migration

[Jason Brown]

+1 for the migration.

Thanks for owning and leading this, Sam

On Fri, Jan 4, 2019 at 02:58 Sam Tunnicliffe  wrote:

> As per the announcement on 7th December 2018[1], ASF infra are planning to
> shutdown the service behind git-wip-us.apache.org and migrate all
> existing repos to gitbox.apache.org
>
> There are further details in the original mail, but apparently one of the
> benefits of the migration is that we'll have full write access via Github,
> including the ability finally to close PRs. This affects the cassandra,
> cassandra-dtest and cassandra-build repos (but not the new
> cassandra-sidecar repo).
>
> A pre-requisite of the migration is to demonstrate consensus within the
> community, so to satisfy that formality I'm starting this thread to gather
> any objections or specific requests regarding the timing of the move.
>
> I'll collate responses in a week or so and file the necessary INFRA Jira.
>
> Thanks,
> Sam
>
> [1]
> https://lists.apache.org/thread.html/667772efdabf49a0a23d585539c127f335477e033f1f9b6f5079aced@%3Cdev.cassandra.apache.org%3E
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Sidecar repo created

[Jason Brown]

Hello all,

In moving the sidecar project forward, I've created a gitbox [1] repo for
the cassandra-sidecar which is located at:

git: https://gitbox.apache.org/repos/asf/cassandra-sidecar.git
github: https://github.com/apache/cassandra-sidecar

There are additional behaviors we can set on the repo [2]:


   1. Any Pull Request that gets opened, closed, reopened or commented on
   can get recorded on the project's mailing list
   2. If a project has a JIRA instance, any PRs or comments on PRs that
   include a JIRA ticket ID will trigger an update on that specific ticket
   (unless opted out of)
   3. GitHub activity can be relayed to IRC channels on the Freenode
   network.

I think we probably want to enable #1 & #2 to maintain parity with what we
have already with the main cassandra repo.

Thanks,

-Jason

[1] https://gitbox.apache.org/
[2] https://reference.apache.org/pmc/github

Re: Proposing an Apache Cassandra Management process

[Jason Brown]

It looks like we have consensus to move forward with the sidecar
I will be shepherding the project, and to that end, I'll create a repo
against which we can start working.

Thanks,

-Jason

On Fri, Nov 30, 2018 at 7:54 AM sankalp kohli 
wrote:

> If no one has more feedback, we should create JIRAs for all the subtasks
> discussed in the proposal. I can see JIRA for Bulk command but not others.
>
> On Mon, Nov 19, 2018 at 2:12 AM dinesh.jo...@yahoo.com.INVALID
>  wrote:
>
> > Thanks, Mick & Stefan for your inputs. What should we do as next steps to
> > move this proposal forward?
> > Dinesh
> >
> > On Sunday, November 18, 2018, 11:57:54 AM PST, Stefan Podkowinski <
> > s...@apache.org> wrote:
> >
> >  My goal for a side car would be to enable more people to contribute to
> > the project, by making it more accessible for anyone who’s not familiar
> > with the Cassandra code base, or not familiar with Java development in
> > general. Although most of the functionality described in the proposal
> > sounds useful to have, I’d already be happy to have a solid REST API for
> > the existing nodetool and JMX functionality. If an official side car,
> > installed separately on each node, would provide that, I’m sure we’d see
> > lots of new tools created by the community (web UIs, cli tools, ..)
> > based on that. This would also be a good foundation for other existing
> > tool to converge upon, e.g. by calling the REST APIs for repair
> > scheduling and progress tracking instead of JMX, or by continually
> > integrating and sharing useful helper calls. This would also give
> > Cassandra devs more leeway to replace some of the existing tooling
> > related code in Cassandra, e.g. by migrating to virtual tables, while at
> > the same time keep providing a stable API through the side car.
> >
> > What I’d also like to point out here is that implementing such a project
> > as an *official* side car, also implies to me having the same standards
> > when it comes to release quality. I’d also really prefer having feature
> > sets matching between Cassandra and the side car, e.g. authentication
> > and SSL should also be supported in the side car from the beginning,
> > ideally without any additional configuration.
> >
> >
> > On 06.11.18 10:40, Dinesh Joshi wrote:
> > > Hi all,
> > >
> > > Joey, Vinay & I have fleshed out the Management process proposal as the
> > very first CIP document (with Jason’s inputs). It is available on the
> cwiki
> > -
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652224
> > >
> > > Please comment on it and provide us any input that you may have. We
> want
> > to ideally time-box the period to 2 weeks so we avoid waiting
> indefinitely.
> > >
> > > Thanks,
> > >
> > > Dinesh
> > >
> > >> On Oct 22, 2018, at 7:30 AM, "dinesh.jo...@yahoo.com.INVALID" <
> > dinesh.jo...@yahoo.com.INVALID> wrote:
> > >>
> > >> Thanks for starting this, Mick. I will flesh it out.
> > >> Dinesh
> > >>
> > >>On Sunday, October 21, 2018, 1:52:10 AM PDT, Mick Semb Wever <
> > m...@apache.org> wrote:
> > >>
> > >>
> > >>> But I'll try to put together a strawman proposal for the doc(s) over
> > the
> > >>> weekend.
> > >>
> > >> I've thrown something quickly together here:
> > >> -
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652201
> > >> -
> >
> https://cwiki.apache.org/confluence/display/CASSANDRA/CIP-1%3A+Proposing+an+Apache+Cassandra+Management+process
> > >>
> > >> The former is a blatant rip-off from the Kafka and Spark design
> > proposal pages that Dinesh previously mentioned. I'd hoped to do more of
> an
> > analysis of the existing C* habits and precedence on design proposals
> > (implicit in jira tickets), but in lei of that this is a strawman to
> start
> > the discussion.
> > >>
> > >> The latter still needs to be fleshed out. Dinesh, can you do this? I
> > can add a subpage/section that describes the alternative/consuming
> > third-party tools out there.
> > >>
> > >> regards,
> > >> Mick
> > >>
> > >> -
> > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >>
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [VOTE] Change Jira Workflow

[Jason Brown]

+1.

On Mon, Dec 17, 2018 at 7:36 AM Michael Shuler 
wrote:

> +1
>
> --
> Michael
>
> On 12/17/18 9:19 AM, Benedict Elliott Smith wrote:
> > I propose these changes <
> https://cwiki.apache.org/confluence/display/CASSANDRA/JIRA+Workflow+Proposals>*
> to the Jira Workflow for the project.  The vote will be open for 72 hours**.
> >
> > I am, of course, +1.
> >
> > * With the addendum of the mailing list discussion <
> https://lists.apache.org/thread.html/e4668093169aa4ef52f2bea779333f04a0afde8640c9a79a8c86ee74@%3Cdev.cassandra.apache.org%3E>;
> in case of any conflict arising from a mistake on my part in the wiki, the
> consensus reached by polling the mailing list will take precedence.
> > ** I won’t be around to close the vote, as I will be on vacation.
> Everyone is welcome to ignore the result until I get back in a couple of
> weeks, or if anybody is eager feel free to close the vote and take some
> steps towards implementation.
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Revisit the proposal to use github PR

[Jason Brown]

To clarify my position: Github PRs are great for *reviewing* code, and the
commentary is much easier to follow imo. But for *merging* code, esp into
our multi-branch strategy, PRs don't fit well, unless there's some
technique I and perhaps others are unaware of.

On Thu, Dec 13, 2018 at 9:47 AM Ariel Weisberg  wrote:

> Hi,
>
> I'm not clear on what github makes worse. It preserves more history then
> the JIRA approach. When people invitably force push their branches you
> can't tell from the link to a branch on JIRA. Github preserves the comments
> and force push history so you know what version of the code each comment
> applied to. Github also tracks when requests for changes are acknowledged
> and resolved.  I have had to make the same change request many times and
> keep track independently whether it was resolved. This has also resulting
> in mistakes getting merged when I missed a comment that was ignored.
>
> Now that github can CC JIRA that also CCs to commits@. It's better then
> JIRA comments because each comment includes a small diff of the code the
> comment applies to. To do that in JIRA I have to manually link to the code
> the PR and most people don't do that for every comment so some of them are
> inscrutable after the fact. Also manually created links sometimes refer to
> references that disappear or get force pushed. It's a bit tricky to get
> right.
>
> To me arguing against leveraging a better code review workflow (whether
> github or some other tool) is like arguing against using source control
> tools. Sure the filesystem and home grown scripts can be used to work
> around lack of source control, but why would you?
>
> I see two complaints so far. One is that github PRs encourage nitpicking.
> I don't see tool based solution to that off the cuff. Another is that
> github doesn't by default CC JIRA. Maybe we can just refuse to accept
> improperly formatted PRs and look into auto rejecting ones that don't refer
> to a ticket?
>
> Ariel
>
> On Thu, Dec 13, 2018, at 12:20 PM, Aleksey Yeschenko wrote:
> > There are some nice benefits to GH PRs, one of them is that we could
> > eventually set up CircleCI hooks that would explicitly prevent commits
> > that don’t pass the tests.
> >
> > But handling multiple branches would indeed be annoying. Would have to
> > either submit 1 PR per branch - which is both tedious and non-atomic -
> > or do a mixed approach, with a PR for the oldest branch, then a manual
> > merge upwards. The latter would be kinda meh, especially when commits
> > for different branches diverge.
> >
> > For me personally, the current setup works quite well, and I mostly
> > share Sylvain’s opinion above, for the same reasons listed.
> >
> > —
> > AY
> >
> > > On 13 Dec 2018, at 08:15, Sylvain Lebresne  wrote:
> > >
> > > Fwiw, I personally find it very useful to have all discussion, review
> > > comments included, in the same place (namely JIRA, since for better or
> > > worse, that's what we use for tracking tickets). Typically, that means
> > > everything gets consistently pushed to the  commits@ mailing list,
> which I
> > > find extremely convenient to keep track of things. I also have a theory
> > > that the inline-comments type of review github PR give you is very
> > > convenient for nitpicks, shallow or spur-of-the-moment comments, but
> > > doesn't help that much for deeper reviews, and that it thus to favor
> the
> > > former kind of review.
> > >
> > > Additionally, and to Benedict's point, I happen to have first hand
> > > experience with a PR-based process for a multi-branch workflow very
> similar
> > > to the one of this project, and suffice to say that I hate it with a
> > > passion.
> > >
> > > Anyway, very much personal opinion here.
> > > --
> > > Sylvain
> > >
> > >
> > > On Thu, Dec 13, 2018 at 2:13 AM dinesh.jo...@yahoo.com.INVALID
> > >  wrote:
> > >
> > >> I've been already using github PRs for some time now. Once you
> specify the
> > >> ticket number, the comments and discussion are persisted in Apache
> Jira as
> > >> work log so it can be audited if desired. However, committers usually
> > >> squash and commit the changes once the PR is approved. We don't use
> the
> > >> merge feature in github. I don't believe github we can merge the
> commit
> > >> into multiple branches through the UI. We would need to merge it into
> one
> > >> branch and then manually merge that commit into other branches. The
> big
> > >> upside of using github PRs is that it makes collaborating a lot
> easier.
> > >> Downside is that it makes it very difficult to follow along the
> progress in
> > >> Apache Jira. The messages that github posts back include huge diffs
> and are
> > >> aweful.
> > >> Dinesh
> > >>
> > >>On Thursday, December 13, 2018, 1:10:12 AM GMT+5:30, Benedict
> Elliott
> > >> Smith  wrote:
> > >>
> > >> Perhaps somebody could summarise the tradeoffs?  I’m a little
> concerned
> > >> about how it would work for our multi-branch workflow.  Would we 

Re: Revisit the proposal to use github PR

[Jason Brown]

I'm with Sylvain and Aleksey. Our multi-branch strategy does not work well
(if at all) with Github PR-style workflows (and, yes, I and most c*
maintainers have played this game). The cassandra-dtests repo is different
as we don't really branch there, so might be fine for Github PRs - except
for the merge commit noise, and that we prefer squashed patches for commit.

We could perhaps be a bit more explicit about "how to contribute" in the
docs [1], wrt branches as well as Github PRs (that we don't do them), but
it's mostly not unreasonable.

Thus, I'm in favor of the current process.

[1] http://cassandra.apache.org/doc/latest/development/patches.html


On Thu, Dec 13, 2018 at 9:20 AM Aleksey Yeschenko 
wrote:

> There are some nice benefits to GH PRs, one of them is that we could
> eventually set up CircleCI hooks that would explicitly prevent commits that
> don’t pass the tests.
>
> But handling multiple branches would indeed be annoying. Would have to
> either submit 1 PR per branch - which is both tedious and non-atomic - or
> do a mixed approach, with a PR for the oldest branch, then a manual merge
> upwards. The latter would be kinda meh, especially when commits for
> different branches diverge.
>
> For me personally, the current setup works quite well, and I mostly share
> Sylvain’s opinion above, for the same reasons listed.
>
> —
> AY
>
> > On 13 Dec 2018, at 08:15, Sylvain Lebresne  wrote:
> >
> > Fwiw, I personally find it very useful to have all discussion, review
> > comments included, in the same place (namely JIRA, since for better or
> > worse, that's what we use for tracking tickets). Typically, that means
> > everything gets consistently pushed to the  commits@ mailing list,
> which I
> > find extremely convenient to keep track of things. I also have a theory
> > that the inline-comments type of review github PR give you is very
> > convenient for nitpicks, shallow or spur-of-the-moment comments, but
> > doesn't help that much for deeper reviews, and that it thus to favor the
> > former kind of review.
> >
> > Additionally, and to Benedict's point, I happen to have first hand
> > experience with a PR-based process for a multi-branch workflow very
> similar
> > to the one of this project, and suffice to say that I hate it with a
> > passion.
> >
> > Anyway, very much personal opinion here.
> > --
> > Sylvain
> >
> >
> > On Thu, Dec 13, 2018 at 2:13 AM dinesh.jo...@yahoo.com.INVALID
> >  wrote:
> >
> >> I've been already using github PRs for some time now. Once you specify
> the
> >> ticket number, the comments and discussion are persisted in Apache Jira
> as
> >> work log so it can be audited if desired. However, committers usually
> >> squash and commit the changes once the PR is approved. We don't use the
> >> merge feature in github. I don't believe github we can merge the commit
> >> into multiple branches through the UI. We would need to merge it into
> one
> >> branch and then manually merge that commit into other branches. The big
> >> upside of using github PRs is that it makes collaborating a lot easier.
> >> Downside is that it makes it very difficult to follow along the
> progress in
> >> Apache Jira. The messages that github posts back include huge diffs and
> are
> >> aweful.
> >> Dinesh
> >>
> >>On Thursday, December 13, 2018, 1:10:12 AM GMT+5:30, Benedict Elliott
> >> Smith  wrote:
> >>
> >> Perhaps somebody could summarise the tradeoffs?  I’m a little concerned
> >> about how it would work for our multi-branch workflow.  Would we open
> >> multiple PRs?
> >>
> >> Could we easily link with external CircleCI?
> >>
> >> It occurs to me, in JIRA proposal mode, that an extra required field
> for a
> >> permalink to GitHub for the patch would save a lot of time I spend
> hunting
> >> for a branch in the comments.
> >>
> >>
> >>
> >>
> >>> On 12 Dec 2018, at 19:20, jay.zhu...@yahoo.com.INVALID wrote:
> >>>
> >>> It was discussed 1 year's ago:
> >> https://www.mail-archive.com/dev@cassandra.apache.org/msg11810.html
> >>> As all Apache projects are moving to gitbox:
> >> https://reference.apache.org/committer/github, should we revisit that
> and
> >> change our review/commit process to use github PR?A good example is
> >> Spark:"Changes to Spark source code are proposed, reviewed and committed
> >> via Github pull requests" (https://spark.apache.org/contributing.html).
> >>> /jay
> >>
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Testing 4.0 upgrading

[Jason Brown]

Hi Tommy,

Thank you for taking the time to start kicking the tires on the upgrade to
4.0. It looks like you've found two bugs:

1) "Unknown column coordinator_port during deserialization" (reported on
3.x nodes)

- looks like the 4.0 node isn't filtering out a column from a system table
that 3.0 doesn't know about. Most likely due to CASSANDRA-7544. Can you
open a JIRA for this, and tag @aweisberg?

2) SSL connection problems

I unserstand the 4.0 -> 3.X connection problem, and documented it at [1] in
MessagingService. TL;DR we don't know the version of a peer when restarting
and need to wait for that peer to connect to the local node and pass it's
correct messaging version (if the local node cannot interpret the handhsake
from the peer). However, why for the inbound connection to the 4.0 node it
is seeing SSLv2 is unclear. Can you open a separate JIRA, and we'll go from
there? In the meantime, maybe enable the JDK's SSL debugging [2] on the 3.x
node to see exactly what it is trying to do? Also, you can enable wire
tracing on the 4.0 node by setting this value to true [3] and recompiling.
We can followup further in the jira.

Thanks!

-Jason

[1]
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/MessagingService.java#L1668
[2]
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ReadDebug.html
[3]
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/async/NettyFactory.java#L84

On Tue, Oct 23, 2018 at 2:44 AM Tommy Stendahl 
wrote:

> Hi,
>
> I have been testing upgrade to 4.0, I started out with a cluster with
> 3.0.15 and server encryption enabled. Due to some issues in my
> environment I did upgrade one of the nodes to 3.11.3, I think this
> turned out to be a good thing since I could observer the behaviour of
> upgrading from both 3.0.15 and 3.11.3 at the same time.
>
> At first I didn't have much success at all, it look like found multiple
> issues mostly with server encryption so I decided to simplify thing and
> disabled server encryption.
>
> So with server encryption disabled the upgrade was working ok, what I
> did notice was exceptions in the 3.0.15 and 3.11.3 nodes once the first
> 4.0 node started.
>
> 3.0.15 exception:
> 2018-10-22T11:05:38.883+0200 ERROR
> [MessagingService-Incoming-/10.216.193.244] CassandraDaemon.java:223
> Exception in thread Thread[MessagingService-Incoming-/10.216.193.244
> ,5,main]
> java.lang.RuntimeException: Unknown column coordinator_port during
> deserialization
>  at
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.db.filter.ColumnFilter$Serializer.deserialize(ColumnFilter.java:447)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:647)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:584)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.io.ForwardingVersionedSerializer.deserialize(ForwardingVersionedSerializer.java:50)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at org.apache.cassandra.net.MessageIn.read(MessageIn.java:98)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:201)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>
> 3.11.3 exception:
> 2018-10-22T11:12:05.060+0200 ERROR
> [MessagingService-Incoming-/10.216.193.244] CassandraDaemon.java:228
> Exception in thread Thread[MessagingService-Incoming-/10.216.193.244
> ,5,main]
> java.lang.RuntimeException: Unknown column coordinator_port during
> deserialization
>  at
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:452)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at
> org.apache.cassandra.db.filter.ColumnFilter$Serializer.deserialize(ColumnFilter.java:482)
>
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at
> org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:760)
>
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at
> org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:697)
>
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at
> org.apache.cassandra.io.ForwardingVersionedSerializer.deserialize(ForwardingVersionedSerializer.java:50)
>
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.net.MessageIn.read(MessageIn.java:123)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at
> 

Re: Moving tickets out of 4.0 post freeze

[Jason Brown]

Before we blanket update all of these, let's make sure they are not
relevant for 4.0 - because some of them actually are. Some are docs, some
are parent tickets, some are testing.

Naively, here are some that are still 4.0:
CASSANDRA-14746 
CASSANDRA-13254 
CASSANDRA-14606 

Thanks,

-Jason

On Tue, Sep 25, 2018 at 4:05 AM, Benedict Elliott Smith  wrote:

> Do we want to manage more versions than we do now?  Why don’t we simply
> align these things to majors, like we’ve typically tried to anyway?
>
> I think it’s anyway better to decide on a strategy and find a versioning
> scheme that matches it, rather than to look for a strategy that matches our
> current versioning scheme.
>
>
>
>
> > On 25 Sep 2018, at 03:07, Mick Semb Wever  wrote:
> >
> >
> >
> >
> >> I'm totally spitballing here on possible uses of meaningful minors.
> >
> > Continuing the splitballing…
> >
> > What about aligning native protocol or sstable format changes with the
> minor version?
> >
> >
> >> Regardless, the OP's statement that new features and improvements
> should go to 4.0.x seems wrong
> >
> > Yeah, I instinctively thought features and improvements would be moved
> to either 4.x or 5.x (not to any subsequent patch version 4.0.x).
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: UDF

[Jason Brown]

Hi Robert,

Thanks for taking on this work. Is this message a heads up that a patch is
coming/complete, or to spawn a discussion about including this in 4.0?

Thanks,

-Jason

On Tue, Sep 11, 2018 at 2:32 AM, Robert Stupp  wrote:

> In an effort to clean up our hygiene and limit the dependencies used by
> UDFs/UDAs, I think we should refactor the UDF code parts and remove the
> dependency to the Java Driver in that area without breaking existing
> UDFs/UDAs.
>
> A working prototype is in this branch: https://github.com/snazy/
> cassandra/tree/feature/remove-udf-driver-dep-trunk <
> https://github.com/snazy/cassandra/tree/feature/remove-
> udf-driver-dep-trunk> . The changes are rather trivial and provide 100%
> backwards compatibility for existing UDFs.
>
> The prototype copies the necessary parts from the Java Driver into the C*
> source tree to org.apache.cassandra.cql3.functions.types and adopts its
> usages - i.e. UDF/UDA code plus CQLSSTableWriter + StressCQLSSTableWriter.
> The latter two classes have a reference to UDF’s UDHelper and had to be
> changed as well.
>
> Some functionality, like type parsing & handling, is duplicated in the
> code base with this prototype - once in the “current” source tree and once
> for UDFs. However, unifying the code paths is not trivial, since the UDF
> sandbox prohibits the use of internal classes (direct and likely indirect
> dependencies).
>
> Robert
>
> —
> Robert Stupp
> @snazy
>
>

Re: NGCC 2018?

[Jason Brown]

+1 to Jon's sentiment. Further, perhaps we should use that time after
GA'ing 4.0 to poll our users what they need/want from the next major
release of the database.

On Wed, Sep 5, 2018 at 9:31 AM, Jonathan Haddad  wrote:

> I'm thinking a month or two after 4.0 would give us time to unwind after
> the release and start to give real thought to big changes coming in the
> next release.  Let's focus on one thing at a time.
>
> On Wed, Sep 5, 2018 at 12:29 PM sankalp kohli 
> wrote:
>
> > A good time for NGCC will be closer to 4.0 release where we can plan what
> > we can put it on 4.0-next. I am not sure doing it now is going to help
> when
> > we are months away from 4.0 release.
> >
> > On Fri, Aug 31, 2018 at 7:42 AM Jay Zhuang 
> wrote:
> >
> > > Are we going to have a dev event next month? Or anything this year? We
> > may
> > > also be able to provide space in bay area and help to organize it.
> > (Please
> > > let us know, so we could get final approval for that).
> > >
> > > On Fri, Jul 27, 2018 at 10:05 AM Jonathan Haddad 
> > > wrote:
> > >
> > > > My interpretation of Nate's statement was that since there would be a
> > > bunch
> > > > of us at Lynn's event, we might as well do NGCC at the same time.
> > > >
> > > > On Thu, Jul 26, 2018 at 9:03 PM Ben Bromhead 
> > > wrote:
> > > >
> > > > > It sounds like there may be an appetite for something, but the NGCC
> > in
> > > > its
> > > > > current format is likely to not be that useful?
> > > > >
> > > > > Is a bay area event focused on C* developers something that is
> > > > interesting
> > > > > for the broader dev community? In whatever format that may be?
> > > > >
> > > > > On Tue, Jul 24, 2018 at 5:02 PM Nate McCall 
> > > wrote:
> > > > >
> > > > > > This was discussed amongst the PMC recently. We did not come to a
> > > > > > conclusion and there were not terribly strong feelings either
> way.
> > > > > >
> > > > > > I don't feel like we need to hustle to get "NGCC" in place,
> > > > > > particularly given our decided focus on 4.0. However, that should
> > not
> > > > > > stop us from doing an additional 'c* developer' event in sept. to
> > > > > > coincide with distributed data summit.
> > > > > >
> > > > > > On Wed, Jul 25, 2018 at 5:03 AM, Patrick McFadin <
> > pmcfa...@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > Ben,
> > > > > > >
> > > > > > > Lynn Bender had offered a space the day before Distributed Data
> > > > Summit
> > > > > in
> > > > > > > September (http://distributeddatasummit.com/) since we are
> both
> > > > > platinum
> > > > > > > sponsors. I thought he and Nate had talked about that being a
> > good
> > > > > place
> > > > > > > for NGCC since many of us will be in town already.
> > > > > > >
> > > > > > > Nate, now that I've spoken for you, you can clarify, :D
> > > > > > >
> > > > > > > Patrick
> > > > > > >
> > > > > >
> > > > > >
> > -
> > > > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > > > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > > > > >
> > > > > > --
> > > > > Ben Bromhead
> > > > > CTO | Instaclustr 
> > > > > +1 650 284 9692
> > > > > Reliability at Scale
> > > > > Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
> > > > >
> > > >
> > > >
> > > > --
> > > > Jon Haddad
> > > > http://www.rustyrazorblade.com
> > > > twitter: rustyrazorblade
> > > >
> > >
> >
>
>
> --
> Jon Haddad
> http://www.rustyrazorblade.com
> twitter: rustyrazorblade
>

Re: [Discuss] Accept GoCQL driver donation

[Jason Brown]

I find this idea interesting and worth a strong discussion.

Something to consider is an argument floating around in the admin tool/side
car discussion: if we take an existing project wholesale, we inherit all of
it's design decision and technical debt (every project has these). On the
other hand, we also get working code that is running in production. I
haven't looked at the gocql code (and probably won't until we're a bit
beyond the 4.0 code freeze), so I can't speak to anything beyond a cursory
reading of the docs.

As we consider bringing in drivers under the project umbrella, we need to
evaluate each driver implementation's merit on a case-by-case basis. I'm
not sure how we divvy up that work, or whom to entrust with that work, but
it's something to keep in mind.

Thanks,

-Jason


On Fri, Aug 31, 2018 at 8:38 AM, Chris Bannister 
wrote:

> I intend to stay on and continue to contribute.
>
> On Fri, 31 Aug 2018 at 4:37 pm, Jonathan Ellis  wrote:
>
> > On Fri, Aug 31, 2018 at 9:14 AM Nate McCall  wrote:
> >
> > > Hi folks,
> > > So I was recently talking with, Chris Bannister  the gocql [0]
> > > maintainer, and he expressed an interest in donating the driver to the
> > > ASF.
> > >
> >
> > Is he looking to continue to maintain it or is he looking to give it a
> good
> > home when he moves on?
> >
> > We could accept this along the same lines as how we took in the dtest
> > > donation - going through the incubator IP clearance process [1], but
> > > in this case it's much simpler as an individual (Chris) owns the
> > > copyright.
> > >
> >
> > Is that actually the case?  Github says 128 contributors, and I don't see
> > any mention of a CLA in
> > https://github.com/gocql/gocql/blob/master/CONTRIBUTING.md.
> >
> > --
> > Jonathan Ellis
> > co-founder, http://www.datastax.com
> > @spyced
> >
>

Re: Supporting multiple JDKs

[Jason Brown]

Some of our java8 code will not compile under java11 (see CASSANDRA-9608);
the symbols have literally been removed (Unsafe.monitorEnter() /
Unsafe.monitorExit() ). Setting -source to "8" will not help. Thus, we need
two compilers for the foreseeable future.



On Thu, Aug 23, 2018 at 3:44 PM, Sumanth Pasupuleti <
sumanth.pasupuleti...@gmail.com> wrote:

> I am not against using a compile-time quick-feedback tool like Animal
> Sniffer either. It is great to have such a tool to know of any obvious bad
> changes right away during development. However, in addition to using such a
> tool, I believe, when we make a release, we should build against the actual
> JDKs we support (that way, we are not making a release just based on the
> result of an external tool), and we should be able to optionally run UTs
> and DTests against the JDK  (i.e. Java7 and Java8 for C* 2.2).
>
> > What do we mean "support multiple JDKs" ?
> > Are we talking Run-time or Compile-time?
> I am talking both - compile-time can for Build, run-time for UTs and
> DTests.
>
> > I think that dtests are always going to be the important defence here,
> and that we simplify it by running dtests on a standardised JDK compile.
> I agree, but its good to have an optional workflow in CircleCI to be able
> to run DTest against the non-standardized JDK as well, which we support
> (Java7 for example, for C*2.2, and Java11 for C* 4.0)
>
> Thanks,
> Sumanth
>
> On Thu, Aug 23, 2018 at 12:06 AM Mick Semb Wever  wrote:
>
> >
> > > > There's a cost-optimisation here in reducing what we have to support.
> > >
> > > I agree and animal sniffer is a great way to ferret out obvious issues.
> > > I am not against using animal sniffer. I'm concerned that there are
> > > various incompatibilities[1] between JDK versions and I am not 100%
> > > certain that animal sniffer will catch all of them.
> >
> >
> > Yeah, but is compiling (and unit tests) really any more effective against
> > behavioural incompatibilities (which also occur from one patch JDK
> version
> > to the next)?
> >
> > I think that dtests are always going to be the important defence here,
> and
> > that we simplify it by running dtests on a standardised JDK compile.
> >
> > regards,
> > Mick
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: Side Car New Repo vs not

[Jason Brown]

+1 for separate repo. For pretty much all the same reasons Aleksey
elucidated.

On Tue, Aug 21, 2018 at 10:20 AM, Aleksey Yeshchenko 
wrote:

> Sure, allow me to elaborate - at least a little bit. But before I do, just
> let me note that this wasn’t a veto -1, just a shorthand for “I don’t like
> this option”.
>
> It would be nice to have sidecar and C* version and release cycles fully
> decoupled. I know it *can* be done when in-tree, but the way we vote on
> releases with tags off current branches would have to change somehow.
> Probably painfully. It would be nice to be able to easily enforce freezes,
> like the upcoming one, on the whole C* repo, while allowing feature
> development on the sidecar. It would be nice to not have sidecar commits in
> emails from commits@ mailing list. It would be nice to not have C* CI
> trigger necessarily on sidecar commits. Groups of people working on the two
> repos will mostly be different too, so what’s the point in sharing the repo?
>
> Having an extra repo with its own set of branches is cheap and easy - we
> already do that with dtests. I like cleanly separated things when coupling
> is avoidable. As such I would prefer the sidecar to live in a separate new
> repo, while still being part of the C* project.
>
> —
> AY
>
> On 21 August 2018 at 17:06:39, sankalp kohli (kohlisank...@gmail.com)
> wrote:
>
> Hi Aleksey,
> Can you please elaborate on the reasons for your -1? This
> way we can make progress towards any one approach.
> Thanks,
> Sankalp
>
> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko 
> wrote:
>
> > FWIW I’m strongly -1 on in-tree approach, and would much prefer a
> separate
> > repo, dtest-style.
> >
> > —
> > AY
> >
> > On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> > jeremiah.jor...@gmail.com) wrote:
> >
> > I think the following is a very big plus of it being in tree:
> > >> * Faster iteration speed in general. For example when we need to add
> a
> > >> new
> > >> JMX endpoint that the sidecar needs, or change something from JMX to
> a
> > >> virtual table (e.g. for repair, or monitoring) we can do all changes
> > >> including tests as one commit within the main repository and don't
> > have
> > >> to
> > >> commit to main repo, sidecar repo,
> >
> > I also don’t see a reason why the sidecar being in tree means it would
> not
> > work in a mixed version cluster. The nodes themselves must work in a
> mixed
> > version cluster during a rolling upgrade, I would expect any management
> > side car to operate in the same manor, in tree or not.
> >
> > This tool will be pretty tightly coupled with the server, and as
> someone
> > with experience developing such tightly coupled tools, it is *much*
> easier
> > to make sure you don’t accidentally break them if they are in tree. How
> > many times has someone updated some JMX interface, updated nodetool,
> and
> > then moved on? Breaking all the external tools not in tree, without
> > realizing it. The above point about being able to modify interfaces and
> the
> > side car in the same commit is huge in terms of making sure someone
> doesn’t
> > inadvertently break the side car while fixing something else.
> >
> > -Jeremiah
> >
> >
> > > On Aug 21, 2018, at 10:28 AM, Jonathan Haddad 
> > wrote:
> > >
> > > Strongly agree with Blake. In my mind supporting multiple versions is
> > > mandatory. As I've stated before, we already do it with Reaper, I'd
> > > consider it a major misstep if we couldn't support multiple with the
> > > project - provided admin tool. It's the same reason dtests are
> separate
> > -
> > > they work with multiple versions.
> > >
> > > The number of repos does not affect distribution - if we want to ship
> > > Cassandra with the admin / repair tool (we should, imo), that can be
> > part
> > > of the build process.
> > >
> > >
> > >
> > >
> > > On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston 
>
> > > wrote:
> > >
> > >> If the sidecar is going to be on a different release cadence, or
> > support
> > >> interacting with mixed mode clusters, then it should definitely be
> in
> > a
> > >> separate repo. I don’t even know how branching and merging would
> work
> > in a
> > >> repo that supports 2 separate release targets and/or mixed mode
> > >> compatibility, but I’m pretty sure it would be a mess.
> > >>
> > >> As a cluster management tool, mixed mode is probably going to be a
> goal
> > at
> > >> some point. As a new project, it will benefit from not being tied to
> > the C*
> > >> release cycle (which would probably delay any sidecar release until
> > >> whenever 4.1 is cut).
> > >>
> > >>
> > >> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.ly...@gmail.com)
>
> >
> > >> wrote:
> > >>
> > >> I think that the pros of incubating the sidecar in tree as a tool
> > first
> > >> outweigh the alternatives at this point of time. Rough tradeoffs
> that
> > I
> > >> see:
> > >>
> > >> Unique pros of in tree sidecar:
> > >> * Faster iteration speed in general. For example when we 

Re: upgrade guava on trunk before 9/1?

[Jason Brown]

Hey Ariel,

Tbqh, not that much. I was mostly thinking from the "I have conflicts on
guava versions in my app because I pull in cassandra and XYZ libraries, and
the transitive dependencies on guava use different versions" POV. Further,
we'll be on this version of guava for 4.0 for at least two years from now.

As I asked, "does anybody feeling strongly?". Personally, I'm sorta +0 to
+0.5, but I was just throwing this out there in case someone does really
think it best we upgrade (and wants to make a contribution).

-Jason




On Wed, Aug 15, 2018 at 7:25 AM, Ariel Weisberg  wrote:

> Hi,
>
> What do we get from Guava in exchange for upgrading?
>
> Ariel
>
> On Wed, Aug 15, 2018, at 10:19 AM, Jason Brown wrote:
> > Hey all,
> >
> > Does anyone feel strongly about upgrading guava on trunk before the 9/1
> > feature freeze for 4.0? We are currently at 23.3 (thanks to
> > CASSANDRA-13997), and the current is 26.0.
> >
> > I took a quick look, and there's about 17 compilation errors. They fall
> > into two categories, both of which appear not too difficult to resolve (I
> > didn't look too closely, tbh).
> >
> > If anyone wants to tackle this LHF I can rustle up some review time.
> >
> > Thanks,
> >
> > -Jason
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

upgrade guava on trunk before 9/1?

[Jason Brown]

Hey all,

Does anyone feel strongly about upgrading guava on trunk before the 9/1
feature freeze for 4.0? We are currently at 23.3 (thanks to
CASSANDRA-13997), and the current is 26.0.

I took a quick look, and there's about 17 compilation errors. They fall
into two categories, both of which appear not too difficult to resolve (I
didn't look too closely, tbh).

If anyone wants to tackle this LHF I can rustle up some review time.

Thanks,

-Jason

Re: G1GC is now default

[Jason Brown]

restored CMS as the default GC alg in CASSANDRA-14636.

Thanks,

-Jason

On Wed, Aug 8, 2018 at 4:33 PM, Jason Brown  wrote:

> Hi Jon,
>
> Thanks for pointing that out. Looks like I missed that during the
> CASSANDRA-9608 review. Robert and I didn't discuss that online or offline,
> so it's something that snuck in.
>
> I don't have a personal preference for GC algs right now, but at a minimum
> it would probably be appropriate to return to CMS. We can certainly have a
> discussion thread about switching the default, but until we have that that,
> I think it's more honest to revert to CMS. If there's no immediate push
> back on reverting, I'll have a patch within a day to revert.
>
> Thanks,
>
> -Jason
>
> On Wed, Aug 8, 2018 at 3:34 PM, Jonathan Haddad  wrote:
>
>> I fired up trunk to check something, and noticed this:
>>
>> INFO  [Service Thread] 2018-08-08 15:01:36,723 GCInspector.java:290 - G1
>> Young Generation GC in 339ms.  G1 Eden Space: 4634705920 -> 0; G1 Old Gen:
>> 1190138352 -> 1435504616; G1 Survivor Space: 406847488 -> 301989888;
>>
>> which I thought was a bit weird, since I was using trunk without making
>> any
>> changes and didn't remember seeing a JIRA where we agreed to make that
>> change.  I looked back and saw it made it in as a side effect of
>> CASSANDRA-9608, but wasn't explicitly discussed in the ticket, and there's
>> no note of it in CHANGES.
>>
>> I'm personally OK with this change as G1 is a safer bet for anyone who
>> uses
>> the defaults, but we should be explicit about the change.  Can anyone
>> think
>> of a reason why we'd need to revert this back to ParNew / CMS?
>>
>> --
>> Jon Haddad
>> http://www.rustyrazorblade.com
>> twitter: rustyrazorblade
>>
>
>

Re: G1GC is now default

[Jason Brown]

Hi Jon,

Thanks for pointing that out. Looks like I missed that during the
CASSANDRA-9608 review. Robert and I didn't discuss that online or offline,
so it's something that snuck in.

I don't have a personal preference for GC algs right now, but at a minimum
it would probably be appropriate to return to CMS. We can certainly have a
discussion thread about switching the default, but until we have that that,
I think it's more honest to revert to CMS. If there's no immediate push
back on reverting, I'll have a patch within a day to revert.

Thanks,

-Jason

On Wed, Aug 8, 2018 at 3:34 PM, Jonathan Haddad  wrote:

> I fired up trunk to check something, and noticed this:
>
> INFO  [Service Thread] 2018-08-08 15:01:36,723 GCInspector.java:290 - G1
> Young Generation GC in 339ms.  G1 Eden Space: 4634705920 -> 0; G1 Old Gen:
> 1190138352 -> 1435504616; G1 Survivor Space: 406847488 -> 301989888;
>
> which I thought was a bit weird, since I was using trunk without making any
> changes and didn't remember seeing a JIRA where we agreed to make that
> change.  I looked back and saw it made it in as a side effect of
> CASSANDRA-9608, but wasn't explicitly discussed in the ticket, and there's
> no note of it in CHANGES.
>
> I'm personally OK with this change as G1 is a safer bet for anyone who uses
> the defaults, but we should be explicit about the change.  Can anyone think
> of a reason why we'd need to revert this back to ParNew / CMS?
>
> --
> Jon Haddad
> http://www.rustyrazorblade.com
> twitter: rustyrazorblade
>

triage of recent dtest failures

[Jason Brown]

Hi all,

Now that the vote is on for the next releases, I've done an initial triage
of the failed dtest runs from 2.2, 3.0, and 3.11 (based on my circleci
runs). Below are the ten dtest failures I found most often (not exhaustive,
but representative), and I opened up corresponding jiras for them.

Let's see if we can get all the tickets assigned and analyzed. If in
researching, a test is indescribably flakey, that is valuble knowledge as
well. Being able to identify flakey tests and label them as such is step
forward, lacking a full fix. With these tickets, it might be that the test
needs to be corrected, and might not necessarily be a problem with
database. In some cases, it will be a problem in casandra, so please keep
that in mind.

Remember, fixing these tests helps *all* of us get to more stable and
reliable releases, and thus makes the database and project as a whole
stronger.

Also, feel free to reach out to me with questions on these.

Thanks!

-Jason


* test_describecluster_more_information_three_datacenters -
nodetool_test.TestNodetool
- versions: 3.11, 3.0, 2.2
https://issues.apache.org/jira/browse/CASSANDRA-14484#


* test_closing_connections - thrift_hsha_test.TestThriftHSHA
- versions: 3.11, 3.0, 2.2
https://issues.apache.org/jira/browse/CASSANDRA-14595


* test_mutation_v5 - write_failures_test.TestWriteFailures
- versions: 3.11 only
https://issues.apache.org/jira/browse/CASSANDRA-14596


* snapshot_test.TestArchiveCommitlog.*
- versions: apparently only 3.0
https://issues.apache.org/jira/browse/CASSANDRA-14597


* test_decommissioned_node_cant_rejoin - topology_test.TestTopology
- versions: seen on 3.0, but may be more
https://issues.apache.org/jira/browse/CASSANDRA-14598


* test_functional - global_row_key_cache_test.TestGlobalRowKeyCache
- versions: apparently only 3.0
https://issues.apache.org/jira/browse/CASSANDRA-14599


* test_system_auth_ks_is_alterable - auth_test.TestAuth
- versions: 3.0 / 3.11
https://issues.apache.org/jira/browse/CASSANDRA-14600


* test_failure_threshold_deletions - paging_test.TestPagingWithDeletions
-versions 3.11 only
https://issues.apache.org/jira/browse/CASSANDRA-14601


* test_sstableofflinerelevel - offline_tools_test.TestOfflineTools
- versions 3.0 only
https://issues.apache.org/jira/browse/CASSANDRA-14602


* test_alter_rf_and_run_read_repair & test_read_repair_chance -
read_repair_test.TestReadRepair
- versions 2.2, 3.0
https://issues.apache.org/jira/browse/CASSANDRA-14603

Re: [VOTE] Release Apache Cassandra 3.0.17 (Take 2)

[Jason Brown]

Ran the tests again, just for sanity sake, and all seems within acceptable
levels (in order to get the data loss bugs resolved).

+1



On Wed, Jul 25, 2018 at 1:24 PM, Nate McCall  wrote:

> +1
>
> On Wed, Jul 25, 2018, 7:47 PM Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 3.0.17.
> >
> > sha1: d52c7b8c595cc0d06fc3607bf16e3f595f016bb6
> > Git:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/3.0.17-tentative
> > Artifacts:
> >
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1165/org/apache/cassandra/apache-cassandra/3.0.17/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1165/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: CHANGES.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/3.0.17-tentative
> > [2]: NEWS.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/3.0.17-tentative
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [VOTE] Release Apache Cassandra 2.2.13

[Jason Brown]

Ran the tests again, just for sanity sake, and all seems within acceptable
levels.

+1

On Wed, Jul 25, 2018 at 1:25 PM, Nate McCall  wrote:

> +1
>
> On Wed, Jul 25, 2018, 7:49 PM Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 2.2.13.
> >
> > sha1: 3482370df5672c9337a16a8a52baba53b70a4fe8
> > Git:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/2.2.13-tentative
> > Artifacts:
> >
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1167/org/apache/cassandra/apache-cassandra/2.2.13/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1167/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: CHANGES.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/2.2.13-tentative
> > [2]: NEWS.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/2.2.13-tentative
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [VOTE] Release Apache Cassandra 3.11.3 (Take 2)

[Jason Brown]

Ran the tests again, just for sanity sake, and all seems within acceptable
levels (in order to get the data loss bugs resolved).

+1



On Wed, Jul 25, 2018 at 1:24 PM, Nate McCall  wrote:

> +1
>
> On Wed, Jul 25, 2018, 7:46 PM Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 3.11.3.
> >
> > sha1: 31d5d870f9f5b56391db46ba6cdf9e0882d8a5c0
> > Git:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/3.11.3-tentative
> > Artifacts:
> >
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1164/org/apache/cassandra/apache-cassandra/3.11.3/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1164/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: CHANGES.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/3.11.3-tentative
> > [2]: NEWS.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/3.11.3-tentative
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: reroll the builds?

[Jason Brown]

I did run the dtests against the last release shas (3.0.16 and 3.11.2).
Notes are all the way at the bottom of the gist about those runs. Circleci
URLs: https://circleci.com/workflow-run/5a1df5a1-f0c1-4ab4-a7db-e5551e7a5d38
/ https://circleci.com/workflow-run/a4369ab0-ae11-497a-8e10-de3995d10f25.

Current HEAD of 3.0 & 3.11 branches have significantly lower failing
dtests, and the failing tests on HEAD are a subset of those from the last
release.


On Tue, Jul 24, 2018 at 9:03 AM, dinesh.jo...@yahoo.com.INVALID <
dinesh.jo...@yahoo.com.invalid> wrote:

> Hi Jason,
> I agree - we should release with the dataloss bug fix. I went over the
> gist - apart from the Python errors and test teardown failures, there seem
> to be a few failures that look legitimate. Any chance you can run the
> dtests on the previous release SHAs and compare the dtest failures? If
> they're the same / similar, we know at least we're at parity with the
> previous release :)
> Dinesh
>
> On Tuesday, July 24, 2018, 8:18:50 AM PDT, Jason Brown <
> jasedbr...@gmail.com> wrote:
>
>  TL;DR We are in a better place than we were for the 3.0.16 and 3.11.2
> releases. The current fails are not fatal, although they warrant
> investigation. My opinion is that due the critical data loss bugs that are
> fixed by CASSANDRA-14513 and CASSANDRA-14515, we should cut the builds now.
>
> I've run the HEAD of the 3.0 and 3.11 branches vs the 3.0.16 and 3.11.2
> release shas, and there are far less failing dtests now. In comparison:
>
> - 3.11
> -- HEAD - 5-6 failing tests
> -- 3.11.2 - 18-20 failures
>
> - 3.0
> -- HEAD - 14-16 failures
> -- 3.0.16 - 22-25 failures
>
> The raw dump of my work can be found here:
> https://gist.github.com/jasobrown/e7ecf6d0bf875d1f4a08ee06ac7eaba0. I've
> applied no effort to clean it up, but it's available (includes links to the
> circleci runs). I haven't completed an exhautive analysis of the failures
> to see how far they go back as things become tricky (or, at least, very
> time intensive to research) with the pytest/python-3 update with
> CASSANDRA-14134. Thus some of the failures might be in the dtests
> themselves (I suspect a couple of the failures are), but most are proabaly
> legit failures.
>
> As this thread is about cutting the releases, I'll save any significiant
> analysis for a followup thread. I will say that the current failures are a
> subset of the previous release's failures, and those failures are not data
> loss bugs.
>
> Overall, I feel far more comfortable getting the data loss fixes out
> without any further delay than waiting for a few minor fixes. I will triage
> the dtest failures over the coming days. There are some open tickets, and
> I'll try to corral those with any new ones.
>
> Thanks,
>
> -Jason
>
>
> On Mon, Jul 23, 2018 at 10:26 AM, dinesh.jo...@yahoo.com.INVALID <
> dinesh.jo...@yahoo.com.invalid> wrote:
>
> > I can help out with the triage / rerunning dtests if needed.
> > Dinesh
> >
> >On Monday, July 23, 2018, 10:22:18 AM PDT, Jason Brown <
> > jasedbr...@gmail.com> wrote:
> >
> >  I spoke with some people over here, and I'm going to spend a day doing a
> > quick triage of the failing dtests. There are some fixes for data loss
> bugs
> > that are critical to get out in these builds, so I'll ensure the current
> > failures are within an acceptable level of flakey-ness in order to
> unblock
> > those fixes.
> >
> > Will have an update shortly ...
> >
> > -Jason
> >
> > On Mon, Jul 23, 2018 at 9:18 AM, Jason Brown 
> wrote:
> >
> > > Hi all,
> > >
> > > First, thanks Joey for running the tests. Your pass/fail counts are
> > > basically what in line with what I've seen for the last several months.
> > (I
> > > don't have an aggregated list anywhere, just observations from recent
> > runs).
> > >
> > > Second, it's beyond me why there's such inertia to actually cutting a
> > > release. We're getting up to almost *six months* since the last
> release.
> > > Are there any grand objections at this point?
> > >
> > > Thanks,
> > >
> > > -Jason
> > >
> > >
> > > On Tue, Jul 17, 2018 at 4:01 PM, Joseph Lynch 
> > > wrote:
> > >
> > >> We ran the tests against 3.0, 2.2 and 3.11 using circleci and there
> are
> > >> various failing dtests but all three have green unit tests.
> > >>
> > >> 3.11.3 tentative (31d5d87, test branch
> > >> <https://circleci.com/gh/vinaykumarchella/cassandra/tree/
> > >> cassandra_3.11_temp_t

Re: reroll the builds?

[Jason Brown]

TL;DR We are in a better place than we were for the 3.0.16 and 3.11.2
releases. The current fails are not fatal, although they warrant
investigation. My opinion is that due the critical data loss bugs that are
fixed by CASSANDRA-14513 and CASSANDRA-14515, we should cut the builds now.

I've run the HEAD of the 3.0 and 3.11 branches vs the 3.0.16 and 3.11.2
release shas, and there are far less failing dtests now. In comparison:

- 3.11
-- HEAD - 5-6 failing tests
-- 3.11.2 - 18-20 failures

- 3.0
-- HEAD - 14-16 failures
-- 3.0.16 - 22-25 failures

The raw dump of my work can be found here:
https://gist.github.com/jasobrown/e7ecf6d0bf875d1f4a08ee06ac7eaba0. I've
applied no effort to clean it up, but it's available (includes links to the
circleci runs). I haven't completed an exhautive analysis of the failures
to see how far they go back as things become tricky (or, at least, very
time intensive to research) with the pytest/python-3 update with
CASSANDRA-14134. Thus some of the failures might be in the dtests
themselves (I suspect a couple of the failures are), but most are proabaly
legit failures.

As this thread is about cutting the releases, I'll save any significiant
analysis for a followup thread. I will say that the current failures are a
subset of the previous release's failures, and those failures are not data
loss bugs.

Overall, I feel far more comfortable getting the data loss fixes out
without any further delay than waiting for a few minor fixes. I will triage
the dtest failures over the coming days. There are some open tickets, and
I'll try to corral those with any new ones.

Thanks,

-Jason


On Mon, Jul 23, 2018 at 10:26 AM, dinesh.jo...@yahoo.com.INVALID <
dinesh.jo...@yahoo.com.invalid> wrote:

> I can help out with the triage / rerunning dtests if needed.
> Dinesh
>
> On Monday, July 23, 2018, 10:22:18 AM PDT, Jason Brown <
> jasedbr...@gmail.com> wrote:
>
>  I spoke with some people over here, and I'm going to spend a day doing a
> quick triage of the failing dtests. There are some fixes for data loss bugs
> that are critical to get out in these builds, so I'll ensure the current
> failures are within an acceptable level of flakey-ness in order to unblock
> those fixes.
>
> Will have an update shortly ...
>
> -Jason
>
> On Mon, Jul 23, 2018 at 9:18 AM, Jason Brown  wrote:
>
> > Hi all,
> >
> > First, thanks Joey for running the tests. Your pass/fail counts are
> > basically what in line with what I've seen for the last several months.
> (I
> > don't have an aggregated list anywhere, just observations from recent
> runs).
> >
> > Second, it's beyond me why there's such inertia to actually cutting a
> > release. We're getting up to almost *six months* since the last release.
> > Are there any grand objections at this point?
> >
> > Thanks,
> >
> > -Jason
> >
> >
> > On Tue, Jul 17, 2018 at 4:01 PM, Joseph Lynch 
> > wrote:
> >
> >> We ran the tests against 3.0, 2.2 and 3.11 using circleci and there are
> >> various failing dtests but all three have green unit tests.
> >>
> >> 3.11.3 tentative (31d5d87, test branch
> >> <https://circleci.com/gh/vinaykumarchella/cassandra/tree/
> >> cassandra_3.11_temp_testing>,
> >> unit tests <https://circleci.com/gh/vinaykumarchella/cassandra/258>
> >> pass, 5
> >> <https://circleci.com/gh/vinaykumarchella/cassandra/256> and 6
> >> <https://circleci.com/gh/vinaykumarchella/cassandra/256#
> >> tests/containers/8>
> >> dtest failures)
> >> 3.0.17 tentative (d52c7b8, test branch
> >> <https://circleci.com/gh/jolynch/workflows/cassandra/tree/3.0-testing>,
> >> unit
> >> tests <https://circleci.com/gh/jolynch/cassandra/110> pass, 14
> >> <https://circleci.com/gh/jolynch/cassandra/112> and 15
> >> <https://circleci.com/gh/jolynch/cassandra/111> dtest failures)
> >> 2.2.13 tentative (3482370, test branch
> >> <https://circleci.com/gh/sumanth-pasupuleti/workflows/cassan
> >> dra/tree/2.2-testing>,
> >> unit tests <https://circleci.com/gh/sumanth-pasupuleti/cassandra/20>
> >> pass, 9
> >> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/21> and 10
> >> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/22#
> >> tests/containers/8>
> >> dtest failures)
> >>
> >> It looks like many (~6) of the failures in 3.0.x are related to
> >> snapshot_test.TestArchiveCommitlog. I'm not sure if this is abnormal.
> >>
> >> I don't see a good historical record to know if these are just flakes,
> but
> >> if we o

Re: reroll the builds?

[Jason Brown]

I spoke with some people over here, and I'm going to spend a day doing a
quick triage of the failing dtests. There are some fixes for data loss bugs
that are critical to get out in these builds, so I'll ensure the current
failures are within an acceptable level of flakey-ness in order to unblock
those fixes.

Will have an update shortly ...

-Jason

On Mon, Jul 23, 2018 at 9:18 AM, Jason Brown  wrote:

> Hi all,
>
> First, thanks Joey for running the tests. Your pass/fail counts are
> basically what in line with what I've seen for the last several months. (I
> don't have an aggregated list anywhere, just observations from recent runs).
>
> Second, it's beyond me why there's such inertia to actually cutting a
> release. We're getting up to almost *six months* since the last release.
> Are there any grand objections at this point?
>
> Thanks,
>
> -Jason
>
>
> On Tue, Jul 17, 2018 at 4:01 PM, Joseph Lynch 
> wrote:
>
>> We ran the tests against 3.0, 2.2 and 3.11 using circleci and there are
>> various failing dtests but all three have green unit tests.
>>
>> 3.11.3 tentative (31d5d87, test branch
>> <https://circleci.com/gh/vinaykumarchella/cassandra/tree/
>> cassandra_3.11_temp_testing>,
>> unit tests <https://circleci.com/gh/vinaykumarchella/cassandra/258>
>> pass, 5
>> <https://circleci.com/gh/vinaykumarchella/cassandra/256> and 6
>> <https://circleci.com/gh/vinaykumarchella/cassandra/256#
>> tests/containers/8>
>> dtest failures)
>> 3.0.17 tentative (d52c7b8, test branch
>> <https://circleci.com/gh/jolynch/workflows/cassandra/tree/3.0-testing>,
>> unit
>> tests <https://circleci.com/gh/jolynch/cassandra/110> pass, 14
>> <https://circleci.com/gh/jolynch/cassandra/112> and 15
>> <https://circleci.com/gh/jolynch/cassandra/111> dtest failures)
>> 2.2.13 tentative (3482370, test branch
>> <https://circleci.com/gh/sumanth-pasupuleti/workflows/cassan
>> dra/tree/2.2-testing>,
>> unit tests <https://circleci.com/gh/sumanth-pasupuleti/cassandra/20>
>> pass, 9
>> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/21> and 10
>> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/22#
>> tests/containers/8>
>> dtest failures)
>>
>> It looks like many (~6) of the failures in 3.0.x are related to
>> snapshot_test.TestArchiveCommitlog. I'm not sure if this is abnormal.
>>
>> I don't see a good historical record to know if these are just flakes, but
>> if we only want to go on green builds perhaps we can either disable the
>> flakey tests or fix them up? If someone feels strongly we should fix
>> particular tests up please link a jira and I can take a whack at some of
>> them.
>>
>> -Joey
>>
>> On Tue, Jul 17, 2018 at 9:35 AM Michael Shuler 
>> wrote:
>>
>> > On 07/16/2018 11:27 PM, Jason Brown wrote:
>> > > Hey all,
>> > >
>> > > The recent builds were -1'd, but it appears the issues have been
>> resolved
>> > > (2.2.13 with CASSANDRA-14423, and 3.0.17 / 3.11.3 reverting
>> > > CASSANDRA-14252). Can we go ahead and reroll now?
>> >
>> > Could someone run through the tests on 2.2, 3.0, 3.11 branches and link
>> > them?  Thanks!
>> >
>> > Michael
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> > For additional commands, e-mail: dev-h...@cassandra.apache.org
>> >
>> >
>>
>
>

Re: reroll the builds?

[Jason Brown]

Hi all,

First, thanks Joey for running the tests. Your pass/fail counts are
basically what in line with what I've seen for the last several months. (I
don't have an aggregated list anywhere, just observations from recent runs).

Second, it's beyond me why there's such inertia to actually cutting a
release. We're getting up to almost *six months* since the last release.
Are there any grand objections at this point?

Thanks,

-Jason


On Tue, Jul 17, 2018 at 4:01 PM, Joseph Lynch  wrote:

> We ran the tests against 3.0, 2.2 and 3.11 using circleci and there are
> various failing dtests but all three have green unit tests.
>
> 3.11.3 tentative (31d5d87, test branch
> <https://circleci.com/gh/vinaykumarchella/cassandra/
> tree/cassandra_3.11_temp_testing>,
> unit tests <https://circleci.com/gh/vinaykumarchella/cassandra/258> pass,
> 5
> <https://circleci.com/gh/vinaykumarchella/cassandra/256> and 6
> <https://circleci.com/gh/vinaykumarchella/cassandra/256#tests/containers/8
> >
> dtest failures)
> 3.0.17 tentative (d52c7b8, test branch
> <https://circleci.com/gh/jolynch/workflows/cassandra/tree/3.0-testing>,
> unit
> tests <https://circleci.com/gh/jolynch/cassandra/110> pass, 14
> <https://circleci.com/gh/jolynch/cassandra/112> and 15
> <https://circleci.com/gh/jolynch/cassandra/111> dtest failures)
> 2.2.13 tentative (3482370, test branch
> <https://circleci.com/gh/sumanth-pasupuleti/workflows/
> cassandra/tree/2.2-testing>,
> unit tests <https://circleci.com/gh/sumanth-pasupuleti/cassandra/20>
> pass, 9
> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/21> and 10
> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/
> 22#tests/containers/8>
> dtest failures)
>
> It looks like many (~6) of the failures in 3.0.x are related to
> snapshot_test.TestArchiveCommitlog. I'm not sure if this is abnormal.
>
> I don't see a good historical record to know if these are just flakes, but
> if we only want to go on green builds perhaps we can either disable the
> flakey tests or fix them up? If someone feels strongly we should fix
> particular tests up please link a jira and I can take a whack at some of
> them.
>
> -Joey
>
> On Tue, Jul 17, 2018 at 9:35 AM Michael Shuler 
> wrote:
>
> > On 07/16/2018 11:27 PM, Jason Brown wrote:
> > > Hey all,
> > >
> > > The recent builds were -1'd, but it appears the issues have been
> resolved
> > > (2.2.13 with CASSANDRA-14423, and 3.0.17 / 3.11.3 reverting
> > > CASSANDRA-14252). Can we go ahead and reroll now?
> >
> > Could someone run through the tests on 2.2, 3.0, 3.11 branches and link
> > them?  Thanks!
> >
> > Michael
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

reroll the builds?

[Jason Brown]

Hey all,

The recent builds were -1'd, but it appears the issues have been resolved
(2.2.13 with CASSANDRA-14423, and 3.0.17 / 3.11.3 reverting
CASSANDRA-14252). Can we go ahead and reroll now?

Thanks,

-Jason

Re: 3.11 currently not building

[Jason Brown]

Thanks for the quick turn around, Lerh and Paulo!

On Thu, Jun 7, 2018 at 3:56 AM, Paulo Motta 
wrote:

> Fixed bad merge to 3.11 with 7bb88deb4c6387fd67114543986774c903860de9.
> Thanks for the heads up!
>
> 2018-06-07 2:34 GMT-03:00 Lerh Chuan Low :
> > Hi guys,
> >
> > I saw your message on IRC Jason, Kurt forwarded it to me. Thanks for the
> > heads up!
> >
> > 3.11 presently doesn't build, I think it may have been due to an
> accidental
> > merge gone bad. The original JIRA
> > https://issues.apache.org/jira/browse/CASSANDRA-13698 has been updated
> with
> > a patch and the tests are running (runs locally for me as well), do
> please
> > have a look when possible, thanks!
> >
> > Lerh
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: 3.11.3

[Jason Brown]

wrt releasing 3.11, +1 to this ... and additionally, 3.0 and 2.2.

I'd propose we cut early next week (Monday) and anything that can get
reviewed/committed from Kurt's list would be great.

On Wed, Jun 6, 2018 at 5:22 AM, kurt greaves  wrote:

> We probably need to release 3.11.3 at some point soon because there's some
> pretty major bug fixes in there and interest has been expressed multiple
> times now for a release.
>
> Currently the only tickets marked for 3.11.3 that aren't complete are:
> https://issues.apache.org/jira/browse/CASSANDRA-14355 - No work has
> started
> on this yet.
> https://issues.apache.org/jira/browse/CASSANDRA-13929 - A lot of work has
> been done here... just need a final review
>
> However we've got quite a few tickets with patches that should probably be
> considered - can't say all are necessary to get into 3.11.3 but seeing as
> they all have patch available and have been around :
> for a while they are worth considering for anyone who could kindly review:
> https://issues.apache.org/jira/browse/CASSANDRA-14242 - Inconsistent
> indexing on static columns :(. needs review
> https://issues.apache.org/jira/browse/CASSANDRA-14415 - performance
> regression fix for DISTINCT
> https://issues.apache.org/jira/browse/CASSANDRA-14204 - Straightforward
> fix
> for nodetool garbagecollect
> https://issues.apache.org/jira/browse/CASSANDRA-14167 - Pretty much ready
> for commit - has been given +1 by Sylvain
> https://issues.apache.org/jira/browse/CASSANDRA-14099 - Needs only a unit
> test
> https://issues.apache.org/jira/browse/CASSANDRA-14085 - performance fix,
> small patch needs review
> https://issues.apache.org/jira/browse/CASSANDRA-13935 - Simple fix to CQL
> output for indexes when dumping schema
> https://issues.apache.org/jira/browse/CASSANDRA-13917 - Needs review, fix
> for compact storage inserts.
> https://issues.apache.org/jira/browse/CASSANDRA-13843 - Needs review.
> Small
> debian packaging fix for heapdump location.
> https://issues.apache.org/jira/browse/CASSANDRA-13834 - Needs an
> "official"
> reviewer and probably a test.
> https://issues.apache.org/jira/browse/CASSANDRA-13618 - Can likely be
> committed. Jeff has already done all the work for it just need to verify
> tests.
> https://issues.apache.org/jira/browse/CASSANDRA-11479 - Waiting for review
> for quite a long time.. unit test fix but seems to include changes to the
> server
>
> Handy JQL:
> project = CASSANDRA AND ((fixVersion = 3.11.x and status = "Patch
> Available" ) OR (fixVersion = 3.11.3 AND status != Resolved)) and Type =
> Bug
>

Re: Improve the performance of CAS

[Jason Brown]

Hey all,

Before we go bananas, let's see if Sylvain, the primary author of the
original patch, has the opportunity to chime with some explanatory notes or
other guidance. There may be some subtle points or considerations that are
not obvious, and I'd hate to lose that context.

Thanks,

-Jason

On Wed, May 16, 2018 at 2:57 PM, Ariel Weisberg  wrote:

> Hi,
>
> I think you are looking at the right low hanging fruit.  Cassandra
> deserves a better consensus protocol, but it's a very big project.
>
> Regards,
> Ariel
> On Wed, May 16, 2018, at 5:51 PM, Dikang Gu wrote:
> > Cool, create a jira for it,
> > https://issues.apache.org/jira/browse/CASSANDRA-14448. I have a draft
> patch
> > working internally, will clean it up.
> >
> > The EPaxos is more complicated, could be a long term effort.
> >
> > Thanks
> > Dikang.
> >
> > On Wed, May 16, 2018 at 2:20 PM, sankalp kohli 
> > wrote:
> >
> > > Hi,
> > > The idea of combining read with prepare sounds good. Regarding
> reducing
> > > the commit round trip, it is possible today by giving a lower
> consistency
> > > level for commit I think.
> > >
> > > Regarding EPaxos, it is a large change and will take longer to land. I
> > > think we should do this as it will help lower the latencies a lot.
> > >
> > > Thanks,
> > > Sankalp
> > >
> > > On Wed, May 16, 2018 at 2:15 PM, Jeremy Hanna <
> jeremy.hanna1...@gmail.com>
> > > wrote:
> > >
> > > > Hi Dikang,
> > > >
> > > > Have you seen Blake’s work on implementing egalitarian paxos or
> epaxos*?
> > > > That might be helpful for the discussion.
> > > >
> > > > Jeremy
> > > >
> > > > * https://issues.apache.org/jira/browse/CASSANDRA-6246
> > > >
> > > > > On May 16, 2018, at 3:37 PM, Dikang Gu  wrote:
> > > > >
> > > > > Hello C* developers,
> > > > >
> > > > > I'm working on some performance improvements of the lightweight
> > > > transitions
> > > > > (compare and set), I'd like to hear your thoughts about it.
> > > > >
> > > > > As you know, current CAS requires 4 round trips to finish, which
> is not
> > > > > efficient, especially in cross DC case.
> > > > > 1) Prepare
> > > > > 2) Quorum read current value
> > > > > 3) Propose new value
> > > > > 4) Commit
> > > > >
> > > > > I'm proposing the following improvements to reduce it to 2 round
> trips,
> > > > > which is:
> > > > > 1) Combine prepare and quorum read together, use only one round
> trip to
> > > > > decide the ballot and also piggyback the current value in response.
> > > > > 2) Propose new value, and then send out the commit request
> > > > asynchronously,
> > > > > so client will not wait for the ack of the commit. In case of
> commit
> > > > > failures, we should still have chance to retry/repair it through
> hints
> > > or
> > > > > following read/cas events.
> > > > >
> > > > > After the improvement, we should be able to finish the CAS
> operation
> > > > using
> > > > > 2 rounds trips. There can be following improvements as well, and
> this
> > > can
> > > > > be a start point.
> > > > >
> > > > > What do you think? Did I miss anything?
> > > > >
> > > > > Thanks
> > > > > Dikang
> > > >
> > > >
> > > > 
> -
> > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Dikang
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Roadmap for 4.0

[Jason Brown]

My understanding, from Nate's summary, was June 1 is the freeze date for
features. I expect we would go for at least 4 months (if not longer)
testing, fixing bugs, early dogfooding, and so on. I also equated June 1
with the data which we would create a 'cassandra-4.0' branch, and thus the
merge order becomes: 3.0->3,11->4.0->trunk.

Is this different from what others are thinking? I'm open to shifting the
actual date, but what about the rest?


On Thu, Apr 5, 2018 at 11:39 AM, Aleksey Yeshchenko <alek...@apple.com>
wrote:

> June feels a bit too early to me as well.
>
> I personally would go prefer end of August / beginning of September.
>
> +1 to the idea of having a fixed date, though, just not this one.
>
> —
> AY
>
> On 5 April 2018 at 19:20:12, Stefan Podkowinski (s...@apache.org) wrote:
>
> June is too early.
>
>
> On 05.04.18 19:32, Josh McKenzie wrote:
> > Just as a matter of perspective, I'm personally mentally diffing from
> > when 3.0 hit, not 3.10.
> >
> >> commit 96f407bce56b98cd824d18e32ee012dbb99a0286
> >> Author: T Jake Luciani <j...@apache.org>
> >> Date: Fri Nov 6 14:38:34 2015 -0500
> >> 3.0 release versions
> > While June feels close to today relative to momentum for a release
> > before this discussion, it's certainly long enough from when the
> > previous traditional major released that it doesn't feel "too soon" to
> > me.
> >
> > On Thu, Apr 5, 2018 at 12:46 PM, sankalp kohli <kohlisank...@gmail.com>
> wrote:
> >> We can take a look on 1st June how things are then decide if we want to
> >> freeze it and whats in and whats out.
> >>
> >> On Thu, Apr 5, 2018 at 9:31 AM, Ariel Weisberg <ar...@weisberg.ws>
> wrote:
> >>
> >>> Hi,
> >>>
> >>> +1 to having a feature freeze date. June 1st is earlier than I would
> have
> >>> picked.
> >>>
> >>> Ariel
> >>>
> >>> On Thu, Apr 5, 2018, at 10:57 AM, Josh McKenzie wrote:
> >>>> +1 here for June 1.
> >>>>
> >>>> On Thu, Apr 5, 2018 at 9:50 AM, Jason Brown <jasedbr...@gmail.com>
> >>> wrote:
> >>>>> +1
> >>>>>
> >>>>> On Wed, Apr 4, 2018 at 8:31 PM, Blake Eggleston <
> beggles...@apple.com>
> >>>>> wrote:
> >>>>>
> >>>>>> +1
> >>>>>>
> >>>>>> On 4/4/18, 5:48 PM, "Jeff Jirsa" <jji...@gmail.com> wrote:
> >>>>>>
> >>>>>> Earlier than I’d have personally picked, but I’m +1 too
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Jeff Jirsa
> >>>>>>
> >>>>>>
> >>>>>> > On Apr 4, 2018, at 5:06 PM, Nate McCall <zznat...@gmail.com>
> >>>>> wrote:
> >>>>>> >
> >>>>>> > Top-posting as I think this summary is on point - thanks,
> >>> Scott!
> >>>>> (And
> >>>>>> > great to have you back, btw).
> >>>>>> >
> >>>>>> > It feels to me like we are coalescing on two points:
> >>>>>> > 1. June 1 as a freeze for alpha
> >>>>>> > 2. "Stable" is the new "Exciting" (and the testing and
> >>> dogfooding
> >>>>>> > implied by such before a GA)
> >>>>>> >
> >>>>>> > How do folks feel about the above points?
> >>>>>> >
> >>>>>> >
> >>>>>> >> Re-raising a point made earlier in the thread by Jeff and
> >>> affirmed
> >>>>>> by Josh:
> >>>>>> >>
> >>>>>> >> –––
> >>>>>> >> Jeff:
> >>>>>> >>>> A hard date for a feature freeze makes sense, a hard date
> >>> for a
> >>>>>> release
> >>>>>> >>>> does not.
> >>>>>> >>
> >>>>>> >> Josh:
> >>>>>> >>> Strongly agree. We should also collectively define what
> >>> "Done"
> >>>>>> looks like
> >>>>>> >>> post freeze so we don't end up in bike-shedding hell like we
> >>> have

Re: Roadmap for 4.0

[Jason Brown]

+1

On Wed, Apr 4, 2018 at 8:31 PM, Blake Eggleston 
wrote:

> +1
>
> On 4/4/18, 5:48 PM, "Jeff Jirsa"  wrote:
>
> Earlier than I’d have personally picked, but I’m +1 too
>
>
>
> --
> Jeff Jirsa
>
>
> > On Apr 4, 2018, at 5:06 PM, Nate McCall  wrote:
> >
> > Top-posting as I think this summary is on point - thanks, Scott! (And
> > great to have you back, btw).
> >
> > It feels to me like we are coalescing on two points:
> > 1. June 1 as a freeze for alpha
> > 2. "Stable" is the new "Exciting" (and the testing and dogfooding
> > implied by such before a GA)
> >
> > How do folks feel about the above points?
> >
> >
> >> Re-raising a point made earlier in the thread by Jeff and affirmed
> by Josh:
> >>
> >> –––
> >> Jeff:
>  A hard date for a feature freeze makes sense, a hard date for a
> release
>  does not.
> >>
> >> Josh:
> >>> Strongly agree. We should also collectively define what "Done"
> looks like
> >>> post freeze so we don't end up in bike-shedding hell like we have
> in the
> >>> past.
> >> –––
> >>
> >> Another way of saying this: ensuring that the 4.0 release is of
> high quality is more important than cutting the release on a specific date.
> >>
> >> If we adopt Sylvain's suggestion of freezing features on a "feature
> complete" date (modulo a "definition of done" as Josh suggested), that will
> help us align toward the polish, performance work, and dog-fooding needed
> to feel great about shipping 4.0. It's a good time to start thinking about
> the approaches to testing, profiling, and dog-fooding various contributors
> will want to take on before release.
> >>
> >> I love how Ben put it:
> >>
> >>> An "exciting" 4.0 release to me is one that is stable and usable
> >>> with no perf regressions on day 1 and includes some of the big
> >>> internal changes mentioned previously.
> >>>
> >>> This will set the community up well for some awesome and exciting
> >>> stuff that will still be in the pipeline if it doesn't make it to
> 4.0.
> >>
> >> That sounds great to me, too.
> >>
> >> – Scott
> >
> > 
> -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Roadmap for 4.0

[Jason Brown]

The only additional tickets I'd like to mention are:

https://issues.apache.org/jira/browse/CASSANDRA-13971 - Automatic
certificate management using Vault
- Stefan's Vault integration work. A sub-ticket, CASSANDRA-14102, addresses
encryption at-rest, subsumes CASSANDRA-9633 (SSTable encryption) - which I
doubt I would be able to get to any time this year. It would definitely be
nice to have a clarified encryption/security story for 4.0.

https://issues.apache.org/jira/browse/CASSANDRA-11990 - Address rows rather
than partitions in SASI
- a nice update for SASI, but not critical.

-Jason

On Sat, Mar 31, 2018 at 6:53 PM, Ben Bromhead  wrote:

> Apologies all, I didn't realize I was responding to this discussion only on
> the @user list. One of the perils of responding to a thread that is on both
> user and dev...
>
> For context, I have included my response to Kurt's previous discussion on
> this topic as it only ended up on the user list.
>
> *After some further discussions with folks offline, I'd like to revive this
> discussion. *
>
> *As Kurt mentioned, to keep it simple I if we can simply build consensus
> around what is in for 4.0 and what is out. We can then start the process of
> working off a 4.0 branch towards betas and release candidates. Again as
> Kurt mentioned, assigning a timeline to it right now is difficult, but
> having a firm line in the sand around what features/patches are in, then
> limiting future 4.0 work to bug fixes will give folks a less nebulous
> target to work on. *
>
> *The other thing to mention is that once we have a 4.0 branch to work off,
> we at Instaclustr have a commitment to dogfooding the release candidates on
> our internal staging and internal production workloads before 4.0 becomes
> generally available. I know other folks have similar commitments and simply
> having a 4.0 branch with a clear list of things that are in or out will
> allow everyone to start testing and driving towards a quality release. *
>
> *The other thing is that there are already a large number of changes ready
> for 4.0, I would suggest not recommending tickets for 4.0 that have not yet
> been finished/have outstanding work unless you are the person working on it
> (or are offering to work on it instead) and can get it ready for review in
> a timely fashion. That way we can build a more realistic working target.
> For other major breaking changes, there is always 5.0 or 4.1 or whatever we
> end up doing :)*
>
> Thinking further about it, I would suggest a similar process that was
> applied to releasing 3.0, in order to get to 4.0:
>
>- Clean up ticket labeling. Move tickets unlikely to make it / be worked
>on for 4.0 to something else (e.g. 4.x or whatever).
>- Tickets labeled 4.0 will be the line in the sand, with some trigger
>("done") event where all features not done by a certain event will
> simply
>move into the next release. For the 3.0 branch, this occurred after a
>large review of 8099. For 4.0 it could simply be resolving all current
>blockers/major tickets tagged 4.0... doesn't have to be / nor is it
>something I would strongly advocate.
>- Once we hit this "done" event. Cut a Cassandra-4.0 branch and start
>the alpha/beta/rc cycle from that branch, with only bugfixes going into
>it
>- This, in my mind, is similar to the 3.0 approach
>https://mail-archives.apache.org/mod_mbox/cassandra-dev/
> 201503.mbox/%3CCALdd-zjAyiTbZksMeq2LxGwLF5LPhoi_
> 4vsjy8JBHBRnsxH%3D8A%40mail.gmail.com%3E,
>but without the subsequent tick-tock :)
>
> There are currently 3 open blockers tagged 4.0, some are old and probably
> not really blockers anymore, there are other tickets that may/should be
> blockers on 4.0:
>
>- https://issues.apache.org/jira/browse/CASSANDRA-13951
>- https://issues.apache.org/jira/browse/CASSANDRA-13994
>- https://issues.apache.org/jira/browse/CASSANDRA-12042
>
> In terms of major tickets that I would like to see land:
>
>- https://issues.apache.org/jira/browse/CASSANDRA-7622 Virtual Tables
>- https://issues.apache.org/jira/browse/CASSANDRA-13628 Internode netty
>- https://issues.apache.org/jira/browse/CASSANDRA-13475 Pluggable
> Storage
>- https://issues.apache.org/jira/browse/CASSANDRA-9633 SSTable
> encryption
>
> Ben
>
> On Mon, Feb 12, 2018 at 11:26 PM Jeff Jirsa  wrote:
>
> >
> > Advantages of cutting a release sooner than later:
> > 1) The project needs to constantly progress forward. Releases are the
> most
> > visible part of that.
> > 2) Having a huge changelog in a release increases the likelihood of bugs
> > that take time to find.
> >
> > Advantages of a slower release:
> > 1) We don't do major versions often, and when we do breaking changes
> > (protocol, file format, etc), we should squeeze in as many as possible to
> > avoid having to roll new majors
> > 2) There are probably few people actually running 3.11 at scale, so
> > probably few 

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

I'm coming to be on-board with #3.

One thing to watch out for (we can't account for it now) is how our
dependencies choose to move forward. If we need to upgrade a jar (netty,
for example) due to some leak or vulnerability, and it only runs on a
higher version, we may be forced to upgrade the base java version. Like, I
said we can't possibly foresee these things, and we'll just have to make a
hard decision if the situation arises, but just something to keep in mind.


On Fri, Mar 23, 2018 at 5:39 AM, Josh McKenzie  wrote:

> >
> > 3) Release 4.0 for Java 8, *optionally* branch 4.1 for Java 11 later
>
> This seems like the best of our bad options, with the addition of
> "optionally".
>
>
> On Fri, Mar 23, 2018 at 8:12 AM, Gerald Henriksen 
> wrote:
>
> > On Fri, 23 Mar 2018 04:54:23 +, you wrote:
> >
> > >I think Michael is right. It would be impossible to make everyone follow
> > >such a fast release scheme, and supporting it will be pressured onto the
> > >various distributions, M$ and Apple.
> > >On the other hand https://adoptopenjdk.net has already done a lot of
> the
> > >work and it's already rumoured they may take up backporting of
> > security/bug
> > >fixes. I'd fully expect a lot of users to collaborate around this (or
> > >similar), and there's no reason we couldn't do our part to contribute.
> >
> > A posting on Reddit yesterday from someone from adoptopenjdk claimes
> > that they will be doing LTS releases starting with Java 11, and there
> > should be updates to their website to reflect that soon:
> >
> > https://www.reddit.com/r/java/comments/86ce66/java_long_term_support/
> >
> > So I guess a wait and see to what they commit could be worthwhile.
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

See the legal-discuss@ thread:
https://mail-archives.apache.org/mod_mbox/www-legal-discuss/201803.mbox/browser
.

TL;DR jlink-based distributions are not gonna fly due to OpenJDK's license,
so let's focus on other paths forward.


On Thu, Mar 22, 2018 at 2:04 PM, Carl Mueller <carl.muel...@smartthings.com>
wrote:

> Is OpenJDK really not addressing this at all? Is that because OpenJDK is
> beholden to Oracle somehow? This is a major disservice to Apache and the
> java ecosystem as a whole.
>
> When java was fully open sourced, it was supposed to free the ecosystem to
> a large degree from Oracle. Why is OpenJDK being so uncooperative? Are they
> that resource strapped? Can no one, from consulting empires, Google, IBM,
> Amazon, and a host of other major companies take care of this?
>
> This is basically OpenSSL all over again.
>
> Deciding on a way to get a stable language runtime isn't our job. It's the
> job of either the runtime authors (OpenJDK) or another group that should
> form around it.
>
> There is no looming deadline on this, is there? Can we just let the dust
> settle on this in the overall ecosystem to see what happens? And again,
> what is the Apache Software Foundation's approach to this that affects so
> many of their projects?
>
> On Wed, Mar 21, 2018 at 12:55 PM, Jason Brown <jasedbr...@gmail.com>
> wrote:
>
> > Well, that was quick. TL;DR Redistributing any part of the OpenJDK is
> > basically a no-go.
> >
> > Thus, that option is off the table.
> >
> > On Wed, Mar 21, 2018 at 10:46 AM, Jason Brown <jasedbr...@gmail.com>
> > wrote:
> >
> > > ftr, I've sent a message to legal-discuss to inquire about the
> licensing
> > > aspect of the OpenJDK as we've been discussing. I believe anyone can
> > follow
> > > the thread by subscribing to the legal-discuss@ ML, or you can wait
> for
> > > updates on this thread as I get them.
> > >
> > > On Wed, Mar 21, 2018 at 9:49 AM, Jason Brown <jasedbr...@gmail.com>
> > wrote:
> > >
> > >> If we went down this path, I can't imagine we would build OpenJDK
> > >> ourselves, but probably build a release with jlink or javapackager. I
> > >> haven't done homework on that yet, but i *think* it uses a blessed
> > OpenJDK
> > >> release for the packaging (or perhaps whatever JDK you happen to be
> > >> compiling/building with). Thus as long as we build/release when an
> > openJDK
> > >> rev is released, we would hypothetically be ok from a secutiry POV.
> > >>
> > >> That being said, Micke's points about multiple architectures and other
> > >> OSes (Windows for sure, macOS not so sure) are a legit concern as
> those
> > >> would need to be separate packages, with separate CI/testing and so on
> > :(
> > >>
> > >> I'm not sure betting the farm on linux disto support is the path to
> > >> happiness, either. Not everyone uses one of the distros mentioned (RH,
> > >> ubuntu), nor does everyone use linux (sure, the vast majority is
> > Linux/x86,
> > >> but we do support Windows deployment and macOS development).
> > >>
> > >> -Jason
> > >>
> > >>
> > >>
> > >> On Wed, Mar 21, 2018 at 9:26 AM, Michael Burman <mibur...@redhat.com>
> > >> wrote:
> > >>
> > >>> On 03/21/2018 04:52 PM, Josh McKenzie wrote:
> > >>>
> > >>> This would certainly mitigate a lot of the core problems with the new
> > >>>> release model. Has there been any public statements of plans/intent
> > >>>> with regards to distros doing this?
> > >>>>
> > >>> Since the latest official LTS version is Java 8, that's the only one
> > >>> with publicly available information For RHEL, OpenJDK8 will receive
> > updates
> > >>> until October 2020.  "A major version of OpenJDK is supported for a
> > period
> > >>> of six years from the time that it is first introduced in any version
> > of
> > >>> RHEL, or until the retirement date of the underlying RHEL platform ,
> > >>> whichever is earlier." [1]
> > >>>
> > >>> [1] https://access.redhat.com/articles/1299013
> > >>>
> > >>> In terms of the burden of bugfixes and security fixes if we bundled a
> > >>>> JRE w/C*, cutting a patch release of C* with a new JRE distribution
> > >>>> would be a really low friction process (add to 

Re: Paying off tech debt and correctly naming things

[Jason Brown]

Jon,

Thanks for bringing up this topic. I'll admit that I've been around this
code base for long enough, and have enough accumulated history, that I
probably can't fully appreciate the impact for a newcomer wrt naming.
However, as Josh points out, this situation probably happens to "every
non-trivially aged code-base ever".

One thing I'd like to add is that with these types of large refactoring
changes, the review effort is non-trivial. This is because the review still
has to ensure that correctness is preserved and it's easy to overlook a
seemingly innocuous change.

That being said, I am supportive of this effort. However, I believe it's
going to be best, for contributor and reviewer, to break it up into
smaller, more digestible pieces. I'd also like to request that we not go
whole hog and try to do everything in a compressed time frame; reviewer
availability is already stretched thin and I'm afraid of deepening the
review queue, especially mine :)

Thanks,

-Jason




On Thu, Mar 22, 2018 at 6:41 AM, Josh McKenzie  wrote:

> > Some of us have big patches in flight, things that actually
> > pay off some technical debt, and dealing with such renames is rebase
> hell :\
> For sure, but with a code-base this old / organically grown, I expect
> this will always be the case. If we're talking something as simple as
> an intellij rename refactor, while menial, couldn't someone with a
> giant patch just do the same thing on their side and spend half an
> hour of their life clicking next? ;)
>
> > That said, there is good time for such renames - it’s during
> > those major refactors and rewrites. When you are
> > changing a subsystem, might as well do the appropriate renames.
> Does that hold true for a code-base with as much static state and
> abstraction leaking / bad factoring as we have? (i.e. every
> non-trivially aged code-base ever) The combined approach takes a
> change that's already non-trivially dealing with complex subsystem
> changes and injects a bunch of trivial renaming noise across unrelated
> subsystems into the signal of an actual logic refactor.
>
> On Thu, Mar 22, 2018 at 9:31 AM, Aleksey Yeshchenko 
> wrote:
> > Poor and out-of-date naming of things is probably the least serious part
> of our technical debt. Bad factoring, and straight-up
> > poorly written components is where it’s really at.
> >
> > Doing a big rename for rename sake alone does more harm than it is good,
> sometimes. Some of us have big patches
> > in flight, things that actually pay off some technical debt, and dealing
> with such renames is rebase hell :\
> >
> > That said, there is good time for such renames - it’s during those major
> refactors and rewrites. When you are
> > changing a subsystem, might as well do the appropriate renames.
> >
> > —
> > AY
> >
> > On 20 March 2018 at 22:04:48, Jon Haddad (j...@jonhaddad.com) wrote:
> >
> > Whenever I hop around in the codebase, one thing that always manages to
> slow me down is needing to understand the context of the variable names
> that I’m looking at. We’ve now removed thrift the transport, but the
> variables, classes and comments still remain. Personally, I’d like to go in
> and pay off as much technical debt as possible by refactoring the code to
> be as close to CQL as possible. Rows should be rows, not partitions, I’d
> love to see the term column family removed forever in favor of always using
> tables. That said, it’s a big task. I did a quick refactor in a branch,
> simply changing the ColumnFamilyStore class to TableStore, and pushed it up
> to GitHub. [1]
> >
> > Didn’t click on the link? That’s ok. The TL;DR is that it’s almost 2K
> LOC changed across 275 files. I’ll note that my branch doesn’t change any
> of the almost 1000 search results of “columnfamilystore” found in the
> codebase and hundreds of tests failed on my branch in CircleCI, so that 2K
> LOC change would probably be quite a bit bigger. There is, of course, a lot
> more than just renaming this one class, there’s thousands of variable names
> using any manner of “cf”, “cfs”, “columnfamily”, names plus comments and
> who knows what else. There’s lots of references in probably every file that
> would have to get updated.
> >
> > What are people’s thoughts on this? We should be honest with ourselves
> and know this isn’t going to get any easier over time. It’s only going to
> get more confusing for new people to the project, and having to figure out
> “what kind of row am i even looking at” is a waste of time. There’s
> obviously a much bigger impact than just renaming a bunch of files, there’s
> any number of patches and branches that would become outdated, plus anyone
> pulling in Cassandra as a dependency would be affected. I don’t really have
> a solution for the disruption other than “leave it in place”, but in my
> mind that’s not a great (or even good) solution.
> >
> > Anyways, enough out of me. My concern for ergonomics and naming might be
> 

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

Well, that was quick. TL;DR Redistributing any part of the OpenJDK is
basically a no-go.

Thus, that option is off the table.

On Wed, Mar 21, 2018 at 10:46 AM, Jason Brown <jasedbr...@gmail.com> wrote:

> ftr, I've sent a message to legal-discuss to inquire about the licensing
> aspect of the OpenJDK as we've been discussing. I believe anyone can follow
> the thread by subscribing to the legal-discuss@ ML, or you can wait for
> updates on this thread as I get them.
>
> On Wed, Mar 21, 2018 at 9:49 AM, Jason Brown <jasedbr...@gmail.com> wrote:
>
>> If we went down this path, I can't imagine we would build OpenJDK
>> ourselves, but probably build a release with jlink or javapackager. I
>> haven't done homework on that yet, but i *think* it uses a blessed OpenJDK
>> release for the packaging (or perhaps whatever JDK you happen to be
>> compiling/building with). Thus as long as we build/release when an openJDK
>> rev is released, we would hypothetically be ok from a secutiry POV.
>>
>> That being said, Micke's points about multiple architectures and other
>> OSes (Windows for sure, macOS not so sure) are a legit concern as those
>> would need to be separate packages, with separate CI/testing and so on :(
>>
>> I'm not sure betting the farm on linux disto support is the path to
>> happiness, either. Not everyone uses one of the distros mentioned (RH,
>> ubuntu), nor does everyone use linux (sure, the vast majority is Linux/x86,
>> but we do support Windows deployment and macOS development).
>>
>> -Jason
>>
>>
>>
>> On Wed, Mar 21, 2018 at 9:26 AM, Michael Burman <mibur...@redhat.com>
>> wrote:
>>
>>> On 03/21/2018 04:52 PM, Josh McKenzie wrote:
>>>
>>> This would certainly mitigate a lot of the core problems with the new
>>>> release model. Has there been any public statements of plans/intent
>>>> with regards to distros doing this?
>>>>
>>> Since the latest official LTS version is Java 8, that's the only one
>>> with publicly available information For RHEL, OpenJDK8 will receive updates
>>> until October 2020.  "A major version of OpenJDK is supported for a period
>>> of six years from the time that it is first introduced in any version of
>>> RHEL, or until the retirement date of the underlying RHEL platform ,
>>> whichever is earlier." [1]
>>>
>>> [1] https://access.redhat.com/articles/1299013
>>>
>>> In terms of the burden of bugfixes and security fixes if we bundled a
>>>> JRE w/C*, cutting a patch release of C* with a new JRE distribution
>>>> would be a really low friction process (add to build, check CI, green,
>>>> done), so I don't think that would be a blocker for the concept.
>>>>
>>>> And do we have someone actively monitoring CVEs for this? Would we ship
>>> a version of OpenJDK which ensures that it works with all the major
>>> distributions? Would we run tests against all the major distributions for
>>> each of the OpenJDK version we would ship after each CVE with each
>>> Cassandra version? Who compiles the OpenJDK distribution we would create
>>> (which wouldn't be the official one if we need to maintain support for each
>>> distribution we support) ? What if one build doesn't work for one distro?
>>> Would we not update that CVE? OpenJDK builds that are in the distros are
>>> not necessarily the pure ones from the upstream, they might include patches
>>> that provide better support for the distribution - or even fix bugs that
>>> are not yet in the upstream version.
>>>
>>> I guess we also need the Windows versions, maybe the PowerPC & ARM
>>> versions also at some point. I'm not sure if we plan to support J9 or other
>>> JVMs at some point.
>>>
>>> We would also need to create CVE reports after each Java CVE for
>>> Cassandra as well I would assume since it would affect us separately (and
>>> updating only the Java wouldn't help).
>>>
>>> To me this sounds like an understatement of the amount of work that
>>> would go to this. Not to mention the bad publicity if Java CVEs are not
>>> actually patched instantly in the Cassandra also (and then each user would
>>> have to validate that the shipped version actually works with their
>>> installation in their hardware since they won't get support for it from the
>>> vendors as it's unofficial package).
>>>
>>>   - Micke
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>>
>>>
>>
>

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

ftr, I've sent a message to legal-discuss to inquire about the licensing
aspect of the OpenJDK as we've been discussing. I believe anyone can follow
the thread by subscribing to the legal-discuss@ ML, or you can wait for
updates on this thread as I get them.

On Wed, Mar 21, 2018 at 9:49 AM, Jason Brown <jasedbr...@gmail.com> wrote:

> If we went down this path, I can't imagine we would build OpenJDK
> ourselves, but probably build a release with jlink or javapackager. I
> haven't done homework on that yet, but i *think* it uses a blessed OpenJDK
> release for the packaging (or perhaps whatever JDK you happen to be
> compiling/building with). Thus as long as we build/release when an openJDK
> rev is released, we would hypothetically be ok from a secutiry POV.
>
> That being said, Micke's points about multiple architectures and other
> OSes (Windows for sure, macOS not so sure) are a legit concern as those
> would need to be separate packages, with separate CI/testing and so on :(
>
> I'm not sure betting the farm on linux disto support is the path to
> happiness, either. Not everyone uses one of the distros mentioned (RH,
> ubuntu), nor does everyone use linux (sure, the vast majority is Linux/x86,
> but we do support Windows deployment and macOS development).
>
> -Jason
>
>
>
> On Wed, Mar 21, 2018 at 9:26 AM, Michael Burman <mibur...@redhat.com>
> wrote:
>
>> On 03/21/2018 04:52 PM, Josh McKenzie wrote:
>>
>> This would certainly mitigate a lot of the core problems with the new
>>> release model. Has there been any public statements of plans/intent
>>> with regards to distros doing this?
>>>
>> Since the latest official LTS version is Java 8, that's the only one with
>> publicly available information For RHEL, OpenJDK8 will receive updates
>> until October 2020.  "A major version of OpenJDK is supported for a period
>> of six years from the time that it is first introduced in any version of
>> RHEL, or until the retirement date of the underlying RHEL platform ,
>> whichever is earlier." [1]
>>
>> [1] https://access.redhat.com/articles/1299013
>>
>> In terms of the burden of bugfixes and security fixes if we bundled a
>>> JRE w/C*, cutting a patch release of C* with a new JRE distribution
>>> would be a really low friction process (add to build, check CI, green,
>>> done), so I don't think that would be a blocker for the concept.
>>>
>>> And do we have someone actively monitoring CVEs for this? Would we ship
>> a version of OpenJDK which ensures that it works with all the major
>> distributions? Would we run tests against all the major distributions for
>> each of the OpenJDK version we would ship after each CVE with each
>> Cassandra version? Who compiles the OpenJDK distribution we would create
>> (which wouldn't be the official one if we need to maintain support for each
>> distribution we support) ? What if one build doesn't work for one distro?
>> Would we not update that CVE? OpenJDK builds that are in the distros are
>> not necessarily the pure ones from the upstream, they might include patches
>> that provide better support for the distribution - or even fix bugs that
>> are not yet in the upstream version.
>>
>> I guess we also need the Windows versions, maybe the PowerPC & ARM
>> versions also at some point. I'm not sure if we plan to support J9 or other
>> JVMs at some point.
>>
>> We would also need to create CVE reports after each Java CVE for
>> Cassandra as well I would assume since it would affect us separately (and
>> updating only the Java wouldn't help).
>>
>> To me this sounds like an understatement of the amount of work that would
>> go to this. Not to mention the bad publicity if Java CVEs are not actually
>> patched instantly in the Cassandra also (and then each user would have to
>> validate that the shipped version actually works with their installation in
>> their hardware since they won't get support for it from the vendors as it's
>> unofficial package).
>>
>>   - Micke
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>
>>
>

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

If we went down this path, I can't imagine we would build OpenJDK
ourselves, but probably build a release with jlink or javapackager. I
haven't done homework on that yet, but i *think* it uses a blessed OpenJDK
release for the packaging (or perhaps whatever JDK you happen to be
compiling/building with). Thus as long as we build/release when an openJDK
rev is released, we would hypothetically be ok from a secutiry POV.

That being said, Micke's points about multiple architectures and other OSes
(Windows for sure, macOS not so sure) are a legit concern as those would
need to be separate packages, with separate CI/testing and so on :(

I'm not sure betting the farm on linux disto support is the path to
happiness, either. Not everyone uses one of the distros mentioned (RH,
ubuntu), nor does everyone use linux (sure, the vast majority is Linux/x86,
but we do support Windows deployment and macOS development).

-Jason



On Wed, Mar 21, 2018 at 9:26 AM, Michael Burman  wrote:

> On 03/21/2018 04:52 PM, Josh McKenzie wrote:
>
> This would certainly mitigate a lot of the core problems with the new
>> release model. Has there been any public statements of plans/intent
>> with regards to distros doing this?
>>
> Since the latest official LTS version is Java 8, that's the only one with
> publicly available information For RHEL, OpenJDK8 will receive updates
> until October 2020.  "A major version of OpenJDK is supported for a period
> of six years from the time that it is first introduced in any version of
> RHEL, or until the retirement date of the underlying RHEL platform ,
> whichever is earlier." [1]
>
> [1] https://access.redhat.com/articles/1299013
>
> In terms of the burden of bugfixes and security fixes if we bundled a
>> JRE w/C*, cutting a patch release of C* with a new JRE distribution
>> would be a really low friction process (add to build, check CI, green,
>> done), so I don't think that would be a blocker for the concept.
>>
>> And do we have someone actively monitoring CVEs for this? Would we ship a
> version of OpenJDK which ensures that it works with all the major
> distributions? Would we run tests against all the major distributions for
> each of the OpenJDK version we would ship after each CVE with each
> Cassandra version? Who compiles the OpenJDK distribution we would create
> (which wouldn't be the official one if we need to maintain support for each
> distribution we support) ? What if one build doesn't work for one distro?
> Would we not update that CVE? OpenJDK builds that are in the distros are
> not necessarily the pure ones from the upstream, they might include patches
> that provide better support for the distribution - or even fix bugs that
> are not yet in the upstream version.
>
> I guess we also need the Windows versions, maybe the PowerPC & ARM
> versions also at some point. I'm not sure if we plan to support J9 or other
> JVMs at some point.
>
> We would also need to create CVE reports after each Java CVE for Cassandra
> as well I would assume since it would affect us separately (and updating
> only the Java wouldn't help).
>
> To me this sounds like an understatement of the amount of work that would
> go to this. Not to mention the bad publicity if Java CVEs are not actually
> patched instantly in the Cassandra also (and then each user would have to
> validate that the shipped version actually works with their installation in
> their hardware since they won't get support for it from the vendors as it's
> unofficial package).
>
>   - Micke
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

fwiw, a naive internet search turned up [1]. tl;dr use the java 9's jlink
(or java8's javapackager) to build a full app+jre package for distribution.

I started digging into the legal aspects, and (trying to) searching
legal-discuss@. May just send an email to them later today to speed up this
discovery process.

[1]
https://steveperkins.com/using-java-9-modularization-to-ship-zero-dependency-native-apps/

On Wed, Mar 21, 2018 at 8:25 AM, Stefan Podkowinski  wrote:

> On 21.03.2018 15:41, Ariel Weisberg wrote:
> > I'm not clear on what building and bundling our own JRE/JDK accomplishes?
>
> If we talk about OpenJDK, there will be only a single Java version
> supported at any time and that is the latest Java version (11, 12, ..).
> There is no overlap between supported versions. Therefor it doesn't
> really make a lot of sense for us to officially support "a few releases
> of the JDK" when we talk about OpenJDK releases. What we'd have to do is
> to keep up with new Java versions by testing them and updating our code
> base if necessary. Keep in mind that branches like 4.0 and 3.11 will
> span several Java versions.
>
> We can do this by communicating a list of branches and corresponding
> Java releases that are officially supported. But we can also just bundle
> and ship the latest OpenJDK release that we know is to be working for
> any Cassandra branch right away, which would avoid any incompatibility
> issues between our releases and JREs installed by the user and is
> probably easier for everyone. But thats pretty much the biggest selling
> point on bundling the JRE, but will probably not happen anyway due to
> the licensing restrictions.
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

Similar to Josh, I like Stefan's idea, a lot. iirc, the Basho folks used to
ship a specific version of erlang with Riak, so it's not a new precedent in
the (nosql) database space.
I'm not sure about the licensing, though, as openjdk is GPLv2 w/ classpath
exception - but I can ask Apache Legal about it.

On Wed, Mar 21, 2018 at 6:10 AM, Josh McKenzie <jmcken...@apache.org> wrote:

> As a gut check, the idea of bundling a JRE with C* appeals to me from
> a "control your variables" perspective. Simplifies quite a bit of this
> problem imo.
>
> On Wed, Mar 21, 2018 at 9:04 AM, Stefan Podkowinski <s...@apache.org>
> wrote:
> > There's also another option, which I just want to mention here for the
> > sake of discussion.
> >
> > Quoting the Oracle Support Roadmap:
> > "Instead of relying on a pre-installed standalone JRE, we encourage
> > application developers to deliver JREs with their applications."
> >
> > I've played around with Java 9 a while ago and also tested creating a
> > self contained JRE using jlink, which you can bundle and ship with your
> > application. So there's a technical solution for that with Java 9. Of
> > course you'd have to clarify licensing issues (OpenJDK is GPLv2 +
> > Classpath exception) first.
> >
> > Bundling a custom JRE along with Cassandra, would be convenient in a way
> > that we can do all the testing against the bundled Java version. We
> > could also switch to a new Java version whenever it fits us. Like e.g.
> > apache-cassandra-4.0.14_openjdk11u321 and two months later release
> > apache-cassandra-4.0.15_openjdk12u123. History has shown that planing
> > and timing new releases isn't always working out for us as expected. I'd
> > rather prefer not having to tightly coordinate our own releases together
> > with OpenJDK releases, if it can be avoided. At the same time I'd like
> > to avoid having users updating to incompatible JREs (think about
> > 8u161/#14173), or have them constantly ask which JRE version to use for
> > which Cassandra version, always with the risk of automatic updates
> > causing unexpected issues. Bundling the JRE may help us with that, as it
> > would become more a matter of testing and getting CI turn green, before
> > we're ready to bundle the next major JRE update, without getting the
> > user involved at all.
> >
> > If you would prefer using a global system JRE, that should still be
> > possible by installing an unbundled Cassandra version, but you'd have to
> > pay attention which Java version to use for which Cassandra release,
> > possibly having to provide patches and do some testing for more recent
> > Cassandra versions, in case of compatibility issues. If we update 3.11
> > to Java 13 in mid 2019, we'd have to provide release candidates that can
> > be used for testing for such incompatibilities by LTS users and have
> > them provide patches, which then have to fully work with Java 13 of
> > course. Otherwise I can't see how to make Oracle/Redhat/IBM/Azul LTS
> > releases work, except on this best effort basis without official support
> > guarantees by us.
> >
> > I'm not too enthusiastic about this perspective. But I wouldn't
> > completely dismiss it either, without talking about all the other
> > options first.
> >
> >
> > On 20.03.2018 22:32, Ariel Weisberg wrote:
> >> Hi,
> >>
> >> Synchronizing with Oracle LTS releases is kind of low value if it's a
> paid offering. But if someone in the community doesn't want to upgrade and
> pays Oracle we don't want to get in the way of that.
> >>
> >> Which is how you end up with what Jordan and ElasticSearch suggest. I'm
> still +1 on that although in my heart of hearts I want  to only support the
> latest OpenJDK on trunk and after we cut a release only change the JDK if
> there is a serious issue.
> >>
> >> It's going to be annoying once we have a serious security or
> correctness issue and we need to move to a later OpenJDK. The majority
> won't be paying Oracle for LTS. I don't think that will happen that often
> though.
> >>
> >> Regards,
> >> Ariel
> >>
> >> If that ends up not working and we find it's a problem to not be getting
> >> On Tue, Mar 20, 2018, at 4:50 PM, Jason Brown wrote:
> >>> Thanks to Hannu and others pointing out that the OracleJDK is a
> >>> *commercial* LTS, and thus not an option. mea culpa for missing the
> >>> "commercial" and just focusing on the "LTS" bit. OpenJDK is is, then.
> >>>
> >>> Stefan's elastic s

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

Thanks to Hannu and others pointing out that the OracleJDK is a
*commercial* LTS, and thus not an option. mea culpa for missing the
"commercial" and just focusing on the "LTS" bit. OpenJDK is is, then.

Stefan's elastic search link is rather interesting. Looks like they are
compiling for both a LTS version as well as the current OpenJDK. They
assume some of their users will stick to a LTS version and some will run
the current version of OpenJDK.

While it's extra work to add JDK version as yet another matrix variable in
addition to our branching, is that something we should consider? Or are we
going to burden maintainers even more? Do we have a choice? Note: I think
this is similar to what Jeremiah is proposed.

@Ariel: Going beyond 3 years could be tricky in the worst case because
bringing in up to 3 years of JDK changes to an older release might mean
some of our dependencies no longer function and now it's not just minor
fixes it's bringing in who knows what in terms of updated dependencies

I'm not sure we have a choice anymore, as we're basically bound to what the
JDK developers choose to do (and we're bound to the JDK ...). However, if
we have the changes necessary for the JDK releases higher than the LTS (if
we following the elastic search model), perhaps it'll be a reasonably
smooth transition?

On Tue, Mar 20, 2018 at 1:31 PM, Jason Brown <jasedbr...@gmail.com> wrote:

> copied directly from dev channel, just to keep with this ML conversation
>
> 08:08:26   Robert Stupp jasobrown: https://www.azul.com/java-
> stable-secure-free-choose-two-three/ and https://blogs.oracle.com/java-
> platform-group/faster-and-easier-use-and-redistribution-of-java-se
> 08:08:38 the 2nd says: "The Oracle JDK will continue as a commercial long
> term support offering"
> 08:08:46 also: http://www.oracle.com/technetwork/java/eol-135779.html
> 08:09:21 the keyword in that cite is "commercial"
> 08:21:21  Michael Shuler a couple more thoughts.. 1) keep C*
> support in step with latest Ubuntu LTS OpenJDK major in main, 2) bundle JRE
> in C* releases? (JDK is not "legal" to bundle)
> 08:23:44  https://www.elastic.co/blog/elasticsearch-
> java-9-and-beyond  - interesting read on that matter
> 08:26:04 can't wait for the infra and CI testing implications.. will be
> lot's of fun ;(
> 08:42:13  Robert Stupp Not sure whether stepping with Ubuntu is
> necessary. It's not so difficult to update apt.source ;)
> 08:42:43 CI ? It just let's your test matrix explode - a litte ;)
> 08:46:48  Michael Shuler yep, we currently `def jdkLabel = 'JDK 1.8
> (latest)'` in job DSL and could easily modify that
>
> On Tue, Mar 20, 2018 at 9:08 AM, Kant Kodali <k...@peernova.com> wrote:
>
>> Java 10 is releasing today!
>>
>> On Tue, Mar 20, 2018 at 9:07 AM, Ariel Weisberg <ar...@weisberg.ws>
>> wrote:
>>
>> > Hi,
>> >
>> > +1 to what Jordan is saying.
>> >
>> > It seems like if we are cutting a release off of trunk we want to make
>> > sure we get N years of supported JDK out of it. For a single LTS
>> release N
>> > could be at most 3 and historically that isn't long enough and it's very
>> > likely we will get < 3 after a release is cut.
>> >
>> > Going beyond 3 years could be tricky in the worst case because bringing
>> in
>> > up to 3 years of JDK changes to an older release might mean some of our
>> > dependencies no longer function and now it's not just minor fixes it's
>> > bringing in who knows what in terms of updated dependencies.
>> >
>> > I think in some cases we are going to need to take a release we have
>> > already cut and make it work with an LTS release that didn't exist when
>> the
>> > release was cut.
>> >
>> > We also need to update how CI works. We should at least build and run a
>> > quick smoke test with the JDKs we are claiming to support and
>> > asynchronously run all the tests on the rather large matrix that now
>> exists.
>> >
>> > Ariel
>> >
>> > On Tue, Mar 20, 2018, at 11:07 AM, Jeremiah Jordan wrote:
>> > > My suggestion would be to keep trunk on the latest LTS by default, but
>> > > with compatibility with the latest release if possible.  Since Oracle
>> > > LTS releases are every 3 years, I would not want to tie us to that
>> > > release cycle?
>> > > So until Java 11 is out that would mean trunk should work under Java
>> 8,
>> > > with the option of being compiled/run under Java 9 or 10.  Once Java
>> 11
>> > > is out we could then switch to 11 only.
>> > >
>> > > -Jeremi

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

copied directly from dev channel, just to keep with this ML conversation

08:08:26   Robert Stupp jasobrown:
https://www.azul.com/java-stable-secure-free-choose-two-three/ and
https://blogs.oracle.com/java-platform-group/faster-and-easier-use-and-redistribution-of-java-se
08:08:38 the 2nd says: "The Oracle JDK will continue as a commercial long
term support offering"
08:08:46 also: http://www.oracle.com/technetwork/java/eol-135779.html
08:09:21 the keyword in that cite is "commercial"
08:21:21  Michael Shuler a couple more thoughts.. 1) keep C* support
in step with latest Ubuntu LTS OpenJDK major in main, 2) bundle JRE in C*
releases? (JDK is not "legal" to bundle)
08:23:44 
https://www.elastic.co/blog/elasticsearch-java-9-and-beyond  - interesting
read on that matter
08:26:04 can't wait for the infra and CI testing implications.. will be
lot's of fun ;(
08:42:13  Robert Stupp Not sure whether stepping with Ubuntu is
necessary. It's not so difficult to update apt.source ;)
08:42:43 CI ? It just let's your test matrix explode - a litte ;)
08:46:48  Michael Shuler yep, we currently `def jdkLabel = 'JDK 1.8
(latest)'` in job DSL and could easily modify that

On Tue, Mar 20, 2018 at 9:08 AM, Kant Kodali <k...@peernova.com> wrote:

> Java 10 is releasing today!
>
> On Tue, Mar 20, 2018 at 9:07 AM, Ariel Weisberg <ar...@weisberg.ws> wrote:
>
> > Hi,
> >
> > +1 to what Jordan is saying.
> >
> > It seems like if we are cutting a release off of trunk we want to make
> > sure we get N years of supported JDK out of it. For a single LTS release
> N
> > could be at most 3 and historically that isn't long enough and it's very
> > likely we will get < 3 after a release is cut.
> >
> > Going beyond 3 years could be tricky in the worst case because bringing
> in
> > up to 3 years of JDK changes to an older release might mean some of our
> > dependencies no longer function and now it's not just minor fixes it's
> > bringing in who knows what in terms of updated dependencies.
> >
> > I think in some cases we are going to need to take a release we have
> > already cut and make it work with an LTS release that didn't exist when
> the
> > release was cut.
> >
> > We also need to update how CI works. We should at least build and run a
> > quick smoke test with the JDKs we are claiming to support and
> > asynchronously run all the tests on the rather large matrix that now
> exists.
> >
> > Ariel
> >
> > On Tue, Mar 20, 2018, at 11:07 AM, Jeremiah Jordan wrote:
> > > My suggestion would be to keep trunk on the latest LTS by default, but
> > > with compatibility with the latest release if possible.  Since Oracle
> > > LTS releases are every 3 years, I would not want to tie us to that
> > > release cycle?
> > > So until Java 11 is out that would mean trunk should work under Java 8,
> > > with the option of being compiled/run under Java 9 or 10.  Once Java 11
> > > is out we could then switch to 11 only.
> > >
> > > -Jeremiah
> > >
> > > On Mar 20, 2018, at 10:48 AM, Jason Brown <jasedbr...@gmail.com>
> wrote:
> > >
> > > >>> Wouldn't that potentially leave us in a situation where we're ready
> > for
> > > > a C* release but blocked waiting on a new LTS cut?
> > > >
> > > > Agreed, and perhaps if we're close enough to a LTS release (say three
> > > > months or less), we could choose to delay (probably with community
> > > > input/vote). If we're a year or two out, then, no, we should not
> wait.
> > I
> > > > think this is what I meant to communicate by "Perhaps we can evaluate
> > this
> > > > over time." (poorly stated, in hindsight)
> > > >
> > > >> On Tue, Mar 20, 2018 at 7:22 AM, Josh McKenzie <
> jmcken...@apache.org>
> > wrote:
> > > >>
> > > >> Need a little clarification on something:
> > > >>
> > > >>> 2) always release cassandra on a LTS version
> > > >> combined with:
> > > >>> 3) keep trunk on the lasest jdk version, assumming we release a
> major
> > > >>> cassandra version close enough to a LTS release.
> > > >>
> > > >> Wouldn't that potentially leave us in a situation where we're ready
> > > >> for a C* release but blocked waiting on a new LTS cut? For example,
> if
> > > >> JDK 9 were the currently supported LTS and trunk was on JDK 11, we'd
> > > >> either have to get trunk to work with 9 or wait for 11 to resolve
&g

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

>> Don't forget that you have to spend bucks to get LTS.

Huh? Is that true? Can you point me to any docs that I may have missed?
That would be an important point to consider.

>> supporting Java 10 should be good enough.

Sure but what about 2 years after we release a major, on a JDK that is 2-4
versions outdated? Are we going to get requests to keep
upgrading/validating all 'live' versions under the JDK flavor of the month?
That's what I'd like to avoid, if it's at all possible.

On Tue, Mar 20, 2018 at 7:56 AM, Robert Stupp <sn...@gmx.de> wrote:

> Don't forget that you have to spend bucks to get LTS.
>
> My hope is that after that Java 9, upcoming releases should be smoother to
> use. I.e. settling the C* release on the Java release that's current at
> that point in time sounds good enough. I.e. my hope is that any C* release
> made for Java X will work with Java X+n (in the foreseeable future). Caveat
> is probably the use of "Unsafe"...
>
> For example, if a major release would be planned for April, supporting
> Java 10 should be good enough. But that C* major release should stay on
> Java 10 and no code that requires a newer Java version must get in.
>
> I'm not sure whether recommending OracleJDK over OpenJDK is required. You
> get some goodies with OracleJDK (CAs for example), sure.
>
>
> On 03/20/2018 03:22 PM, Josh McKenzie wrote:
>
>> Need a little clarification on something:
>>
>> 2) always release cassandra on a LTS version
>>>
>> combined with:
>>
>>> 3) keep trunk on the lasest jdk version, assumming we release a major
>>> cassandra version close enough to a LTS release.
>>>
>> Wouldn't that potentially leave us in a situation where we're ready
>> for a C* release but blocked waiting on a new LTS cut? For example, if
>> JDK 9 were the currently supported LTS and trunk was on JDK 11, we'd
>> either have to get trunk to work with 9 or wait for 11 to resolve
>> that.
>>
>> On Tue, Mar 20, 2018 at 9:32 AM, Jason Brown <jasedbr...@gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>>
>>> TL;DR Oracle has started revving the JDK version much faster, and we need
>>> an agreed upon plan.
>>>
>>> Well, we probably should has this discussion this already by now, but
>>> here
>>> we are. Oracle announced plans to release updated JDK version every six
>>> months, and each new version immediate supercedes the previous in all
>>> ways:
>>> no updates/security fixes to previous versions is the main thing, and
>>> previous versions are EOL'd immediately. In addition, Oracle has planned
>>> parallel LTS versions that will live for three years, and then superceded
>>> by the next LTS; but not immediately EOL'd from what I can tell. Please
>>> see
>>> [1, 2] for Oracle's offical comments about this change ([3] was
>>> particularly useful, imo), [4] and many other postings on the internet
>>> for
>>> discussion/commentary.
>>>
>>> We have a jira [5] where Robert Stupp did most of the work to get us onto
>>> Java 9 (thanks, Robert), but then the announcement of the JDK version
>>> changes happened last fall after Robert had done much of the work on the
>>> ticket.
>>>
>>> Here's an initial proposal of how to move forward. I don't suspect it's
>>> complete, but a decent place to start a conversation.
>>>
>>> 1) receommend OracleJDK over OpenJDK. IIUC from [3], the OpenJDK will
>>> release every six months, and the OracleJDK will release every three
>>> years.
>>> Thus, the OracleJDK is the LTS version, and it just comes from a snapshot
>>> of one of those OpenJDK builds.
>>>
>>> 2) always release cassandra on a LTS version. I don't think we can
>>> reasonably expect operators to update the JDK every six months, on time.
>>> Further, if there are breaking changes to the JDK, we don't want to have
>>> to
>>> update established c* versions due to those changes, every six months.
>>>
>>> 3) keep trunk on the lasest jdk version, assumming we release a major
>>> cassandra version close enough to a LTS release. Currently that seems
>>> reasonable for cassandra 4.0 to be released with java 11 (18.9 LTS)
>>> support. Perhaps we can evaluate this over time.
>>>
>>>
>>> Once we agree on a path forward, *it is impreative that we publish the
>>> decision to the docs* so we can point contributors and operators there,
>>>

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

>> Wouldn't that potentially leave us in a situation where we're ready for
a C* release but blocked waiting on a new LTS cut?

Agreed, and perhaps if we're close enough to a LTS release (say three
months or less), we could choose to delay (probably with community
input/vote). If we're a year or two out, then, no, we should not wait. I
think this is what I meant to communicate by "Perhaps we can evaluate this
over time." (poorly stated, in hindsight)

On Tue, Mar 20, 2018 at 7:22 AM, Josh McKenzie <jmcken...@apache.org> wrote:

> Need a little clarification on something:
>
> > 2) always release cassandra on a LTS version
> combined with:
> > 3) keep trunk on the lasest jdk version, assumming we release a major
> > cassandra version close enough to a LTS release.
>
> Wouldn't that potentially leave us in a situation where we're ready
> for a C* release but blocked waiting on a new LTS cut? For example, if
> JDK 9 were the currently supported LTS and trunk was on JDK 11, we'd
> either have to get trunk to work with 9 or wait for 11 to resolve
> that.
>
> On Tue, Mar 20, 2018 at 9:32 AM, Jason Brown <jasedbr...@gmail.com> wrote:
> > Hi all,
> >
> >
> > TL;DR Oracle has started revving the JDK version much faster, and we need
> > an agreed upon plan.
> >
> > Well, we probably should has this discussion this already by now, but
> here
> > we are. Oracle announced plans to release updated JDK version every six
> > months, and each new version immediate supercedes the previous in all
> ways:
> > no updates/security fixes to previous versions is the main thing, and
> > previous versions are EOL'd immediately. In addition, Oracle has planned
> > parallel LTS versions that will live for three years, and then superceded
> > by the next LTS; but not immediately EOL'd from what I can tell. Please
> see
> > [1, 2] for Oracle's offical comments about this change ([3] was
> > particularly useful, imo), [4] and many other postings on the internet
> for
> > discussion/commentary.
> >
> > We have a jira [5] where Robert Stupp did most of the work to get us onto
> > Java 9 (thanks, Robert), but then the announcement of the JDK version
> > changes happened last fall after Robert had done much of the work on the
> > ticket.
> >
> > Here's an initial proposal of how to move forward. I don't suspect it's
> > complete, but a decent place to start a conversation.
> >
> > 1) receommend OracleJDK over OpenJDK. IIUC from [3], the OpenJDK will
> > release every six months, and the OracleJDK will release every three
> years.
> > Thus, the OracleJDK is the LTS version, and it just comes from a snapshot
> > of one of those OpenJDK builds.
> >
> > 2) always release cassandra on a LTS version. I don't think we can
> > reasonably expect operators to update the JDK every six months, on time.
> > Further, if there are breaking changes to the JDK, we don't want to have
> to
> > update established c* versions due to those changes, every six months.
> >
> > 3) keep trunk on the lasest jdk version, assumming we release a major
> > cassandra version close enough to a LTS release. Currently that seems
> > reasonable for cassandra 4.0 to be released with java 11 (18.9 LTS)
> > support. Perhaps we can evaluate this over time.
> >
> >
> > Once we agree on a path forward, *it is impreative that we publish the
> > decision to the docs* so we can point contributors and operators there,
> > instead of rehashing the same conversation.
> >
> > I look forward to a lively discussion. Thanks!
> >
> > -Jason
> >
> > [1] http://www.oracle.com/technetwork/java/eol-135779.html
> > [2]
> > https://blogs.oracle.com/java-platform-group/faster-and-
> easier-use-and-redistribution-of-java-se
> > [3]
> > https://www.oracle.com/java/java9-screencasts.html?bcid=
> 5582439790001=single-social=events
> > [4]
> > http://blog.joda.org/2018/02/java-9-has-six-weeks-to-live.
> html?utm_source=feedburner_medium=feed_campaign=Feed%3A+
> StephenColebournesBlog+%28Stephen+Colebourne%27s+blog%29
> > [5] https://issues.apache.org/jira/browse/CASSANDRA-9608
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

[DISCUSS] java 9 and the future of cassandra on the jdk

[Jason Brown]

Hi all,


TL;DR Oracle has started revving the JDK version much faster, and we need
an agreed upon plan.

Well, we probably should has this discussion this already by now, but here
we are. Oracle announced plans to release updated JDK version every six
months, and each new version immediate supercedes the previous in all ways:
no updates/security fixes to previous versions is the main thing, and
previous versions are EOL'd immediately. In addition, Oracle has planned
parallel LTS versions that will live for three years, and then superceded
by the next LTS; but not immediately EOL'd from what I can tell. Please see
[1, 2] for Oracle's offical comments about this change ([3] was
particularly useful, imo), [4] and many other postings on the internet for
discussion/commentary.

We have a jira [5] where Robert Stupp did most of the work to get us onto
Java 9 (thanks, Robert), but then the announcement of the JDK version
changes happened last fall after Robert had done much of the work on the
ticket.

Here's an initial proposal of how to move forward. I don't suspect it's
complete, but a decent place to start a conversation.

1) receommend OracleJDK over OpenJDK. IIUC from [3], the OpenJDK will
release every six months, and the OracleJDK will release every three years.
Thus, the OracleJDK is the LTS version, and it just comes from a snapshot
of one of those OpenJDK builds.

2) always release cassandra on a LTS version. I don't think we can
reasonably expect operators to update the JDK every six months, on time.
Further, if there are breaking changes to the JDK, we don't want to have to
update established c* versions due to those changes, every six months.

3) keep trunk on the lasest jdk version, assumming we release a major
cassandra version close enough to a LTS release. Currently that seems
reasonable for cassandra 4.0 to be released with java 11 (18.9 LTS)
support. Perhaps we can evaluate this over time.


Once we agree on a path forward, *it is impreative that we publish the
decision to the docs* so we can point contributors and operators there,
instead of rehashing the same conversation.

I look forward to a lively discussion. Thanks!

-Jason

[1] http://www.oracle.com/technetwork/java/eol-135779.html
[2]
https://blogs.oracle.com/java-platform-group/faster-and-easier-use-and-redistribution-of-java-se
[3]
https://www.oracle.com/java/java9-screencasts.html?bcid=5582439790001=single-social=events
[4]
http://blog.joda.org/2018/02/java-9-has-six-weeks-to-live.html?utm_source=feedburner_medium=feed_campaign=Feed%3A+StephenColebournesBlog+%28Stephen+Colebourne%27s+blog%29
[5] https://issues.apache.org/jira/browse/CASSANDRA-9608

Re: NVIDIA TESLA: The World's Most Advance Data Center GPU's for accelerating demanding HPC workloads

[Jason Brown]

All,

This is completely OFF-TOPIC for this mailing list. Please stop.

-Jason

On Thu, Mar 15, 2018 at 10:09 AM, daemeon reiydelle 
wrote:

> Not so sure they are C* relevant, I build (100's of GPU enabled node) HPC's
> that use them for ML, AI, Graph analytics, etc. with the sources in C* or
> more typically Hadoop/EMR data.
>
>
> <==>
> "Who do you think made the first stone spear? The Asperger guy.
> If you get rid of the autism genetics, there would be no Silicon Valley"
> Temple Grandin
>
>
> *Daemeon C.M. ReiydelleSan Francisco 1.415.501.0198London 44 020 8144 9872*
>
>
> On Thu, Mar 15, 2018 at 8:40 AM, Kenneth Brotman <
> kenbrot...@yahoo.com.invalid> wrote:
>
> > I see things like this
> > https://www.nvidia.com/en-us/data-center/tesla/#section3 as something I
> > might be using in things I help build.  Does anyone have any experience
> > with
> > them?
> >
> >
> >
> > Kenneth Brotman
> >
> >
>

Re: Making RF4 useful aka primary and secondary ranges

[Jason Brown]

I feel like we've had a very similar conversation (not so) recently:
https://lists.apache.org/thread.html/9952c419398a1a2f22e2887e3492f9d6899365f0ea7c2b68d6fbe0d4@%3Cuser.cassandra.apache.org%3E

Which led to the creation of this JIRA:
https://issues.apache.org/jira/browse/CASSANDRA-13645


On Wed, Mar 14, 2018 at 4:23 PM, Carl Mueller 
wrote:

> Since this is basically driver syntactic sugar... Yes I'll try that.
>
>
> On Wed, Mar 14, 2018 at 5:59 PM, Jonathan Haddad 
> wrote:
>
> > You could use a load balancing policy at the driver level to do what you
> > want, mixed with the existing consistency levels as Jeff suggested.
> >
> > On Wed, Mar 14, 2018 at 3:47 PM Carl Mueller <
> carl.muel...@smartthings.com
> > >
> > wrote:
> >
> > > But we COULD have CL2 write (for RF4)
> > >
> > > The extension to this idea is multiple backup/secondary replicas. So
> you
> > > have RF5 or RF6 or higher, but still are performing CL2 against the
> > > preferred first three for both read and write.
> > >
> > > You could also ascertain the general write health of affected ranges
> > before
> > > taking a node down for maintenance from the primary, and then know the
> > > switchover is in good shape. Yes there are CAP limits and race
> conditions
> > > there, but you could get pretty good assurances (all repaired, low/zero
> > > queued hinted handoffs, etc).
> > >
> > > This is essentially like if you had two datacenters, but are doing
> > > local_quorum on the one datacenter. Well, except switchover is a bit
> more
> > > granular if you run out of replicas in the local.
> > >
> > >
> > >
> > > On Wed, Mar 14, 2018 at 5:17 PM, Jeff Jirsa  wrote:
> > >
> > > > Write at CL 3 and read at CL 2
> > > >
> > > > --
> > > > Jeff Jirsa
> > > >
> > > >
> > > > > On Mar 14, 2018, at 2:40 PM, Carl Mueller <
> > > carl.muel...@smartthings.com>
> > > > wrote:
> > > > >
> > > > > Currently there is little use for RF4. You're getting the
> > requirements
> > > of
> > > > > QUORUM-3 but only one extra backup.
> > > > >
> > > > > I'd like to propose something that would make RF4 a sort of more
> > > heavily
> > > > > backed up RF3.
> > > > >
> > > > > A lot of this is probably achievable with strictly driver-level
> > logic,
> > > so
> > > > > perhaps it would belong more there.
> > > > >
> > > > > Basically the idea is to have four replicas of the data, but only
> > have
> > > to
> > > > > practically do QUORUM with three nodes. We consider the first three
> > > > > replicas the "primary replicas". On an ongoing basis for QUORUM
> reads
> > > and
> > > > > writes, we would rely on only those three replicas to satisfy
> > > > > two-out-of-three QUORUM. Writes are persisted to the fourth replica
> > in
> > > > the
> > > > > normal manner of cassandra, it just doesn't count towards the
> QUORUM
> > > > write.
> > > > >
> > > > > On reads, with token and node health awareness by the driver, if
> the
> > > > > primaries are all healthy, two-of-three QUORUM is calculated from
> > > those.
> > > > >
> > > > > If however one of the three primaries is down, read QUORUM is a bit
> > > > > different:
> > > > > 1) if the first two replies come from the two remaining primaries
> and
> > > > > agree, the is returned
> > > > > 2) if the first two replies are a primary and the "hot spare" and
> > those
> > > > > agree, that is returned
> > > > > 3) if the primary and hot spare disagree, wait for the next primary
> > to
> > > > > return, and then take the agreement (hopefully) that results
> > > > >
> > > > > Then once the previous primary comes back online, the read quorum
> > goes
> > > > back
> > > > > to preferring that set, with the assuming hinted handoff and repair
> > > will
> > > > > get it back up to snuff.
> > > > >
> > > > > There could also be some mechanism examining the hinted handoff
> > status
> > > of
> > > > > the four to determine when to reactivate the primary that was down.
> > > > >
> > > > > For mutations, one could prefer a "QUORUM plus" that was a quorum
> of
> > > the
> > > > > primaries plus the hot spare.
> > > > >
> > > > > Of course one could do multiple hot spares, so RF5 could still be
> > > treated
> > > > > as RF3 + hot spares.
> > > > >
> > > > > The goal here is more data resiliency but not having to rely on as
> > many
> > > > > nodes for resiliency.
> > > > >
> > > > > Since the data is ring-distributed, the fact there are primary
> owners
> > > of
> > > > > ranges should still be evenly distributed and no hot nodes should
> > > result
> > > >
> > > > 
> -
> > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > > >
> > > >
> > >
> >
>

Re: Timeout unit tests in trunk

[Jason Brown]

All,

As @kjellman pointed out, the timeouts on ViewTest & ViewBuilderTaskTest
are being addressed in CASSANDRA-14194
 (I have a patch,
almost ready to release).

@dikang if you want to refactor those tests for fun, go for it - but note
that the timeouts are currently not due to size.

I have no idea about the BatchMetricsTest, but I can see if it is related
to the others. @dikang, Do you have any details to share about the
failures?

-Jason

On Tue, Feb 27, 2018 at 5:16 PM, Dinesh Joshi <
dinesh.jo...@yahoo.com.invalid> wrote:

> Yes, give it a go. I am not 100% sure if it will help a whole lot but try
> it out and let's see what happens!
> Dinesh
>
> On Tuesday, February 27, 2018, 2:57:41 PM PST, Dikang Gu <
> dikan...@gmail.com> wrote:
>
>  I took some look at the cql3.ViewTest, it seems too big and timeout very
> often. Any objections if I split it into two or multiple tests?
>
> On Tue, Feb 27, 2018 at 1:32 PM, Michael Kjellman 
> wrote:
>
> > well, turns out we already have a jira tracking the MV tests being broken
> > on trunk. they are legit broken :) thanks jaso
> >
> > https://issues.apache.org/jira/browse/CASSANDRA-14194
> >
> > not sure about the batch test timeout there though.. did you debug it at
> > all by chance?
> >
> >
> > On Feb 27, 2018, at 1:27 PM, Michael Kjellman   > kjell...@apple.com>> wrote:
> >
> > hey dikang: just chatted a little bit about this. proposal: let's add the
> > equivalent of @resource_intensive to unit tests too.. and the first one
> is
> > to stop from running the MV unit tests in the free circleci containers.
> > thoughts?
> >
> > also, might want to bug your management to see if you can get some paid
> > circleci resources. it's game changing!
> >
> > best,
> > kjellman
> >
> > On Feb 27, 2018, at 12:12 PM, Dinesh Joshi  > INVALID> wrote:
> >
> > Some tests might require additional resources to spin up the required
> > components. 2 CPU / 4GB might not be sufficient. You may need to bump up
> > the resources to 8CPU / 16GB.
> > Dinesh
> >
> >  On Tuesday, February 27, 2018, 11:24:34 AM PST, Dikang Gu <
> > dikan...@gmail.com> wrote:
> >
> > Looks like there are a few flaky/timeout unit tests in trunk, wondering
> is
> > there anyone looking at them already?
> >
> > testBuildRange - org.apache.cassandra.db.view.ViewBuilderTaskTest
> > testUnloggedPartitionsPerBatch -
> > org.apache.cassandra.metrics.BatchMetricsTest
> > testViewBuilderResume - org.apache.cassandra.cql3.ViewTest
> >
> > https://circleci.com/gh/DikangGu/cassandra/20
> >
> > --
> > Dikang
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > unsubscr...@cassandra.apache.org>
> > For additional commands, e-mail: dev-h...@cassandra.apache.org > dev-h...@cassandra.apache.org>
> >
> >
> >
>
>
> --
> Dikang
>
>

Re: Cassandra Needs to Grow Up by Version Five!

[Jason Brown]

Hi all,

I'd like to deescalate a bit here.

Since this is an Apache and an OSS project, contributions come in many
forms: code, speaking/advocacy, documentation, support, project management,
and so on. None of these things come for free.

Ken, I appreciate you bring up these usability topics; they are certainly
valid concerns. You've mentioned you are working on posting of some sort
that I think will amount to an enumerated list of the topics/issues you
feel need addressing. Some may be simple changes, some may be more
invasive, some we can consider implementing, some not. I look forward to a
positive discussion.

I think what would be best would be for you to complete that list and work
with the community, in a *positive and constructive manner*, towards
getting it done. That is certainly contributing, and contributing in a big
way: project management. Working with the community is going to be the most
beneficial path for everyone.

Ken, if you feel like you'd like some help getting such an initiative
going, and contributing substantively to it (not necessarily in terms of
code) please feel free to reach out to me directly (jasedbr...@gmail.com).

Hoping this leads somewhere positive, that benefits everyone,

-Jason



On Wed, Feb 21, 2018 at 2:53 PM, Kenneth Brotman <
kenbrot...@yahoo.com.invalid> wrote:

> Hi Akash,
>
> I get the part about outside work which is why in replying to Jeff Jirsa I
> was suggesting the big companies could justify taking it on easy enough and
> you know actually pay the people who would be working at it so those people
> could have a life.
>
> The part I don't get is the aversion to usability.  Isn't that what you
> think about when you are coding?  "Am I making this thing I'm building easy
> to use?"  If you were programming for me, we would be constantly talking
> about what we are building and how we can make things easier for users.  If
> I had to fight with a developer, architect or engineer about usability all
> the time, they would be gone and quick.  How do approach programming if you
> aren't trying to make things easy.
>
> Kenneth Brotman
>
> -Original Message-
> From: Akash Gangil [mailto:akashg1...@gmail.com]
> Sent: Wednesday, February 21, 2018 2:24 PM
> To: dev@cassandra.apache.org
> Cc: u...@cassandra.apache.org
> Subject: Re: Cassandra Needs to Grow Up by Version Five!
>
> I would second Jon in the arguments he made. Contributing outside work is
> draining and really requires a lot of commitment. If someone requires
> features around usability etc, just pay for it, period.
>
> On Wed, Feb 21, 2018 at 2:20 PM, Kenneth Brotman <
> kenbrot...@yahoo.com.invalid> wrote:
>
> > Jon,
> >
> > Very sorry that you don't see the value of the time I'm taking for this.
> > I don't have demands; I do have a stern warning and I'm right Jon.
> > Please be very careful not to mischaracterized my words Jon.
> >
> > You suggest I put things in JIRA's, then seem to suggest that I'd be
> > lucky if anyone looked at it and did anything. That's what I figured too.
> >
> > I don't appreciate the hostility.  You will understand more fully in
> > the next post where I'm coming from.  Try to keep the conversation
> civilized.
> > I'm trying or at least so you understand I think what I'm doing is
> > saving your gig and mine.  I really like a lot of people is this group.
> >
> > I've come to a preliminary assessment on things.  Soon the cloud will
> > clear or I'll be gone.  Don't worry.  I'm a very peaceful person and
> > like you I am driven by real important projects that I feel compelled
> > to work on for the good of others.  I don't have time for people to
> > hand hold a database and I can't get stuck with my projects on the wrong
> stuff.
> >
> > Kenneth Brotman
> >
> >
> > -Original Message-
> > From: Jon Haddad [mailto:jonathan.had...@gmail.com] On Behalf Of Jon
> > Haddad
> > Sent: Wednesday, February 21, 2018 12:44 PM
> > To: u...@cassandra.apache.org
> > Cc: dev@cassandra.apache.org
> > Subject: Re: Cassandra Needs to Grow Up by Version Five!
> >
> > Ken,
> >
> > Maybe it’s not clear how open source projects work, so let me try to
> > explain.  There’s a bunch of us who either get paid by someone or
> > volunteer on our free time.  The folks that get paid, (yay!) usually
> > take direction on what the priorities are, and work on projects that
> > directly affect our jobs.  That means that someone needs to care
> > enough about the features you want to work on them, if you’re not going
> to do it yourself.
> >
> > Now as others have said already, please put your list of demands in
> > JIRA, if someone is interested, they will work on it.  You may need to
> > contribute a little more than you’ve done already, be prepared to get
> > involved if you actually want to to see something get done.  Perhaps
> > learning a little more about Cassandra’s internals and the people
> > involved will reveal some of the design decisions and priorities of the
> project.
> >
> 

Re: Release votes

[Jason Brown]

Hi,

I'm ecstatic others are now running the tests and, more importantly, that
we're having the conversation.

I've become convinced we cannot always have 100% green tests. I am reminded
of this [1] blog post from Google when thinking about flaky tests.
The TL;DR is "flakiness happens", to the tune of about 1.5% of all tests
across Google.

I am in no way advocating that we simply turn a blind eye to broken or
flaky tests, or shrug our shoulders
and rubber stamp a vote, but instead to accept it when it reasonably
applies.
To achieve this, we might need to have discussion at vote/release time (if
not sooner) to triage flaky tests, but I see that as a good thing.

Thanks,

-Jason

[1]
https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html



On Fri, Feb 16, 2018 at 12:47 AM, Dinesh Joshi <
dinesh.jo...@yahoo.com.invalid> wrote:

> I'm new to this project and here are my two cents.
> If there are tests that are constantly failing or flaky and you have had
> releases despite their failures, then they're not useful and can be
> disabled. They can always be reenabled if they are in fact valuable. Having
> 100% blue dashboard is not idealistic IMHO. Hardware failures are harder
> but they can be addressed too.
> I could pitch in to fix the noisy tests or just help in other ways to get
> the dashboard to blue.
> Dinesh
> On Thursday, February 15, 2018, 1:14:33 PM PST, Josh McKenzie <
> jmcken...@apache.org> wrote:
>  >
> > We’ve said in the past that we don’t release without green tests. The PMC
> > gets to vote and enforce it. If you don’t vote yes without seeing the
> test
> > results, that enforces it.
>
> I think this is noble and ideal in theory. In practice, the tests take long
> enough, hardware infra has proven flaky enough, and the tests *themselves*
> flaky enough, that there's been a consistent low-level of test failure
> noise that makes separating signal from noise in this context very time
> consuming. Reference 3.11-test-all for example re:noise:
> https://builds.apache.org/view/A-D/view/Cassandra/job/
> Cassandra-3.11-test-all/test/?width=1024=768
>
> Having spearheaded burning test failures to 0 multiple times and have them
> regress over time, my gut intuition is we should have one person as our
> Source of Truth with a less-flaky source for release-vetting CI (dedicated
> hardware, circle account, etc) we can use as a reference to vote on release
> SHA's.
>
> We’ve declared this a requirement multiple times
>
> Declaring things != changed behavior, and thus != changed culture. The
> culture on this project is one of having a constant low level of test
> failure noise in our CI as a product of our working processes. Unless we
> change those (actually block release w/out green board, actually
> aggressively block merge w/any failing tests, aggressively retroactively
> track down test failures on a daily basis and RCA), the situation won't
> improve. Given that this is a volunteer organization / project, that kind
> of daily time investment is a big ask.
>
> On Thu, Feb 15, 2018 at 1:10 PM, Jeff Jirsa  wrote:
>
> > Moving this to it’s own thread:
> >
> > We’ve declared this a requirement multiple times and then we occasionally
> > get a critical issue and have to decide whether it’s worth the delay. I
> > assume Jason’s earlier -1 on attempt 1 was an enforcement of that earlier
> > stated goal.
> >
> > It’s up to the PMC. We’ve said in the past that we don’t release without
> > green tests. The PMC gets to vote and enforce it. If you don’t vote yes
> > without seeing the test results, that enforces it.
> >
> > --
> > Jeff Jirsa
> >
> >
> > > On Feb 15, 2018, at 9:49 AM, Josh McKenzie 
> wrote:
> > >
> > > What would it take for us to get green utest/dtests as a blocking part
> of
> > > the release process? i.e. "for any given SHA, here's a link to the
> tests
> > > that passed" in the release vote email?
> > >
> > > That being said, +1.
> > >
> > >> On Wed, Feb 14, 2018 at 4:33 PM, Nate McCall 
> > wrote:
> > >>
> > >> +1
> > >>
> > >> On Thu, Feb 15, 2018 at 9:40 AM, Michael Shuler <
> mich...@pbandjelly.org
> > >
> > >> wrote:
> > >>> I propose the following artifacts for release as 3.0.16.
> > >>>
> > >>> sha1: 890f319142ddd3cf2692ff45ff28e71001365e96
> > >>> Git:
> > >>> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > >> shortlog;h=refs/tags/3.0.16-tentative
> > >>> Artifacts:
> > >>> https://repository.apache.org/content/repositories/
> > >> orgapachecassandra-1157/org/apache/cassandra/apache-cassandra/3.0.16/
> > >>> Staging repository:
> > >>> https://repository.apache.org/content/repositories/
> > >> orgapachecassandra-1157/
> > >>>
> > >>> Debian and RPM packages are available here:
> > >>> http://people.apache.org/~mshuler
> > >>>
> > >>> *** This release addresses an important fix for CASSANDRA-14092 ***
> > >>>"Max ttl of 20 years will overflow localDeletionTime"
> > >>>

Re: [VOTE] (Take 2) Release Apache Cassandra 3.0.16

[Jason Brown]

Nate, I can do this.

On Thu, Feb 15, 2018 at 2:51 PM, Nate McCall  wrote:

> >
> > My circleci test runs fail on lack of resources and I understand now
> > that Jason used the circleci configuration from the trunk branch to run
> > the 3.0 and 3.11 branches. These branches should get commits for a
> > working CI configuration in-tree.
> >
>
> Quick follow up - does someone have an issue/action item on updating these:
>
> https://github.com/apache/cassandra/blob/cassandra-3.0/circle.yml
> https://github.com/apache/cassandra/blob/cassandra-3.11/circle.yml
>
> Still showing as out of date(?).
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [VOTE] (Take 3) Release Apache Cassandra 3.11.2

[Jason Brown]

I ran the unit tests and they are green:
https://circleci.com/gh/jasobrown/workflows/cassandra/tree/3.11.2-candidate-2

+1. Thanks, Michael.

On Wed, Feb 14, 2018 at 1:34 PM, Nate McCall  wrote:

> +1
>
> On Thu, Feb 15, 2018 at 10:09 AM, Michael Shuler 
> wrote:
> > I propose the following artifacts for release as 3.11.2.
> >
> > sha1: 1d506f9d09c880ff2b2693e3e27fa58c02ecf398
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/3.11.2-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1158/org/apache/cassandra/apache-cassandra/3.11.2/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1158/
> >
> > Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > *** This release addresses an important fix for CASSANDRA-14092 ***
> > "Max ttl of 20 years will overflow localDeletionTime"
> > https://issues.apache.org/jira/browse/CASSANDRA-14092
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt) https://goo.gl/RLZLrR
> > [2]: (NEWS.txt) https://goo.gl/kpnVHp
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [VOTE] (Take 2) Release Apache Cassandra 3.0.16

[Jason Brown]

I ran the unit tests, and all are green:
https://circleci.com/gh/jasobrown/workflows/cassandra/tree/3.0.16-candidate-2

+1. Thank you, Michael.

On Wed, Feb 14, 2018 at 12:51 PM, Brandon Williams  wrote:

> +1
>
> On Wed, Feb 14, 2018 at 2:40 PM, Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 3.0.16.
> >
> > sha1: 890f319142ddd3cf2692ff45ff28e71001365e96
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/3.0.16-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1157/org/apache/cassandra/apache-cassandra/3.0.16/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1157/
> >
> > Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > *** This release addresses an important fix for CASSANDRA-14092 ***
> > "Max ttl of 20 years will overflow localDeletionTime"
> > https://issues.apache.org/jira/browse/CASSANDRA-14092
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt) https://goo.gl/rLj59Z
> > [2]: (NEWS.txt) https://goo.gl/EkrT4G
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [VOTE FAILED] Release Apache Cassandra 3.0.16

[Jason Brown]

Hi Michael,

Which CI environment? iirc, I had several clean runs in circleci with the
latest commits (will run again today).

Thanks,

-Jasoin

On Wed, Feb 14, 2018 at 9:40 AM, Michael Shuler <mich...@pbandjelly.org>
wrote:

> Thanks for the feedback on 3.0.16 tentative release.
>
> Commit 890f319 (current cassandra-3.0 branch HEAD) fails only one test
> (in both standard and -compression suites) in CI for me. This test has
> failed 20 times in the last 20 runs. Test output attached.
>
> Do we wish to fix this before the next cut, since we're here? :)
>
> --
> Kind regards,
> Michael
>
> On 02/14/2018 07:30 AM, Jason Brown wrote:
> > I think we can attempt another build and vote now.
> >
> > On Tue, Feb 13, 2018 at 3:44 PM, Jason Brown <jasedbr...@gmail.com>
> wrote:
> >
> >> CASSANDRA-14219 is committed and tests look clean (
> https://circleci.com/
> >> workflow-run/d0a2622a-e74f-4c46-b0ad-a84ca063736f).
> >>
> >> On Tue, Feb 13, 2018 at 1:47 PM, Brandon Williams <dri...@gmail.com>
> >> wrote:
> >>
> >>> I change my vote to -1 binding as well.
> >>>
> >>> On Tue, Feb 13, 2018 at 3:43 PM, Jason Brown <jasedbr...@gmail.com>
> >>> wrote:
> >>>
> >>>> -1, binding. Unit tests are broken:
> >>>> https://circleci.com/gh/jasobrown/cassandra/451#tests/containers/50
> >>>>
> >>>> Dave ninja-committed 7df36056b12a13b60097b7a9a4f8155a1d02ff62 to
> update
> >>>> some logging messages, which broke ViewComplexTest. The errors like
> >>> this:
> >>>>
> >>>> junit.framework.AssertionFailedError: Expected error message to
> contain
> >>>> 'Cannot drop column a on base table with materialized views', but got
> >>>> 'Cannot drop column a on base table table_21 with materialized views.'
> >>>>
> >>>> Dave has a followup commit, 40148a178bd9b74b731591aa46b4158efb16b742,
> >>>> which
> >>>> fixed a few of the errors, but there are four outstanding failures. I
> >>>> created CASSANDRA-14219 last week, and assigned it to Dave, but he
> might
> >>>> have missed the notification. Dinesh Joshi has a patch that I will
> >>> review
> >>>> ASAP.
> >>>>
> >>>> Michael, is there a link of where you ran the tests? If so, can you
> >>> include
> >>>> it in the future [VOTE] emails?
> >>>>
> >>>> Thanks,
> >>>>
> >>>> -Jason
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Feb 13, 2018 at 11:03 AM, Jon Haddad <j...@jonhaddad.com>
> wrote:
> >>>>
> >>>>> +1
> >>>>>
> >>>>>> On Feb 13, 2018, at 10:52 AM, Josh McKenzie <jmcken...@apache.org>
> >>>>> wrote:
> >>>>>>
> >>>>>> +1
> >>>>>>
> >>>>>> On Feb 13, 2018 9:20 AM, "Marcus Eriksson" <krum...@gmail.com>
> >>> wrote:
> >>>>>>
> >>>>>>> +1
> >>>>>>>
> >>>>>>> On Tue, Feb 13, 2018 at 1:29 PM, Aleksey Yeshchenko <
> >>>> alek...@apple.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> +1
> >>>>>>>>
> >>>>>>>> —
> >>>>>>>> AY
> >>>>>>>>
> >>>>>>>> On 12 February 2018 at 20:31:23, Michael Shuler (
> >>>>> mich...@pbandjelly.org)
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> I propose the following artifacts for release as 3.0.16.
> >>>>>>>>
> >>>>>>>> sha1: 91e83c72de109521074b14a8eeae1309c3b1f215
> >>>>>>>> Git:
> >>>>>>>> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> >>>>>>>> shortlog;h=refs/tags/3.0.16-tentative
> >>>>>>>> Artifacts:
> >>>>>>>> https://repository.apache.org/content/repositories/
> >>>>>>>> orgapachecassandra-1154/org/apache/cassandra/apache-
> >>>> cassandra/3.0.16/
> >>>>>>>> Staging repository:
> >>>>>>>> https://repository.apache.org/content/repositories/
> >>>>>>>> orgapachecassandra-1154/
> >>>>>>>>
> >>>>>>>> Debian and RPM packages are available here:
> >>>>>>>> http://people.apache.org/~mshuler
> >>>>>>>>
> >>>>>>>> *** This release addresses an important fix for CASSANDRA-14092
> >>> ***
> >>>>>>>> "Max ttl of 20 years will overflow localDeletionTime"
> >>>>>>>> https://issues.apache.org/jira/browse/CASSANDRA-14092
> >>>>>>>>
> >>>>>>>> The vote will be open for 72 hours (longer if needed).
> >>>>>>>>
> >>>>>>>> [1]: (CHANGES.txt) https://goo.gl/rLj59Z
> >>>>>>>> [2]: (NEWS.txt) https://goo.gl/EkrT4G
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>>
> >>>>> 
> -
> >>>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >>>>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >>
> >
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>

Re: [VOTE] Release Apache Cassandra 3.0.16

[Jason Brown]

I think we can attempt another build and vote now.

On Tue, Feb 13, 2018 at 3:44 PM, Jason Brown <jasedbr...@gmail.com> wrote:

> CASSANDRA-14219 is committed and tests look clean (https://circleci.com/
> workflow-run/d0a2622a-e74f-4c46-b0ad-a84ca063736f).
>
> On Tue, Feb 13, 2018 at 1:47 PM, Brandon Williams <dri...@gmail.com>
> wrote:
>
>> I change my vote to -1 binding as well.
>>
>> On Tue, Feb 13, 2018 at 3:43 PM, Jason Brown <jasedbr...@gmail.com>
>> wrote:
>>
>> > -1, binding. Unit tests are broken:
>> > https://circleci.com/gh/jasobrown/cassandra/451#tests/containers/50
>> >
>> > Dave ninja-committed 7df36056b12a13b60097b7a9a4f8155a1d02ff62 to update
>> > some logging messages, which broke ViewComplexTest. The errors like
>> this:
>> >
>> > junit.framework.AssertionFailedError: Expected error message to contain
>> > 'Cannot drop column a on base table with materialized views', but got
>> > 'Cannot drop column a on base table table_21 with materialized views.'
>> >
>> > Dave has a followup commit, 40148a178bd9b74b731591aa46b4158efb16b742,
>> > which
>> > fixed a few of the errors, but there are four outstanding failures. I
>> > created CASSANDRA-14219 last week, and assigned it to Dave, but he might
>> > have missed the notification. Dinesh Joshi has a patch that I will
>> review
>> > ASAP.
>> >
>> > Michael, is there a link of where you ran the tests? If so, can you
>> include
>> > it in the future [VOTE] emails?
>> >
>> > Thanks,
>> >
>> > -Jason
>> >
>> >
>> >
>> > On Tue, Feb 13, 2018 at 11:03 AM, Jon Haddad <j...@jonhaddad.com> wrote:
>> >
>> > > +1
>> > >
>> > > > On Feb 13, 2018, at 10:52 AM, Josh McKenzie <jmcken...@apache.org>
>> > > wrote:
>> > > >
>> > > > +1
>> > > >
>> > > > On Feb 13, 2018 9:20 AM, "Marcus Eriksson" <krum...@gmail.com>
>> wrote:
>> > > >
>> > > >> +1
>> > > >>
>> > > >> On Tue, Feb 13, 2018 at 1:29 PM, Aleksey Yeshchenko <
>> > alek...@apple.com>
>> > > >> wrote:
>> > > >>
>> > > >>> +1
>> > > >>>
>> > > >>> —
>> > > >>> AY
>> > > >>>
>> > > >>> On 12 February 2018 at 20:31:23, Michael Shuler (
>> > > mich...@pbandjelly.org)
>> > > >>> wrote:
>> > > >>>
>> > > >>> I propose the following artifacts for release as 3.0.16.
>> > > >>>
>> > > >>> sha1: 91e83c72de109521074b14a8eeae1309c3b1f215
>> > > >>> Git:
>> > > >>> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
>> > > >>> shortlog;h=refs/tags/3.0.16-tentative
>> > > >>> Artifacts:
>> > > >>> https://repository.apache.org/content/repositories/
>> > > >>> orgapachecassandra-1154/org/apache/cassandra/apache-
>> > cassandra/3.0.16/
>> > > >>> Staging repository:
>> > > >>> https://repository.apache.org/content/repositories/
>> > > >>> orgapachecassandra-1154/
>> > > >>>
>> > > >>> Debian and RPM packages are available here:
>> > > >>> http://people.apache.org/~mshuler
>> > > >>>
>> > > >>> *** This release addresses an important fix for CASSANDRA-14092
>> ***
>> > > >>> "Max ttl of 20 years will overflow localDeletionTime"
>> > > >>> https://issues.apache.org/jira/browse/CASSANDRA-14092
>> > > >>>
>> > > >>> The vote will be open for 72 hours (longer if needed).
>> > > >>>
>> > > >>> [1]: (CHANGES.txt) https://goo.gl/rLj59Z
>> > > >>> [2]: (NEWS.txt) https://goo.gl/EkrT4G
>> > > >>>
>> > > >>>
>> > > >>
>> > >
>> > >
>> > > -
>> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
>> > >
>> > >
>> >
>>
>
>

Re: [VOTE] (Take 2) Release Apache Cassandra 3.11.2

[Jason Brown]

CASSANDRA-14234 (which fixes ReadCommandTest) is now committed. I think we
can attempt another build and vote.

Thanks,

-Jason

On Tue, Feb 13, 2018 at 6:16 PM, Jason Brown <jasedbr...@gmail.com> wrote:

> Thank you, Kurt. I'll get to it within the next 12-24 hours (kids'
> homework time in the U.S. now ;) )
>
> -Jason
>
> On Tue, Feb 13, 2018 at 5:10 PM, kurt greaves <k...@instaclustr.com>
> wrote:
>
>> CASSANDRA-14234 <https://issues.apache.org/jira/browse/CASSANDRA-14234>
>> patch
>> in for ReadCommandTest if anyone wants to take a look.
>>
>> On 14 February 2018 at 00:45, kurt greaves <k...@instaclustr.com> wrote:
>>
>> > ​ReadCommandTest is still not passing. I would be a happy maintainer if
>> >> someone could step up and analyze and contribute a patch; I will
>> review.
>> >
>> >
>> > Having a look. Appears to be just a case of using the same CF for
>> multiple
>> > tests and metrics (tombstone histograms) aren't cleared in between.
>> >
>>
>
>

Re: [VOTE] (Take 2) Release Apache Cassandra 3.11.2

[Jason Brown]

Thank you, Kurt. I'll get to it within the next 12-24 hours (kids' homework
time in the U.S. now ;) )

-Jason

On Tue, Feb 13, 2018 at 5:10 PM, kurt greaves  wrote:

> CASSANDRA-14234 
> patch
> in for ReadCommandTest if anyone wants to take a look.
>
> On 14 February 2018 at 00:45, kurt greaves  wrote:
>
> > ​ReadCommandTest is still not passing. I would be a happy maintainer if
> >> someone could step up and analyze and contribute a patch; I will review.
> >
> >
> > Having a look. Appears to be just a case of using the same CF for
> multiple
> > tests and metrics (tombstone histograms) aren't cleared in between.
> >
>

Re: [VOTE] (Take 2) Release Apache Cassandra 3.11.2

[Jason Brown]

CASSANDRA-14219 is committed, and ViewComplexTest is now green (
https://circleci.com/workflow-run/b1456e1c-c775-4fcb-82a1-ad3a00ced0d8).

ReadCommandTest is still not passing. I would be a happy maintainer if
someone could step up and analyze and contribute a patch; I will review.

On Tue, Feb 13, 2018 at 1:47 PM, Brandon Williams <dri...@gmail.com> wrote:

> I change my vote to -1 binding here, too.
>
> On Tue, Feb 13, 2018 at 3:45 PM, Jason Brown <jasedbr...@gmail.com> wrote:
>
> > -1, binding. Unit tests are broken:
> > https://circleci.com/gh/jasobrown/cassandra/452#tests/containers/60
> > <https://circleci.com/gh/jasobrown/cassandra/451#tests/containers/50>
> >
> > Dave ninja-committed 7df36056b12a13b60097b7a9a4f8155a1d02ff62 to update
> > some logging messages, which broke ViewComplexTest. The errors like this:
> >
> > junit.framework.AssertionFailedError: Expected error message to contain
> > 'Cannot drop column a on base table with materialized views', but got
> > 'Cannot drop column a on base table table_21 with materialized views.'
> >
> > Dave has a followup commit, 40148a178bd9b74b731591aa46b4158efb16b742,
> > which
> > fixed a few of the errors, but there are four outstanding failures. I
> > created CASSANDRA-14219 last week, and assigned it to Dave, but he might
> > have missed the notification. Dinesh Joshi has a patch that I will review
> > ASAP.
> >
> > There is also another test which is failing: testCountWithNoDeletedRow -
> > org.apache.cassandra.db.ReadCommandTest
> >
> > junit.framework.AssertionFailedError: expected:<1> but was:<5>
> >
> > I have not investigated that error. It would be great if someone from the
> > community can do so.
> >
> > Michael, is there a link of where you ran the tests? If so, can you
> include
> > it in the future [VOTE] emails?
> >
> > Thanks,
> >
> > -Jason
> >
> >
> > On Tue, Feb 13, 2018 at 10:53 AM, Josh McKenzie <jmcken...@apache.org>
> > wrote:
> >
> > > +1
> > >
> > > On Feb 13, 2018 9:36 AM, "Gary Dusbabek" <gdusba...@gmail.com> wrote:
> > >
> > > > +1
> > > >
> > > > On Mon, Feb 12, 2018 at 9:40 PM, Michael Shuler <
> > mich...@pbandjelly.org>
> > > > wrote:
> > > >
> > > > > I propose the following artifacts for release as 3.11.2.
> > > > >
> > > > > sha1: 8a5e88f635fdb984505a99a553b5799cedccd06d
> > > > > Git:
> > > > > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > > > > shortlog;h=refs/tags/3.11.2-tentative
> > > > > Artifacts:
> > > > > https://repository.apache.org/content/repositories/
> > > > > orgapachecassandra-1156/org/apache/cassandra/apache-
> > cassandra/3.11.2/
> > > > > Staging repository:
> > > > > https://repository.apache.org/content/repositories/
> > > > > orgapachecassandra-1156/
> > > > >
> > > > > Debian and RPM packages are available here:
> > > > > http://people.apache.org/~mshuler
> > > > >
> > > > > *** This release addresses an important fix for CASSANDRA-14092 ***
> > > > > "Max ttl of 20 years will overflow localDeletionTime"
> > > > > https://issues.apache.org/jira/browse/CASSANDRA-14092
> > > > >
> > > > > The vote will be open for 72 hours (longer if needed).
> > > > >
> > > > > [1]: (CHANGES.txt) https://goo.gl/RLZLrR
> > > > > [2]: (NEWS.txt) https://goo.gl/kpnVHp
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] (Take 2) Release Apache Cassandra 3.11.2

[Jason Brown]

-1, binding. Unit tests are broken:
https://circleci.com/gh/jasobrown/cassandra/452#tests/containers/60


Dave ninja-committed 7df36056b12a13b60097b7a9a4f8155a1d02ff62 to update
some logging messages, which broke ViewComplexTest. The errors like this:

junit.framework.AssertionFailedError: Expected error message to contain
'Cannot drop column a on base table with materialized views', but got
'Cannot drop column a on base table table_21 with materialized views.'

Dave has a followup commit, 40148a178bd9b74b731591aa46b4158efb16b742, which
fixed a few of the errors, but there are four outstanding failures. I
created CASSANDRA-14219 last week, and assigned it to Dave, but he might
have missed the notification. Dinesh Joshi has a patch that I will review
ASAP.

There is also another test which is failing: testCountWithNoDeletedRow -
org.apache.cassandra.db.ReadCommandTest

junit.framework.AssertionFailedError: expected:<1> but was:<5>

I have not investigated that error. It would be great if someone from the
community can do so.

Michael, is there a link of where you ran the tests? If so, can you include
it in the future [VOTE] emails?

Thanks,

-Jason


On Tue, Feb 13, 2018 at 10:53 AM, Josh McKenzie 
wrote:

> +1
>
> On Feb 13, 2018 9:36 AM, "Gary Dusbabek"  wrote:
>
> > +1
> >
> > On Mon, Feb 12, 2018 at 9:40 PM, Michael Shuler 
> > wrote:
> >
> > > I propose the following artifacts for release as 3.11.2.
> > >
> > > sha1: 8a5e88f635fdb984505a99a553b5799cedccd06d
> > > Git:
> > > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > > shortlog;h=refs/tags/3.11.2-tentative
> > > Artifacts:
> > > https://repository.apache.org/content/repositories/
> > > orgapachecassandra-1156/org/apache/cassandra/apache-cassandra/3.11.2/
> > > Staging repository:
> > > https://repository.apache.org/content/repositories/
> > > orgapachecassandra-1156/
> > >
> > > Debian and RPM packages are available here:
> > > http://people.apache.org/~mshuler
> > >
> > > *** This release addresses an important fix for CASSANDRA-14092 ***
> > > "Max ttl of 20 years will overflow localDeletionTime"
> > > https://issues.apache.org/jira/browse/CASSANDRA-14092
> > >
> > > The vote will be open for 72 hours (longer if needed).
> > >
> > > [1]: (CHANGES.txt) https://goo.gl/RLZLrR
> > > [2]: (NEWS.txt) https://goo.gl/kpnVHp
> > >
> > >
> >
>

Re: [VOTE] Release Apache Cassandra 3.0.16

[Jason Brown]

-1, binding. Unit tests are broken:
https://circleci.com/gh/jasobrown/cassandra/451#tests/containers/50

Dave ninja-committed 7df36056b12a13b60097b7a9a4f8155a1d02ff62 to update
some logging messages, which broke ViewComplexTest. The errors like this:

junit.framework.AssertionFailedError: Expected error message to contain
'Cannot drop column a on base table with materialized views', but got
'Cannot drop column a on base table table_21 with materialized views.'

Dave has a followup commit, 40148a178bd9b74b731591aa46b4158efb16b742, which
fixed a few of the errors, but there are four outstanding failures. I
created CASSANDRA-14219 last week, and assigned it to Dave, but he might
have missed the notification. Dinesh Joshi has a patch that I will review
ASAP.

Michael, is there a link of where you ran the tests? If so, can you include
it in the future [VOTE] emails?

Thanks,

-Jason



On Tue, Feb 13, 2018 at 11:03 AM, Jon Haddad  wrote:

> +1
>
> > On Feb 13, 2018, at 10:52 AM, Josh McKenzie 
> wrote:
> >
> > +1
> >
> > On Feb 13, 2018 9:20 AM, "Marcus Eriksson"  wrote:
> >
> >> +1
> >>
> >> On Tue, Feb 13, 2018 at 1:29 PM, Aleksey Yeshchenko 
> >> wrote:
> >>
> >>> +1
> >>>
> >>> —
> >>> AY
> >>>
> >>> On 12 February 2018 at 20:31:23, Michael Shuler (
> mich...@pbandjelly.org)
> >>> wrote:
> >>>
> >>> I propose the following artifacts for release as 3.0.16.
> >>>
> >>> sha1: 91e83c72de109521074b14a8eeae1309c3b1f215
> >>> Git:
> >>> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> >>> shortlog;h=refs/tags/3.0.16-tentative
> >>> Artifacts:
> >>> https://repository.apache.org/content/repositories/
> >>> orgapachecassandra-1154/org/apache/cassandra/apache-cassandra/3.0.16/
> >>> Staging repository:
> >>> https://repository.apache.org/content/repositories/
> >>> orgapachecassandra-1154/
> >>>
> >>> Debian and RPM packages are available here:
> >>> http://people.apache.org/~mshuler
> >>>
> >>> *** This release addresses an important fix for CASSANDRA-14092 ***
> >>> "Max ttl of 20 years will overflow localDeletionTime"
> >>> https://issues.apache.org/jira/browse/CASSANDRA-14092
> >>>
> >>> The vote will be open for 72 hours (longer if needed).
> >>>
> >>> [1]: (CHANGES.txt) https://goo.gl/rLj59Z
> >>> [2]: (NEWS.txt) https://goo.gl/EkrT4G
> >>>
> >>>
> >>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [VOTE] Release Apache Cassandra 2.2.12

[Jason Brown]

+1

On Tue, Feb 13, 2018 at 7:55 AM, Jon Haddad  wrote:

> +1
>
>
> > On Feb 13, 2018, at 7:21 AM, Marcus Eriksson  wrote:
> >
> > +1
> >
> > On Tue, Feb 13, 2018 at 4:18 PM, Gary Dusbabek 
> wrote:
> >
> >> +1
> >>
> >> On Mon, Feb 12, 2018 at 2:30 PM, Michael Shuler  >
> >> wrote:
> >>
> >>> I propose the following artifacts for release as 2.2.12.
> >>>
> >>> sha1: 1602e606348959aead18531cb8027afb15f276e7
> >>> Git:
> >>> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> >>> shortlog;h=refs/tags/2.2.12-tentative
> >>> Artifacts:
> >>> https://repository.apache.org/content/repositories/
> >>> orgapachecassandra-1153/org/apache/cassandra/apache-cassandra/2.2.12/
> >>> Staging repository:
> >>> https://repository.apache.org/content/repositories/
> >>> orgapachecassandra-1153/
> >>>
> >>> Debian and RPM packages are available here:
> >>> http://people.apache.org/~mshuler
> >>>
> >>> *** This release addresses an important fix for CASSANDRA-14092 ***
> >>>"Max ttl of 20 years will overflow localDeletionTime"
> >>>https://issues.apache.org/jira/browse/CASSANDRA-14092
> >>>
> >>> The vote will be open for 72 hours (longer if needed).
> >>>
> >>> [1]: (CHANGES.txt) https://goo.gl/QkJeXH
> >>> [2]: (NEWS.txt) https://goo.gl/A4iKFb
> >>>
> >>>
> >>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [VOTE] Release Apache Cassandra 2.1.20

[Jason Brown]

+1

On Tue, Feb 13, 2018 at 7:20 AM, Marcus Eriksson  wrote:

> +1
>
> On Tue, Feb 13, 2018 at 1:30 PM, Aleksey Yeshchenko 
> wrote:
>
> > +1
> >
> > —
> > AY
> >
> > On 12 February 2018 at 20:30:58, Michael Shuler (mich...@pbandjelly.org)
> > wrote:
> >
> > I propose the following artifacts for release as 2.1.20.
> >
> > sha1: b2949439ec62077128103540e42570238520f4ee
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/2.1.20-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1152/org/apache/cassandra/apache-cassandra/2.1.20/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1152/
> >
> > Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > *** This release addresses an important fix for CASSANDRA-14092 ***
> > "Max ttl of 20 years will overflow localDeletionTime"
> > https://issues.apache.org/jira/browse/CASSANDRA-14092
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt) https://goo.gl/5i2nw9
> > [2]: (NEWS.txt) https://goo.gl/i9Fg2u
> >
> >
>

Re: CASSANDRA-14183 review request -> logback upgrade to fix CVE

[Jason Brown]

Ariel,

>> Option 4, upgrade trunk, update NEWS.TXT in prior versions warning about
the vulnerability

+1 to this. I'll check the ticket, as well.

On Tue, Feb 13, 2018 at 9:45 AM, Ariel Weisberg <ar...@weisberg.ws> wrote:

> Hi,
>
> Option 4, upgrade trunk, update NEWS.TXT in prior versions warning about
> the vulnerability.
>
> Ariel
>
> On Tue, Feb 13, 2018, at 12:28 PM, Ariel Weisberg wrote:
> > Hi,
> >
> > So our options are:
> >
> > 1. Ignore it.
> > Most people aren't using this functionality.
> > Most people aren't and shouldn't be exposing the logging port to
> > untrusted networks
> > But everyone loses at defense in depth (or is it breadth) if they use
> > this functionality and someone might expose the port
> >
> > 2. Remove the offending classes from the 1.1.10 jar
> > My crazy idea, break it, but only for the people using the vulnerable
> > functionality. Possibly no one, but probably someone. Maybe they can
> > upgrade it manually for their usage?
> > This also has an issue when working with maven.
> >
> > 3. Upgrade it
> > Definitely going to break some apps according to Michael Shuler.
> > Happened when he tried it.
> >
> > Certainly we can upgrade in trunk? While we are at it come up to the
> > latest version.
> >
> > Ariel
> >
> > On Tue, Feb 13, 2018, at 12:03 PM, Ariel Weisberg wrote:
> > > Hi,
> > >
> > > I don't think the fix is in 1.1.11 looking at the diff between 1.1.11
> > > and 1.2.0 https://github.com/qos-ch/logback/compare/v_1.1.11...v_1.2.0
> > >
> > > I looked at 1.1.11 and 1.1.10 and didn't see it there either.
> > >
> > > When you say stuff broke do you mean stuff not in the dtests or utests?
> > >
> > > Ariel
> > >
> > > On Tue, Feb 13, 2018, at 11:57 AM, Michael Shuler wrote:
> > > > I tried a logback 1.2.x jar update a number of months ago to fix the
> > > > broken log rotation (try setting rotation to a large number - you'll
> > > > find you only get I think it was 10 files, regardless of setting).
> > > >
> > > > Like we've found updating other jars in the past, this seemingly
> > > > "simple" update broke a number of application components, so we
> rolled
> > > > it back and worked out another log rotation method.
> > > >
> > > > Looking at the logback changelog, I cannot tell if version 1.1.11 is
> > > > fixed for this, or if that might be less breakage? There are a pretty
> > > > significant number of API-looking changes from 1.1.3 to 1.2.3, so I
> do
> > > > not wish to break other user's applications, as I have experienced.
> > > >
> > > > I do not think this should block the current releases, unless someone
> > > > wants to do some significant testing and user outreach for
> tentatively
> > > > breaking their applications.
> > > >
> > > > --
> > > > Michael
> > > >
> > > > On 02/13/2018 10:48 AM, Jason Brown wrote:
> > > > > Ariel,
> > > > >
> > > > > If this is a legit CVE, then we would want to patch all the current
> > > > > versions we support - which is 2.1 and higher.
> > > > >
> > > > > Also, is this worth stopping the current open vote for this patch?
> (Not in
> > > > > a place to look at the patch and affects to impacted branches
> right now).
> > > > >
> > > > > Jason
> > > > >
> > > > > On Tue, Feb 13, 2018 at 08:43 Ariel Weisberg <ar...@weisberg.ws>
> wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> Seems like users could conceivably be using the vulnerable
> component. Also
> > > > >> seems like like we need potentially need to do this as far back
> as 2.1?
> > > > >>
> > > > >> Anyone else have an opinion before I commit this? What version to
> start
> > > > >> from?
> > > > >>
> > > > >> Ariel
> > > > >>
> > > > >> On Tue, Feb 13, 2018, at 5:59 AM, Thiago Veronezi wrote:
> > > > >>> Hi dev team,
> > > > >>>
> > > > >>> Sorry to keep bothering you.
> > > > >>>
> > > > >>> This is just a friendly reminder that I would like to 

Re: CASSANDRA-14183 review request -> logback upgrade to fix CVE

[Jason Brown]

Thanks, Michael and Jeremiah. That’s good input.

Ok, let’s not hold up the vote.

On Tue, Feb 13, 2018 at 08:58 Jeremiah D Jordan <jeremiah.jor...@gmail.com>
wrote:

> s/does affect/does not affect/
>
> > On Feb 13, 2018, at 11:57 AM, Jeremiah D Jordan <
> jeremiah.jor...@gmail.com> wrote:
> >
> > I don’t think we need to stop the vote.  This CVE has been around for a
> while (3/13/2017), and does affect any install I have ever seen.  It
> affects users who manually enable some specific logback features using the
> SocketServer or ServerSocketReceiver component which are not used in our
> default settings (or by any install I have ever seen).
> >
> > -Jeremiah
> >
> >> On Feb 13, 2018, at 11:48 AM, Jason Brown <jasedbr...@gmail.com> wrote:
> >>
> >> Ariel,
> >>
> >> If this is a legit CVE, then we would want to patch all the current
> >> versions we support - which is 2.1 and higher.
> >>
> >> Also, is this worth stopping the current open vote for this patch? (Not
> in
> >> a place to look at the patch and affects to impacted branches right
> now).
> >>
> >> Jason
> >>
> >> On Tue, Feb 13, 2018 at 08:43 Ariel Weisberg <ar...@weisberg.ws> wrote:
> >>
> >>> Hi,
> >>>
> >>> Seems like users could conceivably be using the vulnerable component.
> Also
> >>> seems like like we need potentially need to do this as far back as 2.1?
> >>>
> >>> Anyone else have an opinion before I commit this? What version to start
> >>> from?
> >>>
> >>> Ariel
> >>>
> >>> On Tue, Feb 13, 2018, at 5:59 AM, Thiago Veronezi wrote:
> >>>> Hi dev team,
> >>>>
> >>>> Sorry to keep bothering you.
> >>>>
> >>>> This is just a friendly reminder that I would like to contribute to
> this
> >>>> project starting with a fix for CASSANDRA-14183
> >>>> <https://issues.apache.org/jira/browse/CASSANDRA-14183>.
> >>>>
> >>>> []s,
> >>>> Thiago.
> >>>>
> >>>>
> >>>>
> >>>> On Tue, Jan 30, 2018 at 8:05 AM, Thiago Veronezi <thi...@veronezi.org
> >
> >>>> wrote:
> >>>>
> >>>>> Hi dev team,
> >>>>>
> >>>>> Can one of you guys take a look on this jira ticket?
> >>>>> https://issues.apache.org/jira/browse/CASSANDRA-14183
> >>>>>
> >>>>> It has an a patch available for a known security issue with one of
> the
> >>>>> dependencies. It has only with trivial code changes. It should be
> >>>>> straightforward to review it. Any feedback is very welcome.
> >>>>>
> >>>>> Thanks,
> >>>>> Thiago
> >>>>>
> >>>
> >>> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>>
> >>>
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: CASSANDRA-14183 review request -> logback upgrade to fix CVE

[Jason Brown]

Ariel,

If this is a legit CVE, then we would want to patch all the current
versions we support - which is 2.1 and higher.

Also, is this worth stopping the current open vote for this patch? (Not in
a place to look at the patch and affects to impacted branches right now).

Jason

On Tue, Feb 13, 2018 at 08:43 Ariel Weisberg  wrote:

> Hi,
>
> Seems like users could conceivably be using the vulnerable component. Also
> seems like like we need potentially need to do this as far back as 2.1?
>
> Anyone else have an opinion before I commit this? What version to start
> from?
>
> Ariel
>
> On Tue, Feb 13, 2018, at 5:59 AM, Thiago Veronezi wrote:
> > Hi dev team,
> >
> > Sorry to keep bothering you.
> >
> > This is just a friendly reminder that I would like to contribute to this
> > project starting with a fix for CASSANDRA-14183
> > .
> >
> > []s,
> > Thiago.
> >
> >
> >
> > On Tue, Jan 30, 2018 at 8:05 AM, Thiago Veronezi 
> > wrote:
> >
> > > Hi dev team,
> > >
> > > Can one of you guys take a look on this jira ticket?
> > > https://issues.apache.org/jira/browse/CASSANDRA-14183
> > >
> > > It has an a patch available for a known security issue with one of the
> > > dependencies. It has only with trivial code changes. It should be
> > > straightforward to review it. Any feedback is very welcome.
> > >
> > > Thanks,
> > > Thiago
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: duration based config settings

[Jason Brown]

I like this idea, but tbh I hate overburdening operators with what will
seem them as busywork. Operators have scripts and other infrastructure to
automate the creation of yaml files, and the thoughts of "oh crap, why did
they rename all the properties? And, are these really just simple renames,
or is there something subtly different now?" are onerous and worrysome to
them.

If there's a clever, maintainable, and safe way to implement this change
and be somehow backward compatible I could be on board. However, let's
please keep the operators in mind when making these very externally visible
changes.

Thanks,

-Jason

On Mon, Dec 4, 2017 at 5:33 PM, Jon Haddad  wrote:

> Sure, I’m fine w/ letting the _ms settings work indefinitely.  Can revisit
> retiring them if there’s ever a real need, am OK if that never happens.
>
> I’ll create the JIRA.
>
> > On Dec 4, 2017, at 5:19 PM, Nate McCall  wrote:
> >
> >> I'd be in favour of never retiring the _ms format though - it's almost
> >> free, there's no backward compatibility problems, and it's fairly
> intuitive
> >> so long as we're consistent about it.
> >>
> >
> > Agreed on this point. Overall, this will be excellent for usability.
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Stream is failing while removing the node

[Jason Brown]

Varun,

Please open a JIRA ticket with all the details of what you are seeing.

Thanks,

-Jason

On Mon, Oct 9, 2017 at 7:54 PM, Varun Barala 
wrote:

> Hi developers,
>
> Recently, I was removing one node from the cluster without downtime.
>
> Cluster size :: 3 node [test cluster with nodeA, nodeB, nodeC]
>
>
> *Procedure:-*1 Change RF to 1
> 2 Check all the nodes are up
> 3 Run repair on all the nodes
> 4 Run decommission on node nodeC
> 5 After decommission
> 6 Check node status(status should be *UN*)
> 7 Clean up on all the nodes
> * repeat steps[2..7] for nodeB
>
>
> But we found some weird exception in the logs:-
> *level:WARN  thread:STREAM-IN-/10.142.1.11 
> @timestamp:2017-10-06T02:44:21.567+
> hostname:ip-10-142-1-10.compute.internal
> loggerName:org.apache.cassandra.streaming.compress.CompressedStreamReader
> file:CompressedStreamReader.java:115message:[Stream
> cc2f84d0-aa3e-11e7-a2fc-fd45c9a98801] Error while reading partition
> DecoratedKey(-9223372036854775808, ) from stream on ks='ks1' and
> table='cf1'.  throwable:*
> *level:WARN  thread:STREAM-IN-/10.142.1.11 
> @timestamp:2017-10-06T02:44:21.568+
> hostname:ip-10-142-1-10.compute.internal
> loggerName:org.apache.cassandra.streaming.StreamSession
> file:StreamSession.java:626 message:[Stream
> #cc2f84d0-aa3e-11e7-a2fc-fd45c9a98801] Retrying for following error
> throwable:java.lang.AssertionError: null\nat
> org.apache.cassandra.streaming.compress.CompressedInputStream.read(
> CompressedInputStream.java:106)\nat
> java.io.InputStream.read(InputStream.java:170)\nat
> java.io.InputStream.skip(InputStream.java:224)\nat
> org.apache.cassandra.streaming.StreamReader.drain(
> StreamReader.java:158)\nat
> org.apache.cassandra.streaming.compress.CompressedStreamReader.read(
> CompressedStreamReader.java:129)\nat
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(
> IncomingFileMessage.java:48)\nat
> org.apache.cassandra.streaming.messages.IncomingFileMessage$1.deserialize(
> IncomingFileMessage.java:38)\nat
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(
> StreamMessage.java:56)\nat
> org.apache.cassandra.streaming.ConnectionHandler$
> IncomingMessageHandler.run(ConnectionHandler.java:250)\nat
> java.lang.Thread.run(Thread.java:745)\n*
>
> assertion failing at:-
> https://github.com/apache/cassandra/blob/cassandra-2.1.
> 13/src/java/org/apache/cassandra/streaming/compress/
> CompressedInputStream.java#L106
>
> Could someone please tell me in which scenario this happens, Thanks!!
>
> Thanks in advance!!
>
> Regards
> Varun Barala
>

Re: [VOTE] Release Apache Cassandra 3.0.15

[Jason Brown]

+1

On Mon, Oct 2, 2017 at 11:00 Jon Haddad  wrote:

> +1
>
> > On Oct 2, 2017, at 11:00 AM, Brandon Williams  wrote:
> >
> > +1
> >
> > On Oct 2, 2017 12:58 PM, "Aleksey Yeshchenko"  wrote:
> >
> >> +1
> >>
> >> —
> >> AY
> >>
> >> On 2 October 2017 at 18:18:26, Michael Shuler (mich...@pbandjelly.org)
> >> wrote:
> >>
> >> I propose the following artifacts for release as 3.0.15.
> >>
> >> sha1: b32a9e6452c78e6ad08e371314bf1ab7492d0773
> >> Git:
> >> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> >> shortlog;h=refs/tags/3.0.15-tentative
> >> Artifacts:
> >> https://repository.apache.org/content/repositories/
> >> orgapachecassandra-1150/org/apache/cassandra/apache-cassandra/3.0.15/
> >> Staging repository:
> >> https://repository.apache.org/content/repositories/
> >> orgapachecassandra-1150/
> >>
> >> The Debian packages are available here:
> http://people.apache.org/~mshuler
> >>
> >> The vote will be open for 72 hours (longer if needed).
> >>
> >> [1]: (CHANGES.txt) https://goo.gl/RyuPpw
> >> [2]: (NEWS.txt) https://goo.gl/qxwUti
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>
> >>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [VOTE] Release Apache Cassandra 3.11.1

[Jason Brown]

+1

On Mon, Oct 2, 2017 at 11:01 Aleksey Yeshchenko  wrote:

> +1
>
> —
> AY
>
> On 2 October 2017 at 18:58:57, Michael Shuler (mich...@pbandjelly.org)
> wrote:
>
> I propose the following artifacts for release as 3.11.1.
>
> sha1: 983c72a84ab6628e09a78ead9e20a0c323a005af
> Git:
>
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.11.1-tentative
> Artifacts:
>
> https://repository.apache.org/content/repositories/orgapachecassandra-1151/org/apache/cassandra/apache-cassandra/3.11.1/
> Staging repository:
> https://repository.apache.org/content/repositories/orgapachecassandra-1151/
>
> The Debian packages are available here: http://people.apache.org/~mshuler
>
> The vote will be open for 72 hours (longer if needed).
>
> [1]: (CHANGES.txt) https://goo.gl/dZCRk8
> [2]: (NEWS.txt) https://goo.gl/rh24MX
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [VOTE] Release Apache Cassandra 2.2.11

[Jason Brown]

+1

On Thu, Sep 28, 2017 at 12:05 PM, Brandon Williams  wrote:

> +1
>
> On Sep 28, 2017 1:41 PM, "Michael Shuler"  wrote:
>
> > I propose the following artifacts for release as 2.2.11.
> >
> > sha1: c510e001481637e1f74d9ad176f8dc3ab7ebd1e3
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/2.2.11-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1149/org/apache/cassandra/apache-cassandra/2.2.11/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1149/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt) https://goo.gl/qTG7xU (also RPM file ownership fix)
> > [2]: (NEWS.txt) https://goo.gl/ggdkLH
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: Hints replay incompatible between 2.x and 3.x

[Jason Brown]

Hi Andrew,

This question is best for the user@ list, included here.

Thanks,

-Jason

On Wed, Aug 30, 2017 at 10:00 AM, Andrew Whang 
wrote:

> In evaluating 3.x, we found that hints are unable to be replayed between
> 2.x and 3.x nodes. This introduces a risk during the upgrade path for some
> of our write-heavy clusters - nodes will accumulate upwards of 1TB of hints
> if a node goes/remains down for <1hr.
>
> Any suggestions to mitigate this issue?
>

Re: jira rights

[Jason Brown]

Malcolm,

You should be able to comment on and assign tickets to yourself, and all
the standard Jira use stuff. What specific behavior is not working for you?

Thanks

-Jason

On Fri, Jul 21, 2017 at 08:36 Malcolm Taylor  wrote:

> Please grant me (malcolmt) contributor rights for the Cassandra Jira site.
> thanks,
> Malcolm
>

Re: [VOTE] Release Apache Cassandra 2.1.18

[Jason Brown]

+1

On Wed, Jun 21, 2017 at 17:52 Brandon Williams  wrote:

> +1
>
> On Wed, Jun 21, 2017 at 3:18 PM, Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 2.1.18.
> >
> > sha1: 9369db1dfd92d4eb76284cfb68b1ffb9d22c9b06
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/2.1.18-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1146/org/apache/cassandra/apache-cassandra/2.1.18/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1146/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler/
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt) https://goo.gl/MG5EUZ
> > [2]: (NEWS.txt) https://goo.gl/7bi7K1
> >
> >
>

Re: [VOTE] Release Apache Cassandra 2.2.10

[Jason Brown]

+1

On Wed, Jun 21, 2017 at 16:52 Jeff Jirsa  wrote:

> +1
>
> On Wed, Jun 21, 2017 at 1:20 PM, Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 2.2.10.
> >
> > sha1: 83f28ce3c4eeff75ce70855a56b6155047ce8e9a
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/2.2.10-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1147/org/apache/cassandra/apache-cassandra/2.2.10/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1147/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler/
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt) https://goo.gl/N2ZEkF
> > [2]: (NEWS.txt) https://goo.gl/FAv57Z
> >
> >
>

Re: [VOTE] Release Apache Cassandra 3.11.0

[Jason Brown]

+1

On Tue, Jun 20, 2017 at 10:37 AM, Jeff Jirsa  wrote:

> +1
>
>
> On Mon, Jun 19, 2017 at 4:10 PM, Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 3.11.0.
> >
> > sha1: 88dee7e9d515ad94ecf8f2309f1e6138ec79e1a2
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/3.11.0-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1145/org/apache/cassandra/apache-cassandra/3.11.0/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1145/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt) https://goo.gl/x6SzsQ
> > [2]: (NEWS.txt) https://goo.gl/qxG8MQ
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [VOTE] Release Apache Cassandra 3.0.14

[Jason Brown]

+1

On Mon, Jun 19, 2017 at 14:12 Brandon Williams  wrote:

> +1
>
> On Mon, Jun 19, 2017 at 3:37 PM, Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 3.0.14.
> >
> > sha1: f3e38cb638113c2a23855a104d6082da5bc10ddb
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/3.0.14-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1143/org/apache/cassandra/apache-cassandra/3.0.14/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1143/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt) https://goo.gl/VdkHQH
> > [2]: (NEWS.txt) https://goo.gl/Qkknas
> >
> >
>

dtests and differing behavior across c* versions

[Jason Brown]

Hey all,

As I'm finalizing work on CASSANDRA-8457 and CASSANDRA-12229 (switching to
netty for internode messaging and streaming), I need to update the a
handful of existing dtests due to changes in

a) grepping log for errors (primarily for TLS)
b) byteman injections (primarily for streaming)


As we need dtests to be backward compatible with previous c* versions,
what's the recommended strategy for having the updates for the newer
versions but keeping the existing functionality for the old tests?

For the "grepping log for errors" issue, I'm thinking of something like
this (apologies for poor python and short-handing these):


@since('4.0)
def test_XYZ_feature_40_and_up(self):
  error = "new error"
  test_XYZ_feature(error)

@since('3.0', max_version='3.x')
def test_XYZ_feature_before_40(self):
  error = "old error"
  test_XYZ_feature(error)

def test_XYZ_feature(self, error):
  


Does this seem legit, or is there something better that is recommended?

WRT to the byteman changes, should I take a similar approach, where I split
out the byteman scripts for each variant, and then pass that script's file
name into an overloaded function?

Thanks!

-Jason

Re: dtests and differing behavior across c* versions

[Jason Brown]

bah! sent too soon. will finish up and resend asap

On Sat, Jun 17, 2017 at 8:20 AM, Jason Brown <jasedbr...@gmail.com> wrote:

> Hey all,
>
> As I'm finalizing work on CASSANDRA-8457 and CASSANDRA-12229 (switching to
> netty for internode messaging and streaming), I need to update the a
> handful of existing dtests due to changes in
>
> a) byteman injections (primarily for streaming)
> b) grepping log for errors (primarily for TLS)
>
> As we need dtests to be backward compatible with previous c* versions,
> what's the recommended strategy for having the updates for the newer
> versions but keeping the existing functionality for the old tests?
>
> For the "grepping log for errors" issue, I'm thinking of something like
> this (apologies for poor python and short-handing these):
>
> @since('4.0)
> def test_XYZ_feature_40_and_up()
>   error = "new error"
>   test_XYZ_feature(error)
>
> @since('3.0', max_version='3.x')
> def test_XYZ_feature_before_40()
>   error = "old error"
>   test_XYZ_feature(error)
>
> def test_XYZ_feature(error)
>
>

dtests and differing behavior across c* versions

[Jason Brown]

Hey all,

As I'm finalizing work on CASSANDRA-8457 and CASSANDRA-12229 (switching to
netty for internode messaging and streaming), I need to update the a
handful of existing dtests due to changes in

a) byteman injections (primarily for streaming)
b) grepping log for errors (primarily for TLS)

As we need dtests to be backward compatible with previous c* versions,
what's the recommended strategy for having the updates for the newer
versions but keeping the existing functionality for the old tests?

For the "grepping log for errors" issue, I'm thinking of something like
this (apologies for poor python and short-handing these):

@since('4.0)
def test_XYZ_feature_40_and_up()
  error = "new error"
  test_XYZ_feature(error)

@since('3.0', max_version='3.x')
def test_XYZ_feature_before_40()
  error = "old error"
  test_XYZ_feature(error)

def test_XYZ_feature(error)

Re: Is concurrent_batchlog_writes option used/implemented?

[Jason Brown]

Hey Tomas,

Thanks for finding these errors. Unfortunately, those are problems on the
Datastax-hosted documentation, not the docs hosted by the Apache project.
To fix those problems you should contact Datastax (I don't have a URL handy
rn, but if one of the DS folks who follow this list can add one that would
be great).

I can't look right now, but do we have similar documentation on the Apache
docs?

Thanks,

Jason

On Thu, Jun 15, hose2017 at 01:46 Tomas Repik  wrote:

> And yet another glitch in the ob at:
> https://docs.datastax.com/en/cassandra/3.0/cassandra/configuration/configCassandra_yaml.html#configCassandra_yaml__cqlTruncateequest_timeout_in_ms
>
> I guess it should be truncate_timeout_in_ms instead.
>
> Is there a more proper way I should use to report these kind of issues? If
> yes, thanks for giving any directions.
>
> Tomas
>
> - Original Message -
> > Thanks for information I thought this would be the case ...
> >
> > I found another option that is not documented properly:
> > allocate_tokens_for_local_replication_factor [1] option is not found in
> any
> > config file instead the allocate_tokens_for_keyspace option is present. I
> > guess it is the replacement for the former but I can't see it documented
> > anywhere. Thanks for clarification.
> >
> > Tomas
> >
> > [1]
> >
> https://docs.datastax.com/en/cassandra/3.0/cassandra/configuration/configCassandra_yaml.html#configCassandra_yaml__allocate_tokens_for_local_replication_factor
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: NGCC Proposal (Was Re: NGCC?)

[Jason Brown]

Bah! Jeff beat me to responding with essentially the same message. +1 for
Tuesday and the venue.

On Tue, Jun 13, 2017 at 11:44 AM, Jeff Jirsa  wrote:

> Looks great to me - especially the venue.  Date wise, Tuesday (19th) lets
> people fly in on Monday instead of costing a weekend, so selfishly that
> seems better to me.
>
>
>
> On Mon, Jun 12, 2017 at 1:30 PM, Gary Dusbabek 
> wrote:
>
> > ## Cassandra list email
> >
> > Hi everybody,
> >
> > Here are current thoughts about structure and timing. Your feedback is
> > welcome.
> >
> > Date: One of 18, 19, or 22 September 2017. We are aware the Tokyo user
> > group is planning an event the first week in October. We’re doing our
> best
> > to give a buffer there.
> >
> > Venue: After evaluating a few options in San Antonio, the event space at
> > Geekdom seems like a good fit and the right balance of cost and services
> (
> > goo.gl/tViC72).
> >
> > Format: This will be a one day event, starting in the morning and running
> > through the evening. Here is a proposed agenda we can iterate on:
> >
> > * 8-9am - catered breakfast, conversation
> > * 9-12 - presentations, including a coffee/snack break
> > * 12-1:30pm - catered lunch
> > * 1:30-4:00pm - presentations, including a coffee/snack break
> > * 4-5pm - lightning talks
> > * 5-6:30pm - happy hour and a half
> > * 6:30-9:30pm - dinner at a local restaurant
> >
> > We hope to film the presentations and make them available on Youtube.
> >
> > A few have already reached out offering various resources or assistance.
> > Thank you! Eric or I will be reaching out to coordinate as soon as we are
> > ready to finalize plans.
> >
> > Please chime in if you have suggestions, especially those relating to
> > format.
> >
> > Cheers,
> >
> > Gary
> >
>

Re: Long running compaction on huge hint table.

[Jason Brown]

Varun,

This a message better for the user@ ML.

Thanks,

-Jason

On Tue, May 16, 2017 at 3:41 AM, varun saluja  wrote:

> Hi Experts,
>
> We are facing issue on production cluster. Compaction on system.hint table
> is running from last 2 days.
>
>
> pending tasks: 1
>compaction type   keyspace   table completed  total
>   unit   progress
>   Compaction system   hints   20623021829   877874092407
>  bytes  2.35%
> Active compaction remaining time :   0h27m15s
>
>
> Active compaction remaining time shows in minutes.  But, this is job is
> running like indefinitely.
>
> We have 3 node cluster V 2.1.7. And we ran  write intensive job last week
> on particular table.
> Compaction on this table finished but hint table size is growing
> continuously.
>
> Can someone Please help me.
>
>
> Thanks & Regards,
> Varun Saluja
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Integrating vendor-specific code and developing plugins

[Jason Brown]

I agree the plugins route is the safest and best. However, we already have
platform-specific code in-tree that is semi-unmaintained: the Windows
support. To Sylvain's point, I have little to no idea if I'm going to break
the Windows builds as I don't have access to a Windows machine, nor are we
as a community (as best as I can tell) actively running unit tests or
dtests on Windows.

Further, we support snitches for clouds I suspect we don't typically
run/test on, as well: CloudstackSnitch, GoogleCloudSnitch.

This being said, I don't think we should remove support for Windows or
those snitches. Instead, what I think would be more beneficial, and
certainly more reflecting the Apache Way, is to see if someone in the
community would be willing to maintain those components. Looking at another
Apache project, Mesos has an interesting breakdown of maintainers for
specific components [1]. We might consider adopting a similar idea for
platforms/OSes/architectures/whatevers.

As for where to put the custom code, there's a few different options:

bare minimum: we should have docs pointing to all known third party
implementations of pluggable interfaces
slightly more involved: contrib/ section of third-party contributed plugins
even more involved: in tree like gcp / aws snitches

I'm not really thrilled on the contribs repo, and in-tree certainly has
drawbacks, as well. As I initially stated, it can be on a case-by-case
basis.

Honestly, I don't want to push away contributors if they can add something
to the project - as long as it is maintainable.

Thanks,

-Jason


[1] https://mesos.apache.org/documentation/latest/committers/

On Fri, May 12, 2017 at 4:35 AM, Sylvain Lebresne <sylv...@datastax.com>
wrote:

> On Fri, May 12, 2017 at 12:29 AM, Jason Brown <jasedbr...@gmail.com>
> wrote:
>
>> Hey all,
>>
>> I'm on-board with what Rei is saying. I think we should be open to, and
>> encourage, other platforms/architectures for integration. Of course, it
>> will come down to specific maintainers/committers to do the testing and
>> verification on non-typical platforms. Hopefully those maintainers will
>> also contribute to other parts of the code base, as well, so I see this as
>> another way to bring more folks into the project.
>>
>
> Without going so far as to say we shouldn't merge any
> platform/architecture/vendor specific code ever and for no reason, I
> personally think we should avoid doing so as much as practical and
> encourage the "plugins" route instead. It's just much cleaner imo on
> principle and amounts to good software development hygiene.
>
> I don't want to not be able to commit some changes because it breaks the
> build because there is code for X number of super specific
> platform/architecture I don't have access to/don't know anything about
> and the maintainers are on vacation or hasn't been reachable in a while.
> And what if such maintainer do go away? Sure we can have some "process"
> to remove the code in such cases, but why add that burden on us? Plus
> we, the Apache Cassandra project, would still be seen as the ones that
> drop support for said platform/architecture even though we really have
> no choice if it's something we don't have access to anyway.
>
> And sure, I'm painting a bleak picture here, and we would probably have
> none of those problems in most cases. But if we do start encourage
> actual merge of such code, you can be sure we'll have some of those
> problems at some point.
>
> Encouraging plugins have imo pretty much all the benefits with none of
> the risks. In particular, I'm unconvinced that someone will be
> much more likely to meaningfully contribute to other part of the code
> if his "plugins" is in-tree versus out of it.
>
> *But* I can certainly agree with the part about us not having done a
> good job to promote plugins and it make sense to me to try to improve on
> that.
>
>
>>
>> WRT extensibility, it just requires someone to do the work of making
>> reasonable abstraction points - and documenting them ;). The interesting
>> question comes down to how to host/ship any pluggable dependencies. Much
>> like what we had with jna before they relicensed, we'll probably ship some
>> things in-tree and some things users will have to fetch and deploy
>> themselves; it's a case-by-case basis.
>>
>> Thanks,
>>
>> -Jason
>>
>>
>> On Thu, May 11, 2017 at 2:59 PM, 大平怜 <rei.oda...@gmail.com> wrote:
>>
>> > Hi all,
>> >
>> > In this JIRA ticket https://issues.apache.org/jira
>> /browse/CASSANDRA-13486,
>> > we proposed integrating our code to support a fast flash+FPGA card
>> (called
>> > CAPI Flash) only available in the ppc a

Re: CASSANDRA-13444: Fast Streaming Hist

[Jason Brown]

Hey Fedor,

I'll try to make some time in the next week to get to it, as my hands are
all over streaming for 4.0.

Thanks for being patient,

-Jason

On Tue, May 9, 2017 at 11:16 PM, Fedor Bobin 
wrote:

> Hello,
>
> Could someone provide feedback (and possible merge) my PR [1]
> for CASSANDRA-13444?
>
> I rewrote cpu-critical part of Cassandra. Merging this PR will reduce cpu
> and GC consumtion produced by Compaction threads.
>
> You can find more details in issue description [2].
>
> Thanks.
>
> Feodor Bobin
>
> [1] https://github.com/apache/cassandra/pull/106
> [2] https://issues.apache.org/jira/browse/CASSANDRA-13444
>

Re: Integrating vendor-specific code and developing plugins

[Jason Brown]

Hey all,

I'm on-board with what Rei is saying. I think we should be open to, and
encourage, other platforms/architectures for integration. Of course, it
will come down to specific maintainers/committers to do the testing and
verification on non-typical platforms. Hopefully those maintainers will
also contribute to other parts of the code base, as well, so I see this as
another way to bring more folks into the project.

WRT extensibility, it just requires someone to do the work of making
reasonable abstraction points - and documenting them ;). The interesting
question comes down to how to host/ship any pluggable dependencies. Much
like what we had with jna before they relicensed, we'll probably ship some
things in-tree and some things users will have to fetch and deploy
themselves; it's a case-by-case basis.

Thanks,

-Jason


On Thu, May 11, 2017 at 2:59 PM, 大平怜  wrote:

> Hi all,
>
> In this JIRA ticket https://issues.apache.org/jira/browse/CASSANDRA-13486,
> we proposed integrating our code to support a fast flash+FPGA card (called
> CAPI Flash) only available in the ppc architecture. Although we will keep
> discussing the topics specific to the patch (e.g. documentation, license,
> code quality) in the JIRA, we would like to start in this dev list more
> general discussion about how to (and how not to) merge
> architecture-specific (or vendor-specific) changes.
>
> I think in the end the problem boils down to how to test the
> architecture-specific code. The original contributors of the
> architecture-specific code can keep "supporting" the code in a sense that
> when a problem arises they can fix it and send a patch, but the committers
> cannot test it anyway.  Are there any other factors we must consider?
>
> Also, in this particular case, it is relatively easy to turn the code
> change into a plugin because it extended the already-pluggable RowCache.  I
> feel Cassandra has promoted the plugins not so much as other pluggable
> software have done like Eclipse, Apache HTTP server, fluentd, etc.  For
> example, they have a list of plugins in their Web pages.  I think if the
> community wants to encourage developers to maintain vendor-specific code as
> plugins outside of the main source tree, a deeper commitment to the plugin
> ecosystem would be appreciated.
>
> What do you think?
>
>
> Thanks,
> Rei Odaira
>

Re: Soliciting volunteers for flaky dtests on trunk

[Jason Brown]

I've taken
CASSANDRA-13507
CASSANDRA-13517

-Jason


On Wed, May 10, 2017 at 9:45 PM, Lerh Chuan Low 
wrote:

> I'll try my hand on https://issues.apache.org/jira/browse/CASSANDRA-13182.
>
> On 11 May 2017 at 05:59, Blake Eggleston  wrote:
>
> > I've taken CASSANDRA-13194, CASSANDRA-13506, CASSANDRA-13515,
> > and CASSANDRA-13372 to start
> >
> > On May 10, 2017 at 12:44:47 PM, Ariel Weisberg (ar...@weisberg.ws)
> wrote:
> >
> > Hi,
> >
> > The dev list murdered my rich text formatted email. Here it is
> > reformatted as plain text.
> >
> > The unit tests are looking pretty reliable right now. There is a long
> > tail of infrequently failing tests but it's not bad and almost all
> > builds succeed in the current build environment. In CircleCI it seems
> > like unit tests might be a little less reliable, but still usable.
> >
> > The dtests on the other hand aren't producing clean builds yetl. There
> > is also a pretty diverse set of failing tests.
> >
> > I did a bit of triaging of the flakey dtests. I started by cataloging
> > everything, but what I found is that the long tail of flakey dtests is
> > very long indeed so I narrowed focus to just the top frequently failing
> > tests for now. See https://goo.gl/b96CdO
> >
> > I created spreadsheet with some of the failing tests. Links to JIRA,
> > last time the test was seen failing, and how many failures I found in
> > Apache Jenkins across the 3 dtest builds. There are a lot of failures
> > not listed. There would be 50+ entries if I cataloged each one.
> >
> > There are two hard failing tests, but both are already moving along:
> > CASSANDRA-13229 (Ready to commit, assigned Alex Petrov, Paulo Motta
> > reviewing, last updated April 2017) dtest failure in
> > topology_test.TestTopology.size_estimates_multidc_test
> > CASSANDRA-13113 (Ready to commit, assigned Alex Petrov, Sam T Reviewing,
> > last updated March 2017) test failure in
> > auth_test.TestAuth.system_auth_ks_is_alterable_test
> >
> > I think the tests we should tackle first are on this sheet in priority
> > order https://goo.gl/S3khv1
> >
> > Suite: bootstrap_test
> > Test: TestBootstrap.simultaneous_bootstrap_test
> > JIRA: https://issues.apache.org/jira/browse/CASSANDRA-13506
> > Last failure: 5/5/2017
> > Counted failures: 45
> >
> > Suite: repair_test
> > Test: incremental_repair_test.TestIncRepair.compaction_test
> > JIRA: https://issues.apache.org/jira/browse/CASSANDRA-13194
> > Last failure: 5/4/2017
> > Counted failures: 44
> >
> > Suite: sstableutil_test
> > Test: SSTableUtilTest.compaction_test
> > JIRA: https://issues.apache.org/jira/browse/CASSANDRA-13182
> > Last failure: 5/4/2017
> > Counted failures: 35
> >
> > Suite: paging_test
> > Test: TestPagingWithDeletions.test_ttl_deletions
> > JIRA: https://issues.apache.org/jira/browse/CASSANDRA-13507
> > Last failure: 4/25/2017
> > Counted failures: 31
> >
> > Suite: repair_test
> > Test: incremental_repair_test.TestIncRepair.multiple_repair_test
> > JIRA: https://issues.apache.org/jira/browse/CASSANDRA-13515
> > Last failed: 5/4/2017
> > Counted failures: 18
> >
> > Suite: cqlsh_tests
> > Test: cqlsh_copy_tests.CqlshCopyTest.test_bulk_round_trip_*
> > JIRA:
> > https://issues.apache.org/jira/issues/?jql=project%20%
> > 3D%20CASSANDRA%20AND%20status%20in%20(Open%2C%20%22In%
> > 20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22%
> > 2C%20%22Ready%20to%20Commit%22%2C%20%22Awaiting%
> > 20Feedback%22)%20AND%20text%20~%20%22CqlshCopyTest%22
> > Last failed: 5/8/2017
> > Counted failures: 23
> >
> > Suite: paxos_tests
> > Test: TestPaxos.contention_test_many_threads
> > JIRA: https://issues.apache.org/jira/browse/CASSANDRA-13517
> > Last failed: 5/8/2017
> > Counted failures: 15
> >
> > Suite: repair_test
> > Test: TestRepair
> > JIRA:
> > https://issues.apache.org/jira/issues/?jql=status%20%3D%
> > 20Open%20AND%20text%20~%20%22dtest%20failure%20repair_test%22
> > Last failure: 5/4/2017
> > Comment: No one test fails a lot but the number of failing tests is
> > substantial
> >
> > Suite: cqlsh_tests
> > Test: cqlsh_tests.CqlshSmokeTest.[test_insert | test_truncate |
> > test_use_keyspace | test_create_keyspace]
> > JIRA: No JIRA yet
> > Last failed: 4/22/2017
> > count: 6
> >
> > If you have spare cycles you can make a huge difference in test
> > stability by picking off one of these.
> >
> > Regards,
> > Ariel
> >
> > On Wed, May 10, 2017, at 12:45 PM, Ariel Weisberg wrote:
> > > Hi all,
> > >
> > > The unit tests are looking pretty reliable right now. There is a long
> > > tail of infrequently failing tests but it's not bad and almost all
> > > builds succeed in the current build environment. In CircleCI it seems
> > > like unit tests might be a little less reliable, but still usable.
> > > The dtests on the other hand aren't producing clean builds yetl. There
> > > is also a pretty diverse set of failing tests.
> > > I did a bit of triaging of the flakey dtests. I started by cataloging

Re: Cassandra on RocksDB experiment result

[Jason Brown]

I'm +1 on the idea of a pluggable storage engine. There's clearly a
bandwidth problem for developing/reviewing/maintain multiple storage
engines, but I think having the interface is a good thing and can enhance
testability.

At a minimum I think it's worthwhile to explore the storage engine
interface, although it may turn out that it's infeasible/impractical given
the current system. And that's OK.

Thanks,

-Jason


On Thu, Apr 20, 2017 at 4:25 PM, Jeff Jirsa  wrote:

> Let's try to make this actionable. Long time
> contributors/committers/members of the PMC (especially you guys who have
> been working on internals for 4-8 years):
>
> Setting aside details of the implementation, does anyone feel that
> pluggable storage in itself is inherently a bad idea (so much so that you'd
> -1 it if someone else did the work)?
>
> If we can establish loose consensus on it being something generally
> acceptable (assuming someone can come up with an interface/abstraction upon
> which everyone can agree), then it seems like the next step is working on
> defining the proper interface.
>
> - Jeff
>
>
> On Wed, Apr 19, 2017 at 9:21 AM, Dikang Gu  wrote:
>
> > Hi Cassandra developers,
> >
> > This is Dikang from Instagram, I'd like to share you some experiment
> > results we did recently, to use RocksDB as Cassandra's storage engine. In
> > the experiment, I built a prototype to integrate Cassandra 3.0.12 and
> > RocksDB on single column (key-value) use case, shadowed one of our
> > production use case, and saw about 4-6X P99 read latency drop during peak
> > time, compared to 3.0.12. Also, the P99 latency became more predictable
> as
> > well.
> >
> > Here is detailed note with more metrics:
> >
> > https://docs.google.com/document/d/1Ztqcu8Jzh4USKoWBgDJQw82DBurQm
> > sV-PmfiJYvu_Dc/edit?usp=sharing
> >
> > Please take a look and let me know your thoughts. I think the biggest
> > latency win comes from we get rid of most Java garbages created by
> current
> > read/write path and compactions, which reduces the JVM overhead and makes
> > the latency to be more predictable.
> >
> > We are very excited about the potential performance gain. As the next
> step,
> > I propose to make the Cassandra storage engine to be pluggable (like
> Mysql
> > and MongoDB), and we are very interested in providing RocksDB as one
> > storage option with more predictable performance, together with
> community.
> >
> > Thanks.
> >
> > --
> > Dikang
> >
>

Re: [VOTE] Release Apache Cassandra 3.0.13

[Jason Brown]

Alex and Sam T think CASSANDRA-13113 is not a deal breaker in terms of
regression (from reading their comments on the JIRA), and Sam mentioned to
me that it only affects trunk, so I'm going to +1 and will poke on
CASSAADNRA-13113 to get it done as soon as it can be :)

-Jason

On Thu, Apr 13, 2017 at 6:22 AM, Tommy Stendahl  wrote:

> non-binding +1
>
>
>
> On 2017-04-11 20:59, Michael Shuler wrote:
>
>> I propose the following artifacts for release as 3.0.13.
>>
>> sha1: 91661ec296c6d089e3238e1a72f3861c449326aa
>> Git:
>> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=sho
>> rtlog;h=refs/tags/3.0.13-tentative
>> Artifacts:
>> https://repository.apache.org/content/repositories/orgapache
>> cassandra-1142/org/apache/cassandra/apache-cassandra/3.0.13/
>> Staging repository:
>> https://repository.apache.org/content/repositories/orgapache
>> cassandra-1142/
>>
>> The Debian and RPM packages are available here:
>> http://people.apache.org/~mshuler/
>>
>> The vote will be open for 72 hours (longer if needed).
>>
>> [1]: (CHANGES.txt) https://goo.gl/xBbbHa
>> [2]: (NEWS.txt) https://goo.gl/PlsFmm
>>
>>
>

Re: [VOTE] Release Apache Cassandra 3.0.13

[Jason Brown]

+1

On Tue, Apr 11, 2017 at 4:26 PM, Brandon Williams  wrote:

> +1
>
> On Tue, Apr 11, 2017 at 1:59 PM, Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 3.0.13.
> >
> > sha1: 91661ec296c6d089e3238e1a72f3861c449326aa
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/3.0.13-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1142/org/apache/cassandra/apache-cassandra/3.0.13/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1142/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler/
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt) https://goo.gl/xBbbHa
> > [2]: (NEWS.txt) https://goo.gl/PlsFmm
> >
>

Re: [VOTE] self-assignment of jira tickets

[Jason Brown]

With 8 binding +1's, 9 non-binding +1's, and no -1, tis vote passes. I'll
request INFRA to make the change.

Thanks,

-Jason

On Wed, Mar 29, 2017 at 8:58 PM, Ben Bromhead <b...@instaclustr.com> wrote:

> +1 nb
>
> On Wed, 29 Mar 2017 at 12:20 Aleksey Yeschenko <alek...@apache.org> wrote:
>
> > +1 super binding.
> >
> > --
> > AY
> >
> > On 29 March 2017 at 16:22:03, Jason Brown (jasedbr...@gmail.com) wrote:
> >
> > Hey all,
> >
> > Following up my thread from a week or two ago (
> >
> > https://lists.apache.org/thread.html/0665f40c7213654e9981714
> 1972c003a2131aba7a1c63d6765db75c5@%3Cdev.cassandra.apache.org%3E
> > ),
> > I'd like to propose a vote to change to allow any potential contributor
> to
> > assign a jira to themselves without needing to be added to the
> contributors
> > group first.
> >
> > https://issues.apache.org/jira/browse/INFRA-11950 is an example of how
> to
> > get this done with INFRA.
> >
> > Vote will be open for 72 hours.
> >
> > Thanks,
> >
> > -Jason Brown
> >
> --
> Ben Bromhead
> CTO | Instaclustr <https://www.instaclustr.com/>
> +1 650 284 9692
> Managed Cassandra / Spark on AWS, Azure and Softlayer
>

[VOTE] self-assignment of jira tickets

[Jason Brown]

Hey all,

Following up my thread from a week or two ago (
https://lists.apache.org/thread.html/0665f40c7213654e99817141972c003a2131aba7a1c63d6765db75c5@%3Cdev.cassandra.apache.org%3E),
I'd like to propose a vote to change to allow any potential contributor to
assign a jira to themselves without needing to be added to the contributors
group first.

https://issues.apache.org/jira/browse/INFRA-11950 is an example of how to
get this done with INFRA.

Vote will be open for 72 hours.

Thanks,

-Jason Brown

Re: [VOTE] self-assignment of jira tickets

[Jason Brown]

oops, hit sent too early. will send another email shortly

On Wed, Mar 29, 2017 at 6:14 AM, Jason Brown <jasedbr...@gmail.com> wrote:

> Hey all,
>
> Following up on my emai
>

Re: [DISCUSS] Implementing code quality principles, and rules (was: Code quality, principles and rules)

[Jason Brown]

re: code coverage (this could become it's own thread, and probably should)

As somebody working on a new large feature (CASSANDRA-8457), I have a few
thoughts/observations about code coverage. As a preface, though, I'll point
out that we have jacoco (http://www.jacoco.org/jacoco/trunk/index.html)
already incorporated into the build, so it's quite easy to generate code
coverage reports ('ant codecoverage' runs the tests and creates the jacoco
binary, and 'ant jacoco-report' creates an html dump).

As CASSANDRA-8457 is largely additive, it's pretty easy for me to confirm
the code coverage, as it's a whole new package - so clearly it's pretty
obvious to what i didn't cover in the test suite. Contrast this against the
rest of the code base. It may be harder to discern, from a simple
percentage output, just how much of an impact the added tests have across
the entire code base. Further, if a given patch added minor functionality
(say a one-liner fix), plus implementing some toString() functions, you
might actually see the coverage go *down* if the patch doesn't include
tests for the toString().

Also, for things that just can't be unit tested in the current code base,
but where correctness can be ensured with a dtest, that wouldn't get
included by coverage tools. Of course, this is where incremental changes to
the code base to allow unit testing are really awesome and appreciated.
I've been encouraging some folks to do that with their submissions. It
makes their patches a bit more involved (for them to code and for me to
review), but hopefully it improves the overall quality of the code.

Thus, imho, for new/large features, sure, having a high coverage is an
awesome thing: I'd think a minimum of 80%, with a "highly recommended"
target of > 90%. For smaller fixes, I'm not so sure, but I think a
reasonable agreement between the contributor and reviewer, based on the
guidelines coming from Blake's forthcoming doc, seem like a good point of
departure.

On Tue, Mar 28, 2017 at 5:57 AM, Josh McKenzie  wrote:

> Hold on now - there's no reason to throw the baby out with the bathwater on
> this. Code coverage on it's own is admittedly a questionable metric, but
> bad metrics are better than *no* metrics if taken with context.
>
> Most code coverage tools will allow you to drill down to the granularity
> we'd need to confirm that new code (new, as in entirely new file, etc) was
> covered, and to delta against a previously run coverage metric to make sure
> there's an uptick in modules where we make changes. It's quite a bit more
> involved / extra overhead compared to what we do now; I'd argue we probably
> need to learn to crawl before we can walk (as in, actually write tests for
> the code that goes into the project and do a *lot* more jmh benchmarking
> for perf changes).
>
> Definitely like the direction this is headed, but it sounds like some
> cart-before-horse to me if we integrate point 4 with the rest of them. Also
> - for point 2 who's going to do the work to correlate flaky failures to the
> changes that introduced them, or are we just going to accept that flaky
> tests get added and deal with them retroactively? I assume we could also
> set up some kind of CI job that would run a new test, say, 20 times and
> greenlight it for patches adding completely new tests? Might be a bad idea,
> might not.
>
> On Tue, Mar 28, 2017 at 1:13 AM, Blake Eggleston 
> wrote:
>
> > In addition to it’s test coverage problem, the project has a general
> > testability problem, and I think it would be more effective to introduce
> > some testing guidelines and standards that drive incremental improvement
> of
> > both, instead of requiring an arbitrary code coverage metric be hit,
> which
> > doesn’t tell the whole story anyway.
> >
> > It’s not ready yet, but I’ve been putting together a testing standards
> > document for the project since bringing it up in the “Code quality,
> > principles and rules” email thread a week or so ago.
> >
> > On March 27, 2017 at 4:51:31 PM, Edward Capriolo (edlinuxg...@gmail.com)
> > wrote:
> > On Mon, Mar 27, 2017 at 7:03 PM, Josh McKenzie 
> > wrote:
> >
> > > How do we plan on verifying #4? Also, root-cause to tie back new code
> > that
> > > introduces flaky tests (i.e. passes on commit, fails 5% of the time
> > > thereafter) is a non-trivial pursuit (thinking #2 here), and a pretty
> > > common problem in this environment.
> > >
> > > On Mon, Mar 27, 2017 at 6:51 PM, Nate McCall 
> wrote:
> > >
> > > > I don't want to lose track of the original idea from François, so
> > > > let's do this formally in preparation for a vote. Having this all in
> > > > place will make transition to new testing infrastructure more
> > > > goal-oriented and keep us more focused moving forward.
> > > >
> > > > Does anybody have specific feedback/discussion points on the
> following
> > > > (awesome, IMO) proposal:
> > > >
> > > 

Re: [VOTE] Ask Infra to move github notification emails to pr@

[Jason Brown]

+1
On Mon, Mar 20, 2017 at 18:21 Anthony Grasso 
wrote:

> +1
>
> On 21 March 2017 at 09:32, Jeff Jirsa  wrote:
>
> > There's no reason for the dev list to get spammed everytime there's a
> > github PR. We know most of the time we prefer JIRAs for real code PRs,
> but
> > with docs being in tree and low barrier to entry, we may want to accept
> > docs through PRs ( see https://issues.apache.org/
> > jira/browse/CASSANDRA-13256
> > , and comment on it if you disagree).
> >
> > To make that viable, we should make it not spam dev@ with every comment.
> > Therefore I propose we move github PR comments/actions to pr@ so as
> > not to clutter the dev@ list.
> >
> > Voting to remain open for 72 hours.
> >
> > - Jeff
> >
>

Re: [VOTE] Ask Infra to move github notification emails to commits@

[Jason Brown]

+1

On Mon, Mar 20, 2017 at 3:02 PM, Brandon Williams  wrote:

> +1, we've had to explain this a thousand times here.
>
> On Mon, Mar 20, 2017 at 5:00 PM, Jeff Jirsa  wrote:
>
> > There's no reason for the dev list to get spammed everytime there's a
> > github PR. We know most of the time we prefer JIRAs for real code PRs,
> but
> > with docs being in tree and low barrier to entry, we may want to accept
> > docs through PRs ( see https://issues.apache.org/
> > jira/browse/CASSANDRA-13256
> > , and comment on it if you disagree).
> >
> > To make that viable, we should make it not spam dev@ with every comment.
> > Therefore I propose we move github PR comments/actions to commits@ so as
> > not to clutter the dev@ list.
> >
> > Voting to remain open for 72 hours.
> >
> > - Jeff
> >
>

Re: Code quality, principles and rules

[Jason Brown]

To François's point about code coverage for new code, I think this makes a
lot of sense wrt large features (like the current work on 8457/12229/9754).
It's much simpler to (mentally, at least) isolate those changed sections
and it'll show up better in a code coverage report. With small patches,
that might be harder to achieve - however, as the patch should come with
*some* tests (unless it's a truly trivial patch), it might just work itself
out.

On Fri, Mar 17, 2017 at 11:19 AM, Jason Brown <jasedbr...@gmail.com> wrote:

> As someone who spent a lot of time looking at the singletons topic in the
> past, Blake brings a great perspective here. Figuring out and communicating
> how best to test with the system we have (and of course incrementally
> making that system easier to work with/test) seems like an achievable goal.
>
> On Fri, Mar 17, 2017 at 10:17 AM, Edward Capriolo <edlinuxg...@gmail.com>
> wrote:
>
>> On Fri, Mar 17, 2017 at 12:33 PM, Blake Eggleston <beggles...@apple.com>
>> wrote:
>>
>> > I think we’re getting a little ahead of ourselves talking about DI
>> > frameworks. Before that even becomes something worth talking about, we’d
>> > need to have made serious progress on un-spaghettifying Cassandra in the
>> > first place. It’s an extremely tall order. Adding a DI framework right
>> now
>> > would be like throwing gasoline on a raging tire fire.
>> >
>> > Removing singletons seems to come up every 6-12 months, and usually
>> > abandoned once people figure out how difficult they are to remove
>> properly.
>> > I do think removing them *should* be a long term goal, but we really
>> need
>> > something more immediately actionable. Otherwise, nothing’s going to
>> > happen, and we’ll be having this discussion again in a year or so when
>> > everyone’s angry that Cassandra 5.0 still isn’t ready for production, a
>> > year after it’s release.
>> >
>> > That said, the reason singletons regularly get brought up is because
>> doing
>> > extensive testing of anything in Cassandra is pretty much impossible,
>> since
>> > the code is basically this big web of interconnected global state.
>> Testing
>> > anything in isolation can’t be done, which, for a distributed database,
>> is
>> > crazy. It’s a chronic problem that handicaps our ability to release a
>> > stable database.
>> >
>> > At this point, I think a more pragmatic approach would be to draft and
>> > enforce some coding standards that can be applied in day to day
>> development
>> > that drive incremental improvement of the testing and testability of the
>> > project. What should be tested, how it should be tested. How to write
>> new
>> > code that talks to the rest of Cassandra and is testable. How to fix
>> bugs
>> > in old code in a way that’s testable. We should also have some
>> guidelines
>> > around refactoring the wildly untested sections, how to get started,
>> what
>> > to do, what not to do, etc.
>> >
>> > Thoughts?
>>
>>
>> To make the conversation practical. There is one class I personally really
>> want to refactor so it can be tested:
>>
>> https://github.com/apache/cassandra/blob/trunk/src/java/org/
>> apache/cassandra/net/OutboundTcpConnection.java
>>
>> There is little coverage here. Questions like:
>> what errors cause the connection to restart?
>> when are undropable messages are dropped?
>> what happens when the queue fills up?
>> Infamous throw new AssertionError(ex); (which probably bubble up to
>> nowhere)
>> what does the COALESCED strategy do in case XYZ.
>> A nifty label (wow a label you just never see those much!)
>> outer:
>> while (!isStopped)
>>
>> Comments to jira's that probably are not explicitly tested:
>> // If we haven't retried this message yet, put it back on the queue to
>> retry after re-connecting.
>> // See CASSANDRA-5393 and CASSANDRA-12192.
>>
>> If I were to undertake this cleanup, would there actually be support? IE
>> if
>> this going to turn into an "it aint broken. don't fix it thing" or a "we
>> don't want to change stuff just to add tests" . Like will someone pledge
>> to
>> agree its kinda wonky and merge the effort in < 1 years time?
>>
>
>

Re: Code quality, principles and rules

[Jason Brown]

>> do we have plan to integrate with a dependency injection framework?

No, we (the maintainers) have been pretty much against more frameworks due
to performance reasons, overhead, and dependency management problems.

On Thu, Mar 16, 2017 at 2:04 PM, Qingcun Zhou  wrote:

> Since we're here, do we have plan to integrate with a dependency injection
> framework like Dagger2? Otherwise it'll be difficult to write unit test
> cases.
>
> On Thu, Mar 16, 2017 at 1:16 PM, Edward Capriolo 
> wrote:
>
> > On Thu, Mar 16, 2017 at 3:10 PM, Jeff Jirsa  wrote:
> >
> > >
> > >
> > > On 2017-03-16 10:32 (-0700), François Deliège 
> > > wrote:
> > > >
> > > > To get this started, here is an initial proposal:
> > > >
> > > > Principles:
> > > >
> > > > 1. Tests always pass.  This is the starting point. If we don't care
> > > about test failures, then we should stop writing tests. A recurring
> > failing
> > > test carries no signal and is better deleted.
> > > > 2. The code is tested.
> > > >
> > > > Assuming we can align on these principles, here is a proposal for
> their
> > > implementation.
> > > >
> > > > Rules:
> > > >
> > > > 1. Each new release passes all tests (no flakinesss).
> > > > 2. If a patch has a failing test (test touching the same code path),
> > the
> > > code or test should be fixed prior to being accepted.
> > > > 3. Bugs fixes should have one test that fails prior to the fix and
> > > passes after fix.
> > > > 4. New code should have at least 90% test coverage.
> > > >
> > > First I was
> > > I agree with all of these and hope they become codified and followed. I
> > > don't know anyone who believes we should be committing code that breaks
> > > tests - but we should be more strict with requiring green test runs,
> and
> > > perhaps more strict with reverting patches that break tests (or cause
> > them
> > > to be flakey).
> > >
> > > Ed also noted on the user list [0] that certain sections of the code
> > > itself are difficult to test because of singletons - I agree with the
> > > suggestion that it's time to revisit CASSANDRA-7837 and CASSANDRA-10283
> > >
> > > Finally, we should also recall Jason's previous notes [1] that the
> actual
> > > test infrastructure available is limited - the system provided by
> > Datastax
> > > is not generally open to everyone (and not guaranteed to be permanent),
> > and
> > > the infrastructure currently available to the ASF is somewhat limited
> > (much
> > > slower, at the very least). If we require tests passing (and I agree
> that
> > > we should), we need to define how we're going to be testing (or how
> we're
> > > going to be sharing test results), because the ASF hardware isn't going
> > to
> > > be able to do dozens of dev branch dtest runs per day in its current
> > form.
> > >
> > > 0: https://lists.apache.org/thread.html/f6f3fc6d0ad1bd54a6185ce7bd7a2f
> > > 6f09759a02352ffc05df92eef6@%3Cuser.cassandra.apache.org%3E
> > > 1: https://lists.apache.org/thread.html/5fb8f0446ab97644100e4ef987f36e
> > > 07f44e8dd6d38f5dc81ecb3cdd@%3Cdev.cassandra.apache.org%3E
> > >
> > >
> > >
> > Ed also noted on the user list [0] that certain sections of the code
> itself
> > are difficult to test because of singletons - I agree with the suggestion
> > that it's time to revisit CASSANDRA-7837 and CASSANDRA-10283
> >
> > Thanks for the shout out!
> >
> > I was just looking at a patch about compaction. The patch was to
> calculate
> > free space correctly in case X. Compaction is not something that requires
> > multiple nodes to test. The logic on the surface seems simple: find
> tables
> > of similar size and select them and merge them. The reality is it turns
> out
> > now to be that way. The coverage itself both branch and line may be very
> > high, but what the code does not do is directly account for a wide
> variety
> > of scenarios. Without direct tests you end up with a mental approximation
> > of what it does and that varies from person to person and accounts for
> the
> > cases that fit in your mind. For example, you personally are only running
> > LevelDB inspired compaction.
> >
> > Being that this this is not a multi-node problem you should be able to re
> > factor this heavily. Pulling everything out to a static method where all
> > the parameters are arguments, or inject a lot of mocks in the current
> code,
> > and develop some scenario based coverage.
> >
> > That is how i typically "rescue" code I take over. I look at the
> nightmare
> > and say, "damn i am really afraid to touch this". I construct 8 scenarios
> > that test green. Then I force some testing into it through careful re
> > factoring. Now, I probably know -something- about it. Now, you are fairly
> > free to do a wide ranging refactor, because you at least counted for 8
> > scenarios and you put unit test traps so that some rules are enforced.
> (Or
> > the person changing the code has to actively REMOVE 

jira ticket assignment

[Jason Brown]

Hey all,

Currently, when a new contributor wants to have a ticket assigned to
themselves, they cannot due to JIRA permissions. The contributor (or
someone on behalf of the contributor) usually needs to contact the
dev-channel and ask for the magical permissions to be granted, and there's
only a handful of folks who can grant that.

Is anyone aware of why this restriction is in place, or feel strongly it
should exist? I'd like to change the JIRA defaults and let anyone assign
themselves to a ticket (see
https://issues.apache.org/jira/browse/INFRA-11950 for an example of another
Apache project that has done this already).

Thoughts?

If no strong objections I'll set up a PMC vote within a day or two.

Thanks,

-Jason

Re: Testing and jira tickets

[Jason Brown]

To Ariel's point, I don't think we can expect all contributors to run all
utesss/dtests, especially when the patch spans multiple branches. On that
front, I, like Ariel and many others, typically create our own branch of
the patch and have executed the tests. I think this is a reasonable system,
if slightly burdensome, given our project and our needs/demands on it.

Whoever runs the test, is it reasonable to expect the results to become
available on the ticket? As the CI is moving to the Apache infrastructure,
that will be the final arbiter of " patch works or breaks things", but I
suspect executing the tests anywhere else will still be a very good
indicator of the viability of the patch.

Thoughts?

On Thu, Mar 9, 2017 at 12:31 PM, Ariel Weisberg <ar...@weisberg.ws> wrote:

> Hi,
>
> Before this change I had already been queuing the jobs myself as a
> reviewer. It also happens to be that many reviewers are committers. I
> wouldn't ask contributors to run the dtests/utests for any purpose other
> then so that they know the submission is done.
>
> Even if they did and they pass it doesn't matter. It only matters if it
> passes in CI. If it fails in CI but passes on their desktop it's not
> good enough so we have to run in CI anyways.
>
> If a reviewer is not a committer. Well they can ask someone else to do
> it? I know we have issues with responsiveness, but I would make myself
> available for that. It shouldn't be a big problem because if someone is
> doing a lot of reviews they should be a committer right?
>
> Regards,
> Ariel
>
> On Thu, Mar 9, 2017, at 01:51 PM, Jason Brown wrote:
> > Hey all,
> >
> > A nice convention we've stumbled into wrt to patches submitted via Jira
> > is
> > to post the results of unit test and dtest runs to the ticket (to show
> > the
> > patch doesn't break things). Many contributors have used the
> > DataStax-provided cassci system, but that's not the best long term
> > solution. To that end, I'd like to start a conversation about what is the
> > best way to proceed going forward, and then add it to the "How to
> > contribute" docs.
> >
> > As an example, should contributors/committers run dtests and unit tests
> > on
> > *some* machine (publicly available or otherwise), and then post those
> > results to the ticket? This could be a link to a build system, like what
> > we
> > have with cassci, or just  upload the output of the test run(s).
> >
> > I don't have any fixed notions, and am looking forward to hearing other's
> > ideas.
> >
> > Thanks,
> >
> > -Jason
> >
> > p.s. a big thank you to DataStax for providing the cassci system
>

Re: Testing and jira tickets

[Jason Brown]

Jon and Brandon,

I'd actually like to narrow the discussion, and keep it focused to my
original topic. Those are two excellent topics that should be addressed,
and the solution(s) might be the same or similar as the outcome of this.
However, I feel they deserve their own message thread.

Thanks for understanding,

-Jason

On Thu, Mar 9, 2017 at 11:27 AM, Brandon Williams <dri...@gmail.com> wrote:

> Let me further broaden this discussion to include github branches, which
> are often linked on tickets, and then later deleted.  This forces a person
> to search through git to actually see the patch, and that process can be a
> little rough (especially since we all know if you're gonna make a typo,
> it's going to be in the commit, and it's probably going to be the ticket
> number.)
>
> On Thu, Mar 9, 2017 at 1:00 PM, Jonathan Haddad <j...@jonhaddad.com> wrote:
>
> > If you don't mind, I'd like to broaden the discussion a little bit to
> also
> > discuss performance related patches.  For instance, CASSANDRA-13271 was a
> > performance / optimization related patch that included *zero* information
> > on if there was any perf improvement or a regression as a result of the
> > change, even though I've asked twice for that information.
> >
> > In addition to "does this thing break anything" we should be asking "how
> > does this patch affect performance?" (and were the appropriate docs
> > included, but that's another topic altogether)
> >
> > On Thu, Mar 9, 2017 at 10:51 AM Jason Brown <jasedbr...@gmail.com>
> wrote:
> >
> > > Hey all,
> > >
> > > A nice convention we've stumbled into wrt to patches submitted via Jira
> > is
> > > to post the results of unit test and dtest runs to the ticket (to show
> > the
> > > patch doesn't break things). Many contributors have used the
> > > DataStax-provided cassci system, but that's not the best long term
> > > solution. To that end, I'd like to start a conversation about what is
> the
> > > best way to proceed going forward, and then add it to the "How to
> > > contribute" docs.
> > >
> > > As an example, should contributors/committers run dtests and unit tests
> > on
> > > *some* machine (publicly available or otherwise), and then post those
> > > results to the ticket? This could be a link to a build system, like
> what
> > we
> > > have with cassci, or just  upload the output of the test run(s).
> > >
> > > I don't have any fixed notions, and am looking forward to hearing
> other's
> > > ideas.
> > >
> > > Thanks,
> > >
> > > -Jason
> > >
> > > p.s. a big thank you to DataStax for providing the cassci system
> > >
> >
>