Re: [VOTE] Release dtest-api 0.0.12

[Oleksandr Petrov]

+1

On Mon 24. Jan 2022 at 13:57, Andrés de la Peña 
wrote:

> +1
>
> On Mon, 24 Jan 2022 at 12:29, Brandon Williams  wrote:
>
>> +1
>>
>> On Thu, Jan 13, 2022 at 12:17 PM Mick Semb Wever  wrote:
>> >
>> > Proposing the test build of in-jvm dtest API 0.0.12 for release.
>> >
>> > Repository:
>> >
>> https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.12
>> >
>> > Candidate SHA:
>> >
>> https://github.com/apache/cassandra-in-jvm-dtest-api/commit/207d6cee2d01552f794d322ec05a7577bcab08e0
>> > tagged with 0.0.12
>> >
>> > Artifacts:
>> >
>> https://repository.apache.org/content/repositories/orgapachecassandra-1252/org/apache/cassandra/dtest-api/0.0.12/
>> >
>> > Key signature: A4C465FEA0C552561A392A61E91335D77E3E87CB
>> >
>> > Changes since last release:
>> >   * CASSANDRA-17214:Add IInstance.isValid() with default true return
>> value
>> >
>> >
>> > The vote will be open for 24 hours. Everyone who has tested the build
>> > is invited to vote. Votes by PMC members are considered binding. A
>> > vote passes if there are at least three binding +1s.
>>
> --
alex p

Welcome Sumanth Pasupuleti as Apache Cassandra Committer

[Oleksandr Petrov]

The PMC members are pleased to announce that Sumanth Pasupuleti has
recently accepted the invitation to become committer.

Sumanth, thank you for all your contributions to the project over the years.

Congratulations and welcome!

The Apache Cassandra PMC members

Re: [DISCUSS] Releasable trunk and quality

[Oleksandr Petrov]

I'll merge 16262 and the Harry blog-post that accompanies it shortly.
Having 16262 merged will significantly reduce the amount of resistance one
has to overcome in order to write a fuzz test. But this, of course, only
covers short/small/unit-test-like tests.

For longer running tests, I guess for now we will have to rely on folks
(hopefully) running long fuzz tests and reporting issues. But eventually
it'd be great to have enough automation around it so that anyone could do
that and where test results are public.

In regard to long-running tests, currently with Harry we can run three
kinds of long-running tests:
1. Stress-like concurrent write workload, followed by periods of quiescence
and then validation
2. Writes with injected faults, followed by repair and validation
3. Stress-like concurrent read/write workload with fault injection without
validation, for finding rare edge conditions / triggering possible
exceptions

Which means that quorum read and write paths (for all kinds of schemas,
including all possible kinds of read and write queries), compactions,
repairs, read-repairs and hints are covered fairly well. However things
like bootstrap and other kinds of range movements aren't. It'd be great to
expand this, but it's been somewhat difficult to do, since last time a
bootstrap test was attempted, it has immediately uncovered enough issues to
keep us busy fixing them for quite some time. Maybe it's about time to try
that again.

For short tests, you can think of Harry as a tool to save you time and
allow focusing on higher-level test meaning rather than creating schema and
coming up with specific values to insert/select.

Thanks
--Alex



On Tue, Nov 2, 2021 at 5:30 PM Ekaterina Dimitrova 
wrote:

> Did I hear my name? 😁
> Sorry Josh, you are wrong :-) 2 out of 30 in two months were real bugs
> discovered by pflaky tests and one of them was very hard to hit. So 6-7%. I
> think that report I sent back then didn’t come through so the topic was
> cleared in a follow up mail by Benjamin; with a lot of sweat but we kept to
> the promised 4.0 standard.
>
> Now back to this topic:
> - green CI without enough test coverage is nothing more than green CI
> unfortunately to me.  I know this is an elephant but I won’t sleep well
> tonight if I don’t mention it.
> - I believe the looping of tests mentioned by Berenguer can help for
> verifying no new weird flakiness is introduced by new tests added. And of
> course it helps a lot during fixing flaky tests, I think that’s clear.
>
>  I think that it would be great if each such test
> > > (or
> > > > test group) was guaranteed to have a ticket and some preliminary
> > analysis
> > > > was done to confirm it is just a test problem before releasing the
> new
> > > > version
>
> Probably not bad idea. Preliminary analysis. But we need to get into the
> cadence of regular checking our CI; divide and conquer on regular basis
> between all of us. Not to mention it is way easier to follow up recently
> introduced issues with the people who worked on stuff then trying to find
> out what happened a year ago in a rush before a release. I agree it is not
> about the number but what stays behind it.
>
> Requiring all tests to run pre every merge, easily we can add this in
> circle but there are many people who don’t have access to high resources so
> again they won’t be able to run absolutely everything. At the end
> everything is up to the diligence of the reviewers/committers. Plus
> official CI is Jenkins and we know there are different infra related
> failures in the different CIs. Not an easy topic, indeed. I support running
> all tests, just having in mind all the related issues/complications.
>
> I would say in my mind upgrade tests are particularly important to be green
> before a release, too.
>
> Seems to me we have the tools, but now it is time to organize the rhythm in
> an efficient manner.
>
> Best regards,
> Ekaterina
>
>
> On Tue, 2 Nov 2021 at 11:06, Joshua McKenzie  wrote:
>
> > To your point Jacek, I believe in the run up to 4.0 Ekaterina did some
> > analysis and something like 18% (correct me if I'm wrong here) of the
> test
> > failures we were considering "flaky tests" were actual product defects in
> > the database. With that in mind, we should be uncomfortable cutting a
> > release if we have 6 test failures since there's every likelihood one of
> > them is a surfaced bug.
> >
> > ensuring our best practices are followed for every merge
> >
> > I totally agree but I also don't think we have this codified (unless I'm
> > just completely missing something - very possible! ;)) Seems like we have
> > different circle configs, different sets of jobs being run, Harry /
> Hunter
> > (maybe?) / ?? run on some but not all commits and/or all branches,
> > manual performance testing on specific releases but nothing surfaced
> > formally to the project as a reproducible suite like we used to have
> years
> > ago (primitive though it was at the time with what it c

Re: Paging in bytes (CASSANDRA-11745)

[Oleksandr Petrov]

I personally think this change should be small enough to just be done via
jira without introducing a CEP. But since paging generally has seen some
problems, it'd be good to understand the design before flushing it out in
code.

One of the ways where we probably should be considerate is the paging
state. Paging state is passed from the coordinator to client, and then from
the client to a (potentially different) coordinator. If the coordinator
speaks the internode protocol of a different version, this can potentially
lead to problems. That said, if we have transactional metadata implemented
and can reliably track versions in the cluster, and switch between epochs
in which an effective minimal version for all coordinators supports a
specific version of client state, this should not be a problem at all.

On Fri, Oct 22, 2021 at 9:07 AM Jacek Lewandowski <
lewandowski.ja...@gmail.com> wrote:

> Hi
>
> I wanted to start working on extending the paging mechanism to support
> memory based limits. My question is whether such a change requires a CEP or
> discussion in Jira is enough?
>
> Thanks,
> - - -- --- -  -
> Jacek Lewandowski
>


-- 
alex p

Re: Save CircleCI resources with optional test jobs

[Oleksandr Petrov]

In his latest email he pointed to the Slack
> > > discussion that happened and where agreement was reached.
> > > I see your email was sent two days after he said it is implemented. I
> > guess
> > > we had to be more explicit that  this email announces lazy consensus.
> > >
> > > Anyway, I am really sorry your email was missed and please, believe me
> > when
> > > I say I would have considered it as the ticket was not committed yet at
> > > that point! I am almost sure something similar happened to Andres
> knowing
> > > how extremely punctual and fair he is.
> > >
> > > The main goal was not to push on every commit as many times it would be
> > > unnecessary waste of resources and spending credits. At the same time
> new
> > > workflows with one precommit button were added to ensure people can run
> > all
> > > tests we as a community require with one click before a commit. Links
> > with
> > > the different versions/suggestions of the workflow are published on the
> > > ticket.
> > >
> > > Yes, now we need to click on the build. It was agreed that many times
> > > people push even just to preserve intermediate work and continue later
> > > without caring of build or anything. If for some reason it is important
> > to
> > > you or anyone else from the community to build on every commit, we can
> > open
> > > a ticket to change that. It will save a click or two in the case of the
> > > in-jvm upgrade tests. I would like to point out that Andres was also
> > > experimenting to ensure that the graph will be still as much readable
> as
> > > possible.
> > >
> > > Benedict, on your comment of feature request - we can do that. I am
> also
> > > sceptic as you if that will happen but this doesn’t mean we can’t give
> > it a
> > > try. Who knows, maybe others are also raising the topic and they might
> > > consider it.
> > >
> > > Honestly, I personally support the current workflow and approval
> required
> > > but if it is not acceptable to others and skipping the build approval
> > click
> > > is what others would prefer, I will open a ticket to restore that part.
> > > Please let me know and one more time, apologize for missing your email,
> > > Alex.
> > >
> > > Best regards,
> > > Ekaterina
> > >
> > >
> > >
> > > On Wed, 20 Oct 2021 at 9:01, bened...@apache.org 
> > > wrote:
> > >
> > > > I think this would be fine if there were a way for approval of later
> > steps
> > > > to trigger the build automatically. As it is we have to traverse the
> > > > dependency graph manually, which is a bit weird.
> > > >
> > > > I can’t figure out a way to do this with the CircleCI API however. It
> > > > might be a feature request, and perhaps we can at least trigger the
> > build
> > > > until we get that (which may never happen).
> > > >
> > > > From: Oleksandr Petrov 
> > > > Date: Wednesday, 20 October 2021 at 13:35
> > > > To: dev 
> > > > Subject: Re: Save CircleCI resources with optional test jobs
> > > > I thought this discussion was still ongoing, but it looks like
> > > > CASSANDRA-16882 is now committed.
> > > >
> > > > Could you give some context on why at least compilation is not done
> on
> > > > every commit now?
> > > >
> > > >
> > > > On Fri, Oct 8, 2021 at 4:12 PM Oleksandr Petrov <
> > > > oleksandr.pet...@gmail.com>
> > > > wrote:
> > > >
> > > > > I personally rarely push tickets/post a patch in an raw state, but
> > since
> > > > I
> > > > > almost always have to approve jobs when it gets close to commit, I
> > don't
> > > > > mind also confirming utest runs. I'd say it'd be good to run at
> very
> > > > least
> > > > > a compilation step on every commit. I think it's fine if
> > > > dtests/utests/jvm
> > > > > tests require approval.
> > > > >
> > > > > On Wed, Oct 6, 2021 at 4:22 PM Andrés de la Peña <
> > adelap...@apache.org>
> > > > > wrote:
> > > > >
> > > > >> Hello all,
> > > > >>
> > > > >> The changes in CircleCI proposed in the previous messages are
> 

Re: Save CircleCI resources with optional test jobs

[Oleksandr Petrov]

I thought this discussion was still ongoing, but it looks like
CASSANDRA-16882 is now committed.

Could you give some context on why at least compilation is not done on
every commit now?


On Fri, Oct 8, 2021 at 4:12 PM Oleksandr Petrov 
wrote:

> I personally rarely push tickets/post a patch in an raw state, but since I
> almost always have to approve jobs when it gets close to commit, I don't
> mind also confirming utest runs. I'd say it'd be good to run at very least
> a compilation step on every commit. I think it's fine if dtests/utests/jvm
> tests require approval.
>
> On Wed, Oct 6, 2021 at 4:22 PM Andrés de la Peña 
> wrote:
>
>> Hello all,
>>
>> The changes in CircleCI proposed in the previous messages are implemented
>> in CASSANDRA-16882. They try to include the feedback from both the
>> reviewers and the Slack discussion at
>> https://the-asf.slack.com/archives/CK23JSY2K/p1631627458109000.
>>
>> The patch modifies the CircleCI config to have two pairs of j8/j11
>> workflows:
>>
>> * The javaX_pre-commit_tests workflows are meant to be used when a patch
>> is
>> definitively ready for final review and/or commit. They have a single
>> approval step that starts all the basic tests, including unit tests,
>> dtests, etc. Additionally, it has approval steps for those tests that are
>> not generally required in every ticket, such as upgrade tests and the
>> multiplexer. This pair of workflows is quite similar to what we currently
>> have, and the main difference is that there is an approval step to start
>> the build.
>>
>> * The javaX_separate_tests workflows are meant to be used in intermediate
>> steps and special cases such as fixing flaky tests. All the jobs in these
>> workflows have an individual approval step, so one can run any test job
>> without wasting resources in the others. For example, it makes possible to
>> repeatedly run a unit test without firing the entire JVM dtests.
>>
>> Here is an example of how the workflows would look like:
>>
>> https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-trunk-v04
>>
>> Hopefully this approach would help us to save resources in intermediate
>> development steps and special cases, while keeping the current concept of
>> mandatory tests.
>>
>> If no one disagrees with this approach I'll commit the changes in a few
>> days.
>>
>> On Fri, 27 Aug 2021 at 11:54, Andrés de la Peña 
>> wrote:
>>
>> > Hi,
>> >
>> > CASSANDRA-16882 has patches for any of the mentioned configurations
>> aimed
>> > to save CircleCI resources.
>> >
>> > There are CircleCI runs showing how each approach would look like, plus
>> an
>> > additional fourth option:
>> >
>> > 1. Make the entire workflow optional:
>> >
>> https://app.circleci.com/pipelines/github/adelapena/cassandra/800/workflows/9cb8ca7b-ab57-431e-a22b-643d61c92c29
>> > 2. Make all the test jobs optional:
>> >
>> https://app.circleci.com/pipelines/github/adelapena/cassandra/798/workflows/a859cfbc-fdf8-4468-beb9-b2ee17dc1ae3
>> > 3. Make all the mandatory test jobs depend on a single optional job:
>> >
>> https://app.circleci.com/pipelines/github/adelapena/cassandra/802/workflows/0372f5d6-d1f0-4f0e-91a3-aa75a2712bae
>> > 4. Combine 2 and 3 to have approval jobs for both individual tests and
>> the
>> > grouped mandatory tests:
>> >
>> https://app.circleci.com/pipelines/github/adelapena/cassandra/803/workflows/08ae07d5-6a1e-4e5b-bc0c-32bdc9b9f190
>> >
>> > I think that the fourth option gives us more flexibility, because it
>> > allows us to start any combination of tests we want while it keeps the
>> > concept of required tests.
>> >
>> > On Thu, 12 Aug 2021 at 17:47, Andrés de la Peña <
>> a.penya.gar...@gmail.com>
>> > wrote:
>> >
>> >> Hello all,
>> >>
>> >> The current CircleCI configuration automatically runs the unit tests,
>> JVM
>> >> dtests and cqhshlib tests. This is done by default for every commit or,
>> >> with some configuration, for every push.
>> >>
>> >> Along the lifecycle of a ticket it is quite frequent to have multiple
>> >> commits and pushes, all running these test jobs. I'd say that
>> frequently it
>> >> is not necessary to run the tests for some of those intermediate
>> commits
>> >> and pushes. For example, one can show proofs of concept, or have
>&g

Re: [VOTE] CEP-15: General Purpose Transactions

[Oleksandr Petrov]

1. +1
2. +1
3. +1

On Thu, Oct 14, 2021 at 6:31 PM bened...@apache.org 
wrote:

> Hi everyone,
>
> I would like to start a vote on this CEP, split into three sub-decisions,
> as discussion has been circular for some time.
>
> 1. Do you support adopting this CEP?
> 2. Do you support the transaction semantics proposed by the CEP for
> Cassandra?
> 3. Do you support an incremental approach to developing transactions in
> Cassandra, leaving scope for future development?
>
> The first vote is a consensus vote of all committers, the second and third
> however are about project direction and therefore are simple majority votes
> of the PMC.
>
> Recall that all -1 votes must be accompanied by an explanation. If you
> reject the CEP only on grounds (2) or (3) you should not veto the proposal.
> If a majority reject grounds (2) or (3) then transaction developments will
> halt for the time being.
>
> This vote will be open for 72 hours.
>


-- 
alex p

Re: Tradeoffs for Cassandra transaction management

[Oleksandr Petrov]

I realise this is not contributing to this discussion, but this email is
very difficult to read because it seems like something has happened with
formatting. For me it gets displayed as a single paragraph with no line
breaks.

There seems to be some overlap between the image uploaded to imgur and this
email, but some things are only present in the email and not on the image.

On Sat, Oct 9, 2021 at 6:54 PM Jonathan Ellis  wrote:

> * Hi all,After calling several times for a broader discussion of goals and
> tradeoffs around transaction management in the CEP-15 thread, I’ve put
> together a short analysis to kick that off.Here is a table that summarizes
> the state of the art for distributed transactions that offer
> serializability, i.e., a superset of what you can get with LWT.  (The most
> interesting option that this eliminates is RAMP.)Since I'm not sure how
> this will render outside gmail, I've also uploaded it here:
> https://imgur.com/a/SCZ8jex
> SpannerCockroachCalvin/FaunaSLOG (see
> below)Write latencyGlobal Paxos, plus 2pc for multi-partition.For
> intercontinental replication this is 100+ms.  Cloud Spanner does not allow
> truly global deployments for this reason.Single-region Paxos, plus 2pc.
> I’m not very clear on how this works but it results in non-strict
> serializability.I didn’t find actual numbers for CR other than “2ms in a
> single AZ” which is not a typical scenario.Global Raft.  Fauna posts actual
> numbers of ~70ms in production which I assume corresponds to a multi-region
> deployment with all regions in the USA.  SLOG paper says true global Calvin
> is 200+ms.Single-region Paxos (common case) with fallback to multi-region
> Paxos.Under 10ms.Scalability bottlenecksLocks held during cross-region
> replicationSame as SpannerOLLP approach required when PKs are not known in
> advance (mostly for indexed queries) -- results in retries under
> contentionSame as CalvinRead latency at serial consistencyTimestamp from
> Paxos leader (may be cross-region), then read from local replica.Same as
> Spanner, I thinkSame as writesSame as writesMaximum serializability
> flavorStrictUn-strictStrictStrictSupport for other isolation
> levels?SnapshotNoSnapshot (in Fauna)Paper mentions dropping from
> strict-serializable to only serializable.  Probably could also support
> Snapshot like Fauna.Interactive transaction support (req’d for
> SQL)YesYesNoNoPotential for grafting onto C*NightmareNightmareReasonable,
> Calvin is relatively simple and the storage assumptions it makes are
> minimalI haven’t thought about this enough. SLOG may require versioned
> storage, e.g. see this comment
> <
> http://dbmsmusings.blogspot.com/2019/10/introducing-slog-cheating-low-latency.html?showComment=1570497003296#c5976719429355924873
> >.(I
> have not included Accord here because it’s not sufficiently clear to me how
> to create a full transaction manager from the Accord protocol, so I can’t
> analyze many of the properties such a system would have.  The most obvious
> solution would be “Calvin but with Accord instead of Raft”, but since
> Accord already does some Calvin-like things that seems like it would result
> in some suboptimal redundancy.)After putting the above together it seems to
> me that the two main areas of tradeoff are, 1. Is it worth giving up local
> latencies to get full global consistency?  Most LWT use cases use
> LOCAL_SERIAL.  While all of the above have more efficient designs than LWT,
> it’s still true that global serialization will require 100+ms in the
> general case due to physical transmission latency.  So a design that allows
> local serialization with EC between regions, or a design (like SLOG) that
> automatically infers a “home” region that can do local consensus in the
> common case without giving up global serializability, is desirable.2. Is it
> worth giving up the possibility of SQL support, to get the benefits of
> deterministic transaction design?  To be clear, these benefits include very
> significant ones around simplicity of design, higher write throughput, and
> (in SLOG) lower read and write latencies.I’ll doubleclick on #2 because it
> was asserted in the CEP-15 thread that Accord could support SQL by applying
> known techniques on top.  This is mistaken.  Deterministic systems like
> Calvin or SLOG or Accord can support queries where the rows affected are
> not known in advance using a technique that Abadi calls OLLP (Optimistic
> Lock Location Prediction), but this does not help when the transaction
> logic is not known in advance.Here is Daniel Abadi’s explanation of OLLP
> from “An Overview of Deterministic Database Systems
> <
> https://cacm.acm.org/magazines/2018/9/230601-an-overview-of-deterministic-database-systems/fulltext?mobile=false
> >:”In
> practice, deterministic database systems that use ordered locking do not
> wait until runtime for transactions to determine their access-sets.
> Instead, they use a technique called OLLP where 

Re: [DISCUSS] CEP-15: General Purpose Transactions

[Oleksandr Petrov]

> I support this proposal. From what I can understand, this proposal  moves
us towards having the building blocks we need to correctly deliver some of
the most often requested features in Cassandra.

Same here. I also support this proposal and believe it opens up many new
opportunities (while not limiting us / not narrowing our future options),
can help us implement features we've all wanted to have implemented for
years, and make significant improvements in the subsystems that were a
source of issues for a long time.

I think it's also good to start with CAS batches: it's a great way to make
the feature available and work incrementally. After this lands, people will
be able to use Accord/MPT in different subsystems and get busy
implementing all sorts of other features and improvements on top of it.




On Sat, Oct 9, 2021 at 4:18 PM Joseph Lynch  wrote:

> > With the proposal hitting the one-month mark, the contributors are
> interested in gauging the developer community's response to the proposal.
>
> I support this proposal. From what I can understand, this proposal
> moves us towards having the building blocks we need to correctly
> deliver some of the most often requested features in Cassandra. For
> example it seems to unlock: batches that actually work, registers that
> offer fast compare and swap, global secondary indices that can be
> correctly maintained, and more. Therefore, given the benefit to the
> community, I support working towards that foundation that will allow
> us to build solutions in Cassandra that pay consensus closer to
> mutation instead of lazily at read/repair time.
>
> I think the feedback in this thread around interface (what statements
> will this facilitate and how will the library integrate with Cassandra
> itself), performance (how fast will these transactions be, will we
> offer bounded stale reads, etc ...), and implementation (how does this
> compare/contrast with other consensus approaches) has been
> informative, but at this point I think it makes sense to start trying
> to make incremental progress towards a functional integration to
> discover any remaining areas for improvement.
>
> Cheers and thank you!
> -Joey
>
>
>
> On Thu, Oct 7, 2021 at 10:51 AM C. Scott Andreas 
> wrote:
> >
> > Hi Jonathan,
> >
> > Following up on my message yesterday as it looks like our replies may
> have crossed en route.
> >
> > Thanks for bumping your message from earlier in our discussion. I
> believe we have addressed most of these questions on the thread, in
> addition to offering a presentation on this and related work at ApacheCon,
> a discussion hosted following that presentation at ApacheCon, and in ASF
> Slack. Contributors have further offered an opportuntity to discuss
> specific questions via videoconference if it helps to speak live. I'd be
> happy to do so as well.
> >
> > Since your original message, discussion has covered a lot of ground on
> the related databases you've mentioned:
> > – Henrik has shared expertise related to MongoDB and its implementation.
> > – You've shared an overview of Calvin.
> > – Alex Miller has helped us review the work relative to other Paxos
> algorithms and identified a few great enhancements to incorporate.
> > – The paper discusses related approaches in FoundationDB, CockroachDB,
> and Yugabyte.
> > – Subsequent discussion has contrasted the implementation to DynamoDB,
> Google Cloud BigTable, and Google Cloud Spanner (noting specifically that
> the protocol achieves Spanner's 1x round-trip without requiring specialized
> hardware).
> >
> > In my reply yesterday, I've attempted to crystallize what becomes
> possible via CQL: one-shot multi-partition transactions in the first
> implementation and a 4x latency reduction on writes / 2x latency reduction
> on reads relative to today; along with the ability to build upon this work
> to enable interactive transactions in the future.
> >
> > I believe we've exercised the questions you've raised and am grateful
> for the ground we've covered. If you have further questions that are
> difficult to exercise via email, please let me know if you'd like to
> arrange a call (open-invite); we'd be happy to discuss live as well.
> >
> > With the proposal hitting the one-month mark, the contributors are
> interested in gauging the developer community's response to the proposal.
> We warrant our ability to focus durably on the project; execute this
> development on ASF JIRA in collaboration with other contributors; engage
> with members of the developer and user community on feedback, enhancements,
> and bugs; and intend deliver it to completion at a standard of readiness
> suitable for production transactional systems of record.
> >
> > Thanks,
> >
> > – Scott
> >
> > On Oct 6, 2021, at 8:25 AM, C. Scott Andreas 
> wrote:
> >
> >
> >
> > Hi folks,
> >
> > Thanks for discussion on this proposal, and also to Benedict who’s been
> fielding questions on the list!
> >
> > I’d like to restate the goals and proble

Re: Save CircleCI resources with optional test jobs

[Oleksandr Petrov]

I personally rarely push tickets/post a patch in an raw state, but since I
almost always have to approve jobs when it gets close to commit, I don't
mind also confirming utest runs. I'd say it'd be good to run at very least
a compilation step on every commit. I think it's fine if dtests/utests/jvm
tests require approval.

On Wed, Oct 6, 2021 at 4:22 PM Andrés de la Peña 
wrote:

> Hello all,
>
> The changes in CircleCI proposed in the previous messages are implemented
> in CASSANDRA-16882. They try to include the feedback from both the
> reviewers and the Slack discussion at
> https://the-asf.slack.com/archives/CK23JSY2K/p1631627458109000.
>
> The patch modifies the CircleCI config to have two pairs of j8/j11
> workflows:
>
> * The javaX_pre-commit_tests workflows are meant to be used when a patch is
> definitively ready for final review and/or commit. They have a single
> approval step that starts all the basic tests, including unit tests,
> dtests, etc. Additionally, it has approval steps for those tests that are
> not generally required in every ticket, such as upgrade tests and the
> multiplexer. This pair of workflows is quite similar to what we currently
> have, and the main difference is that there is an approval step to start
> the build.
>
> * The javaX_separate_tests workflows are meant to be used in intermediate
> steps and special cases such as fixing flaky tests. All the jobs in these
> workflows have an individual approval step, so one can run any test job
> without wasting resources in the others. For example, it makes possible to
> repeatedly run a unit test without firing the entire JVM dtests.
>
> Here is an example of how the workflows would look like:
>
> https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-trunk-v04
>
> Hopefully this approach would help us to save resources in intermediate
> development steps and special cases, while keeping the current concept of
> mandatory tests.
>
> If no one disagrees with this approach I'll commit the changes in a few
> days.
>
> On Fri, 27 Aug 2021 at 11:54, Andrés de la Peña 
> wrote:
>
> > Hi,
> >
> > CASSANDRA-16882 has patches for any of the mentioned configurations aimed
> > to save CircleCI resources.
> >
> > There are CircleCI runs showing how each approach would look like, plus
> an
> > additional fourth option:
> >
> > 1. Make the entire workflow optional:
> >
> https://app.circleci.com/pipelines/github/adelapena/cassandra/800/workflows/9cb8ca7b-ab57-431e-a22b-643d61c92c29
> > 2. Make all the test jobs optional:
> >
> https://app.circleci.com/pipelines/github/adelapena/cassandra/798/workflows/a859cfbc-fdf8-4468-beb9-b2ee17dc1ae3
> > 3. Make all the mandatory test jobs depend on a single optional job:
> >
> https://app.circleci.com/pipelines/github/adelapena/cassandra/802/workflows/0372f5d6-d1f0-4f0e-91a3-aa75a2712bae
> > 4. Combine 2 and 3 to have approval jobs for both individual tests and
> the
> > grouped mandatory tests:
> >
> https://app.circleci.com/pipelines/github/adelapena/cassandra/803/workflows/08ae07d5-6a1e-4e5b-bc0c-32bdc9b9f190
> >
> > I think that the fourth option gives us more flexibility, because it
> > allows us to start any combination of tests we want while it keeps the
> > concept of required tests.
> >
> > On Thu, 12 Aug 2021 at 17:47, Andrés de la Peña <
> a.penya.gar...@gmail.com>
> > wrote:
> >
> >> Hello all,
> >>
> >> The current CircleCI configuration automatically runs the unit tests,
> JVM
> >> dtests and cqhshlib tests. This is done by default for every commit or,
> >> with some configuration, for every push.
> >>
> >> Along the lifecycle of a ticket it is quite frequent to have multiple
> >> commits and pushes, all running these test jobs. I'd say that
> frequently it
> >> is not necessary to run the tests for some of those intermediate commits
> >> and pushes. For example, one can show proofs of concept, or have
> multiple
> >> rounds of review before actually running the tests. Running the tests
> for
> >> every change can produce an unnecessary expense of CircleCI resources.
> >>
> >> I think we could make running those tests optional, as well as clearly
> >> specifying in the documentation what are the tests runs that are
> mandatory
> >> before actually committing. We could do this in different ways:
> >>
> >> 1. Make the entire CircleCI workflow optional, so the build job requires
> >> manual approval. Once the build is approved the mandatory test jobs
> would
> >> be run without any further approval, exactly as it's currently done.
> >>
> >> 2. Make all the test jobs optional, so every test job requires manual
> >> approval, and the documentation specifies which tests are mandatory in
> the
> >> final steps of a ticket.
> >>
> >> 3. Make all the mandatory test jobs depend on a single optional job, so
> >> we have a single button to optionally run all the mandatory tests.
> >>
> >> I think any of these changes, or a combination of them, would
> >> significantly reduce

Re: [VOTE] Release dtest-api 0.0.10

[Oleksandr Petrov]

With 6 +1s, and no -1s, the vote passes.

On Wed, Oct 6, 2021 at 8:21 AM Dinesh Joshi 
wrote:

> +1
>
> Dinesh
>
> > On Oct 5, 2021, at 7:27 PM, Joshua McKenzie 
> wrote:
> >
> > +1
> >
> >> On Tue, Oct 5, 2021 at 2:15 PM Brandon Williams 
> wrote:
> >>
> >> +1
> >>
> >>> On Tue, Oct 5, 2021 at 11:47 AM Oleksandr Petrov
> >>>  wrote:
> >>>
> >>> Proposing the test build of in-jvm dtest API 0.0.10 for release.
> >>>
> >>> Repository:
> >>>
> >>
> https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.10
> >>>
> >>> Candidate SHA:
> >>>
> >>
> https://github.com/apache/cassandra-in-jvm-dtest-api/commit/2139b4c85e319b17afbdea2f653152d1e1895fc6
> >>> tagged with 0.0.10
> >>>
> >>> Artifacts:
> >>>
> >>
> https://repository.apache.org/content/repositories/orgapachecassandra-1249/org/apache/cassandra/dtest-api/0.0.10/
> >>>
> >>> Key signature: A4C465FEA0C552561A392A61E91335D77E3E87CB
> >>>
> >>> Changes since last release:
> >>>  * CASSANDRA-17013: CEP-10 Simulator Improvements
> >>>
> >>>
> >>> The vote will be open for 24 hours. Everyone who has tested the build
> >>> is invited to vote. Votes by PMC members are considered binding. A
> >>> vote passes if there are at least three binding +1s.
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>
> >>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
alex p

[VOTE] Release dtest-api 0.0.10

[Oleksandr Petrov]

Proposing the test build of in-jvm dtest API 0.0.10 for release.

Repository:
https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.10

Candidate SHA:
https://github.com/apache/cassandra-in-jvm-dtest-api/commit/2139b4c85e319b17afbdea2f653152d1e1895fc6
tagged with 0.0.10

Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1249/org/apache/cassandra/dtest-api/0.0.10/

Key signature: A4C465FEA0C552561A392A61E91335D77E3E87CB

Changes since last release:
  * CASSANDRA-17013: CEP-10 Simulator Improvements


The vote will be open for 24 hours. Everyone who has tested the build
is invited to vote. Votes by PMC members are considered binding. A
vote passes if there are at least three binding +1s.

Re: [VOTE] Release Apache Cassandra 4.0.0 (third time is the charm)

[Oleksandr Petrov]

+1

On Fri, Jul 23, 2021 at 7:40 AM Jeff Jirsa  wrote:

> +1
>
> > On Jul 22, 2021, at 3:41 PM, Brandon Williams <
> brandonwilli...@apache.org> wrote:
> >
> > I am proposing the test build of Cassandra 4.0.0 for release.
> >
> > sha1: 902b4d31772eaa84f05ffdc1e4f4b7a66d5b17e6
> > Git:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/4.0.0-tentative
> > Maven Artifacts:
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1244/org/apache/cassandra/cassandra-all/4.0.0/
> >
> > The Source and Build Artifacts, and Debian and RPM packages and
> > repositories are available here:
> > https://dist.apache.org/repos/dist/dev/cassandra/4.0.0/
> >
> > The vote will be open for 72 hours (longer if needed). Everyone who
> > has tested the build is invited to vote. Votes by PMC members are
> > considered binding. A vote passes if there are at least three binding
> > +1s and no -1's.
> >
> > [1]: CHANGES.txt:
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/4.0.0-tentative
> > [2]: NEWS.txt:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/4.0.0-tentative
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
alex p

Re: [VOTE] Release dtest-api 0.0.8

[Oleksandr Petrov]

+1

On Fri, Jul 9, 2021 at 12:29 PM Mick Semb Wever  wrote:

> Proposing the test build of in-jvm dtest API 0.0.8 for release.
>
> Repository:
>
> https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.8
>
> Candidate SHA:
>
> https://github.com/apache/cassandra-in-jvm-dtest-api/commit/db3cdf5823fbc49c6b5c53dc2b15330d3883fd09
> tagged with 0.0.8
>
> Artifacts:
>
> https://repository.apache.org/content/repositories/orgapachecassandra-1240/org/apache/cassandra/dtest-api/0.0.8/
>
> Key signature: A4C465FEA0C552561A392A61E91335D77E3E87CB
>
>
> Changes since last release:
>   * CASSANDRA-16649: Update Versions for trunk as 4.1 and new release
> branch cassandra-4.0
>   * CASSANDRA-16649: Introduce SemVer4j for version representation, parsing
> and handling
>
> The vote will be open for 24 hours. Everyone who has tested the build
> is invited to vote. Votes by PMC members are considered binding. A
> vote passes if there are at least three binding +1s.
>


-- 
alex p

Re: [DISCUSSION] Should we mark DROP COMPACT STORAGE as experimental

[Oleksandr Petrov]

Thank you for bringing this subject up.

> not ready for production use unless users fully understand what they are
doing.

Thing is, there's no easy way around dropping compact storage.  At the
moment of writing of 10857, we have collectively decided that we'll
document that the new columns are going to be shown, and have added a
client protocol option that would hide/show columns depending on the mode
we're running it in for anyone who upgrades. This makes it harder to make
a transition for anyone who controls only the server side, since you have
to account for how clients would behave whenever they see a new column. We
did try to patch around the shown columns, but because of ColumnFilter this
also turned out to be non-trivial, or at least not worth the effort for the
moment.

One of the things mentioned in this list (primary key liveness) is also
existing as a difference between UPDATE and INSERT, but I'm not sure if
it's properly documented. Similar to some other nuances, such as nulls in
clustering keys on partitions that only have a static row. We did recently
discuss some of these not-commonly-known cases with Benjamin and some other
folks. So it might be worth documenting those, too.

Problem with compact storage is that very few people want to touch it, and
it requires a non-trivial amount of "institutional" knowledge and
remembering things about Thrift. I think it's OK to mark the feature as
experimental, but remembering how we haven't made significant improvements
to things we have previously marked as experimental, this one may not
materialise into something final, too.

What would a complete, non-foot-gun solution for dropping compact storage
entail? If we're talking about avoiding showing columns to users, there are
ways to achieve this without rewriting sstables, for example, by
introducing "hidden" columns in table metadata. However, if we want to
preserve deletion semantics, I'm not sure if we're doing it right at all:
we'll just trade one source of difference for partition liveness for insert
queries for the other, so I'd say that, by executing ALTER TABLE statement,
you're accepting that after it propagates, there will be at least some
difference in behaviour and semantics. We did discuss this in C-16069, and
my thesis back then was that replacing special-casing for compact tables
with special casing for tables that "used to be compact" isn't  bringing us
closer to the final solution.

To summarise, I don't mind if we mark this feature experimental, but if we
want to ever make it complete, we have to discuss what we do with each of
the special cases. And it may very well be that we just need to add
explicit hidden columns to metadata, and allow nulls for clusterings, maybe
several more small changes. Unless we define what it would take to get this
feature out of experimental state, and actually make an effort to resolve
these issues, I'd just put a huge warning and call it a power-user feature.


On Fri, Jun 4, 2021 at 5:01 PM Joshua McKenzie  wrote:

> >
> > not ready for production use unless users fully understand what they are
> > doing.
>
> This statement stood out to me - in my opinion we should think carefully
> about the surface area of the user interfaces on new features before we add
> more cognitive burden to our users. We already have plenty of "foot-guns"
> in the project and should only add more if absolutely necessary.
>
> Further, marking this as experimental would be another feature we've
> released and then retroactively marked as experimental; that's a habit we
> should not get into.
>
> On balance, my .02 is the benefits to our end users and operators of
> getting 4.0 to GA outweigh the costs of flagging this as experimental now
> so I'm a +1 to the flagging idea, but I think there's some valuable lessons
> for us to learn in retrospect from not just this feature but others like it
> in the past.
>
> Curious to hear Alex' thoughts about this situation in particular as author
> of C-10857. I recall that being a pretty painful slog so apologies in
> advance for picking at this scab. :)
>
>
>
> On Fri, Jun 4, 2021 at 9:44 AM Brandon Williams  wrote:
>
> > +1
> >
> > On Fri, Jun 4, 2021, 3:53 AM Benjamin Lerer  wrote:
> >
> > > Hi everybody,
> > >
> > > There are a significant amount of issues with DROP COMPACT STORAGE that
> > can
> > > be pretty surprising for users.
> > > To name a few:
> > > * Some hidden columns will show up changing the resultset returned for
> > > wildcard queries
> > > * As COMPACT tables did not have primary key liveness there empty rows
> > > inserted AFTER the ALTER will be returned whereas the one inserted
> before
> > > the ALTER will not.
> > > * Also due to the lack of primary key liveness the amount of SSTables
> > being
> > > read will increase resulting in slower queries
> > > * After DROP COMPACT it becomes possible to ALTER the table in a way
> that
> > > makes all the row disappears
> > > * There is a loss of functionality around nul

Re: Pushing CASSANDRA-16262 out of 4.0-rc

[Oleksandr Petrov]

In the absence of a fuzz testing tool I would probably support excluding
this ticket from GA, but speaking from recent experience it feels to me
that fixing bugs is blocking us from further fuzz testing more than fuzz
testing is blocking us from releases. In other words, you run a fuzz test
for a relatively short amount of time, and then spend several days to fix
and merge the issue that was triggered to unblock further testing.


In a relatively short amount of time, we've been able to hit at least these
four issues:

   - Group By in-jvm paging issue:
   https://issues.apache.org/jira/browse/CASSANDRA-16427
   - Group By breaks range tombstone closer:
   https://issues.apache.org/jira/browse/CASSANDRA-16431
   - Reverse iteration + paging:
   https://issues.apache.org/jira/browse/CASSANDRA-16435
   - NPE in Slice#make on RT + partition deletion reconciliation:
   https://issues.apache.org/jira/browse/CASSANDRA-16453

I think we would've hit the first one without a fuzz testing tool, since it
was a relatively obvious one, but looking at the output from Harry it was
immediately clear what's going on, so I still consider its output useful.
There was one more issue that was discovered independently, but was also
caught with a fuzz test tool later. Its prior discovery with a test tool
was blocked by the fix of an issue that was occurring more frequently.

Maybe it's ok to release a release candidate, but we probably should still
wait a bit to cut a final release. WDYT?




On Mon, Feb 1, 2021 at 1:33 PM Benjamin Lerer 
wrote:

> It seems to me that CASSANDRA-16180
> , CASSANDRA-16262
>  and
> CASSANDRA-16181
>  already highly
> improve the test coverage on distributed read and write paths.
> CASSANDRA-16262 sounds like a nice to have but should probably not block
> 4.0 GA
> I am +1 to push it out of 4.0-rc.
>
> Thanks to you and Andres for all the work you contributed on
> CASSANDRA-15579
> . It is some great
> work :-)
>
> On Thu, Jan 28, 2021 at 6:45 PM Caleb Rackliffe 
> wrote:
>
> > Hey everyone,
> >
> > I wanted to have a quick conversation about CASSANDRA-16262
> > . As I mentioned
> in
> > the Jira
> > <
> >
> https://issues.apache.org/jira/browse/CASSANDRA-16262?focusedCommentId=17273891&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17273891
> > >
> > ...
> >
> > *"I was chatting w/ Andres de la Peña
> > 
> > the
> > other day, and it feels like there's an argument for allowing 4.0 to
> > release without this work being complete. We've certainly come a long way
> > w/ CASSANDRA-15579
> >  already, filling
> > in
> > a number of gaps that existed."*
> >
> > Any strong opinions out there?
> >
>


-- 
alex p

Re: [VOTE] Release Apache Cassandra 3.11.10

[Oleksandr Petrov]

This vote has passed, after 3 days, with seven binding +1s, three
non-binding +1s, and no -1s.

On Mon, Feb 1, 2021 at 2:57 PM Brandon Williams  wrote:

> +1
>
> On Fri, Jan 29, 2021, 6:50 AM Oleksandr Petrov  >
> wrote:
>
> > Proposing the test build of Cassandra 3.11.10 for release.
> >
> > sha1: 181a4969290f1c756089b2993a638fe403bc1314
> > Git:
> >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.11.10-tentative
> > Maven Artifacts:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1231/org/apache/cassandra/cassandra-all/3.11.10/
> >
> > The Source and Build Artifacts, and the Debian and RPM packages and
> > repositories, are available here:
> > https://dist.apache.org/repos/dist/dev/cassandra/3.11.10/
> >
> > The vote will be open for 72 hours (longer if needed). Everyone who has
> > tested the build is invited to vote. Votes by PMC members are considered
> > binding. A vote passes if there are at least three binding +1s and no
> -1's.
> >
> > [1]: CHANGES.txt:
> >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/3.11.10-tentative
> > [2]: NEWS.txt:
> >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/3.11.10-tentative
> >
>


-- 
alex p

Re: [VOTE] Release Apache Cassandra 3.0.24

[Oleksandr Petrov]

This vote has passed, after 3 days, with seven binding +1s, three
non-binding +1s, and no -1s.

On Mon, Feb 1, 2021 at 2:57 PM Brandon Williams  wrote:

> +1
>
> On Fri, Jan 29, 2021, 6:48 AM Oleksandr Petrov  >
> wrote:
>
> > Proposing the test build of Cassandra 4.0-beta4 for release.
> >
> > sha1: 6748ecd63cae047b5b0e8c3165088252954e9d5f
> > Git:
> >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.0.24-tentative
> > Maven Artifacts:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1229/org/apache/cassandra/cassandra-all/3.0.24/
> >
> > The Source and Build Artifacts, and the Debian and RPM packages and
> > repositories, are available here:
> > https://dist.apache.org/repos/dist/dev/cassandra/3.0.24/
> >
> > The vote will be open for 72 hours (longer if needed). Everyone who has
> > tested the build is invited to vote. Votes by PMC members are considered
> > binding. A vote passes if there are at least three binding +1s and no
> -1's.
> >
> > [1]: CHANGES.txt:
> >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/3.0.24-tentative
> > [2]: NEWS.txt:
> >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/3.0.24-tentative
> >
>


-- 
alex p

[RELEASE] Apache Cassandra 3.0.24 released

[Oleksandr Petrov]

The Cassandra team is pleased to announce the release of Apache Cassandra
version 3.0.24.

Apache Cassandra is a fully distributed database. It is the right choice
when you need scalability and high availability without compromising
performance.

 http://cassandra.apache.org/

Downloads of source and binary distributions are listed in our download
section:

 http://cassandra.apache.org/download/

This version is a bug fix release[1] on the 3.0 series. As always, please
pay attention to the release notes[2] and Let us know[3] if you were to
encounter any problem.

Enjoy!

[1]: CHANGES.txt
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/cassandra-3.0.24
[2]: NEWS.txt
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/cassandra-3.0.24
[3]: https://issues.apache.org/jira/browse/CASSANDRA

[RELEASE] Apache Cassandra 3.11.10 released

[Oleksandr Petrov]

The Cassandra team is pleased to announce the release of Apache Cassandra
version 3.11.10.

Apache Cassandra is a fully distributed database. It is the right choice
when you need scalability and high availability without compromising
performance.

 http://cassandra.apache.org/

Downloads of source and binary distributions are listed in our download
section:

 http://cassandra.apache.org/download/

This version is a bug fix release[1] on the 3.11 series. As always, please
pay attention to the release notes[2] and Let us know[3] if you were to
encounter any problem.

Enjoy!

[1]: CHANGES.txt
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/cassandra-3.11.10
[2]: NEWS.txt
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/cassandra-3.11.10
[3]: https://issues.apache.org/jira/browse/CASSANDRA

Re: [VOTE] Release Apache Cassandra 3.0.24

[Oleksandr Petrov]

> Proposing the test build of Cassandra 4.0-beta4 for release.

Correction: test build of 3.0.24. The rest looks right.

On Fri, Jan 29, 2021 at 1:48 PM Oleksandr Petrov 
wrote:

> Proposing the test build of Cassandra 4.0-beta4 for release.
>
> sha1: 6748ecd63cae047b5b0e8c3165088252954e9d5f
> Git:
>
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.0.24-tentative
> Maven Artifacts:
>
> https://repository.apache.org/content/repositories/orgapachecassandra-1229/org/apache/cassandra/cassandra-all/3.0.24/
>
> The Source and Build Artifacts, and the Debian and RPM packages and
> repositories, are available here:
> https://dist.apache.org/repos/dist/dev/cassandra/3.0.24/
>
> The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>
> [1]: CHANGES.txt:
>
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/3.0.24-tentative
> [2]: NEWS.txt:
>
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/3.0.24-tentative
>


-- 
alex p

[VOTE] Release Apache Cassandra 3.11.10

[Oleksandr Petrov]

Proposing the test build of Cassandra 3.11.10 for release.

sha1: 181a4969290f1c756089b2993a638fe403bc1314
Git:
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.11.10-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1231/org/apache/cassandra/cassandra-all/3.11.10/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/3.11.10/

The vote will be open for 72 hours (longer if needed). Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s and no -1's.

[1]: CHANGES.txt:
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/3.11.10-tentative
[2]: NEWS.txt:
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/3.11.10-tentative

[VOTE] Release Apache Cassandra 3.0.24

[Oleksandr Petrov]

Proposing the test build of Cassandra 4.0-beta4 for release.

sha1: 6748ecd63cae047b5b0e8c3165088252954e9d5f
Git:
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.0.24-tentative
Maven Artifacts:
https://repository.apache.org/content/repositories/orgapachecassandra-1229/org/apache/cassandra/cassandra-all/3.0.24/

The Source and Build Artifacts, and the Debian and RPM packages and
repositories, are available here:
https://dist.apache.org/repos/dist/dev/cassandra/3.0.24/

The vote will be open for 72 hours (longer if needed). Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s and no -1's.

[1]: CHANGES.txt:
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/3.0.24-tentative
[2]: NEWS.txt:
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/3.0.24-tentative

Re: [DISCUSS] Bringing protocol v5 out of beta and dropping support from 3.11.x

[Oleksandr Petrov]

> Nit-picking,  "beta" isn't sounding like the most accurate classifier
here. It sounds to me more like it is "in development", i.e. 'dev', rather
than beta.

I agree that `dev` rather than `beta` is a better classifier. We probably
should rename it at least internally and in the spec. That might be harder
to do in drivers since usually they're offering backward-compatible APIs;
maybe there we could deprecate `allowBetaVersions` and introduce
`allowDevelopmentVersions`.

On Tue, Dec 15, 2020 at 6:29 PM Sam Tunnicliffe  wrote:

> > What's the verdict now for CASSANDRA-14973 ?
>
> My aim is to have a C* patch and PRs for the python and java drivers this
> week, but really there's nothing to block cutting a new C* beta now (or
> whenever we're ready).
>
> > On 15 Dec 2020, at 15:39, Mick Semb Wever  wrote:
> >
> >>
> >> To use a beta flag, one absolutely has to have matching server
> >> and client versions, since otherwise things can break in unexpected
> ways.
> >> In fact, this specific issue makes it easy since you'd see that
> something
> >> has changed immediately.
> >>
> >
> >
> > That could certainly deserve better documentation. For example, an extra
> > sentence at
> >
> https://github.com/apache/cassandra/blob/trunk/doc/native_protocol_v5.spec#L140-L142
> >
> > And it would also be awesome to see this on the website docs. It's a bit
> of
> > tribal knowledge and crawling jira tickets atm.
> >
> > I think it can also be made more explicit here by what we mean by
> "matching
> > server and client versions". To my understanding this is not explicit
> > versions within V5, as there is only that, but just
> > different build/implementation versions. And that this basically also
> means
> > mixed server versions are outside the scope of the beta flag.
> >
> > Nit-picking,  "beta" isn't sounding like the most accurate classifier
> here.
> > It sounds to me more like it is "in development", i.e. 'dev', rather than
> > beta.
> >
> >
> >> +1 to have 4.0 with v5.
> >
> >
> > I agree, v6 is not needed. What's the verdict now for CASSANDRA-14973 ?
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
alex p

Re: [DISCUSS] Bringing protocol v5 out of beta and dropping support from 3.11.x

[Oleksandr Petrov]

I wanted to point out that beta version of the protocol was created [1]
only for development purposes:

> This is primarily useful for driver authors to start work against a new
protocol version when the work on that spans multiple releases. Users would
not generally be expected to utilize this flag, although it could
potentially be used to offer early feedback on new protocol features.

Even at the time of creating [1], it was expected that development of the
new protocol version would span multiple versions. Unless release of the
new protocol version strictly coincides with a major version bump, even a
major one. To use a beta flag, one absolutely has to have matching server
and client versions, since otherwise things can break in unexpected ways.
In fact, this specific issue makes it easy since you'd see that something
has changed immediately.

+1 to have 4.0 with v5.



[1] https://issues.apache.org/jira/browse/CASSANDRA-12142

On Fri, Dec 11, 2020 at 5:02 PM Sam Tunnicliffe  wrote:

>
> After a little bit of investigation, it turns out that we can't bring v5
> out of beta in 3.11 as there are already a number of v5 features which 3.11
> doesn't support. This necessarily couples the C*, protocol and driver
> versions. For instance, java driver 3.0.x is able to support v5 with 3.11,
> but not with 4.0 (and the inverse is true of driver 3.10.x). Going back to
> the genesis of v5 in CASSANDRA-10786, this was always the intent:
>
> "v5 beta" will mean different things at different times, and there
> might be errors when an older driver version (supporting "v5 beta with new
> feature A") connects to a newer C* version (supporting "v5 beta with
> new features A and B"). But with the provision that "bad things can happen
> when you're in beta" that's acceptable, so we don't need to make things any
> more complicated." [1]
>
> This simplifies things, we promote v5 out of beta in trunk only with no
> need to bump the framing format to v6-beta. If clients currently using
> v5-beta on 3.11.x want to upgrade their drivers, they must either revert to
> v4 or upgrade the cluster to 4.0. Because CI uses the same python driver
> version for all C* branches, we'll need to make dtests which require v5
> exclusive to 4.0, which I'll take care of. Tickets already exist for the
> python and java drivers ([2],[3],[4]), but we'll need to bundle a SNAPSHOT
> java driver with C* temporarily for testing. I'll file a 4.0-rc blocker to
> replace it with a release version, as we've done previously.
>
> [1]
> https://issues.apache.org/jira/browse/CASSANDRA-10786?focusedCommentId=15344506&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15344506
> [2] https://datastax-oss.atlassian.net/browse/PYTHON-1232
> [3] https://datastax-oss.atlassian.net/browse/JAVA-2704
> [4] https://datastax-oss.atlassian.net/browse/JAVA-2705
>
>
> > On 9 Dec 2020, at 11:02, Sam Tunnicliffe  wrote:
> >
> >
> >
> >> On 9 Dec 2020, at 02:41, Sumanth Pasupuleti <
> sumanth.pasupuleti...@gmail.com> wrote:
> >>
> >> +1 to moving v5-beta changes in trunk to new protocol v6.
> >>
> >> Regarding 3.11 and v5, I suppose we could say, v5 could never get
> matured
> >> beyond beta, but not sure if it would be confusing to see v6 while v5 is
> >> still in beta (curious on others' thoughts as well)
> >>
> >> On Tue, Dec 8, 2020 at 12:33 PM Mick Semb Wever  wrote:
> >>
>  …move to v6 for the new protocol to avoid this issue
> >>>
> >>>
> >>> +1,  feels more the "grown-up thing to do".
> >>>
> >>>
> > Perhaps we should skip v5…
>  We could finalise v5 as it was prior to the new framing format, then
> >>> create v6-beta in trunk only with the 15299 changes.
> >>>
> >>>
> >>> Would v5 come out of beta then in 3.11?, or would it stay as beta
> forever,
> >>> with the focus on maturing v6 in 4.0+?
> >>>
> >
> >
> > Technically, it *could* come out of beta in 4.0, but stay as it is in
> 3.11. That is, the v5 protocol itself would be identical in both C*
> versions, but there would be some considerations on the client side.
> Namely, if connecting to a cluster with any 3.11 nodes, clients would need
> to set the beta flag in CQL frame (meaning a frame in v5 terms). Any 4.0
> nodes in the cluster would simply ignore the flag, as it wouldn't be
> required for v5 connections. So the most straightforward path would be
> probably to promote v5 out of beta in the next 3.11 and 4.0-beta releases.
> Clients would just need to ensure that they stay on a *current* driver
> (i.e. one which sets that flag for v5) until after upgrading clusters to
> either the latest 3.11 or 4.0-beta releases.
> >
> > I have patches for C*, java and python drivers ready, so I'll file a
> JIRA and open some PRs.
> >
> >
> >
> >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
>
> -

[VOTE] Release dtest-api 0.0.7

[Oleksandr Petrov]

Proposing the test build of in-jvm dtest API 0.0.7 for release.

Repository:https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.7

Candidate 
SHA:https://github.com/apache/cassandra-in-jvm-dtest-api/commit/d5174b1f44b7d9cb919d4975b4d437041273c09c
tagged with 0.0.7
Artifact:https://repository.apache.org/content/repositories/orgapachecassandra-1225/org/apache/cassandra/dtest-api/0.0.7/

Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C

Changes since last release:

  * CASSANDRA-16136: Add Metrics to instance API
  * CASSANDRA-16272: Nodetool assert apis do not include the new
stdout and stderr in the failure message

The vote will be open for 24 hours. Everyone who has tested the build
is invited to vote. Votes by PMC members are considered binding. A
vote passes if there are at least three binding +1s.

-- Alex

Re: Minimal 4.0 COMPACT STORAGE backport

[Oleksandr Petrov]

These are some great (albeit somewhat hard) questions!

> * when you drop compact storage and have some SSTables stored in
compact storage and some SSTables stored normally, how do we present data
correctly?

There is no difference in SSTable format between normal and compact
storage, the difference is only in how Table Metadata is represented. In
other words, in code.

* does upgradesstables and/or compaction migrate off the compact
storage format into the normal format?

I think above answers this: the only way to migrate off compact storage is
to change table metadata, and accept the differences mentioned above.

* how do we know it is safe to remove?

Maybe if we're explicit about it and say that, for example, by version X
compact storage will not be supported at all, and document all the
pitfalls, people will migrate their applications, one at a time. There's
been a few suggestions about how to mitigate, including client options in
10857, and introducing "hiding" columns in 15811, and many more that were
discussed but not implemented.

Hope this won't discourage further discussions, but I suggest we keep this
thread focused on pros/cons of a suggested intermediate solution (bring
back a minimal subset of CS), and collect ideas about migration, cutoff
date and final deprecation in a separate thread or a document. I personally
do not have a full answer to "when it is safe to remove", and it seems like
the best we can do is to create a clear procedure.

On Thu, Oct 22, 2020 at 6:38 PM Ekaterina Dimitrova 
wrote:

> Definitely that is not off the table.
> Also, we talked this morning for a plan for additional testing to be
> created as part of CASSANDRA-15588
>
> On Thu, 22 Oct 2020 at 12:33, David Capwell  wrote:
>
> > I am in favor of bringing it back, but I do feel we should also plan how
> to
> > get it removed as well.
> >
> > Few examples, would love to see fleshed out
> >
> > * when you drop compact storage and have some SSTables stored in compact
> > storage and some SSTables stored normally, how do we present data
> > correctly?
> > * does upgradesstables and/or compaction migrate off the compact storage
> > format into the normal format?
> > * how do we know it is safe to remove?
> >
> >
> >
> > On Thu, Oct 22, 2020 at 6:55 AM Ekaterina Dimitrova <
> e.dimitr...@gmail.com
> > >
> > wrote:
> >
> > > Hi Alex,
> > > Thanks for bringing this up.
> > > I am with you for returning part of the code back and considering this
> > as a
> > > bug.
> > > I truly believe it is too late in the release to document changed
> > behavior.
> > > I think this contradicts with the project’s promise for no breaking
> > > changes. This should have been documented in alpha.
> > > Best regards,
> > > Ekaterina
> > >
> > > On Thu, 22 Oct 2020 at 3:20, Oleksandr Petrov <
> > oleksandr.pet...@gmail.com>
> > > wrote:
> > >
> > > > Since this is an important subject, I thought it also makes sense to
> > > start
> > > > a mailing list thread.
> > > >
> > > > You may know that in 4.0 there was a plan to drop compact storage and
> > > > related code. However, there are several behavioural changes related
> to
> > > > compact storage, and difference in visible behaviour between "normal"
> > and
> > > > compact tables are larger than most of us have anticipated: we first
> > > > thought there’ll be only “appearing column” in dense case, but
> there’s
> > > > implicit nulls in clusterings thing, and row vs column deletion now,
> > TTL,
> > > > and more.
> > > >
> > > > Some of the recent issues on the subject are: CASSANDRA-16048
> > > > <https://issues.apache.org/jira/browse/CASSANDRA-16048>, which
> allows
> > to
> > > > ignore these differences. The other one was an attempt to improve the
> > > user
> > > > experience of anyone still using compact storage: CASSANDRA-15811
> > > > <https://issues.apache.org/jira/browse/CASSANDRA-15811>.
> > > >
> > > > Easily reproducible differences are:
> > > >
> > > > (1) hidden columns show up, which breaks SELECT * queries
> > > > (2) DELETE v and UPDATE v WITH TTL would result into row removals in
> > > > non-dense compact tables (CASSANDRA-16069
> > > > <https://issues.apache.org/jira/browse/CASSANDRA-16069>)
> > > > (3) INSERT allows skipping clusterings, which are filled with nulls
> by
> > > > default.
&g

Minimal 4.0 COMPACT STORAGE backport

[Oleksandr Petrov]

Since this is an important subject, I thought it also makes sense to start
a mailing list thread.

You may know that in 4.0 there was a plan to drop compact storage and
related code. However, there are several behavioural changes related to
compact storage, and difference in visible behaviour between "normal" and
compact tables are larger than most of us have anticipated: we first
thought there’ll be only “appearing column” in dense case, but there’s
implicit nulls in clusterings thing, and row vs column deletion now, TTL,
and more.

Some of the recent issues on the subject are: CASSANDRA-16048
, which allows to
ignore these differences. The other one was an attempt to improve the user
experience of anyone still using compact storage: CASSANDRA-15811
.

Easily reproducible differences are:

(1) hidden columns show up, which breaks SELECT * queries
(2) DELETE v and UPDATE v WITH TTL would result into row removals in
non-dense compact tables (CASSANDRA-16069
)
(3) INSERT allows skipping clusterings, which are filled with nulls by
default.

Some of these are tricky to support, as 15811 has shown. Anyone who might
want to upgrade to 4.0 while still using compact storage might be affected
by being forced into one of these behaviours.

Possible solutions are to document these behaviours, or to bring back a
minimal set of COMPACT STORAGE and keep supporting these in 4.0

It looks like it is possible to leave some of the functionality related to
DENSE flag and allow it to be present in 4.0, but only for these three (and
potential related, however not directly visible) cases.

You can find more details on the subject here:
https://issues.apache.org/jira/browse/CASSANDRA-16217

Thank you,

-- Alex

Re: [VOTE] Release dtest-api 0.0.6

[Oleksandr Petrov]

With 7 +1 votes, and no -1s, the vote passes.

On Fri, Oct 9, 2020 at 1:12 AM Nate McCall  wrote:

> +1
>
> On Thu, Oct 8, 2020 at 9:20 PM Oleksandr Petrov <
> oleksandr.pet...@gmail.com>
> wrote:
>
> > Proposing the test build of in-jvm dtest API 0.0.6 for release.
> >
> > Repository:
> >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.6
> >
> > Candidate SHA:
> >
> >
> https://github.com/apache/cassandra-in-jvm-dtest-api/commit/9efeb731b6ff4036fa822b0282b27d273975cd6fTT
> > tagged with 0.0.6
> > Artifact:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1220/org/apache/cassandra/dtest-api/0.0.6/
> >
> > Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C
> >
> > Changes since last release:
> >
> >   * CASSANDRA-16148: Add IInstance#getReleaseVersionString
> >
> > The vote will be open for 24 hours. Everyone who has tested the build is
> > invited to vote. Votes by PMC members are considered binding. A vote
> passes
> > if there are at least three binding +1s.
> >
> > -- Alex
> >
>


-- 
alex p

[VOTE] Release dtest-api 0.0.6

[Oleksandr Petrov]

Proposing the test build of in-jvm dtest API 0.0.6 for release.

Repository:
https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.6

Candidate SHA:
https://github.com/apache/cassandra-in-jvm-dtest-api/commit/9efeb731b6ff4036fa822b0282b27d273975cd6fTT
tagged with 0.0.6
Artifact:
https://repository.apache.org/content/repositories/orgapachecassandra-1220/org/apache/cassandra/dtest-api/0.0.6/

Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C

Changes since last release:

  * CASSANDRA-16148: Add IInstance#getReleaseVersionString

The vote will be open for 24 hours. Everyone who has tested the build is
invited to vote. Votes by PMC members are considered binding. A vote passes
if there are at least three binding +1s.

-- Alex

Re: [VOTE] Release dtest-api 0.0.5

[Oleksandr Petrov]

0.0.5 is now released:
https://repository.apache.org/service/local/repositories/releases/content/org/apache/cassandra/dtest-api/0.0.5/

On Thu, Oct 1, 2020 at 8:01 AM Oleksandr Petrov 
wrote:

> With 8 +1 votes, and no -1s, the vote passes.
>
>
> On Fri, Sep 25, 2020 at 5:50 PM Yifan Cai  wrote:
>
>> +1 nb
>>
>> 
>> From: Jon Meredith 
>> Sent: Friday, September 25, 2020 8:39:31 AM
>> To: dev@cassandra.apache.org 
>> Subject: Re: [VOTE] Release dtest-api 0.0.5
>>
>> +1 (non-binding)
>>
>> On Fri, Sep 25, 2020 at 9:16 AM Marcus Eriksson 
>> wrote:
>> >
>> > +1
>> >
>> > On 25 September 2020 at 17:13:36, Chris Lohfink (clohfin...@gmail.com)
>> wrote:
>> > > +1
>> > >
>> > > On Fri, Sep 25, 2020 at 10:11 AM Caleb Rackliffe
>> > > wrote:
>> > >
>> > > > +1
>> > > >
>> > > > On Fri, Sep 25, 2020 at 10:08 AM Brandon Williams
>> > > > wrote:
>> > > >
>> > > > > +1
>> > > > >
>> > > > > On Fri, Sep 25, 2020, 9:45 AM Oleksandr Petrov <
>> > > > oleksandr.pet...@gmail.com
>> > > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Proposing the test build of in-jvm dtest API 0.0.5 for release.
>> > > > > >
>> > > > > > Repository:
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.5
>> > > > > > Candidate SHA:
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> https://github.com/apache/cassandra-in-jvm-dtest-api/commit/f900334d2f61f0b10640ba7ae15958f26df72d92
>> > > > > > tagged with 0.0.5
>> > > > > > Artifact:
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> https://repository.apache.org/content/repositories/orgapachecassandra-1219/org/apache/cassandra/dtest-api/0.0.5/
>> > > > > >
>> > > > > > Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C
>> > > > > >
>> > > > > > Changes since last release:
>> > > > > >
>> > > > > > * CASSANDRA-16109: If user has not set nodeCount, use the node
>> id
>> > > > > > topology size
>> > > > > > * CASSANDRA-16057: Update in-jvm dtest to expose stdout and
>> stderr
>> > > > for
>> > > > > > nodetool
>> > > > > > * CASSANDRA-16120: Add ability for jvm-dtest to grep instance
>> logs
>> > > > > > * CASSANDRA-16101: Add method to ignore uncaught throwables
>> > > > > > * CASSANDRA-16109: Collect dc/rack information and validate when
>> > > > > building
>> > > > > > * CASSANDRA-15386: Default to 3 datadirs in in-jvm dtests
>> > > > > > * CASSANDRA-16101: Add method to fetch uncaught exceptions
>> > > > > >
>> > > > > > The vote will be open for 24 hours. Everyone who has tested the
>> build
>> > > > is
>> > > > > > invited to vote. Votes by PMC members are considered binding. A
>> vote
>> > > > > passes
>> > > > > > if there are at least three binding +1s.
>> > > > > >
>> > > > > > -- Alex
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> > For additional commands, e-mail: dev-h...@cassandra.apache.org
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>
>>
>
> --
> alex p
>


-- 
alex p

Re: [VOTE] Release dtest-api 0.0.5

[Oleksandr Petrov]

With 8 +1 votes, and no -1s, the vote passes.


On Fri, Sep 25, 2020 at 5:50 PM Yifan Cai  wrote:

> +1 nb
>
> 
> From: Jon Meredith 
> Sent: Friday, September 25, 2020 8:39:31 AM
> To: dev@cassandra.apache.org 
> Subject: Re: [VOTE] Release dtest-api 0.0.5
>
> +1 (non-binding)
>
> On Fri, Sep 25, 2020 at 9:16 AM Marcus Eriksson 
> wrote:
> >
> > +1
> >
> > On 25 September 2020 at 17:13:36, Chris Lohfink (clohfin...@gmail.com)
> wrote:
> > > +1
> > >
> > > On Fri, Sep 25, 2020 at 10:11 AM Caleb Rackliffe
> > > wrote:
> > >
> > > > +1
> > > >
> > > > On Fri, Sep 25, 2020 at 10:08 AM Brandon Williams
> > > > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > On Fri, Sep 25, 2020, 9:45 AM Oleksandr Petrov <
> > > > oleksandr.pet...@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Proposing the test build of in-jvm dtest API 0.0.5 for release.
> > > > > >
> > > > > > Repository:
> > > > > >
> > > > > >
> > > > >
> > > >
> https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.5
> > > > > > Candidate SHA:
> > > > > >
> > > > > >
> > > > >
> > > >
> https://github.com/apache/cassandra-in-jvm-dtest-api/commit/f900334d2f61f0b10640ba7ae15958f26df72d92
> > > > > > tagged with 0.0.5
> > > > > > Artifact:
> > > > > >
> > > > > >
> > > > >
> > > >
> https://repository.apache.org/content/repositories/orgapachecassandra-1219/org/apache/cassandra/dtest-api/0.0.5/
> > > > > >
> > > > > > Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C
> > > > > >
> > > > > > Changes since last release:
> > > > > >
> > > > > > * CASSANDRA-16109: If user has not set nodeCount, use the node id
> > > > > > topology size
> > > > > > * CASSANDRA-16057: Update in-jvm dtest to expose stdout and
> stderr
> > > > for
> > > > > > nodetool
> > > > > > * CASSANDRA-16120: Add ability for jvm-dtest to grep instance
> logs
> > > > > > * CASSANDRA-16101: Add method to ignore uncaught throwables
> > > > > > * CASSANDRA-16109: Collect dc/rack information and validate when
> > > > > building
> > > > > > * CASSANDRA-15386: Default to 3 datadirs in in-jvm dtests
> > > > > > * CASSANDRA-16101: Add method to fetch uncaught exceptions
> > > > > >
> > > > > > The vote will be open for 24 hours. Everyone who has tested the
> build
> > > > is
> > > > > > invited to vote. Votes by PMC members are considered binding. A
> vote
> > > > > passes
> > > > > > if there are at least three binding +1s.
> > > > > >
> > > > > > -- Alex
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
alex p

[VOTE] Release dtest-api 0.0.5

[Oleksandr Petrov]

Proposing the test build of in-jvm dtest API 0.0.5 for release.

Repository:
https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.5
Candidate SHA:
https://github.com/apache/cassandra-in-jvm-dtest-api/commit/f900334d2f61f0b10640ba7ae15958f26df72d92
tagged with 0.0.5
Artifact:
https://repository.apache.org/content/repositories/orgapachecassandra-1219/org/apache/cassandra/dtest-api/0.0.5/

Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C

Changes since last release:

  * CASSANDRA-16109: If user has not set nodeCount, use the node id
topology size
  * CASSANDRA-16057: Update in-jvm dtest to expose stdout and stderr for
nodetool
  * CASSANDRA-16120: Add ability for jvm-dtest to grep instance logs
  * CASSANDRA-16101: Add method to ignore uncaught throwables
  * CASSANDRA-16109: Collect dc/rack information and validate when building
  * CASSANDRA-15386: Default to 3 datadirs in in-jvm dtests
  * CASSANDRA-16101: Add method to fetch uncaught exceptions

The vote will be open for 24 hours. Everyone who has tested the build is
invited to vote. Votes by PMC members are considered binding. A vote passes
if there are at least three binding +1s.

-- Alex

Re: Cassandra Contributor Meeting to focus on outstanding 4.0 issues

[Oleksandr Petrov]

Is it possible to find a time slot that is earlier than 12:30PM PST? It's
9:30PM for most of Europe [1], and this meeting is rather important for
many of us to attend.

[1] https://savvytime.com/converter/pst-to-sgt-cet-gmt/12-30pm

On Fri, Sep 25, 2020 at 2:23 AM Patrick McFadin  wrote:

> Hi everyone,
>
> First, I want to acknowledge some of the raw conversations today in the
> cassandra-dev slack channel. It was probably well overdue and if not for
> 2020 and what a wonderful year this has been, we might have gotten there
> earlier. I really appreciate how everyone who participated kept the
> conversation from completely devolving into something that wouldn't get us
> anywhere as a project. I also want to say that I hope we don't forget to
> pause and give each other a break. People are already tense and on edge
> from months of pandemics, politics, social unrest, disasters, child care
> issues and a lot of other things on a long list. I find it helpful to
> assume that everyone I talk to is stressed out, behind schedule and
> probably has a boss pressuring them to do something. We share a really
> unique common bond by willfully choosing to work on distributed systems. I
> hope we find more reasons to come together than be apart in the future.
>
> If I can even attempt to sum up the important themes from a very long
> conversation. There is a tension between the frozen codebase leading to 4.0
> and the desire to add new features beyond 4.0.  One way to eliminate that
> tension is to get 4.0 wrapped up. What is left to get 4.0 done isn't clear
> to many and there is a desire to get clear on the remaining items.
>
> I would like to contribute some of my energy and time to help this along.
> Standing on the shoulders of giants here, a LOT of work has already gone
> into this topic. Josh, Jordan and Jon (J^3 ?) kept a steady strain on
> updating jiras and status. This Kanban is really helpful:
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355&quickFilter=1661
>
> There is an epic that was discussed in slack:
> https://issues.apache.org/jira/browse/CASSANDRA-15536
>
> There are 17 separate jiras as dependencies, of which only 2 are marked as
> completed. 3 have no assigned owner. I have put a Zoom call on the
> contributor meeting calendar to discuss those three first. I had to weave
> it between ApacheCon sessions but hopefully this will help in velocity.
> Agenda
> posted, feel free to alter as you see fit. September 29, 1230PM PST:
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/2020-09-29+Apache+Cassandra+Contributor+Meeting+-+4.0+push+edition
>
> I mentioned this is Slack. We are so close. Beta 2 is out there, Harry just
> dropped and Jiras have been closing. I get excited when I think of the
> impact this will have. My entire life has now somehow become dependent on
> Cassandra working. When I take a pic with my iPhone. Play games on my
> PlayStation. Use curb-side pickup at Target. Order food on GrubHub.
> Everybody here helped make that happen and there are a lot of great
> engineers that will be deploying 4.0 and loving it. Most importantly, I
> reliably get food when I'm playing Ghost of Tsushima and take a selfie for
> my finsta. #priorities I would love to stick it to 2020 and make "Shipping
> 4.0" a positive event as we leave this stinking year behind.
>
> Patrick
>


-- 
alex p

Re: Creating a branch for 5.0 …?

[Oleksandr Petrov]

> So, here's what I've done, in an effort to make a space for both of these
groups to operate: the exact same thing we've done for every release in the
past. I created a branch for the 4.0 release.

I agree that everyone is free to work on whatever they want, but it seems
like having a 4.next branch isn't a prerequisite for any new (or ongoing)
development.

I appreciate the cheerful tone in which this message was written, but I
still think we should try and find an agreement on such things before
putting them into action.

On Thu, Sep 24, 2020 at 4:31 PM Brandon Williams  wrote:

> It's been a while now for this thread, but it seems to me that it has
> been established:
>
> 1. This is an opensource project and anyone is free to work on any
> part of it that they choose. Nobody has authority over this other than
> the contributor.
> 2. Some people are concerned that allowing innovation (in code) will
> make 4.0 take longer to release and cause them the pain of merging up
> yet another branch.
>
> So, here's what I've done, in an effort to make a space for both of
> these groups to operate: the exact same thing we've done for every
> release in the past.  I created a branch for the 4.0 release.
>
> WHAT HAVE YOU DONE?!
>
> Alright, calm down.  You're in group 2, don't worry:  you have your
> 4.0 branch! You can completely focus on it, and if that's all you want
> to do, that's fine! YOU DO NOT NEED TO MERGE TO TRUNK.  Those who wish
> to volunteer their time merging from 4.0 to trunk will pick up this
> mantle.  If nobody does and I get exhausted, we'll just abort it and
> delete the branch, no big deal.  One more time to make this crystal
> clear: IF YOU DO NOT WANT TO MERGE FROM 4.0 TO TRUNK, YOU ARE ABSOLVED
> FROM THIS DUTY.  The branch is named, unsurprisingly, 'cassandra-4.0'
>
> As for those who wish to begin working on new features, trunk is now
> open for business.
>
> The merge order is now 2.1->2.2->3.0->3.11->4.0 and _optionally_,
> should you _choose_, trunk.
>
> The show must go on.
>
> On Wed, Sep 16, 2020 at 12:08 PM Dinesh Joshi  wrote:
> >
> > Maybe you should ask these people to bring their contributions or issues
> directly to the dev list. You don’t need to disclose their names or contact
> information.
> >
> > Contributing to the project involves engaging the community. We’re still
> open to discussions even if the patches may not land immediately.
> >
> > If they don’t talk to the dev list and won’t make a case for their
> contribution (assuming it’s a big one) we can’t discuss possible ways
> forward to accept it.
> >
> > It also seems that these folks are interested in contributing new
> features to Cassandra. When the community is working towards stabilizing
> 4.0, contributing to that effort will help build goodwill. We’re averse to
> one off, drive-by contributions. I am not assuming that’s the case but the
> fact is that we’re discussing 5.0 here.
> >
> > Dinesh
> >
> > > On Sep 16, 2020, at 6:00 AM, Joshua McKenzie 
> wrote:
> > >
> > > People aren't lining up waiting to contribute to the project until we
> > > accept non-4.0 quality-based contributions. There is a discrete window
> of
> > > opportunity where we as a project can make a first impression on folks
> > > interested in joining our community, and signals from people, the data
> we
> > > have available about contributors, as well as basic logic are all
> > > consistent that we are turning away new contributors, likely
> permanently.
> > > They're moving on to other projects, since "apparently the Cassandra
> > > project isn't interested in new contributions" (interviewees words 2
> weeks
> > > ago, not mine). Or same sentiment expressed by multiple major companies
> > > looking to find a storage coordination layer to put in front of their
> > > storage offerings, for instance.
> > >
> > > And sorry I can't give you specific names, dates, quotations, and/or
> > > contact information Benedict; it seems this rankles you as you
> continue to
> > > use terms like "hypothetical" and "mythical" to describe the very real
> > > humans I have spoken with over the course of the past year on this
> topic.
> > > If my constraints of confidentiality from the people I've interacted
> with
> > > are unacceptable for you in a discussion like this and you don't trust
> me
> > > enough to know I wouldn't overtly lie to try and shift an Overton
> Window,
> > > we should probably go ahead and agree to disagree on this conversation
> and
> > > let committers go forward and do what they think best for the project.
> > >
> > >
> > >
> > >> On Wed, Sep 16, 2020 at 5:09 AM, Benedict Elliott Smith <
> bened...@apache.org
> > >>> wrote:
> > >>
> > >> I know. I recognise that is a frustrating aspect of this discussion.
> It is
> > >> something hard to move on.
> > >>
> > >> So how about we wait until there's a concrete example we can discuss
> as a
> > >> community? If we don't have one, it doesn't seem pressing.
> > >>
> > >> On 16/09/2020, 08

Re: [DISCUSS] CEP-7 Storage Attached Index

[Oleksandr Petrov]

e are some broader implications of using the
> > trie that
> > > reach outside SAI itself, but talking about that would involve some
> > bits of
> > > information DataStax might not be ready to share?
> > >
> > > On Wed, Sep 23, 2020 at 11:00 AM Jeremiah D Jordan <
> jeremiah.jordan@
> > > gmail.com> wrote:
> > >
> > > Short question: looking forward, how are we going to maintain three
> > 2i
> > > implementations: SASI, SAI, and 2i?
> > >
> > > I think one of the goals stated in the CEP is for SAI to have
> parity
> > with
> > > 2i such that it could eventually replace it.
> > >
> > > On Sep 23, 2020, at 10:34 AM, Oleksandr Petrov <
> > >
> > > oleksandr.pet...@gmail.com> wrote:
> > >
> > > Short question: looking forward, how are we going to maintain three
> > 2i
> > > implementations: SASI, SAI, and 2i?
> > >
> > > Another thing I think this CEP is missing is rationale and
> motivation
> > > about why trie-based indexes were chosen over, say, B-Tree. We did
> > have a
> > > short discussion about this on Slack, but both arguments that I've
> > heard
> > > (space-saving and keeping a small subset of nodes in memory) work
> > only
> > >
> > > for
> > >
> > > the most primitive implementation of a B-Tree. Fully-occupied
> prefix
> > >
> > > B-Tree
> > >
> > > can have similar properties. There's been a lot of research on
> > B-Trees
> > >
> > > and
> > >
> > > optimisations in those. Unfortunately, I do not have an
> > implementation
> > > sitting around for a direct comparison, but I can imagine
> situations
> > when
> > > B-Trees may perform better because of simpler
> > >
> > > construction.
> > >
> > > Maybe we should even consider prototyping a prefix B-Tree to have a
> > more
> > > fair comparison.
> > >
> > > Thank you,
> > > -- Alex
> > >
> > > On Thu, Sep 10, 2020 at 9:12 AM Jasonstack Zhao Yang <
> > jasonstack.zhao@
> > > gmail.com> wrote:
> > >
> > > Thank you Patrick for hosting Cassandra Contributor Meeting for
> CEP-7
> > >
> > > SAI.
> > >
> > > The recorded video is available here:
> > >
> > > https://cwiki.apache.org/confluence/display/CASSANDRA/
> > > 2020-09-01+Apache+Cassandra+Contributor+Meeting
> > >
> > > On Tue, 1 Sep 2020 at 14:34, Jasonstack Zhao Yang <
> > jasonstack.zhao@gmail.
> > > com>
> > > wrote:
> > >
> > > Thank you, Charles and Patrick
> > >
> > > On Tue, 1 Sep 2020 at 04:56, Charles Cao 
> > wrote:
> > >
> > > Thank you, Patrick!
> > >
> > > On Mon, Aug 31, 2020 at 12:59 PM Patrick McFadin <
> pmcfa...@gmail.com
> > >
> > > wrote:
> > >
> > > I just moved it to 8AM for this meeting to better accommodate APAC.
> > >
> > > Please
> > >
> > > see the update here:
> > >
> > > https://cwiki.apache.org/confluence/display/CASSANDRA/
> > > 2020-08-01+Apache+Cassandra+Contributor+Meeting
> > >
> > > Patrick
> > >
> > > On Mon, Aug 31, 2020 at 10:04 AM Charles Cao  >
> > >
> > > wrote:
> > >
> > > Patrick,
> > >
> > > 11AM PST is a bad time for the people in the APAC timezone. Can we
> > move it
> > > to 7 or 8AM PST in the morning to accommodate their needs ?
> > >
> > > ~Charles
> > >
> > > On Fri, Aug 28, 2020 at 4:37 PM Patrick McFadin <
> pmcfa...@gmail.com
> > >
> > > wrote:
> > >
> > > Meeting scheduled.
> > >
> > > https://cwiki.apache.org/confluence/display/CASSANDRA/
> > > 2020-08-01+Apache+Cassandra+Contributor+Meeting
> > >
> > > Tuesday September 1st, 11AM PST. I added a basic bullet for the
> > >
> > > agenda
> > >
> > >

Re: [DISCUSS] CEP-7 Storage Attached Index

[Oleksandr Petrov]

I did see a bit about "future parity and beyond" which is more or less an
obvious goal. I think CEP should be more upfront with "eventually replace
it" bit, since it raises the question about what the people who are using
other index implementations can expect.

On Wed, Sep 23, 2020 at 6:00 PM Jeremiah D Jordan 
wrote:

> > Short question: looking forward, how are we going to maintain three 2i
> > implementations: SASI, SAI, and 2i?
>
> I think one of the goals stated in the CEP is for SAI to have parity with
> 2i such that it could eventually replace it.
>
>
> > On Sep 23, 2020, at 10:34 AM, Oleksandr Petrov <
> oleksandr.pet...@gmail.com> wrote:
> >
> > Short question: looking forward, how are we going to maintain three 2i
> > implementations: SASI, SAI, and 2i?
> >
> > Another thing I think this CEP is missing is rationale and motivation
> > about why trie-based indexes were chosen over, say, B-Tree. We did have a
> > short discussion about this on Slack, but both arguments that I've heard
> > (space-saving and keeping a small subset of nodes in memory) work only
> for
> > the most primitive implementation of a B-Tree. Fully-occupied prefix
> B-Tree
> > can have similar properties. There's been a lot of research on B-Trees
> and
> > optimisations in those. Unfortunately, I do not have an
> > implementation sitting around for a direct comparison, but I can imagine
> > situations when B-Trees may perform better because of simpler
> construction.
> > Maybe we should even consider prototyping a prefix B-Tree to have a more
> > fair comparison.
> >
> > Thank you,
> > -- Alex
> >
> >
> >
> > On Thu, Sep 10, 2020 at 9:12 AM Jasonstack Zhao Yang <
> > jasonstack.z...@gmail.com> wrote:
> >
> >> Thank you Patrick for hosting Cassandra Contributor Meeting for CEP-7
> SAI.
> >>
> >> The recorded video is available here:
> >>
> >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/2020-09-01+Apache+Cassandra+Contributor+Meeting
> >>
> >> On Tue, 1 Sep 2020 at 14:34, Jasonstack Zhao Yang <
> >> jasonstack.z...@gmail.com>
> >> wrote:
> >>
> >>> Thank you, Charles and Patrick
> >>>
> >>> On Tue, 1 Sep 2020 at 04:56, Charles Cao  wrote:
> >>>
> >>>> Thank you, Patrick!
> >>>>
> >>>> On Mon, Aug 31, 2020 at 12:59 PM Patrick McFadin 
> >>>> wrote:
> >>>>>
> >>>>> I just moved it to 8AM for this meeting to better accommodate APAC.
> >>>> Please
> >>>>> see the update here:
> >>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/2020-08-01+Apache+Cassandra+Contributor+Meeting
> >>>>>
> >>>>> Patrick
> >>>>>
> >>>>> On Mon, Aug 31, 2020 at 10:04 AM Charles Cao 
> >>>> wrote:
> >>>>>
> >>>>>> Patrick,
> >>>>>>
> >>>>>> 11AM PST is a bad time for the people in the APAC timezone. Can we
> >>>>>> move it to 7 or 8AM PST in the morning to accommodate their needs ?
> >>>>>>
> >>>>>> ~Charles
> >>>>>>
> >>>>>> On Fri, Aug 28, 2020 at 4:37 PM Patrick McFadin  >>>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Meeting scheduled.
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/2020-08-01+Apache+Cassandra+Contributor+Meeting
> >>>>>>>
> >>>>>>> Tuesday September 1st, 11AM PST. I added a basic bullet for the
> >>>> agenda
> >>>>>> but
> >>>>>>> if there is more, edit away.
> >>>>>>>
> >>>>>>> Patrick
> >>>>>>>
> >>>>>>> On Thu, Aug 27, 2020 at 11:31 AM Jasonstack Zhao Yang <
> >>>>>>> jasonstack.z...@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> +1
> >>>>>>>>
> >>>>>>>> On Thu, 27 Aug 2020 at 04:52, Ekaterina Dimitrova <
> >>>>>> e.dimitr...@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>&g

Re: [DISCUSS] CEP-7 Storage Attached Index

[Oleksandr Petrov]

Short question: looking forward, how are we going to maintain three 2i
implementations: SASI, SAI, and 2i?

Another thing I think this CEP is missing is rationale and motivation
about why trie-based indexes were chosen over, say, B-Tree. We did have a
short discussion about this on Slack, but both arguments that I've heard
(space-saving and keeping a small subset of nodes in memory) work only for
the most primitive implementation of a B-Tree. Fully-occupied prefix B-Tree
can have similar properties. There's been a lot of research on B-Trees and
optimisations in those. Unfortunately, I do not have an
implementation sitting around for a direct comparison, but I can imagine
situations when B-Trees may perform better because of simpler construction.
Maybe we should even consider prototyping a prefix B-Tree to have a more
fair comparison.

Thank you,
-- Alex



On Thu, Sep 10, 2020 at 9:12 AM Jasonstack Zhao Yang <
jasonstack.z...@gmail.com> wrote:

> Thank you Patrick for hosting Cassandra Contributor Meeting for CEP-7 SAI.
>
> The recorded video is available here:
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/2020-09-01+Apache+Cassandra+Contributor+Meeting
>
> On Tue, 1 Sep 2020 at 14:34, Jasonstack Zhao Yang <
> jasonstack.z...@gmail.com>
> wrote:
>
> > Thank you, Charles and Patrick
> >
> > On Tue, 1 Sep 2020 at 04:56, Charles Cao  wrote:
> >
> >> Thank you, Patrick!
> >>
> >> On Mon, Aug 31, 2020 at 12:59 PM Patrick McFadin 
> >> wrote:
> >> >
> >> > I just moved it to 8AM for this meeting to better accommodate APAC.
> >> Please
> >> > see the update here:
> >> >
> >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/2020-08-01+Apache+Cassandra+Contributor+Meeting
> >> >
> >> > Patrick
> >> >
> >> > On Mon, Aug 31, 2020 at 10:04 AM Charles Cao 
> >> wrote:
> >> >
> >> > > Patrick,
> >> > >
> >> > > 11AM PST is a bad time for the people in the APAC timezone. Can we
> >> > > move it to 7 or 8AM PST in the morning to accommodate their needs ?
> >> > >
> >> > > ~Charles
> >> > >
> >> > > On Fri, Aug 28, 2020 at 4:37 PM Patrick McFadin  >
> >> > > wrote:
> >> > > >
> >> > > > Meeting scheduled.
> >> > > >
> >> > >
> >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/2020-08-01+Apache+Cassandra+Contributor+Meeting
> >> > > >
> >> > > > Tuesday September 1st, 11AM PST. I added a basic bullet for the
> >> agenda
> >> > > but
> >> > > > if there is more, edit away.
> >> > > >
> >> > > > Patrick
> >> > > >
> >> > > > On Thu, Aug 27, 2020 at 11:31 AM Jasonstack Zhao Yang <
> >> > > > jasonstack.z...@gmail.com> wrote:
> >> > > >
> >> > > > > +1
> >> > > > >
> >> > > > > On Thu, 27 Aug 2020 at 04:52, Ekaterina Dimitrova <
> >> > > e.dimitr...@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > +1
> >> > > > > >
> >> > > > > > On Wed, 26 Aug 2020 at 16:48, Caleb Rackliffe <
> >> > > calebrackli...@gmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > +1
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Wed, Aug 26, 2020, 3:45 PM Patrick McFadin <
> >> pmcfa...@gmail.com>
> >> > > > > > wrote:
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > > This is related to the discussion Jordan and I had about
> the
> >> > > > > > contributor
> >> > > > > > >
> >> > > > > > > > Zoom call. Instead of open mic for any issue, call it
> based
> >> on a
> >> > > > > > > discussion
> >> > > > > > >
> >> > > > > > > > thread or threads for higher bandwidth discussion.
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > > > I would be happy to schedule on for next week to
> >> specifically
> >> > > discuss
> >> > > > > > >
> >> > > > > > > > CEP-7. I can attach the recorded call to the CEP after.
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > > > +1 or -1?
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > > > Patrick
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > > > On Tue, Aug 25, 2020 at 7:03 AM Joshua McKenzie <
> >> > > > > jmcken...@apache.org>
> >> > > > > > >
> >> > > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > > > > >
> >> > > > > > >
> >> > > > > > > > > > Does community plan to open another discussion or CEP
> on
> >> > > > > > >
> >> > > > > > > > modularization?
> >> > > > > > >
> >> > > > > > > > >
> >> > > > > > >
> >> > > > > > > > > We probably should have a discussion on the ML or
> monthly
> >> > > contrib
> >> > > > > > call
> >> > > > > > >
> >> > > > > > > > > about it first to see how aligned the interested
> >> contributors
> >> > > are.
> >> > > > > > > Could
> >> > > > > > >
> >> > > > > > > > do
> >> > > > > > >
> >> > > > > > > > > that through CEP as well but CEP's (at least thus far
> >> sans k8s
> >> > > > > > > operator)
> >> > > > > > >
> >> > > > > > > > > tend to start with a strong, deeply thought out point of
> >> view
> >> > > being
> >> > > > > > >
> >> > > > > > > > > expressed.
> >> > > > > > >
> >> > 

Re: [VOTE] Accept the Harry donation

[Oleksandr Petrov]

+1

On Thu, Sep 17, 2020 at 6:28 PM Blake Eggleston
 wrote:

> +1
>
> > On Sep 16, 2020, at 2:45 AM, Mick Semb Wever  wrote:
> >
> > This vote is about officially accepting the Harry donation from Alex
> Petrov
> > and Benedict Elliott Smith, that was worked on in CASSANDRA-15348.
> >
> > The Incubator IP Clearance has been filled out at
> > http://incubator.apache.org/ip-clearance/apache-cassandra-harry.html
> >
> > This vote is a required part of the IP Clearance process. It follows the
> > same voting rules as releases, i.e. from the PMC a minimum of three +1s
> and
> > no -1s.
> >
> > Please cast your votes:
> >   [ ] +1 Accept the contribution into Cassandra
> >   [ ] -1 Do not
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
alex p

Re: [DISCUSS] CEP-8 Drivers Donation

[Oleksandr Petrov]

I've tried to find this information in the document above, but couldn't
find an explicit mention of it, possibly because it's implied. If we take
java-driver as an example, is this CEP proposing to contribute all branches
(3.x and 4.x) or only the latest one?

Thank you,
-- Alex

On Wed, Jul 29, 2020 at 12:16 PM Alexandre Dutra <
alexandre.du...@datastax.com> wrote:

> Hi Dinesh,
>
> While we think that maintaining the drivers together as a long-term goal
> would be beneficial as it would allow the drivers to evolve together and
> adopt similar design decisions, we also understand that donating 7 drivers
> at once may put the whole operation at risk, as some folks here already
> pointed out.
>
> The current CEP text actually advocates for a phased approach (see
> "Approach" section):
>
> [...] the donation process will be iterative, and will start with only the
> > Java driver in a first phase; then, in a second phase, it will be
> extended
> > to the remaining drivers.
> > Once the Java driver is transferred, and before the others are
> > transferred, we will revise the methodology described in this CEP, and if
> > necessary, revise its parameters and adjust them accordingly. A second
> CEP
> > may be required if the changes to the methodology are found to be
> > substantial.
> >
>
> What do you think of this idea, does it sound reasonable ?
>
> Thanks,
>
> Alex
>
> On Mon, Jul 27, 2020 at 11:53 PM Dinesh Joshi  wrote:
>
> > Alexandre,
> >
> > I gave the document a quick read and TL;DR seems to be that we'd like to:
> >
> > 1. Bring in all drivers
> > 2. They're part of the C* project
> >
> > If so, I would prefer if we add the Java driver first to the project.
> > Based on that experience we can move forward with other drivers.
> However, I
> > am open to the idea of adding all the drivers to the project.
> >
> > Please correct me if I misunderstood something.
> >
> > Thanks,
> >
> > Dinesh
> >
> > > On Jul 26, 2020, at 8:41 PM, Nate McCall  wrote:
> > >
> > > This is fantastic, Alexandre, thanks for putting this together!
> > >
> > > I put an initial set of comments/concerns on there and will loop back
> > later
> > > this week after other folks have  a go.
> > >
> > > Cheers,
> > > -Nate
> > >
> > > On Sat, Jul 25, 2020 at 7:35 AM Alexandre Dutra <
> > > alexandre.du...@datastax.com> wrote:
> > >
> > >> Hi,
> > >>
> > >> As per the CEP process guidelines, I'm starting a formal DISCUSS
> > >> thread to resume the conversation
> > >> started here[1].
> > >>
> > >> On behalf of the developers who maintain the DataStax drivers, I
> > >> confirm that we are offering to
> > >> donate to the Apache Software Foundation all seven DataStax-funded
> > >> drivers, and are therefore
> > >> opening this CEP[2] to discussion.
> > >>
> > >> In the CEP document linked above, we tried our best to address all the
> > >> issues that we anticipate
> > >> going forward with this endeavor, but the task is big: your
> > >> suggestions, remarks and comments are
> > >> most welcome.
> > >>
> > >> As stated previously, we understand that we're all primarily focused
> > >> on the C* 4.0 release
> > >> currently; therefore we don't mind deferring any concrete progress on
> > >> this until after C* 4.0 is
> > >> out.
> > >>
> > >> However, as you can judge from the CEP draft document, the donation is
> > >> likely to have many
> > >> implications on sensitive areas, e.g. on project governance; which is
> > >> why we think best to open the
> > >> discussion as early as possible, so that we all have enough time to
> > >> determine what we think best
> > >> suits the project going forward.
> > >>
> > >> Kind regards,
> > >>
> > >> Alex Dutra
> > >>
> > >> [1]
> > >>
> >
> https://lists.apache.org/thread.html/ra7caa1dd42ccaa04bcabfbc33233995c125c655f9a3cdb2c7bd8e9f7%40%3Cdev.cassandra.apache.org%3E
> > >> [2]
> > >>
> >
> https://docs.google.com/document/d/1e0SsZxjeTabzrMv99pCz9zIkkgWjUd4KL5Yp0GFzNnY/edit?usp=sharing
> > >>
> > >> --
> > >> Alexandre Dutra
> > >> alexandre.du...@datastax.com
> > >>
> > >> -
> > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >>
> > >>
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>
> --
> Alexandre Dutra
> e. alexandre.du...@datastax.com
> w. www.datastax.com
>


-- 
alex p

[VOTE] Release dtest-api 0.0.4

[Oleksandr Petrov]

Proposing the test build of in-jvm dtest API 0.0.4 for release.

Repository:
https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.4
Candidate SHA:
https://github.com/apache/cassandra-in-jvm-dtest-api/commit/b481c85f9ce49e17b47765800ca5c2b396b1fa73
tagged
with 0.0.4
Artifact:
https://repository.apache.org/content/repositories/orgapachecassandra-1206/org/apache/cassandra/dtest-api/0.0.4/
Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C

Changes since last release:

  * CASSANDRA-15920: Make SimpleQueryResult a container for client
warnings, and expose those warnings via QueryResult

The vote will be open for 24 hours. Everyone who has tested the build is
invited to vote. Votes by PMC members are considered binding. A vote passes
if there are at least three binding +1s.

-- Alex

Re: [VOTE] Release dtest-api 0.0.3

[Oleksandr Petrov]

With 5 +1 (including my own) and no -1, the vote passes.

On Fri, Jul 3, 2020 at 7:37 PM Jon Haddad  wrote:

> +1
>
> On Fri, Jul 3, 2020 at 7:03 AM Brandon Williams  wrote:
>
> > +1
> >
> > On Fri, Jul 3, 2020, 4:57 AM Oleksandr Petrov <
> oleksandr.pet...@gmail.com>
> > wrote:
> >
> > > Proposing the test build of in-jvm dtest API 0.0.3 for release.
> > >
> > > Repository:
> > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.3
> > > Candidate SHA:
> > >
> > >
> >
> https://github.com/apache/cassandra-in-jvm-dtest-api/commit/3f01a1743a0dcc9d0b054e12bd53f808dd1adc49
> > > tagged with 0.0.3
> > > Artifact:
> > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1205/org/apache/cassandra/dtest-api/0.0.3/
> > > Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C
> > >
> > > Changes since last release:
> > >
> > >   * CASSANDRA-15851: Add instance initializer
> > >
> > > The vote will be open for 24 hours. Everyone who has tested the build
> is
> > > invited to vote. Votes by PMC members are considered binding. A vote
> > passes
> > > if there are at least three binding +1s.
> > >
> > > -- Alex
> > >
> >
>


-- 
alex p

[VOTE] Release dtest-api 0.0.3

[Oleksandr Petrov]

Proposing the test build of in-jvm dtest API 0.0.3 for release.

Repository:
https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.3
Candidate SHA:
https://github.com/apache/cassandra-in-jvm-dtest-api/commit/3f01a1743a0dcc9d0b054e12bd53f808dd1adc49
tagged with 0.0.3
Artifact:
https://repository.apache.org/content/repositories/orgapachecassandra-1205/org/apache/cassandra/dtest-api/0.0.3/
Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C

Changes since last release:

  * CASSANDRA-15851: Add instance initializer

The vote will be open for 24 hours. Everyone who has tested the build is
invited to vote. Votes by PMC members are considered binding. A vote passes
if there are at least three binding +1s.

-- Alex

Re: [DISCUSS] CASSANDRA-13994

[Oleksandr Petrov]

> 1) Would you block the release over this ticket?

I would definitely not block the release on this ticket.

> 2) Would you prioritize this ticket over testing?

Same here, I would prioritise testing.

> 3) Does fixing this ticket make 4.0 a more stable release?

I wanted to give some context: I wrote that in August 2018. While I still
believe it is important to get rid of this code, I'm disinclined to merge
it into 4.0.

Given that the patch is rather big (421 additions and 1,480 deletions) and
touches many important places, including parser, I would be extremely
cautious to merge it that late in release cycle. It would be great to also
hear arguments that would justify the risk.

Thank you for starting this discussion,
-- Alex



On Tue, May 26, 2020 at 5:20 PM Ekaterina Dimitrova <
ekaterina.dimitr...@datastax.com> wrote:

> Dear all,
>
> Following the ticket review sent on 12th May I wanted to bring up
> https://issues.apache.org/jira/browse/CASSANDRA-13994: Remove COMPACT
>
> STORAGE internals before 4.0 release.
>
> It is already under review by Dinesh Joshi and Alex Petrov. Not a
> blocker but already under review.
>
> Below are my responses to the questions brought up.
>
>
> 1) Would you block the release over this
>
> ticket? - probably not
>
> 2) Would you prioritize this ticket over testing? - already
> implemented but if there are some big changes needed after the review,
> I doubt it we will want to prioritize over the testing
>
> 3) Does fixing
> this ticket make 4.0 a more stable release? - I will just cite Alex
> Petrov who reported this Jira and I think the rest of us would agree
> with him here.
>
> "I would say it's quite important to clean up compact storage
> internals in 4.0 before the release. It should have no visible
> side-effects, but it'd be very good to have as it simplifies multiple
> code paths."
>
>
> Ekaterina Dimitrova
> e. ekaterina.dimitr...@datastax.com
> w. www.datastax.com
>


-- 
alex p

Re: List of serious issues fixed in 3.0.x

[Oleksandr Petrov]

Wanted to add some that I remembered:

  * https://issues.apache.org/jira/browse/CASSANDRA-12811 - data
resurrection, but was marked as normal because was discovered with a test.
Should've marked it as critical.
  * https://issues.apache.org/jira/browse/CASSANDRA-12956 - data loss
(commit log isn't replayed on custom 2i exception)
  * https://issues.apache.org/jira/browse/CASSANDRA-12144 -
undeletable/duplicate rows problem; can be considered data resurrection
and/or sstable corruption.



On Thu, May 7, 2020 at 6:55 PM Joshua McKenzie  wrote:

> "ML is plaintext bro" - thanks Mick. ಠ_ಠ
>
> Since we're stuck in the late 90's, here's some links to a gsheet:
>
> Defects by month:
> https://docs.google.com/spreadsheets/d/1Qt8lLIiqVvK7mlSML7zsmXdAc-LsvktFW5RXJDRtN8k/edit#gid=1584867240
> Defects by component:
> https://docs.google.com/spreadsheets/d/1Qt8lLIiqVvK7mlSML7zsmXdAc-LsvktFW5RXJDRtN8k/edit#gid=1946109279
> Defects by type:
> https://docs.google.com/spreadsheets/d/1Qt8lLIiqVvK7mlSML7zsmXdAc-LsvktFW5RXJDRtN8k/edit#gid=385136105
>
> On Thu, May 7, 2020 at 12:31 PM Joshua McKenzie 
> wrote:
>
>> Hearing the images got killed by the web server. Trying from gmail (sorry
>> for spam). Time to see if it's the apache smtp server or the list culling
>> images:
>>
>> ---
>> I did a little analysis on this data (any defect marked with fixversion
>> 4.0 that rose to the level of critical in terms of availability,
>> correctness, or corruption/loss) and charted some things the rest of the
>> project community might find interesting:
>>
>> 1: Critical (availability, correctness, corruption/loss) defects fixed
>> per month since about 6 months before 3.11.0:
>> [image: monthly.png]
>>
>> 2: Components in which critical defects arose (note: bright red bar ==
>> sum of 3 dark red):
>> [image: Total Defects by Component.png]
>>
>> 3: Type of defect found and fixed (bright red: cluster down or permaloss,
>> dark red: temp corrupt/loss, yellow: incorrect response):
>>
>> [image: Total Defects by Type.png]
>>
>> My personal takeaways from this: a ton of great defect fixing work has
>> gone into 4.0. I'd love it if we had both code coverage analysis for
>> testing on the codebase as well as data to surface where hotspots of
>> defects are in the code that might need further testing (caveat: many have
>> voiced their skepticism of the value of this type of data in the past in
>> this project community, so that's probably another conversation to have on
>> another thread)
>>
>> Hope someone else finds the above interesting if not useful.
>>
>> --
>> Joshua McKenzie
>>
>> On Thu, May 7, 2020 at 12:24 PM Joshua McKenzie 
>> wrote:
>>
>>> I did a little analysis on this data (any defect marked with fixversion
>>> 4.0 that rose to the level of critical in terms of availability,
>>> correctness, or corruption/loss) and charted some things the rest of the
>>> project community might find interesting:
>>>
>>> 1: Critical (availability, correctness, corruption/loss) defects fixed
>>> per month since about 6 months before 3.11.0:
>>> [image: monthly.png]
>>>
>>> 2: Components in which critical defects arose (note: bright red bar ==
>>> sum of 3 dark red):
>>> [image: Total Defects by Component.png]
>>>
>>> 3: Type of defect found and fixed (bright red: cluster down or
>>> permaloss, dark red: temp corrupt/loss, yellow: incorrect response):
>>>
>>> [image: Total Defects by Type.png]
>>>
>>> My personal takeaways from this: a ton of great defect fixing work has
>>> gone into 4.0. I'd love it if we had both code coverage analysis for
>>> testing on the codebase as well as data to surface where hotspots of
>>> defects are in the code that might need further testing (caveat: many have
>>> voiced their skepticism of the value of this type of data in the past in
>>> this project community, so that's probably another conversation to have on
>>> another thread)
>>>
>>> Hope someone else finds the above interesting if not useful.
>>>
>>> ~Josh
>>>
>>>
>>> On Wed, May 6, 2020 at 3:38 PM Dinesh Joshi  wrote:
>>>
 Hi Sankalp,

 Thanks for bringing this up. At the very minimum, I hope we have
 regression tests for the specific issues we have fixed.

 I personally think, the project should focus on building a
 comprehensive test suite. However, some of these issues can only be
 detected at scale. We need users to test* C* in their environment for their
 use-cases. Ideally these folks stand up large clusters and tee their
 traffic to the new cluster and report issues.

 If we had an automated test suite that everyone can run at a large
 scale that would be even better.

 Thanks,

 Dinesh


 * test != starting C* in a few nodes and looking at logs.

 > On May 6, 2020, at 10:11 AM, sankalp kohli 
 wrote:
 >
 > Hi,
 >I want to share some of the serious issues that were found and
 fixed in
 > 3.0.x. I

Re: Scope of CASSANDRA-14557 (Default and Minimum RF)

[Oleksandr Petrov]

I think it is reasonable that system keyspaces would get initialized with a
default replication factor, assuming ones that were already
initalized would remain intact (however, this should be the same for
user-created keyspaces).

Assuming it doesn't change the current behaviour, and default and min rf,
when unset, act the same way current version would, the only thing we
should probably add is a line in cassandra.conf that default and min
replication factors will also apply to system keyspaces.

On Wed, May 13, 2020 at 9:01 PM Sumanth Pasupuleti <
sumanth.pasupuleti...@gmail.com> wrote:

> Hi,
>
> Based on Alex's suggestion on the ticket, wanted to reach out to clarify on
> the current scope of  default and minimum replication factors that 14557
> defines, and gather thoughts to farm for dissent.
>
> Both the configurations (default and minimum) apply not just to user
> keyspaces, but also to system keyspaces. For instance, this can be handy in
> deployments that use authenticated C* clusters where operators have to
> "remember" to set system_auth keyspace's RF to a value higher than 1. In
> such cases, setting default_rf = 3 for example (which I suppose is common
> in most deployments) would ensure all the system keyspaces (including
> system_auth) come up with RF=3.
>
> It can be helpful to note that, this patch by default does not cause any
> change to the replication factors, reason being, the default values of
> these configurations are set to [defaultRF=1, minimumRF=0] to not induce
> any changes that folks may not expect, but rather offers knobs to define
> what a sane default RF should be, and have a gate on any new keyspaces
> being created with an RF lower than minimumRF.
>
> Curious to know your thoughts.
>
> Thanks,
> Sumanth
>


-- 
alex p

Re: Calling for release managers (Committers and PMC)

[Oleksandr Petrov]

Not sure if I’m too late to the game but you can count me in, too!

On Tue 12. May 2020 at 18:42, Mick Semb Wever  wrote:

> Thanks everyone!
>
> I've created a confluence page for the onboarding process.
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Managers+Onboarding
>
> There's some prerequisites there for people to read and follow up on
> in their own time.
> Provide feedback and ask questions as needed.
>
>
> On Tue, 12 May 2020 at 14:10, Aleksey Yeshchenko
>  wrote:
> >
> > Please add me as well.
> >
> > > On 12 May 2020, at 09:51, Sam Tunnicliffe  wrote:
> > >
> > > Me too.
> > >
> > >> On 11 May 2020, at 20:23, Jake Luciani  wrote:
> > >>
> > >> Happy to lend a hand
> > >>
> > >> On Mon, May 11, 2020 at 3:12 PM Eric Evans  >
> > >> wrote:
> > >>
> > >>> I can take a turn.
> > >>>
> > >>> On Fri, May 8, 2020 at 11:10 AM Vinay Chella <
> vinaykumar...@gmail.com>
> > >>> wrote:
> > 
> >  I would like to help as well.
> > 
> > 
> >  On Fri, May 8, 2020 at 8:54 AM Chris Lohfink 
> > >>> wrote:
> > 
> > > I'd like to get involved in this as well.
> > >
> > > On Thu, May 7, 2020 at 2:06 PM Jon Meredith  >
> > >>> wrote:
> > >
> > >> Sign me up.
> > >>
> > >> On Thu, May 7, 2020 at 12:36 PM Robert Stupp 
> wrote:
> > >>>
> > >>> I can help
> > >>>
> > >>> --
> > >>> Robert Stupp
> > >>> @snazy
> > >>>
> >  Am 07.05.2020 um 20:29 schrieb Mick Semb Wever  >:
> > 
> >  The Cassandra release process has had some improvements to
> > >>> better in
> >  line with the ASF guidelines: sha256 & sha512 checksums, staged
> >  artefacts in svnpubsub, dep and rpm repositories complete and
> > >>> signed
> >  in staging, and separate scripts and manual steps merged
> > >>> together.
> > 
> >  The updated documentation for cutting, voting, and publishing a
> >  release is found here:
> > 
> > >>
> > >>>
> https://cassandra.apache.org/doc/latest/development/release_process.html
> > 
> >  I am hoping to get as many Committers* and PMC members
> > >>> interested as
> >  possible for cutting a future release.
> > 
> >  Who is interested? How many names can I get :-)
> > 
> >  The more that are interested then the easier it is to take turns
> > >>> and
> >  be flexible depending on our own availability each time. I will
> > >>> help
> >  out everyone on their first run. Indeed most of my motivation in
> >  getting involved with the release process was to make it all as
> > > simple
> >  and as forgettable as possible, so the role of the role manager
> > >>> can
> >  change easily from release to release.
> > 
> >  *When a Committer cuts a release, a PMC member has to perform
> the
> > > very
> >  last post-vote publish step.
> > 
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
> --
alex p

Re: [VOTE] Release dtest-api 0.0.2

[Oleksandr Petrov]

The vote passed with 4 +1 votes (including mine) and 0 -1s.

On Thu, Apr 30, 2020 at 10:51 AM Marcus Eriksson  wrote:

> +1
>
>
> On 29 April 2020 at 17:08:57, David Capwell (dcapw...@apple.com.invalid)
> wrote:
> > +1.
> > Checked integration with Cassandra and upgrade
> >
> > Sent from my iPhone
> >
> > > On Apr 29, 2020, at 4:01 AM, Mick Semb Wever wrote:
> > >
> > > 
> > >>
> > >> Proposing the test build of in-jvm dtest API 0.0.2 for release.
> > >
> > >
> > > +1
> > >
> > > Checked:
> > > - copyright and license
> > > - changelog
> > > - build, tests, rat
> > > - dependencies
> > > - gpg signatures (to fingerprint)
> > > - checksums
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
alex p

[VOTE] Release dtest-api 0.0.2

[Oleksandr Petrov]

Proposing the test build of in-jvm dtest API 0.0.2 for release.

Repository:
https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.2
Candidate SHA:
https://github.com/apache/cassandra-in-jvm-dtest-api/commit/6d7679a349d7500554d6fbdcdd3f82030805b240
tagged
with 0.0.2
Artifact:
https://repository.apache.org/content/repositories/orgapachecassandra-1204/org/apache/cassandra/dtest-api/0.0.2/
Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C

Changes since last release:

  * CASSANDRA-15684: Improve error codes in NodeToolResult to produce
better errors and to allow Any style message checks
  * CASSANDRA-15713: Make shared class filter for InstanceClassLoader
pluggable
  * CASSANDRA-15714: Add support for replacing logback with alternate
logger config (like log4j2)
  * CASSANDRA-15756: In-jvm dtest IInstance and ICoordinator should use
QueryResult as the base API
  * CASSANDRA-15733: Cluster builder should be provided to the factory and
expose state

The vote will be open for 24 hours as agreed in [1]. Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s.

-- Alex

[1] https://www.mail-archive.com/dev@cassandra.apache.org/msg14897.html

Re: Simplify voting rules for in-jvm-dtest-api releases

[Oleksandr Petrov]

Thank you for suggestions, Jeremiah!

I first really liked the idea of jitpack since I thought it clones
repository and builds stuff locally. However, it seems like they build on
their machines in docker container. While this is something that could be
useful in many cases, I think just using snapshots with hash would yield a
similar result.

Another suggestion from Jeremiah was to use submodules, which could be
helpful for IDE. We can explore this in future, I just do not have capacity
at the moment to figure out how to make sure it all builds with ant to make
it work nicely for developers.

@Nate McCall   I've added some information about
deployment to readme file [1], and have also posted how to build a snapshot
[2]. I'll add information about the length of vote with a reference to this
mailing list discussion for history.

-- Alex


[1]
https://github.com/apache/cassandra-in-jvm-dtest-api/blob/master/README.md
[2]
https://github.com/apache/cassandra-in-jvm-dtest-api/commit/29d055b7cffc66a852505660930c980c185138a1


On Fri, Apr 17, 2020 at 1:34 AM J. D. Jordan 
wrote:

> I was taking with Alex on slack earlier today brainstorming ideas and two
> that might work are using a git submodule to reference the code by git
> hash, so no release needed, or using jitpack.io to be able to pull the
> jar down by git hash without doing a release.
>
> Does anyone find either of those options more appealing than 1/2/3?
>
> -Jeremiah
>
> > On Apr 16, 2020, at 6:14 PM, David Capwell  wrote:
> >
> > Not a fan of 2 or 3.  For #2 there is also talk about getting rid of the
> > jars in /lib so that would complicate things.
> >
> > I think frequent releases with snapshots per commit is good.  Agree with
> > Nate we should document this so we have something we can always point to.
> >
> >> On Thu, Apr 16, 2020 at 2:54 PM Nate McCall  wrote:
> >>
> >> (1) sounds reasonable to me. I'd like us to document the vote cycle and
> >> release train specifics on cassandra.a.o somewhere (developer and
> releases
> >> pages maybe?). Nothing exhaustive, just 'we do X with Y'.
> >>
> >>
> >> On Thu, Apr 16, 2020 at 11:03 PM Oleksandr Petrov <
> >> oleksandr.pet...@gmail.com> wrote:
> >>
> >>> I've posted the question on the legal-discussion mailing list, and got
> >> some
> >>> helpful responses.
> >>>
> >>> We can't work around the vote, best we can do is make it shorter (3 +1
> >>> votes / 24 hours). We have several options now:
> >>>
> >>> 1. Release SNAPSHOT builds prefixed with in-jvm dtest commit SHAs and
> cut
> >>> release every week or so (release-train if you wish)
> >>> 2. Avoid using Apache repository for releases altogether, and just push
> >>> jars to Cassandra repository
> >>> 3. Make this code "unofficial" (publish and manage outside Apache)
> >>>
> >>> I'm not a big fan of (2), since we already tried that with Python and
> >> Java
> >>> drivers, also I'm not sure about binaries in git. As regards (3), I'm
> not
> >>> sure if this makes it harder for the folks who rely on Apache legal
> >>> framework for contributions.
> >>>
> >>> Unless there are strong opinions against (1), which seems to be a
> >>> reasonable middle ground, we can do it. Please let me know if you also
> >> have
> >>> other ideas.
> >>>
> >>> Thank you,
> >>> -- Alex
> >>>
> >>> On Wed, Apr 15, 2020 at 10:33 PM Jeremiah D Jordan <
> >> jerem...@datastax.com>
> >>> wrote:
> >>>
> >>>> I think as long as we don’t publish the artifacts to maven central or
> >>> some
> >>>> other location that is for “anyone” we do not need a formal release.
> >> Even
> >>>> then since the artifact is only meant for use by people developing C*
> >>> that
> >>>> might be fine.
> >>>>
> >>>> If artifacts are only for use by individuals actively participating in
> >>> the
> >>>> development process, then no formal release is needed.  See the
> >>> definition
> >>>> of “release” and “publication” found here:
> >>>>
> >>>> http://www.apache.org/legal/release-policy.html#release-definition
> >>>>> DEFINITION OF "RELEASE" <
> >>>> http://www.apache.org/legal/release-policy.html#release-definition>
> >>>>> Generic

Re: Simplify voting rules for in-jvm-dtest-api releases

[Oleksandr Petrov]

I've posted the question on the legal-discussion mailing list, and got some
helpful responses.

We can't work around the vote, best we can do is make it shorter (3 +1
votes / 24 hours). We have several options now:

1. Release SNAPSHOT builds prefixed with in-jvm dtest commit SHAs and cut
release every week or so (release-train if you wish)
2. Avoid using Apache repository for releases altogether, and just push
jars to Cassandra repository
3. Make this code "unofficial" (publish and manage outside Apache)

I'm not a big fan of (2), since we already tried that with Python and Java
drivers, also I'm not sure about binaries in git. As regards (3), I'm not
sure if this makes it harder for the folks who rely on Apache legal
framework for contributions.

Unless there are strong opinions against (1), which seems to be a
reasonable middle ground, we can do it. Please let me know if you also have
other ideas.

Thank you,
-- Alex

On Wed, Apr 15, 2020 at 10:33 PM Jeremiah D Jordan 
wrote:

> I think as long as we don’t publish the artifacts to maven central or some
> other location that is for “anyone” we do not need a formal release. Even
> then since the artifact is only meant for use by people developing C* that
> might be fine.
>
> If artifacts are only for use by individuals actively participating in the
> development process, then no formal release is needed.  See the definition
> of “release” and “publication” found here:
>
> http://www.apache.org/legal/release-policy.html#release-definition
> > DEFINITION OF "RELEASE" <
> http://www.apache.org/legal/release-policy.html#release-definition>
> > Generically, a release is anything that is published beyond the group
> that owns it. For an Apache project, that means any publication outside the
> development community, defined as individuals actively participating in
> development or following the dev list.
> >
> > More narrowly, an official Apache release is one which has been endorsed
> as an "act of the Foundation" by a PMC.
> >
> >
>
> > PUBLICATION <http://www.apache.org/legal/release-policy.html#publication
> >
> > Projects SHALL publish official releases and SHALL NOT publish
> unreleased materials outside the development community.
> >
> > During the process of developing software and preparing a release,
> various packages are made available to the development community for
> testing purposes. Projects MUST direct outsiders towards official releases
> rather than raw source repositories, nightly builds, snapshots, release
> candidates, or any other similar packages. The only people who are supposed
> to know about such developer resources are individuals actively
> participating in development or following the dev list and thus aware of
> the conditions placed on unreleased materials.
> >
>
>
> -Jeremiah
>
> > On Apr 15, 2020, at 3:05 PM, Nate McCall  wrote:
> >
> > Open an issue with the LEGAL jira project and ask there.
> >
> > I'm like 62% sure they will say nope. The vote process and the time for
> > such is to allow for PMC to review the release to give the ASF a
> reasonable
> > degree of assurance for indemnification. However, we might have a fair
> > degree of leeway so long as we do 'vote', it's test scope (as Mick
> pointed
> > out) and the process for such is published somewhere?
> >
> > Cheers,
> > -Nate
> >
> > On Thu, Apr 16, 2020 at 7:20 AM Oleksandr Petrov <
> oleksandr.pet...@gmail.com>
> > wrote:
> >
> >> The most important thing for the purposes of what we’re trying to
> achieve
> >> is to have a unique non overridable version. In principle, a unique tag
> >> with release timestamp should be enough, as long as we can uniquely
> >> reference it.
> >>
> >> However, even then, I’d say release frequency (establishing “base”) for
> >> releases should be at least slightly relaxed compared to Cassandra
> itself.
> >>
> >> I will investigate whether it is possible to avoid voting for test only
> >> dependencies, since I’d much rather have it under Apache umbrella, as I
> was
> >> previously skeptical of a dependency that I believed shouldn’t have been
> >> locked under contributor’s GitHub.
> >>
> >> If test only no-vote option isn’t possible due to legal reasons, we can
> >> proceed with snapshot+timestamp and release-rebase with a 24h simplified
> >> vote.
> >>
> >> Thanks,
> >> —Alex
> >>
> >> On Wed 15. Apr 2020 at 19:24, Mick Semb Wever  wrote:
> >>
> >>>> Apache release r

Re: Simplify voting rules for in-jvm-dtest-api releases

[Oleksandr Petrov]

The most important thing for the purposes of what we’re trying to achieve
is to have a unique non overridable version. In principle, a unique tag
with release timestamp should be enough, as long as we can uniquely
reference it.

However, even then, I’d say release frequency (establishing “base”) for
releases should be at least slightly relaxed compared to Cassandra itself.

I will investigate whether it is possible to avoid voting for test only
dependencies, since I’d much rather have it under Apache umbrella, as I was
previously skeptical of a dependency that I believed shouldn’t have been
locked under contributor’s GitHub.

If test only no-vote option isn’t possible due to legal reasons, we can
proceed with snapshot+timestamp and release-rebase with a 24h simplified
vote.

Thanks,
—Alex

On Wed 15. Apr 2020 at 19:24, Mick Semb Wever  wrote:

> > Apache release rules were made for first-class projects. I would like to
> > propose simplifying voting rules for in-jvm-dtest-api project [1].
>
>
> I am not sure the PMC can simply vote away the ASF release rules here.
> But it should be possible to implement the proposal by stepping away
> from what the ASF considers a release and work with "nightlies" or
> snapshots. The purpose of an "ASF release" has little value to
> in-jvm-dtest-api, IIUC.
>
> For example you can't put artifacts into a public maven repository
> without a formal release vote. AFAIK the vote process is there for the
> sake of the legal protections the ASF extends to all its projects,
> over any notion of technical quality of the release cut.
>
> And we are not supposed to be including binaries in the source code
> artifacts, at least not for anything that runtime code depends on.
>
> Solutions to this could be…
>  - allowing snapshot versions of test scope dependencies*, downloaded
> from the ASF's snapshot repository⁽¹⁾
>  - making an exception for a binary if only used in test scope (i
> believe this is ok),
>  - move in-jvm-dtest-api out of ASF (just have it as a github project,
> and publish as needed to a maven repo)
>
> You could also keep using `mvn release:prepare` to cut the versions,
> but just not deploy them to ASF's distribution channels.
>
> This whole area of ASF procedures is quite extensive, so i'd
> definitely appreciate being correctly contradicted :-)
>
> My vote would be to take the approach of using the snapshot
> repository. Semantic versioning has limited value here, and you would
> be able to have a jenkins build push the latest in-jvm-dtest-api
> artifact into the snapshot repository using a uniqueVersion so that it
> can be referenced safely in the build.xml (avoiding having to check
> the jar file into the cassandra/lib/ folder).
>
> regards,
> Mick
>
> ⁽¹⁾ https://repository.apache.org/content/groups/snapshots/
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
> --
alex p

Re: [VOTE] Release Apache Cassandra 4.0-alpha4

[Oleksandr Petrov]

+1

On Wed, Apr 15, 2020 at 10:21 AM Benjamin Lerer 
wrote:

> +1
>
>
>
> On Tue, Apr 14, 2020 at 5:50 PM Blake Eggleston
>  wrote:
>
> > +1
> >
> > > On Apr 14, 2020, at 5:09 AM, e.dimitr...@gmail.com wrote:
> > >
> > > I also can’t see them. I think it matters to which interface is the
> > link.
> > >
> > > And +1 from me, thanks!
> > >
> > >> On 14 Apr 2020, at 7:53, Erick Ramirez 
> > wrote:
> > >>
> > >> 
> > >>>
> > >>>
> >  All java8 UTs, jvmdtests and dtests pass
> > 
> > >>>
> >
> https://circleci.com/workflow-run/d7b3f62d-c9ad-43d6-9152-2655e27feccb?signup-404=true
> > >>>
> > >>>
> > >>> Is anyone else able to see this^ circleci page?
> > >>> For me, it never loads, and isn't the first time I've been unable to
> > >>> see others' circleci results.
> > >>>
> > >>
> > >> All that shows up for me is:
> > >>
> > >> Workflows  >>  null  >>  null  >> null
> > >> 0 jobs in this workflow
> > >>
> > >>
> > >> and a spinning widget.
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>


-- 
alex p

Simplify voting rules for in-jvm-dtest-api releases

[Oleksandr Petrov]

Hi everyone,

Apache release rules were made for first-class projects. I would like to
propose simplifying voting rules for in-jvm-dtest-api project [1].

A bit of background: in-jvm-dtest-api is a project that is used by all
active Cassandra branches (2.2, 3.0, 3.11, and trunk) to unify interfaces
that allows creating clusters and running tests, much like Python dtests,
just with a potential to run and develop them faster. Previously, anyone
could break API compatibility by committing to only one of the branches and
not updating the other one, which has happened on several occasions and has
went unnoticed, and has added work for people who had to bring changes to
more than one branch. This unified API and tests are particularly useful
for upgrade tests, but are also good for any kind of testing.

Since this project was made to simplify contributions to in-jvm dtests,
it'd be great if making changes to this project would actually be simple.
Before that, in order to make changes in in-jvm-dtest API, we required
only +1 from a contributor and a committer could just commit the change.

I would say that in order to cut a (minor) release of in-jvm-dtest-api you
should:

1. Get a +1 from a contributor who can review and test your change
2. Get a +1 from one of committers who are familiar with in-jvm dtests (we
have enough, I just don't want to volunteer anyone)

This will guard us from unnecessary changes, and add an extra pair of eyes
for things that influence moore than one branch, but leave us flexible
enough to be able to move forward without conducting a vote.

Since in-jvm-dtest-api is only used to test Cassandra, and isn't intended
for production purposes, this is a low-risk change in procedure. Moreover,
even if we package in-jvm-dtest-api with some Cassandra release, there will
be an additional vote on the release, where anyone who has concerns about
in-jvm-dtest-api changes can still voice them.

Please let me know if you'd like more information about in-jvm-dtest API,
or have comments about this change in procedure.

Thank you,
-- Alex
[1] https://github.com/apache/cassandra-in-jvm-dtest-api

Re: server side describe

[Oleksandr Petrov]

> My understanding is that a majority of people ended up in favor of
a DESCRIBE approach. Robert made a patch for that approach (according to
his comment it was discussed with Chris beforehand).

This sounds like just a switch of API (in other words, you use the same
string generation, but return it via different means). From the (little)
time I've spent looking at the patch, I remember that there wasn't much
common code between the two (sorry if I'm remembering it wrong).

If the discussion is about whether or not to include _some_ version of the
patch, I think including it makes sense, especially given the patch was
there for a while and was not committed for non-technical reasons. If we're
trying to decide _which_ patch to commit, I'd personally focus on the
original patch (to foster recognition), get it reviewed, and pick any
changes that make it better from the follow-up one (to foster
collaboration).

> Where do we draw the line?

I would discuss changes on the case-by-case basis. Some of the things we
have explicitly (as a community) agreed to commit are still in progress,
including several client protocol changes. And, to my understanding, if
those are committed, it'll be what community has agreed upon. All further
changes that weren't discussed yet - we can talk over. If it's something as
trivial as server-side describe that doesn't risk stability but adds a lot
of value, it may make sense to include. But I woudln't attempt to come up
with a general rule for what we may or may not consider for now.


On Mon, Apr 6, 2020 at 3:21 PM Benjamin Lerer 
wrote:

> We have already discussed it for several days and I believe that all the
> points have been brought to the table.
>
> I took the time to read CASSANDRA-14825 this morning to understand the full
> story.
> All the discussion has been about using Virtual Tables versus using
> DESCRIBE.
> My understanding is that a majority of people ended up in favor of a
> DESCRIBE approach (skipping the details on purpose here).
> Robert made a patch for that approach (according to his comment it was
> discussed with Chris beforehand).
>
> The question here is should: we agree to put it in 4.0 even if the version
> is frozen for improvements.
>
> My main reason against it is that if we broke the freeze for every ticket
> we believe might be useful we will end up delaying 4.0 a lot.
> Are there not other tickets that are more valuable than CASSANDRA-14825,
> that are not included in 4.0?
> Where do we draw the line?
>
> I believe that if we were able to answer that question it would suddenly
> become much easier to agree on which tickets we put in 4.0.
>
>
>
>
>
>
>
>
> On Sun, Apr 5, 2020 at 1:25 AM Benedict Elliott Smith  >
> wrote:
>
> > There it is.  I knew it would show up eventually.
> >
> > On 04/04/2020, 06:26, "bened...@apache.org" 
> > wrote:
> >
> > > scope creep.
> >
> > I think it is unfair to label this scope creep; it would have to be
> > newly considered for 4.0 for it to fall under that umbrella.
> >
> > I don't personally mind if it lands, but this was discussed at length
> > on multiple occasions over the past year, and only stalled because of a
> > combination of lack of etiquette, and a lack of leadership from e.g. PMC
> in
> > resolving various political questions over the course of events.
> >
> > I also struggle to see how this would invalidate testing in any
> > significant way?  It doesn't modify any existing behaviour.
> >
> > 
> > From: Joshua McKenzie 
> > Sent: 01 April 2020 19:24
> > To: dev@cassandra.apache.org 
> > Subject: Re: server side describe
> >
> > This looks like a feature that'd potentially invalidate some testing
> > that's
> > been done and we've been feature frozen for over a year and a half.
> > Also:
> > scope creep.
> >
> > My PoV is we hold off. If we get into a cadence of more frequent
> > releases
> > we'll have it soon enough.
> >
> > On Wed, Apr 1, 2020 at 3:03 PM  wrote:
> >
> > > Hi,
> > > Normally I ping the person on the ticket or in Slack to ask him/her
> > for
> > > status update and whether I can help. Then probably he/she gives
> me a
> > > direction.
> > > If I can’t find the person anymore, I just use my best judgement
> and
> > > coordinate with people who might know better than me.
> > > For now this strategy worked for me personally.
> > > Hope this helps
> > > Ekaterina
> > >
> > > Sent from my iPhone
> > >
> > > > On 1 Apr 2020, at 14:27, Jon Haddad  wrote:
> > > >
> > > > Hey folks,
> > > >
> > > > I was looking through our open JIRAs and realized we hadn't
> merged
> > in
> > > > server side describe calls yet.  The ticket died off a ways ago,
> > and I
> > > > pinged Chris about it yesterday.  He's got a lot of his plate and
> > won't
> > > be
> > > > able to work on it anytime soon.  I still think we should include
>

Re: [VOTE] Release dtest-api 0.0.1

[Oleksandr Petrov]

Just to make sure to mention it explicitly: I'm holding off the release
itself till https://issues.apache.org/jira/browse/CASSANDRA-15652 is merged.

On Thu, Mar 26, 2020 at 11:43 AM Oleksandr Petrov <
oleksandr.pet...@gmail.com> wrote:

> The vote passed with 4 +1 votes (including mine) and 0 -1s.
>
> Thank you,
> -- Alex
>
> On Thu, Mar 26, 2020 at 10:42 AM Sam Tunnicliffe  wrote:
>
>> +1
>>
>> Sam
>>
>> > On 23 Mar 2020, at 09:36, Oleksandr Petrov 
>> wrote:
>> >
>> > Thanks to everyone who participated in the previous vote, I appreciate
>> your
>> > feedback.
>> >
>> > @Mick, thank you for sending a patch and helping to figure out the
>> release
>> > procedure and specifics.
>> >
>> > Proposing the test build of in-jvm dtest API 0.0.1 for release.
>> >
>> > Repository:
>> >
>> https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.1
>> > Candidate SHA:
>> >
>> https://github.com/apache/cassandra-in-jvm-dtest-api/commit/7a2f029676d3b9294f10263742aa4ba07b9abcfd
>> > /
>> > tagged with 0.0.1
>> > Artifact:
>> >
>> https://repository.apache.org/content/repositories/orgapachecassandra-1201/org/apache/cassandra/dtest-api/0.0.1/
>> > Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C
>> >
>> > Changes since the previous vote (see [3] for details):
>> >
>> >  * code is now in master branch; pointing to the right SHA
>> >  * release version is 0.0.1, not 0.0.3
>> >  * there's only a source jar, no source zip; checksum files are now
>> correct
>> >  * NOTICE file is now added, missing license headers added, RAT checks
>> are
>> > passing
>> >
>> > The vote will be open for 72 hours (longer if needed). Everyone who has
>> > tested the build is invited to vote. Votes by PMC members are considered
>> > binding. A vote passes if there are at least three binding +1s.
>> >
>> > To make sure, my key is still not in Cassandra KEYS file, there's an
>> issue
>> > [2] open to track it.
>> >
>> > -- Alex
>> >
>> > [1] https://issues.apache.org/jira/browse/CASSANDRA-15539
>> > [2] https://issues.apache.org/jira/browse/CASSANDRA-15652
>> > [3] https://www.mail-archive.com/dev@cassandra.apache.org/msg14741.html
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>
>>
>
> --
> alex p
>


-- 
alex p

Re: [VOTE] Release dtest-api 0.0.1

[Oleksandr Petrov]

The vote passed with 4 +1 votes (including mine) and 0 -1s.

Thank you,
-- Alex

On Thu, Mar 26, 2020 at 10:42 AM Sam Tunnicliffe  wrote:

> +1
>
> Sam
>
> > On 23 Mar 2020, at 09:36, Oleksandr Petrov 
> wrote:
> >
> > Thanks to everyone who participated in the previous vote, I appreciate
> your
> > feedback.
> >
> > @Mick, thank you for sending a patch and helping to figure out the
> release
> > procedure and specifics.
> >
> > Proposing the test build of in-jvm dtest API 0.0.1 for release.
> >
> > Repository:
> >
> https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.1
> > Candidate SHA:
> >
> https://github.com/apache/cassandra-in-jvm-dtest-api/commit/7a2f029676d3b9294f10263742aa4ba07b9abcfd
> > /
> > tagged with 0.0.1
> > Artifact:
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1201/org/apache/cassandra/dtest-api/0.0.1/
> > Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C
> >
> > Changes since the previous vote (see [3] for details):
> >
> >  * code is now in master branch; pointing to the right SHA
> >  * release version is 0.0.1, not 0.0.3
> >  * there's only a source jar, no source zip; checksum files are now
> correct
> >  * NOTICE file is now added, missing license headers added, RAT checks
> are
> > passing
> >
> > The vote will be open for 72 hours (longer if needed). Everyone who has
> > tested the build is invited to vote. Votes by PMC members are considered
> > binding. A vote passes if there are at least three binding +1s.
> >
> > To make sure, my key is still not in Cassandra KEYS file, there's an
> issue
> > [2] open to track it.
> >
> > -- Alex
> >
> > [1] https://issues.apache.org/jira/browse/CASSANDRA-15539
> > [2] https://issues.apache.org/jira/browse/CASSANDRA-15652
> > [3] https://www.mail-archive.com/dev@cassandra.apache.org/msg14741.html
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
alex p

Re: CircleCI config change?

[Oleksandr Petrov]

If you just want to disable them for your own branch/branches - I don't see
any problem with that, I'm just not sure if it's a good default.

In a similar spirit, I know that some folks are running the script to set
up HIGHRES environment for circle ci every time they create a branch even
though it's not a default.

On Wed, Mar 25, 2020 at 2:26 PM  wrote:

> Hi Aleks,
> Thanks for those fixes and pointing out this issue.
> I didn’t ask for tests not to be used and disabled. I asked whether we can
> make them not to run on every single push to circleci as they are not
> always needed. Intermediate pushes just to save pieces of work do not
> require immediate CI run. That was my point.
> Ekaterina
>
> Sent from my iPhone
>
> > On 25 Mar 2020, at 9:07, Oleksandr Petrov 
> wrote:
> >
> > I recently had to fix at least two problems that could've been
> prevented by
> > running tests and checks that are about to be turned off by default ([1]
> > and [2]).
> >
> > I'd like to point out that this has happened while checks were
> > theoretically enabled, and both problems could've been prevented. This
> > doesn't seem to be a merge problem, or something that showed up only
> after
> > the merge.
> >
> > I might be misunderstanding motivation for this, but my impression was
> that
> > we, as a community, are striving to be able to have working version on
> > every commit merged to master, and possibly even block merging in case
> > tests don't pass. It'd be great to hear more about why this could be
> > helpful.
> >
> > [1] Ninja fix: fix eclipse warnings that were broken during
> CASSANDRA-15528
> > <
> https://github.com/apache/cassandra/commit/a01d05d9a73211fb91c068e133d78ef8ccf34b4e
> >
> > [2] Ninja fix: Fix unit tests that were broken during CASSANDRA-15303.
> > <
> https://github.com/apache/cassandra/commit/b29af2925cddacb4ab8b429b31917748781fbe5d
> >
> >
> >> On Tue, Mar 24, 2020 at 9:01 PM Joshua McKenzie 
> >> wrote:
> >>
> >> Am I understanding correctly - this isn't disabling tests, just changing
> >> when they're triggered (i.e. automatic to manual)?
> >>
> >> So, for instance, smaller interim commits don't trigger a CI run and
> thus
> >> costs?
> >>
> >> On Tue, Mar 24, 2020 at 3:34 PM David Capwell 
> wrote:
> >>
> >>>>
> >>>> I want to change it so it could be a manual choice whether to do it or
> >>> not.
> >>>
> >>>
> >>> Could you explain the motivations for disabling the tests by default?
> My
> >>> personal stance is all tests should run (we disable a lot, at least
> >> HIGHER
> >>> should enable all...), not a fan of disabling tests.
> >>>
> >>> On Tue, Mar 24, 2020 at 12:14 PM Ekaterina Dimitrova <
> >>> ekaterina.dimitr...@datastax.com> wrote:
> >>>
> >>>> Hello everyone,
> >>>> Hope this email finds you well!
> >>>>
> >>>> Just a heads up that I plan to open a Jira and change the CircleCI
> >>> config.
> >>>> Currently unit tests and in-jvm tests are triggered automatically on
> >>> every
> >>>> commit.
> >>>> I want to change it so it could be a manual choice whether to do it or
> >>> not.
> >>>>
> >>>> Anyone against that who really needs the current setup? Is there any
> >>>> background information I miss?
> >>>>
> >>>> Ekaterina Dimitrova | Software Engineer
> >>>> ekaterina.dimitr...@datastax.com | datastax.com
> >>>> <
> >>>>
> >>>
> >>
> http://datastax.com/?utm_campaign=FY20Q2_CONSTELLATION&utm_+medium=email&utm_source=signature
> >>>>>
> >>>>
> >>>
> >>
> >
> >
> > --
> > alex p
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
alex p

Re: CircleCI config change?

[Oleksandr Petrov]

I recently had to fix at least two problems that could've been prevented by
running tests and checks that are about to be turned off by default ([1]
and [2]).

I'd like to point out that this has happened while checks were
theoretically enabled, and both problems could've been prevented. This
doesn't seem to be a merge problem, or something that showed up only after
the merge.

I might be misunderstanding motivation for this, but my impression was that
we, as a community, are striving to be able to have working version on
every commit merged to master, and possibly even block merging in case
tests don't pass. It'd be great to hear more about why this could be
helpful.

[1] Ninja fix: fix eclipse warnings that were broken during CASSANDRA-15528

[2] Ninja fix: Fix unit tests that were broken during CASSANDRA-15303.


On Tue, Mar 24, 2020 at 9:01 PM Joshua McKenzie 
wrote:

> Am I understanding correctly - this isn't disabling tests, just changing
> when they're triggered (i.e. automatic to manual)?
>
> So, for instance, smaller interim commits don't trigger a CI run and thus
> costs?
>
> On Tue, Mar 24, 2020 at 3:34 PM David Capwell  wrote:
>
> > >
> > > I want to change it so it could be a manual choice whether to do it or
> > not.
> >
> >
> > Could you explain the motivations for disabling the tests by default?  My
> > personal stance is all tests should run (we disable a lot, at least
> HIGHER
> > should enable all...), not a fan of disabling tests.
> >
> > On Tue, Mar 24, 2020 at 12:14 PM Ekaterina Dimitrova <
> > ekaterina.dimitr...@datastax.com> wrote:
> >
> > > Hello everyone,
> > > Hope this email finds you well!
> > >
> > > Just a heads up that I plan to open a Jira and change the CircleCI
> > config.
> > > Currently unit tests and in-jvm tests are triggered automatically on
> > every
> > > commit.
> > > I want to change it so it could be a manual choice whether to do it or
> > not.
> > >
> > > Anyone against that who really needs the current setup? Is there any
> > > background information I miss?
> > >
> > > Ekaterina Dimitrova | Software Engineer
> > > ekaterina.dimitr...@datastax.com | datastax.com
> > > <
> > >
> >
> http://datastax.com/?utm_campaign=FY20Q2_CONSTELLATION&utm_+medium=email&utm_source=signature
> > > >
> > >
> >
>


-- 
alex p

[VOTE] Release dtest-api 0.0.1

[Oleksandr Petrov]

Thanks to everyone who participated in the previous vote, I appreciate your
feedback.

@Mick, thank you for sending a patch and helping to figure out the release
procedure and specifics.

Proposing the test build of in-jvm dtest API 0.0.1 for release.

Repository:
https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.1
Candidate SHA:
https://github.com/apache/cassandra-in-jvm-dtest-api/commit/7a2f029676d3b9294f10263742aa4ba07b9abcfd
/
tagged with 0.0.1
Artifact:
https://repository.apache.org/content/repositories/orgapachecassandra-1201/org/apache/cassandra/dtest-api/0.0.1/
Key signature: 9E66CEC6106D578D0B1EB9BFF1000962B7F6840C

Changes since the previous vote (see [3] for details):

  * code is now in master branch; pointing to the right SHA
  * release version is 0.0.1, not 0.0.3
  * there's only a source jar, no source zip; checksum files are now correct
  * NOTICE file is now added, missing license headers added, RAT checks are
passing

The vote will be open for 72 hours (longer if needed). Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s.

To make sure, my key is still not in Cassandra KEYS file, there's an issue
[2] open to track it.

-- Alex

[1] https://issues.apache.org/jira/browse/CASSANDRA-15539
[2] https://issues.apache.org/jira/browse/CASSANDRA-15652
[3] https://www.mail-archive.com/dev@cassandra.apache.org/msg14741.html

Re: [VOTE] Release dtest-api 0.0.3

[Oleksandr Petrov]

I've prepared a new release. Not issuing a vote just yet, wanted to check
it first.

> You need to sign with your own key, and the public key needs to be in our
KEYS file.
Ok, so this should be fine then. I need, however, to add my key to the KEYS
file, created jira for this:
https://issues.apache.org/jira/browse/CASSANDRA-15652

> What was the guide you were using?
I've used this release guide: https://infra.apache.org/maven-releases.html

Release RSA:
https://github.com/apache/cassandra-in-jvm-dtest-api/commit/b4e8724615a931d0b38b59beb08c4c227aa8996e
New artifact:
https://repository.apache.org/content/repositories/orgapachecassandra-1199/org/apache/cassandra/dtest-api/0.0.1/

I've left sources artifact, but added md5 and sha checksums.

Thank you for helping out with this,
-- Alex


On Sat, Mar 21, 2020 at 10:45 AM Oleksandr Petrov <
oleksandr.pet...@gmail.com> wrote:

> Thanks for bringing these up!
>
> >  this fixes everything but the signing key used issue
> https://github.com/apache/cassandra-in-jvm-dtest-api/pull/2
> Great, I'll just merge this together with my branch to master to have a
> ref. Thank you for taking time to resolve these.
>
> >  * Source artifacts does not compile. They depend on snapshot
> dependency, see below.
> I did mention this explicitly in my original email. There's no Cassandra
> artifact and we can not release it yet because it depends on this
> repository. To resolve this, I'll remove tests and dependency on
> cassandra jar for now just to publish. These tests won't do us any good
> here until Cassandra artifacts are released anyways.
>
> > * There's no copyright or NOTICE file in source jar artifact.
> I'll double-check, but I do not remember this mentioned in the Apache
> guide I've followed, it's worth adding this information there if it's
> missing.
>
> > * What key has been used to sign?
> Can you provide more specific details on that? Apache guide I've followed
> said you have to sign with your own key, which was what I've done. If this
> is not the case, it'd be great to know which key I should use. I did upload
> public key to ubuntu keystore for verification to.
>
> >  * The scm SHA is not mentioned in the vote.
> True; branch is not merged since Cassandra patch that depends on it is not
> finalized. But to comply to Apache processes we can just merge the branch.
>
> >  * There's a ".git" directory in the source jar artifact.
> Interesting. I've used `mvn release:prepare/perform`, and expected it to
> take care of it.
>
> On Fri, Mar 20, 2020 at 7:39 PM Mick Semb Wever  wrote:
>
>> > The vote will be open for 72 hours (longer if needed). Everyone who has
>> > tested the build is invited to vote. Votes by PMC members are considered
>> > binding. A vote passes if there are at least three binding +1s.
>> >
>>
>>
>> -1
>>
>> A few things here don't meet the requirements.
>>
>>  * There's no copyright or NOTICE file in source jar artifact.
>>  * The license is not present in all files (eg AssertUtils.java)
>>  * What key has been used to sign?
>>  * Source artifacts does not compile. They depend on snapshot dependency,
>> see below.
>>  * There's a ".git" directory in the source jar artifact.
>>
>> Additionally,
>>  * `mvn rat:check` does not pass. (relates back to license and .git
>> directory)
>>  * There's unnecessary duplicate source artifacts.
>>  * The source zip file does not have sha256 or sha512 checksums.
>>  * The contents of the source zip artifact do not match what's in scm.
>>  * The scm SHA is not mentioned in the vote.
>>  * Where's the scm tag for this scm SHA?
>>  * Erroneous `.asc.asc` files.
>>
>>
>> The build failure I get is:
>> ```
>> [ERROR] Failed to execute goal on project dtest-api: Could not resolve
>> dependencies for project
>> org.apache.cassandra:dtest-api:jar:0.0.4-SNAPSHOT:
>> Could not find artifact
>> org.apache.cassandra:in-jvm-dtest-cassandra-tryout:jar:0.0.1-2.2-1 in
>> central (https://repo.maven.apache.org/maven2) -> [Help 1]
>> ```
>>
>> The source zip artifact can just be removed (not generated) as there's no
>> need (afaik) for any artifacts outside of the maven repository. But add
>> those manually added files into git. This will solve the sha256 and sha512
>> problem, and that the build that doesn't match scm contents.
>>
>>
>> I've got a few hours in front of me and will try to send some PRs to fix
>> what I can here.
>>
>> regards,
>> Mick
>>
>
>
> --
> alex p
>


-- 
alex p

Re: [VOTE] Release dtest-api 0.0.3

[Oleksandr Petrov]

Thanks for bringing these up!

>  this fixes everything but the signing key used issue
https://github.com/apache/cassandra-in-jvm-dtest-api/pull/2
Great, I'll just merge this together with my branch to master to have a
ref. Thank you for taking time to resolve these.

>  * Source artifacts does not compile. They depend on snapshot dependency,
see below.
I did mention this explicitly in my original email. There's no Cassandra
artifact and we can not release it yet because it depends on this
repository. To resolve this, I'll remove tests and dependency on
cassandra jar for now just to publish. These tests won't do us any good
here until Cassandra artifacts are released anyways.

> * There's no copyright or NOTICE file in source jar artifact.
I'll double-check, but I do not remember this mentioned in the Apache guide
I've followed, it's worth adding this information there if it's missing.

> * What key has been used to sign?
Can you provide more specific details on that? Apache guide I've followed
said you have to sign with your own key, which was what I've done. If this
is not the case, it'd be great to know which key I should use. I did upload
public key to ubuntu keystore for verification to.

>  * The scm SHA is not mentioned in the vote.
True; branch is not merged since Cassandra patch that depends on it is not
finalized. But to comply to Apache processes we can just merge the branch.

>  * There's a ".git" directory in the source jar artifact.
Interesting. I've used `mvn release:prepare/perform`, and expected it to
take care of it.

On Fri, Mar 20, 2020 at 7:39 PM Mick Semb Wever  wrote:

> > The vote will be open for 72 hours (longer if needed). Everyone who has
> > tested the build is invited to vote. Votes by PMC members are considered
> > binding. A vote passes if there are at least three binding +1s.
> >
>
>
> -1
>
> A few things here don't meet the requirements.
>
>  * There's no copyright or NOTICE file in source jar artifact.
>  * The license is not present in all files (eg AssertUtils.java)
>  * What key has been used to sign?
>  * Source artifacts does not compile. They depend on snapshot dependency,
> see below.
>  * There's a ".git" directory in the source jar artifact.
>
> Additionally,
>  * `mvn rat:check` does not pass. (relates back to license and .git
> directory)
>  * There's unnecessary duplicate source artifacts.
>  * The source zip file does not have sha256 or sha512 checksums.
>  * The contents of the source zip artifact do not match what's in scm.
>  * The scm SHA is not mentioned in the vote.
>  * Where's the scm tag for this scm SHA?
>  * Erroneous `.asc.asc` files.
>
>
> The build failure I get is:
> ```
> [ERROR] Failed to execute goal on project dtest-api: Could not resolve
> dependencies for project org.apache.cassandra:dtest-api:jar:0.0.4-SNAPSHOT:
> Could not find artifact
> org.apache.cassandra:in-jvm-dtest-cassandra-tryout:jar:0.0.1-2.2-1 in
> central (https://repo.maven.apache.org/maven2) -> [Help 1]
> ```
>
> The source zip artifact can just be removed (not generated) as there's no
> need (afaik) for any artifacts outside of the maven repository. But add
> those manually added files into git. This will solve the sha256 and sha512
> problem, and that the build that doesn't match scm contents.
>
>
> I've got a few hours in front of me and will try to send some PRs to fix
> what I can here.
>
> regards,
> Mick
>


-- 
alex p

[VOTE] Release dtest-api 0.0.3

[Oleksandr Petrov]

This is a somewhat special vote that requires at least minimal
understanding of the patch it is related to [1], since it's also the first
release.

This version has to be released _before_ Cassandra version that its future
versions are released, since we can't rely on unreleased artifact in
Cassandra, since any change in dtest api would break Cassandra builds,
which we'd like to avoid. There's a detailed description of the release
plan in 15539.

If you have any questions and/or concerns, now is the second-best time to
raise them.

Code: https://github.com/apache/cassandra-in-jvm-dtest-api
Artifact:
https://repository.apache.org/content/repositories/orgapachecassandra-1195/org/apache/cassandra/dtest-api/0.0.3/

The vote will be open for 72 hours (longer if needed). Everyone who has
tested the build is invited to vote. Votes by PMC members are considered
binding. A vote passes if there are at least three binding +1s.

-- Alex

[1] https://issues.apache.org/jira/browse/CASSANDRA-15539

Protocol-impacting (internode and client) changes for 4.0

[Oleksandr Petrov]

Hi,

During NGCC/ACNA19 we've had quite a few conversations around the 4.0
release. Many (minor) features and changes suggested during that time are
possible to implement in 4.next without any problem. However, some changes
that seem to be very important for the community, which got mentioned in
several conversations, are not possible to implement without protocol
changes. By *protocol* changes here I mean both native and client protocol.

Here's a shortlist of the issues in question:
https://issues.apache.org/jira/browse/CASSANDRA-15349 Add “Going away”
message to the client protocol
https://issues.apache.org/jira/browse/CASSANDRA-15350 Add CAS “uncertainty”
and “contention" messages that are currently propagated as a
WriteTimeoutException.
https://issues.apache.org/jira/browse/CASSANDRA-15351 Allow configuring
timeouts on the per-request basis
https://issues.apache.org/jira/browse/CASSANDRA-15352 Replica failure
propagation to coordinator and client
https://issues.apache.org/jira/browse/CASSANDRA-15299 Improve checksumming
and compression in protocol v5-beta

And, less importantly - CASSANDRA-14683 (paging state issue).

My suggestion would be to lift a freeze for all (or at least some) of these
issues, since they seem to be quite important for operators and each one of
them is extremely low risk, which means that any validation effort that has
already happened won't have to be re-done. All of the issues are fairly
easy to implement, which means they won't delay the release.

To my best knowledge, there's no client that fully supports 4.0, I think
doing this now actually makes sense, meaning that driver implementers won't
really have to redo anything.

Your thoughts on this are welcome,
-- Alex

Re: "4.0: TBD" -> "4.0: Est. Q4 2019"?

[Oleksandr Petrov]

Maybe a bit off-topic:

Before we cut a release, we should make sure we take care of beta protocol
[1], include released driver versions [2] and remove compact storage
remainders [3]. Third one is optional, but I'd argue we should do it sooner
rather than later.

[1] https://issues.apache.org/jira/browse/CASSANDRA-14973
[2] https://issues.apache.org/jira/browse/CASSANDRA-13951
[3] https://issues.apache.org/jira/browse/CASSANDRA-13994



On Sat, Jun 22, 2019 at 1:25 AM Sumanth Pasupuleti <
sumanth.pasupuleti...@gmail.com> wrote:

> Thanks for the feedback Scott. I have incorporated all the incremental
> feedback I have thus far.
>
> Looking for any additional feedback folks may have.
>
> https://docs.google.com/document/d/1bS6sr-HSrHFjZb0welife6Qx7u3ZDgRiAoENMLYlfz8/edit#
>
> On Tue, Jun 11, 2019 at 11:54 AM Scott Andreas 
> wrote:
>
> > Thanks for starting this discussion, Sumanth! Added a round of comments
> as
> > well.
> >
> > Summarizing my non-binding feedback: I feel that many of the items under
> > "Alpha" and "Beta" should be achieved prior to the release of an alpha,
> > especially those related to correctness/safety, scope lock, feature
> > completeness, deprecation, and backwards compatibility. Establishing a
> > higher standard for official project releases (even at the alpha and beta
> > stage) will help us really polish the final build together.
> >
> > Ideally, I feel that contributors should have completed extensive
> > testing/validation to ensure that no critical or severe bugs exist prior
> to
> > the release of an alpha (e.g., data loss, consistency violations,
> incorrect
> > responses to queries, etc). Perhaps we can add a line to this effect.
> >
> > Ensuring that we've met that bar prior to alpha will help us focus the
> > final stages of the release on gathering feedback from users + developers
> > to validate tooling and automation; compatibility with less commonly-used
> > client libraries, testing new features, evaluating performance and
> > stability under their workloads, etc.
> >
> > – Scott
> >
> > On 6/11/19, 6:45 AM, "Sumanth Pasupuleti" <
> > sumanth.pasupuleti...@gmail.com> wrote:
> >
> > Thanks for the feedback on the product stages/ release life cycle
> > document.
> > I have incorporated the suggestions and looking for any additional
> > feedback
> > folks may have.
> >
> >
> https://docs.google.com/document/d/1bS6sr-HSrHFjZb0welife6Qx7u3ZDgRiAoENMLYlfz8/edit#
> >
> > Thanks,
> > Sumanth
> >
> > On Tue, May 28, 2019 at 10:43 PM Scott Andreas  >
> > wrote:
> >
> > > Echoing Jon’s point here –
> > >
> > > JH: “My thinking is I'd like to be able to recommend 4.0.0 as a
> > production
> > > ready
> > > database for business critical cases”
> > >
> > > I feel that this is a standard that is both appropriate and
> > achievable,
> > > and one I’m legitimately excited about.
> > >
> > > Re: the current state of the test plan wiki in Confluence, I owe
> > another
> > > pass through. There has been a lot of progress here, but I’ve let
> > perfect
> > > be the enemy of the good in getting updates out. I’ll complete that
> > pass
> > > later this week.
> > >
> > > Cheers,
> > >
> > > — Scott
> > >
> > > > On May 28, 2019, at 10:48 AM, Dinesh Joshi 
> > wrote:
> > > >
> > > > +1. Wiki could be useful to document what the overall plan. Jira
> to
> > > track progress.
> > > >
> > > > Dinesh
> > > >
> > > >>> On May 28, 2019, at 10:20 AM, Joshua McKenzie <
> > jmcken...@apache.org>
> > > wrote:
> > > >>>
> > > >>>
> > > >>> The unofficial rule is to not upgrade to prod till .10 is cut.
> > > >>
> > > >> FWIW, I believe it's historically .6. Which is still not a great
> > look
> > > for
> > > >> the project.
> > > >>
> > > >> There's a ton of work going into testing 4.0 already.
> > > >>
> > > >> While I intuitively and anecdotally (from the people I've
> > backchanneled
> > > >> with) believe this to be true as well, the referenced wiki
> > page[1] and
> > > >> jql[2] doesn't look like it's an up to date reflection of the
> > testing
> > > >> efforts going on. Is there another place this information is
> > stored /
> > > >> queryable we can surface to people to keep us all coordinated?
> > > >>
> > > >> [1]
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans
> > > >> [2]
> > > >>
> > >
> >
> https://issues.apache.org/jira/browse/CASSANDRA-14862?jql=project%20%3D%20CASSANDRA%20AND%20%20labels%20%3D%204.0-QA
> > > >>
> > > >> On Tue, May 28, 2019 at 12:57 PM sankalp kohli <
> > kohlisank...@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> Hi Jon,
> > > >>>  When you say 4.0 release, how do u match it with 3.0
> > minor
> > > >>> releases. The unofficial rule is to not upgra

Re: Stabilising Internode Messaging in 4.0

[Oleksandr Petrov]

Sorry to pick only a few points to address, but I think these are ones
where I can contribute productively to the discussion.

> In principle, I agree with the technical improvements you
mention (backpressure / checksumming / etc). These things should be there.
Are they a hard requirement for 4.0?

One thing that comes to mind is protocol versioning and consistency. If
changes adding checksumming and handshake do not make it to 4.0, we grow
the upgrade matrix and have to put changes to the separate protocol
version. I'm not sure how many other internode protocol changes we have
planned for 4.next, but this is definitely something we should keep in mind.

> 2. We should not be measuring complexity in LoC with the exception that
all 20k lines *do need to be review* (not just the important parts and
because code refactoring tools have bugs too) and more lines take more time.

Everything should definitely be reviewed. But with different rigour: one
thing is to review byte arithmetic and protocol formats and completely
different thing is to verify that Verb moved from one place to the other is
still used. Concentrating on a smaller subset doesn't make a patch smaller,
just helps to guide reviewers and observers, so my primary goal was to help
people, hence my reference to the jira comment I'm working on.


On Wed, Apr 10, 2019 at 6:13 PM Sankalp Kohli 
wrote:

> I think we should wait for testing doc on confluence to come up and
> discuss what all needs to be added there to gain confidence.
>
> If the work is more to split the patch into smaller parts and delays 4.0
> even more, can we use time in adding more test coverage, documentation and
> design docs to this component?  Will that be a middle ground here ?
>
> Examples of large changes not going well is due to lack of testing,
> documentation and design docs not because they were big. Being big adds to
> the complexity and increased the total bug count but small changes can be
> equally worse in terms of impact.
>
>
> > On Apr 10, 2019, at 8:53 AM, Jordan West  wrote:
> >
> > There is a lot of discuss here so I’ll try to keep my opinions brief:
> >
> > 1. The bug fixes are a requirement in order to have a stable 4.0. Whether
> > they come from this patch or the original I have less of an opinion. I do
> > think its important to minimize code changes at this time in the
> > development/freeze cycle — including large refactors which add risk
> despite
> > how they are being discussed here. From that perspective, I would prefer
> to
> > see more targeted fixes but since we don’t have them and we have this
> patch
> > the decision is different.
> >
> > 2. We should not be measuring complexity in LoC with the exception that
> all
> > 20k lines *do need to be review* (not just the important parts and
> because
> > code refactoring tools have bugs too) and more lines take more time.
> > Otherwise, its a poor metric for how long this will take to review.
> > Further, it seems odd that the authors are projecting how long it will
> take
> > to review — this should be the charge of the reviewers and I would like
> to
> > hear from them on this. Its clear this a complex patch — as risky as
> > something like 8099 (and the original rewrite by Jason). We should treat
> it
> > as such and not try to merge it in quickly for the sake of it, repeating
> > past mistakes. The goal of reviewing the messaging service work was to do
> > just that. It would be a disservice to rush in these changes now. If the
> > goal is to get the patch in that should be the priority, not completing
> it
> > “in two weeks”. Its clear several community members have pushed back on
> > that and are not comfortable with the time frame.
> >
> > 3. If we need to add special forks of Netty classes to the code because
> of
> > “how we use Netty” that is a concern to me w.r.t to quality, stability,
> and
> > maintenance. Is there a place that documents/justifies our
> non-traditional
> > usage (I saw some in JavaDocs but found it lacking in *why* but had a lot
> > of how/what was changed). Given folks in the community have decent
> > relationships with the Netty team perhaps we should leverage that as
> well.
> > Have we reached out to them?
> >
> > 4. In principle, I agree with the technical improvements you mention
> > (backpressure / checksumming / etc). These things should be there. Are
> they
> > a hard requirement for 4.0? In my opinion no and Dinesh has done a good
> job
> > of providing some reasons as to why so I won’t go into much detail here.
> In
> > short, a bug and a missing safety mechanism are not the same thing. Its
> > also important to consider how many users a change like that covers and
> for
> > what risk — we found a bug in 13304 late in review, had it slipped
> through
> > it would have subjected users to silent corruption they thought couldn’t
> > occur anymore because we included the feature in a prod Cassandra
> release.
> >
> > 5. The patch could seriously benefit from some com

Re: Stabilising Internode Messaging in 4.0

[Oleksandr Petrov]

To be fair, even though the patch totals to 20K LoC, the core of the patch
is within reasonable bounds (around net.async.*). There are many changes
because of the code that got moved around. Some parts changes look large
because Java is quite a verbose language (e.g., metric tables).
Unfortunately, I do not see a good way to split out the bug fixes specific
to netty refactor from some other changes, since some things were
fundamentally reworked.

I'll leave a more elaborate comment that summarises the way I've been
approaching the patch review that might help to identify the "important"
parts that one should concentrate on while reviewing.

I've been reviewing the patch and can say that it has several advantages,
including the fact that incoming and outgoing paths are now easier to test
(e.g., write unit tests for). This has helped to validate rather complex
scenarios, such as handshake protocol, back pressure, dropping messages,
large messages, expiry, counting and more. Patch also integrates well with
in-jvm tests, which allowed to verify several behaviors which otherwise
would've been somewhat difficult to reproduce.

I do agree that validating this patch is no easy endeavor, but having that
said, at that point, we would have to invest a similar amount of time to
validate 4.0 with and without this patch.

I'm personally leaning towards 4.0, since this helps to keep the community
unified testing the same version all together instead of some concentrating
on making 4.0 work, while some are skipping it and proceeding with 4.next.


On Wed, Apr 10, 2019 at 4:55 PM Benedict Elliott Smith 
wrote:

> Appreciate everyone's input. It sounds like there's broad agreement that
> the fixes need to make it into 4.0, which I'm really pleased to see. The
> question seems to be moving towards scope and timing.
>
> TL;DR: This patch is probably the most efficient route to a stable 4.0.
> The patch is already reviewed, extensively tested, and more thorough
> testing is coming. From our perspective, a timeline for merging this is on
> the order of two weeks.
>
> To best answer the question of ideal scope and timing, it's perhaps
> worthwhile considering what our ideal approach would look like, given
> realistic breathing room, and then how other pressures might prefer to
> modify that. In my opinion, both views endorse proceeding with the patch
> largely as it stands, though I will highlight some of the strongest cases
> for splitting.
>
> To answer the first question: Were I starting with 3.0, and producing a
> major refactor of messaging, would I have produced this patch, or one very
> like it? The answer is yes.
>
> These changes have been designed together, intentionally. By making them
> simultaneously, we not only reduce the review and work burden, we reduce
> the overall risk by ensuring all of the semantics of each component are
> designed in harmony. A trickle of changes requires subtly updating the
> semantics of each component, across multiple constantly shifting interfaces
> between old and new. Each time we touch one of these interfaces, we risk
> failing to properly modify (or more likely hack) one side to understand the
> other, and carrying those misunderstandings and hacks through to later
> patches.
>
> Each time we do this, it is costly in both time and risk. The project
> can't really afford either, even in the wider picture sense. There is
> already too much work to do, and risk to mitigate.
>
> Just as importantly, most of these changes are necessary, and actually fix
> bugs. For example, the “performance improvements” from re-implementing
> Netty classes are actually to avoid bugs in our usage, since they are
> designed for use only on the event loop.
>
> - Using our own allocator avoids cross-thread recycling, which can cause
> unbounded heap growth in Netty today (and is the topic of active discussion
> on the Netty bug tracker [1]). We already have our own allocator that
> works, and arguably it is lowest risk to repurpose it. We also reduce risk
> by using existing ByteBuffer methods, keeping the code more idiomatic.
> - AsyncPromise is used to avoid blocking the event loop. Modifying and
> invoking listeners on a DefaultPromise requires taking a mutex, and we do
> this on threads besides the event loop. This could stall the event loop for
> an entire scheduler quanta (or more, if the owning thread is low priority
> or has been extremely busy recently), which is a bug for a real-time
> networking thread that could be servicing dozens of connections.
> - FutureCombiner in Netty is not thread-safe, and we need it to be.
>
> Some are perhaps not bug fixes, but a fundamental part of the design of
> the patch. You either get them for free, or you write a different patch.
> For example, checksumming, which falls naturally out of framing - which is
> integral to the new message flow. Or dropping messages eagerly when reading
> off the wire, which is simply natural to do in this design. Back pressure
> f

Re: [VOTE] Change Jira Workflow

[Oleksandr Petrov]

+1

On Mon, Dec 17, 2018 at 7:12 PM Nate McCall  wrote:
>
> On Tue, Dec 18, 2018 at 4:19 AM Benedict Elliott Smith
>  wrote:
> >
> > I propose these changes 
> > *
> >  to the Jira Workflow for the project.  The vote will be open for 72 
> > hours**.
> >
>
>
> +1
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>


-- 
alex p

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Reviewer for CASSANDRA-14055

[Oleksandr Petrov]

Hi everyone,

I still couldn't find enough time to review the
https://issues.apache.org/jira/browse/CASSANDRA-14055 for which I'm deeply
sorry.

It's a SASI issue related to index redistribution.
Unfortunately, this week doesn't look very promising in terms of time as
well, so I'm making a call for reviewer if anyone has capacity to take it.

If there are any takers, please feel free to assign yourself as reviewer.

Thank you

Best regards,
Alex Petrov

Re: V5 as a protocol beta version in 3.11

[Oleksandr Petrov]

This is an option, you're right. In this case v5 will have just one
feature, however, and the only feature (Duration type) should work with via
CustomTypes through v4.

Looks like the Jira numbers were off, so let me do it again:

In 3.11 we have:
  * CASSANDRA-12838 - Extend native protocol flags and add supported
versions to the SUPPORTED response
  * CASSANDRA-12142 - Add "beta" version native protocol flag
  * CASSANDRA-12850 - Add the duration type to the protocol V5 <-- (this
one should also work with v4)

In 4.0 we have
  * CASSANDRA-10786 - Include hash of result set metadata in prepared
statement id

And the options:

  * (1) remove v5 from 3.11 by reverting #12838 and #12142
  * (2) support v5 in 3.11 forever and backport #10786
  * (3) bump 4.0 version to v6 and make sure that #10786 is v6

Question with (1) is mostly whether or not we would like to cut another
version release because of (in essence) #12838 only, since #12142 is not
relevant in the context and #12850 will still work.



On Tue, Nov 7, 2017 at 4:19 PM Jonathan Haddad  wrote:

> The other option, to avoid having two different v5 implementations, is to
> bump 4.0’s protocol version to 6.
> On Tue, Nov 7, 2017 at 6:48 AM Jeremiah D Jordan <
> jeremiah.jor...@gmail.com>
> wrote:
>
> > My 2 cents.  When we added V5 to 3.x wasn’t it added as a beta protocol
> > for tick/tock stuff and known that when a new version came out it would
> > most possibly break the older releases V5 beta stuff? Or at the very
> least
> > add new things to V5.  So I see no reason to need to add more new
> features
> > to 3.11 v5.
> >
> > -Jeremiah
> >
> > > On Nov 7, 2017, at 9:41 AM, Oleksandr Petrov <
> oleksandr.pet...@gmail.com>
> > wrote:
> > >
> > > Hi everyone,
> > >
> > > Currently, 3.11 supports V5 as a protocol version. However, all new
> > > features are now going to 4.0, which is going to be a new feature
> > release.
> > >
> > > Right now we have two v5 features:
> > >
> > >   - CASSANDRA-10786 <
> > https://issues.apache.org/jira/browse/CASSANDRA-10786>
> > >   - CASSANDRA-12838 <
> > https://issues.apache.org/jira/browse/CASSANDRA-12838>
> > >
> > >
> > > #12838 is adding duration type, which is a nice addition. #10786 is
> also
> > > useful, but is more of an edge cases for users with huge clusters
> and/or
> > > frequent schema changes.
> > >
> > > If we leave v5 in 3.11, we'll have to always backport all v5 features
> to
> > > 3.11. This is something that hasn't been done in #10786. So the
> question
> > > is: are we ready to commit and support v5 in 3.11 "forever", or should
> we
> > > stop until it went too far and remove v5 from 3.11 since it's still in
> > beta
> > > there.
> > >
> > > Looking forward to hear your opinion,
> > >
> > >
> > > --
> > > Alex Petrov
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>
-- 
Alex Petrov

V5 as a protocol beta version in 3.11

[Oleksandr Petrov]

Hi everyone,

Currently, 3.11 supports V5 as a protocol version. However, all new
features are now going to 4.0, which is going to be a new feature release.

Right now we have two v5 features:

   - CASSANDRA-10786 
   - CASSANDRA-12838 


#12838 is adding duration type, which is a nice addition. #10786 is also
useful, but is more of an edge cases for users with huge clusters and/or
frequent schema changes.

If we leave v5 in 3.11, we'll have to always backport all v5 features to
3.11. This is something that hasn't been done in #10786. So the question
is: are we ready to commit and support v5 in 3.11 "forever", or should we
stop until it went too far and remove v5 from 3.11 since it's still in beta
there.

Looking forward to hear your opinion,


-- 
Alex Petrov

Compact Storage and SuperColumn Tables in 4.0/trunk

[Oleksandr Petrov]

As you may know, SuperColumn Tables did not work in 3.x the way they worked in 
2.x. In order to provide everyone with a reasonable upgrade path, we've been 
working on CASSANDRA-12373[1], that brings in support for SuperColumn tables as 
close to 2.x as possible. The patch is planned to land cassandra-3.0 and 
cassandra-3.11 branches only, since the patch for trunk will require even more 
work and, since thrift is not supported on trunk/4.0, it makes it much less 
appealing or even necessary. The idea behind the support for SuperColumns was 
always only to allow people to smoothly migrate off them in 3.0/3.11 world, not 
to have them as a primary feature.

SuperColumns are not the only type of Compact Table, there are more. After 
CASSANDRA-8099[2], Compact Tables are special-cased and have special schema 
layout with some columns hidden from CQL, that allows them to be used from 
Thrift. But, except for the fact they’re accessible from Thrift, there are no 
advantages to use them with the new storage. In order to allow people to 
“expose” the internal structure of the compact tables to make them fully 
accessible in CQL, CASSANDRA-10857[3] was created.

In the light of the fact that 4.0 will not have reasonable SuperColumn support 
(due to related complexity and amount of special-cases required to support it 
in 4.0) and a possibility drop a Compact Storage flag and expose them as 
“normal" tables, there was an idea of removing the Compact Tables from 4.x 
altogether. 


Leaving Compact Storage in 3.x only will make the table metadata a bit lighter 
and allow us to remove some special cases required for their support. Doing it 
during the major release, provided with a reasonable upgrade path (same 
functionality from both Thrift and CQL for all compact tables, including Super 
Column ones) through 3.x/3.11, sounds like the best option that we have right 
now.

It’d be good if you could voice your support of this idea (or raise possible 
concerns, if there are any).


There will be additional discussion and a proposal on how to allow “online” 
COMPACT STORAGE flag drop in CASSANDRA-10857 later this (or the following week).

Best Regards, 
Alex
 
[1] https://issues.apache.org/jira/browse/CASSANDRA-12373 

[2] https://issues.apache.org/jira/browse/CASSANDRA-8099 

[3] https://issues.apache.org/jira/browse/CASSANDRA-10857 

CASSANDRA-13004 FAQ

[Oleksandr Petrov]

Hi everyone,

There were many people asking similar questions about the CASSANDRA-13004.
It might be that the issue itself and release notes are somewhat hard to
grasp or might sound ambiguous, so here's a bit more elaborate explanation
what 13004 means in terms of the upgrade process, how it manifests and what
exactly to do:
https://gist.github.com/ifesdjeen/9cacb1ccd934374f707125d78f2fbcb6

If you're running 3.0+ Cassandra you might want to read it just in case.

If you have any proposals in terms of how to modify/improve it, please ping
me or comment on the gist, I'll make sure to adjust the text accordingly.

Best regards

-- 
Alex Petrov

Re: 3.10 release status: blocked on dtest

[Oleksandr Petrov]

#13025 was updated yesterday. It just needs some feedback, but we know what
the problem is there.

On Wed, Jan 4, 2017 at 5:32 PM Michael Shuler 
wrote:

> On 12/20/2016 03:48 PM, Michael Shuler wrote:
> > Current release blockers in JIRA on the cassandra-3.11 branch are:
> >
> > https://issues.apache.org/jira/browse/CASSANDRA-12617
> > https://issues.apache.org/jira/browse/CASSANDRA-13058
>
> and https://issues.apache.org/jira/browse/CASSANDRA-13025
>
> CASSANDRA-13058 is unassigned, but was just updated (thanks Stefan!).
> The other tickets are assigned, but have not been updated in a while.
>
> JQL for 3.10:
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20fixVersion%20%3D%203.10%20AND%20resolution%20%3D%20Unresolved
>
> --
> Kind regards,
> Michael
>
-- 
Alex Petrov

Re: CASSANDRA-8596

[Oleksandr Petrov]

I would recommend several things that could help this (and probably many
other patches that accidentally went under radar) to get accepted. It's
hard to keep an enormous amount of issues in memory, so we try to rely on
simple rules that help everyone to see current status and understand what
to do next with the ticket:

   * if you are planning to work on ticket, assign it to yourself.
Sometimes it's also good to comment on it (if there was previous activity),
as someone else might be already working on it but forgot to mention it or
assign it. Doesn't happen often though.
  * If you're working on it, put it into "work in progress" status.
  * If you have finished working on the patch and think it's ready for
review, put it into "ready for review" state.
  * (bonus) ping someone on cassandra-dev IRC channel to get your patch
submitted for CI. Most of time however, this will be done by reviewer
anyways. But having a passing build is always reassuring.

I know that committers (and active contributors) are frequently reviewing
"ready for review" tickets and try to provide feedback. Relying only on
ticket comments is a bit hard, since it's too easy to lose track of them.

Sometimes it's possible that the next person who can review the ticket will
only have time in a couple of weeks. Everyone does their best. But if you
feel that it went forgotten, just leave another comment.

Hope this helps!

On Tue, Nov 29, 2016 at 9:19 AM Vladimir Yudovin 
wrote:

> Hi,
>
>
>
> in the light of broader community involvement  I would like to bring
> attentions to the https://issues.apache.org/jira/browse/CASSANDRA-8596 I
> open with proposed patch.
>
> It's stuck for almost 1.5 years.
>
>
>
> I would like to help with Cassandra development and improvement, and I
> have several other fixes, some has JIRA, some not.
>
>
>
> Best regards, Vladimir Yudovin,
>
> Winguzone - Cloud Cassandra Hosting
>
>
>
>
>
> --
Alex Petrov

Segfaults in Unit Test Results on CI

[Oleksandr Petrov]

Hi everyone,

Wanted to ask everyone to take extra care when you see error messages such
as:

Forked Java VM exited abnormally. Please note the time in the report
does not reflect the time until the VM exit.


in your unit test reports on cassci. When you see such a failure, please
always check the full log to see what happened to the JVM, since in the
full log output there might be more information on why JVM exited, which
may be a serious issue, for example segfault due to native memory access.

Since junit already does report it all, I just wanted to stress that if you
see such things, just take a minute to check out logs in more details. If
you already do it this way - it's great and awesome.

If you would like to see an example of such report, you can refer to this
issue [1]

[1] https://issues.apache.org/jira/browse/CASSANDRA-12957

-- 
Alex Petrov

Re: [VOTE] Release Apache Cassandra 3.10 (Take 3)

[Oleksandr Petrov]

>From just a quick glance, I can say at least some of the tests are either
PA or are getting there:

For example:
http://cassci.datastax.com/job/cassandra-3.X_novnode_dtest/lastCompletedBuild/testReport/paging_test/TestPagingData/test_paging_with_filtering_on_partition_key/
Should be fixed by
https://issues.apache.org/jira/browse/CASSANDRA-12666

http://cassci.datastax.com/job/cassandra-3.X_testall/lastCompletedBuild/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testAllowFilteringOnPartitionKeyWithSecondaryIndex/
Should be fixed by
https://issues.apache.org/jira/browse/CASSANDRA-12651

Although there can be more tests in PA.

On Sun, Nov 20, 2016 at 10:35 PM sankalp kohli 
wrote:

> Hi,
> I see the following test runs are failing. Are they for this release?
>
> http://cassci.datastax.com/job/cassandra-3.X_utest_cdc/
> http://cassci.datastax.com/job/cassandra-3.X_testall/
> http://cassci.datastax.com/job/cassandra-3.X_offheap_dtest/
> http://cassci.datastax.com/job/cassandra-3.X_novnode_dtest/
> http://cassci.datastax.com/job/cassandra-3.X_dtest/40/
>
> Thanks,
> Sankalp
>
>
> On Sun, Nov 20, 2016 at 12:50 PM, Nate McCall  wrote:
>
> > +1
> >
> > On Sat, Nov 19, 2016 at 7:08 AM, Michael Shuler 
> > wrote:
> > > I propose the following artifacts for release as 3.10.
> > >
> > > sha1: 96d67b109a2ef858c2753bbb9853d01460cb8f8e
> > > Git:
> > > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/3.10-tentative
> > > Artifacts:
> > > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1135/org/apache/cassandra/apache-cassandra/3.10/
> > > Staging repository:
> > > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1135/
> > >
> > > The Debian packages are available here: http://people.apache.org/~
> > mshuler
> > >
> > > The vote will be open for 72 hours (longer if needed).
> > >
> > > [1]: (CHANGES.txt) https://goo.gl/vah74X
> > > [2]: (NEWS.txt) https://goo.gl/In3vp7
> > >
> >
>
-- 
Alex Petrov

Re: [VOTE] Release Apache Cassandra 3.10 (Take 2)

[Oleksandr Petrov]

-1

Sorry but I have to -1 that one, with the following explanation

One of the features in 3.10 breaks SASI in quite a significant way. The
issue was introduced in #11990 [1] and described in #12877 [2]. If there
are more than 8 items in partition that have same index value, index file
will get corrupted and returned results will be incorrect.

I think we should rather revert the patch and go ahead with a release
without it and change the patch to work correctly, possibly making it's
scope slightly bigger. I am working on a design document for the change.

[1] https://issues.apache.org/jira/browse/CASSANDRA-11990
[2] https://issues.apache.org/jira/browse/CASSANDRA-12877

On Wed, Nov 9, 2016 at 11:19 AM Gary Dusbabek  wrote:

> +1
>
> On Tue, Nov 8, 2016 at 8:09 PM, Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 3.10.
> >
> > sha1: 072b5271a88328b909b230d0e30df1c7476fdb3f
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/3.10-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1133/org/apache/cassandra/apache-cassandra/3.10/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1133/
> >
> >
> >
> > The Debian packages are available here:
> > http://people.apache.org/~mshuler
> >
> >
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> >
> >
> > [1]: (CHANGES.txt) https://goo.gl/TZa9a7
> >
> > [2]: (NEWS.txt) https://goo.gl/FSI1a4
> >
>
-- 
Alex Petrov

Re: Broader community involvement in 4.0 (WAS Re: Rough roadmap for 4.0)

[Oleksandr Petrov]

Recently there was another discussion on documentation and comments [1]

On one hand, documentation and comments will help newcomers to familiarise
themselves with the codebase. On the other - one may get up to speed by
reading the code and adding some docs. Such things may require less
oversight and can play some role in improving diversity / increasing an
amount of involved people.

Same thing with tests. There are some areas where tests need some
refactoring / improvements, or even just splitting them from one file to
multiple. It's a good way to experience the process and get involved into
discussion.

For that, we could add some issues with subtasks (just a few for starters)
or even just a wiki page with a doc/test wishlist where everyone could add
a couple of points.

Docs and tests could be used in addition to lhf issues, helping people,
having comprehensive and quick process and everything else that was
mentioned in this thread.

Thank you.

[1]
http://mail-archives.apache.org/mod_mbox/cassandra-dev/201605.mbox/%3ccakkz8q088ojbvhycyz2_2eotqk4y-svwiwksinpt6rr9pop...@mail.gmail.com%3E

On Mon, Nov 7, 2016 at 5:38 PM Aleksey Yeschenko  wrote:

> Agreed.
>
> --
> AY
>
> On 7 November 2016 at 16:38:07, Jeff Jirsa (jeff.ji...@crowdstrike.com)
> wrote:
>
> ‘Accepted’ JIRA status seems useful, but would encourage something more
> explicit like ‘Concept Accepted’ or similar to denote that the concept is
> agreed upon, but the actual patch itself may not be accepted yet.
>
> /bikeshed.
>
> On 11/7/16, 2:56 AM, "Ben Slater"  wrote:
>
> >Thanks Dave. The shepherd concept sounds a lot like I had in mind (and a
> >better name).
> >
> >One other thing I noted from the Mesos process - they have an “Accepted”
> >jira status that comes after open and means “at least one Mesos developer
> >thought that the ideas proposed in the issue are worth pursuing further”.
> >Might also be something to consider as part of a process like this?
> >
> >Cheers
> >Ben
> >
> >On Mon, 7 Nov 2016 at 09:37 Dave Lester  wrote:
> >
> >> Hi Ben,
> >>
> >> A few ideas to add to your suggestions [inline]:
> >>
> >> On 2016-11-06 13:51 (-0800), Ben Slater <
> https://urldefense.proofpoint.com/v2/url?u=http-3A__ben.slater-40instaclustr.com&d=DgIFaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=yfYEBHVkX6l0zImlOIBID0gmhluYPD5Jje-3CtaT3ow&m=0Ynrto5MaNdgc2fOUtxv50ouikBU_P7VEv6KNub9Bhk&s=MAZTdq4wfrTiqh7nImEMcFWtTrsixRFOX7Pi0SKqQv0&e=
> >
> >> wrote:
> >> > Hi All,
> >> >
> >> > I thought I would add a couple of observations and suggestions as
> someone
> >> > who has both personally made my first contributions to the project in
> the
> >> > last few months and someone in a leadership role in an organisation
> >> > (Instaclustr) that is feeling it’s way through increasing our
> >> contributions
> >> > as an organisation.
> >> >
> >> > Firstly - an observation on contribution experience and what I think
> is
> >> > likely to make people want to contribute again:
> >> > 1) The worst thing that can happen is for your contribution to be
> >> > completely ignored.
> >> > 2) The second worst thing is for it to be rejected without a good
> >> > explanation (that you can learn from) or with hostility.
> >> > 3) Having it rejected with a good reason is not a bad thing (you
> learn)
> >> > 4) Having it accepted is, of course, the best!
> >> >
> >> > With this as a background I would suggest a couple of thing that help
> >> make
> >> > sure (3) and (4) are always more common that (1) and (2) (good
> outcomes
> >> are
> >> > probably more common than bad at the moment but we’ve experienced all
> >> four
> >> > scenarios in the last few months):
> >> > 1) I think some process of assigning a committer of a “sponsor” of a
> >> change
> >> > (which would probably mean committers volunteering) before it
> commences
> >> > would be useful. You can kind of do this at the moment by creating a
> JIRA
> >> > and asking for comment but I think the process is a bit unclear and a
> bit
> >> > intimidating for people starting off and it would be nice to know who
> was
> >> > your primary reviewer for a piece of work. (Or maybe this process does
> >> > exist and I don’t know about.)
> >>
> >> I've seen this approach before and it that can reduce ambiguity on the
> >> state of contributions; the Apache Mesos project has a shepherding
> system
> >> similar to this. I would shy away from the term "sponsor" since it could
> >> infer a non-voluntary relationship between contributors and volunteer
> >> committers.
> >>
> >> From the Mesos docs: "Find a shepherd to collaborate on your patch. A
> >> shepherd is a Mesos committer that will work with you to give you
> feedback
> >> on your proposed design, and to eventually commit your change into the
> >> Mesos source tree." More info on how they approach this is in both their
> >> newbie guide:
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mesos.apache.org_documentation_newbie-2Dguide_&d=DgIFaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1

Re: Cleanup after yourselves please

[Oleksandr Petrov]

It makes sense to make them work for backwards compatibility. There was no
announcement that if you migrated to 3.x they wouldn't work, so everyone
would most likely expect a clear upgrade path. it's not bringing them back
from the dead, but rather helping people who want to upgrade to have this
option.

Unit tests will be completely rewritten I suspect.
On Tue, 18 Oct 2016 at 18:04, Michael Kjellman 
wrote:

> Gotcha, I didn't know we were actually bringing them back from the dead!
>
> That being said, won't the unit tests need to be re-writtten (or at least
> refactored) after your work? Couldn't we use /* */ comments instead of
> every single line one by one? Given we use source control couldn't we
> remove the dead code and get it from the revision history if we need it in
> the future?
>
> > On Oct 18, 2016, at 8:18 AM, Oleksandr Petrov <
> oleksandr.pet...@gmail.com> wrote:
> >
> > I'm currently working on actually making Super Columns work in CQL
> context.
> > Currently they do not really work[1].
> >
> > It's not a very small piece of work. It was in the pipeline for some
> time,
> > although there most likely were more important things that had to be
> worked
> > on. I understand your disappointment and am sorry you stumbled upon this.
> > But for now you may just disregard the commented tests. My branch is
> going
> > to be ready for review soon.
> >
> > [1] https://issues.apache.org/jira/browse/CASSANDRA-12373
> >
> >
> > On Tue, Oct 18, 2016 at 5:10 PM Michael Kjellman <
> > mkjell...@internalcircle.com> wrote:
> >
> >> There was a bunch of tests hastily and messly commented out line by line
> >> (*whyy?*) ColumnFamilyStoreTest with comments that they are pending
> >> SuperColumns support post 8099.
> >>
> >> Could those responsible please cleanup after themselves? It's been a
> while
> >> since 8099 was committed in the first place and I don't see us adding
> Super
> >> Column support at this point and the unit tests surly will need to be
> >> rewritten anyways.
> >>
> >> As my mother always said, pick your dirty wet towel in the hamper off
> the
> >> floor and put it in the hamper please
> >>
> >> best,
> >> kjellman
> >>
> >> Sent from my iPhone
> >
> > --
> > Alex Petrov
>
> --
Alex Petrov

Re: Cleanup after yourselves please

[Oleksandr Petrov]

I'm currently working on actually making Super Columns work in CQL context.
Currently they do not really work[1].

It's not a very small piece of work. It was in the pipeline for some time,
although there most likely were more important things that had to be worked
on. I understand your disappointment and am sorry you stumbled upon this.
But for now you may just disregard the commented tests. My branch is going
to be ready for review soon.

[1] https://issues.apache.org/jira/browse/CASSANDRA-12373


On Tue, Oct 18, 2016 at 5:10 PM Michael Kjellman <
mkjell...@internalcircle.com> wrote:

> There was a bunch of tests hastily and messly commented out line by line
> (*whyy?*) ColumnFamilyStoreTest with comments that they are pending
> SuperColumns support post 8099.
>
> Could those responsible please cleanup after themselves? It's been a while
> since 8099 was committed in the first place and I don't see us adding Super
> Column support at this point and the unit tests surly will need to be
> rewritten anyways.
>
> As my mother always said, pick your dirty wet towel in the hamper off the
> floor and put it in the hamper please
>
> best,
> kjellman
>
> Sent from my iPhone

-- 
Alex Petrov

Re: Failing tests 2016-09-28

[Oleksandr Petrov]

No, not really. It's just that dtests currently expect same behaviour for
3.x and up.


On Wed, Sep 28, 2016 at 8:41 PM Aleksey Yeschenko 
wrote:

> We had one accidental merge from 3.0 into 3.9 (looking at you, you know
> who you are), so could be.
>
> --
> AY
>
> On 28 September 2016 at 17:48:27, Philip Thompson (
> philip.thomp...@datastax.com) wrote:
>
> That ticket was only supposed to be committed to 3.10 and 3.0.x. Was it
> accidentally also merged into 3.9?
>
> On Wed, Sep 28, 2016 at 12:40 PM, Oleksandr Petrov <
> oleksandr.pet...@gmail.com> wrote:
>
> > LWTTester issues are leftovers from the bad merge in
> > https://github.com/riptano/cassandra-dtest/pull/1214
> >
> > I've rebased and currently re-running tests.
> >
> > On Wed, Sep 28, 2016 at 6:23 PM Philip Thompson <
> > philip.thomp...@datastax.com> wrote:
> >
> > > Hi All,
> > >
> > > cassandra-3.9:
> > > ===
> > > testall: All passed!
> > >
> > > ===
> > > dtest: 4 failures
> > >
> > > cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
> > > .test_round_trip_with_authentication
> > > Flaky test, needs a Jira ticket
> > >
> > >
> > > cql_tests.LWTTester
> > > .conditional_deletes_on_static_columns_with_null_values_test
> > >
> > > cql_tests.LWTTeste
> > > r.conditional_deletes_on_static_columns_with_null_values_batch_test
> > >
> > > cql_tests.LWTTester
> > > .conditional_updates_on_static_columns_with_null_values_test
> > > These three failures are all the same, and need a ticket
> > > ===
> > > upgrade: 661 failures
> > >
> > > I won't enumerate these here. 12697 should resolve them, and also
> > explains
> > > where they're all coming from
> > >
> > > ===
> > > novnode: All passed!
> > >
> > > ===
> > >
> > > trunk:
> > >
> > > ===
> > > testall: 13 failures
> > >
> > > org.apache.cassandra.config.DatabaseDescriptorRefTest
> > > .testDatabaseDescriptorRef
> > >
> > > org.apache.cassandra.config.DatabaseDescriptorRefTest
> > > .testDatabaseDescriptorRef-compression
> > > CASSANDRA-12677 for the two failures above.
> > >
> > > org.apache.cassandra.service.RemoveTest
> > > .testLocalHostId
> > > CASSANDRA-12714
> > >
> > > 10 unlisted timeouts
> > >
> > > ===
> > > dtest: 4 failures
> > >
> > > materialized_views_test.TestMaterializedViews
> > > .clustering_column_test
> > >
> > > materialized_views_test.TestMaterializedViews
> > > .insert_test
> > >
> > > materialized_views_test.TestMaterializedViews
> > > .query_all_new_column_test
> > >
> > > materialized_views_test.TestMaterializedViews
> > > .query_new_column_test
> > >
> > > No need to file a jira for these, I've fixed them already.
> > >
> > > ===
> > > upgrade: 662 failures
> > >
> > > I won't enumerate these here. 12697 should resolve them, and also
> > explains
> > > where they're all coming from
> > >
> > > ===
> > > novnode: 9 failures
> > >
> > > Same 8 paging failures. CASSANDRA-12666
> > >
> > > compaction_test.TestCompaction_with_DateTieredCompactionStrategy
> > > .bloomfilter_size_test
> > > CASSANDRA-12711
> > >
> > > Thanks,
> > > Philip
> > >
> > --
> > Alex Petrov
> >
>
-- 
Alex Petrov

Re: Failing tests 2016-09-28

[Oleksandr Petrov]

Might be I have only made differentiation for trunk and 2.x.

I'll investigate and report on JIRA. Thanks for assigning.

On Wed, Sep 28, 2016 at 6:48 PM Philip Thompson <
philip.thomp...@datastax.com> wrote:

> That ticket was only supposed to be committed to 3.10 and 3.0.x. Was it
> accidentally also merged into 3.9?
>
> On Wed, Sep 28, 2016 at 12:40 PM, Oleksandr Petrov <
> oleksandr.pet...@gmail.com> wrote:
>
> > LWTTester issues are leftovers from the bad merge in
> > https://github.com/riptano/cassandra-dtest/pull/1214
> >
> > I've rebased and currently re-running tests.
> >
> > On Wed, Sep 28, 2016 at 6:23 PM Philip Thompson <
> > philip.thomp...@datastax.com> wrote:
> >
> > > Hi All,
> > >
> > > cassandra-3.9:
> > > ===
> > > testall: All passed!
> > >
> > > ===
> > > dtest: 4 failures
> > >
> > > cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
> > >  .test_round_trip_with_authentication
> > > Flaky test, needs a Jira ticket
> > >
> > >
> > > cql_tests.LWTTester
> > >  .conditional_deletes_on_static_columns_with_null_values_test
> > >
> > > cql_tests.LWTTeste
> > >  r.conditional_deletes_on_static_columns_with_null_values_batch_test
> > >
> > > cql_tests.LWTTester
> > >  .conditional_updates_on_static_columns_with_null_values_test
> > > These three failures are all the same, and need a ticket
> > > ===
> > > upgrade: 661 failures
> > >
> > > I won't enumerate these here. 12697 should resolve them, and also
> > explains
> > > where they're all coming from
> > >
> > > ===
> > > novnode: All passed!
> > >
> > > ===
> > >
> > > trunk:
> > >
> > > ===
> > > testall: 13 failures
> > >
> > > org.apache.cassandra.config.DatabaseDescriptorRefTest
> > >  .testDatabaseDescriptorRef
> > >
> > > org.apache.cassandra.config.DatabaseDescriptorRefTest
> > >  .testDatabaseDescriptorRef-compression
> > > CASSANDRA-12677 for the two failures above.
> > >
> > > org.apache.cassandra.service.RemoveTest
> > >  .testLocalHostId
> > > CASSANDRA-12714
> > >
> > > 10 unlisted timeouts
> > >
> > > ===
> > > dtest: 4 failures
> > >
> > > materialized_views_test.TestMaterializedViews
> > >  .clustering_column_test
> > >
> > > materialized_views_test.TestMaterializedViews
> > >  .insert_test
> > >
> > > materialized_views_test.TestMaterializedViews
> > >  .query_all_new_column_test
> > >
> > > materialized_views_test.TestMaterializedViews
> > >  .query_new_column_test
> > >
> > > No need to file a jira for these, I've fixed them already.
> > >
> > > ===
> > > upgrade: 662 failures
> > >
> > > I won't enumerate these here. 12697 should resolve them, and also
> > explains
> > > where they're all coming from
> > >
> > > ===
> > > novnode: 9 failures
> > >
> > > Same 8 paging failures. CASSANDRA-12666
> > >
> > > compaction_test.TestCompaction_with_DateTieredCompactionStrategy
> > >  .bloomfilter_size_test
> > > CASSANDRA-12711
> > >
> > > Thanks,
> > > Philip
> > >
> > --
> > Alex Petrov
> >
>
-- 
Alex Petrov

Re: Failing tests 2016-09-28

[Oleksandr Petrov]

LWTTester issues are leftovers from the bad merge in
https://github.com/riptano/cassandra-dtest/pull/1214

I've rebased and currently re-running tests.

On Wed, Sep 28, 2016 at 6:23 PM Philip Thompson <
philip.thomp...@datastax.com> wrote:

> Hi All,
>
> cassandra-3.9:
> ===
> testall: All passed!
>
> ===
> dtest: 4 failures
>
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
>  .test_round_trip_with_authentication
> Flaky test, needs a Jira ticket
>
>
> cql_tests.LWTTester
>  .conditional_deletes_on_static_columns_with_null_values_test
>
> cql_tests.LWTTeste
>  r.conditional_deletes_on_static_columns_with_null_values_batch_test
>
> cql_tests.LWTTester
>  .conditional_updates_on_static_columns_with_null_values_test
> These three failures are all the same, and need a ticket
> ===
> upgrade: 661 failures
>
> I won't enumerate these here. 12697 should resolve them, and also explains
> where they're all coming from
>
> ===
> novnode: All passed!
>
> ===
>
> trunk:
>
> ===
> testall: 13 failures
>
> org.apache.cassandra.config.DatabaseDescriptorRefTest
>  .testDatabaseDescriptorRef
>
> org.apache.cassandra.config.DatabaseDescriptorRefTest
>  .testDatabaseDescriptorRef-compression
> CASSANDRA-12677 for the two failures above.
>
> org.apache.cassandra.service.RemoveTest
>  .testLocalHostId
> CASSANDRA-12714
>
> 10 unlisted timeouts
>
> ===
> dtest: 4 failures
>
> materialized_views_test.TestMaterializedViews
>  .clustering_column_test
>
> materialized_views_test.TestMaterializedViews
>  .insert_test
>
> materialized_views_test.TestMaterializedViews
>  .query_all_new_column_test
>
> materialized_views_test.TestMaterializedViews
>  .query_new_column_test
>
> No need to file a jira for these, I've fixed them already.
>
> ===
> upgrade: 662 failures
>
> I won't enumerate these here. 12697 should resolve them, and also explains
> where they're all coming from
>
> ===
> novnode: 9 failures
>
> Same 8 paging failures. CASSANDRA-12666
>
> compaction_test.TestCompaction_with_DateTieredCompactionStrategy
>  .bloomfilter_size_test
> CASSANDRA-12711
>
> Thanks,
> Philip
>
-- 
Alex Petrov

Re: Failing tests 2016-09-15

[Oleksandr Petrov]

I've been able to reproduce both SASI statics (saved sstables, going to
take a closer look) and 11031 tests with novnode (looks like paging problem
that was not appearing when all parts of partition key were locked), will
create a Jira ticket today.

On Fri, Sep 16, 2016 at 7:24 AM Joel Knighton 
wrote:

> cassandra-3.9: No new runs
>
>
> trunk
> ===
> testall: 6 failures
>   org.apache.cassandra.cql3.KeyCacheCqlTest
>   .test2iKeyCachePathsShallowIndexEntry
>
>   org.apache.cassandra.cql3.KeyCacheCqlTest
>   .test2iKeyCachePathsShallowIndexEntry-compression
>   CASSANDRA-12650 for the two failures above. New flaky failure.
>
>   org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest
>   .testAllowFilteringOnPartitionKeyWithSecondaryIndex
>
>   org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest
>   .testAllowFilteringOnPartitionKeyWithSecondaryIndex-compression
>   CASSANDRA-12651 for the two failures above. New flaky failure.
>
>   org.apache.cassandra.index.sasi.SASIIndexTest
>   .testMultiExpressionQueriesWhereRowSplitBetweenSSTables
>   Looks like an environmental problem where the forked JVM exited.
>   I'm holding off on creating a JIRA for now.
>
>   org.apache.cassandra.index.sasi.SASIIndexTest
>   .testStaticIndex-compression
>   CASSANDRA-12652. New flaky failure.
>
> ===
> dtest: 4 failures
>   cdc_test.TestCDC.test_cdc_data_available_in_cdc_raw
>   CASSANDRA-11811. Known flaky failure.
>
>   materialized_views_test.TestMaterializedViews
>   .add_node_after_mv_test
>   CASSANDRA-12140. Known flaky failure.
>
>   materialized_views_test.TestMaterializedViews
>   .really_complex_repair_test
>   CASSANDRA-12475. Known flaky failure.
>
>   snitch_test.TestGossipingPropertyFileSnitch
>   .test_prefer_local_reconnect_on_listen_address
>   A typo fix was committed to trunk without updating the test
>   looking for the log message.
>
> ===
> novnode: 6 failures
>   paging_test.TestPagingData
>   .test_paging_with_filtering_on_partition_key
>
>   paging_test.TestPagingData
>   .test_paging_with_filtering_on_partition_key_on_clustering_columns
>
>   paging_test.TestPagingData
>
>
> .test_paging_with_filtering_on_partition_key_on_clustering_columns_with_contains
>
>   paging_test.TestPagingData
>   .test_paging_with_filtering_on_partition_key_on_counter_columns
>   Four new failures, bisect suggests they are due to
>   CASSANDRA-11031. Only failed on novnode. I've asked Alex
>   Petrov to take a look. No JIRA yet.
>
>   snitch_test.TestGossipingPropertyFileSnitch
>   .test_prefer_local_reconnect_on_listen_address
>   Same as the vnode failure above.
>
>   replication_test.SnitchConfigurationUpdateTest
>   .test_rf_collapse_property_file_snitch
>   New flaky failure. No JIRA created yet.
>
> ===
> upgrade: All passed!
>
-- 
Alex Petrov

Re: Failing tests 2016-09-14

[Oleksandr Petrov]

> SelectTest start to be pretty big.

Agree, I've started to get that feeling as well.

On Thu, Sep 15, 2016 at 9:42 AM Benjamin Lerer 
wrote:

> SelectTest start to be pretty big. It makes sense to splitting it into
> separate TestClasses. For example we could extract all the filtering tests
> into a new TestClass: FilteringTest or SelectWithFilteringTest.
>
> On Thu, Sep 15, 2016 at 8:34 AM, Oleksandr Petrov <
> oleksandr.pet...@gmail.com> wrote:
>
> > > CASSANDRA-11031
> >
> > Yes, sorry for delay with #11031 dtests. I've ran updated dtests
> yesterday
> > and they were clean to merge. I just wanted to make sure someone else
> takes
> > a quick glance. By now they're merged, so hopefully today it's going to
> be
> > better.
> >
> > As regards environmental timeouts, it looks like certain methods are more
> > prone to this (in particular, view filtering test does quite a lot). I
> > realise they don't hang, they just execute slower on CI machine than we
> > anticipate. But what should we do with it generally? Increasing timeouts
> > won't really help, so what comes to mind is:
> >   * Splitting tests
> >   * Modularising to make sure unnecessary components don't get started
> >   * Running "slow-prone" tests sequentially to make sure they get enough
> > processor time
> >   * Taking a deeper look, might be there's a performance issue hiding
> > behind
> >   * Thread-dumping in case there is some sort of deadlock that's hard to
> > reproduce on "faster" machine (however improbable that might sound)
> >
> > Since it looks like generally tests are in much better shape, it might
> be a
> > good point to start thinking about those timing out ones.
> >
> >
> >
> >
> > On Thu, Sep 15, 2016 at 7:51 AM Joel Knighton <
> joel.knigh...@datastax.com>
> > wrote:
> >
> > > cassandra-3.9
> > > ===
> > > testall: 8 failures
> > >   org.apache.cassandra.cql3.ViewFilteringTest
> > >   .testPartitionKeyAndClusteringKeyFilteringRestrictions
> > >
> > >   org.apache.cassandra.cql3.ViewFilteringTest
> > >   .testMVCreationSelectRestrictions
> > >
> > >   org.apache.cassandra.cql3.ViewTest.testCompoundPartitionKey
> > >
> > >   org.apache.cassandra.cql3.validation.entities.UFTest.testEmptyString
> > >
> > >   org.apache.cassandra.cql3.validation.operations.AggregationTest
> > >   .testFunctionsWithCompactStorage
> > >
> > >   org.apache.cassandra.cql3.validation.operations.SelectTest
> > >   .testAllowFiltering
> > >   These six test failures are due to environmental timeouts.
> > >
> > >   org.apache.cassandra.db.compaction
> > >   .TimeWindowCompactionStrategyTest
> > >   .testDropExpiredSSTables-compression
> > >   New flaky failure. CASSANDRA-12645 opened.
> > >
> > >   org.apache.cassandra.service.RemoveTest.testBadHostId
> > > CASSANDRA-12487. Flaky failure in a test utility setup method.
> > >
> > > ===
> > > dtest: 1 failure
> > >   user_types_test.TestUserTypes.test_type_as_part_of_pkey
> > > Should have been fixed as part of CASSANDRA-11031. Incorrect
> > > version gating still - I'll follow up and get this fixed tomorrow.
> > >
> > > ===
> > > novnode: 4 failures
> > >   user_types_test.TestUserTypes.test_type_as_part_of_pkey
> > >   Same as above.
> > >
> > >   cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
> > >   .test_bulk_round_trip_with_single_core
> > >   New failure - looks like a schema agreement problem. A JIRA
> > >   hasn't been created yet.
> > >
> > >   cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
> > >   .test_reading_max_insert_errors
> > >   New failure - looks like Netty detected a leak. A JIRA hasn't
> been
> > >   created yet.
> > >
> > >   batch_test.TestBatch.logged_batch_doesnt_throw_uae_test
> > >   CASSANDRA-12383. Flaky failure.
> > >
> > > ===
> > > upgrade: 1 failure
> > >   upgrade_tests.cql_tests
> > >   .TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_3_x
> > >   .bug_5732_test
> > >   CASSANDRA-12457. Patch available that needs a reviewer.
> > >
> > >
> > > Since there's a few open opportunities based on 3.9 failures, I'm only
> > > covering 3.9 on today's email.
> > >
> > --
> > Alex Petrov
> >
>
-- 
Alex Petrov

Re: Failing tests 2016-09-14

[Oleksandr Petrov]

> CASSANDRA-11031

Yes, sorry for delay with #11031 dtests. I've ran updated dtests yesterday
and they were clean to merge. I just wanted to make sure someone else takes
a quick glance. By now they're merged, so hopefully today it's going to be
better.

As regards environmental timeouts, it looks like certain methods are more
prone to this (in particular, view filtering test does quite a lot). I
realise they don't hang, they just execute slower on CI machine than we
anticipate. But what should we do with it generally? Increasing timeouts
won't really help, so what comes to mind is:
  * Splitting tests
  * Modularising to make sure unnecessary components don't get started
  * Running "slow-prone" tests sequentially to make sure they get enough
processor time
  * Taking a deeper look, might be there's a performance issue hiding
behind
  * Thread-dumping in case there is some sort of deadlock that's hard to
reproduce on "faster" machine (however improbable that might sound)

Since it looks like generally tests are in much better shape, it might be a
good point to start thinking about those timing out ones.




On Thu, Sep 15, 2016 at 7:51 AM Joel Knighton 
wrote:

> cassandra-3.9
> ===
> testall: 8 failures
>   org.apache.cassandra.cql3.ViewFilteringTest
>   .testPartitionKeyAndClusteringKeyFilteringRestrictions
>
>   org.apache.cassandra.cql3.ViewFilteringTest
>   .testMVCreationSelectRestrictions
>
>   org.apache.cassandra.cql3.ViewTest.testCompoundPartitionKey
>
>   org.apache.cassandra.cql3.validation.entities.UFTest.testEmptyString
>
>   org.apache.cassandra.cql3.validation.operations.AggregationTest
>   .testFunctionsWithCompactStorage
>
>   org.apache.cassandra.cql3.validation.operations.SelectTest
>   .testAllowFiltering
>   These six test failures are due to environmental timeouts.
>
>   org.apache.cassandra.db.compaction
>   .TimeWindowCompactionStrategyTest
>   .testDropExpiredSSTables-compression
>   New flaky failure. CASSANDRA-12645 opened.
>
>   org.apache.cassandra.service.RemoveTest.testBadHostId
> CASSANDRA-12487. Flaky failure in a test utility setup method.
>
> ===
> dtest: 1 failure
>   user_types_test.TestUserTypes.test_type_as_part_of_pkey
> Should have been fixed as part of CASSANDRA-11031. Incorrect
> version gating still - I'll follow up and get this fixed tomorrow.
>
> ===
> novnode: 4 failures
>   user_types_test.TestUserTypes.test_type_as_part_of_pkey
>   Same as above.
>
>   cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
>   .test_bulk_round_trip_with_single_core
>   New failure - looks like a schema agreement problem. A JIRA
>   hasn't been created yet.
>
>   cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
>   .test_reading_max_insert_errors
>   New failure - looks like Netty detected a leak. A JIRA hasn't been
>   created yet.
>
>   batch_test.TestBatch.logged_batch_doesnt_throw_uae_test
>   CASSANDRA-12383. Flaky failure.
>
> ===
> upgrade: 1 failure
>   upgrade_tests.cql_tests
>   .TestCQLNodes3RF3_Upgrade_current_2_1_x_To_indev_3_x
>   .bug_5732_test
>   CASSANDRA-12457. Patch available that needs a reviewer.
>
>
> Since there's a few open opportunities based on 3.9 failures, I'm only
> covering 3.9 on today's email.
>
-- 
Alex Petrov

Re: Support Multi-Tenant in Cassandra

[Oleksandr Petrov]

There's a ticket on filtering (#11031), although I would not count on
filtering in production.

As it was already mentioned in the ticket itself, filtering is a highly
inefficient operation. it was thought as aid for people who're exploring
data and/or can structure query in such a way that it will at least be
local (for example, with IN or EQ query on the partition key and filtering
out results from the small partition). However, filtering on the Partition
Key assumes that _every_ replica has to be queried for the results, as we
do not know which partitions are going to be holding the data. Having every
query in your system to rely on filtering, big amount of data and high load
will eventually have substantial negative impact on performance.

I'm not sure what's the amount of tenants you're working with, although
I've seen setups where tenancy was solved by using multiple keyspaces,
which helps to completely isolate the data, avoid filtering. Given that
you've tried splitting sstables on tenant_id, that might be solved by using
multiple keyspaces. This will also help with server resource isolation and
most of the issues you've raised.


On Fri, Jul 15, 2016 at 10:10 AM Romain Hardouin
 wrote:

> I don't use C* in such a context but out of curiosity did you set
> the request_scheduler to RoundRobin or did you implement your own scheduler?
> Romain
> Le Vendredi 15 juillet 2016 8h39, jason zhao yang <
> zhaoyangsingap...@gmail.com> a écrit :
>
>
>  Hi,
>
> May I ask is there any plan of extending functionalities related to
> Multi-Tenant?
>
> Our current approach is to define an extra PartitionKey called "tenant_id".
> In my use cases, all tenants will have the same table schemas.
>
> * For security isolation: we customized GRANT statement to be able to
> restrict user query based on the "tenant_id" partition.
>
> * For getting all data of single tenant, we customized SELECT statement to
> support allow filtering on "tenant_id" partition key.
>
> * For server resource isolation, I have no idea how to.
>
> * For per-tenant backup restore, I was trying a
> tenant_base_compaction_strategy to split sstables based on tenant_id. it
> turned out to be very inefficient.
>
> What's community's opinion about submitting those patches to Cassandra? It
> will be great if you guys can share the ideal Multi-Tenant architecture for
> Cassandra?
>
> jasonstack
>
>
>

-- 
Alex Petrov

Re: Documentation on a new CQL feature of 3.6

[Oleksandr Petrov]

As far as I understand this wording, it's correct. Prior to 3.6 *filtering*
was not allowed on clustering columns. It was allowed to do non-filtering
queries involving clustering columns, although you could not specify any
clustering column (or combine multiple range queries). Now it is allowed.

So the wording "clustering columns can be defined in WHERE clauses* if
ALLOW FILTERING is also used even if a secondary index is not created*" is
correct (emphasis to indicate the context you specified). Might be that
secondary index part sounded like the behaviour was somewhat similar before.

For the background, you can check
https://issues.apache.org/jira/browse/CASSANDRA-11310

Having that said, I'll add a little guide for querying capabilities for the
current version in scope of 8700.

Thank you

On Wed, Jun 15, 2016 at 5:03 PM Giampaolo Trapasso <
giampaolo.trapa...@radicalbit.io> wrote:

> Hi to all,
>
> DS Documentation says that
> *In Cassandra 3.6 and later, clustering columns can be defined in WHERE
> clauses if ALLOW FILTERING is also used even if a secondary index is not
> created. The table definition is given and then the SELECT command. Note
> that race_start_date is a clustering column that has no secondary index.*
>
> This seemed strange to me, since since in past I did queries using
> clustering columns.
>
> I did this quick check:
>
> cqlsh:test> show version
> [cqlsh 5.0.1 | Cassandra 2.2.5-SNAPSHOT | CQL spec 3.3.1 | Native protocol
> v4]
> cqlsh:test> SELECT * FROM calendar WHERE race_start_date='2015-06-13' ALLOW
> FILTERING;
>
>  race_id | race_start_date | race_end_date | race_name
> -+-+---+---
>
> (0 rows)
> cqlsh:test> SELECT * FROM calendar WHERE race_end_date='2015-06-13' ALLOW
> FILTERING;
> InvalidRequest: code=2200 [Invalid query] message="PRIMARY KEY column
> "race_end_date" cannot be restricted as preceding column "race_start_date"
> is not restricted"
> cqlsh:test>
>
> As you can see, in < 3.6 you can put clustering columns in queries as long
> as you respect the "preceding column" constraint. IMHO, that line should be
> changed saying that an arbitrary clustering column can be used from 3.6,
> and the example should use the 'race_end_date".
>
> This is DS documentation. Did not find something similar in community
> documentation (anycase 8700 is a work in progress, I will check in future).
> Let me know if I'm missing some point.
>
> Giampaolo
>
-- 
Alex Petrov

Re: Cassandra Read Path Code Navigation

[Oleksandr Petrov]

Hi,

The query behaviour should not rely on the compaction. It'd be great to
have a Jira ticket for that.

It'd also be very useful if you described your setup a bit more, are you
using SASI for like queries (assuming wildcards in the '*Size_s*')?

As regarding the read path, there's a nice talk by Tyler Hobbs about
read/write paths which might get you started:
https://www.youtube.com/watch?v=9Id5me7QFHU

Briefly, there are several paths, each one used in it's own setting and
multiple things play together for the query execution. It all starts with
the SelectStatement, which determines whether it's a single partition
(SinglePartitionReadCommand) or partition range read
(PartitionRangeReadCommand).

StatementRestrictions are responsible for everything related to the WHERE
clause of the query, including index queries and filtering. You can see
that restrictions are separated into PartitionKey restrictions,
ClusteringColumn Restrictions and NonPrimary column restrictions. Here, the
bounds are created for the queries that have keys specified in the order
that makes it possible to query without filtering and filter restrictions
are added otherwise.

Usually it's not necessary to go all the way down to the SSTables, as read
path is very well abstracted from the underlying storage, since the read
involves Memtable lookup as well. Also, if the query returns too many
results, it seems that the engine itself did a good job of reading results
from disk, but some of them were not filtered out.

Hope this helps,

On Tue, Jun 14, 2016 at 7:26 AM Bhuvan Rawal  wrote:

> Hi All,
>
> Im debugging a issue in Cassandra 3.5 which was reported in user mailing
> list earlier, is pretty critical to solve at our end. ill give a brief
> intro: On issuing this query:;
>
> select id,filter_name from navigation_bucket_filter where id=2429 and
> filter_name='*Size_s*';
>
>  id   | filter_name
> --+--
>  2429 | AdditionalProperty_s
>  2429 |Brand
> more rows---
>  2429 |   Size_s
> more rows---
>  2429 | sdFullfilled
>  2429 |   sellerCode
>
> (16 rows)
>
> Whereas *only one result was expected* (Row bearing filter_name - Size_s),
> we got that result but along with 15 other unexpected rows..
>
> Total number of rows in the partition are 20 (Verified using select
> id,filter_name from navigation_bucket_filter where id=2429;) as well as
> json dump. We are wondering why Cassandra could not filter the results
> completely. I have checked that the data is intact by taking json dump and
> validating using sstabledump tool.
>
> The issue was resolved on production by using nodetool compact, but
> debugging it is critical as to what led to this and issuing manual
> compaction may not be possible everytime.
>
> I copied the sstables of the particular table onto my local machine and
> *have
> been able to reproduce the same* issue, while trying to run Cassandra in
> debug mode I have been able to connect my IDE with it but unfortunately I
> have not been able to navigate really far in the Read Path. Will be glad to
> get a some pointers on where in the code SSTables are read and partition is
> filtered.
>
> Secondly, I wanted to know if there is a possible way by which we can read
> the other SSTable files (Partition Index) Filter.db, Statistics.db, et al
> as well as Commitlog. If such a utility does not exist currently but can be
> created from existing classes pls let me know as well would love to build
> and share one.
>
>
> Best Regards,
> Bhuvan Rawal
>
-- 
Alex Petrov

Re: How to debug a unit test?

[Oleksandr Petrov]

I'm not entirely sure how to debug in terminal. But you can set break
points and debug in eclipse and IntelliJ without a problem.

I strongly recommend reading the contributing guide:
https://wiki.apache.org/cassandra/HowToContribute

On Fri, 20 May 2016 at 03:18, Mahdi Mohammadi  wrote:

> Hi,
>
> I can run a single unit test using `ant test -Dtest.name=TraceCqlTest`.
> How can I debug the test using gdb (in the terminal)?
>
> Best Regards
>
-- 
Alex Petrov

Re: Error running 'ant test'

[Oleksandr Petrov]

It passes for me, and seems to work on the CI.
Rule of thumb for me personally is:
  * make sure that everything cleaned up (no c* instances running, no tests
running at same time)
  * make sure the latest version is pulled
  * in case something failed, try running more than once

If it constantly fails, fire up the issue.
If it failed just that one time, forget about it for now, it'll pop up
again, so you'll think of it then.
If it fails every other time, fire up the issue and explain that it doesn't
fail all the time.


On Thu, May 19, 2016 at 11:43 AM Mahdi Mohammadi  wrote:

> I ran your command and test is successful but still there are tests which
> are failing and I am confused.
> My Java version is 1.8.0_74 and OS is 64 bit Debian (Vagrant box).
>
> When I run: *ant clean && ant test -Dtest.name=CassandraIndexTest *(both on
> trunk and cassandra-3.0), I get error:
>
>* [junit] INFO  09:40:24 Initializing
> cql_test_keyspace.table_24.table_24_c_idx*
> *[junit] INFO  09:40:24 Drop Keyspace 'cql_test_keyspace_alt'*
> *[junit] -  ---*
> *[junit] Testcase:
>
> indexOnRegularColumn(org.apache.cassandra.index.internal.CassandraIndexTest):
> FAILED*
> *[junit] Got less rows than expected. Expected 1 but got 0*
> *[junit] junit.framework.AssertionFailedError: Got less rows than
> expected. Expected 1 but got 0*
> *[junit] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:818)*
> *[junit] at
>
> org.apache.cassandra.index.internal.CassandraIndexTest$TestScript.run(CassandraIndexTest.java:660)*
> *[junit] at
>
> org.apache.cassandra.index.internal.CassandraIndexTest.indexOnRegularColumn(CassandraIndexTest.java:70)*
> *[junit]*
> *[junit]*
> *[junit] Testcase:
>
> indexOnNonFrozenListWithReplaceOperation(org.apache.cassandra.index.internal.CassandraIndexTest):
> FAILED*
> *[junit] Got less rows than expected. Expected 1 but got 0*
> *[junit] junit.framework.AssertionFailedError: Got less rows than
> expected. Expected 1 but got 0*
> *[junit] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:818)*
> *[junit] at
>
> org.apache.cassandra.index.internal.CassandraIndexTest$TestScript.run(CassandraIndexTest.java:660)*
> *[junit] at
>
> org.apache.cassandra.index.internal.CassandraIndexTest.indexOnNonFrozenListWithReplaceOperation(CassandraIndexTest.java:148)*
> *[junit]*
> *[junit]*
> *[junit] Testcase:
>
> indexOnNonFrozenMapValuesWithReplaceOperation(org.apache.cassandra.index.internal.CassandraIndexTest):
>FAILED*
> *[junit] Got less rows than expected. Expected 1 but got 0*
> *[junit] junit.framework.AssertionFailedError: Got less rows than
> expected. Expected 1 but got 0*
> *[junit] at
> org.apache.cassandra.cql3.CQLTester.assertRows(CQLTester.java:818)*
> *[junit] at
>
> org.apache.cassandra.index.internal.CassandraIndexTest$TestScript.run(CassandraIndexTest.java:627)*
> *[junit] at
>
> org.apache.cassandra.index.internal.CassandraIndexTest.indexOnNonFrozenMapValuesWithReplaceOperation(CassandraIndexTest.java:212)*
> *[junit]*
> *[junit]*
> *[junit] Test org.apache.cassandra.index.internal.CassandraIndexTest
> FAILED*
>
>
>
>
> Best Regards
>
> On Thu, May 19, 2016 at 3:21 PM, Oleksandr Petrov <
> oleksandr.pet...@gmail.com> wrote:
>
> > I just tried it locally, `ant clean && ant test
> > -Dtest.name=TopKSamplerTest` works just fine (tried the other test, too).
> > What's your JDK version? I can't see anything obvious that is wrong. If
> the
> > branch is up-to-date, it usually has to build.
> >
> >
> > On Thu, May 19, 2016 at 5:12 AM Mahdi Mohammadi 
> wrote:
> >
> > > I am trying to run `ant test` on cassandra-3.0 branch but get some
> > errors:
> > >
> > > *[junit] Testcase:
> > >
> > >
> >
> testSamplerSingleInsertionsEqualMulti(org.apache.cassandra.utils.TopKSamplerTest):
> > >FAILED*
> > > *[junit] expected:<{item2=2, item1=1, item8=8, item7=7, item9=9,
> > > item4=4, item10=10, item3=3, item6=6, item5=5}> but was:<{item2=2,
> > item1=1,
> > > item8=8, item7=7, item9=9, item4=4, item3=3, item6=6,*
> > > *item5=5}>*
> > > *[junit] junit.framework.AssertionFailedError: expected:<{item2=2,
> > > item1=1, item8=8, item7=7, item9=9, item4=4, item10=10, item3=3,
> item6=6,
> > > item5=5}> but was:<{item2=2, item1=1, item8=8, item7=7,*
> > > *item9=9, item4=4, item3=3, item6=6

Re: How cassandra ensures consistency when adding or removing a node?

[Oleksandr Petrov]

I think that this article [1] covers most of the concepts (see key
concepts) quite well.
I am not aware of any article that explains the whole process, though.

Briefly, there are several processes/concepts that are somewhat related to
that subject: token ownership, replica, coordinator and gossip.
Ensuring consistency in small cluster (amount of replica <= amount of
nodes) is more or less straightforward. In this case, when node bootstraps,
it notifies all the replicas, information about that node gets added to
`pending nodes`, all nodes know about the bootstrapping node, as otherwise
streaming would not even start.
Having a coordinator outside of replica for the partition/token you're
querying is a bit more complex, as it involves the knowledge about the
joined node that's distributed over gossip.

There are two properties that can improve the situation with range
movements: cassandra.consistent.rangemovement
and cassandra.consistent.simultaneousmoves.allow. First one disallows ring
changes in case there's any node in replica is offline. In addition to
that, it makes sure there are no moves within the ring. In that case, if
you're connected to coordinator that's a part of replica, data has to be
placed correctly. The data will be moved and any inconsistencies will be
eventually fixed with a repair (answering your question, there will be no
data lost during this process).

(I tried to provide information according to my best knowledge, although if
anyone sees something wrong, please indicate accordingly)

[1] https://dzone.com/articles/introduction-apache-cassandra

On Thu, May 19, 2016 at 5:58 AM Renjie Liu  wrote:

> BTW, is there any article explaining the process? I think this will help us
> understand it better.
>
> On Thu, May 19, 2016 at 11:28 AM Renjie Liu 
> wrote:
>
> > Thanks, I'll read the code.
> >
> > On Thu, May 19, 2016 at 11:02 AM Jeff Jirsa 
> > wrote:
> >
> >>
> >>
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/locator/TokenMetadata.java#L731-L754
> >>
> >>
> >> And
> >>
> >>
> >>
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/locator/TokenMetadata.java#L60-L88
> >>
> >>
> >>
> >> Cassandra keeps a map of joining and leaving nodes, and does extra
> writes
> >> to the appropriate nodes for mutations created after the streaming is
> >> calculated.
> >>
> >>
> >>
> >> On 5/18/16, 7:33 PM, "Renjie Liu"  wrote:
> >>
> >> >Hi, cassandra devs:
> >> >I'm learning cassandra and I can understand most of the techniques
> used.
> >> >But I can't understand how cassandra ensures consistency when
> >> >adding/removing a node? It seems that when a node joins the dht ring,
> >> some
> >> >node need to transferring data to the new node using streaming. But the
> >> >data may still get updated while transferring, so the new node can
> never
> >> >catch up with it. How cassandra handles this? Will cassandra lose data
> >> >during this process?
> >> >--
> >> >Liu, Renjie
> >> >Software Engineer, MVAD
> >
> > --
> > Liu, Renjie
> > Software Engineer, MVAD
> >
> --
> Liu, Renjie
> Software Engineer, MVAD
>
-- 
Alex Petrov

Re: Error running 'ant test'

[Oleksandr Petrov]

I just tried it locally, `ant clean && ant test
-Dtest.name=TopKSamplerTest` works just fine (tried the other test, too).
What's your JDK version? I can't see anything obvious that is wrong. If the
branch is up-to-date, it usually has to build.


On Thu, May 19, 2016 at 5:12 AM Mahdi Mohammadi  wrote:

> I am trying to run `ant test` on cassandra-3.0 branch but get some errors:
>
> *[junit] Testcase:
>
> testSamplerSingleInsertionsEqualMulti(org.apache.cassandra.utils.TopKSamplerTest):
>FAILED*
> *[junit] expected:<{item2=2, item1=1, item8=8, item7=7, item9=9,
> item4=4, item10=10, item3=3, item6=6, item5=5}> but was:<{item2=2, item1=1,
> item8=8, item7=7, item9=9, item4=4, item3=3, item6=6,*
> *item5=5}>*
> *[junit] junit.framework.AssertionFailedError: expected:<{item2=2,
> item1=1, item8=8, item7=7, item9=9, item4=4, item10=10, item3=3, item6=6,
> item5=5}> but was:<{item2=2, item1=1, item8=8, item7=7,*
> *item9=9, item4=4, item3=3, item6=6, item5=5}>*
> *[junit] at
>
> org.apache.cassandra.utils.TopKSamplerTest.testSamplerSingleInsertionsEqualMulti(TopKSamplerTest.java:39)*
> *[junit]*
>
> Or this one:
>
> *[junit] Testcase:
> executeTriggerOnThriftInsert(org.apache.cassandra.triggers.TriggersTest):
> Caused an ERROR*
> *[junit] Trigger trigger_1 already exists*
> *[junit] org.apache.cassandra.exceptions.InvalidRequestException:
> Trigger trigger_1 already exists*
> *[junit] at
>
> org.apache.cassandra.cql3.statements.CreateTriggerStatement.announceMigration(CreateTriggerStatement.java:85)*
> *[junit] at
>
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:93)*
> *[junit] at
>
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)*
> *[junit] at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237)*
> *[junit] at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:252)*
> *[junit] at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:247)*
> *[junit] at
> org.apache.cassandra.triggers.TriggersTest.setup(TriggersTest.java:92)*
> *[junit]*
> *[junit]*
>
> Am I missing something?
>
> Best Regards
>
-- 
Alex Petrov

Re: [Proposal] Mandatory comments

[Oleksandr Petrov]

I think that Netty is a good example of comments, at least for me they
often were helpful, both from perspective of working on the code and from
user perspective (when something seems to be working not as expected, you
can usually find out reasons for it from the code in a straightforward
way).

I definitely support the idea of "high-level comments" (someone said
Class-level ones). Alongside, some fields might be good to describe,
especially if they have multiple purposes. If methods have conceptual
names, they're also worth commenting. All parts where protocol is implied
may reference where one could lookup how certain things are serialized
(more applicable for the collections, probably).

I can bring up several examples from top of my head of the things I could
volunteer to start commenting straight away: NativeCell (could be helpful
to document the protocol format there), CollectionSerializer, CellPath
serializer, ModificationStatement and things related to CAS, things like
values and bounds as clustering in Restrictions (and maybe recently
refactored Restriction hierarchy in general), Value (Terminal) classes in
collections, Markers, Operations, Column Conditions, DataLimits,
ColumnFamilyStore (or at least parts of it), Memtable.

Sorry for the lengthy list: it was something I stumbled upon recently, and
gladly having tests and navigating through the code helped to understand
what is going on in majority of cases.

But I wanted to bring up several positive examples as well: Cell class is
very well documented, describing it's purpose and the protocol format,
UnfilteredSerializer is quite good in terms of the protocol and overall
description, too, DataRange (describes some "tricks", too),
CqlSStableWriter, SSTable. There are many more examples, too.

So definitely count me in on improving comments. I like writing (as you can
note from the length of this email), so I'd be also very happy to help to
improve.



On Wed, May 4, 2016 at 7:21 PM Jonathan Ellis  wrote:

> On Wed, May 4, 2016 at 2:27 AM, Sylvain Lebresne 
> wrote:
>
> > On Tue, May 3, 2016 at 6:57 PM, Eric Evans 
> > wrote:
> >
> > > On Mon, May 2, 2016 at 11:26 AM, Sylvain Lebresne <
> sylv...@datastax.com>
> > > wrote:
> > > > Looking forward to other's opinions and feedbacks on this proposal.
> > >
> > > We might want to leave just a little wiggle room for judgment on the
> > > part of the reviewer, for the very simple cases.  Documenting
> > > something like setFoo(int) with "Sets foo" can get pretty tiresome for
> > > everyone, and doesn't add any value.
> > >
> >
> > I knew someone was going to bring this :). In principle, I don't really
> > disagree. In practice though,
> > I suspect it's sometimes just easier to adhere to such simple rule
> somewhat
> > strictly. In particular,
> > I can guarantee that we don't all agree where the border lies between
> what
> > warrants a javadoc
> > and what doesn't. Sure, there is a few cases where you're just
> paraphrasing
> > the method name
> > (and while it might often be the case for getters and setters, it's worth
> > noting that we don't really
> > do much of those in C*), but how hard is it to write a one line comment?
> > Surely that's a negligeable
> > part of writing a patch and we're not that lazy.
> >
>
> I'm more concerned that this kind of boilerplate commenting obscures rather
> than clarifies.  When I'm reading code i look for comments to help me
> understand key points, points that aren't self-evident.  If we institute a
> boilerplate "comment everything" rule then I lose that signpost.
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>
--