Re: Proof of concept for Cassandra docs conversion

[Murukesh Mohanan]

Is the AsciiDoc source also on GitHub?

Some things that I noticed:

- The FAQ sidebar navigation is great!  The TOC list at the top isn't
needed at all now.
- In the breadcrumb trail (e.g., ASCIIDOC_POC > Cassandra > FAQ, or
ASCIIDOC_POC > Cassandra Documentation > Glossary), *Cassandra* in the
Cassandra pages isn't a link but Cassandra Documentation is.
- Some of the code blocks are off (e.g,
https://polandll.github.io/site/ASCIIDOC_POC/4.0/cassandra/cql/indexes.html#create-index-statement,
https://polandll.github.io/site/ASCIIDOC_POC/4.0/cassandra/cql/mvs.html)
- Pandoc clobbered the "MV select" part in
https://polandll.github.io/site/ASCIIDOC_POC/4.0/cassandra/cql/mvs.html
into the preceding note.

I wonder if it will be worth it to try rSt -> HTML (using the current build
tools) -> AsciiDoc (using Pandoc).

Yours,
Murukesh Mohanan


On Thu, 11 Jun 2020 at 03:21, Lorina Poland  wrote:

> Hello all!
>
> Based on an earlier discussion about moving the OSS C* docs to
> asciidoc, to allow more flexibility in maintaining the docs, I've done
> a proof of concept about what it would take to accomplish a
> conversion. I converted rSt files to asciidoc files using pandoc, did
> some additional editing, and use antora (antora.org) as a static site
> generator to build the docs. The result is here:
>
> https://polandll.github.io/site/ASCIIDOC_POC/4.0/cassandra/getting_started/configuring.html#changing-the-location-of-directoriesThe
> editing of the docs is NOT complete, but I completed enough to feel
> confident that this process can be accomplished. Some YAML
> configuration for antora was required, and I did a minimum of UI
> configuration (added color banner, logo). Looking for feedback and
> questions anyone may have.
>
> Thanks,
>
> Lorina Poland (DataStax tech writer)
>

Re: [DISCUSS] governance on the Apache Cassandra project

[Murukesh Mohanan]

A couple of thoughts:

1. It seems all rules on voting are predicated on the question being
binary. Perhaps we should also tack on a section for cases where we have to
pick among multiple options (a simple plurality, maybe).
2. Should this itself be a CEP? (E.g., Python's governance model was
proposed in PEPs 8010 through 8016 ([1][2] etc.) and codified in PEP 13
[3], PHP's voting system was iterated upon recently in their equivalent, an
RFC [4]) Or perhaps not, since the current CEP description suggests that
CEPs are exclusively for code changes? (If that's the case, we could
discuss that at a later date, and move on with this proposal first.)

[1]: https://www.python.org/dev/peps/pep-8015/
[2]: https://www.python.org/dev/peps/pep-8016/
[3]: https://www.python.org/dev/peps/pep-0013/
[4]: https://wiki.php.net/rfc/abolish-narrow-margins

Yours,
Murukesh Mohanan


On Fri, 5 Jun 2020 at 01:54, Joshua McKenzie  wrote:

> Hello project!
>
> The pmc has been discussing how we make decisions as a pmc, how we make
> decisions as a project of committers and contributors, what decisions are
> made where, and how those decisions are ratified and by whom. We came to
> the conclusion that there's value in having a more formal (though
> lightweight) structure around these topics as well as start to enumerate
> some expectations on how we interact with each other on the project as it
> matures.
>
> A link to the current draft of the governance doc is here:
>
> https://docs.google.com/document/d/1wOrJBkgudY2BxEVtubq9IbiFFC3d3efJSj9OIrGcqQ8/edit#
>
> The doc is only 2 pages long; if you're interested in engaging in a
> discussion about how we evolve and collaborate as a project, please take
> some time to read through the doc, think through things, and engage on this
> thread here.
>
> Thanks everyone, and looking forward to a great discussion!
>
> ~Josh McKenzie
>

Re: Staging website at cassandra.staged.apache.org

[Murukesh Mohanan]

For clarification, when you say "this would clean the master
branch history", would the content directory be removed from past commits
using bfg or similar tools?

(I'm fine with either way; just curious.)

On Fri, 1 May 2020, 07:12 Anthony Grasso,  wrote:

> Hi everyone,
>
> Thanks to hard work by Mick every push to the cassandra-website *src/*
> directory in master is now automatically deployed to
> https://cassandra.staged.apache.org/. The automation is carried out by a
> Jenkins job at https://ci-cassandra.apache.org/job/cassandra-website/.
> This
> is really useful because it allows us to preview our changes in the staging
> site before we push them to the production site!
>
> With the above change in place, the process to deploy a change to the
> production site is.
> 1. Commit your change to the *src/* directory in the master branch. Jenkins
> will now automatically generate the (HTML) site pages in *content/*
> directory on the asf-staging branch.
> 2. Preview your changes on the staging site:
> https://cassandra.staged.apache.org/.
> 3. If the changes look good, then merge the *content/* directory on the
> asf-staging branch to the master branch using:
>
>
> $ git merge asf-staging
> $ git push
>
>
> An issue with the above process is the master branch history is now going
> to get polluted with the auto-generated content commits. Even without the
> Jenkins automation, the process to publish to the production site still had
> the issue where commits to the master branch included both the *src/* and
> *content/* directory changes. A small one line change to the *src/*
> directory, could result in changes to hundreds of pages in the *content/*
> directory.
>
> Rather than serving the production site from the *content/* directory on
> the master branch, is there any objection to serving it from the asf-site
> branch? This would mean that the last step in the above process would be
> performed on the asf-site branch rather than the master branch. In
> addition, there would be no need to have a *content/* directory on the
> master branch. The *content/* directory from which the production site is
> served would live in the asf-site branch. This would clean the master
> branch history, hiding the generated *content/* directory and the unwieldy
> content changes generated by Jenkins user.
>
> Let me know what you think about the proposed change.
>
> Regards,
> Anthony
>
> On Tue, 21 Apr 2020 at 16:40, Mick Semb Wever  wrote:
>
> > For our cassandra-website repository, any changes to our website can now
> > first be staged at https://cassandra.staged.apache.org/
> >
> > The staged website comes from the content/ directory on the `asf-staging`
> > branch.
> >
> > regards,
> > Mick
> >
>

Shouldn't GitHub emails go to some other mailing list?

[Murukesh Mohanan]

Don't we have a pr@ mailing list for these?

Yours,
Murukesh Mohanan


On Thu, 27 Feb 2020 at 03:56, GitBox  wrote:

> dcapwell commented on a change in pull request #1: Introduce the extracted
> in-JVM DTest API
> URL:
> https://github.com/apache/cassandra-in-jvm-dtest-api/pull/1#discussion_r384691389
>
>
>
>  ##
>  File path:
> src/main/java/org/apache/cassandra/distributed/shared/Versions.java
>  ##
>  @@ -0,0 +1,201 @@
> +/*
> + * Licensed to the Apache Software Foundation (ASF) under one
> + * or more contributor license agreements.  See the NOTICE file
> + * distributed with this work for additional information
> + * regarding copyright ownership.  The ASF licenses this file
> + * to you under the Apache License, Version 2.0 (the
> + * "License"); you may not use this file except in compliance
> + * with the License.  You may obtain a copy of the License at
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +
> +package org.apache.cassandra.distributed.shared;
> +
> +import java.io.File;
> +import java.net.MalformedURLException;
> +import java.net.URL;
> +import java.nio.file.Paths;
> +import java.util.*;
> +import java.util.regex.Matcher;
> +import java.util.regex.Pattern;
> +import java.util.stream.Collectors;
> +
> +public class Versions
>
>  Review comment:
>FYI one thing I have been seeing is that 2 distinct versions with the
> same major.minor is unclear which one gets picked up (think current branch
> is 3.0 and you pull in the 3.0 dtest for the previous release)
>
>This should be improved, but will be so much easier once this JIRA is
> completed!
>
> 
> This is an automated message from the Apache Git Service.
> To respond to the message, please log on to GitHub and use the
> URL above to go to the specific comment.
>
> For queries about this service, please contact Infrastructure at:
> us...@infra.apache.org
>
>
> With regards,
> Apache Git Services
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Apache Cassandra Virtual meetings

[Murukesh Mohanan]

Do we need to moderate heavily from the get-go, or should we implement all
this after a couple of trial calls to see how bad things are?

On Sat, 10 Aug 2019, 08:27 Scott Andreas,  wrote:

> On the "virtual" side --
>
> I've spent some time this week reviewing how the Kubernetes community
> conducts their weekly meetings. References are at the end of this message.
>
> If we'd like to hold occasional virtual meetings among the dev and user
> community, here are some things that may help make them successful:
>
> 1. Agenda: Discussed and committed on the dev list in advance, published
> on Confluence, with speakers confirmed and topic durations assigned.
> 2. Video Call: To be dialed into by confirmed speakers, eliminating the
> need for heavy WebEx-style moderation, several minutes of "please mute,
> everyone mute...," or difficulty enforcing ASF code of conduct if someone
> joins an open call in bad faith. Google Meet appears to be free for up to
> 25 participating / presenting in a call.
> 3. Live stream / broadcast: Google Meet meetings can also be broadcast
> live to YouTube and recorded for non-participating attendees to watch
> (e.g., via EveryCord now that Hangouts on Air is discontinued). This would
> enable a model in which a smaller number of speakers could present, with
> anyone in the community able to join and watch live or at a later time.
> 4. Moderation: An emcee to introduce the agenda, hold speakers to time to
> ensure discussion on one item doesn't prevent the rest from being reached,
> and represent community Q
> 5. Community Q: We could conduct Q with community members via ASF
> Slack during the call – e.g., a moderator reading questions shared via
> Slack, answered by presenters. Text-based Q also mitigates the risk of
> runaway "not really a question but more of a comment..." dialogue.
> 6. Notes: As others mentioned, notes should be prepared (live or after the
> fact) and archived on Confluence alongside the agenda.
> 7. Decisions: Happen on the dev list, though discussion and
> consensus-building may occur during calls like this.
>
> Hosting / moderating calls like this takes planning and effort, but it
> seems very doable if others feel it would be valuable to our dev and user
> community. Happy to help if so.
>
> – Scott
>
> ---
>
> References:
> [1] Meeting doc:
> https://github.com/kubernetes/community/blob/master/events/community-meeting.md
> [2] Agenda:
> https://docs.google.com/document/d/1VQDIAB0OqiSjIHI8AWMvSdceWhnz56jNpZrLs6o7NJY/edit#heading=h.en8cy6hno0c6
> [3] K8s Zoom Guidelines:
> https://github.com/kubernetes/community/blob/master/communication/zoom-guidelines.md
> [4] K8s Moderation Guidelines:
> https://github.com/kubernetes/community/blob/master/communication/moderation.md
> [5] Example recorded meeting: https://www.youtube.com/watch?v=6uZScaWEb08
>
> On 8/9/19, 10:27 AM, "sankalp kohli"  wrote:
>
> @Dinesh/Nate: Yes we need to decide on the timing and we can always
> change
> them as we go
> @Joshua/Gary: We will publish notes on the mailing list. If we need to
> make
> a decision, we will still need to get it voted on the ML. We should not
> have a case where someone misses the boat because they could not
> attend one
> of these. So ML is a big part of this.
>
> Additional feedback welcome.
>
> On Fri, Aug 9, 2019 at 6:28 AM Gary Dusbabek 
> wrote:
>
> > Would publishing notes to the ML be sufficient? Apache board
> meetings work
> > this way.
> >
> > Gary.
> >
> > On Wed, Aug 7, 2019 at 4:51 PM Nate McCall 
> wrote:
> >
> > > We can do the time mostly fair if we alternate back and forth
> between PST
> > > morning and evening. This will at least let most folks attend
> every other
> > > meeting.
> > >
> > > I agree with Josh's sentiment on the discussions. We can do it, we
> just
> > > have to be aware of it and defer things to Jira and/or ML.
> > >
> > > On Thu, Aug 8, 2019 at 12:42 AM Joshua McKenzie <
> jmcken...@apache.org>
> > > wrote:
> > >
> > > > The one thing we need to keep in mind is the "If it didn't
> happen on a
> > > > mailing list, it didn't happen  >"
> > > > philosophy of apache projects. Shouldn't constrain us too much
> as the
> > > > nuance is:
> > > >
> > > > *"Discussions and plan proposals often happen at events, in chats
> > (Slack,
> > > > IRC, IM, etc.) or other synchronous places. But all final
> decisions
> > about
> > > > executing on the plan, checking in the new code, or launching the
> > website
> > > > must be made by the community asyncrhonously on the mailing
> list."*
> > > >
> > > > So long as we keep that in mind (and maybe push it back to 8am
> PST
> > since
> > > > 9am can get pretty ugly for some of the more eastern european /
> asian
> > > > countries), makes sense to me.
> > > >
> > > > On Tue, Aug 6, 2019 at 

Re: JIRA Workflow Proposals

[Murukesh Mohanan]

On Tue, 11 Dec 2018 at 10:51, Benedict Elliott Smith
 wrote:
>
> On 10 Dec 2018, at 16:21, Ariel Weisberg  wrote:
> >
> > Hi,
> >
> > RE #1, does this mean if you submit a ticket and you are not a contributor 
> > you can't modify any of the fields including description or adding/removing 
> > attachments?
>
> Attachment operations have their own permissions, like comments.  Description 
> would be prohibited though.  I don’t see this as a major problem, really; it 
> is generally much more useful to add comments.  If we particularly want to 
> make a subset of fields editable there is a workaround, though I’m not sure 
> anybody would have the patience to implement it:  
> https://confluence.atlassian.com/jira/how-can-i-control-the-editing-of-issue-fields-via-workflow-149834.html
>  
> 
>

That would be disappointing. Descriptions with broken or no formatting
aren't rare (say, command output without code formatting). And every
now and then the description might need to be updated. For example, in
https://issues.apache.org/jira/browse/CASSANDRA-10989, the link to the
paper had rotted, but I managed to find a new one, so I could edit it
in. If such a change had to be posted as a comment, it might easily
get lost in some of the more active issues.

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Implicit Casts for Arithmetic Operators

[Murukesh Mohanan]

I think you're conflating two things here. There's the loss resulting from
using some operators, and loss involved in casting. Dividing an integer by
another integer to obtain an integer result can result in loss, but there's
no implicit casting there and no loss due to casting.  Casting an integer
to a float can also result in loss. So dividing an integer by a float, for
example, with an implicit cast has an additional avenue for loss: the
implicit cast for the operands so that they're of the same type. I believe
this discussion so far has been about the latter, not the loss from the
operations themselves.

On Wed, 3 Oct 2018 at 18:35 Benjamin Lerer 
wrote:

> Hi,
>
> I would like to try to clarify things a bit to help people to understand
> the true complexity of the problem.
>
> The *float *and *double *types are inexact numeric types. Not only at the
> operation level.
>
> If you insert 676543.21 in a *float* column and then read it, you will
> realize that the value has been truncated to 676543.2.
>
> If you want accuracy the only way is to avoid those inexact types.
> Using *decimals
> *during operations will mitigate the problem but will not remove it.
>
>
> I do not recall PostgreSQL behaving has described. If I am not mistaken in
> PostgreSQL *SELECT 3/2* will return *1*. Which is similar to what MS SQL
> server and Oracle do. So all thoses databases will lose precision if you
> are not carefull.
>
> If you truly need precision you can have it by using exact numeric types
> for your data types. Of course it has a cost on performance, memory and
> disk usage.
>
> The advantage of the current approach is that it give you the choice. It is
> up to you to decide what you need for your application. It is also in line
> with the way CQL behave everywhere else.
>
-- 

Muru

Re: Reaper as cassandra-admin

[Murukesh Mohanan]

Is there a roadmap or release schedule, so we can get an idea of what
the Reaper devs have planned for it?


Yours,
Murukesh Mohanan

On Tue, 28 Aug 2018 at 10:02, Jeff Jirsa  wrote:
>
> Can you get all of the contributors cleared?
> What’s the architecture? Is it centralized? Is there a sidecar?
>
>
> > On Aug 27, 2018, at 5:36 PM, Jonathan Haddad  wrote:
> >
> > Hey folks,
> >
> > Mick brought this up in the sidecar thread, but I wanted to have a clear /
> > separate discussion about what we're thinking with regard to contributing
> > Reaper to the C* project.  In my mind, starting with Reaper is a great way
> > of having an admin right now, that we know works well at the kind of scale
> > we need.  We've worked with a lot of companies putting Reaper in prod (at
> > least 50), running on several hundred clusters.  The codebase has evolved
> > as a direct result of production usage, and we feel it would be great to
> > pair it with the 4.0 release.  There was a LOT of work done on the repair
> > logic to make things work across every supported version of Cassandra, with
> > a great deal of documentation as well.
> >
> > In case folks aren't aware, in addition to one off and scheduled repairs,
> > Reaper also does cluster wide snapshots, exposes thread pool stats, and
> > visualizes streaming (in trunk).
> >
> > We're hoping to get some feedback on our side if that's something people
> > are interested in.  We've gone back and forth privately on our own
> > preferences, hopes, dreams, etc, but I feel like a public discussion would
> > be healthy at this point.  Does anyone share the view of using Reaper as a
> > starting point?  What concerns to people have?
> > --
> > Jon Haddad
> > http://www.rustyrazorblade.com
> > twitter: rustyrazorblade
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Side Car New Repo vs not

[Murukesh Mohanan]

FWIW, I think it's possible to merge in a separate repository into a
subdirectory while keeping git history, but I don't know if the other way
will be possible if commits span other parts of the repo as well\* (which
will likely happen sooner or later). So a separate repo is a choice we can
backtrack from if it proves problematic later.


\* it may be possible, but the commit messages might not make much sense
after that.

On Fri, 24 Aug 2018, 09:12 Benedict Elliott Smith, 
wrote:

> +1 also for separate repo
>
> > On 24 Aug 2018, at 01:11, Jeff Jirsa  wrote:
> >
> > +1 for separate repo
> >
> >
> > --
> > Jeff Jirsa
> >
> >
> >> On Aug 23, 2018, at 1:00 PM, sankalp kohli 
> wrote:
> >>
> >> Separate repo is in a majority so far. Please reply to this thread with
> >> your responses.
> >>
> >> On Tue, Aug 21, 2018 at 4:34 PM Rahul Singh <
> rahul.xavier.si...@gmail.com>
> >> wrote:
> >>
> >>> +1 for separate repo. Especially on git. Maybe make it a submodule.
> >>>
> >>> Rahul
> >>> On Aug 21, 2018, 3:33 PM -0500, Stefan Podkowinski ,
> >>> wrote:
>  I'm also currently -1 on the in-tree option.
> 
>  Additionally to what Aleksey mentioned, I also don't see how we could
>  make this work with the current build and release process. Our scripts
>  [0] for creating releases (tarballs and native packages), would need
>  significant work to add support for an independent side-car. Our ant
>  based build process is also not a great start for adding new tasks,
> let
>  alone integrating other tool chains for web components for a potential
> >>> UI.
> 
>  [0] https://git-wip-us.apache.org/repos/asf?p=cassandra-builds.git
> 
> 
> > On 21.08.18 19:20, Aleksey Yeshchenko wrote:
> > Sure, allow me to elaborate - at least a little bit. But before I do,
> >>> just let me note that this wasn’t a veto -1, just a shorthand for “I
> don’t
> >>> like this option”.
> >
> > It would be nice to have sidecar and C* version and release cycles
> >>> fully decoupled. I know it *can* be done when in-tree, but the way we
> vote
> >>> on releases with tags off current branches would have to change
> somehow.
> >>> Probably painfully. It would be nice to be able to easily enforce
> freezes,
> >>> like the upcoming one, on the whole C* repo, while allowing feature
> >>> development on the sidecar. It would be nice to not have sidecar
> commits in
> >>> emails from commits@ mailing list. It would be nice to not have C* CI
> >>> trigger necessarily on sidecar commits. Groups of people working on
> the two
> >>> repos will mostly be different too, so what’s the point in sharing the
> repo?
> >
> > Having an extra repo with its own set of branches is cheap and easy -
> >>> we already do that with dtests. I like cleanly separated things when
> >>> coupling is avoidable. As such I would prefer the sidecar to live in a
> >>> separate new repo, while still being part of the C* project.
> >
> > —
> > AY
> >
> > On 21 August 2018 at 17:06:39, sankalp kohli (kohlisank...@gmail.com
> )
> >>> wrote:
> >
> > Hi Aleksey,
> > Can you please elaborate on the reasons for your -1? This
> > way we can make progress towards any one approach.
> > Thanks,
> > Sankalp
> >
> > On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko <
> alek...@apple.com>
> > wrote:
> >
> >> FWIW I’m strongly -1 on in-tree approach, and would much prefer a
> >>> separate
> >> repo, dtest-style.
> >>
> >> —
> >> AY
> >>
> >> On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> >> jeremiah.jor...@gmail.com) wrote:
> >>
> >> I think the following is a very big plus of it being in tree:
>  * Faster iteration speed in general. For example when we need to
> >>> add a
>  new
>  JMX endpoint that the sidecar needs, or change something from
> >>> JMX to a
>  virtual table (e.g. for repair, or monitoring) we can do all
> >>> changes
>  including tests as one commit within the main repository and
> >>> don't
> >> have
>  to
>  commit to main repo, sidecar repo,
> >>
> >> I also don’t see a reason why the sidecar being in tree means it
> >>> would not
> >> work in a mixed version cluster. The nodes themselves must work in a
> >>> mixed
> >> version cluster during a rolling upgrade, I would expect any
> >>> management
> >> side car to operate in the same manor, in tree or not.
> >>
> >> This tool will be pretty tightly coupled with the server, and as
> >>> someone
> >> with experience developing such tightly coupled tools, it is *much*
> >>> easier
> >> to make sure you don’t accidentally break them if they are in tree.
> >>> How
> >> many times has someone updated some JMX interface, updated nodetool,
> >>> and
> >> then moved on? Breaking all the external tools not in tree, without
> >> realizing it. The above point 

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Murukesh Mohanan]

If anything, riding on their coattails will mean we'll drop support a year
or two later, after we see their experiences - which will be nicely in line
with the Python upstream EOL date of 2020. (Of course, distro vendors will
provide support after that, say till 2025.) So perhaps 2020 should be a
reasonable deadline for Cassandra to drop support too? Assuming C* version
X EOLing in 2025 will be the last to have support for Python 2 is released
around 2020.
On Sat, Jun 2, 2018 at 2:01 Jonathan Haddad  wrote:

> And that's why I said supporting both with six is the right path
> forward, later dropping support for 2.  I'm not advocating we drop 2
> support now, and I'm not asking for any sort of commitment.  I didn't
> think adding support for 3 would be so controversial.
> On Fri, Jun 1, 2018 at 9:40 AM Jeremiah D Jordan
>  wrote:
> >
> > The community of people doing python development and the community of
> people running Cassandra servers are not the same.  I am not fine riding
> the coat tails of libraries used in python development.  As others have
> stated we need to be following the lead of the OS vendors that people will
> be deploying Cassandra on top of.  And those will not be dropping Python 2
> at the end of the year.
> >
> > -Jeremiah
> >
> > > On Jun 1, 2018, at 12:37 PM, Jonathan Haddad 
> wrote:
> > >
> > > Both can work.  I did a lot of the work on the port of the Python
> > > driver's object mapper (formerly cqlengine) to Python 3.  It's
> > > reasonably straightforward if you use the six library.
> > >
> > > Both pandas and numpy are dropping support for Python 2 at the end of
> > > this year.  I'm fine with riding on their coattails.
> > > On Fri, Jun 1, 2018 at 9:21 AM Russell Bateman 
> wrote:
> > >>
> > >> Support for, but not the very script, right? Because, as gently
> pointed
> > >> out by several realists here, Python 2 is far from dead and arguably
> > >> still the majority usage. That's only just now beginning to change. I
> > >> think it will be more than 2 years before people begin asking what
> > >> Python 2 was.
> > >>
> > >>
> > >> On 06/01/2018 10:10 AM, Jonathan Haddad wrote:
> > >>> Supporting both as a next step is logical, removing support for 2 in
> the
> > >>> next year or two seems reasonable enough. Gotta rip the band aid off
> at
> > >>> some point.
> > >>>
> > >>> On Fri, Jun 1, 2018 at 2:34 AM Michael Burman 
> wrote:
> > >>>
> > >>>> Hi,
> > >>>>
> > >>>> Deprecating in this context does not mean removing it or it being
> > >>>> replaced by 3 (RHEL 7.x will remain with Python 2.x as default). It
> > >>>> refers to future versions (>7), but there are none at this point. It
> > >>>> appears Ubuntu has deviated from Debian in this sense, but Debian
> has
> > >>>> not changed yet (likely Debian 10 will, but that's not out yet and
> has
> > >>>> no announced release date).
> > >>>>
> > >>>> Thus, 2.x still remains the most used version for servers. And
> servers
> > >>>> deployed at this point of time will use these versions for years.
> > >>>>
> > >>>>- Micke
> > >>>>
> > >>>>
> > >>>> On 06/01/2018 10:52 AM, Murukesh Mohanan wrote:
> > >>>>> On 2018/06/01 07:40:04, Michael Burman 
> wrote:
> > >>>>>> IIRC, there's no major distribution yet that defaults to Python 3
> (I
> > >>>>>> think Ubuntu & Debian are still defaulting to Python 2 also).
> This will
> > >>>>>> happen eventually (maybe), but not yet. Discarding Python 2
> support
> > >>>>>> would mean more base-OS work for most people wanting to run
> Cassandra
> > >>>>>> and that's not a positive thing.
> > >>>>>>
> > >>>>> Ubuntu since 16.04 defaults to Python 3:
> > >>>>>
> > >>>>>> Python2 is not installed anymore by default on the server, cloud
> and
> > >>>> the touch images, long live Python3! Python3 itself has been
> upgraded to
> > >>>> the 3.5 series. -
> > >>>>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.ubuntu.com_XenialXerus_ReleaseNotes-23Python-5F3=DwIBaQ=adz96Xi0w1RHqtPMowiL2g=CNZK3RiJDLqhsZDG6FQGnX

Re: Urgent help needed : Cassandra benchmarking with Cassandra Stress tool

[Murukesh Mohanan]

Attachments don't work well with mailing lists. You might want to post the
yaml to a GitHub gist or some other public pastebin site and provide the
link instead. Also, I think this topic is more suited to the user@ mailing
list rather then this development-focused mailing list.
On Sat, Jun 2, 2018 at 1:20 Vinay Vongour 
wrote:

> Hi all
>
>
>
> Hope you all doing good
>
> I am working on benchmarking Cassandra using Cassandra stress tool and
> results are not very impressive. I really some help to tune Cassandra to
> get great results. Below is my environment and test results and attached is
> Cassandra.yaml I am using for the test.
>
>
>
> I am running single node Cassandra virtual machine and 1 SSD disk in
> VMWare ESXi 6.7 eventually I want to scale the database to 4 fours once I
> get the expected results in the single node. Attached is the Cassandra.yaml
> file is used for configuring database.
>
>
>
> The result I am getting is not very impressive operations/sec is very low
> and the latency is very high for the reads here is the results it got from
> the test.
>
> Load command: Cassandra-stress write n=40 -rate threads=96 -node
> $node1 -log file=load_400M.log
>
>
>
> *Results:*
>
> Op rate   :2,347 op/s  [READ: 2,347 op/s]
>
> Partition rate:2,347 pk/s  [READ: 2,347 pk/s]
>
> Row rate  :2,347 row/s [READ: 2,347 row/s]
>
> Latency mean  :  381.1 ms [READ: 381.1 ms]
>
> Latency median:  327.4 ms [READ: 327.4 ms]
>
> Latency 95th percentile   :  836.2 ms [READ: 836.2 ms]
>
> Latency 99th percentile   : 1138.8 ms [READ: 1,138.8 ms]
>
> Latency 99.9th percentile : 1695.5 ms [READ: 1,695.5 ms]
>
> Latency max   : 6253.7 ms [READ: 6,253.7 ms]
>
> Total partitions  :  1,000,000 [READ: 1,000,000]
>
> Total errors  :  0 [READ: 0]
>
> Total GC count: 0
>
> Total GC memory   : 0.000 KiB
>
> Total GC time :0.0 seconds
>
> Avg GC time   :NaN ms
>
> StdDev GC time:0.0 ms
>
> Total operation time  : 00:07:06
>
>
>
> The write performance is really impressive with less latency and good
> operation/sec and when we compare it with reads I feel something is wrong.
>
> Read command: Cassandra-stress read n=1  no-warmup cl=one -mode
> native cql3 -schema keyspace=”keyspace1” -rate threads=96 -node $node1 -log
> file=read_10M_test.log
>
>
>
> Results:
>
> Op rate   :   35,979 op/s  [WRITE: 35,979 op/s]
>
> Partition rate:   35,979 pk/s  [WRITE: 35,979 pk/s]
>
> Row rate  :   35,979 row/s [WRITE: 35,979 row/s]
>
> Latency mean  :3.3 ms [WRITE: 3.3 ms]
>
> Latency median:2.2 ms [WRITE: 2.2 ms]
>
> Latency 95th percentile   :8.2 ms [WRITE: 8.2 ms]
>
> Latency 99th percentile   :   17.6 ms [WRITE: 17.6 ms]
>
> Latency 99.9th percentile :   75.6 ms [WRITE: 75.6 ms]
>
> Latency max   : 1840.3 ms [WRITE: 1,840.3 ms]
>
> Total partitions  : 1,600,000,000 [WRITE: 1,600,000,000]
>
> Total errors  :  0 [WRITE: 0]
>
> Total GC count: 0
>
> Total GC memory   : 0.000 KiB
>
> Total GC time :0.0 seconds
>
> Avg GC time   :NaN ms
>
> StdDev GC time:0.0 ms
>
> Total operation time  : 12:21:10
>
>
>
>
>
> When I compare the both read and write performance they are not at all
> consistent and I feel there is some kind of bottleneck with the reads
> during test.
>
>
>
> Could you please look into these details and let me know where I am
> missing I have been struggling hard to get good results out of Cassandra
> database but I couldn’t figure out where I am missing.
>
>
>
> Please let me know if you need additional information about the Cassandra
> setup.
>
>
>
>
>
> Thanks and Regards
>
> Vinay Kumar Vongour
>
>
>
>
>
>
>
> Thanks and Regards
>
> Vinay Kumar Vongour
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org

-- 

Muru

Re: Planning to port cqlsh to Python 3 (CASSANDRA-10190)

[Murukesh Mohanan]

On 2018/06/01 07:40:04, Michael Burman  wrote: 
> 
> IIRC, there's no major distribution yet that defaults to Python 3 (I 
> think Ubuntu & Debian are still defaulting to Python 2 also). This will 
> happen eventually (maybe), but not yet. Discarding Python 2 support 
> would mean more base-OS work for most people wanting to run Cassandra 
> and that's not a positive thing.
> 

Ubuntu since 16.04 defaults to Python 3:

> Python2 is not installed anymore by default on the server, cloud and the 
> touch images, long live Python3! Python3 itself has been upgraded to the 3.5 
> series. - https://wiki.ubuntu.com/XenialXerus/ReleaseNotes#Python_3  

RHEL 7.5 deprecates Python 2 
(https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.5_release_notes/chap-red_hat_enterprise_linux-7.5_release_notes-deprecated_functionality).



-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: question on running cassandra-dtests

[Murukesh Mohanan]

On Tue, Mar 27, 2018 at 6:47 Ariel Weisberg <ar...@weisberg.ws> wrote:

> Hi,
>
> Are you deleting the venv before creating it? You shouldn't really need to
> use sudo for the virtualenv. That is going to make things potentially
> wonky. Naming it cassandra-dtest might also do something wonky if you have
> a cassandra-dtest directory already. I usually name it just venv and place
> it in the same subdir as the requirements file.
>
> Also running sudo is going to create a new shell and then exit the shell
> immediately so when you install the requirements it might be doing it not
> in the venv, but in whatever is going on inside the sudo shell.


Yep, looking at the logs, that's probably the issue. When activating a venv
(with `source .../bin/activate`), it sets environment variables (`PATH`,
`PYTHONHOME` etc.) so that the virtualenv's Python, pip are used instead of
the system Python and pip. sudo defaults to using a clean PATH and
resetting most of the user's environment, so the effects of the venv are
lost when running in sudo.


The advantage of virtualenv is not needing to mess with system packages at
> all so sudo is inadvisable when creating, activating, and pip installing
> things.
>
> You might need to use pip3 instead of pip, but I suspect that in a correct
> venv pip is going to point to pip3.
>
> Ariel
>
> On Mon, Mar 26, 2018, at 5:31 PM, Tyagi, Preetika wrote:
> > Yes, that's correct. I followed README and ran all below steps to create
> > virtualenv. Attached is the output of all commands I ran successfully
> > except the last one i.e. pytest.
> >
> > Could you please let me know if you see anything wrong or missing?
> >
> > Thanks,
> > Preetika
> >
> > -Original Message-
> > From: Ariel Weisberg [mailto:ar...@weisberg.ws]
> > Sent: Monday, March 26, 2018 9:32 AM
> > To: dev@cassandra.apache.org
> > Subject: Re: question on running cassandra-dtests
> >
> > Hi,
> >
> > Your environment is python 2.7 when it should be python 3.
> > See:
> > >   File "/usr/local/lib/python2.7/dist-packages/_pytest/assertion/
> > > rewrite.py", line 213, in load_module
> >
> > Are you using virtualenv to create a python 3 environment to use with
> the tests?
> >
> > From README.md:
> >
> > **Note**: While virtualenv isn't strictly required, using virtualenv is
> > almost always the quickest path to success as it provides common base
> > setup across various configurations.
> >
> > 1. Install virtualenv: ``pip install virtualenv`` 2. Create a new
> > virtualenv: ``virtualenv --python=python3 --no-site-packages ~/dtest``
> > 3. Switch/Activate the new virtualenv: ``source ~/dtest/bin/activate``
> > 4. Install remaining DTest Python dependencies: ``pip install -r /path/
> > to/cassandra-dtest/requirements.txt``
> >
> > Regards,
> > Ariel
> >
> > On Mon, Mar 26, 2018, at 11:13 AM, Tyagi, Preetika wrote:
> > > I was able to run requirements.txt with success. Below is the error I
> get:
> > >
> > > Traceback (most recent call last):
> > >   File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py",
> > > line 371, in _importconftest
> > > mod = conftestpath.pyimport()
> > >   File "/usr/local/lib/python2.7/dist-packages/py/_path/local.py",
> > > line 668, in pyimport
> > > __import__(modname)
> > >   File "/usr/local/lib/python2.7/dist-packages/_pytest/assertion/
> > > rewrite.py", line 213, in load_module
> > > py.builtin.exec_(co, mod.__dict__)
> > >   File "/usr/local/lib/python2.7/dist-packages/py/_builtin.py", line
> > > 221, in exec_
> > > exec2(obj, globals, locals)
> > >   File "", line 7, in exec2
> > >   File "/home//conftest.py", line 11, in 
> > > from itertools import zip_longest
> > > ImportError: cannot import name zip_longest
> > > ERROR: could not load /home//conftest.py
> > >
> > > Thanks,
> > > Preetika
> > >
> > > -Original Message-
> > > From: Murukesh Mohanan [mailto:murukesh.moha...@gmail.com]
> > > Sent: Sunday, March 25, 2018 10:48 PM
> > > To: dev@cassandra.apache.org
> > > Subject: Re: question on running cassandra-dtests
> > >
> > > The complete error is needed. I get something similar if I hadn't run
> > > `pip3 install -r requirements.txt`:
> > >
> > > Traceback (most recent call last):
> > >   File "/usr/local/lib/python3.6/site-packages/_pytest/c

Re: question on running cassandra-dtests

[Murukesh Mohanan]

The complete error is needed. I get something similar if I hadn't run `pip3 
install -r requirements.txt`:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/_pytest/config.py", line 328, in 
_getconftestmodules
return self._path2confmods[path]
KeyError: local('/home/muru/dev/cassandra-dtest')

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/_pytest/config.py", line 359, in 
_importconftest
return self._conftestpath2mod[conftestpath]
KeyError: local('/home/muru/dev/cassandra-dtest/conftest.py')

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/_pytest/config.py", line 365, in 
_importconftest
mod = conftestpath.pyimport()
  File "/usr/local/lib/python3.6/site-packages/py/_path/local.py", line 668, in 
pyimport
__import__(modname)
  File "/usr/local/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", 
line 212, in load_module
py.builtin.exec_(co, mod.__dict__)
  File "/home/muru/dev/cassandra-dtest/conftest.py", line 13, in 
from dtest import running_in_docker, 
cleanup_docker_environment_before_test_execution
  File "/home/muru/dev/cassandra-dtest/dtest.py", line 12, in 
import cassandra
ModuleNotFoundError: No module named 'cassandra'
ERROR: could not load /home/muru/dev/cassandra-dtest/conftest.py

Of course, `pip3 install -r requirements.txt` creates an `src` directory with 
appropriate branches of ccm and cassandra-driver checked out.

If you have run `pip3 install -r requirements.txt`, then something else is 
wrong and we need the complete error log.

On 2018/03/23 20:22:47, "Tyagi, Preetika"  wrote: 
> Hi All,
> 
> I am trying to setup and run Cassandra-dtests so that I can write some tests 
> for a JIRA ticket I have been working on.
> This is the repo I am using: https://github.com/apache/cassandra-dtest
> I followed all the instructions and installed dependencies.
> 
> However, when I run "pytest -cassandra-dir= directory>
> 
> It throws the error "could not load /conftest.py.
> 
> I checked that this file (conftest.py) exists in Cassandra-dtest source root 
> and I'm not sure why it cannot find it. Does anyone have any idea what might 
> be going wrong here?
> 
> I haven't used dtests before so I wonder if I'm missing something here.
> 
> Thanks,
> Preetika
> 
> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: A JIRA proposing a seperate repository for the online documentation

[Murukesh Mohanan]

a PMC (Project Management
> > > > Committee); and that in the new repository . . .
> > > >
> > > >
> > > >
> > > > Please see the jira. I hope it's a good answer to everyone.
>
> --
> Eric Evans
> john.eric.ev...@gmail.com
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
> --

Murukesh Mohanan,
Yahoo! Japan

Re: A JIRA proposing a seperate repository for the online documentation

[Murukesh Mohanan]

I think this was how it was in the dark ages, with the wiki and all. I
believe the reason why they shifted to in-tree docs is that this way,
people who make changes to the code are more likely to make the
corresponding doc changes as well, and reviewers have it easier to ensure
docs are updated with new patches. The wiki was often left behind the code.
So I they will move back to an out-of-tree doc system again. The way
Michael put it in CASSANDRA-13907, the main blocker behind having docs for
multiple versions online is that it's a pain just to get the docs for trunk
updated. Once the current site update process is improved, multiple
versions can more easily be added.


On Thu, Mar 15, 2018 at 1:22 Kenneth Brotman <kenbrot...@yahoo.com.invalid>
wrote:

> https://issues.apache.org/jira/browse/CASSANDRA-14313
>
>
>
> For some reason I'm told by many committers that we should not have sets of
> documentation for other versions than the current version in a tree for
> that
> version.  This has made it difficult, maybe impossible to have
> documentation
> for all the supported versions on the website at one time.
>
>
>
> As a solution I propose that we maintain the online documentation in a
> separate repository that is managed as the current repository under the
> guidance of the Apache Cassandra PMC (Project Management Committee); and
> that in the new repository . . .
>
>
>
> Please see the jira.  I hope it's a good answer to everyone.
>
>
>
> KennethBrotman
>
>
>
>
>
> --

Murukesh Mohanan,
Yahoo! Japan

Re: Can API be an alternative for MBeans?

[Murukesh Mohanan]

You might want to keep an eye on
https://issues.apache.org/jira/browse/CASSANDRA-7622 (I suspect you might
already be doing so, but just in case...)

On Thu, 22 Feb 2018 at 17:57 Jacques-Henri Berthemet <
jacques-henri.berthe...@genesys.com> wrote:

> Hi,
>
> What would be great would be to be able to query stats using CQL on some
> "virtual" systems tables. Exposing a REST API would be another endpoint to
> secure.
>
> --
> Jacques-Henri Berthemet
>
> -Original Message-
> From: Nicolas Guyomar [mailto:nicolas.guyo...@gmail.com]
> Sent: Thursday, February 22, 2018 9:51 AM
> To: dev@cassandra.apache.org
> Subject: Re: Can API be an alternative for MBeans?
>
> Hi,
>
> Jolokia 'for instance) is making exposing MBean with Http so easy (with
> "just"  a simple jar addition) that I think this is not really needed
> within Cassandra
>
>
> On 22 February 2018 at 09:10, Venkata Hari Krishna Nukala <
> n.v.harikrishna.apa...@gmail.com> wrote:
>
> > Hi,
> >
> > I saw lots of information exposed through MBeans (like status, cfstats
> > etc...). I feel exposing them like as API has few advantages like, it
> > is more open (different types of clients can use) and more expressible
> > for request and response.  Does the option of exposing such
> > functionality through  API (REST) was considered at any point of time?
> > Are there any advantages or compelling reasons to stick with MBeans?
> >
> > Thanks!
> >
>
-- 

Murukesh Mohanan,
Yahoo! Japan

Re: Possibly cassandra 3.0.9 bug?

[Murukesh Mohanan]

Hi Alex,

It's still not visible... I don't think the mailing list supports image
attachments. Maybe you can create an issue on JIRA with the attachments?

Thanks,
Muru
On Thu, 16 Nov 2017 at 17:00 Alex Circus <circus.alexan...@gmail.com> wrote:

> Hi Pavel,
>
> I'm attaching it again. I use gmail app from browser. Please check now.
>
> Thanks,
> Alex.
>
> On Thu, Nov 16, 2017 at 9:17 AM, Pavel Drankov <titant...@gmail.com>
> wrote:
>
>> Hi Alex,
>>
>> I don't see any attached image. Can you please send it one more time?
>>
>> Best wishes,
>> Pavel
>>
>> On 16 November 2017 at 01:04, Alex Circus <circus.alexan...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > *On short:*
>> > I use cassandra 3.0.9 in a cluster of 6 nodes.
>> > 1. I create a keyspace called test:
>> > CREATE KEYSPACE business WITH replication = {'class':
>> > 'SimpleStrategy', 'replication_factor': '3'}  AND durable_writes = true;
>> > 2. I create table called test:
>> >
>> > CREATE TABLE test.test (
>> >
>> > test_id bigint,
>> >
>> > test_value text
>> >
>> > PRIMARY KEY (test_id)
>> >
>> > )
>> >
>> > 3. I insert test_id=23 and test_value=some very large string/html (like
>> > 406088 chars utf8).
>> >
>> > 4. I query for test_id=35 and I get timeout (even with clqsh
>> > --request-timeout=3600)...
>> >
>> > 5. If I run the above on an existing cassandra cluster with cassa 2.0
>> the
>> > select returns instantlyThe Java heap size is 8GB and in JMX I see
>> max
>> > 4GB used of these 8 GB in the new cluster
>> >
>> >
>> > *Detailed:*
>> >
>> > The above was just a test. The real scenario is:
>> >
>> > I migrated some tables from an old cassa (2.0) cluster with 9 nodes into
>> > another with 6 nodes and with cassa 3.0.9 and there was a lot of
>> > problems
>> >
>> > I have a table like this:
>> >
>> > CREATE TABLE table (
>> >   id text,
>> >   ts text,
>> >   score decimal,
>> >   type text,
>> >   values text,
>> >   PRIMARY KEY (id, ts)
>> > ) WITH CLUSTERING ORDER BY (ts DESC)
>> >
>> > and the following query (which returns instantly):
>> >
>> > SELECT * FROM keyspace.table WHERE id='someId' AND ts IN
>> ('2017-10-15','2017-10-16','2017-10-17','2017-10-18','2017-10-19','2017-10-20','2017-10-21','2017-10-22','2017-10-23','2017-10-24','2017-10-25','2017-10-26','2017-10-27','2017-10-28','2017-10-29','2017-10-30','2017-10-31','2017-11-01','2017-11-02','2017-11-03','2017-11-04','2017-11-05','2017-11-06');
>> >
>> > *If I add another day in the IN clause, the response never comes (even
>> > after 10 minutes!!!):*
>> >
>> > SELECT * FROM keyspace.table WHERE id='someId' AND ts IN
>> > ('2017-10-15','2017-10-16','2017-10-17','2017-10-18','
>> > 2017-10-19','2017-10-20','2017-10-21','2017-10-22','
>> > 2017-10-23','2017-10-24','2017-10-25','2017-10-26','
>> > 2017-10-27','2017-10-28','2017-10-29','2017-10-30','
>> > 2017-10-31','2017-11-01','2017-11-02','2017-11-03','
>> > 2017-11-04','2017-11-05','2017-11-06', *'2017-11-07'*);
>> >
>> > *The 'values' column may have large json data. *
>> >
>> > I managed to trace one of the timeouts by looking into system_trace
>> > keyspace. Please look into the attached image and see the last process
>> took
>> > 10 minutes!!!
>> >
>> > I think there is some size limit somewhere because in* the IN clause *if
>> > I have 23 params it works(under 1 second), but with more(1+) it fails.
>> The
>> > rows are the same size (same json size on all). In node2 of those 6 it
>> > works with 24 params. In node1 and node3 no. The other nodes I haven't
>> > checked yet.
>> >
>> > I saw no concluding logs except this one from cassa's debug.log (in the
>> > moment of the timeout or very close to that):
>> >
>> > *DEBUG [Thrift:2608] 2017-11-15 13:48:05,611 ReadCallback.java:126 -
>> Timed
>> > out; received 0 of 1 responses*
>> >
>> > I think this problem has the same root cause as the one from the test
>> > (large html text) and it is related to some memory limit by code
>> somewhere.
>> >
>> >
>> > Thank you,
>> >
>> > Alex.
>> > [image: screenshot.png]
>> >
>> >
>>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org

-- 

Murukesh Mohanan,
Yahoo! Japan

Are the NGCC 2017 videos published somewhere?

[Murukesh Mohanan]

I think videos were recorded of the NGCC 2017 sessions. Are those published 
somewhere, and if not, will they be?

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Cassandra DTests deadlocks on random test

[Murukesh Mohanan]

I have had this problem too on similarly spec'd OpenStack instances (but
I'm reasonably certain I didn't include the resource-intensive tests). My
solution was to run the dtests in small batches (say 5-10 each), with a
timeout (say 1.2x the max for 5 tests from a good run). Kill the test if
exceeds the timeout, that way you lose only 5-10 test results.

Another thing I did was to run the tests in docker, killing the instance
itself on timeout and ensuring no orphan processes. This also allows you to
run two or more test sets in parallel (which is not otherwise possible due
to the use of CCM).

On Thu, Oct 19, 2017, 22:58 Michael Shuler <mich...@pbandjelly.org> wrote:

> 7.6G RAM may be a little bit too small, we've seen similar random hangs
> in the past on non-resource-intensive tests on m3.large. It doesn't
> appear you are skipping resource-intensive tests. Our standard dtest
> instance type has been an m3.xlarge, and the resource-intensive tests
> are run m3.2xlarge. (or something comparable on RAM & SSD)
>
> dtest run command (excludes resource-intensive tests):
>
> https://github.com/apache/cassandra-builds/blob/master/build-scripts/cassandra-dtest.sh#L53
>
> dtest-large run command (only resource-intensive tests):
>
> https://github.com/apache/cassandra-builds/blob/master/build-scripts/cassandra-dtest.sh#L61
>
> Running dtest without excluding resource-intensive will run everything.
>
> If you wish to troubleshoot your tests when they hang, there should be
> /tmp/dtest-X directories with the ccm cluster left on disk from the
> hung test, since they never get to cleanup stage.
>
> --
> Michael
>
> On 10/19/2017 05:29 AM, Sergey La wrote:
> > Hi!
> >
> > I have created the patch for the Cassandra version 3.0.14 and trying to
> > test it using the cassandra dtests.
> >
> > Problem is - dtests deadlocks at some random tests, time and again - on
> > unpatched  3.0.14 version of Cassandra.
> >
> > What I have done.
> >
> > I cloned the cassandra repository (origin
> > http://git-wip-us.apache.org/repos/asf/cassandra.git), and checked out
> to
> > tags/cassandra-3.0.14 - head is on
> f3e38cb638113c2a23855a104d6082da5bc10ddb.
> >
> > Then I have cloned the cassandra-dtest repo (origin  git://
> > github.com/riptano/cassandra-dtest.git). Head is on
> > 6843d76d0a85ad82edf889e8280b87786dc48486.
> >
> > I setup dtests according to this instructions:
> > https://github.com/riptano/cassandra-dtest/blob/master/INSTALL.md
> >
> > In addition, I have setup JAVA8_HOME and JAVA_HOME variables to public
> jre
> > of my 1.8.0_144 jdk.
> >
> > I start testing using this command:
> > JAVA8_HOME=$JAVA8_HOME nosetests --with-flaky  --with-xunit
> > --xunit-file=out.xunit.xml  --force-flaky --max-runs=3 --verbose
> > --debug-log=err.debug.nose.txt  1> out.txt 2> err.txt
> > I run the tests on x86_64 CentOS 7 with 7.6G of RAM.
> >
> > Problem symptoms:
> > During the "normal" run (I have got only 1 "normal" run in 5 attempts),
> > err.txt is updated constantly with name of the test recently completed,
> and
> > in the end out.xunit.xml file appears, with test summary results.
> nosetests
> > process exits.
> >
> > During the "problem" run tests stop progressing (err.txt was not modified
> > for 10 hours), out.xunit.xml is not appearing, nosetests process runs. I
> > killed java processes, but nothing changed for 2 hours - nosetests
> process
> > still runs, but files are unchanged.
> >
> > Any help would be appreciated,
> > Sergey
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
> --

Murukesh Mohanan,
Yahoo! Japan

Re: question on the code formatter

[Murukesh Mohanan]

The wiki seems to be outdated. See
https://github.com/apache/cassandra/blob/trunk/doc/source/development/ide.rst
:


> The project generated by the ant task ``generate-idea-files`` contains
nearly everything
> you need to debug Cassandra and execute unit tests.
>
> * Run/debug defaults for JUnit
> * Run/debug configuration for Cassandra daemon
> * License header for Java source files
> * Cassandra code style
> * Inspections

You can just run `generate-idea-files` and then open the project in IDEA.
Code style settings should be automatically picked up by IDEA.

On Fri, 15 Sep 2017 at 14:46 Tyagi, Preetika <preetika.ty...@intel.com>
wrote:

> Hi all,
>
> I was trying to configure the Cassandra code formatter and downloaded
> IntelliJ-codestyle.jar from this link:
> https://wiki.apache.org/cassandra/CodeStyle
>
> After extracting this JAR, I was able to import codestyle/Default_1_.xml
> into my project and formatting seemed to work.
>
> However, I'm wondering what options/code.style.schemes.xml file is exactly
> used for? Could anyone please give me an idea if I need to configure this
> as well?
>
> Thanks,
> Preetika
>
>
> --

Murukesh Mohanan,
Yahoo! Japan

Re: Porting cqlsh to Python 3

[Murukesh Mohanan]

Related: https://issues.apache.org/jira/browse/CASSANDRA-10190

On 2017-07-20 18:17 (+0900), Tomas Repik  wrote: 
> Hi,
> 
> the clock for Python 2.7 is ticking [1], and yes, there are still more than 
> two years, but sooner or later cqlsh should be ported to Python 3. Is anybody 
> already working on it or just considers to work on it? What is the long time 
> plan for cqlsh? Should it be in Python forever or is the another plan? What 
> was the original reason for the client for Cassandra being written in python? 
> 
> Too many questions? Sorry for that, I'm just curious. My main idea was to get 
> involved in the discussion about cqlsh and python 3. And if there is not one 
> already I would like to start it.
> 
> Tomas
> 
> [1] https://pythonclock.org/
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Three scripts needed to run the server, Why?

[Murukesh Mohanan]

Also, you can use bash to debug bin/cassandra:

PS4=' $BASH_SOURCE:$LINENO:   ' bash -x bin/cassandra

This should print the filename of the file being executed/sourced and the line 
number being currently executed, so it should be easier to find out what 
happened, where and when. Of course, /bin/sh need not be bash, but I'm not sure 
what the equivalent method would be for dash or other shells.

On 2017-07-12 00:15 (+0900), "Murukesh Mohanan"<murukesh.moha...@gmail.com> 
wrote: 
> What you complain about may be useful to someone else who might appreciate 
> the added flexibility. I'd personally be opposed to a single script, as I'd 
> rather not edit something that might cause conflicts or be overwritten on 
> upgrades (the location of the include and environment files being 
> configurable mean that they can be in an entirely different corner of the 
> filesystem).
> 
> I can also think of cases where having two configurable files is useful. For 
> example, as an administrator, I'd keep everything in the cassandra install 
> directory read-only except for upgrades, then keep a common include file for 
> my users with some common configuration for my server, and let the users use  
> `$CASSANDRA_CONF` (the directory where the environment file is) to configure 
> everything else they wish for running their instances of Cassandra taking 
> advantage of the common install and base setup. Admittedly this isn't a 
> common use case.
> 
> If you're modifying bin/cassandra, then you're doing it wrong, IMHO. Only two 
> files need to be examined: the (an?) included file and the environment file. 
> And if you simply need to override a setting, then, you can just use the 
> environment file as the ultimate override, since it is sourced after the 
> include (not by it).
> 
> On 2017-07-11 23:39 (+0900), Tomas Repik <tre...@redhat.com> wrote: 
> > Thanks for the answer, it did not help much. I have read this several times 
> > and this I already know, It still does not answer the question, why there 
> > is the need for three files instead of a single file. Not to mention 
> > multiple different config files.
> > All these files are more or less configuration file which set up the 
> > environment and properties of the server. Why can't there be a single file 
> > that one would modify in order to tweak the server to his or her needs. In 
> > the current situation you have to search many different files to find the 
> > place where the option is configured.
> > 
> > - Original Message -
> > > 
> > > The bin/cassandra script has an explanation
> > > (https://github.com/apache/cassandra/blob/trunk/bin/cassandra#L24):
> > > 
> > > # As a convenience, a fragment of shell is sourced in order to set one or
> > > # more of these variables. This so-called `include' can be placed in a
> > > # number of locations and will be searched for in order. The lowest
> > > # priority search path is the same directory as the startup script, and
> > > # since this is the location of the sample in the project tree, it should
> > > # almost work Out Of The Box.
> > > #
> > > # Any serious use-case though will likely require customization of the
> > > # include. For production installations, it is recommended that you copy
> > > # the sample to one of /usr/share/cassandra/cassandra.in.sh,
> > > # /usr/local/share/cassandra/cassandra.in.sh, or
> > > # /opt/cassandra/cassandra.in.sh and make your modifications there.
> > > #
> > > #[...]
> > > #
> > > # If you would rather configure startup entirely from the environment, you
> > > # can disable the include by exporting an empty CASSANDRA_INCLUDE, or by
> > > # ensuring that no include files exist in the aforementioned search list.
> > > # Be aware that you will be entirely responsible for populating the needed
> > > # environment variables.
> > > 
> > > You can use just a single environment file, if you so wish.
> > > 
> > 
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > 
> > 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Three scripts needed to run the server, Why?

[Murukesh Mohanan]

What you complain about may be useful to someone else who might appreciate the 
added flexibility. I'd personally be opposed to a single script, as I'd rather 
not edit something that might cause conflicts or be overwritten on upgrades 
(the location of the include and environment files being configurable mean that 
they can be in an entirely different corner of the filesystem).

I can also think of cases where having two configurable files is useful. For 
example, as an administrator, I'd keep everything in the cassandra install 
directory read-only except for upgrades, then keep a common include file for my 
users with some common configuration for my server, and let the users use  
`$CASSANDRA_CONF` (the directory where the environment file is) to configure 
everything else they wish for running their instances of Cassandra taking 
advantage of the common install and base setup. Admittedly this isn't a common 
use case.

If you're modifying bin/cassandra, then you're doing it wrong, IMHO. Only two 
files need to be examined: the (an?) included file and the environment file. 
And if you simply need to override a setting, then, you can just use the 
environment file as the ultimate override, since it is sourced after the 
include (not by it).

On 2017-07-11 23:39 (+0900), Tomas Repik  wrote: 
> Thanks for the answer, it did not help much. I have read this several times 
> and this I already know, It still does not answer the question, why there is 
> the need for three files instead of a single file. Not to mention multiple 
> different config files.
> All these files are more or less configuration file which set up the 
> environment and properties of the server. Why can't there be a single file 
> that one would modify in order to tweak the server to his or her needs. In 
> the current situation you have to search many different files to find the 
> place where the option is configured.
> 
> - Original Message -
> > 
> > The bin/cassandra script has an explanation
> > (https://github.com/apache/cassandra/blob/trunk/bin/cassandra#L24):
> > 
> > # As a convenience, a fragment of shell is sourced in order to set one or
> > # more of these variables. This so-called `include' can be placed in a
> > # number of locations and will be searched for in order. The lowest
> > # priority search path is the same directory as the startup script, and
> > # since this is the location of the sample in the project tree, it should
> > # almost work Out Of The Box.
> > #
> > # Any serious use-case though will likely require customization of the
> > # include. For production installations, it is recommended that you copy
> > # the sample to one of /usr/share/cassandra/cassandra.in.sh,
> > # /usr/local/share/cassandra/cassandra.in.sh, or
> > # /opt/cassandra/cassandra.in.sh and make your modifications there.
> > #
> > #[...]
> > #
> > # If you would rather configure startup entirely from the environment, you
> > # can disable the include by exporting an empty CASSANDRA_INCLUDE, or by
> > # ensuring that no include files exist in the aforementioned search list.
> > # Be aware that you will be entirely responsible for populating the needed
> > # environment variables.
> > 
> > You can use just a single environment file, if you so wish.
> > 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Three scripts needed to run the server, Why?

[Murukesh Mohanan]

The bin/cassandra script has an explanation 
(https://github.com/apache/cassandra/blob/trunk/bin/cassandra#L24):

# As a convenience, a fragment of shell is sourced in order to set one or
# more of these variables. This so-called `include' can be placed in a 
# number of locations and will be searched for in order. The lowest 
# priority search path is the same directory as the startup script, and
# since this is the location of the sample in the project tree, it should
# almost work Out Of The Box.
#
# Any serious use-case though will likely require customization of the
# include. For production installations, it is recommended that you copy
# the sample to one of /usr/share/cassandra/cassandra.in.sh,
# /usr/local/share/cassandra/cassandra.in.sh, or 
# /opt/cassandra/cassandra.in.sh and make your modifications there.
#
#[...]
# 
# If you would rather configure startup entirely from the environment, you
# can disable the include by exporting an empty CASSANDRA_INCLUDE, or by 
# ensuring that no include files exist in the aforementioned search list.
# Be aware that you will be entirely responsible for populating the needed
# environment variables.

You can use just a single environment file, if you so wish.

On 2017-07-11 23:08 (+0900), Tomas Repik  wrote: 
> Greetings,
> 
> I've been working with Cassandra for more than a year but I still wonder 
> about one thing:
> 
> To run the server there is a bash script (cassandra) which uses another 
> script (cassandra.in.sh) which uses yet another bash script 
> (cassandra-env.sh).
> What is the reason behind this?
> Why there is not only a single file setting up the environment and running 
> the server? 
> 
> Thanks for your answers
> 
> Tomas
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: All clear to import dtest

[Murukesh Mohanan]

Will the Github repo continue to get updates (perhaps as a mirror of the
ASF one)? Just checking to see if any changes need to be made in the near
future for an in-house CI pipeline running dtests.

On Fri, 16 Jun 2017 at 09:39 Nate McCall <zznat...@gmail.com> wrote:

> We got through all the legal shenanigans necessary to import dtest
> into the project:
> https://issues.apache.org/jira/browse/CASSANDRA-13584
>
> This is just a heads up that I have created CASSANDRA-13613 for
> tracking the actual import and will be adding some sub-tasks next week
> for the actual migration into ASF infra and housekeeping around such.
>
> To sum up previous mail list conversations [0], we will be creating a
> new "cassandra-dtest" git repo at the ASF with the same committer list
> as the project.
>
> Anybody have any remaining issues or concerns here?
>
> Thanks,
> -Nate
>
> [0]
> https://lists.apache.org/thread.html/840fd900fb7f6568bfa008d122d4375b708c1f7f1b5929018118d5d5@%3Cdev.cassandra.apache.org%3E
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
> --

Murukesh Mohanan,
Yahoo! Japan

Splitting up CHANGES

[Murukesh Mohanan]

Python recently made a switch to Github, and I was an interested onlooker. One 
thing I found relevant to Cassandra was a new tool added by Larry Hastings, 
called blurb (https://github.com/larryhastings/blurb):

> blurb is a tool designed to rid CPython core development of the scourge of 
> Misc/NEWS conflicts.
> 
> The core concept: split Misc/NEWS into many separate files that, when 
> concatenated back together in 
> sorted order, reconstitute the original Misc/NEWS file. After that, Misc/NEWS 
> could be deleted from the
> CPython repo and thereafter rendered on demand (e.g. when building a 
> release). When checking in a ?
> change to CPython, the checkin process will write out a new file that sorts 
> into the correct place, using 
> a filename unlikely to have a merge conflict.

The CHANGES file strikes me as being a similar pain point. I think we could do 
with something like blurb for Cassandra. Blurb is, at the moment, very 
Python-centric, but I think it could be made more extensible and generic. 
Thoughts?

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Can we kill the wiki?

[Murukesh Mohanan]

I wonder if the recent influx has anything to do with GSoC. The student
application period begins in a few days. I don't see any Cassandra issues
on the GSoC ideas list, though.

On Sat, 18 Mar 2017 at 10:40 Anthony Grasso <anthony.gra...@gmail.com>
wrote:

+1 to killing the wiki as well. If that is not possible, we should at least
put a note on there saying it is deprecated and point people to the new
docs.

On 18 March 2017 at 08:09, Jonathan Haddad <j...@jonhaddad.com> wrote:

> +1 to killing the wiki.
>
> On Fri, Mar 17, 2017 at 2:08 PM Blake Eggleston <beggles...@apple.com>
> wrote:
>
> > With CASSANDRA-8700, docs were moved in tree, with the intention that
> they
> > would replace the wiki. However, it looks like we’re still getting
> regular
> > requests to edit the wiki. It seems like we should be directing these
> folks
> > to the in tree docs and either disabling edits for the wiki, or just
> > removing it entirely, and replacing it with a link to the hosted docs.
> I'd
> > prefer we just remove it myself, makes things less confusing for
> newcomers.
> >
> > Does that seem reasonable to everyone?
>

-- 

Murukesh Mohanan,
Yahoo! Japan

Re: What would a pluggable logging implementation look like?

[Murukesh Mohanan]

Thanks! Based on this and a suggestion by Jon, I'm working on a pluggable 
query-logging implementation. Can somebody have a quick look at the last patch 
submitted for CASSANDRA-13001 and tell me if I'm heading in the right direction?

Quoting my comment for that file to summarize:

- a new interface o.a.c.db.monitoring.IQueryLogger, with one function void 
logQueries(MonitoringTask.AggregatedOperations operations, 
MonitoringTask.OperationType type, long elapsed), accordingly, various elements 
in MonitoringTask have been made public
- Two classes implementing it, o.a.c.d.m.QueryDebugLogger and 
{{o.a.c.d.m.QueryTableLogger. The former implements the current debug.log 
behaviour and is default, and the latter saves stats and metadata to a table 
(hard-coded to perf.slow_log, looking for better ideas on this).
- New configuration setting slow_query_logger, works like authenticator.
- Rolled up both timed out and slow operation logging in MonitoringTask to use 
the interface. Accordingly cleaned up MonitoringTask. Perhaps the configuration 
setting should be renamed, again looking for ideas.

On 2017-03-01 19:42 (+0900), Romain Hardouin <romainh...@yahoo.fr.INVALID> 
wrote: 
> Hi,
> I think you have to look at how authenticator/authorizer/role_manager are 
> handled.e.g. 
> https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L103https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/auth/AllowAllAuthenticator.java
> Best,Romain 
> Le Mercredi 1 mars 2017 6h19, Murukesh Mohanan 
> <murukesh.moha...@gmail.com> a écrit :
>  
>  I'm looking at CASSANDRA-13001 (pluggable slow query logging / handling). I 
> wrote a hacky patch, where my main goal was to touch as few files as possible 
> - so I did what I could within MonitoringTask, mostly. However, it seems that 
> I completely misunderstood what the feature request was. Jon Haddad noted 
> that pluggable means:
> 
> > 1. It's going to be java code
> > 2. the pluggable thing implements an interface defined in cassandra.
> > 3. the class would be compiled and dropped in lib (loaded into classpath 
> > automatically)
> > 4. The class can be specified in the yaml and is loaded by Class.forName() 
> > to pull the interface in
> > 
> > We would need to convert the current slow query logger into a class of the 
> > defined interface and have it be the default if no class is specified in 
> > the yaml.
> 
> Can someone point me to an existing implementation of this, that I can learn 
> from? A previous patch that contributed something similar, perhaps?
> 
> 
>

What would a pluggable logging implementation look like?

[Murukesh Mohanan]

I'm looking at CASSANDRA-13001 (pluggable slow query logging / handling). I 
wrote a hacky patch, where my main goal was to touch as few files as possible - 
so I did what I could within MonitoringTask, mostly. However, it seems that I 
completely misunderstood what the feature request was. Jon Haddad noted that 
pluggable means:

> 1. It's going to be java code
> 2. the pluggable thing implements an interface defined in cassandra.
> 3. the class would be compiled and dropped in lib (loaded into classpath 
> automatically)
> 4. The class can be specified in the yaml and is loaded by Class.forName() to 
> pull the interface in
> 
> We would need to convert the current slow query logger into a class of the 
> defined interface and have it be the default if no class is specified in the 
> yaml.

Can someone point me to an existing implementation of this, that I can learn 
from? A previous patch that contributed something similar, perhaps?

Where would a Cassandra equivalent of mysqldumpslow be placed?

[Murukesh Mohanan]

For CASSANDRA-13000 (slow query log analysis tool), I uploaded a script. If I 
were to submit it as a patch, where should I place it? In bin/, or tools/bin/? 
Since it is not directly concerned with running Cassandra, it seems like it 
should be in tools/bin. Or, is there a third location?

Re: [RELEASE] Apache Cassandra 3.0.11 released

[Murukesh Mohanan]

This is speculation on my part, but this might be due to the software used
to set up the repository. reprepro is very simple to use (and so, somewhat
popular), but it only supports one version of a package at a time. Older
deb files are not removed, but they do get dropped from the Packages file.
This repo might be setup using reprepro.

On Fri, 24 Feb 2017 at 08:29 Julien Anguenot <jul...@anguenot.org> wrote:

> Hey,
>
> Any reasons why old versions get removed from the Debian repository when a
> new version gets promoted?
>
> For instance, here one would expect to still be able to:
>
>   $ apt-get install cassandra=3.0.10
>
> But after that release only the latest 3.0.11 is available:
>
>
> http://dl.bintray.com/apache/cassandra/dists/30x/main/binary-amd64/Packages
>
> Old versions are still available from there:
>
>http://dl.bintray.com/apache/cassandra/pool/main/c/cassandra/ <
> http://dl.bintray.com/apache/cassandra/pool/main/c/cassandra/>
>
> But it requires to manually download and put them back to the apt cache.
>
> It is quite handy for point releases when a rollback is required, and
> possible, because of regressions.
>
> Thanks.
>
>J.
>
> --
> Julien Anguenot (@anguenot)
>
> > On Feb 21, 2017, at 12:03 PM, Michael Shuler <mich...@pbandjelly.org>
> wrote:
> >
> > The Cassandra team is pleased to announce the release of Apache
> > Cassandra version 3.0.11.
> >
> > Apache Cassandra is a fully distributed database. It is the right choice
> > when you need scalability and high availability without compromising
> > performance.
> >
> > http://cassandra.apache.org/
> >
> > Downloads of source and binary distributions are listed in our download
> > section:
> >
> > http://cassandra.apache.org/download/
> >
> > This version is a bug fix release[1] on the 3.0 series. As always,
> > please pay attention to the release notes[2] and Let us know[3] if you
> > were to encounter any problem.
> >
> > Enjoy!
> >
> > [1]: (CHANGES.txt)
> >
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/cassandra-3.0.11
> > [2]: (NEWS.txt)
> >
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/cassandra-3.0.11
> > [3]: https://issues.apache.org/jira/browse/CASSANDRA
> >
>
> --

Murukesh Mohanan,
Yahoo! Japan

Re: Showing a new property in DESCRIBE TABLE output

[Murukesh Mohanan]

Yes, that was it. The file to modify was cassandra/metadata.py in
cassandra-driver, which has a list of recognized table options (
https://github.com/datastax/python-driver/blob/master/cassandra/metadata.py#L2169).
Sorry for not posting the implementation, but I've followed the changes
made for adding the CDC option (
https://github.com/apache/cassandra/commit/e31e216234c6b57a531cae607e0355666007deb2).
Naturally, it didn't contain the changes made in another repository.

OT: I wonder if cassandra-driver can be added as a submodule to cassandra,
instead of embedding it as zip file. Since submodule updates are also part
of the commit history, that will make it easier to spot related changes in
the driver.

On Wed, 25 Jan 2017 at 07:56 Blake Eggleston <beggles...@apple.com> wrote:

I haven't seen your implementation, but the likely cause of your problem is
either that the new parameter isn't being sent over the client protocol, or
that cqlsh is ignoring it. The cqlsh output of DESCRIBE TABLE seems to be
generated by the TableMetadata class in the python driver (see the
as_cql_query method). Dropping a breakpoint in there would probably be a
good place to start.
On January 24, 2017 at 7:07:38 AM, Murukesh Mohanan (
murukesh.moha...@gmail.com) wrote:

I'm having a go at CASSANDRA-13002 (
https://issues.apache.org/jira/browse/CASSANDRA-12403), by adding a new
table property which will override the global slow_query_log_timeout_in_ms
setting. It works, but I can't get it to show up in cqlsh DESCRIBE TABLE
output. For example, this is what I get:

cqlsh> DESCRIBE TABLE foo.bar;

CREATE TABLE foo.bar (
id uuid PRIMARY KEY,
name text
) WITH bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND cdc = true
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 1001
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';

cqlsh> select table_name, slow_query_log_timeout_in_ms from
system_schema.tables where table_name = 'bar' allow filtering;

table_name | slow_query_log_timeout_in_ms
+--
bar | 103

The property (which is also called `slow_query_log_timeout_in_ms`) shows up
in the system_schema table.

It seems that the file to modify would be
src/java/org/apache/cassandra/db/ColumnFamilyStoreCQLHelper.java, but I
didn't have any luck modifying it.

Any pointers, please?



--

Murukesh Mohanan,
Yahoo! Japan

-- 

Murukesh Mohanan,
Yahoo! Japan

Showing a new property in DESCRIBE TABLE output

[Murukesh Mohanan]

I'm having a go at CASSANDRA-13002 (
https://issues.apache.org/jira/browse/CASSANDRA-12403), by adding a new
table property which will override the global slow_query_log_timeout_in_ms
setting. It works, but I can't get it to show up in cqlsh DESCRIBE TABLE
output. For example, this is what I get:

cqlsh> DESCRIBE TABLE foo.bar;

CREATE TABLE foo.bar (
id uuid PRIMARY KEY,
name text
) WITH bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND cdc = true
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 1001
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';

cqlsh> select table_name, slow_query_log_timeout_in_ms from
system_schema.tables  where table_name = 'bar' allow filtering;

 table_name | slow_query_log_timeout_in_ms
+--
bar |  103

The property (which is also called `slow_query_log_timeout_in_ms`) shows up
in the system_schema table.

It seems that the file to modify would be
src/java/org/apache/cassandra/db/ColumnFamilyStoreCQLHelper.java, but I
didn't have any luck modifying it.

Any pointers, please?



-- 

Murukesh Mohanan,
Yahoo! Japan