Latest changes to CircleCI testing workflow

2019-03-15 Thread Stefan Podkowinski
tldr; make sure to read the new instructions in the .circleci directory, if you’re a circleci user: https://github.com/apache/cassandra/tree/trunk/.circleci It’s been a while since I last reached out on dev- regarding proposed changes to our CircleCI setup [0]. Meanwhile Marcus and I have

Re: cqlsh tests and Python 3

2019-02-12 Thread Stefan Podkowinski
Previous discussion can be found here: https://lists.apache.org/thread.html/cbc50f5ac085ac759b52eb7e87277a3b82e2773c6d507c4b525d@%3Cdev.cassandra.apache.org%3E On 11.02.19 19:58, Ariel Weisberg wrote: Hi, Do you mean Python 2/3 compatibility? This has been discussed earlier and I think

Re: [VOTE] Release Apache Cassandra 2.1.21

2019-02-04 Thread Stefan Podkowinski
4 AM Nate McCall wrote: +1 on the release of 2.1.21 (let's focus on that in the spirit of these other votes we have up right now). I don't feel the need to be absolutist about something being EOL. On Sun, Feb 3, 2019 at 1:47 AM Stefan Podkowinski wrote: What are we voting on here? Rele

Re: [VOTE] Release Apache Cassandra 2.1.21

2019-02-03 Thread Stefan Podkowinski
What are we voting on here? Releasing the 2.1.21 candidate, or that 2.1 would become EOL? Please let's have separate votes on that, if you want to propose putting 2.1 EOL (which I'm strongly -1). On 03.02.19 01:32, Michael Shuler wrote: *EOL* release for the 2.1 series. There will be no new

Re: Who should be in our distribution KEYS file?

2019-01-09 Thread Stefan Podkowinski
If creating native deb/rpm package signatures and repo metadata signatures is the only issue that stops us from releasing more often and sharing the burden in doing so, then I'd be happy to discuss dropping this altogether. We can always distribute binary packages along with checksums and a

Re: Who should be in our distribution KEYS file?

2019-01-07 Thread Stefan Podkowinski
I don't see any reason to have any keys in there, except from release managers who are signing releases. Additional keys from other developers may even harm security, by creating more opportunities for compromising keys. On 07.01.19 11:29, Mick Semb Wever wrote: And when should it get

Re: JIRA Workflow Proposals

2018-12-05 Thread Stefan Podkowinski
Thanks Benedict and everyone involved putting up the proposal! It really deserves some more feedback and I realize that I'm a bit late for that and probably missed a good deal of the conversation so far. I'd still like to share some of my notes that I've taken while reading through it, for the

Re: JIRA Reports in Confluence

2018-11-22 Thread Stefan Podkowinski
Thanks for sorting out components across all these tickets. I really like the idea of having predefined reports. Looking at how tickets are grouped between 4.0, 4.0.x and 4.x, we should probably do some cleanup for the "fix version" attribute as well. We use to set a ultimate version once a patch

Re: Proposing an Apache Cassandra Management process

2018-11-18 Thread Stefan Podkowinski
My goal for a side car would be to enable more people to contribute to the project, by making it more accessible for anyone who’s not familiar with the Cassandra code base, or not familiar with Java development in general. Although most of the functionality described in the proposal sounds useful

Proposed changes to CircleCI testing workflow

2018-10-26 Thread Stefan Podkowinski
I'd like to give you a quick update on the work that has been done lately on running tests using CircleCI. Please let me know if you have any objections or don't think this is going into the right direction, or have any other feedback! We've been using CircleCI for a while now and results are

Re: [DISCUSS] changing default token behavior for 4.0

2018-09-22 Thread Stefan Podkowinski
There already have been some discussions on this here: https://issues.apache.org/jira/browse/CASSANDRA-13701 The mentioned blocker there on the token allocation shouldn't exist anymore. Although it would be good to get more feedback on it, in case we want to enable it by default, along with new

Proposing an Apache Cassandra Management process

2018-09-09 Thread Stefan Podkowinski
Does it have to be a single project with functionality provided by multiple plugins? Designing a plugin API at this point seems to be a bit early and comes with additional complexity around managing plugins in general. I was more thinking into the direction of: "what can we do to enable people to

Re: Supporting multiple JDKs

2018-09-08 Thread Stefan Podkowinski
I really don't see any benefit at all in having any additional Java 1.7 specific build and testing changes for the 2.2 branch. The 2.2 version is reaching EOL and will only get critical patches until then anyways. I also can't remember any reports on regressions in 2.2 bug fix releases specific to

Re: Side Car New Repo vs not

2018-08-21 Thread Stefan Podkowinski
I'm also currently -1 on the in-tree option. Additionally to what Aleksey mentioned, I also don't see how we could make this work with the current build and release process. Our scripts [0] for creating releases (tarballs and native packages), would need significant work to add support for an

Re: Proposing an Apache Cassandra Management process

2018-08-18 Thread Stefan Podkowinski
. On 18.08.18 17:44, Sankalp Kohli wrote: > The thread for side car is months old and no one has opposed to it and hence > someone developed it. I am not sure how else you get consensus. > > Regarding separate repo, how do we get consensus? > >> On Aug 18, 2018, at 05:19, Stef

Re: Proposing an Apache Cassandra Management process

2018-08-18 Thread Stefan Podkowinski
I don't see that we have reached sufficient consensus on this yet. We've had a brief discussion about the pros and cons of in-tree cassandra vs separate ASF repo here, but let's not frame it like it's either or. From my perspective, there hasn't been any final decision yet whether the proposed

Re: GitHub PR ticket spam

2018-08-06 Thread Stefan Podkowinski
+1 for worklog option Here's an example ticket from Arrow, where they seem to be using the same approach: https://issues.apache.org/jira/browse/ARROW-2583 On 05.08.2018 09:56, Mick Semb Wever wrote: >> I find this a bit annoying while subscribed to commits@, >> especially since we created pr@

Re: Tracking Testing, Release Status, and Build Health

2018-07-30 Thread Stefan Podkowinski
On 30.07.2018 02:04, Scott Andreas wrote: > I’m curious on the dev community’s thoughts on how best to organize > information like this. My thinking is that by having a space to share this, > the community can be more informed on each others’ work toward testing, build > health, and active

GitHub PR ticket spam

2018-07-30 Thread Stefan Podkowinski
Looks like we had some active PRs recently to discuss code changes in detail on GitHub, which I think is something we agreed is perfectly fine, in addition to the usual Jira ticket. What bugs me a bit is that for some reasons any comments on the PR would be posted to the Jira ticket as well. I'm

Re: [VOTE] Branching Change for 4.0 Freeze

2018-07-12 Thread Stefan Podkowinski
+1 (assuming merging patches on documentation will always be possible, as it's not effecting the code base) On 11.07.18 23:46, sankalp kohli wrote: > Hi, > As discussed in the thread[1], we are proposing that we will not branch > on 1st September but will only allow following merges into

Scratch an itch (was: [VOTE] Branching Change for 4.0 Freeze)

2018-07-12 Thread Stefan Podkowinski
These are some valid concerns. But I don’t really see it that way after thinking about it. We already have restrictions and consensus based practices in place that may discourage new contributors. E.g. if someone submits a patch to enable a different GC by default in 2.1, that’s probably not going

Re: [VOTE] Release Apache Cassandra 2.2.13

2018-07-04 Thread Stefan Podkowinski
+1 On 02.07.2018 22:10, Michael Shuler wrote: > I propose the following artifacts for release as 2.2.13. > > sha1: 9ff78249a0a5e87bd04bf9804ef1a3b29b5e1645 > Git: > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/2.2.13-tentative > > Artifacts: >

Re: Real time bad query logging framework in C*

2018-06-20 Thread Stefan Podkowinski
Jaydeep, thanks for taking this discussion to the dev list. I think it's the best place to introduce new idea, discuss them in general and how they potentially fit in. As already mention in the ticket, I do share your assessment that we should try to improve making operational issue more visible

Re: Tombstone passed GC period causes un-repairable inconsistent data

2018-06-20 Thread Stefan Podkowinski
Sounds like an older issue that I tried to address two years ago: https://issues.apache.org/jira/browse/CASSANDRA-11427 As you can see, the result hasn't been as expected and we got some unintended side effects based on the patch. I'm not sure I'd be willing to give this another try, considering

Re: [DISCUSS] Cassandra and future Java

2018-05-30 Thread Stefan Podkowinski
right Java version - >> and not let the package manger just pull the newest available. >> The version-string from adoptopenjdk for example is one of these “minor >> issues"... >> >> — >> Robert Stupp >> @snazy >> >> On 28. May 2018, at 15:46, S

Re: [DISCUSS] Cassandra and future Java

2018-05-28 Thread Stefan Podkowinski
The main issue that I see, for supporting both Java 8 + 11, is testing. We should first decide how this would effect builds.apache.org, or how we're going to do CI testing in general for that situation. There are probably also smaller issues that we're not aware of yet, such as which Java

Re: Roadmap for 4.0

2018-04-12 Thread Stefan Podkowinski
Maybe people would have preferred to know early about potential deadlines, before investing a lot of time into "pet ticket" contributions? It's hard enough to make assumptions about if and when contributions make it into a release, but with feature freeze deadlines falling from the sky any time,

Re: Roadmap for 4.0

2018-04-05 Thread Stefan Podkowinski
June is too early. On 05.04.18 19:32, Josh McKenzie wrote: > Just as a matter of perspective, I'm personally mentally diffing from > when 3.0 hit, not 3.10. > >> commit 96f407bce56b98cd824d18e32ee012dbb99a0286 >> Author: T Jake Luciani >> Date: Fri Nov 6 14:38:34 2015 -0500

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

2018-03-23 Thread Stefan Podkowinski
I think it's pretty safe to assume that Java 8 will stay around much longer than by the end of the year, after Oracle dropped their official maintainer role. I also think that we don't have to worry that much how exactly Java 8 is going to be supported. It's a mature enough version that I wouldn't

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

2018-03-21 Thread Stefan Podkowinski
The idea was not about building a custom JDK and ship it along with Cassandra, rather than using the new modular run-time images feature [0] introduced in Java 9. See also the link posted by Jason [1] for an practical introduction. [0] http://openjdk.java.net/jeps/220 [1]

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

2018-03-21 Thread Stefan Podkowinski
On 21.03.2018 15:41, Ariel Weisberg wrote: > I'm not clear on what building and bundling our own JRE/JDK accomplishes? If we talk about OpenJDK, there will be only a single Java version supported at any time and that is the latest Java version (11, 12, ..). There is no overlap between supported

Re: [DISCUSS] java 9 and the future of cassandra on the jdk

2018-03-21 Thread Stefan Podkowinski
There's also another option, which I just want to mention here for the sake of discussion. Quoting the Oracle Support Roadmap: "Instead of relying on a pre-installed standalone JRE, we encourage application developers to deliver JREs with their applications." I've played around with Java 9 a

Re: Debug logging enabled by default since 2.2

2018-03-20 Thread Stefan Podkowinski
Are you suggesting to move all messages currently logged via debug() to info() with the additional marker set, or only particular messages? On 19.03.2018 19:51, Paulo Motta wrote: > Thanks for the constructive input and feedback! From this discussion > it seems like overloading the DEBUG level

Re: Debug logging enabled by default since 2.2

2018-03-19 Thread Stefan Podkowinski
I'd agree that INFO should be the default. Turning on the DEBUG logging can cause notable performance issues and I would not enable it on production systems unless I really have to. That's why I created  CASSANDRA-12696 for 4.0, so you'll be able to at least only partially enable DEBUG based on

Re: [Patch Available for Review!] CASSANDRA-14134: Migrate dtests to use pytest and python3

2018-01-03 Thread Stefan Podkowinski
cle.com> >> wrote: >> >> Comments Inline: Thanks for giving this a go!! >> >>> On Jan 2, 2018, at 6:10 AM, Stefan Podkowinski <s...@apache.org> wrote: >>> >>> I was giving this a try today with some mixed results. First of all, >>> runnin

Re: [Patch Available for Review!] CASSANDRA-14134: Migrate dtests to use pytest and python3

2018-01-02 Thread Stefan Podkowinski
I was giving this a try today with some mixed results. First of all, running pytest locally would fail with an "ccmlib.common.ArgumentError: Unknown log level NOTSET" error for each test. Although I created a new virtualenv for that as described in the readme (thanks for updating!) and use both of

Re: Proposal: github pull requests for all code changes

2017-12-12 Thread Stefan Podkowinski
There's nothing that stops people from using github to discuss code changes. Many jiras already link to gh branches that can be used to review and comment code. But it's not always the best place to do so. The high level discussion should always take place on Jira. Although I'd have no problem to

Re: CCM dependency in dtests

2017-12-01 Thread Stefan Podkowinski
assing. having logic you need >> to fix in 3 repos isn’t ideal at all. >> >>> On Nov 27, 2017, at 4:05 AM, Stefan Podkowinski <s...@apache.org> wrote: >>> >>> Just wanted to bring a recent discussion about how to use ccm from >>> d

CCM dependency in dtests

2017-11-27 Thread Stefan Podkowinski
Just wanted to bring a recent discussion about how to use ccm from dtests to your attention: https://github.com/apache/cassandra-dtest/pull/13 Basically the idea is to not depend on a released ccm artifact, but to use a dedicated git branch in the ccm repo instead for executing dtests. Motivation

Integrating cloud and 3rd party security solutions

2017-10-20 Thread Stefan Podkowinski
I've been recently looking into how we could improve security in Cassandra by integrating external solutions. There are very interesting projects out there, such as Vault[0], but also a growing list of security related APIs offered by cloud providers. Today Cassandra can already be customized by

Re: Proposal to retroactively mark materialized views experimental

2017-10-04 Thread Stefan Podkowinski
Introducing feature flags for enabling or disabling different code paths is not sustainable in the long run. It's hard enough to keep up with integration testing with the couple of Jenkins jobs that we have. Running jobs for all permutations of flags that we keep around, would turn out

Re: [RESULT][VOTE] Ask Infra to move github notification emails to pr@

2017-07-28 Thread Stefan Podkowinski
Can we forward notifications for the new cassandra-dtest repo there as well? On 24.03.2017 18:59, Jeff Jirsa wrote: > With 6 binding +1s, 6 non-binding +1s, and no -1s of any kind, the vote > passes, I'll ask for a new mailing list and get this transitioned. > > - Jeff > > On 2017-03-20 15:32

Re: CHANGES.txt

2017-07-21 Thread Stefan Podkowinski
Thanks your responses! Seems like all of you prefer to have both trivial and non-trivial updates in CHANGES.txt. I'm going to keep that in mind, but will continue to omit them for documentation edits. On 18.07.2017 23:49, kurt greaves wrote: > I agree that all patches should be added to

CHANGES.txt

2017-07-18 Thread Stefan Podkowinski
Has there been any consensus in the past about what goes into CHANGES.txt and what not? My naive assumption was that the intended audience for the file are users who want to know about changes between new releases. Having that in mind, I skipped changes.txt once in a while for updates that have no

Re: sstabledump expects jna 5.1.0

2017-07-18 Thread Stefan Podkowinski
I haven't been able to reproduce this on Ubuntu or CentOS. Which OS do you use? Did you install a pre-build package or tarball? On 18.07.2017 11:43, Micha wrote: > Hello, > > when calling sstabledump from cassandra 3.11 I get the error: > > > "There is an incompatible JNA native library

Re: New contribution - Burst Hour Compaction Strategy

2017-06-09 Thread Stefan Podkowinski
Hello Pedro Thanks for being interested in contributing to Apache Cassandra. Creating a new compaction strategy is not an easy task and there are several things you can do to make it more obvious for other developers to understand what you're up to. First of all, if using github, changes to the

Re: Integrating vendor-specific code and developing plugins

2017-06-03 Thread Stefan Podkowinski
I'd suggest to use the git docs for the new pages, so we can accept pull requests for adding other plugins. [1] We can also link there from the main pages. Maybe the community page would be a good place for that. [1] https://cassandra.apache.org/doc/latest/development/documentation.html On

Status on new nodes for builds.apache.org

2017-06-02 Thread Stefan Podkowinski
Just a quick heads up for everyone interested in the jobs history at builds.apache.org or who wants to run devbranch jobs there. A couple of Jenkins nodes are not working correctly, which is causing jobs to abort abnormally during start. You'd either have to rebuild until you hit a working node,

Re: Potential block issue for 3.0.13: schema version id mismatch while upgrading

2017-05-31 Thread Stefan Podkowinski
On 31.05.2017 09:34, Stefania Alborghetti wrote: > There shouldn't be too many people with 3.0.13 in > production however, since upgrading to 3.0.13 is broken in the first place. Keep in mind that there are always people upgrading from 2.x, especially since we had a couple of important bug fixes

Re: Guidelines on testing

2017-04-25 Thread Stefan Podkowinski
I don't see any reasons not to make this part of our guidelines. The idea of having a list of what should be tested in each kind of test makes sense. I also like the examples how to improve tests dealing with global state. Some of the integration test cases, such as "dry

Re: Documentation contributors guide

2017-03-20 Thread Stefan Podkowinski
As nobody seems to object, you can now find a patch for the proposed document in the corresponding Jira ticket: https://issues.apache.org/jira/browse/CASSANDRA-13256 On 03/17/2017 08:33 PM, Stefan Podkowinski wrote: > There's recently been a discussion about the wiki and how we sho

Re: Documentation contributors guide

2017-03-17 Thread Stefan Podkowinski
reset --soft origin/trunk git commit (add proper commit message and a "Merges #" text to automatically close the PR) On 03/17/2017 09:03 PM, Jeff Jirsa wrote: > > > On 2017-03-17 12:33 (-0700), Stefan Podkowinski <s...@apache.org> wrote: > >> As you can se

Documentation contributors guide

2017-03-17 Thread Stefan Podkowinski
There's recently been a discussion about the wiki and how we should continue to work on the documentation in general. One of my suggestions was to start giving users a clearer guideline how they are able to contribute to our documentation, before having a technical discussion around tools and

Re: Testing and jira tickets

2017-03-16 Thread Stefan Podkowinski
y%20created%20ASC > > We're thankfully still in a place where these tickets are at least being > created, but unless there's a body of people that are digging in to fix > those test failures they're just going to keep growing. > > On Fri, Mar 10, 2017 at 5:03 AM, Stefan Podkowinsk

Re: Contribute to the Cassandra wiki

2017-03-13 Thread Stefan Podkowinski
Agreed. Let's not give up on this as quickly. My suggestion is to at least provide a getting started guide for writing docs, before complaining about too few contributions. I'll try to draft something up this week. What people are probably not aware of is how easy it is to contribute docs through

Re: Testing and jira tickets

2017-03-10 Thread Stefan Podkowinski
If I remember correctly, the requirement of providing test results along with each patch was because of tick-tock, where the goal was to have stable release branches at all times. Without CI for testing each individual commit on all branches, this just won't work anymore. But would that really be

Re: Official CS docs

2017-03-01 Thread Stefan Podkowinski
Hi Benjamin I think the best way to catch up with the motivation behind this is by reading the following dev post and linked jiras: https://lists.apache.org/thread.html/029e1273675260630e4973ba301f71a8de5a9d7e294a7b2df6eed65f@%3Cdev.cassandra.apache.org%3E What are your suggestions to improve

Re: Wrapping up tick-tock

2017-01-17 Thread Stefan Podkowinski
t think we should force a major just because n months have > >passed. I think we should up the major only when we have significant > > (possibly breaking) changes/features. It would seem odd to have a 6.0 > >that's basically the same as 4.0 (in terms of features and &

Re: Wrapping up tick-tock

2017-01-11 Thread Stefan Podkowinski
I honestly don't understand the release cadence discussion. The 3.x branch is far from production ready. Is this really the time to plan the next major feature releases on top of it, instead of focusing to stabilize 3.x first? Who knows how long that would take, even if everyone would exclusively

Re: Proposals for releases - 4.0 and beyond

2016-11-19 Thread Stefan Podkowinski
I’d like to suggest an option similar to what Jeremiah described and that would basically follow the Ubuntu LTS release model [1], but with shorter time periods. The idea would be to do a stable release every 6 months with 1 year bug fixing support. At the same time, every third stable release

Re: Rough roadmap for 4.0

2016-11-16 Thread Stefan Podkowinski
>From my understanding, this will also effect EOL dates of other branches. "We will maintain the 2.2 stability series until 4.0 is released, and 3.0 for six months after that.". On Wed, Nov 16, 2016 at 5:34 AM, Nate McCall wrote: > Agreed. As long as we have a goal I don't

Re: Rough roadmap for 4.0

2016-11-04 Thread Stefan Podkowinski
There has been a lot of discussions about diversity and getting new contributors and I think this aspect should be kept in mind as well when talking about a roadmap, additionally to the listed tickets that are already in the pipeline. What can inspiring developers contribute to 4.0 that would move

Re: CASSANDRA-10993 Approaches

2016-08-18 Thread Stefan Podkowinski
>From my perspective, one of the most important reasons for RxJava would be the strategic option to integrate reactive streams [1] in the overall Cassandra architecture at some point in the future. Reactive streams would allow to design back pressure fundamentally different compared to what we do