Re: [DISCUSS] Allow UPDATE on settings virtual table to change running configuration

2023-03-01 Thread David Capwell
otations one seems most reasonable >> to me and I didn’t have the chance to consider any others. Volatile seems >> fragile and unclear as a differentiator. I agree >> >> On Tue, 28 Feb 2023 at 17:47, Maxim Muzafarov >> mailto:mmu...@apache.org>> wrote: >

Re: [DISCUSS] Allow UPDATE on settings virtual table to change running configuration

2023-02-22 Thread David Capwell
I guess back to the point of the thread, we need a way to know what configs are mutable for the settings virtual table, so need some way to denote that the config replica_filtering_protection.cached_rows_fail_threshold is mutable. Given the way that the yaml config works, we can’t rely on the

Re: Intra-project dependencies

2023-02-16 Thread David Capwell
After a lot of effort I think this branch is in a good state, accord feels mostly like its in-tree and all the complexity of git is hidden mostly. I would love more feedback as the patch is in a usable state > On Jan 30, 2023, at 3:16 PM, David Capwell wrote: > > I took a stab at

Re: [DISCUSS] Moving system property names to the CassandraRelevantProperties

2023-02-09 Thread David Capwell
g a good question! Sure, a new checkstyle rule was added > to address this case for production and test classes. > > On Thu, 9 Feb 2023 at 19:40, David Capwell wrote: >> >> All properties meant to be used only for tests would have a prefix like >> "ca

Re: [DISCUSS] Moving system property names to the CassandraRelevantProperties

2023-02-09 Thread David Capwell
> All properties meant to be used only for tests would have a prefix like > "cassandra.test.name.of.property" and production properties would be > "cassandra.xyx". Once this is done, we can filter them out in vtable so there > would not be any test-related properties in production. Test

Re: [VOTE] CEP-21 Transactional Cluster Metadata

2023-02-07 Thread David Capwell
+1 > On Feb 7, 2023, at 7:15 AM, Jeremiah D Jordan > wrote: > > +1 nb > >> On Feb 6, 2023, at 10:15 AM, Sam Tunnicliffe wrote: >> >> Hi everyone, >> >> I would like to start a vote on this CEP. >> >> Proposal: >>

Re: Implicitly enabling ALLOW FILTERING on virtual tables

2023-02-03 Thread David Capwell
> I don't think the assumption that "virtual tables will always be small and > always fit in memory" is a safe one. Agree, there is a repair ticket to have the coordinating node do network queries to peers to resolve the table (rather than operator querying everything, allow the coordinator

Re: Welcome Patrick McFadin as Cassandra Committer

2023-02-02 Thread David Capwell
Congrats and welcome! =) > On Feb 2, 2023, at 10:53 AM, J. D. Jordan wrote: > > Congrats! > >> On Feb 2, 2023, at 12:47 PM, Christopher Bradford >> wrote: >> >>  >> Congrats Patrick! Well done. >> >> On Thu, Feb 2, 2023 at 10:44 AM Aaron Ploetz > > wrote:

Re: Merging CEP-15 to trunk

2023-02-01 Thread David Capwell
> It's been mentioned "always shippable trunk according to circleci". That's > not true: we are always shippable according to *either* CI. There are folk > just using ci-cassandra for our pre-commit gateway. It is important that you > don't trash the other CI system, particularly when it

Re: Merging CEP-15 to trunk

2023-01-31 Thread David Capwell
e that I did $ git show f8243f41c9e96c4a0390558086ece078b0b6b15c commit f8243f41c9e96c4a0390558086ece078b0b6b15c Author: David Capwell Date: Mon Jan 9 13:20:58 2023 -0800 Ninja: Add AccordTestUtils.parse which was missing in the latest commit diff --git a/test/unit/org/apache/cassandra/service/accord

Re: Merging CEP-15 to trunk

2023-01-30 Thread David Capwell
sts should be run before merge. There are examples of Jenkins only tests that are not run, but again this is due to existing limitations with Jenkins. > On Jan 30, 2023, at 3:33 PM, Henrik Ingo wrote: > > On Tue, Jan 31, 2023 at 1:28 AM David Capwell <mailto:dcapw...@ap

Re: Merging CEP-15 to trunk

2023-01-30 Thread David Capwell
> Does this mean there have also been nightly jenkins builds running? Is there > a history of such test results visible somewhere? If yes, I think that lends > a lot of credibility to the claim the process was as rigorous as it is for > trunk, and looking at the build history for a few minutes

Re: Intra-project dependencies

2023-01-30 Thread David Capwell
I took a stab at creating a patch that I think addresses most of the comments I saw in this thread, would love feedback in https://issues.apache.org/jira/browse/CASSANDRA-18204 Given that the leading solution is git submodules I went down

Re: [DISCUSSION] Framework for Internal Collection Exposure and Monitoring API Alignment

2023-01-30 Thread David Capwell
ortant thing here is to >> come into agreement about the future of JMX and what we will do or not as a >> community. Also, how much time people are able to invest. I guess this will >> influence any directions to be taken here. >> >> [1] >> https://lists.apache.org/thre

Re: Merging CEP-15 to trunk

2023-01-27 Thread David Capwell
> I've learned that when I have defended the need (or right, if appealing to > the Governance texts...) for contributors to be able to review a feature > branch at the time it is merged to trunk - which for Accord is now - that a > common reaction to this is that doing a review of Accord now

Re: [DISCUSSION] Framework for Internal Collection Exposure and Monitoring API Alignment

2023-01-26 Thread David Capwell
I took a look and I see the result is an interface that looks like the vtable interface, that is then used by vtables and JMX? My first thought is why not just use the vtable logic? I also wonder about if we should care about JMX? I know many wish to migrate (its going to be a very long

Re: Intra-project dependencies

2023-01-19 Thread David Capwell
Thanks for the reply, my replies are inline to your inline replies =D > On Jan 19, 2023, at 2:39 PM, Mick Semb Wever wrote: > > > Thanks David for the detailed write up. Replies inline… > > > We tried in-tree for in-jvm dtest and found that this broke every other > commit… maintaining the

Re: Intra-project dependencies

2023-01-18 Thread David Capwell
Been out, sorry for just catching up now… I feel this thread pidgin hold on the word Accord and ignored the fact we are dealing with this pain today with python/jvm dtest and trying to improve that would help the project…. We also have other related projects that we are developing in parallel

Re: Should we change 4.1 to G1 and offheap_objects ?

2023-01-12 Thread David Capwell
I am cool with updating NEWS in 4.1.1 to recommend the change and change it in 4.2/5.0 > On Jan 12, 2023, at 10:56 AM, Josh McKenzie wrote: > > Potential compromise: We change it in trunk, and we NEWS.txt in the minor > about that change in trunk, why, and recommend users consider qualifying

Re: Introducing mockito-inline library among test dependencies

2023-01-11 Thread David Capwell
+1. We already use mockito. Also that library is basically empty, its just defining configs for extensions (see https://github.com/mockito/mockito/tree/main/subprojects/inline/src/main/resources/mockito-extensions

Re: [DISCUSSION] Cassandra's code style and source code analysis

2022-12-22 Thread David Capwell
Im good with 3 and 4. > On Dec 22, 2022, at 10:41 AM, Derek Chen-Becker wrote: > > I vote for #4. I've always used the convention of having stdlib stuff > first, external stuff second, and same-project imports last. I guess > increasing order of specificity? > > Happy Holidays! > > Derek > >

Re: Issue when creating a stream session

2022-12-16 Thread David Capwell
This sounds like a bug to me, but would be good to get feedback from others who have touched Streaming… Repair will fail if membership notifies that a participate node was removed, so I think it makes sense for Streaming to also follow this behavior. > On Dec 14, 2022, at 1:22 PM, Natnael

Re: [DISCUSSION] New dependencies for SAI CEP-7

2022-12-14 Thread David Capwell
> As such I would prefer to keep using the carrotsearch generators Works for me; I am cool with the added test dependency. > On Dec 14, 2022, at 7:13 AM, Mike Adamson wrote: > > I have had a look at whether we could use the QuickTheories in our randomized > testing and come to the following

Re: [DISCUSSION] New dependencies for SAI CEP-7

2022-12-13 Thread David Capwell
library was only added to provide us with a rich set of > random generators. I am happy to look at removing this library if its > inclusion is contentious. > > > On Mon, 12 Dec 2022 at 19:41, David Capwell <mailto:dcapw...@apple.com>> wrote: >> com.carrotsearch.ran

Re: [DISCUSSION] New dependencies for SAI CEP-7

2022-12-12 Thread David Capwell
> com.carrotsearch.randomizedtesting.randomizedtesting-runner 2.1.2 - test > dependency Can you talk more about why? There are several ways to do random testing in-tree ATM, so wondering why we need another one > On Dec 8, 2022, at 6:51 AM, Mike Adamson wrote: > > Hi, > > I wanted to

Re: [DISCUSSION] If we fix code that used default encoding to now be UTF-8... is this a regression?

2022-11-28 Thread David Capwell
t; Cheers, > > Derek > > On Fri, Nov 4, 2022 at 2:09 PM David Capwell wrote: >> >> Testing out linter trying to see if it can solve a case for Simulator and >> see we have 25 cases where we don’t add the encoding and rely on default, >>

Re: [DISCUSSION] Cassandra's code style and source code analysis

2022-11-28 Thread David Capwell
I am strong +1 to new linters, I have been working on SpotBugs but not sent a patch due to sickness and holidays… About the check style as the source of truth for the style guides, I am +1 to this as well… I feel that wiki is a bad place for this and we can use the check style file to generate

Re: Should we change 4.1 to G1 and offheap_objects ?

2022-11-16 Thread David Capwell
Getting poked in Slack to be more explicit in this thread… Switching to G1 on trunk, +1 Switching to G1 on 4.1, -1. 4.1 is about to be released and this isn’t a bug fix but a perf improvement ticket and as such should go through validation that the perf improvements are seen, there is not

Re: Should we change 4.1 to G1 and offheap_objects ?

2022-11-09 Thread David Capwell
CASSANDRA-12029/CASSANDRA-7486 I am not in favor of doing for 4.1, we spend time validating the current settings, so changing at the last minute adds risk; so rather push that to 4.2/5.0 > On Nov 9, 2022, at 11:25 AM, Brandon Williams wrote: > > CMS was deprecated in JDK 9, I don't see a

Re: [DISCUSSION] Add SpotBugs to the build

2022-11-09 Thread David Capwell
Since there was not major pushback, ill see this as green line for JIRA, ill file later today > On Nov 8, 2022, at 9:46 AM, David Capwell wrote: > > Thanks all for the replies, I hope I am posting a summary of all the feedback > > 1) double check with legal due to LGPL >

Re: [DISCUSSION] Add SpotBugs to the build

2022-11-08 Thread David Capwell
Thanks all for the replies, I hope I am posting a summary of all the feedback 1) double check with legal due to LGPL 2) need way to opt-out during development similar to checkstyle 3) patch should be in the same spirit as the other build changes, hiding details in .build and following naming 4)

[DISCUSSION] Add SpotBugs to the build

2022-11-07 Thread David Capwell
I was thinking that it would be good to add SpotBugs (https://spotbugs.github.io) into our build to help find bugs earlier in the life cycle. SpotBugs is LGPL but as it is used only in the build and not to within this project, then this should be fine with Apache. The motivation for adding

[DISCUSSION] If we fix code that used default encoding to now be UTF-8... is this a regression?

2022-11-04 Thread David Capwell
Testing out linter trying to see if it can solve a case for Simulator and see we have 25 cases where we don’t add the encoding and rely on default, which is based off the system… If we attempt to fix these cases, I am wondering if this is a regression… it “might” be the case someone set

Re: A proposal for refactoring the CircleCI config

2022-11-02 Thread David Capwell
Here is the ticket I was talking about https://issues.apache.org/jira/browse/CASSANDRA-17600 > On Nov 2, 2022, at 1:29 PM, Derek Chen-Becker wrote: > >> For the parallel param logic, sounds fine to me. Not sure if this would >> also

Re: A proposal for refactoring the CircleCI config

2022-11-02 Thread David Capwell
Thanks for starting this thread! If I am reading correctly, there are 2 main improvements proposed: move parallel value to a param and have our “patches” update those and not the jobs themselves, look into / use matrix to define the set of jobs we need… For the parallel param logic, sounds

Re: [DISCUSS] Potential circleci config and workflow changes

2022-10-25 Thread David Capwell
be able to run ci >>> 2. People that have resources and want to run ci faster should be able to >>> do so (assuming the ci of record could serve to be faster) >>> 3. ci should be stable >>> >>> Thus far we haven't landed on 1 system that satisfies al

Re: [DISCUSS] Potential circleci config and workflow changes

2022-10-21 Thread David Capwell
our jobs to what actually is needed… I hate that we all do something different (MID, HIGH, and custom HIGH) > On Oct 21, 2022, at 10:39 AM, David Capwell wrote: > > I am cool with removing circle if apache CI is stable and works, we do need > to solve the non-committer

Re: [DISCUSS] Potential circleci config and workflow changes

2022-10-21 Thread David Capwell
I am cool with removing circle if apache CI is stable and works, we do need to solve the non-committer issue but would argue that partially exists in circle today (you can be a non-commuter with a paid account, but you can’t be a non-committer with a free account) > On Oct 20, 2022, at 2:20

Re: [DISCUSS] Potential circleci config and workflow changes

2022-10-19 Thread David Capwell
> 1. Tune parallelism levels per job (David and Ekaterina have insight on this) +1 to this! I drastically lower our parallelism as only python-dtest upgrade tests need many resources… What I do for JVM unit/jvm-dtest is the following def java_parallelism(src_dir, kind, num_file_in_worker,

Re: [Discuss] CASSANDRA-17914: Modernize CQLSH's with argparse for CLI arts

2022-10-17 Thread David Capwell
As long as this doesn’t break the CLI interface, +1 from me; less dependencies are better and valuable work! > On Sep 29, 2022, at 6:52 AM, Berenguer Blasi wrote: > > +1 > > On 29/9/22 15:42, Derek Chen-Becker wrote: >> +1 from me. It sounds like a good opportunity! >> >> Cheers, >> >>

Re: [DISCUSS] Adding dependency on agrona

2022-09-23 Thread David Capwell
+1 from me > On Sep 23, 2022, at 1:21 AM, Branimir Lambov > wrote: > > The usage in the trie memtable is only for volatile access to buffers. In > this case I chose the library instead of reimplementing the functionality > (e.g. as methods in `ByteBufferUtil`) because the relevant interface

Re: CEP-15 multi key transaction syntax

2022-09-21 Thread David Capwell
a problem with the grammar today, I >> think it'd probably be worth the time to sort that out? >> >> >> >> On Wed, Sep 21, 2022 at 12:42 PM David Capwell > <mailto:dcapw...@apple.com>> wrote: >> Caleb is making great progress on this, and I ha

Re: CEP-15 multi key transaction syntax

2022-09-21 Thread David Capwell
Caleb is making great progress on this, and I have been working on CQL fuzz testing the new grammar to make sure we flesh out cases quickly; one thing we hit was about mixing conditional and non-conditional updates; will use a example to better show BEGIN TRANSACTION LET a = (SELECT * FROM

Re: [DISCUSS] Adding dependency on agrona

2022-09-21 Thread David Capwell
I am +1 to adding, good library, but agree with Benedict it would be good to know “why”. > On Sep 21, 2022, at 5:40 AM, Benedict wrote: > > In principle no, it’s a high quality library. But it might help to briefly > outline what it’s used for. I assume it is instead of ByteBuffer? In which

Re: [DISCUSS] Removing support for java 8

2022-08-31 Thread David Capwell
+1 to remove from trunk > On Aug 30, 2022, at 7:54 PM, Caleb Rackliffe wrote: > > +1 on removing 8 for trunk > > On Tue, Aug 30, 2022 at 2:42 PM Jon Haddad > wrote: > +1 to removal of 8 in trunk. > > On 2022/08/29 20:09:55 Blake Eggleston wrote: > > Hi all,

Re: [DISCUSS] LWT UPDATE semantics with + and - when null

2022-08-31 Thread David Capwell
s consistency with counters is important. But they are >> a unique eventually consistent data type, and I am inclined to default >> standard numeric types to behave as SQL does, since they write a new value >> rather than a “delta” >> >> It is far from optimal to h

[DISCUSS] LWT UPDATE semantics with + and - when null

2022-08-30 Thread David Capwell
4.1 added the ability for LWT to support "UPDATE ... SET name = name + 42", but we never really fleshed out with the larger community what the semantics should be in the case where the column or row are NULL; I opened up https://issues.apache.org/jira/browse/CASSANDRA-17857 for this issue. As I

Re: Cassandra project biweekly status update 2022-06-14

2022-06-16 Thread David Capwell
This has come up in CASSANDRA-16844, so wanted to bring to a wider audience. As of this moment, 3 breaking changes have been detected in 4.1 (1 of them was merged into 4.0 as well), and how to resolve this has been talked about a lot, so felt its best to have this conversation here. We've

Re: Appetite for a 4.1-alpha1 ?

2022-05-18 Thread David Capwell
Works for me > On May 18, 2022, at 7:36 AM, Josh McKenzie wrote: > > +1 from me on the grounds that I expect users to be more inclined to test an > alpha build of 4.1 rather than finding and pulling down a nightly. > Expectations of stability differ. > > On Wed, May 18, 2022, at 10:18 AM,

Re: [DISCUSSION] CASSANDRA-17562 and CASSANDRA-15254; future of JMX and Virtual tables

2022-05-13 Thread David Capwell
s/I don’t feel its non-trivial/I feel its non-trivial/. Update support requires a good amount of testing, so isn’t as simple as calling the existing setters (mostly defining a Function that works for callers for each complex type) > On May 13, 2022, at 10:26 AM, David Capwell wr

Re: [DISCUSSION] CASSANDRA-17562 and CASSANDRA-15254; future of JMX and Virtual tables

2022-05-13 Thread David Capwell
CASSANDRA-15254 only adds support for updating, we already have support for viewing (getter). The work to support mostly gets complicated for complex types such as collections, so I don’t feel its non-trivial as we have complex types in our config that has to be dealt with, so since there

Re: How we flag tickets as blockers during freeze

2022-05-09 Thread David Capwell
I am in favor of option 1. If you are 4.1.0 and not resolved, then we either need to kick you out of the 4.1.0 release (as you are not a blocker) or you are a blocker for that release and must be fixed in 4.1.0 > On May 9, 2022, at 2:49 PM, bened...@apache.org wrote: > > I think this is close

Re: JMX exposing non-standard java classes, to fix requires a breaking change

2022-05-05 Thread David Capwell
-Dcassandra.settings.jmx_hide_non_java_exceptions=false Does anyone see any issues about this? > On Apr 6, 2022, at 9:35 AM, David Capwell wrote: > > I noticed in DatabaseDescriptor there are setters that are leaking it to JMX > since cassandra-3.0. I am not sure whether we can

Re: JMX exposing non-standard java classes, to fix requires a breaking change

2022-04-06 Thread David Capwell
erent understanding so I want to align us to ensure we > don’t break tools by fixing or not stuff… > In many setters we were actually not doing the same checks we do on > Startup too… I consider this being a bug. > > On Tue, 5 Apr 2022 at 18:21, David Capwell wrote: > >>

JMX exposing non-standard java classes, to fix requires a breaking change

2022-04-05 Thread David Capwell
There are 2 places that expose non-standard java classes, so JMX only works if and only if the JMX client also has Cassandra's jars, else they will fail; the 2 examples are org.apache.cassandra.service.StorageServiceMBean#enableAuditLog throws

Re: Dropping Python 3.6 support in 4.1

2022-04-05 Thread David Capwell
We still mostly support CentOS 6 =D, I say mostly due to the fact I am not aware of any formal supports, just informal patches getting older OSes working again. > On Apr 5, 2022, at 4:33 AM, Stefan Miklosovic > wrote: > > But ... this begs another question to be asked - until when we want to

Re: [DISCUSSION] Cassandra-17515 - course of action

2022-04-04 Thread David Capwell
I am good with making >= 0 valid in both setters and config parsing, negatives should error or produce defaults > On Apr 4, 2022, at 10:45 AM, Caleb Rackliffe wrote: > > I'm for making >= 0 valid in both the setters and on startup. In the setters, > I'm fine with either translating negative

Re: [DISCUSSION] Shall we update Constants.KEY_DTEST_API_CONFIG_CHECK to true by default for in-jvm upgrade tests?

2022-04-04 Thread David Capwell
I am cool with checking by default and disabling for tests that need it, it may also make more sense to add an allow list so tests can explicitly say which configs to ignore (though this sounds painful to implement). > On Apr 4, 2022, at 9:11 AM, Jon Meredith wrote: > > I think checking the

Re: [VOTE] Release dtest-api 0.0.13

2022-03-31 Thread David Capwell
+1 > On Mar 31, 2022, at 9:53 AM, Alex Petrov wrote: > > Proposing the test build of in-jvm dtest API 0.0.13 for release. > > Repository: > https://gitbox.apache.org/repos/asf?p=cassandra-in-jvm-dtest-api.git;a=shortlog;h=refs/tags/0.0.13 > >

Re: [DISCUSS] Should we deprecate / freeze python dtests

2022-03-30 Thread David Capwell
gt; > features. > > > > Perhaps this has improved recently and we no longer need to worry about > > extending the framework or duplicating code when testing new functionality. > > > > Em ter., 29 de mar. de 2022 às 15:12, Ekaterina Dimitrova > > mailto:e.dimitr

Re: [DISCUSS] Should we deprecate / freeze python dtests

2022-03-29 Thread David Capwell
e initialization and thus they test the test server rather than the >> real node. Other than that, it can be problematic to test upgrades when the >> starting version must run with a different Java version than the end >> release. One more thing I've been observing sometimes

Re: [DISCUSS] Next release cut

2022-03-28 Thread David Capwell
I agree with * Fork may 1st * Freeze cassandra-4.1 on may 1st > On Mar 9, 2022, at 1:23 AM, Benjamin Lerer wrote: > > We also have the requirement that there will be no release without green CI. > Do we cut a release branch on the 1st May if we don't yet have a green CI and > can't be making

Re: [DISCUSS] Should we deprecate / freeze python dtests

2022-03-28 Thread David Capwell
I am back and the work for trunk to support vnode is at the last stage of review; I had not planned to backport the changes to other branches (aka, older branches would only support single token), so if someone would like to pick up this work it is rather LHF after 17332 goes in (see trunk

Re: [RELEASE] Apache Cassandra 4.0.2 released

2022-02-13 Thread David Capwell
removed(maybe replaced by different solution) and this is just a > placeholder. > >> On Sun, 13 Feb 2022 at 8:34, David Capwell wrote: >> In CASSANDRA-17166 I am adding a test that detects breaking changes; this is >> not compile time, rather test case time… >&g

Re: [RELEASE] Apache Cassandra 4.0.2 released

2022-02-13 Thread David Capwell
In CASSANDRA-17166 I am adding a test that detects breaking changes; this is not compile time, rather test case time… We sadly don’t document when a config is free to remove, we should do that to enhance our warnings to users as well as make clear when free to remove; we spoke about this in

Re: [VOTE] Formalizing our CI process

2022-01-12 Thread David Capwell
+1 > On Jan 12, 2022, at 8:39 AM, Joseph Lynch wrote: > > On Wed, Jan 12, 2022 at 3:25 AM Berenguer Blasi > wrote: >> >> jenkins CI was at 2/3 flakies consistently post 4.0 release. > > That is really impressive and I absolutely don't mean to downplay that > achievement. > >> Then things

Re: [VOTE] Formalizing our CI process

2022-01-10 Thread David Capwell
> Finalize on the canonical set of tests and JDK env to run pre-commit (JIRA > Pending; to be linked) Would be nice to get that defined, but I am cool with +1 for the rest and deferring solving that till after > On Jan 10, 2022, at 1:18 PM, Joshua McKenzie wrote: > > +1 > > On Mon, Jan

Re: [DISCUSS] Releasable trunk and quality

2022-01-04 Thread David Capwell
> The more I think on it, the more I am anyway strongly -1 on having some > bifurcated commit process. We should decide on a uniform commit process for > the whole project, for all patches, whatever that may be. Making the process stable and handle all the random things we need to handle takes

Re: [DISCUSS] Next release cut

2022-01-03 Thread David Capwell
4.1 target of may works for me, we may want to remove a few things in Config.java if not wrapped up on time (thinking track warnings and guardrails, though that should be handled by then) > On Dec 14, 2021, at 5:02 AM, Paulo Motta wrote: > > +1 to cut 4.1 in May. It would be great to attain

Re: [DISCUSS] Nested YAML configs for new features

2021-12-03 Thread David Capwell
ls, verbs/nouns, and specific terms >>> (period, abort, limit, datacenter vs dc, etc), but ideally we would have >> a >>> common end goal for the config file. >>> >>>> leave non-features to CASSANDRA-15234 >>> >>> IMO 15234 has sailed – it’s bee

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread David Capwell
epresentation of nested configs, then this would be a brand new > feature; we currently show the nested structure in cassandra.yaml > >> On Nov 29, 2021, at 11:58 AM, David Capwell >> wrote: >> >> Thanks everyone for the comments, I hope below is a good summary of a

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread David Capwell
bility to inline objects), so if we did allow flat config representation of nested configs, then this would be a brand new feature; we currently show the nested structure in cassandra.yaml > On Nov 29, 2021, at 11:58 AM, David Capwell > wrote: > > Thanks everyone for the comm

Re: [DISCUSS] Nested YAML configs for new features

2021-11-29 Thread David Capwell
Thanks everyone for the comments, I hope below is a good summary of all the talking points? We already use nested configs (networking, seed provider, commit log/hint compression, back pressure, etc.) Flat configs are easier for grep, but can be solved with grep -A/-B and/or yq It would be

Re: [DISCUSS] Nested YAML configs for new features

2021-11-19 Thread David Capwell
should support nested and flat, so if your infra works better with flat then you could use track_warnings.coordinator_read_size.warn_threshold_kb: 0 track_warnings.coordinator_read_size.abort_threshold_kb: 0 > On Nov 19, 2021, at 1:34 PM, David Capwell wrote: > >> With the flat

Re: [DISCUSS] Nested YAML configs for new features

2021-11-19 Thread David Capwell
t is flat, I can just grep "track_warnings" and I have >>> them all. >>> >>> Can you elaborate on your last bullet point? Parsing layer ... What do >>> you mean specifically? >>> >>> Thanks >>> >>> On Fri, 19 Nov 2021 at

Re: [DISCUSS] Nested YAML configs for new features

2021-11-19 Thread David Capwell
se? If it is flat, I can just grep "track_warnings" and I have > them all. > > Can you elaborate on your last bullet point? Parsing layer ... What do > you mean specifically? > > Thanks > > On Fri, 19 Nov 2021 at 19:36, David Capwell wrote: >>

[DISCUSS] Nested YAML configs for new features

2021-11-19 Thread David Capwell
This has been brought up in a few tickets, so pushing to the dev list. CASSANDRA-15234 - Standardise config and JVM parameters CASSANDRA-16896 - hard/soft limits for queries CASSANDRA-17147 - Guardrails prototype In short, do we as a project wish to move "new features" into nested YAML when the

Re: [VOTE] CEP-17: SSTable format API

2021-11-15 Thread David Capwell
+1 > On Nov 15, 2021, at 12:18 PM, bened...@apache.org wrote: > > +1 > > From: Brandon Williams > Date: Monday, 15 November 2021 at 19:47 > To: dev@cassandra.apache.org > Subject: Re: [VOTE] CEP-17: SSTable format API > +1 > > On Mon, Nov 15, 2021 at 1:43 PM Branimir Lambov wrote: >> >> Hi

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

2021-11-15 Thread David Capwell
und for sometime, we can start a > separate discussion about whether we want to put some guarantees on them. > > - - -- --- - - > Jacek Lewandowski > > > On Wed, Nov 10, 2021 at 9:01 PM David Capwell > wrote: > >> If this gets descope

Re: [DISCUSS] Mentoring newcomers

2021-11-12 Thread David Capwell
I am cool helping. > On Nov 12, 2021, at 10:29 AM, Ekaterina Dimitrova > wrote: > > I am in too > > On Fri, 12 Nov 2021 at 13:23, wrote: > >> I am interested as well. >> >> Sent from my iPhone >> >>> On 12. Nov 2021, at 19:01, Paulo Motta wrote: >>> >>> Count me in. >>> Em sex.,

Re: [VOTE] CEP-3: Guardrails

2021-11-11 Thread David Capwell
+1 > On Nov 11, 2021, at 7:10 AM, Sumanth Pasupuleti > wrote: > > +1 > > On Thu, Nov 11, 2021 at 6:10 AM Gary Dusbabek wrote: > >> +1 >> >> On Thu, Nov 11, 2021 at 5:30 AM Andrés de la Peña >> wrote: >> >>> Hi everyone, >>> >>> I would like to start a vote on this CEP. >>> >>>

Re: [DISCUSS] CEP-3: Guardrails

2021-11-10 Thread David Capwell
a lot of sense to >>>> me, I have updated the CEP to mention that. I think we don't need to >> decide >>>> yet whether it would be done through JMX and/or virtual tables. >>>> >>>> On Mon, 1 Nov 2021 at 20:35, C. Scott Andreas >>&

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

2021-11-10 Thread David Capwell
re’s a lot that needs to be done over the coming years to >> improve the internal structure of the project, and unduly entrenching the >> current state of affairs would be a huge potential harm of these efforts to >> modularise the codebase. >> >> From: David Capwell >

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

2021-11-09 Thread David Capwell
t'd be valuable for us to go through the codebase and annotate >> interfaces as intended to be exposed to 3rd parties; this has bothered me >> for years. Especially as we come up on a large number of new cleanups, >> refactorings, and potentially genericizing some subsystems into API's

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

2021-11-09 Thread David Capwell
gt; designed_ for external usage. /sigh >> >> I think it'd be valuable for us to go through the codebase and annotate >> interfaces as intended to be exposed to 3rd parties; this has bothered me >> for years. Especially as we come up on a large number of new cleanups, >> re

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

2021-11-09 Thread David Capwell
e many such > interfaces, I see no reason to block adding more of them while change > policies are discussed. > > -Jeremiah > >> On Nov 9, 2021, at 10:44 AM, David Capwell >> wrote: >> >> I still have one outstanding comment, but this is a comment for several

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

2021-11-09 Thread David Capwell
re > we ready to move forward to a vote? > > Regards, > Branimir > > On Tue, Nov 2, 2021 at 7:15 PM David Capwell > wrote: > >>> I apologize I did not mention those things explicitly. All the places >> where >>> sstable files are accessed di

Re: [DISCUSS] Creating a new slack channel for newcomers

2021-11-08 Thread David Capwell
> Do newcomers feel confident enough to ask questions there? We get a few a quarter, guess the bigger question is “why do we not get more”? I don’t have issues following another channel, glad to help where I can. Another channel is low effort so I don’t have any issues trying to see if it

Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer

2021-11-08 Thread David Capwell
Congrats! > On Nov 8, 2021, at 4:00 AM, Benjamin Lerer wrote: > > Congrats !  > > Le ven. 5 nov. 2021 à 20:20, Jordan West a écrit : > >> Congratulations Sumanth! >> >> On Fri, Nov 5, 2021 at 12:17 PM Stefan Miklosovic < >> stefan.mikloso...@instaclustr.com> wrote: >> >>> Welcome! >>>

Re: [DISCUSS] Releasable trunk and quality

2021-11-03 Thread David Capwell
> It’s hard to gate commit on a clean CI run when there’s flaky tests I agree, this is also why so much effort was done in 4.0 release to remove as much as possible. Just over 1 month ago we were not really having a flaky test issue (outside of the sporadic timeout issues; my circle ci runs

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

2021-11-02 Thread David Capwell
> I apologize I did not mention those things explicitly. All the places where > sstable files are accessed directly would have to be refactored. Works for me > Speaking about the implementation, one idea I was thinking about was that > the factories for formats are registered using Java's native

Re: [DISCUSS] Releasable trunk and quality

2021-11-01 Thread David Capwell
he.org/view/patches/job/Cassandra-devbranch-test/build <https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-test/build>; no support to run against your other branches). > On Nov 1, 2021, at 3:03 PM, David Capwell wrote: > >> How do we define what "releasable

Re: [DISCUSS] Releasable trunk and quality

2021-11-01 Thread David Capwell
> How do we define what "releasable trunk" means? One thing I would love is for us to adopt a “run all tests needed to release before commit” mentality, and to link a successful run in JIRA when closing (we talked about this once in slack). If we look at CircleCI we currently do not run all

Re: [DISCUSS] CEP-3: Guardrails

2021-11-01 Thread David Capwell
If anyone wants to bite off making https://github.com/apache/cassandra/blob/ab920c30310a8c095ba76b363142b8e74cbf0a0a/src/java/org/apache/cassandra/db/virtual/SettingsTable.java

Re: [DISCUSS] CEP-3: Guardrails

2021-11-01 Thread David Capwell
Under "Migrating existing cassandra.yaml warn/fail thresholds”, I recently added a few things which are basically guardrails, so should be included in this set; they are configured by track_warnings (coordinator_read_size, local_read_size, and row_index_size). With track_warnings I setup the

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

2021-11-01 Thread David Capwell
this as an expansion of CASSANDRA-7443 and cleanup of any changes > that came after it and broke the intended capability. > > Regards, > Branimir > > On Thu, Oct 28, 2021 at 7:43 PM David Capwell > wrote: > >> Sorry about that; used -1/+1 to show preference, not bind

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

2021-10-28 Thread David Capwell
to express our > preferences and defer to the collective opinion where possible. True -1s > should very rarely appear. > > > From: David Capwell > Date: Wednesday, 27 October 2021 at 15:33 > To: dev@cassandra.apache.org > Subject: Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

2021-10-27 Thread David Capwell
Reading the CEP I don’t see any mention to the systems which access SSTables; such as streaming (small callout to zero-copy-streaming with ZeroCopyBigTableWriter) and repair. If you are abstracting out BigTableReader then you are not dealing with the implementation assumptions that users of

Re: [DISCUSS] How to implement backward compatibility (CASSANDRA-17048)

2021-10-27 Thread David Capwell
> should we let users enable the new uuid ids later when they are sure they > will not downgrade in the future? I strongly believe new features should be off by default and give a “good enough” grace period before enabling by default (while still offering support to disable). As long as this

Re: [QUESTION] Should JVM dtests be resistant to System.exit called in the node thread?

2021-10-27 Thread David Capwell
> It is also possible to install security manager which prevents System.exit. > If so, should this be implemented globally for all the JVM dtests or > individually for JVM dtests which need that? I'm asking because when > System.exit is called unexpectedly, the test is not reported as failed -

<    1   2   3   >