In case you were wondering, if you switch your ToDo list structure enough times you're bound to have something slip through the cracks. Like, say, a periodic project status update email. If you're curious where I landed: "All tools are comparably insufficient in different ways". /sigh
Don't be like me; be better. Last update was about 7 weeks ago. Have we all been lounging around all summer, taking vacation and doing nothing useful? We released 4.0.11 yesterday and the votes ongoing for 4.1.3, so I'm going to go with a "No". :) Either that or we're the kind of folks who work on open-source code for fun on vacation. ¯\_(ツ)_/¯ Here's the CHANGES.txt for 4.1.3: https://github.com/apache/cassandra/blob/4.1.3-tentative/CHANGES.txt Hit up the dev list to vote! There's a community survey! Go take it. Please. :) From Patrick's thread on the dev ML: > There are only 2 questions required, and the rest are all optional, so > answer whatever you can. It’s all helpful information. > > https://forms.gle/KVNd7UmUfcBuoNvF7 > > The survey will run until July 29, 2023. Once completed, the results will > be anonymized and the results posted on http://cassandra.apache.org *[New Contributors Getting Started] * https://the-asf.slack.com in #cassandra-dev is where we congregate. (reply to me on this email if you need an invite for your account), and reach out to the @cassandra_mentors alias with any questions about the code. Or just ask a public question about JMX vs. REST and watch the hornets nest come to life! :D For a "where should I get started?" list of curated work, you can check that out here: https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2160&quickFilter=2162. Anything in the "ToDo" column is a great place to get started. Some useful links: - Getting Started with Development on C*: https://cassandra.apache.org/_/development/gettingstarted.html - Building and IDE integration (worktrees are your friend; msg me on slack if you need pointers): https://cassandra.apache.org/_/development/ide.html - Code Style: https://cassandra.apache.org/_/development/code_style.html *[Dev mailing list] * https://lists.apache.org/list?dev@cassandra.apache.org:dfr=2023-5-30|dto=2023-7-20: Hm; 35 topics compared to the 52 from the last "oops I took too long" update. Looks like some were reasonably dense. What have we been up to? Stefan Miklosovic was looking into the addition of wiremock into the project as part of CASSANDRA-16555: https://lists.apache.org/thread/s4q0159wsrw19kcco555xjnt6b60z8jx. No updates on that in about a month; seemed like there wasn't big concern on it but some questions about documentation since we have a couple mocking libs already. This was merged in as part of https://issues.apache.org/jira/browse/CASSANDRA-16555. Jordan West reached out about changing our default SSL provider to either ACCP or tc-native: https://lists.apache.org/thread/lx43340ycmm2t22ds13wmjzxj5vvjly9. Some users have seen pretty significant reduction in CPU usage by switching over; this is being tracked in https://issues.apache.org/jira/browse/CASSANDRA-18624. Some changes in the DropWizard JMXReporter were making a whole slew of people sad. API's people! They're like marriage, or better yet, having children: It's a long-term commitment. Maxim Muzafarov has a patch available in CASSANDRA-14667 for this. Dev thread: https://lists.apache.org/thread/g3v06c3xtrgtcsr4x52nwj6v7dj9l6dl, JIRA: https://issues.apache.org/jira/browse/CASSANDRA-14667 As a heads up to everyone, ant generate-idea-files now generates for JDK17 as well. ML thread: https://lists.apache.org/thread/0hnf7wtcf86pn6b2lydd4kqcfjw8n4t9, JIRA: https://issues.apache.org/jira/browse/CASSANDRA-18467 Jacek Lewandowski hit the list to update folks w/his plans around limiting paging by bytes and queries by memory; we're working through this on CASSANDRA-11745, though likely not before a freeze for 5.0 whenever that may be. ML thread: https://lists.apache.org/thread/9zymojzgq5sw9jpmghf36nh8yn1kwz6k, JIRA: https://issues.apache.org/jira/browse/CASSANDRA-11745 Brad Schoening reached out about a potential change in argument parsing for cassandra-stress over to the Apache Commons CLI library here: https://lists.apache.org/thread/hbg25nn1jobx69zrl1syv1vl3qc318tn. Ekaterina pointed out we need to update it but otherwise it's been pretty radio silent for 9 days; probably have a lazy consensus there. Changes here *could* disrupt folks that have automated work using cassandra-stress which is worth keeping in mind. Given how a) aggressively internal this tool is, and b) the number of different stress tools out there targeting end-users (tlp-stress, NoSQLBench, etc), my personal .02 is that we go for it and unify those options. If anyone else disagrees, now's the time. Looks like we're going to deprecate the CloudstackSnitch snitch in 5.0 and let users know they're more or less on their own if they're using it since it's Very Not Maintained: https://lists.apache.org/thread/2zxvbjoynhvsr33cc9h9xv99sdvjldn8. Ah, static analysis. The thing we love to argue debate about. Well, *one* of the things. Jacek's looking to tidy things up a bit: https://lists.apache.org/thread/xot4xnoolgss34tq9ptb8mjy8pzjpbb7. Kind of noodling on when we run checks, where, on what targets in build.xml, on what targets in the build. Jacek digested and restated what we agreed upon on that thread in the context of his focus here: https://lists.apache.org/thread/66j9nmff89y0hft6z4cqgw6s37p25s5m, or for the lazy / efficient: > - Remove the checkstyle dependency from "jar" and "test" > - Create a single "check" target that includes all the checks we expect > to pass in the CI (currently Checkstyle, RAT, and Eclipse-Warnings), making > this task the default. On the above topic / thread, for the record I don't have any magic to share from my local env. SpotBugs and SonarLint still both largely think our code-base is a wasteland of sorrow and regret, however demoting them to warnings and using them to tidy up files when I touch them for other work has been a pretty effective strategy for me to capture some of their value without the *rather massive* initial lift of majorly tuning their rulesets to match our... particular... customs. :) Both are pretty simple to bind keymaps for "current file analysis" style approaches. The cqlsh version and C* version mismatch is now making German Eichberger sad (as it should): https://lists.apache.org/thread/yk167v55cx7g1k8c0cmom06hothrv6q2. We're tracking some work to document min required version inside a vtable in CASSANDRA-18666 https://issues.apache.org/jira/browse/CASSANDRA-18666; no assignee on that so if anyone's interested, hop on board. Definitely a great step in the right direction. Some good discussion started by Jeff Widman in conjunction w/Brad Shoening around the pypi cqlsh library: https://lists.apache.org/thread/sy3p2b2tncg1bk6x3r0r60y10dm6l18d. The proposed goal: When a Cassandra release happens, the build/release process automatically publishes cqlsh to https://pypi.org/project/cqlsh/. Looks like there's user interest and a CEP draft coming - thanks to both of you for working on this. These kinds of things make a big difference for our users. A discussion about what qualifies as an API w/regards to our tooling stretched long here: https://lists.apache.org/thread/72j5qfgbttjcmylhcmfq1ptboh641ns0. For good reason; it's a pretty thorny one where we have a lot of debt, and the topic keeps coming back like a zombie on slack every few months which means it's a pain-point and probably worth applying some energy to as a project community. My understanding of where we reached on the thread: 1. We want to be able to revise and evolve our user-facing output 2. We need stable alternatives that can be programmatically parsed if we want 1 3. We should version the output for all of that Also, typos in API's are like annoying little splinters stuck in our minds for eternity. So we should try and avoid those. Jakub Zytka has quite a lot of detailed thoughts and analyses about some assertions during reads that may be fixeable in the future with TCM (Transactional Cluster Metadata) coming into play. See the ML thread here: https://lists.apache.org/thread/zmq3ky65cp9orgcjhfcox08s49bzsc2g. Related jira tickets: CASSANDRA-18591: https://issues.apache.org/jira/browse/CASSANDRA-18591 CASSANDRA-18589: https://issues.apache.org/jira/browse/CASSANDRA-18589 Mick and a few of us have been grinding away at changing over the CI scripts for ASF CI to being in-tree and more generic. Well, really Mick has been grinding away and the peanut gallery (i.e. review squad) has just been sitting back and critiquing his work and providing ideas, as one does. Sorry Mick, and great work! ;) I started a thread discussing what our "definition of done" for a pre-commit CI system should be. Basically, we have a high level goal: tests run on pre-commit in any environment should give us high confidence they won't disrupt the ASF CI (i.e. introduce flakes). Thread is here: https://lists.apache.org/thread/cgmlgxknjl2s8f3wwlpkqn5xoc1dq582. There's a gdoc linked from there as well; on the whole it looks like we have a decent consensus there so I'll likely encode it in the wiki w/a link to the dev ML as "lazy consensus achieved" for future posterity. Berenguer Blasi is wrestling with some of the data structure format and optimization on disk checking into whether we have bytes to ditch due to lack of need on max dates or, as Benedict points out, to potentially just delta encode in even smaller space. ML list is here: https://lists.apache.org/thread/3jlhwgg6k6ckcg0x1jsfgf1gdn4n8zd9. Worth a read through and if you have the time and inclination, do engage. SAI. The promise of native indexes across your C* data... looks like Caleb, Mike, and the crew are bringing this forward: https://lists.apache.org/thread/of3zgtcoyrwggkz9w2jjv2qpkhh49pbt. This is a lovely update from Caleb just letting everyone know where they are in their work and how close they are to merging into 5.0. 5.0 really is going to be a pretty interesting release, eh? Great work everyone! *[Checking in on CI] * https://butler.cassandra.apache.org/#/ So since 5/30: 3.0: 7 -> 9 • Tthe last few were around 5. Looks a little jumpy; not sure what's up there. 3.11: 11 -> 16 • Had a recent bump up around June 21 where it looks likje our average hopped up from the 11 we had prior to 16. Probably worth looking into. 4.0: 14 -> 24 • This looks like a recent anomaly. Since we have multi-branch recent anomalies, I'd look to env first (which is one thing we're looking to improve with things like CASSANDRA-18133) 4.1: 4 -> 12 • Though recently as low as 0, 3 a couple weeks ago. Flakey flakes: https://butler.cassandra.apache.org/#/ci/upstream/compare/Cassandra-4.1/cassandra-4.1. trunk: 15 -> 16 • Yet again, recent spike anomaly. Was as low / consistent as 5-ish. Maybe average high single digits. So all in all? Not a terrible place to be entering the "freezing season", whenever that may land. We've held steady for over a quarter now. That said, we have SAI, maybe accord and TCM looking to merge, so there's some really big feature branches coming down the pike. Please make sure you do full post-commit suites (i.e. not just the pre-commit circle subset for instance) before merging things in; the last thing we need is a big blowout of flaky failing tests to hit we then need to grind through before we can release 5.0. *[What's been closed out] * QuickFilter remains accurate: https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=484&quickFilter=2278 Here's a JQL query that ought to get you the same if you want to look at it in the web interface: https://issues.apache.org/jira/issues/?filter=12352614&jql=project%20%3D%20cassandra%20AND%20(resolved%20%3E%202023-05-20%20AND%20resolution%20%3D%20%22Fixed%22)%20ORDER%20BY%20fixversion%20DESC%2C%20type%20desc *Closed since 2023-05-20** * *Missing versions:** * [Task](CASSANDRA-17429)Use only one image for all CircleCI builds; Switch to "latest" tag(e.dimitrova) [Improvement](CASSANDRA-15698)DOC - Review Download page(polandll) [Bug](CASSANDRA-15801)Tiny documentation fix in Architecture/Overview(polandll) [Bug](CASSANDRA-18541)AUTH requests use too much resources() [Bug](CASSANDRA-18678)CCM does not select Java version properly(jlewandowski) *5.x tasks:** * 5.x[Task](CASSANDRA-18494)Upgrade to lucene-core 9.7.0(jbellis) 5.x[Improvement](CASSANDRA-18398)CEP-25: Trie-indexed SSTable format(blambov) 5.x[Improvement](CASSANDRA-18345)Enable streaming SAI components as part of repair(pkolaczk) 5.x[Improvement](CASSANDRA-18515)Optimize Initial Concurrency Selection for Range Read Algorithm During SAI Queries(mike_tr_adamson) 5.x[Improvement](CASSANDRA-18490)Checksum per-SSTable and per-column SAI components after streaming(pkolaczk) 5.x[Improvement](CASSANDRA-18479)Add basic text tokenisation and analysis(mike_tr_adamson) 5.x[Improvement](CASSANDRA-18615)CREATE INDEX Modifications for Initial Release of SAI(maedhroz) *5.0 tasks:** * 5[Task](CASSANDRA-18453)Use WithProperties to ensure that system properties are handled(bernardo.botella) 5[Task](CASSANDRA-18519)CEP-15: (C*) Add notion of CommandsForRanges and make this durable in C*(dcapwell) 5[Task](CASSANDRA-18506)Deduplicate the MixedModeAvailability upgrade jvm-dtests(mck) 5[Task](CASSANDRA-17433)Revise use of pytz in Python >= 3.9(lmtrombone) 5[Task](CASSANDRA-18555)Expose decommission state to nodetool info(chovatia.jayd...@gmail.com) 5[Task](CASSANDRA-18285)[CircleCI] Add JDK 11 Java and Python upgrade tests in trunk(e.dimitrova) 5[Task](CASSANDRA-18467)Update generate-idea-files for J17(jakubzytka) 5[Task](CASSANDRA-18133)In-tree build scripts(mck) 5[Task](CASSANDRA-18645)Upgrade guava on trunk(e.dimitrova) *5.0 new features:** * 5[New Feature](CASSANDRA-18500)Add guardrail for partition size(adelapena) 5[New Feature](CASSANDRA-17194)Migrate thresholds for compaction to guardrails(adelapena) 5[New Feature](CASSANDRA-18613)Add support for vectors on UDFs(adelapena) *5.0 improvements:** * 5[Improvement](CASSANDRA-15046)Add a "history" command to cqlsh. Perhaps "show history"?(bschoeni) 5[Improvement](CASSANDRA-18544)Make cassandra-stress able to read all credentials from a file(smiklosovic) 5[Improvement](CASSANDRA-18561)Extend Accord MessageType with a side effect flag(aleksey) 5[Improvement](CASSANDRA-12972)Print header and statistics for cassandra-stress output with arbitrary frequency(smiklosovic) 5[Improvement](CASSANDRA-18566)Avoid unnecessary deserialization of terminal arguments when executing CQL functions(adelapena) 5[Improvement](CASSANDRA-18563)Convert AccordStateCache cache from write-through to write-back(aleksey) 5[Improvement](CASSANDRA-18305)Enhance nodetool compactionstats with existing MBean metrics(manish.c.ghildi...@gmail.com) 5[Improvement](CASSANDRA-18504)Added support for type VECTOR<type, dimension>(dcapwell) 5[Improvement](CASSANDRA-18639)Add duration to sstablemetadata(timothytu) 5[Improvement](CASSANDRA-18438)Refactor cloud snitches to get rid of duplicate code(smiklosovic) 5[Improvement](CASSANDRA-18617)Disable the deprecated keyspace/table thresholds and convert them to Guardrails(djatnieks) 5[Improvement](CASSANDRA-18659)Upgrade apache commons cli to 1.5.0(timothytu) 5[Improvement](CASSANDRA-18652)Improve vector value validation errors(adelapena) *5.0 bugs:** * 5[Bug](CASSANDRA-18569)Bti shouldn't be available in compatibility mode(bereng) 5[Bug](CASSANDRA-14227)Extend maximum expiration date(bereng) 5[Bug](CASSANDRA-17698)sstabledump errors when dumping data from index(maxwellguo) 5[Bug](CASSANDRA-18593)CircleCI: Add separate approval step for oa utests on separate workflows(adelapena) 5[Bug](CASSANDRA-18626)compaction_tombstone_warning_threshold and compaction_large_partition_warning_threshold cause deprecation warnings(adelapena) 5[Bug](CASSANDRA-18660)transient_replication_ring_test.py::TestTransientReplicationRing::test_bootstrap_and_cleanup(brandon.williams) *4.1 tickets:** * 4.1.3[Task](CASSANDRA-18603)Move checkstyle files into .build(bereng) 4.1.3[Improvement](CASSANDRA-18588)Slow builds due to checkstyle(bereng) 4.1.3[Bug](CASSANDRA-18338)Test failure: org.apache.cassandra.distributed.test.ByteBuddyExamplesTest.countTest(bereng) *4.0 tickets:** * 4.0.12[Bug](CASSANDRA-18658)Microbench tests fail on JDK11(marianne-manaog) 4.0.11[Improvement](CASSANDRA-18546)Remove unnecessary shuffling of GossipDigests in Gossiper#makeRandomGossipDigest(smiklosovic) 4.0.11[Improvement](CASSANDRA-18513)Limit the read size on the row level(jlewandowski) 4.0.11[Bug](CASSANDRA-18540)negotiatedProtocolMustBeAcceptedProtocolTest tests fail with "TLSv1.1 failed to negotiate" on JDK17(djatnieks) 4.0.11[Bug](CASSANDRA-18512)nodetool describecluster command is not showing correct Down count.(ranju) 4.0.11[Bug](CASSANDRA-18596)Assertion error when describing mv as table(masokol) 4.0.11[Bug](CASSANDRA-18560)Incorrect IP used for gossip across DCs with prefer_local=true(brandon.williams) 4.0.10[Improvement](CASSANDRA-18550)Improve nodetool enable{audit,fullquery}log, CVE-2023-30601(marcuse) 4.0.10[Improvement](CASSANDRA-18400)Nodetool info should report on the new networking cache(qannap) *3.11 tickets:** * 3.11.16[Task](CASSANDRA-18664)Move jflex from runtime to build dependencies(jlewandowski) 3.11.16[Improvement](CASSANDRA-18511)Add support for JMX in the in-jvm dtest framework(drohrer) 3.11.16[Bug](CASSANDRA-17818)Fix error message handling when trying to use CLUSTERING ORDER with non-clustering column(qannap) 3.11.16[Bug](CASSANDRA-18543)Waiting for gossip to settle does not wait for live endpoints(smiklosovic) 3.11.16[Bug](CASSANDRA-18346)Error Unknown column during deserialization missing keyspace and table name(manish.c.ghildi...@gmail.com) 3.11.16[Bug](CASSANDRA-18643)jackson-core vulnerability: CVE-2022-45688(brandon.williams) 3.11.16[Bug](CASSANDRA-18647)CASTing a float to decimal adds wrong digits(smiklosovic) 3.11.16[Bug](CASSANDRA-14204)Remove unrepaired SSTables from garbage collection when only_purge_repaired_tombstones is true to avoid AssertionError in nodetool garbagecollect(smiklosovic) *3.0 tickets:** * 3.0.x[Bug](CASSANDRA-18609)snappy-java vulnerability: CVE-2023-34453(brandon.williams) 3.0.30[New Feature](CASSANDRA-16555)Add support for AWS Ec2 IMDSv2(smiklosovic) 3.0.30[Improvement](CASSANDRA-17997)Improve git branch handling for CircleCI generate.sh(dchenbecker) 3.0.30[Bug](CASSANDRA-18553)Generate.sh -s param to skip autodetection of tests(bereng) 3.0.30[Bug](CASSANDRA-17302)Test Failure: dtest-offheap.topology_test.TestTopology.test_decommissioned_node_cant_rejoin(brandon.williams) 3.0.30[Bug](CASSANDRA-18575)Backport Cassandra-10508 Remove hard-coded SSL cipher suites and protocols(xgerman42) 3.0.30[Bug](CASSANDRA-18630)jackson-databind-2.13.2.2.jar vulnerability: CVE-2023-35116(brandon.williams) 3.0.30[Bug](CASSANDRA-18608)snappy-java vulnerability: CVE-2023-34455, CVE-2023-34454, CVE-2023-34453(brandon.williams) 3.0.30[Bug](CASSANDRA-12183)compaction_history system table does not capture all historical compaction sessions(smiklosovic) 3.0.30[Bug](CASSANDRA-18650)Upgrade owasp to 8.3.1(brandon.williams) 3.0.30[Bug](CASSANDRA-18562)guava vulnerability CVE-2023-2976(brandon.williams) 3.0.30[Bug](CASSANDRA-18649)netty-all vulnerability: CVE-2023-34462(brandon.williams) 3.0.29[Bug](CASSANDRA-18552)Debian packaging source should exclude git subdirectory(mck) *... 1.0? Oh My.** * 1.0.0[Improvement](CASSANDRA-18548)[Analytics] Add .asf.yaml file to the Cassandra Analytics repository(frankgh) 2.2.20[Bug](CASSANDRA-18558)remove dh_python use from debian packaging(brandon.williams) *N/A:** * NA[Task](CASSANDRA-18545)[Analytics] Abstract mTLS provisioning via a SecretsProvider(frankgh) NA[Task](CASSANDRA-18578)[Analytics] Circle CI Configuration for Cassandra Analytics(frankgh) NA[Task](CASSANDRA-18600)[Analytics] Add NOTICE.txt in Cassandra Analytics(frankgh) NA[Task](CASSANDRA-18629)BLOG - Clarify wording on 3.x EOL post(patrick) NA[New Feature](CASSANDRA-18605)Cassandra Analytics - Adding support for TTL & Timestamps for bulk writes(Jyothsnakonisa) NA[New Feature](CASSANDRA-18067)On-disk numeric index(mike_tr_adamson) NA[Improvement](CASSANDRA-18521)Unify CQLTester#waitForIndex and SAITester#waitForIndexQueryable(adelapena) NA[Improvement](CASSANDRA-18599)Cassandra Analytics - Upgrade to use JUnit 5(drohrer) NA[Improvement](CASSANDRA-18574)[Analytics] Fix example documentation(frankgh) NA[Improvement](CASSANDRA-18631)[Cassandra Analytics] Add Release Audit Tool (RAT) plugin(frankgh) NA[Improvement](CASSANDRA-18633)[Analytics] Add Caching of Node Settings to Improve Efficiency(semchyshyn) NA[Bug](CASSANDRA-18669)Concurrency is broken in the block balanced tree for queries(mike_tr_adamson) NA[Bug](CASSANDRA-18662)[Cassandra Analytics] Fix cassandra-analytics-core-example(frankgh) Looks like we have some tickets that could use fixversions and some that could use updated ones. I'm sure we'll catch these in the run up to 5.0 but if anyone wants to take a quick peek at the tickets you're responsible for that are a little off that'd be great. Phew. Novella complete. Thanks as always everyone! ~Josh
