Re: [DISCUSS] Harry in-tree

2023-11-25 Thread Josh McKenzie
Strong +1 to including harry in-tree and further, integrating a harry stress 
soak into our pre-commit and post-commit CI.

On Fri, Nov 24, 2023, at 5:10 PM, Alex Petrov wrote:
> Unfortunately my Harry talk got declined. Of course I’ll be happy to talk 
> about Harry and how it can be useful for contributors and about people’s 
> expectations. My talk is going to be about TCM again this time.
> 
> I will make sure examples are in place and are expressive by the summit.
> 
> On Fri, Nov 24, 2023, at 6:18 PM, Jeremy Hanna wrote:
>> I'm excited for Harry to come in-tree to improve the project stability and 
>> quality.  I know you're doing a talk at the Cassandra Summit about Harry to 
>> go over it.  If there's anything that can be done as part of this process to 
>> improve onboarding for Harry too, that would be great.  I'm just thinking 
>> about examples and things like that so people new to Harry can more easily 
>> write and run tests, test new features, and have a standard process for 
>> reporting findings.
>> 
>> Thanks Alex and all involved!
>> 
>> Jeremy
>> 
>>> On Nov 24, 2023, at 9:43 AM, Alex Petrov  wrote:
>>> 
>>> Hi everyone,
>>> 
>>> With TCM landed, there will be way more Harry tests in-tree: we are using 
>>> it for many coordination tests, and there's now a simulator test that uses 
>>> Harry. During development, Harry has allowed us to uncover and resolve 
>>> numerous elusive edge cases.
>>> 
>>> I had conversations with several folks, and wanted to propose to move 
>>> harry-core to Cassandra test tree. This will substantially 
>>> simplify/streamline co-development of Cassandra and Harry. With a new 
>>> HistoryBuilder API that has helped to find and trigger [1] [2] and [3], it 
>>> will also be much more approachable.
>>> 
>>> Besides making it easier for everyone to develop new fuzz tests, it will 
>>> also substantially lower the barrier to entry. Currently, debugging an 
>>> issue found by Harry involves a cumbersome process of rebuilding and 
>>> transferring jars between Cassandra and Harry, depending on which side you 
>>> modify. This not only hampers efficiency but also deters broader adoption. 
>>> By merging harry-core into the Cassandra test tree, we eliminate this 
>>> barrier.
>>> 
>>> Thank you,
>>> --Alex
>>> 
>>> [1] https://issues.apache.org/jira/browse/CASSANDRA-19011
>>> [2] https://issues.apache.org/jira/browse/CASSANDRA-18993
>>> [3] https://issues.apache.org/jira/browse/CASSANDRA-18932
> 


Re: CEP-21 - Transactional cluster metadata merged to trunk

2023-11-25 Thread Mick Semb Wever
Great work Sam, Alex & Marcus !



> There are about 15-20 flaky or failing tests in total, spread over several
> test jobs[2] (i.e. single digit failures in a few of these). We have filed
> JIRAs for the failures and are working on getting those fixed as a top
> priority. CASSANDRA-19055[3] is the umbrella ticket for this follow up work.
>
> There are also a number of improvements we will work on in the coming
> weeks, we will file JIRAs for those early next week and add them as
> subtasks to CASSANDRA-19055.
>


Can we get these tests temporarily annotated as skipped while all the
subtickets to 19055 are being worked on ?

As we have seen from CASSANDRA-18166 and CASSANDRA-19034 there's a lot of
overhead now on 5.0 tickets having to navigate around these failures in
trunk CI runs.

Also, we're still trying to figure out how to do repeated runs for a patch
so big… (the list of touched tests was too long for circleci, i need to
figure out what the limit is and chunk it into separate circleci configs) …
and it probably makes sense to wait until most of 19055 is done (or tests
are temporarily annotated as skipped).