[
https://issues.apache.org/jira/browse/CASSANDRA-15583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17167880#comment-17167880
]
Berenguer Blasi commented on CASSANDRA-15583:
---------------------------------------------
Update: I expect CASSANDRA-15991 to go into review soon. After several days
adding UTs just for the smaller tools and just sanity checking their input args
on a happy path, I've come to realize this task is even bigger, fiddlier and
brittler than you can imagine:
* UT has been added for basic sanity checking params to not fail the tool.
Some broken or missing options in docs have been found. I intend to fix those
in an upcoming ticket.
* Just adding proper UT for params (boundary, poisoned, random,... params)
would entail _a lot_ of work as many tools work only on the happy path but
crash otherwise. So there is a lot of hidden cost fixing the tools as you add
new test cases. I.e. negative numbers for a 'size' and similar input
validation/guardrails
* If we got to have good params UT, then you'd have to focus on that tool
specifics and UT it as most have 0% coverage atm. That again is a huge task
bigger than the 2 before this one.
* Then there are the big ones: nodetool, cassandra-stress, cqlsh... which
again are on their own bigger than the 3 before this one.
Currently 15583 seeds and unblocks tooling testing. But providing good UT for
all tools would imply a multi-month effort and I don't dislike anybody that
much to whish him work on that after my recent experience lol.
Here is what I intend to do:
# Fix the broken options and docs
# Create tickets to UT each tool and flag them LHF if applicable imo
# Create tickets for the bigger tools
With that we can scope tickets in/out at a doable pace + newcomers can help
with the LHF ones. That leaves 15583 more as a seeding/unblocking & surface
area discovery ticket for UT tooling than actually _solving_ it. So I don't
know what was the original spirit of the ticket and if it makes sense to close
it or not?
See CASSANDRA-15991 PR for details. I'll ping when it's ready for review so we
can merge.
> 4.0 quality testing: Tooling, Bundled and First Party
> -----------------------------------------------------
>
> Key: CASSANDRA-15583
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15583
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest, Test/unit
> Reporter: Josh McKenzie
> Assignee: Berenguer Blasi
> Priority: Normal
> Fix For: 4.0-beta
>
>
> Reference [doc from
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
> for context.
> *Shepherd: Sam Tunnicliffe*
> Test plans should cover bundled first-party tooling and CLIs such as
> nodetool, cqlsh, and new tools supporting full query and audit logging
> (CASSANDRA-13983, CASSANDRA-12151).
> ||Tool||UX test||UT coverage||dtest coverage||Comments||
> |Nodetool|(x)|(x)|(!)|Deserves it's own ticket. Not all the sub commands are
> tested. Dtest also test nodetool as a side effect|
> |Cqlsh|(x)|(x)|(!)|Deserves it's own ticket|
> |Cassandra-stress|(x)|(x)|(x)|Some UT. Deserves it's own ticket|
> |debug-cql|(x)|(x)|(x)|Deserves it's own ticket |
> |fqltool|(x)|(/)|(!)|Deserves it's own ticket |
> |auditlogviewer|(/)|(!)|(!)|UX tests: C-15991. Docs missing option -i|
> |*Sstable utilities*| | | | |
> |sstabledump|(/)|(x)|(!)|UX tests: C-15991|
> |sstableexpiredblockers|(/)|(x)|(!)|UX tests: C-15991|
> |sstablelevelreset|(/)|(x)|(!)|UX tests: C-15991|
> |sstableloader|(x)|(x)|(!)|Needs it's own ticket probably|
> |sstablemetadata|(/)|(x)|(x)|Ran in dtests, no dedicated test, UX tests:
> C-15991, missing options in docs, brittle args parsing|
> |sstableofflinerelevel|(/)|(x)|(!)|UX tests: C-15991, brittle args parsing|
> |sstablerepairedset|(/)|(x)|(x)|Ran in dtests, no dedicated test, UX tests:
> C-15991, brittle args parsing|
> |sstablescrub|(/)|(x)|(!)|UX tests: C-15991 but missing options in docs|
> |sstablesplit|(/)|(x)|(!)|UX tests: C-15991|
> |sstableupgrade|(/)|(x)|(!)|UX tests: C-15991|
> |sstableutil|(/)|(x)|(!)|UX tests: C-15991|
> |sstableverify|(/)|(x)|(!)|UX tests: C-15991 but missing options in docs +
> broken token_range option|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
