[
https://issues.apache.org/jira/browse/CASSANDRA-17277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493385#comment-17493385
]
Josh McKenzie commented on CASSANDRA-17277:
-------------------------------------------
Ok, got something working locally with the following usage ergonomics:
{code:java}
usage: jenkins-jira-integration.py [-h] [--jenurl JENURL] --jenuser JENUSER
--jenpass JENPASS --jiraurl JIRAURL --jirauser JIRAUSER --jirapass JIRAPASS
--branch b
[--buildnum n] [--verbose]
Parse Jenkins build output and update JIRA tickets with results for a single
branch
required arguments:
--jenurl JENURL Jenkins server url
--jenuser JENUSER JIRA server url
--jenpass JENPASS JIRA username
--jiraurl JIRAURL JIRA server url
--jirauser JIRAUSER JIRA username
--jirapass JIRAPASS JIRA password
--branch b Branch Versions to pull Jenkins results for
--buildnum n Build number to process for CI status
optional arguments:
--verbose Verbose logging
{code}
The usage of the above can be triggered upon completion of a CI run in Jenkins
that passes in the branch and number of the build; the rest of it is taken care
of by the script.
The current data and format of results it'd post into a JIRA comment (I ran
this against build 940 on trunk; arbitrary pick):
-----------------------------------------------------------------------
[CI Results]
Branch: trunk, build number: 940
jenkins url: [https://ci-cassandra.apache.org/job/Cassandra-trunk/940/]
JIRA: 17352
commit url:
[https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=5c9ba06dd31157cd224af2cec75521fefe2c9883]
affected paths:
* src/java/org/apache/cassandra/config/DatabaseDescriptor.java
* src/java/org/apache/cassandra/cql3/functions/UDFunction.java
* src/java/org/apache/cassandra/config/Config.java
* src/java/org/apache/cassandra/cql3/functions/ThreadAwareSecurityManager.java
Build Result: UNSTABLE
Passing Tests: 42908
Failing Tests: 11
||Test|Failures|JIRA|
|dtest.cqlsh_tests.test_cqlsh_copy.TestCqlshCopy.test_explicit_column_order_reading|1
of 17|[No JIRA
found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]|
|org.apache.cassandra.distributed.upgrade.CompactStorageUpgradeTest.compactStorageUpgradeTest|4
of 17|[No JIRA
found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]|
|dtest.replica_side_filtering_test.TestSecondaryIndexes.test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions|1
of 17|[No JIRA
found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]|
|dtest.snapshot_test.TestArchiveCommitlog.test_archive_commitlog_point_in_time_with_active_commitlog|1
of 17|[No JIRA
found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]|
|dtest.repair_tests.incremental_repair_test.TestIncRepair.test_repaired_tracking_with_partition_deletes|1
of 17|[No JIRA
found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]|
|dtest.paging_test.TestPagingDatasetChanges.test_data_change_impacting_later_page|1
of 17|[No JIRA
found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]|
|dtest.user_functions_test.TestUserFunctions.test_udf_with_udt|1 of 17|[No JIRA
found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]|
|org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testNonblockingShouldMaintainSteadyDiskUsage|13
of 17|CASSANDRA-17233|
For an engineer that worked on a ticket (in this case CASSANDRA-17352), the
implication is that special attention should be paid to any test failure with
only 1 failure in the last N builds as this is the first time we've seen it so
it could have been caused by this change. Notably, [the butler run for
940|https://butler.cassandra.apache.org/#/ci/upstream/compare/Cassandra-trunk/trunk]
doesn't show the {{TestArchiveCommitlog}} failure, however in Jenkins if you
go to all test failure results for the build [you see it
there|https://ci-cassandra.apache.org/job/Cassandra-trunk/940/testReport/]
The above output is created by taking the passed in build number and walking
back (and locally caching) the results of all valid previous builds. This local
data is kept in a simple JSON file per branch which we then use for subsequent
calculation of the # of runs we've seen a given test in and the # of failures;
this is significantly faster (and less offensive) than repeatedly downloading
50+M of text data per build.
The inference of the Jira column is based on a query with summary ~ the last 2
tokens of the test name . delimited. If it finds more than 1 it should link to
the same open test failures kanban board as linked when no Jira ticket is
found. This functionality is of course limited to our consistency in having at
least the last 2 tokens of a test failure name in the ticket summary, but
that's standard operating procedure for the build lead role so shouldn't prove
too brittle. In the event we have 2 failures it'll just redirect to the kanban
board; some developer work required but minimal.
I'd _like_ to be able to have the test name link to the jenkins server for the
test history, but that requires parsing out which test suite a given test comes
from by its name alone; relatively trivial but something I could take as a
follow-up to tinker with.
And last but not least, if this Jira inference proves consistent we could
augment this script to automatically create Jira tickets upon finding test
failures for which we don't yet have tickets.
> Automate updating tickets with CI results after merge
> -----------------------------------------------------
>
> Key: CASSANDRA-17277
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17277
> Project: Cassandra
> Issue Type: Task
> Reporter: Josh McKenzie
> Assignee: Josh McKenzie
> Priority: Normal
>
> From [the wiki
> article|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280]
> {quote}ci-cassandra run bot updating ticket in JIRA w/state of test run for
> the SHA (JIRA Pending; to be linked)
> {quote}
> We already run CI for every commit (see
> [example|https://ci-cassandra.apache.org/job/Cassandra-trunk/]). The goal is
> to have automation that'll tie the results of a commit to the original JIRA
> ticket and update it automatically w/a comment indicating:
> * The results of the CI run (duration, pass, fail, # failures)
> * Any potential regressions in CI from the merge w/links to job history
> ([example|https://ci-cassandra.apache.org/job/Cassandra-trunk/912/testReport/junit/dtest.cqlsh_tests.test_cqlsh_copy/TestCqlshCopy/test_bulk_round_trip_with_timeouts/history/])
> From a workflow perspective we want to optimize for having as minimal
> friction as possible to see the impact of one's commit on ci-cassandra and
> rapidly verify if their change appears to have destabilized any tests on that
> infrastructure.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
