[
https://issues.apache.org/jira/browse/CASSANDRA-17277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17495667#comment-17495667
]
Josh McKenzie edited comment on CASSANDRA-17277 at 2/21/22, 5:40 PM:
---------------------------------------------------------------------
bq. I don't get what the intent of this sentence is…
Butler appeared to be parsing test details whereas this script parses the "all"
panel on failures. Build 940 in butler didn't reflect the count of failures
consistent with the "All" panel, because they didn't match in Jenkins. This
wasn't intended as a slight on butler or a question about it's correctness,
more an observation that the details butler was linking to surfaced that
Jenkins appeared to have inconsistent test data details.
bq. This then becomes much more than a cache, it's an important store of
history.
Let's rename the /cache/ directory it creates to /history/ then. :)
bq. because it cannot fetch anything older than 30 builds that and subsequent
commits are going to get associated with past flakies
Hm. This combined w/the Jenkins flakes isn't ideal if we were to run this
script from jenkins agents. If we were to have a long-running centralized
location from which this runs with, for instance, the last successfully parsed
build per branch and step up one (the API for Jenkins gives us that), we could
just set it up to check on the hour and then update tickets.
I like the "run when build is done" flow if we had jenkins agents do it, but I
think having the longer-term test history is probably more important if we have
to choose between the two.
was (Author: jmckenzie):
bq. I don't get what the intent of this sentence is…
Butler appeared to be parsing test details whereas this script parses the "all"
panel on failures. Build 940 in butler didn't reflect the count of failures
consistent with the "All" panel, because they didn't match in Jenkins. This
wasn't intended as a slight on butler or a question about it's correctness,
more an observation that the details butler was linking too surfaced that
Jenkins appeared to have inconsistent test data details.
bq. This then becomes much more than a cache, it's an important store of
history.
Let's rename the /cache/ directory it creates to /history/ then. :)
bq. because it cannot fetch anything older than 30 builds that and subsequent
commits are going to get associated with past flakies
Hm. This combined w/the Jenkins flakes isn't ideal if we were to run this
script from jenkins agents. If we were to have a long-running centralized
location from which this runs with, for instance, the last successfully parsed
build per branch and step up one (the API for Jenkins gives us that), we could
just set it up to check on the hour and then update tickets.
I like the "run when build is done" flow if we had jenkins agents do it, but I
think having the longer-term test history is probably more important if we have
to choose between the two.
> Automate updating tickets with CI results after merge
> -----------------------------------------------------
>
> Key: CASSANDRA-17277
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17277
> Project: Cassandra
> Issue Type: Task
> Components: Build, CI
> Reporter: Josh McKenzie
> Assignee: Josh McKenzie
> Priority: Normal
>
> From [the wiki
> article|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280]
> {quote}ci-cassandra run bot updating ticket in JIRA w/state of test run for
> the SHA (JIRA Pending; to be linked)
> {quote}
> We already run CI for every commit (see
> [example|https://ci-cassandra.apache.org/job/Cassandra-trunk/]). The goal is
> to have automation that'll tie the results of a commit to the original JIRA
> ticket and update it automatically w/a comment indicating:
> * The results of the CI run (duration, pass, fail, # failures)
> * Any potential regressions in CI from the merge w/links to job history
> ([example|https://ci-cassandra.apache.org/job/Cassandra-trunk/912/testReport/junit/dtest.cqlsh_tests.test_cqlsh_copy/TestCqlshCopy/test_bulk_round_trip_with_timeouts/history/])
> From a workflow perspective we want to optimize for having as minimal
> friction as possible to see the impact of one's commit on ci-cassandra and
> rapidly verify if their change appears to have destabilized any tests on that
> infrastructure.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
