[
https://issues.apache.org/jira/browse/CASSANDRA-16625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17339243#comment-17339243
]
Andres de la Peña edited comment on CASSANDRA-16625 at 5/4/21, 7:20 PM:
------------------------------------------------------------------------
{quote}One thing that I am worried about and I am wondering whether there is a
way to make our lives easier - under Artifacts we will have all the logs from
all the runs which is super helpful but currently I don't see any
identification of which logs are for which run... So in case of 3 fails in 1000
runs I am not sure how we can easily identify the logs we need. Any ideas?
Probably we can try to add some identification to the logs for example. WDYT?
{quote}
Good idea, I have added a {{passes}}/{{fails}} prefix to the artifacts' paths,
so it's easier to find the failing tests. The run number is also included in
the path. I have also added the per-test standard output to the saved
artifacts, so we don't have all the runner's output in a single gigantic file.
Here are a couple of example runs, you can see that the artifacts are saved in
directories such as {{logs/passes/}}, {{logs/fails/}}, etc.:
||ticket||config||run||
|CASSANDRA-16627|[diff|https://github.com/adelapena/cassandra/compare/16625-trunk..16625-trunk-run-16627]|[3359|https://app.circleci.com/pipelines/github/adelapena/cassandra/389/workflows/cd4d6b0a-1a66-4086-b173-b1e1303a11eb/jobs/3359]|
|CASSANDRA-16598|[diff|https://github.com/adelapena/cassandra/compare/16625-trunk..16625-trunk-run-16598]|[3346|https://app.circleci.com/pipelines/github/adelapena/cassandra/388/workflows/034b59a0-11ed-469f-bb24-d13922f8a942/jobs/3346]|
was (Author: adelapena):
bq. One thing that I am worried about and I am wondering whether there is a way
to make our lives easier - under Artifacts we will have all the logs from all
the runs which is super helpful but currently I don't see any identification of
which logs are for which run... So in case of 3 fails in 1000 runs I am not
sure how we can easily identify the logs we need. Any ideas? Probably we can
try to add some identification to the logs for example. WDYT?
Good idea, have added a {{passes}}/{{fails}} prefix to the artifacts' paths, so
it's easier to find the failing tests. I have also added the per-test standard
output to the saved artifacts. Here are a couple of example runs, you can see
that the artifacts are saved in directories such as {{junitxml/passes}},
{{junitxml/fails}}, etc.:
||ticket||config||run||
|CASSANDRA-16627|[diff|https://github.com/adelapena/cassandra/compare/16625-trunk..16625-trunk-run-16627]|[3359|https://app.circleci.com/pipelines/github/adelapena/cassandra/389/workflows/cd4d6b0a-1a66-4086-b173-b1e1303a11eb/jobs/3359]|
|CASSANDRA-16598|[diff|https://github.com/adelapena/cassandra/compare/16625-trunk..16625-trunk-run-16598]|[3346|https://app.circleci.com/pipelines/github/adelapena/cassandra/388/workflows/034b59a0-11ed-469f-bb24-d13922f8a942/jobs/3346]|
> Add a CircleCI job to run some tests repeatedly
> -----------------------------------------------
>
> Key: CASSANDRA-16625
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16625
> Project: Cassandra
> Issue Type: Task
> Components: CI
> Reporter: Andres de la Peña
> Assignee: Andres de la Peña
> Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I think it could be useful to have an optional CircleCI job to run some
> specific tests n times. That way, tickets could attach CircleCI runs showing
> that the changes don't make a certain ticket flaky or, conversely, that they
> fix a flaky test. Doing this systematically should mitigate the risk of
> introducing new flaky tests, and I guess it would be more convenient and easy
> to share than running the tests locally or on a private CI system.
> It would also be nice to have something similar in Jenkins, but I'm focusing
> this ticket on CircleCI because it's available also for non-committers, so
> assignees can run their tests before setting the tickets as ready for review.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
