[jira] [Commented] (CASSANDRA-17043) CircleCI dtest multiplexer with MIDRES needs more resources
[
https://issues.apache.org/jira/browse/CASSANDRA-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430552#comment-17430552
]
Ekaterina Dimitrova commented on CASSANDRA-17043:
-
Ha... I am sure I saw it and I had the impression it was changed on all
branches with the idea that they need more resources but seems I didn't see it
correctly.
Anyway, 25 with the docs advice for people should be good enough. Thanks!
> CircleCI dtest multiplexer with MIDRES needs more resources
> ---
>
> Key: CASSANDRA-17043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17043
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Low
> Fix For: 3.0.26, 3.11.12, 4.0.2, 4.1
>
>
> The CircleCI jobs for regular dtests jobs have more resources in MIDRES,
> which is necessary for some dtests to reliably success. However, the dtest
> multiplexer uses the same resources for LOWRES and MIDRES.
> I think that the dtest multiplexer should always use the same resources as
> the regular dtests. Using too small resources in the multiplexer can lead to
> failures that don't reproduce in the regular dtest jobs, like the one we
> found in
> [CASSANDRA-16334|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> when trying to repeatedly run a resource-hungry dtest, or like [this other
> one|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> while running {{test_network_topology}}.
> This happens because I forgot to update the diff patch when adding the
> multiplexer. This doesn't affect HIGHRES because in that case the patch
> changes the configuration of the test executor, while in MIDRES a new
> executor is defined.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17043) CircleCI dtest multiplexer with MIDRES needs more resources
[
https://issues.apache.org/jira/browse/CASSANDRA-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430471#comment-17430471
]
Andres de la Peña commented on CASSANDRA-17043:
---
Thanks for the review :)
Committed to 3.0 as
[ee6cd06afb66317212d681117d460afebf1ceb31|https://github.com/apache/cassandra/commit/ee6cd06afb66317212d681117d460afebf1ceb31]
and merged to
[3.11|https://github.com/apache/cassandra/commit/7e95c92d00df83d8333852e37813a871a1aada93],
[4.0|https://github.com/apache/cassandra/commit/72f3b7901879aba902f92b45215a25ee130ff057]
and
[trunk|https://github.com/apache/cassandra/commit/3a950b45c321e051a9744721408760c568c05617].
There was a little typo on 3.0/MIDRES wrongly changing the parallelism for
{{repeated_upgrade_dtest}} on 3.0/MIDRES to 100, while we want to keep the
original 25. I have fixed it on commit.
> CircleCI dtest multiplexer with MIDRES needs more resources
> ---
>
> Key: CASSANDRA-17043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17043
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Low
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x
>
>
> The CircleCI jobs for regular dtests jobs have more resources in MIDRES,
> which is necessary for some dtests to reliably success. However, the dtest
> multiplexer uses the same resources for LOWRES and MIDRES.
> I think that the dtest multiplexer should always use the same resources as
> the regular dtests. Using too small resources in the multiplexer can lead to
> failures that don't reproduce in the regular dtest jobs, like the one we
> found in
> [CASSANDRA-16334|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> when trying to repeatedly run a resource-hungry dtest, or like [this other
> one|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> while running {{test_network_topology}}.
> This happens because I forgot to update the diff patch when adding the
> multiplexer. This doesn't affect HIGHRES because in that case the patch
> changes the configuration of the test executor, while in MIDRES a new
> executor is defined.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17043) CircleCI dtest multiplexer with MIDRES needs more resources
[
https://issues.apache.org/jira/browse/CASSANDRA-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430264#comment-17430264
]
Ekaterina Dimitrova commented on CASSANDRA-17043:
-
I just checked the CI results - seems that except flaky tests or lack of enough
resources with lower config, the things were run as per the expectations.
The config is applied as per what we agreed.
I tried the patches locally and everything looks as expected to me.
Thank you! :)
> CircleCI dtest multiplexer with MIDRES needs more resources
> ---
>
> Key: CASSANDRA-17043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17043
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Low
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x
>
>
> The CircleCI jobs for regular dtests jobs have more resources in MIDRES,
> which is necessary for some dtests to reliably success. However, the dtest
> multiplexer uses the same resources for LOWRES and MIDRES.
> I think that the dtest multiplexer should always use the same resources as
> the regular dtests. Using too small resources in the multiplexer can lead to
> failures that don't reproduce in the regular dtest jobs, like the one we
> found in
> [CASSANDRA-16334|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> when trying to repeatedly run a resource-hungry dtest, or like [this other
> one|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> while running {{test_network_topology}}.
> This happens because I forgot to update the diff patch when adding the
> multiplexer. This doesn't affect HIGHRES because in that case the patch
> changes the configuration of the test executor, while in MIDRES a new
> executor is defined.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17043) CircleCI dtest multiplexer with MIDRES needs more resources
[
https://issues.apache.org/jira/browse/CASSANDRA-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430062#comment-17430062
]
Andres de la Peña commented on CASSANDRA-17043:
---
Mentioning parallelism to the wiki page is a great idea, in the end that's
something that can be tuned depending on the particular test that is be run.
The process to edit the config is quite manual and having 12 different versions
of the config file it's quite likely that I have made a mistake, so I'd thank
you if you take a look at them :)
> CircleCI dtest multiplexer with MIDRES needs more resources
> ---
>
> Key: CASSANDRA-17043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17043
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Low
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x
>
>
> The CircleCI jobs for regular dtests jobs have more resources in MIDRES,
> which is necessary for some dtests to reliably success. However, the dtest
> multiplexer uses the same resources for LOWRES and MIDRES.
> I think that the dtest multiplexer should always use the same resources as
> the regular dtests. Using too small resources in the multiplexer can lead to
> failures that don't reproduce in the regular dtest jobs, like the one we
> found in
> [CASSANDRA-16334|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> when trying to repeatedly run a resource-hungry dtest, or like [this other
> one|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> while running {{test_network_topology}}.
> This happens because I forgot to update the diff patch when adding the
> multiplexer. This doesn't affect HIGHRES because in that case the patch
> changes the configuration of the test executor, while in MIDRES a new
> executor is defined.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17043) CircleCI dtest multiplexer with MIDRES needs more resources
[
https://issues.apache.org/jira/browse/CASSANDRA-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430058#comment-17430058
]
Ekaterina Dimitrova commented on CASSANDRA-17043:
-
I agree with everything said, the point for not exhausting all resources of
your organization is a good one. I don't think there is perfect answer but what
you suggest sounds reasonable. What I can suggest and add to the wiki page is
advise to people around parallelism so they can make an informed decision
whether to use the default one or change it. What do you think?
I also looked at the current patches you posted and they look reasonable to me.
I can test also the patches later today. Thank you!
> CircleCI dtest multiplexer with MIDRES needs more resources
> ---
>
> Key: CASSANDRA-17043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17043
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Low
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x
>
>
> The CircleCI jobs for regular dtests jobs have more resources in MIDRES,
> which is necessary for some dtests to reliably success. However, the dtest
> multiplexer uses the same resources for LOWRES and MIDRES.
> I think that the dtest multiplexer should always use the same resources as
> the regular dtests. Using too small resources in the multiplexer can lead to
> failures that don't reproduce in the regular dtest jobs, like the one we
> found in
> [CASSANDRA-16334|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> when trying to repeatedly run a resource-hungry dtest, or like [this other
> one|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> while running {{test_network_topology}}.
> This happens because I forgot to update the diff patch when adding the
> multiplexer. This doesn't affect HIGHRES because in that case the patch
> changes the configuration of the test executor, while in MIDRES a new
> executor is defined.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17043) CircleCI dtest multiplexer with MIDRES needs more resources
[
https://issues.apache.org/jira/browse/CASSANDRA-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430051#comment-17430051
]
Andres de la Peña commented on CASSANDRA-17043:
---
{quote}In that sense it makes sense to me to raise the parallelism for the
MIDRES multiplexer for the upgrade in-jvm tests for example. Why ? Because the
whole suite is 10-20 test classes, but the default run in a loop which I would
assume most people will run is 100. So if we generalize this means to me 10
times more time for execution if we think that most tests need similar amount
of time to run(do they?).
{quote}
I think that the amount of time to run tests varies wildly even for the same
type of tests. Not only we have very different tests, but we can also multiplex
either a method or its containing suite. If the suite has ten methods, that run
will take ten times more time. So I think it's difficult to adjust parallelism
for an expected run time if we don't know what test is going to be repeated,
differently to the relative fixed sets of tests that we have for the standard
jobs.
The current parallelisms of 4/25/100 for low/mid/high resources save some of
the overhead of starting new runners in low and mid configs, although I don't
know how noticeable would it be in practice. Another reason to not use a very
high parallelism is that I understand that the maximum number of concurrent
runners is limited per organization, so a job with a very high parallelism can
produce starvation in other users. I understand that when users with access to
high resources choose to run with low/mid config they are not in a big hurry
and they can wait a bit longer in order to don't exhaust the resources for
other users. Having a very high parallelism in these configs could make this
type of not-invasive runs more difficult to do.
That said, I have no clue about what parallelism would be better for the
average case, although the current values have been good to me by now. I'd be
happy to increase the parallelism of {{repeated_dtest}} if we find it too low,
what value for what config would you suggest?
Here are the updated patches with some multiplexer runs:
||Branch||CI low||CI mid||CI high||
|[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...adelapena:17043-3.0]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1057/workflows/cd90268d-8af7-4d8c-9522-a277398acdc4]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1051/workflows/2550a673-5fc2-4748-b1c6-1ce56b5b2002]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1048/workflows/9d8a08dd-0db3-4a9e-911f-eb64a87741ff]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...adelapena:17043-3.11]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1055/workflows/2412eeca-2a58-424c-9986-d6fb71cd13d4]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1054/workflows/fa78a6e8-be03-4ea8-9169-f92b03cc2a54]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1050/workflows/a7f29c05-ea57-40dd-84dd-e695514ca392]|
|[4.0|https://github.com/apache/cassandra/compare/cassandra-4.0...adelapena:17043-4.0]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1056/workflows/08393fb6-17db-4d91-b3ed-5d7800c56dc1]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1056/workflows/23eeb750-e163-406c-9430-aa37e67b0717]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1053/workflows/15d171cd-cc45-4718-ac83-90710c725097]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1053/workflows/d5c4ef81-04d0-4081-8a9c-f5f1753debd8]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1049/workflows/3047dfae-3b8f-4716-8c18-db4a2fcc8fd5]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1049/workflows/7de5327d-360c-407d-8ab6-0b80fb58ddaf]|
|[trunk|https://github.com/apache/cassandra/compare/trunk...adelapena:17043-trunk]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1060/workflows/34a75acf-e786-4b6f-a4c6-9d878e7f1351]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1060/workflows/51beffb0-2ccc-4ce0-b7e4-0cabac0ac223]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1061/workflows/4842863d-b324-4e0f-a95f-a14ec1fb66d4]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1061/workflows/09bef172-f992-4ab5-b945-97966246a927]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1062/workflows/eff4b3c0-1a90-440a-a432-4ebabe716ff4]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1062/workflows/8a5d32ba-eef9-4008-9d03-3f9cd59115f5]|
Of course the running times of the multiplexer jobs above would have been very
different if I had chosen different tests to be run repeatedly.
> CircleCI dtest multiplexer with MIDRES
[jira] [Commented] (CASSANDRA-17043) CircleCI dtest multiplexer with MIDRES needs more resources
[
https://issues.apache.org/jira/browse/CASSANDRA-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429788#comment-17429788
]
Ekaterina Dimitrova commented on CASSANDRA-17043:
-
{quote}I understand that the parallelisms for the regular test jobs are based
on the number of tests in that job, while the parallelism for the multiplexer
jobs should be based on the number of repetitions, which defaults to 100.
{quote}
Parallelism leads to shorter time for tests execution so what I meant Is that
for the regular job run of all unit tests, python dtests etc we made extensive
testing to come up with a good tradeoff between time to run them and used
CircleCI credits. We can't really make the same testing to check in a loop
every test how much time it needs but we can approximate in general what would
be an expected time to run for 100 tests of certain type on top of the minimum
requirements for resources for those tests to run. About LOWRES config - there
is a limit for the free tier anyway so I guess people will get limited anyway.
And my understanding for the HIGHRES configuration is that it just exercises
all resources and gets the quickest CI runs possible.
In that sense it makes sense to me to raise the parallelism for the MIDRES
multiplexer for the upgrade in-jvm tests for example. Why ? Because the whole
suite is 10-20 test classes, but the default run in a loop which I would assume
most people will run is 100. So if we generalize this means to me 10 times more
time for execution if we think that most tests need similar amount of time to
run(do they?).
The proposed changes look good to me in general but we might need to raise the
python dtests parallelism as I know they require a lot of resources. Can we run
some of the classes to see how that will go, for a direction? Does this make
sense?
> CircleCI dtest multiplexer with MIDRES needs more resources
> ---
>
> Key: CASSANDRA-17043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17043
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Low
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x
>
>
> The CircleCI jobs for regular dtests jobs have more resources in MIDRES,
> which is necessary for some dtests to reliably success. However, the dtest
> multiplexer uses the same resources for LOWRES and MIDRES.
> I think that the dtest multiplexer should always use the same resources as
> the regular dtests. Using too small resources in the multiplexer can lead to
> failures that don't reproduce in the regular dtest jobs, like the one we
> found in
> [CASSANDRA-16334|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> when trying to repeatedly run a resource-hungry dtest, or like [this other
> one|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> while running {{test_network_topology}}.
> This happens because I forgot to update the diff patch when adding the
> multiplexer. This doesn't affect HIGHRES because in that case the patch
> changes the configuration of the test executor, while in MIDRES a new
> executor is defined.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17043) CircleCI dtest multiplexer with MIDRES needs more resources
[
https://issues.apache.org/jira/browse/CASSANDRA-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429469#comment-17429469
]
Andres de la Peña commented on CASSANDRA-17043:
---
I understand that the parallelisms for the regular test jobs are based on the
number of tests in that job, while the parallelism for multiplexer jobs should
be based on the number of repetitions, which defaults to 100.
I think that we can do is simply use dedicated executors for the multiplexer
jobs, so they can use the same resource class as the original job, but with a
different parallelism. I gave it a go in [the branch for
3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...adelapena:17043-3.0],
and I'll update the other branches if the approach looks good.
The defaults parallelisms for the multiplexer jobs are 4 for lowres, 25 for
midres and 100 for highres.
> CircleCI dtest multiplexer with MIDRES needs more resources
> ---
>
> Key: CASSANDRA-17043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17043
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Low
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x
>
>
> The CircleCI jobs for regular dtests jobs have more resources in MIDRES,
> which is necessary for some dtests to reliably success. However, the dtest
> multiplexer uses the same resources for LOWRES and MIDRES.
> I think that the dtest multiplexer should always use the same resources as
> the regular dtests. Using too small resources in the multiplexer can lead to
> failures that don't reproduce in the regular dtest jobs, like the one we
> found in
> [CASSANDRA-16334|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> when trying to repeatedly run a resource-hungry dtest, or like [this other
> one|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> while running {{test_network_topology}}.
> This happens because I forgot to update the diff patch when adding the
> multiplexer. This doesn't affect HIGHRES because in that case the patch
> changes the configuration of the test executor, while in MIDRES a new
> executor is defined.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17043) CircleCI dtest multiplexer with MIDRES needs more resources
[
https://issues.apache.org/jira/browse/CASSANDRA-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429299#comment-17429299
]
Andres de la Peña commented on CASSANDRA-17043:
---
Oh, I totally forgot about the multiplexer jobs for upgrade tests! I understand
that those jobs should use exactly the same resource class as the equivalent
job for regular upgrade tests. I have updated the patches and config files to
use the same executors as the corresponding jobs:
||Branch||
|[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...adelapena:17043-3.0]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...adelapena:17043-3.11]|
|[4.0|https://github.com/apache/cassandra/compare/cassandra-4.0...adelapena:17043-4.0]|
|[trunk|https://github.com/apache/cassandra/compare/trunk...adelapena:17043-trunk]|
However those executors have a quite low parallelism, which might be enough for
running the relatively small set of regular upgrade jobs but that can be too
low for the repeated runs of a specific test. Perhaps the multiplexer jobs
should use separate, dedicated executors with the same resource class and more
parallelism? Or could we just increase the parallelism of the current
executors? wdyt?
> CircleCI dtest multiplexer with MIDRES needs more resources
> ---
>
> Key: CASSANDRA-17043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17043
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Low
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x
>
>
> The CircleCI jobs for regular dtests jobs have more resources in MIDRES,
> which is necessary for some dtests to reliably success. However, the dtest
> multiplexer uses the same resources for LOWRES and MIDRES.
> I think that the dtest multiplexer should always use the same resources as
> the regular dtests. Using too small resources in the multiplexer can lead to
> failures that don't reproduce in the regular dtest jobs, like the one we
> found in
> [CASSANDRA-16334|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> when trying to repeatedly run a resource-hungry dtest, or like [this other
> one|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> while running {{test_network_topology}}.
> This happens because I forgot to update the diff patch when adding the
> multiplexer. This doesn't affect HIGHRES because in that case the patch
> changes the configuration of the test executor, while in MIDRES a new
> executor is defined.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17043) CircleCI dtest multiplexer with MIDRES needs more resources
[
https://issues.apache.org/jira/browse/CASSANDRA-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17429073#comment-17429073
]
Ekaterina Dimitrova commented on CASSANDRA-17043:
-
Good catch!
{quote}I think that the dtest multiplexer should always use the same resources
as the regular dtests.
{quote}
I was looking into the 3.11 MIDRES and I think we need to update also the patch
for _repeated_jvm_upgrade_dtests_ and _repeated_upgrade_dtest_ for the same
reason. What do you think?
> CircleCI dtest multiplexer with MIDRES needs more resources
> ---
>
> Key: CASSANDRA-17043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17043
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Low
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x
>
>
> The CircleCI jobs for regular dtests jobs have more resources in MIDRES,
> which is necessary for some dtests to reliably success. However, the dtest
> multiplexer uses the same resources for LOWRES and MIDRES.
> I think that the dtest multiplexer should always use the same resources as
> the regular dtests. Using too small resources in the multiplexer can lead to
> failures that don't reproduce in the regular dtest jobs, like the one we
> found in
> [CASSANDRA-16334|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> when trying to repeatedly run a resource-hungry dtest, or like [this other
> one|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> while running {{test_network_topology}}.
> This happens because I forgot to update the diff patch when adding the
> multiplexer. This doesn't affect HIGHRES because in that case the patch
> changes the configuration of the test executor, while in MIDRES a new
> executor is defined.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17043) CircleCI dtest multiplexer with MIDRES needs more resources
[
https://issues.apache.org/jira/browse/CASSANDRA-17043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428902#comment-17428902
]
Andres de la Peña commented on CASSANDRA-17043:
---
This simple patches modify the patch file that we use to generate the MIDRES
CircleCI config file, so the dtest multiplexer jobs use the same resources as
the regular dtests jobs:
||patch||CI||
|[3.0|https://github.com/adelapena/cassandra/commit/3acd025e8f4f8396b95a3231275dc6b778cae9aa]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1030/workflows/deb0d4c5-3d2a-423a-93b7-5ad1bc692f78]|
|[3.11|https://github.com/adelapena/cassandra/commit/ca4a19d89380f46dd29d630ba7b5c6c25696c729]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1028/workflows/9309e88e-7386-4cac-871b-0a7641b64b96]|
|[4.0|https://github.com/adelapena/cassandra/commit/8d385d7462417a44d04cfc30c48fe5c6066a60ac]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1029/workflows/db1b57c8-4166-4d6e-a528-1949811e7565]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1029/workflows/bbdc1224-1bae-4b33-bf2e-c1df344e1146]|
|[trunk|https://github.com/adelapena/cassandra/commit/3973e410d34cb0d003fc74014b4b3bee8a165a21]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1027/workflows/ee7e4ffd-3eec-49fd-ad55-028731501811]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1027/workflows/eaa21c86-b148-45df-b01a-d100175ba8ca]|
[~edimitrova] could you please review?
> CircleCI dtest multiplexer with MIDRES needs more resources
> ---
>
> Key: CASSANDRA-17043
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17043
> Project: Cassandra
> Issue Type: Task
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> The CircleCI jobs for regular dtests jobs have more resources in MIDRES,
> which is necessary for some dtests to reliably success. However, the dtest
> multiplexer uses the same resources for LOWRES and MIDRES.
> I think that the dtest multiplexer should always use the same resources as
> the regular dtests. Using too small resources in the multiplexer can lead to
> failures that don't reproduce in the regular dtest jobs, like the one we
> found in
> [CASSANDRA-16334|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> when trying to repeatedly run a resource-hungry dtest, or like [this other
> one|https://app.circleci.com/pipelines/github/adelapena/cassandra/1020/workflows/63908694-e4d7-40b1-9418-7a4b87826233/jobs/9422]
> while running {{test_network_topology}}.
> This happens because I forgot to update the diff patch when adding the
> multiplexer. This doesn't affect HIGHRES because in that case the patch
> changes the configuration of the test executor, while in MIDRES a new
> executor is defined.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
