[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428766#comment-17428766
 ] 

Andres de la Peña commented on CASSANDRA-16882:
---

Committed to 3.0 as 
[efc27d693f325814ef8d502844ea31a431d8a84e|https://github.com/apache/cassandra/commit/efc27d693f325814ef8d502844ea31a431d8a84e]
 and merged to 
[3.11|https://github.com/apache/cassandra/commit/733c72b98ffb8808f9e7b61c0722d6a1d03cc951],
 
[4.0|https://github.com/apache/cassandra/commit/2da274787fb704170c14bf430b6396eabb50ce61]
 and 
[trunk|https://github.com/apache/cassandra/commit/31b2d133df486e90862ee5e365cbb4a00462e2ea].

Thanks for the reviews and feedback.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427307#comment-17427307
 ] 

Andres de la Peña commented on CASSANDRA-16882:
---

Agree, the current patch doesn't run anything at all unless jobs are approved. 
I think we are ready to commit.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427288#comment-17427288
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16882:
-

I am also +1 and I personally agree with [~stefan.miklosovic]. After all 
sometimes we just push code at the end of the day just to be sure we have 
backup of our ongoing work.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Stefan Miklosovic (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427026#comment-17427026
 ] 

Stefan Miklosovic commented on CASSANDRA-16882:
---

I / we at our company would appreciate if all was optional. We are pushing a 
lot and it does not mean we want to build it every time doing so. Maybe 1 out 
of 10 pushes is the actual commit we want to build.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Berenguer Blasi (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426049#comment-17426049
 ] 

Berenguer Blasi commented on CASSANDRA-16882:
-

I am happy to try this. Seems like an improvement to me and we can readjust 
later if needed. +1 from me, let's see what Ekaterina sees as well as she's 
been involved a lot as well.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425744#comment-17425744
 ] 

Andres de la Peña commented on CASSANDRA-16882:
---

Here are the patches for all the branches:
||patch||CI||
|[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...adelapena:16882-option-7-3.0-v05]|[workflows|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-3.0-v05]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...adelapena:16882-option-7-3.11-v05]|[workflows|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-3.11-v05]|
|[4.0|https://github.com/apache/cassandra/compare/cassandra-4.0...adelapena:16882-option-7-4.0-v05]|[workflows|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-4.0-v05]|
|[trunk|https://github.com/apache/cassandra/compare/trunk...adelapena:16882-option-7-trunk-v05]|[workflows|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-trunk-v05]|

I have modified the {{java8_separate_tests}}/{{java11_separate_tests}} 
workflows to have an additional approval step for the build. With this change 
commits and pushes don't run anything at all until manually started. It adds an 
extra click for those workflows, but I think that this way the graphs keep 
better symmetry with the 
{{java8_pre-commit_tests}}/{{java11_pre-commit_tests}}, so all the four 
workflows have the same organisation and number of columns.

As for 3.0 and 3.11, I have tried to homogenise the way the jobs are organised. 
Specifically, the approval steps depended on the build job, so one had to wait 
for the completion of the build before starting the optional tests. I have 
modified this to adopt the 4.0/trunk organisation, where the approval jobs 
don't depend on the build and it's the approved job what depends on both the 
build and its approval job.

I have sent [this 
message|https://lists.apache.org/thread.html/r02822bd5071029b95037b44a2d31fe5bb1c98d9f2391fd0443ca8684%40%3Cdev.cassandra.apache.org%3E]
 to the dev mail list informing about the proposed changes.

Just to summarise what we get with this ticket:
 * Intermediate commits/pushes don't spend any resources at all unless their 
jobs are manually approved.
 * For changes that are ready for a final round of review or that are ready to 
commit, the {{javaX_pre-commit_tests}} workflows have a single and easily 
visible button that runs the most relevant tests. In the future we can always 
change the set of tests that should be run here.
 * For intermediate steps or special cases such as fixing flaky tests, the 
{{javaX_separate_tests}} workflows allow to run any combination of tests 
individually.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a 

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Berenguer Blasi (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424879#comment-17424879
 ] 

Berenguer Blasi commented on CASSANDRA-16882:
-

I don't have a strong preference here but lgtm so far. +1

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424677#comment-17424677
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16882:
-

Hey [~adelapena], [~dcapwell], I was wondering where do we stand with this 
ticket? I want to update with this ticket info the CircleCI wiki page I was 
working on before I push it for final review. Do we have an agreement? :)

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421391#comment-17421391
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16882:
-

{quote}bq. In both cases we don't have to wait to press any of the optional 
buttons in the pre-commit workflow. The difference is that with 
[this|https://app.circleci.com/pipelines/github/adelapena/cassandra/893/workflows/47ae65f7-b8ca-4046-a124-4bc86ceb52e8]
 approach the build is always run (needed or not) and we have to press the 
{{start_pre-commit_tests}} button to start the grouped tests. In contrast, with 
[the other 
approach|https://app.circleci.com/pipelines/github/adelapena/cassandra/892/workflows/be35fdcb-d865-46c3-8370-55315010a2b9]
 nothing is built by default and we have to press the {{start_jx_build}} to 
start both the build and the test. In other words, the second approach saves us 
running the build job in intermediate commits. This job is relatively 
lightweight compared to the tests so I'm not sure it is worth the extra column, 
wdyt?
{quote}
I don't have stronger preference. That one more column is not the most 
beautiful one but it is also not a big deal. I will defer this to the others if 
someone has strong preference :) 

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421279#comment-17421279
 ] 

Andres de la Peña commented on CASSANDRA-16882:
---

{quote}I was referring to the _root_ {{start_j8_build}} 
[here|https://app.circleci.com/pipelines/github/adelapena/cassandra/892/workflows/894331d0-23cb-4def-9a1e-596142485c63]
 i.e. But it's a nit.
{quote}
I see. Unfortunately we can't directly connect the root {{start_j8_build}} to 
the leaf tests in an efficient way, we need the intermediate {{j8_build}} job 
to build the artifacts that will be reused by the the leaf test jobs. But it 
probably makes sense to rename the {{start_jx_build}} tasks to 
[{{start_jx_tests}}|https://app.circleci.com/pipelines/github/adelapena/cassandra/911/workflows/83c749c8-e61b-4f32-a49f-48ab268da4fc]
 or, probably better, 
[{{start_pre-commit_tests}}|https://app.circleci.com/pipelines/github/adelapena/cassandra/912/workflows/c867a240-19e5-4588-82d4-7bf9b81764f4].

[Here|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-trunk-v04]
 is the set of workflows generated by every commit with the last changes.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Berenguer Blasi (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421192#comment-17421192
 ] 

Berenguer Blasi commented on CASSANDRA-16882:
-

I was referring to the _root_ {{start_j8_build}} 
[here|https://app.circleci.com/pipelines/github/adelapena/cassandra/892/workflows/894331d0-23cb-4def-9a1e-596142485c63]
 i.e. But it's a nit.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420905#comment-17420905
 ] 

Andres de la Peña commented on CASSANDRA-16882:
---

{quote}Around the simplified version - visually only two columns look better to 
me but on the other hand my understanding is that now we won't have to wait 
until the build is ready to start the tests but we start build which triggers 
the tests on its own? Is that correct? I don't have strong preference around 
any of the two versions.
{quote}
In both cases we don't have to wait to press any of the optional buttons in the 
pre-commit workflow. The difference is that with 
[this|https://app.circleci.com/pipelines/github/adelapena/cassandra/893/workflows/47ae65f7-b8ca-4046-a124-4bc86ceb52e8]
 approach the build is always run (needed or not) and we have to press the 
{{start_pre-commit_tests}} button to start the grouped tests. In contrast, with 
[the other 
approach|https://app.circleci.com/pipelines/github/adelapena/cassandra/892/workflows/be35fdcb-d865-46c3-8370-55315010a2b9]
 nothing is built by default and we have to press the {{start_jx_build}} to 
start both the build and the test. In other words, the second approach saves us 
running the build job in intermediate commits. This job is relatively 
lightweight compared to the tests so I'm not sure it is worth the extra column, 
wdyt?
{quote}I would rename the root 'j8_build' alike jobs, that trigger the 
downstream actual tests, to 'j8_start_tests'
{quote}
I'm not sure I understand this. Do you mean the {{j8_build}} jobs in 
[this|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-6-trunk-v05]
 set of workflows? There {{j8_build}} is a job that actually do the building. 
Would renaming it to {{j8_start_tests}} mean that it would become a job 
requiring manual approval? In that case we would still need a {{j8_build}} job 
depending on {{j8_start_tests}}, since approval jobs can't do the project 
building. This is more or less what [this 
approach|https://app.circleci.com/pipelines/github/adelapena/cassandra/892/workflows/894331d0-23cb-4def-9a1e-596142485c63]
 does, although the {{start_jx_build}} job could be renamed to 
{{start_jx_tests}} in the case of the pre-commit workflows.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Berenguer Blasi (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419612#comment-17419612
 ] 

Berenguer Blasi commented on CASSANDRA-16882:
-

+1 to separate workflows. I would rename the root 'j8_build' alike jobs, that 
trigger the downstream actual tests, to 'j8_start_tests' i.e. Also +1 to 
collapsing all the cqlsh tests that we never exercise individually into a 
single 'box' would declutter the graph significantly.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419516#comment-17419516
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16882:
-

IMHO the 4 workflows look less overwhelming. I will take care of adding that to 
the wiki page I was working on when we agree on the version we use and commit 
this ticket.

Around the simplified version - visually only two columns look better to me but 
on the other hand my understanding is that now we won't have to wait until the 
build is ready to start the tests but we start build which triggers the tests 
on its own? Is that correct? I don't have strong preference around any of the 
two versions.

 

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419160#comment-17419160
 ] 

Andres de la Peña commented on CASSANDRA-16882:
---

[~e.dimitrova] thanks for the feedback.
{quote}Unfortunately the list of jobs became so long that going over the button 
for pre-commit jobs I can't follow anymore which jobs the arrows reach. Not 
sure we can do anything more about that unfortunately.
{quote}
Agree, I think that the approach with separate workflows is cleaner.
{quote}Also, I guess all the failures are due to the lower resources.
{quote}
Yes, I started it with low resources in an attempt to make the connections in 
the graph easier to follow, trying to highlight what tests are started by the 
pre-commit button.
{quote}It is not related really to what you do, that is the way it was setup 
from before but I was wondering - why do we have separate jobs for vnodes and 
no vnodes for the cqlsh tests, but one job for the Python DTests that triggers 
both versions?
{quote}
I have no clue about why it is that way. I have simplified it to start cqlsh 
tests with and without vnodes with a single button, as we do with dtests. I 
have also renamed the two pairs of workflows to {{javax_pre-commit_tests}} and 
{{javax_separate_tests}}, any suggestions are welcome. The approach can be 
found 
[here|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-6-trunk-v05].

On a slight variation, 
[here|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-trunk]
 I have tried to simplify the buttons for the pre-commit workflow so instead of 
a button to start the pre-commit tests we have a button to start the build, and 
the mandatory tests depend only on the build.

[~dcapwell] what do you think about the separate workflows approach?

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17418871#comment-17418871
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16882:
-

Sorry for the delay, I had the impression you were still fixing something. Back 
to this.
{quote}The workflow with an optional pre-commit job running most of the tests 
would look like 
[this|https://app.circleci.com/pipelines/github/adelapena/cassandra/843/workflows/da24c91a-0b6a-4370-a516-823cc4ad574d],
 and like 
[this|https://app.circleci.com/pipelines/github/adelapena/cassandra/843/workflows/f21093c4-1d9a-41b0-aaea-51c1b5c3c187]
 for Java 11.
{quote}
I have two comments on this one:
 * It is not related really to what you do, that is the way it was setup from 
before but I was wondering - why do we have separate jobs for vnodes and no 
vnodes for the cqlsh tests, but one job for the tests that triggers both 
versions?
 * Unfortunately the workflow became so long that going over the button for 
pre-commit jobs I can't follow anymore which jobs it reaches. Not sure we can 
do anything more about that unfortunately.

Also, I guess all the failures are due to the lower resources. 

About the second suggestion - I think you miss buttons for the cqlsh tests if I 
look at the right build as the link points to two builds but the other one miss 
the pre-commit button. This one looks cleaner but not sure when we add the 
missing buttons that this will not change. 

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415639#comment-17415639
 ] 

Andres de la Peña commented on CASSANDRA-16882:
---

The workflow with an optional pre-commit job running most of the tests would 
look like 
 
[this|https://app.circleci.com/pipelines/github/adelapena/cassandra/843/workflows/da24c91a-0b6a-4370-a516-823cc4ad574d],
 and like 
[this|https://app.circleci.com/pipelines/github/adelapena/cassandra/843/workflows/f21093c4-1d9a-41b0-aaea-51c1b5c3c187]
 for Java 11.

The central column looks a bit messy due to the repetition of the jobs that can 
be started either individually or with the pre-commit job, although the left 
column indicating what jobs have been started is quite clear. These repetitions 
are there because, as mentioned before, CircleCI doesn't allow disjunctions in 
job dependencies. An alternative approach would be placing the pre-commit stuff 
in a separate workflow with a single approval step.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415586#comment-17415586
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16882:
-

Consensus reached in [Slack 
|https://the-asf.slack.com/archives/CK23JSY2K/p1631627458109000|https://urldefense.proofpoint.com/v2/url?u=https-3A__the-2Dasf.slack.com_archives_CK23JSY2K_p1631627458109000=DwMBaQ=adz96Xi0w1RHqtPMowiL2g=4OZU_3AS7gzaAWcWmh7XcRWUDDtEKP19i124LAT3eqA=T3FK_onArHR7_m3-P5eSQ9MmpqYVqp4sN_w8UjU2PJA=DapuifvZx4abu3CJmhG3Sk1saI602WGAWzr0119-CqU=],
 publishing a summary here:
 * Instead of having a minimum required tests, the community agreed we can have 
a pre-commit job which covers all jobs except the python upgrade tests. Those 
shouldn't be skipped by anyone when they make sense (not for fixing unrelated 
test bug for example as they are taking all resources).
 * We can request a required field from Infra for Test build link similar to 
the Source Control Link one we currently have.

Please feel free to add anything I might have missed. Thank you to everyone who 
spend the time for a fruitful discussion. 

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415058#comment-17415058
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16882:
-

{quote}bq. thanks for pointing this out in Slack.
{quote}
We realize that the mail thread got lost in all current discussions so we 
wanted to be fair and ping people before merging anything.
{quote}this JIRA worries me... we have a thread on the dev list that tests are 
getting unstable, and now we want to disable tests by default... I rather us 
opt-into to disabling than opt-in of testing
{quote}
I think we are actually on the same page. No one opts in for less testing 
pre-merge to the code base. I think no committer would accept that. The idea is 
to give an option to people to run only whatever they need on commit and not 
waste resources or look for workarounds. (like the build only on PR option for 
example.) 

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415042#comment-17415042
 ] 

Brandon Williams commented on CASSANDRA-16882:
--

bq. we have a thread on the dev list that tests are getting unstable

There is also a thread specifically for this ticket.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17415036#comment-17415036
 ] 

David Capwell commented on CASSANDRA-16882:
---

thanks for pointing this out in Slack.  I have a script which rewrites our yaml 
to disable all the approvals as we need to know CI is clean before merging... 
right now I am seeing that when I am ready to merge a ticket I find out the 
optional tests were broken between now and my previous time committing 
something (I am committing roughly every 3-4 weeks, and find things keep 
breaking within this gap)... so for me I am worried about disabling unit tests.

Rather than changing LOWER/HIGHER, what if we had another file which doesn't 
trigger anything and when you want this behavior you opt-in?

bq. no one has a strong opinion here and move on with whatever we consider a 
good fit here

this JIRA worries me... we have a thread on the dev list that tests are getting 
unstable, and now we want to disable tests by default...  I rather us opt-into 
to disabling than opt-in of testing

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414312#comment-17414312
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16882:
-

Should we consider that no one has a strong opinion here and move on with 
whatever we consider a good fit here? Or maybe make a last ping in Slack before 
final decision in case someone missed the mail amongst all the latest 
discussions?

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408128#comment-17408128
 ] 

Andres de la Peña commented on CASSANDRA-16882:
---

All the options are good to me, I think that my favourites are 2 and 4 because 
the give us more flexibility. IMO the concept of mandatory tests is a bit 
sloppy and the set of tests to be run depends on the specific changes, but I'm 
not against having it if others find it useful.

I have sent this email to the dev list showing the four options: 
https://lists.apache.org/thread.html/r2fc18a52752d2f7e8fb0dcde901e1d56722f79f2c53432627a8dfc13%40%3Cdev.cassandra.apache.org%3E

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405486#comment-17405486
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16882:
-

I think I am in for option 3 or 4. Honestly, I don't find the graph so 
confusing as what you would need is to approve jobs only in the first column to 
the left. That is clear and neat. For me personally, I don't see way more 
complex, it is already quite dense graph. But I would suggest to post both 
options in Slack or the dev list for feedback. 

I think people didn't mind this improvement on the mailing list but I am no 
sure that they imagined how this will look like.

What do you think?

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404433#comment-17404433
 ] 

Andres de la Peña commented on CASSANDRA-16882:
---

I'm adding a fourth option that combines approaches 2 and 3, so the mandatory 
tests can be started either individually or all together with a single start 
button:

||Option||Branch||CI||
|1​|[16882-option-1-trunk|https://github.com/adelapena/cassandra/tree/16882-option-1-trunk]​|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/800/workflows/9cb8ca7b-ab57-431e-a22b-643d61c92c29]
 
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/800/workflows/3e26fd7e-5c5a-4ec3-8af9-4c247d96556a]​|
|2|[16882-option-2-trunk|https://github.com/adelapena/cassandra/tree/16882-option-2-trunk]​|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/798/workflows/a859cfbc-fdf8-4468-beb9-b2ee17dc1ae3]
 
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/798/workflows/a4a86879-e283-4aa9-8121-c51fa79095e6]​|
|3|[16882-option-3-trunk|https://github.com/adelapena/cassandra/tree/16882-option-3-trunk]​|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/799/workflows/91f90e3a-e032-4d57-ba60-45d925c07c99]
 
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/799/workflows/265a64f2-70b6-4a88-8045-89bdf50e5d8d]​|
|4|[16882-option-4-trunk|https://github.com/adelapena/cassandra/tree/16882-option-4-trunk]​|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/801/workflows/3b044fbb-0fda-4b30-9544-cdc259f8f09b]
 
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/801/workflows/4c205d19-22ea-4ae8-8618-09c9ec7dcbe9]​|
This gives us the flexibility of the second approach with the click savings of 
the third approach. However, the downside is that is done by duplicating the 
jobs, because CircleCI doesn't allow disjunctions in job dependencies. That 
leaves us with a more complex graph, and I'm afraid that could be more 
confusing than just writing in the doc what tests are mandatory.

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404404#comment-17404404
 ] 

Andres de la Peña commented on CASSANDRA-16882:
---

CC [~edimitrova]

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404399#comment-17404399
 ] 

Andres de la Peña commented on CASSANDRA-16882:
---

Here are drafts of what each approach would look like for trunk:
||Option||Branch||CI||
|1​|[16882-option-1-trunk|https://github.com/adelapena/cassandra/tree/16882-option-1-trunk]​|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/800/workflows/9cb8ca7b-ab57-431e-a22b-643d61c92c29]
 
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/800/workflows/3e26fd7e-5c5a-4ec3-8af9-4c247d96556a]​|
|2|[16882-option-2-trunk|https://github.com/adelapena/cassandra/tree/16882-option-2-trunk]​|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/798/workflows/a859cfbc-fdf8-4468-beb9-b2ee17dc1ae3]
 
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/798/workflows/a4a86879-e283-4aa9-8121-c51fa79095e6]​|
|3|[16882-option-3-trunk|https://github.com/adelapena/cassandra/tree/16882-option-3-trunk]​|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/799/workflows/91f90e3a-e032-4d57-ba60-45d925c07c99]
 
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/799/workflows/265a64f2-70b6-4a88-8045-89bdf50e5d8d]​|

> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
>  Issue Type: Task
>  Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI 
> workflows as it was proposed in [this 
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
>  sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM 
> dtests and cqhshlib tests. This is done by default for every commit or, with 
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits 
> and pushes, all running these test jobs. I'd say that frequently it is not 
> necessary to run the tests for some of those intermediate commits and pushes. 
> For example, one can show proofs of concept, or have multiple rounds of 
> review before actually running the tests. Running the tests for every change 
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly 
> specifying in the documentation what are the tests runs that are mandatory 
> before actually committing. We could do this in different ways:
>  # Make the entire CircleCI workflow optional, so the build job requires
>  manual approval. Once the build is approved the mandatory test jobs would
>  be run without any further approval, exactly as it's currently done.
>  # Make all the test jobs optional, so every test job requires manual 
> approval, and the documentation specifies which tests are mandatory in the 
> final steps of a ticket.
>  # Make all the mandatory test jobs depend on a single optional job, so we 
> have a single button to optionally run all the mandatory tests.
> I think any of these changes, or a combination of them, would significantly
>  reduce the usage of resources without making things less tested. The only
>  downside I can think of is that we would need some additional clicks on the
>  CircleCI GUI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org