[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16793506#comment-16793506
]
Marcus Eriksson commented on CASSANDRA-14806:
-
committed to 2.2 {{7333c2b04d6a876ff9734ca576caecace843fdab}} and merged up,
thanks!
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Legacy/Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Normal
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792901#comment-16792901
]
Stefan Podkowinski commented on CASSANDRA-14806:
Looks like your latest builds can be found at:
* [2.2|https://circleci.com/workflow-run/6ac9d6c4-3968-4a6e-8cbe-cad8c7880961]
* [3.0|https://circleci.com/workflow-run/50120c8c-8641-4761-aaa2-64214ed9290a]
* [3.11|https://circleci.com/workflow-run/db5e3bb2-4b8a-456a-8d5b-6dc943fea0fe]
*
[trunk|https://circleci.com/workflow-run/e1eec049-70f1-4a07-b3e9-1c519cb26888]
There seems to be a significant number of 2.2 dtests (see also CASSANDRA-14866)
and the upgrade tests failing, so I did some reference runs against the
unmodified branches. Turns out we see the same kind of failures there as well,
so it shouldn't be related to the patch at all.
* [3.0 - upgrade
tests|https://circleci.com/gh/spodkowinski/cassandra/675#tests/containers/11]
* [3.11 - upgrade
tests|https://circleci.com/gh/spodkowinski/cassandra/673#tests/containers/11]
* [2.2 -
dtests|https://circleci.com/workflow-run/0dae4fa9-7b00-4814-9726-87ad513bb110]
+1, I think we're ready to get this merged now.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Legacy/Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Normal
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770954#comment-16770954
]
Stefan Podkowinski commented on CASSANDRA-14806:
Doing a test run using the docker image mentioned above will finish in a bit
less than an hour:
https://circleci.com/gh/spodkowinski/cassandra/652
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Legacy/Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16753986#comment-16753986
]
Stefan Podkowinski commented on CASSANDRA-14806:
I think ccm must be updated first for the new git url, so the tests actually
run. Can you try {{spod/cassandra-testing-ubuntu1810-java11-w-dependencies}} as
new docker image?
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Legacy/Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750011#comment-16750011
]
Marcus Eriksson commented on CASSANDRA-14806:
-
I added the upgrade tests target to the branches, runs;
* [2.2|https://circleci.com/gh/krummas/cassandra/1317#tests/containers/15]
([reference
run|https://circleci.com/gh/krummas/cassandra/1381#tests/containers/11])
* [3.0|https://circleci.com/gh/krummas/cassandra/1326] ([reference
run|https://circleci.com/gh/krummas/cassandra/1382#tests/containers/11])
* [3.11|https://circleci.com/gh/krummas/cassandra/1385] ([reference
run|https://circleci.com/gh/krummas/cassandra/1383#tests/containers/11])
* [trunk|https://circleci.com/gh/krummas/cassandra/1367] ([reference
run|https://circleci.com/gh/krummas/cassandra/1384#tests/containers/11])
I have not yet looked in to why the patched runs are much quicker than the
reference ones (8 minutes vs 1h)
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Legacy/Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725733#comment-16725733
]
Stefan Podkowinski commented on CASSANDRA-14806:
LGTM, let's get this merged after holidays.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725043#comment-16725043
]
Marcus Eriksson commented on CASSANDRA-14806:
-
fixed the utest_compression run on the 2.2 branch;
https://circleci.com/workflow-run/f71902c6-050c-4080-a54f-3a2699536f24
still failures, but they look unrelated to these changes
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16724185#comment-16724185
]
Stefan Podkowinski commented on CASSANDRA-14806:
Looks good, except the [weird utest_compression
run|https://circleci.com/gh/krummas/cassandra/1263]. Thanks for talking the
extra miles backporting this.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722988#comment-16722988
]
Marcus Eriksson commented on CASSANDRA-14806:
-
2.2:
code: https://github.com/krummas/cassandra/commits/stefan/14806-2.2
test runs:
https://circleci.com/gh/krummas/workflows/cassandra/tree/stefan%2F14806-2.2
reference run (unpatched 2.2):
https://circleci.com/workflow-run/bf69001c-756b-4c43-a754-652066dca07f
3.0
code: https://github.com/krummas/cassandra/commits/stefan/14806-3.0
test runs:
https://circleci.com/gh/krummas/workflows/cassandra/tree/stefan%2F14806-3.0
3.11
code: code: https://github.com/krummas/cassandra/commits/stefan/14806-3.11
test runs:
https://circleci.com/gh/krummas/workflows/cassandra/tree/stefan%2F14806-3.11
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719007#comment-16719007
]
Marcus Eriksson commented on CASSANDRA-14806:
-
We should probably do this change to 2.2+ branches as well, without the
build.xml changes and config-2_1.yml without the J11 targets
Otherwise there will be different ways to enable high/low resources on the
active branches, which would be quite painful
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718999#comment-16718999
]
Marcus Eriksson commented on CASSANDRA-14806:
-
nope, I think this looks good
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718994#comment-16718994
]
Stefan Podkowinski commented on CASSANDRA-14806:
Latest build looks really nice!
I didn't realize that CQL tests are already executed as part of the unit test
job, as they have a separate ant target. We don't need a dedicated job for
them, if that's being the case.
Timed out jvm dtests should be a separate issue (CASSANDRA-14922) and not
related to our changes. They won't affect regular unit tests with low
resources. Let's keep them in the standard unit test job.
So I think we're done here, or is there anything you like to add?
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718945#comment-16718945
]
Marcus Eriksson commented on CASSANDRA-14806:
-
and a low resource run:
[https://circleci.com/workflow-run/55cc3115-b85a-47cf-b326-ba55707c857c]
started all jobs except for the dtest ones
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718833#comment-16718833
]
Marcus Eriksson commented on CASSANDRA-14806:
-
bq. We should use java11-executor for the prerequisite clone & build step.
so I was trying to figure out how to solve
[this|https://circleci.com/gh/krummas/cassandra/1143#tests/containers/1]
problem:
{code}
java.lang.NoSuchMethodError:
java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
at
org.apache.cassandra.io.compress.CompressedSequentialWriterReopenTest$BadCompressor.compress(CompressedSequentialWriterReopenTest.java:119)
at
org.apache.cassandra.io.compress.CompressedSequentialWriterReopenTest.badCompressor2(CompressedSequentialWriterReopenTest.java:55)
{code}
which happens when we build using java8 + java11:
{code}
$ JAVA8_HOME=../jdk1.8.0_181.jdk/... JAVA_HOME=.../jdk-11.jdk/... ant clean jar
$ JAVA_HOME=../jdk1.8.0_181.jdk/... ant test
{code}
ie - we build on java11 and run the tests on java8. Looking at how we build the
cassandra jar vs how the tests are built, the difference is that when building
the multi-target, we explicitly state that we should use the java8 compiler
(and we also fork the jvm, which is required, otherwise {{}} is dropped since compilerarg is passed when
forking the jvm.).
bq. I'd prefer to leave the cql tests
these are not the cqlsh tests, these are the cql ant tests, which are run by
default by the standard unit test target
bq. I wouldn't move them yet and execute along with regular unit test, unless
they won't be able to execute on low resources settings.
I'll try them out on low-resource settings, if they pass in a reasonable time,
I'll add them to auto-run
Pushed a few fixes to the branch, latest test run here:
https://circleci.com/workflow-run/35c0e60e-f37c-4400-bf64-dd28f6ce5682
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717555#comment-16717555
]
Stefan Podkowinski commented on CASSANDRA-14806:
Looks good overall.
* We should use java11-executor for the prerequisite clone & build step. This
will catch commits that fail to compile in java11 and will also make sure we're
testing the same kind of build that we'd ship on releases. This would work for
dtests, but not unit tests, hence the use of the `clean` target, before
actually running the unit tests. This would slow down tests for
run_parallel_junit_tests, but running j11 dtests with a j8 build isn't ideal
either. But please feel to go ahead if you feel test runtime is more important
for you here, as should would prioritize java8 and may come up with a different
solution for java11 dtests later.
* New distributed unit tests support: Thanks for adding this. I wouldn't move
them yet and execute along with regular unit test, unless they won't be able to
execute on low resources settings.
* I'd prefer to leave the cql tests in, so we can continue to work on them.
Keep them in but just don't run them if you're not interested. See my last
comment on CASSANDRA-14298.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716723#comment-16716723
]
Marcus Eriksson commented on CASSANDRA-14806:
-
made changes here: http://github.com/krummas/cassandra/tree/stefan/14806-2 -
could you have a look [~spo...@gmail.com]?
test run: https://circleci.com/workflow-run/886620a2-fdef-488a-8aed-16cdbf61c1d6
* java 8 dtests & unit tests work fine
* java 11 dtests run, but seems byteman tests all fail ({{ERROR : Unexpected
exception transforming class org.apache.cassandra.db.ReadCommandVerbHandler
using rule "disable data reads" loaded from
./byteman/read_repair/stop_data_reads.btm line 8
java.lang.IllegalArgumentException: Unsupported class file major version 55}})
* java 11 unit tests has a few failures, most due to
{{java.lang.ClassCastException: class
jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class
java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and
java.net.URLClassLoader are in module java.base of loader 'bootstrap')}} in the
new distributed tests
* compression tests fail: {{java.lang.ClassNotFoundException:
/tmp/cassandra/test/distributed/org/apache/cassandra/distributed/DistributedReadWritePathTest}}
* long utests look like they are missing a
{{DatabaseDescriptor.daemonInitialization()}}, will try to fix
* CQL utests look broken
We should probably try to at least fix the non-java11 tests before merging this
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712884#comment-16712884
]
Stefan Podkowinski commented on CASSANDRA-14806:
Let's just get rid of the pre-commit hook completely then.
How about just adding the following steps to the instructions and handle things
manually?
Switching between low/high resource configurations:
* cp config.yml.HIGHRES config.yml # use LOWRES to revert
Updating the original config:
* Make your changes to config-2_1.yml (keep this for lowres settings)
* Generate valid config using circleci cli tool:
circleci config process config-2_1.yml > config.yml.LOWRES
* Generate version for highres settings
patch -o config-2_1.yml.HIGHRES config-2_1.yml config-2_1.yml.high_res.patch
&& \
circleci config process config-2_1.yml.HIGHRES > config.yml.HIGHRES && \
rm config-2_1.yml.HIGHRES
* Copy either HIGH or LOWRES to config.yml to make changes effective
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688607#comment-16688607
]
Stefan Podkowinski commented on CASSANDRA-14806:
I'd also slightly prefer the second option for generating both versions with
low and high settings. Please go ahead if you have any ideas on how to change
the scripting for that.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686337#comment-16686337
]
Marcus Eriksson commented on CASSANDRA-14806:
-
bq. Another option would probably to generate two versions of the config.yml
yeah that sounds better I think, let me know if you want me to do those final
changes
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684587#comment-16684587
]
Stefan Podkowinski commented on CASSANDRA-14806:
Patching the generated {{config.yml}} seems to be more painful compared to the
original file. But I don't really mind if you want to go down that way. Another
option would probably to generate two versions of the {{config.yml}}: one using
low settings ({{config_low.yml}}) as currently provided and another patched
version ({{config_high.yml}}) and hard link or copy any of these to
{{config.yml}} as needed.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683777#comment-16683777
]
Marcus Eriksson commented on CASSANDRA-14806:
-
Just tried this on a new machine, have to say it is a bit annoying having to
install circleci cli just to make a tiny change to a config file
Maybe we could have the shipped {{.patch}} apply to the generated file instead?
This way only people doing changes to {{circleci-2.1.yml}} would have to
install the tool. Drawback would be that the person changing circle-2.1.yml
would have to also provide a new patch for the generated file (though, it is
likely this is the case even if we patch {{circleci-2.1.yml}} & generate
{{config.yml}})
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16665217#comment-16665217
]
Stefan Podkowinski commented on CASSANDRA-14806:
Looks your cli version is slightly outdated, mine is 0.1.3541+215b0f9
(installed via brew).
You can either run the {{.circleci/pre-commit-hook.sh}} manually or add it as a
commit hook locally. Totally up to you. I've amended the readme to make this
clearer.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16665082#comment-16665082
]
Marcus Eriksson commented on CASSANDRA-14806:
-
bq. by committing the proposed changes as config-2_1.yml, run it through circle
config process and commit the output as config.yml?
yeah makes sense - this also makes sure we only commit a valid config. A bit of
weirdness though, the generated conf you committed has escaped {{/}}s in the
comments, mine doesn't, so I get a big diff by just regenerating the committed
config-2_1.yml (I guess circleci cli version difference or something? Mine is
0.1.3257+b197d90)
bq. work automatically using git commit hooks.
yes, this could be a nice feature, personally though I would probably prefer
just having a small script that does it for me, like
{{.circleci/activate_high_resource.sh}} mostly to avoid regenerating the config
file on every commit (we could probably add a check that only regenerates the
config if the modification time config-2_1.yml is newer than the one on
config.yml).
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662232#comment-16662232
]
Stefan Podkowinski commented on CASSANDRA-14806:
[~krummas]:
"Maybe we could ship a .circleci/high_resource.patch so that enabling the high
resource would simply be git am .circleci/high_resource.patch?"
"Waiting until 2.1 config is out of 'preview' mode might also be good?"
How about killing three birds with one stone, by committing the proposed
changes as {{config-2_1.yml}}, run it through {{circle config process}} and
commit the output as {{config.yml}}? We can also support some on-the-fly
patching, using a script and even make this work automatically using git commit
hooks. See my latest commit in WIP branch.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662228#comment-16662228
]
Stefan Podkowinski commented on CASSANDRA-14806:
[~jmeredithco], the goal of this ticket was to make it easier to integrate
additional jobs. The ant and pytest calls should be parameterizable, to allow
you to call the commands with a custom target and custom parameters. Ideally
this should make it possible to implement the JaCoCo coverage as well, simply
by adding a small job with the correct parameters and add it to the workflow.
Just let me know if this wouldn't work for your case and if we need additional
changes to make that happen. We can also do this, once this patch is merged.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660687#comment-16660687
]
Jon Meredith commented on CASSANDRA-14806:
--
Looks like we had the same ideas on the CircleCI config - I was trying to add
targets that generated coverage information with JaCoCo in CASSANDRA-14788.
Obviously the two changes conflict, I can look at reworking the coverage
changes on top of this one once it has merged, or you're welcome to incorporate
the changes in this patch if you would prefer.
Also, I'm not sure if you've had problems running the burn tests on the high
resource configuration. I hit problems locally on 8 / 12 core machines running
it on machines that are not configured to use very large heaps.
CASSANDRA-14790 makes the long buffer test pass for me, however the long btree
test has been a little flaky just running locally.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660450#comment-16660450
]
Marcus Eriksson commented on CASSANDRA-14806:
-
this looks good to me, a few comments (mostly circle-ci-config-feature-related);
* It would be nice to at some point get rid of the {{<<:}} stuff as it is not
really covered in the circleci docs, this probably needs additional circleci
features though
* We can probably remove {{with_dtest_jobs_only}} now? Or do you have a good
use case for it?
https://github.com/krummas/cassandra/commit/8e6732694648cdf69a64bcbf416226ddae1f95b7
* We should probably allow {{test-compression}} to be run in parallel, this
requires build.xml changes though, so we can do it in a later ticket
* Maybe we could ship a {{.circleci/high_resource.patch}} so that enabling the
high resource would simply be {{git am .circleci/high_resource.patch}}?
As discussed offline, we should probably give a heads up to the mailing list
(especially the fact that you now need to enable 'build processing') to get
this to work. Waiting until 2.1 config is out of 'preview' mode might also be
good?
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/be25579d-3cbb-4258-9e19-b1f571873850]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14806) CircleCI workflow improvements and Java 11 support
[
https://issues.apache.org/jira/browse/CASSANDRA-14806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649871#comment-16649871
]
Stefan Podkowinski commented on CASSANDRA-14806:
I've now basically rewritten the CircleCI config to make heavily use of
parameters to avoid redundant copy and pasting of script snippets. Combined
with job approvals, we can now have a broader selection of tests. So far I've
added: ant long-test, test-burn, cql-test, test-compression, stress-test,
fqltool-test. Cqlsh dtests will be added in CASSANDRA-14298.
Unfortunately I wasn't able to solve the requirement for not having to manually
edit the config for switching between high/free resource settings. CircleCI
support was so kind to look at the issue and latest config, but wasn't able to
advise a solution either. So I guess we still have to live with that situation
for a while.
> CircleCI workflow improvements and Java 11 support
> --
>
> Key: CASSANDRA-14806
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14806
> Project: Cassandra
> Issue Type: Improvement
> Components: Build, Testing
>Reporter: Stefan Podkowinski
>Assignee: Stefan Podkowinski
>Priority: Major
>
> The current CircleCI config could use some cleanup and improvements. First of
> all, the config has been made more modular by using the new CircleCI 2.1
> executors and command elements. Based on CASSANDRA-14713, there's now also a
> Java 11 executor that will allow running tests under Java 11. The {{build}}
> step will be done using Java 11 in all cases, so we can catch any regressions
> for that and also test the Java 11 multi-jar artifact during dtests, that
> we'd also create during the release process.
> The job workflow has now also been changed to make use of the [manual job
> approval|https://circleci.com/docs/2.0/workflows/#holding-a-workflow-for-a-manual-approval]
> feature, which now allows running dtest jobs only on request and not
> automatically with every commit. The Java8 unit tests still do, but that
> could also be easily changed if needed. See [example
> workflow|https://circleci.com/workflow-run/08ecb879-9aaa-4d75-84d6-b00dc9628425]
> with start_ jobs being triggers needed manual approval for starting the
> actual jobs.
> There was some churn in manually editing the config for paid and non-paid
> resource tiers before. This has been mostly mitigated now by using project
> settings instead, for overriding lower defaults (see below) and scheduling
> dtests on request, which will only run on paid accounts nonetheless, so we
> use high settings for these right away. The only issue left is the question
> how we may be able to dynamically adjust the {{resource_class}} and
> {{parallelism}} settings for unit tests. So at this point, the CircleCI
> config will work for both paid and non-paid by default, but paid accounts
> will see slower unit test results, as only medium instances are used (ie.
> 15min instead of 4min).
> Attention CircleCI paid account users: you'll have to add
> "{{CCM_MAX_HEAP_SIZE: 2048M}}" and "{{CCM_HEAP_NEWSIZE: 512M}}" to your
> project's environment settings or create a context, to override the lower
> defaults for free instances!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
