[jira] [Comment Edited] (CASSANDRA-19558) Standalone jenkinsfile first round bug fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840770#comment-17840770 ] Michael Semb Wever edited comment on CASSANDRA-19558 at 4/25/24 11:49 AM: -- Back to (3)… New Patches - trunk https://github.com/apache/cassandra/compare/trunk...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters - 5.0 https://github.com/apache/cassandra/compare/cassandra-5.0...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters-5.0 CI - 5.0 post-commit https://ci-cassandra.apache.org/job/Cassandra-devbranch-5/21 - 5.0 pre-commit internal [^CASSANDRA-19558-50-33-ci_summary.html] , [^CASSANDRA-19558-50-33-results_details.tar.xz] - trunk pre-commit internal [^CASSANDRA-19558-trunk-11-ci_summary.html] , [^CASSANDRA-19558_#8_results_details.tar.xz] No related failures! was (Author: michaelsembwever): Back to (3)… New Patches - trunk https://github.com/apache/cassandra/compare/trunk...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters - 5.0 https://github.com/apache/cassandra/compare/cassandra-5.0...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters-5.0 CI 5.0 post-commit https://ci-cassandra.apache.org/job/Cassandra-devbranch-5/21 5.0 pre-commit internal [^CASSANDRA-19558-50-33-ci_summary.html] , [^CASSANDRA-19558-50-33-results_details.tar.xz] trunk pre-commit internal [^CASSANDRA-19558-trunk-11-ci_summary.html] , [^CASSANDRA-19558_#8_results_details.tar.xz] No related failures! > Standalone jenkinsfile first round bug fixes > > > Key: CASSANDRA-19558 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19558 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: CASSANDRA-19558_50_#5_ci_summary.html, > CASSANDRA-19558_50_#5_results_details.tar.xz, > CASSANDRA-19558-5.0_#13_ci_summary.html, > CASSANDRA-19558-5.0_#13_results_details.tar.xz, > CASSANDRA-19558-5.0_#16_ci_summary.html, > CASSANDRA-19558-5.0_#16_results_details.tar.xz, > CASSANDRA-19558-50-33-ci_summary.html, > CASSANDRA-19558-50-33-results_details.tar.xz, > CASSANDRA-19558-trunk-11-ci_summary.html, CASSANDRA-19558_#8_ci_summary.html, > CASSANDRA-19558_#8_results_details.tar.xz > > > A few follow up improvements and bug fixes for the standalone jenkinsfile. > - add at top a list of test failures in ci_summary.html > - docker scripts always try to login (as base images need to be pulled too) > - move simulator-dtests to large containers (they need 8g just heap) > - in ubuntu2004_test.docker make sure /home/cassandra exists and has correct > perms (from marcuse) > - persist the jenkinsfile parameters from run to run (important for the > post-commit jobs to keep their non-default branch and profile values) (was > CASSANDRA-19536) > - increase jvm-dtest splits from 8 to 12 > - when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile > generateTestReports() with manual wget of test files, allowing the summary > phase to be run on any agent (copyArtifact would take >4hrs otherwise) (was > INFRA-25694) > - copy ci_summary.html and results_details.tar.xz to nightlies -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19558) Standalone jenkinsfile first round bug fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840332#comment-17840332 ] Michael Semb Wever edited comment on CASSANDRA-19558 at 4/25/24 8:08 AM: - Still issues with docker rate limits, now solely caused by pulling alpine:latest Fixing by pushing alpine:3.19.1 into our jfrog too, reference this comment: https://issues.apache.org/jira/browse/CASSANDRA-18931?focusedCommentId=17840325=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17840325 Another issue was with having the .git subdirectory included in the stash, and unstashing it on top of existing git working directories on ci-cassandra. The only reason .git was being stashed was because a bug had crept into the deb/rpm scripts where they failed if git didn't work. The scripts have been fixed to work on non-versioned working directories, and .git subdirectory is no longer stashed. was (Author: michaelsembwever): Still issues with docker rate limits, now solely caused by pulling alpine:latest Pushing alpine:3.19.1 into our jfrog too, reference this comment: https://issues.apache.org/jira/browse/CASSANDRA-18931?focusedCommentId=17840325=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17840325 > Standalone jenkinsfile first round bug fixes > > > Key: CASSANDRA-19558 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19558 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: CASSANDRA-19558_50_#5_ci_summary.html, > CASSANDRA-19558_50_#5_results_details.tar.xz, > CASSANDRA-19558-5.0_#13_ci_summary.html, > CASSANDRA-19558-5.0_#13_results_details.tar.xz, > CASSANDRA-19558-5.0_#16_ci_summary.html, > CASSANDRA-19558-5.0_#16_results_details.tar.xz, > CASSANDRA-19558_#8_ci_summary.html, CASSANDRA-19558_#8_results_details.tar.xz > > > A few follow up improvements and bug fixes for the standalone jenkinsfile. > - add at top a list of test failures in ci_summary.html > - docker scripts always try to login (as base images need to be pulled too) > - move simulator-dtests to large containers (they need 8g just heap) > - in ubuntu2004_test.docker make sure /home/cassandra exists and has correct > perms (from marcuse) > - persist the jenkinsfile parameters from run to run (important for the > post-commit jobs to keep their non-default branch and profile values) (was > CASSANDRA-19536) > - increase jvm-dtest splits from 8 to 12 > - when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile > generateTestReports() with manual wget of test files, allowing the summary > phase to be run on any agent (copyArtifact would take >4hrs otherwise) (was > INFRA-25694) > - copy ci_summary.html and results_details.tar.xz to nightlies -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19558) Standalone jenkinsfile first round bug fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839287#comment-17839287 ] Michael Semb Wever edited comment on CASSANDRA-19558 at 4/23/24 7:31 PM: - A number of issues have been identified. – need to capture the generateTestReports logs, – fix ant version in centos7-build.docker – remove docker login (if credentials exist then docker is logged in) – prefetch docker images from jfrog (to reduce dockerhub pull rate limits) – use scripts from cassandra-builds to clean and report on agents – stream xz where possible (not an issue, just perf improvement) The docker pull rate limit was the most serious, blocking. patches… 1. this is part of CASSANDRA-18594 (most of it is already running (manually deployed) to ci-cassandra) [https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:cassandra-builds:mck/18594] ( [https://github.com/apache/cassandra-builds/commit/eb3eb5e] ) 2. and for CASSANDRA-19558 [https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:cassandra-builds:mck/19558] ( [https://github.com/thelastpickle/cassandra-builds/commit/9f8ae9dcacd0d744992cc4eaf50f29a8836ffdbd] ) 3. and then (also for CASSANDRA-19558 ) but comes last (i can test it manually once (2) is committed) [https://github.com/apache/cassandra/commit/92c0cb7] (this is on top of the previous commit (that already passed review) in [https://github.com/apache/cassandra/compare/apache:cassandra:cassandra-5.0...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters-5.0|https://github.com/apache/cassandra/compare/apache:cassandra:cassandra-5.0...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters-5.0] ) was (Author: michaelsembwever): A number of issues have been identified. – need to capture the generateTestReports logs, – fix ant version in centos7-build.docker – remove docker login (if credentials exist then docker is logged in) – prefetch docker images from jfrog (to reduce dockerhub pull rate limits) – use scripts from cassandra-builds to clean and report on agents – stream xz where possible (not an issue, just perf improvement) The docker pull rate limit was the most serious, blocking. patches… 1. this is part of CASSANDRA-18594 (most of it is already running (manually deployed) to ci-cassandra) [https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:cassandra-builds:mck/18594] ( [https://github.com/apache/cassandra-builds/commit/eb3eb5e] ) 2. and for CASSANDRA-19558 [https://github.com/thelastpickle/cassandra-builds/compare/mck/18594...thelastpickle:cassandra-builds:mck/19558] ( [https://github.com/thelastpickle/cassandra-builds/commit/9f8ae9dcacd0d744992cc4eaf50f29a8836ffdbd] ) 3. and then (also for CASSANDRA-19558 ) but comes last (i can test it manually once (2) is committed) [https://github.com/apache/cassandra/commit/92c0cb7] (this is on top of the previous commit (that already passed review) in [https://github.com/apache/cassandra/compare/apache:cassandra:cassandra-5.0...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters-5.0|https://github.com/apache/cassandra/compare/apache:cassandra:cassandra-5.0...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters-5.0] ) > Standalone jenkinsfile first round bug fixes > > > Key: CASSANDRA-19558 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19558 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: CASSANDRA-19558_50_#5_ci_summary.html, > CASSANDRA-19558_50_#5_results_details.tar.xz, > CASSANDRA-19558-5.0_#13_ci_summary.html, > CASSANDRA-19558-5.0_#13_results_details.tar.xz, > CASSANDRA-19558-5.0_#16_ci_summary.html, > CASSANDRA-19558-5.0_#16_results_details.tar.xz, > CASSANDRA-19558_#8_ci_summary.html, CASSANDRA-19558_#8_results_details.tar.xz > > > A few follow up improvements and bug fixes for the standalone jenkinsfile. > - add at top a list of test failures in ci_summary.html > - docker scripts always try to login (as base images need to be pulled too) > - move simulator-dtests to large containers (they need 8g just heap) > - in ubuntu2004_test.docker make sure /home/cassandra exists and has correct > perms (from marcuse) > - persist the jenkinsfile parameters from run to run (important for the > post-commit jobs to keep their non-default branch and profile values) (was > CASSANDRA-19536) > - increase jvm-dtest splits from 8 to 12 > - when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile > generateTestReports() with manual wget of test files, allowing the summary > phase to be run on any agent (copyArtifact would
[jira] [Comment Edited] (CASSANDRA-19558) Standalone jenkinsfile first round bug fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839287#comment-17839287 ] Michael Semb Wever edited comment on CASSANDRA-19558 at 4/23/24 7:26 PM: - A number of issues have been identified. – need to capture the generateTestReports logs, – fix ant version in centos7-build.docker – remove docker login (if credentials exist then docker is logged in) – prefetch docker images from jfrog (to reduce dockerhub pull rate limits) – use scripts from cassandra-builds to clean and report on agents – stream xz where possible (not an issue, just perf improvement) The docker pull rate limit was the most serious, blocking. patches… 1. this is part of CASSANDRA-18594 (most of it is already running (manually deployed) to ci-cassandra) [https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:cassandra-builds:mck/18594] ( [https://github.com/apache/cassandra-builds/commit/eb3eb5e] ) 2. and for CASSANDRA-19558 [https://github.com/thelastpickle/cassandra-builds/compare/mck/18594...thelastpickle:cassandra-builds:mck/19558] ( [https://github.com/thelastpickle/cassandra-builds/commit/9f8ae9dcacd0d744992cc4eaf50f29a8836ffdbd] ) 3. and then (also for CASSANDRA-19558 ) but comes last (i can test it manually once (2) is committed) [https://github.com/apache/cassandra/commit/92c0cb7] (this is on top of the previous commit (that already passed review) in [https://github.com/apache/cassandra/compare/apache:cassandra:cassandra-5.0...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters-5.0|https://github.com/apache/cassandra/compare/apache:cassandra:cassandra-5.0...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters-5.0] ) was (Author: michaelsembwever): A number of issues have been identified. – need to capture the generateTestReports logs, – fix ant version in centos7-build.docker – remove docker login (if credentials exist then docker is logged in) – prefetch docker images from jfrog (to reduce dockerhub pull rate limits) – use scripts from cassandra-builds to clean and report on agents – stream xz where possible (not an issue, just perf improvement) The docker pull rate limit was the most serious, blocking. patches… 1. this is part of CASSANDRA-18594 (most of it is already running (manually deployed) to ci-cassandra) [https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:cassandra-builds:mck/18594] ( [https://github.com/apache/cassandra-builds/commit/eb3eb5e] ) 2. and for CASSANDRA-19558 [https://github.com/thelastpickle/cassandra-builds/compare/mck/18594...thelastpickle:cassandra-builds:mck/19558] ( [https://github.com/thelastpickle/cassandra-builds/commit/9f8ae9dcacd0d744992cc4eaf50f29a8836ffdbd] ) 3. and then (also for CASSANDRA-19558 ) but comes last (i can test it manually once (2) is committed) [https://github.com/apache/cassandra/commit/b3e1e40658e99c37f2a142e247ca4305fcc52eb0] (this is on top of the previous commit (that already passed review) in [https://github.com/apache/cassandra/compare/apache:cassandra:cassandra-5.0...thelastpickle:cassandra:mck/mck/jenkinsfile-persist-parameters-5.0-test] ) > Standalone jenkinsfile first round bug fixes > > > Key: CASSANDRA-19558 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19558 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: CASSANDRA-19558_50_#5_ci_summary.html, > CASSANDRA-19558_50_#5_results_details.tar.xz, > CASSANDRA-19558-5.0_#13_ci_summary.html, > CASSANDRA-19558-5.0_#13_results_details.tar.xz, > CASSANDRA-19558-5.0_#16_ci_summary.html, > CASSANDRA-19558-5.0_#16_results_details.tar.xz, > CASSANDRA-19558_#8_ci_summary.html, CASSANDRA-19558_#8_results_details.tar.xz > > > A few follow up improvements and bug fixes for the standalone jenkinsfile. > - add at top a list of test failures in ci_summary.html > - docker scripts always try to login (as base images need to be pulled too) > - move simulator-dtests to large containers (they need 8g just heap) > - in ubuntu2004_test.docker make sure /home/cassandra exists and has correct > perms (from marcuse) > - persist the jenkinsfile parameters from run to run (important for the > post-commit jobs to keep their non-default branch and profile values) (was > CASSANDRA-19536) > - increase jvm-dtest splits from 8 to 12 > - when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile > generateTestReports() with manual wget of test files, allowing the summary > phase to be run on any agent (copyArtifact would take >4hrs otherwise) (was > INFRA-25694) > - copy ci_summary.html and
[jira] [Comment Edited] (CASSANDRA-19558) Standalone jenkinsfile first round bug fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840060#comment-17840060 ] Michael Semb Wever edited comment on CASSANDRA-19558 at 4/23/24 10:55 AM: -- (1) is committed. wrt (2). hit a few small problems. INFRA-25738 being one (though unrelated to the patch). needed to make it work for the <5 jobs, and against different possible python versions (>=3.6). these are fixed. i'm ready to commit (2) which is now [https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:cassandra-builds:mck/19558] (excluding the throwaway commit, ofc) Otherwise we have good test runs in #3 to #7 in [https://ci-cassandra.apache.org/job/Cassandra-devbranch-before-5-artifacts/] and in #588 to #594 in [https://ci-cassandra.apache.org/job/Cassandra-4.1-artifacts/] For (3) it is also needed ensuring it works when github is down (or goes down mid build)… was (Author: michaelsembwever): (1) is committed. wrt (2). hit a few small problems. INFRA-25738 being one (though unrelated to the patch). needed to make it work for the <5 jobs, and against different possible python versions (>=3.6). these are fixed. i'm ready to commit (2) which is now https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:cassandra-builds:mck/19558 Otherwise we have good test runs in #3 to #7 in [https://ci-cassandra.apache.org/job/Cassandra-devbranch-before-5-artifacts/] and in #588 to #594 in [https://ci-cassandra.apache.org/job/Cassandra-4.1-artifacts/] For (3) it is also needed ensuring it works when github is down (or goes down mid build)… > Standalone jenkinsfile first round bug fixes > > > Key: CASSANDRA-19558 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19558 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: CASSANDRA-19558_50_#5_ci_summary.html, > CASSANDRA-19558_50_#5_results_details.tar.xz, > CASSANDRA-19558-5.0_#13_ci_summary.html, > CASSANDRA-19558-5.0_#13_results_details.tar.xz, > CASSANDRA-19558-5.0_#16_ci_summary.html, > CASSANDRA-19558-5.0_#16_results_details.tar.xz, > CASSANDRA-19558_#8_ci_summary.html, CASSANDRA-19558_#8_results_details.tar.xz > > > A few follow up improvements and bug fixes for the standalone jenkinsfile. > - add at top a list of test failures in ci_summary.html > - docker scripts always try to login (as base images need to be pulled too) > - move simulator-dtests to large containers (they need 8g just heap) > - in ubuntu2004_test.docker make sure /home/cassandra exists and has correct > perms (from marcuse) > - persist the jenkinsfile parameters from run to run (important for the > post-commit jobs to keep their non-default branch and profile values) (was > CASSANDRA-19536) > - increase jvm-dtest splits from 8 to 12 > - when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile > generateTestReports() with manual wget of test files, allowing the summary > phase to be run on any agent (copyArtifact would take >4hrs otherwise) (was > INFRA-25694) > - copy ci_summary.html and results_details.tar.xz to nightlies -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19558) Standalone jenkinsfile first round bug fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840060#comment-17840060 ] Michael Semb Wever edited comment on CASSANDRA-19558 at 4/23/24 10:51 AM: -- (1) is committed. wrt (2). hit a few small problems. INFRA-25738 being one (though unrelated to the patch). needed to make it work for the <5 jobs, and against different possible python versions (>=3.6). these are fixed. i'm ready to commit (2) which is now https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:cassandra-builds:mck/19558 Otherwise we have good test runs in #3 to #7 in [https://ci-cassandra.apache.org/job/Cassandra-devbranch-before-5-artifacts/] and in #588 to #594 in [https://ci-cassandra.apache.org/job/Cassandra-4.1-artifacts/] For (3) it is also needed ensuring it works when github is down (or goes down mid build)… was (Author: michaelsembwever): (1) is committed. wrt (2). hit a few small problems. INFRA-25738 being one (though unrelated to the patch). needed to make it work for the <5 jobs, and against different possible python versions (>=3.6). these are fixed. i'm ready to commit (2). Otherwise we have good test runs in #3 to #7 in [https://ci-cassandra.apache.org/job/Cassandra-devbranch-before-5-artifacts/] and in #588 to #594 in [https://ci-cassandra.apache.org/job/Cassandra-4.1-artifacts/] For (3) it is also needed ensuring it works when github is down (or goes down mid build)… > Standalone jenkinsfile first round bug fixes > > > Key: CASSANDRA-19558 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19558 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: CASSANDRA-19558_50_#5_ci_summary.html, > CASSANDRA-19558_50_#5_results_details.tar.xz, > CASSANDRA-19558-5.0_#13_ci_summary.html, > CASSANDRA-19558-5.0_#13_results_details.tar.xz, > CASSANDRA-19558-5.0_#16_ci_summary.html, > CASSANDRA-19558-5.0_#16_results_details.tar.xz, > CASSANDRA-19558_#8_ci_summary.html, CASSANDRA-19558_#8_results_details.tar.xz > > > A few follow up improvements and bug fixes for the standalone jenkinsfile. > - add at top a list of test failures in ci_summary.html > - docker scripts always try to login (as base images need to be pulled too) > - move simulator-dtests to large containers (they need 8g just heap) > - in ubuntu2004_test.docker make sure /home/cassandra exists and has correct > perms (from marcuse) > - persist the jenkinsfile parameters from run to run (important for the > post-commit jobs to keep their non-default branch and profile values) (was > CASSANDRA-19536) > - increase jvm-dtest splits from 8 to 12 > - when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile > generateTestReports() with manual wget of test files, allowing the summary > phase to be run on any agent (copyArtifact would take >4hrs otherwise) (was > INFRA-25694) > - copy ci_summary.html and results_details.tar.xz to nightlies -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19558) Standalone jenkinsfile first round bug fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840060#comment-17840060 ] Michael Semb Wever edited comment on CASSANDRA-19558 at 4/23/24 10:50 AM: -- (1) is committed. wrt (2). hit a few small problems. INFRA-25738 being one (though unrelated to the patch). needed to make it work for the <5 jobs, and against different possible python versions (>=3.6). these are fixed. i'm ready to commit (2). Otherwise we have good test runs in #3 to #7 in [https://ci-cassandra.apache.org/job/Cassandra-devbranch-before-5-artifacts/] and in #588 to #594 in [https://ci-cassandra.apache.org/job/Cassandra-4.1-artifacts/] For (3) it is also needed ensuring it works when github is down (or goes down mid build)… was (Author: michaelsembwever): (1) is committed. working on (2). hit a few small problems. INFRA-25738 being one (though unrelated to the patch). needed to make it work for the <5 jobs, and against different possible python versions (>=3.6), and ensuring it works when github is down (or goes down mid build)… Otherwise we have good test runs in #3 to #7 in [https://ci-cassandra.apache.org/job/Cassandra-devbranch-before-5-artifacts/] and in #588 to #594 in [https://ci-cassandra.apache.org/job/Cassandra-4.1-artifacts/] > Standalone jenkinsfile first round bug fixes > > > Key: CASSANDRA-19558 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19558 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: CASSANDRA-19558_50_#5_ci_summary.html, > CASSANDRA-19558_50_#5_results_details.tar.xz, > CASSANDRA-19558-5.0_#13_ci_summary.html, > CASSANDRA-19558-5.0_#13_results_details.tar.xz, > CASSANDRA-19558-5.0_#16_ci_summary.html, > CASSANDRA-19558-5.0_#16_results_details.tar.xz, > CASSANDRA-19558_#8_ci_summary.html, CASSANDRA-19558_#8_results_details.tar.xz > > > A few follow up improvements and bug fixes for the standalone jenkinsfile. > - add at top a list of test failures in ci_summary.html > - docker scripts always try to login (as base images need to be pulled too) > - move simulator-dtests to large containers (they need 8g just heap) > - in ubuntu2004_test.docker make sure /home/cassandra exists and has correct > perms (from marcuse) > - persist the jenkinsfile parameters from run to run (important for the > post-commit jobs to keep their non-default branch and profile values) (was > CASSANDRA-19536) > - increase jvm-dtest splits from 8 to 12 > - when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile > generateTestReports() with manual wget of test files, allowing the summary > phase to be run on any agent (copyArtifact would take >4hrs otherwise) (was > INFRA-25694) > - copy ci_summary.html and results_details.tar.xz to nightlies -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19558) Standalone jenkinsfile first round bug fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17840060#comment-17840060 ] Michael Semb Wever edited comment on CASSANDRA-19558 at 4/23/24 10:48 AM: -- (1) is committed. working on (2). hit a few small problems. INFRA-25738 being one (though unrelated to the patch). needed to make it work for the <5 jobs, and against different possible python versions (>=3.6), and ensuring it works when github is down (or goes down mid build)… Otherwise we have good test runs in #3 to #7 in [https://ci-cassandra.apache.org/job/Cassandra-devbranch-before-5-artifacts/] and in #588 to #594 in [https://ci-cassandra.apache.org/job/Cassandra-4.1-artifacts/] was (Author: michaelsembwever): (1) is committed. working on (2). hit a few small problems. INFRA-25738 being one (though unrelated to the patch). needed to make it work for the <5 jobs, and against different possible python versions (>=3.6), and ensuring it works when github is down (or goes down mid build)… > Standalone jenkinsfile first round bug fixes > > > Key: CASSANDRA-19558 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19558 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: CASSANDRA-19558_50_#5_ci_summary.html, > CASSANDRA-19558_50_#5_results_details.tar.xz, > CASSANDRA-19558-5.0_#13_ci_summary.html, > CASSANDRA-19558-5.0_#13_results_details.tar.xz, > CASSANDRA-19558-5.0_#16_ci_summary.html, > CASSANDRA-19558-5.0_#16_results_details.tar.xz, > CASSANDRA-19558_#8_ci_summary.html, CASSANDRA-19558_#8_results_details.tar.xz > > > A few follow up improvements and bug fixes for the standalone jenkinsfile. > - add at top a list of test failures in ci_summary.html > - docker scripts always try to login (as base images need to be pulled too) > - move simulator-dtests to large containers (they need 8g just heap) > - in ubuntu2004_test.docker make sure /home/cassandra exists and has correct > perms (from marcuse) > - persist the jenkinsfile parameters from run to run (important for the > post-commit jobs to keep their non-default branch and profile values) (was > CASSANDRA-19536) > - increase jvm-dtest splits from 8 to 12 > - when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile > generateTestReports() with manual wget of test files, allowing the summary > phase to be run on any agent (copyArtifact would take >4hrs otherwise) (was > INFRA-25694) > - copy ci_summary.html and results_details.tar.xz to nightlies -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19558) Standalone jenkinsfile first round bug fixes
[
https://issues.apache.org/jira/browse/CASSANDRA-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837446#comment-17837446
]
Michael Semb Wever edited comment on CASSANDRA-19558 at 4/15/24 9:13 PM:
-
Ok, new round.
Two key improvements:
- {{options { durabilityHint('PERFORMANCE_OPTIMIZED') } }}
-
{{System.setProperty("org.jenkinsci.plugins.durabletask.BourneShellScript.USE_BINARY_WRAPPER",
"true");}}
Explicitly printing exit codes has also been added to the docker scripts.
Patch
- trunk
https://github.com/apache/cassandra/compare/trunk...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters
- 5.0
https://github.com/apache/cassandra/compare/cassandra-5.0...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters-5.0
CI
- 5.0 (jdk11 only)
https://ci-cassandra.apache.org/job/Cassandra-5-devbranch/108/
- 5.0 pre-commit internal [^CASSANDRA-19558-5.0_#13_ci_summary.html] ,
[^CASSANDRA-19558-5.0_#13_results_details.tar.xz]
- 5.0 post-commit internal …wip…
- 5.0 …wip…
- trunk …wip…
was (Author: michaelsembwever):
Ok, new round.
Patch
- trunk
https://github.com/apache/cassandra/compare/trunk...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters
- 5.0
https://github.com/apache/cassandra/compare/cassandra-5.0...thelastpickle:cassandra:mck/jenkinsfile-persist-parameters-5.0
CI
- 5.0 (jdk11 only)
https://ci-cassandra.apache.org/job/Cassandra-5-devbranch/108/
- 5.0 pre-commit internal [^CASSANDRA-19558-5.0_#13_ci_summary.html] ,
[^CASSANDRA-19558-5.0_#13_results_details.tar.xz]
- 5.0 post-commit internal …wip…
- 5.0 …wip…
- trunk …wip…
> Standalone jenkinsfile first round bug fixes
>
>
> Key: CASSANDRA-19558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19558
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Attachments: CASSANDRA-19558_50_#5_ci_summary.html,
> CASSANDRA-19558_50_#5_results_details.tar.xz,
> CASSANDRA-19558-5.0_#13_ci_summary.html,
> CASSANDRA-19558-5.0_#13_results_details.tar.xz,
> CASSANDRA-19558_#8_ci_summary.html, CASSANDRA-19558_#8_results_details.tar.xz
>
>
> A few follow up improvements and bug fixes for the standalone jenkinsfile.
> - add at top a list of test failures in ci_summary.html
> - docker scripts always try to login (as base images need to be pulled too)
> - move simulator-dtests to large containers (they need 8g just heap)
> - in ubuntu2004_test.docker make sure /home/cassandra exists and has correct
> perms (from marcuse)
> - persist the jenkinsfile parameters from run to run (important for the
> post-commit jobs to keep their non-default branch and profile values) (was
> CASSANDRA-19536)
> - increase jvm-dtest splits from 8 to 12
> - when on ci-cassandra, replace use of copyArtifacts in Jenkinsfile
> generateTestReports() with manual wget of test files, allowing the summary
> phase to be run on any agent (copyArtifact would take >4hrs otherwise) (was
> INFRA-25694)
> - copy ci_summary.html and results_details.tar.xz to nightlies
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
