[
https://issues.apache.org/jira/browse/HBASE-24936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418189#comment-17418189
]
Duo Zhang commented on HBASE-24936:
-----------------------------------
We were listed again by the INFRA team to eat too much space and they mentioned
nightlies.apache.org again...
This is what I got from the jenkins file of Cassandra team:
{code}
stage('Summary') {
steps {
sh "rm -fR cassandra-builds"
sh "git clone --depth 1 --single-branch
https://gitbox.apache.org/repos/asf/cassandra-builds.git";
sh "./cassandra-builds/build-scripts/cassandra-test-report.sh"
junit testResults:
'**/build/test/**/TEST*.xml,**/cqlshlib.xml,**/nosetests.xml',
testDataPublishers: [[$class: 'StabilityTestDataPublisher']]
// the following should fail on any installation other than
ci-cassandra.apache.org
// TODO: keep jenkins infrastructure related settings in
`cassandra_job_dsl_seed.groovy`
warnError('cannot send notifications') {
script {
changes = formatChanges(currentBuild.changeSets)
echo "changes: ${changes}"
}
slackSend channel: '#cassandra-builds', message: ":apache:
<${env.BUILD_URL}|${currentBuild.fullDisplayName}> completed:
${currentBuild.result}.
<https://github.com/apache/cassandra/commit/${env.GIT_COMMIT}|${env.GIT_COMMIT}>\n${changes}"
emailext to: '[email protected]', subject: "Build
complete: ${currentBuild.fullDisplayName} [${currentBuild.result}]
${env.GIT_COMMIT}", presendScript:
'${FILE,path="cassandra-builds/jenkins-dsl/cassandra_email_presend.groovy"}',
body: '''
-------------------------------------------------------------------------------
Build ${ENV,var="JOB_NAME"} #${BUILD_NUMBER} ${BUILD_STATUS}
URL: ${BUILD_URL}
-------------------------------------------------------------------------------
Changes:
${CHANGES}
-------------------------------------------------------------------------------
Failed Tests:
${FAILED_TESTS,maxTests=500,showMessage=false,showStack=false}
-------------------------------------------------------------------------------
For complete test report and logs see
https://nightlies.apache.org/cassandra/${JOB_NAME}/${BUILD_NUMBER}/
'''
}
sh "echo \"cassandra-builds at: `git -C cassandra-builds log -1
--pretty=format:'%h %an %ad %s'`\" > builds.head"
sh "find . -type f -name \\*.head -exec cat {} \\;"
sh "xz TESTS-TestSuites.xml"
sh "echo \"For test report and logs see
https://nightlies.apache.org/cassandra/${JOB_NAME}/${BUILD_NUMBER}/\"";
}
post {
always {
sshPublisher(publishers: [sshPublisherDesc(configName:
'Nightlies', transfers: [sshTransfer(remoteDirectory:
'cassandra/${JOB_NAME}/${BUILD_NUMBER}/', sourceFiles:
'TESTS-TestSuites.xml.xz')])])
}
}
}
{code}
Let's see if we could use these scripts too...
> review Jenkins build artifacts
> ------------------------------
>
> Key: HBASE-24936
> URL: https://issues.apache.org/jira/browse/HBASE-24936
> Project: HBase
> Issue Type: Task
> Reporter: Sean Busbey
> Assignee: Sean Busbey
> Priority: Critical
> Attachments: usage-hbase-find-flaky.zip, usage-hbase-full.zip
>
>
> Post move to the ci-hadoop build servers we are now the biggest user of
> space. That is not a problem in and of itself, but the master node has run
> out of disk space twice now. As of this snapshot we are using 125GB of
> storage and the next largest project is only using 20GB.
> https://paste.apache.org/kyrds
> We should review our builds for any issues and come up with expectations for
> what our steady-state disk usage should look like
> * we are supposed to compress any test logs (usually this gets us 90-99%
> space savings)
> * we are supposed to clean up workspaces when jobs are done
> * we are supposed to keep a fixed window of prior builds (either by days or
> number of runs)
> If all of our jobs are currently following these guidelines, another
> possibility is to push the artifacts we need over to
> [nightlies.a.o|https://nightlies.apache.org/authoring.html]. Barring that, we
> should formally request asf infra set up [a plugin for storing artifact on
> s3|https://plugins.jenkins.io/artifact-manager-s3/].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
