Re: Docker agent build script wrapper

[Oleksandr Rudak]

>
> We have reached acceptable solution for the problem. It is loading groovy 
> script with black magic around closure. Attaching code for the community 
> benefit.
>

It boils down to construct in the loaded groovy module:
def container(myImageName, Closure body){
docker.image(myImageName).inside(dockerArgs(), body)
}


 And then using it passing closure body:
def myImage = 'php:latest'
def modelId = drun.modelId()

echo modelId

drun.container(myImage) {
stage ('setup'){
sh 'id'
echo 'value = ' + modelId
}

stage('stage 1'){
sh '/usr/local/bin/php -version'
}
}

There is also custom step returning map of properties from the environment 
implemented as Java class:
public class DrunPropertiesStep extends Step {
...
public static class Execution extends SynchronousStepExecution {
private static final long serialVersionUID = 1;

Execution(StepContext context) {
super(context);
}

@Override
protected Object run() throws Exception {
Run run = getContext().get(Run.class);
//todo: implement

JSONObject json = Utils.getJson(run.getRootDir().getPath(), 
run.getLogReader());

Map map = new HashMap();
Iterator iter = json.keys();
while( iter.hasNext() ) {
String key = iter.next().toString();

map.put(key, json.getString(key));
}

return map;
}
}
...
}

Hope it will help to prevent someone from banging a head against the 
keyboard :)

Thanks
Alex

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/3cd3005a-9a64-48a4-b67b-2cae076d0e93%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


drun.groovy
Description: Binary data


Jenkinsfile
Description: Binary data

Docker agent build script wrapper

[Oleksandr Rudak]

Hi,

We are building scientific model running and testing environment product. 
We use Jenkins as a core building, integration and model running tool. Now 
our users are scientist and have little technological background so we look 
to fold our current Jenkinsfile into something more compact and 
transparent. We've tried few things. SimpleBuildStep, which is too basic to 
handle agent/docker stuff. Step is too complicated as it involved 
distributed execution parallelism. Also we tried 'load' to load Jenkinsfile 
generated in the working directory. All ways have pros/cons but none 
provides in full what we try to accomplish.

Any ideas how to do such wrapping? I think about custom `agent` which would 
provide docker context and will imply the rest of the configuration and 
hide it from user. Possibly some workflow extension or something like this. 
Any thoughts are welcome.

Below is the anticipated Jenkinsfile how it would ideally look. Attached is 
the original Jenkins with detailed steps we want to cloak from user.

node {
stage ('setup') {
setupEnv (dockerImage:'dockerImage123')
}
stage ('train') {
runNotebook (notebookPath:'notebookPath212')
// { prepare, run, archive, legion }
// runNotebook (notebookPath:'notebookPath213')
}
stage ('deploy') {
deployModel ()
}
stage ('test') {
testModel ()
}
}



Thanks
Alex

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-dev/5a2441d6-72ef-45f7-a415-356510c8a9cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
pipeline {
agent { 
docker {
image 'drun.kharlamov.biz/drun/base-python-image:latest'
args '-v drun:/drun -u 0:0 --network=drun_drun_root -e 
"DOCKER_TLS_VERIFY=1" -e "DOCKER_HOST=${DOCKER_HOST}" -e 
"DOCKER_CERT_PATH=${DOCKER_CERT_PATH}" -e 
"MODEL_SERVER_URL=${DRUN_MODEL_SERVER_URL}"'
}
}
stages {
stage('clone'){
steps {
sh '''
pip install -i https://drun.kharlamov.biz/pypi/ 
drun
'''
}
}
stage('run notebook'){
steps {
sh '''
jupyter nbconvert --execute 
"Digit-Recognition/Recognizing digits.ipynb"
cp "Digit-Recognition/Recognizing digits.html" 
notebook.html
mkdir -p release-models
cp -rf /drun/* release-models/
'''
}
}
stage('archive notebook artifacts'){
steps {
archiveArtifacts 
'release-models/image_recognize.model'
archiveArtifacts 'Digit-Recognition/*.html'
archiveArtifacts 'notebook.html'
}
}
stage('deploy to legion'){
steps {
sh '''
legion build 
release-models/image_recognize.model
legion deploy --model-id recognize_digits
'''
sleep time: 20, unit: 'SECONDS'
}
}
stage('run tests'){
steps {
sh '''
cd "Digit-Recognition/tests"
nosetests --with-xunit
'''
junit 'Digit-Recognition/tests/nosetests.xml'
}
}
}
}