subject:"\[1\/2\] hbase git commit\: HBASE\-18147 POC jenkinsfile for nightly checks."

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks.

2017-07-19 Thread busbey

Repository: hbase
Updated Branches:
  refs/heads/branch-1.1-HBASE-18147 [created] a60afbad9
  refs/heads/branch-1.2-HBASE-18147 [created] ae1e6d26b


HBASE-18147 POC jenkinsfile for nightly checks.

* Jenkinsfile that works for all current branches.
* adds dev-support script for setting parameters for our yetus nightly 
invocation
* adds ruby tools to dockerfile
* adds rubocop to dockerfile
* adds ruby-lint to dockerfile
* adds perlcritic to dockerfile


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/ae1e6d26
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/ae1e6d26
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/ae1e6d26

Branch: refs/heads/branch-1.2-HBASE-18147
Commit: ae1e6d26bf454cb53382053c1e0430ef50e3f0e0
Parents: 7d2175e
Author: Sean Busbey 
Authored: Thu Jul 20 00:51:07 2017 -0500
Committer: Sean Busbey 
Committed: Thu Jul 20 00:54:20 2017 -0500

--
 dev-support/Jenkinsfile| 269 
 dev-support/docker/Dockerfile  |  29 
 dev-support/hbase_nightly_yetus.sh |  86 ++
 3 files changed, 384 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/ae1e6d26/dev-support/Jenkinsfile
--
diff --git a/dev-support/Jenkinsfile b/dev-support/Jenkinsfile
new file mode 100644
index 000..5ff3d82
--- /dev/null
+++ b/dev-support/Jenkinsfile
@@ -0,0 +1,269 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+pipeline {
+  agent {
+node {
+  label 'Hadoop'
+}
+  }
+  triggers {
+cron('@daily')
+  }
+  options {
+buildDiscarder(logRotator(numToKeepStr: '30'))
+timeout (time: 6, unit: 'HOURS')
+timestamps()
+  }
+  environment {
+TOOLS = "${env.WORKSPACE}/tools"
+// where we check out to across stages
+BASEDIR = "${env.WORKSPACE}/component"
+YETUS_RELEASE = '0.5.0'
+// where we'll write everything from different steps.
+OUTPUT_RELATIVE_GENERAL = 'output-general'
+OUTPUTDIR_GENERAL = "${env.WORKSPACE}/output-general"
+OUTPUT_RELATIVE_JDK7 = 'output-jdk7'
+OUTPUTDIR_JDK7 = "${env.WORKSPACE}/output-jdk7"
+OUTPUT_RELATIVE_JDK8 = 'output-jdk8'
+OUTPUTDIR_JDK8 = "${env.WORKSPACE}/output-jdk8"
+PROJECT = 'hbase'
+PROJET_PERSONALITY = 
'https://raw.githubusercontent.com/apache/hbase/master/dev-support/hbase-personality.sh'
+// This section of the docs tells folks not to use the javadoc tag. older 
branches have our old version of the check for said tag.
+AUTHOR_IGNORE_LIST = 
'src/main/asciidoc/_chapters/developer.adoc,dev-support/test-patch.sh'
+WHITESPACE_IGNORE_LIST = '.*/generated/.*'
+// output from surefire; sadly the archive function in yetus only works on 
file names.
+ARCHIVE_PATTERN_LIST = 
'TEST-*.xml,org.apache.h*-output.txt,org.apache.h*.txt'
+// These tests currently have known failures. Once they burn down to 0, 
remove from here so that new problems will cause a failure.
+TESTS_FILTER = 
'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop'
+BRANCH_SPECIFIC_DOCKERFILE = "${env.BASEDIR}/dev-support/docker/Dockerfile"
+EXCLUDE_TESTS_URL = 
'https://builds.apache.org/job/HBase-Find-Flaky-Tests/lastSuccessfulBuild/artifact/excludes/'
+  }
+  parameters {
+booleanParam(name: 'USE_YETUS_PRERELEASE', defaultValue: false, 
description: '''Check to use the current HEAD of apache/yetus rather than our 
configured release.
+
+Should only be used manually when e.g. there is some non-work-aroundable 
issue in yetus we are checking a fix for.''')
+booleanParam(name: 'DEBUG', defaultValue: false, description: 'Produce a 
lot more meta-information.')
+  }
+  stages {
+stage ('yetus install') {
+  steps {
+sh  '''#!/usr/bin/env bash
+echo "Ensure we have a copy of Apache Yetus."
+if [[ true !=  "${USE_YETUS_PRERELEASE}" ]]; then
+  YETUS_DIR="${WORKSPACE}/yetus-${YETUS_RELEASE}"
+  echo "Checking for Yetus

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks. [Forced Update!]

2017-07-13 Thread busbey

Repository: hbase
Updated Branches:
  refs/heads/HBASE-18147 7df26b81c -> 96ba853b8 (forced update)
  refs/heads/branch-1.2-HBASE-18147 793a6827b -> 93f17f691 (forced update)


HBASE-18147 POC jenkinsfile for nightly checks.

* adds ruby tools to dockerfile
* adds rubocop to dockerfile
* adds ruby-lint to dockerfile
* adds perlcritic to dockerfile
* TODO branch with mulitple jdk versions


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/93f17f69
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/93f17f69
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/93f17f69

Branch: refs/heads/branch-1.2-HBASE-18147
Commit: 93f17f691589bb8ebddc587f29b6e6c46408f1d7
Parents: 2857b75
Author: Sean Busbey 
Authored: Tue Jul 4 15:12:38 2017 -0400
Committer: Sean Busbey 
Committed: Thu Jul 13 09:08:50 2017 -0500

--
 dev-support/Jenkinsfile   | 193 +
 dev-support/docker/Dockerfile |  29 ++
 2 files changed, 222 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/93f17f69/dev-support/Jenkinsfile
--
diff --git a/dev-support/Jenkinsfile b/dev-support/Jenkinsfile
new file mode 100644
index 000..98e70dd
--- /dev/null
+++ b/dev-support/Jenkinsfile
@@ -0,0 +1,193 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+pipeline {
+  agent {
+node {
+  label 'Hadoop'
+}
+  }
+  // work around for YETUS-508, requires maven outside of the dockerfile
+  tools {
+maven 'Maven (latest)'
+  }
+  triggers {
+cron('@daily')
+  }
+  options {
+buildDiscarder(logRotator(numToKeepStr: '30'))
+timeout (time: 6, unit: 'HOURS')
+timestamps()
+  }
+  environment {
+TOOLS = "${env.WORKSPACE}/tools"
+// where we check out to across stages
+BASEDIR = "${env.WORKSPACE}/component"
+YETUS_RELEASE = '0.4.0'
+// where we'll write everything from different steps.
+OUTPUT_RELATIVE = 'output'
+OUTPUTDIR = "${env.WORKSPACE}/output"
+  }
+  parameters {
+booleanParam(name: 'USE_YETUS_PRERELEASE', defaultValue: false, 
description: '''Check to use the current HEAD of apache/yetus rather than our 
configured release.
+
+Should only be used manually when e.g. there is some non-work-aroundable 
issue in yetus we are checking a fix for.''')
+booleanParam(name: 'DEBUG', defaultValue: false, description: 'Produce a 
lot more meta-information.')
+  }
+  stages {
+stage ('yetus check') {
+  environment {
+PROJECT = 'hbase'
+PROJET_PERSONALITY = 
'https://raw.githubusercontent.com/apache/hbase/master/dev-support/hbase-personality.sh'
+// This section of the docs tells folks not to use the javadoc tag. 
older branches have our old version of the check for said tag.
+AUTHOR_IGNORE_LIST = 
'src/main/asciidoc/_chapters/developer.adoc,dev-support/test-patch.sh'
+WHITESPACE_IGNORE_LIST = '.*/generated/.*'
+// output from surefire; sadly the archive function in yetus only 
works on file names.
+ARCHIVE_PATTERN_LIST = 
'TEST-*.xml,org.apache.h*-output.txt,org.apache.h*.txt'
+// These tests currently have known failures. Once they burn down to 0, remove 
from here so that new problems will cause a failure.
+TESTS_FILTER = 
'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop'
+BRANCH_SPECIFIC_DOCKERFILE = 
"${env.BASEDIR}/dev-support/docker/Dockerfile"
+EXCLUDE_TESTS_URL = 
'https://builds.apache.org/job/HBase-Find-Flaky-Tests/lastSuccessfulBuild/artifact/excludes/'
+  }
+  steps {
+// TODO we can move the yetus install into a different stage and then use 
stash to deploy it.
+sh  '''#!/usr/bin/env bash
+printenv
+echo "Ensure we have a copy of Apache Yetus."
+if [[ true !=  "${USE_YETUS_PRERELEASE}" ]]; then
+  YETUS_DIR="${WORKSPACE}/yetus-${YETUS_RELEASE}"
+  echo "Checking for Yetus ${YETUS_RELEASE} in '${YETUS_DIR}'"
+  if [ !

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks. [Forced Update!]

2017-07-13 Thread busbey

Repository: hbase
Updated Branches:
  refs/heads/HBASE-18147 ce2f9bdf0 -> 7df26b81c (forced update)
  refs/heads/branch-1.2-HBASE-18147 25f955681 -> 793a6827b (forced update)


HBASE-18147 POC jenkinsfile for nightly checks.

* adds ruby tools to dockerfile
* adds rubocop to dockerfile
* adds ruby-lint to dockerfile
* adds perlcritic to dockerfile
* TODO branch with mulitple jdk versions


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/7df26b81
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/7df26b81
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/7df26b81

Branch: refs/heads/HBASE-18147
Commit: 7df26b81c53c3d1641518d8a0bf731ae090db326
Parents: 22df926
Author: Sean Busbey 
Authored: Tue Jul 4 15:12:38 2017 -0400
Committer: Sean Busbey 
Committed: Thu Jul 13 09:01:44 2017 -0500

--
 dev-support/Jenkinsfile   | 196 +
 dev-support/docker/Dockerfile |  29 ++
 2 files changed, 225 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/7df26b81/dev-support/Jenkinsfile
--
diff --git a/dev-support/Jenkinsfile b/dev-support/Jenkinsfile
new file mode 100644
index 000..dba9158
--- /dev/null
+++ b/dev-support/Jenkinsfile
@@ -0,0 +1,196 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+pipeline {
+  agent {
+node {
+  label 'Hadoop'
+}
+  }
+  // work around for YETUS-508, requires maven outside of the dockerfile
+  tools {
+maven 'Maven (latest)'
+  }
+  triggers {
+cron('@daily')
+  }
+  options {
+buildDiscarder(logRotator(numToKeepStr: '30'))
+timeout (time: 6, unit: 'HOURS')
+timestamps()
+  }
+  environment {
+TOOLS = "${env.WORKSPACE}/tools"
+// where we check out to across stages
+BASEDIR = "${env.WORKSPACE}/component"
+YETUS_RELEASE = '0.4.0'
+// where we'll write everything from different steps.
+OUTPUT_RELATIVE = 'output'
+OUTPUTDIR = "${env.WORKSPACE}/output"
+  }
+  parameters {
+booleanParam(name: 'USE_YETUS_PRERELEASE', defaultValue: false, 
description: '''Check to use the current HEAD of apache/yetus rather than our 
configured release.
+
+Should only be used manually when e.g. there is some non-work-aroundable 
issue in yetus we are checking a fix for.''')
+booleanParam(name: 'DEBUG', defaultValue: flase, description: 'Produce a 
lot more meta-information.')
+  }
+  stages {
+stage ('yetus check') {
+  environment {
+PROJECT = 'hbase'
+PROJET_PERSONALITY = 
'https://raw.githubusercontent.com/apache/hbase/master/dev-support/hbase-personality.sh'
+// This section of the docs tells folks not to use the javadoc tag. 
older branches have our old version of the check for said tag.
+AUTHOR_IGNORE_LIST = 
'src/main/asciidoc/_chapters/developer.adoc,dev-support/test-patch.sh'
+WHITESPACE_IGNORE_LIST = '.*/generated/.*'
+// output from surefire; sadly the archive function in yetus only 
works on file names.
+ARCHIVE_PATTERN_LIST = 
'TEST-*.xml,org.apache.h*-output.txt,org.apache.h*.txt'
+// These tests currently have known failures. Once they burn down to 0, remove 
from here so that new problems will cause a failure.
+TESTS_FILTER = 
'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop'
+BRANCH_SPECIFIC_DOCKERFILE = 
"${env.BASEDIR}/dev-support/docker/Dockerfile"
+EXCLUDE_TESTS_URL = 
'https://builds.apache.org/job/HBase-Find-Flaky-Tests/lastSuccessfulBuild/artifact/excludes/'
+  }
+  steps {
+// TODO we can move the yetus install into a different stage and then use 
stash to deploy it.
+sh  '''#!/usr/bin/env bash
+printenv
+echo "Ensure we have a copy of Apache Yetus."
+if [[ true !=  "${USE_YETUS_PRERELEASE}" ]]; then
+  YETUS_DIR="${WORKSPACE}/yetus-${YETUS_RELEASE}"
+  echo "Checking for Yetus ${YETUS_RELEASE} in '${YETUS_DIR}'"
+  if [ ! -d

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks. [Forced Update!]

2017-07-13 Thread busbey

Repository: hbase
Updated Branches:
  refs/heads/HBASE-18147 dbf67e723 -> ce2f9bdf0 (forced update)
  refs/heads/branch-1.2-HBASE-18147 1e73dd19d -> 25f955681 (forced update)


HBASE-18147 POC jenkinsfile for nightly checks.

* adds ruby tools to dockerfile
* adds rubocop to dockerfile
* adds ruby-lint to dockerfile
* adds perlcritic to dockerfile
* TODO branch with mulitple jdk versions


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/25f95568
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/25f95568
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/25f95568

Branch: refs/heads/branch-1.2-HBASE-18147
Commit: 25f955681cac5c8e843f503e805ca2b82ae11058
Parents: 2857b75
Author: Sean Busbey 
Authored: Tue Jul 4 15:12:38 2017 -0400
Committer: Sean Busbey 
Committed: Thu Jul 13 08:14:27 2017 -0500

--
 dev-support/Jenkinsfile   | 193 +
 dev-support/docker/Dockerfile |  29 ++
 2 files changed, 222 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/25f95568/dev-support/Jenkinsfile
--
diff --git a/dev-support/Jenkinsfile b/dev-support/Jenkinsfile
new file mode 100644
index 000..9436eb3
--- /dev/null
+++ b/dev-support/Jenkinsfile
@@ -0,0 +1,193 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+pipeline {
+  agent {
+node {
+  label 'Hadoop'
+}
+  }
+  // work around for YETUS-508, requires maven outside of the dockerfile
+  tools {
+maven 'Maven (latest)'
+  }
+  triggers {
+cron('@daily')
+  }
+  options {
+buildDiscarder(logRotator(numToKeepStr: '30'))
+timeout (time: 6, unit: 'HOURS')
+timestamps()
+  }
+  environment {
+TOOLS = "${env.WORKSPACE}/tools"
+// where we check out to across stages
+BASEDIR = "${env.WORKSPACE}/component"
+YETUS_RELEASE = '0.4.0'
+// where we'll write everything from different steps.
+OUTPUT_RELATIVE = 'output'
+OUTPUTDIR = "${env.WORKSPACE}/output"
+  }
+  parameters {
+booleanParam(name: 'USE_YETUS_PRERELEASE', defaultValue: false, 
description: '''Check to use the current HEAD of apache/yetus rather than our 
configured release.
+
+Should only be used manually when e.g. there is some non-work-aroundable 
issue in yetus we are checking a fix for.''')
+booleanParam(name: 'DEBUG', defaultValue: flase, description: 'Produce a 
lot more meta-information.')
+  }
+  stages {
+stage ('yetus check') {
+  environment {
+PROJECT = 'hbase'
+PROJET_PERSONALITY = 
'https://raw.githubusercontent.com/apache/hbase/master/dev-support/hbase-personality.sh'
+// This section of the docs tells folks not to use the javadoc tag. 
older branches have our old version of the check for said tag.
+AUTHOR_IGNORE_LIST = 
'src/main/asciidoc/_chapters/developer.adoc,dev-support/test-patch.sh'
+WHITESPACE_IGNORE_LIST = '.*/generated/.*'
+// output from surefire; sadly the archive function in yetus only 
works on file names.
+ARCHIVE_PATTERN_LIST = 
'TEST-*.xml,org.apache.h*-output.txt,org.apache.h*.txt'
+// These tests currently have known failures. Once they burn down to 0, remove 
from here so that new problems will cause a failure.
+TESTS_FILTER = 
'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop'
+BRANCH_SPECIFIC_DOCKERFILE = 
"${env.BASEDIR}/dev-support/docker/Dockerfile"
+EXCLUDE_TESTS_URL = 
'https://builds.apache.org/job/HBase-Find-Flaky-Tests/lastSuccessfulBuild/artifact/excludes/'
+  }
+  steps {
+// TODO we can move the yetus install into a different stage and then use 
stash to deploy it.
+sh  '''#!/usr/bin/env bash
+printenv
+echo "Ensure we have a copy of Apache Yetus."
+if [[ true !=  "${USE_YETUS_PRERELEASE}" ]]; then
+  YETUS_DIR="${WORKSPACE}/yetus-${YETUS_RELEASE}"
+  echo "Checking for Yetus ${YETUS_RELEASE} in '${YETUS_DIR}'"
+  if [ !

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks. [Forced Update!]

2017-07-13 Thread busbey

Repository: hbase
Updated Branches:
  refs/heads/HBASE-18147 ad0bf704c -> dbf67e723 (forced update)
  refs/heads/branch-1.2-HBASE-18147 e6e4da9b7 -> 1e73dd19d (forced update)


HBASE-18147 POC jenkinsfile for nightly checks.

* adds ruby tools to dockerfile
* adds rubocop to dockerfile
* adds ruby-lint to dockerfile
* adds perlcritic to dockerfile
* TODO branch with mulitple jdk versions
* TODO verify flaky list is in use


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/dbf67e72
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/dbf67e72
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/dbf67e72

Branch: refs/heads/HBASE-18147
Commit: dbf67e7230e1c24afff0abaa172f174134cadf4a
Parents: 22df926
Author: Sean Busbey 
Authored: Tue Jul 4 15:12:38 2017 -0400
Committer: Sean Busbey 
Committed: Thu Jul 13 01:09:59 2017 -0500

--
 dev-support/Jenkinsfile   | 174 +
 dev-support/docker/Dockerfile |  29 +++
 2 files changed, 203 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/dbf67e72/dev-support/Jenkinsfile
--
diff --git a/dev-support/Jenkinsfile b/dev-support/Jenkinsfile
new file mode 100644
index 000..91becf1
--- /dev/null
+++ b/dev-support/Jenkinsfile
@@ -0,0 +1,174 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+pipeline {
+  agent {
+node {
+  label 'Hadoop'
+}
+  }
+  // work around for YETUS-508, requires maven outside of the dockerfile
+  tools {
+maven 'Maven (latest)'
+  }
+  triggers {
+cron('@daily')
+  }
+  options {
+buildDiscarder(logRotator(numToKeepStr: '30'))
+timeout (time: 6, unit: 'HOURS')
+timestamps()
+  }
+  environment {
+TOOLS = "${env.WORKSPACE}/tools"
+// where we check out to across stages
+BASEDIR = "${env.WORKSPACE}/component"
+YETUS_RELEASE = '0.4.0'
+// where we'll write everything from different steps.
+OUTPUTDIR = "${env.WORKSPACE}/artifacts"
+  }
+  parameters {
+booleanParam(name: 'USE_YETUS_PRERELEASE', defaultValue: false, 
description: '''Check to use the current HEAD of apache/yetus rather than our 
configured release.
+
+Should only be used manually when e.g. there is some non-work-aroundable 
issue in yetus we are checking a fix for.''')
+  }
+  stages {
+stage ('yetus check') {
+  environment {
+PROJECT = 'hbase'
+PROJET_PERSONALITY = 
'https://raw.githubusercontent.com/apache/hbase/master/dev-support/hbase-personality.sh'
+// This section of the docs tells folks not to use the javadoc tag. 
older branches have our old version of the check for said tag.
+AUTHOR_IGNORE_LIST = 
'src/main/asciidoc/_chapters/developer.adoc,dev-support/test-patch.sh'
+WHITESPACE_IGNORE_LIST = '.*/generated/.*'
+// output from surefire; sadly the archive function in yetus only 
works on file names.
+ARCHIVE_PATTERN_LIST = 
'TEST-*.xml,org.apache.h*-output.txt,org.apache.h*.txt'
+// These tests currently have known failures. Once they burn down to 0, remove 
from here so that new problems will cause a failure.
+TESTS_FILTER = 
'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop'
+BRANCH_SPECIFIC_DOCKERFILE = 
"${env.BASEDIR}/dev-support/docker/Dockerfile"
+EXCLUDE_TESTS_URL = 
'https://builds.apache.org/job/HBase-Find-Flaky-Tests/lastSuccessfulBuild/artifact/excludes/'
+  }
+  steps {
+// TODO we can move the yetus install into a different stage and then use 
stash to deploy it.
+sh  '''#!/usr/bin/env bash
+printenv
+echo "Ensure we have a copy of Apache Yetus."
+if [[ true !=  "${USE_YETUS_PRERELEASE}" ]]; then
+  YETUS_DIR="${WORKSPACE}/yetus-${YETUS_RELEASE}"
+  echo "Checking for Yetus ${YETUS_RELEASE} in '${YETUS_DIR}'"
+  if [ ! -d "${YETUS_DIR}" ]; then
+echo "New download of Apache Yetus version ${YETUS_RELEASE}."
+rm -rf

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks. [Forced Update!]

2017-07-12 Thread busbey

Repository: hbase
Updated Branches:
  refs/heads/HBASE-18147 28a3b5ede -> 13300585f (forced update)
  refs/heads/branch-1.2-HBASE-18147 149bbba02 -> e6e4da9b7 (forced update)


HBASE-18147 POC jenkinsfile for nightly checks.

* adds ruby tools to dockerfile
* adds rubocop to dockerfile
* adds ruby-lint to dockerfile
* adds perlcritic to dockerfile
* TODO branch with mulitple jdk versions
* TODO verify flaky list is in use


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/e6e4da9b
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/e6e4da9b
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/e6e4da9b

Branch: refs/heads/branch-1.2-HBASE-18147
Commit: e6e4da9b70a29639279e683f29b8837a7d639fbe
Parents: 2857b75
Author: Sean Busbey 
Authored: Tue Jul 4 15:12:38 2017 -0400
Committer: Sean Busbey 
Committed: Wed Jul 12 20:28:53 2017 -0500

--
 dev-support/Jenkinsfile   | 170 +
 dev-support/docker/Dockerfile |  29 +++
 2 files changed, 199 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/hbase/blob/e6e4da9b/dev-support/Jenkinsfile
--
diff --git a/dev-support/Jenkinsfile b/dev-support/Jenkinsfile
new file mode 100644
index 000..27538c7
--- /dev/null
+++ b/dev-support/Jenkinsfile
@@ -0,0 +1,170 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+pipeline {
+  agent {
+node {
+  label 'Hadoop'
+}
+  }
+  // work around for YETUS-508, requires maven outside of the dockerfile
+  tools {
+maven 'Maven (latest)'
+  }
+  triggers {
+cron('@daily')
+  }
+  options {
+buildDiscarder(logRotator(numToKeepStr: '30'))
+timeout (time: 6, unit: 'HOURS')
+timestamps()
+  }
+  environment {
+TOOLS = "${env.WORKSPACE}/tools"
+// BASEDIR must be here because otherwise it won't work in post
+BASEDIR = "${env.WORKSPACE}/component"
+YETUS_RELEASE = '0.4.0'
+// OUTPUTDIR must be here because otherwise it won't work in post
+OUTPUTDIR = "${env.WORKSPACE}/artifacts"
+  }
+  parameters {
+booleanParam(name: 'USE_YETUS_PRERELEASE', defaultValue: false, 
description: '''Check to use the current HEAD of apache/yetus rather than our 
configured release.
+
+Should only be used manually when e.g. there is some non-work-aroundable 
issue in yetus we are checking a fix for.''')
+  }
+  stages {
+stage ('yetus check') {
+  environment {
+PROJECT = 'hbase'
+PROJET_PERSONALITY = 
'https://raw.githubusercontent.com/apache/hbase/master/dev-support/hbase-personality.sh'
+// This section of the docs tells folks not to use the javadoc tag. 
older branches have our old version of the check for said tag.
+AUTHOR_IGNORE_LIST = 
'src/main/asciidoc/_chapters/developer.adoc,dev-support/test-patch.sh'
+WHITESPACE_IGNORE_LIST = '.*/generated/.*'
+// These tests currently have known failures. Once they burn down to 0, remove 
from here so that new problems will cause a failure.
+TESTS_FILTER = 
'cc,checkstyle,javac,javadoc,pylint,shellcheck,whitespace,perlcritic,ruby-lint,rubocop'
+BRANCH_SPECIFIC_DOCKERFILE = 
"${env.BASEDIR}/dev-support/docker/Dockerfile"
+EXCLUDE_TESTS_URL = 
'https://builds.apache.org/job/HBase-Find-Flaky-Tests/lastSuccessfulBuild/artifact/excludes/'
+  }
+  steps {
+// TODO we can move the yetus install into a different stage and then use 
stash to deploy it.
+sh  '''#!/usr/bin/env bash
+printenv
+echo "Ensure we have a copy of Apache Yetus."
+if [[ true !=  "${USE_YETUS_PRERELEASE}" ]]; then
+  YETUS_DIR="${WORKSPACE}/yetus-${YETUS_RELEASE}"
+  echo "Checking for Yetus ${YETUS_RELEASE} in '${YETUS_DIR}'"
+  if [ ! -d "${YETUS_DIR}" ]; then
+echo "New download of Apache Yetus version ${YETUS_RELEASE}."
+rm -rf "${WORKSPACE}/.gpg"
+mkdir -p "${WORKSPACE}/.gpg"
+chmod -R 700 "${WORKSPACE}/.gpg"
+
+echo "install yetus project KEYS"
+curl -L

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks.

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks. [Forced Update!]

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks. [Forced Update!]

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks. [Forced Update!]

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks. [Forced Update!]

[1/2] hbase git commit: HBASE-18147 POC jenkinsfile for nightly checks. [Forced Update!]

6 matches

Site Navigation

Mail list logo

Footer information