This is an automated email from the ASF dual-hosted git repository.
sekikn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/bigtop.git
The following commit(s) were added to refs/heads/master by this push:
new f8f3a14 BIGTOP-3276. Update supported distros in the toolchain
manifests for v1.5.0
f8f3a14 is described below
commit f8f3a142b293916ac107ab1d233601cc827681bc
Author: Kengo Seki <[email protected]>
AuthorDate: Fri Dec 6 14:19:54 2019 +0900
BIGTOP-3276. Update supported distros in the toolchain manifests for v1.5.0
Signed-off-by: Evans Ye <[email protected]>
---
README.md | 10 +++++++++
bigtop-deploy/puppet/README.md | 2 +-
bigtop_toolchain/README.md | 14 ++++++------
bigtop_toolchain/bin/puppetize.sh | 22 ++++++++++---------
bigtop_toolchain/manifests/cleanup.pp | 2 +-
.../{deployment-tools.pp => deployment_tools.pp} | 2 +-
.../{development-tools.pp => development_tools.pp} | 2 +-
bigtop_toolchain/manifests/env.pp | 11 ++++++++--
bigtop_toolchain/manifests/gnupg.pp | 2 +-
bigtop_toolchain/manifests/jdk.pp | 25 +++++++++++++++++++---
bigtop_toolchain/manifests/packages.pp | 24 ++++++++++++++++-----
.../{puppet-modules.pp => puppet_modules.pp} | 4 ++--
...-modules-prereq.pp => puppet_modules_prereq.pp} | 4 ++--
bigtop_toolchain/manifests/renv.pp | 2 +-
build.gradle | 6 +++---
15 files changed, 92 insertions(+), 40 deletions(-)
diff --git a/README.md b/README.md
index b0c516c..97d53d7 100644
--- a/README.md
+++ b/README.md
@@ -156,6 +156,16 @@ __On all systems, Building Apache Bigtop requires certain
set of tools__
This build task expected Puppet to be installed; user has to have sudo
permissions. The task will pull down and install
all development dependencies, frameworks and SDKs, required to build the
stack on your platform.
+ Before executing the above command, user can use the following script to
install Puppet:
+
+ sudo bigtop_toolchain/bin/puppetize.sh
+
+ Note for CentOS (and RHEL, which is not supported officially but on a best
effort basis) 8 users: on these distros,
+ puppetize.sh installs the puppet command into /opt/puppetlabs/bin, which is
not included usually in secure_path defined in /etc/sudoers.
+ So users may have to add that path to secure_path manually.
+ Also, RHEL 8 users may have to enable their subscriptions themselves for
using EPEL.
+ cf. https://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F
+
To immediately set environment after running toolchain, run
. /etc/profile.d/bigtop.sh
diff --git a/bigtop-deploy/puppet/README.md b/bigtop-deploy/puppet/README.md
index 727594b..5c15bfb 100644
--- a/bigtop-deploy/puppet/README.md
+++ b/bigtop-deploy/puppet/README.md
@@ -103,7 +103,7 @@ gradle toolchain-puppetmodules # if you already have JAVA
installed
or
```
-puppet apply --modulepath=<path_to_bigtop> -e "include
bigtop_toolchain::puppet-modules"
+puppet apply --modulepath=<path_to_bigtop> -e "include
bigtop_toolchain::puppet_modules"
```
This will install the following module(s) for you:
diff --git a/bigtop_toolchain/README.md b/bigtop_toolchain/README.md
index 372ae77..9fcc333 100644
--- a/bigtop_toolchain/README.md
+++ b/bigtop_toolchain/README.md
@@ -20,11 +20,11 @@ bigtop-toolchain
Puppet module for configuring a host for building BigTop. It installs:
-**Apache Ant 1.9.4**
+**Apache Ant 1.9**
-**OpenJDK 1.7**
+**OpenJDK 1.8**
-**Apache Maven 3.2.5**
+**Apache Maven 3.5**
**Gradle 2.4**
@@ -52,11 +52,11 @@ Or installed as a whole with:
include bigtop_toolchain::installer
}
-It will create a user jenkins with the required environment variables set for
+It will create a user jenkins with the required environment variables set for
building BigTop:
```
MAVEN_HOME=/usr/local/maven
-JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64
+JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk.x86_64
ANT_HOME=/usr/local/ant
GRADLE_HOME=/usr/local/gradle
PATH=$MAVEN_HOME/bin:$ANT_HOME/bin:$FORREST_HOME/bin:$GRADLE_HOME/bin:$PATH
@@ -74,7 +74,7 @@ where <path_to_bigtop> is the cloned git repo.
This is a separated set of manifests that helps to setup tools for Bigtop
deployment.
The usage is as below:
- puppet apply --modulepath=<path_to_bigtop> -e "include
bigtop_toolchain::deployment-tools"
+ puppet apply --modulepath=<path_to_bigtop> -e "include
bigtop_toolchain::deployment_tools"
By applying the snippet, Vagrant will be installed(the Docker installation
will be added soon).
@@ -86,7 +86,7 @@ As Groovy isn't required (yet!) for creation of a Bigtop
stack, this environment
In case you system doesn't have already installed version of Bigtop
recommended Groovy environment,
you should be able to so easily by running
- puppet apply --modulepath=<path_to_bigtop> -e "include
bigtop_toolchain::development-tools"
+ puppet apply --modulepath=<path_to_bigtop> -e "include
bigtop_toolchain::development_tools"
Potentially, we'll be adding more development tools in this manifest.
diff --git a/bigtop_toolchain/bin/puppetize.sh
b/bigtop_toolchain/bin/puppetize.sh
index 70eeb90..2422719 100755
--- a/bigtop_toolchain/bin/puppetize.sh
+++ b/bigtop_toolchain/bin/puppetize.sh
@@ -21,22 +21,18 @@ if [ -f /etc/os-release ]; then
fi
case ${ID}-${VERSION_ID} in
- fedora-26)
- dnf -y install yum-utils
- dnf -y check-update
- dnf -y install hostname findutils curl sudo unzip wget puppet
puppetlabs-stdlib procps-ng
+ fedora-31)
+ dnf -y install yum-utils
+ dnf -y check-update
+ dnf -y install hostname findutils curl sudo unzip wget puppet
puppetlabs-stdlib procps-ng
;;
- ubuntu-16.04)
+ ubuntu-16.04 | ubuntu-18.04)
apt-get update
apt-get -y install wget curl sudo unzip puppet
software-properties-common puppet-module-puppetlabs-apt
puppet-module-puppetlabs-stdlib
;;
- debian-9*)
+ debian-9* | debian-10*)
apt-get update
apt-get -y install wget curl sudo unzip puppet
puppet-module-puppetlabs-apt puppet-module-puppetlabs-stdlib systemd-sysv
- ;;
- opensuse-42.3)
- zypper --gpg-auto-import-keys install -y curl sudo unzip wget puppet
suse-release ca-certificates-mozilla net-tools tar
- puppet module install puppetlabs-stdlib
;;
centos-7*)
rpm -ivh
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
@@ -47,6 +43,12 @@ case ${ID}-${VERSION_ID} in
yum -y install hostname curl sudo unzip wget puppet
puppet module install puppetlabs-stdlib --version 4.12.0
;;
+ centos-8* | rhel-8*)
+ rpm -Uvh https://yum.puppet.com/puppet5-release-el-8.noarch.rpm
+ dnf -y check-update
+ dnf -y install puppet-agent
+ /opt/puppetlabs/bin/puppet module install puppetlabs-stdlib
+ ;;
*)
echo "Unsupported OS ${ID}-${VERSION_ID}."
exit 1
diff --git a/bigtop_toolchain/manifests/cleanup.pp
b/bigtop_toolchain/manifests/cleanup.pp
index 94929e4..edf20a0 100644
--- a/bigtop_toolchain/manifests/cleanup.pp
+++ b/bigtop_toolchain/manifests/cleanup.pp
@@ -15,7 +15,7 @@
class bigtop_toolchain::cleanup {
$packager_cleanup = $operatingsystem ? {
- /(?i:(centos|fedora|amazon))/ => 'yum clean all',
+ /(?i:(centos|fedora|redhat|amazon))/ => 'yum clean all',
/(?i:(SLES|opensuse))/ => 'zypper clean -a',
/Ubuntu|Debian/ => 'apt-get clean',
}
diff --git a/bigtop_toolchain/manifests/deployment-tools.pp
b/bigtop_toolchain/manifests/deployment_tools.pp
similarity index 95%
rename from bigtop_toolchain/manifests/deployment-tools.pp
rename to bigtop_toolchain/manifests/deployment_tools.pp
index cd0104c..41a7522 100644
--- a/bigtop_toolchain/manifests/deployment-tools.pp
+++ b/bigtop_toolchain/manifests/deployment_tools.pp
@@ -13,6 +13,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-class bigtop_toolchain::deployment-tools {
+class bigtop_toolchain::deployment_tools {
include bigtop_toolchain::vagrant
}
diff --git a/bigtop_toolchain/manifests/development-tools.pp
b/bigtop_toolchain/manifests/development_tools.pp
similarity index 94%
rename from bigtop_toolchain/manifests/development-tools.pp
rename to bigtop_toolchain/manifests/development_tools.pp
index 0e26394..a627d80 100644
--- a/bigtop_toolchain/manifests/development-tools.pp
+++ b/bigtop_toolchain/manifests/development_tools.pp
@@ -13,6 +13,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-class bigtop_toolchain::development-tools {
+class bigtop_toolchain::development_tools {
include bigtop_toolchain::groovy
}
diff --git a/bigtop_toolchain/manifests/env.pp
b/bigtop_toolchain/manifests/env.pp
index 2c336df..d8ac051 100644
--- a/bigtop_toolchain/manifests/env.pp
+++ b/bigtop_toolchain/manifests/env.pp
@@ -25,10 +25,17 @@ class bigtop_toolchain::env {
'x86_64' : { $arch = "x86_64"}
}
case $operatingsystem {
- 'Ubuntu','Debian': {
+ 'Debian': {
+ if $::operatingsystemmajrelease =~ /^\d$/ {
+ $javahome = "/usr/lib/jvm/${java}-openjdk-$arch"
+ } else {
+ $javahome = "/usr/lib/jvm/adoptopenjdk-8-hotspot-$arch"
+ }
+ }
+ 'Ubuntu': {
$javahome = "/usr/lib/jvm/${java}-openjdk-$arch"
}
- 'Fedora','Centos', 'Amazon': {
+ 'Fedora','Centos','RedHat','Amazon': {
$javahome = "/usr/lib/jvm/${java}"
}
'OpenSuSE' : {
diff --git a/bigtop_toolchain/manifests/gnupg.pp
b/bigtop_toolchain/manifests/gnupg.pp
index be76f05..8857370 100644
--- a/bigtop_toolchain/manifests/gnupg.pp
+++ b/bigtop_toolchain/manifests/gnupg.pp
@@ -16,7 +16,7 @@
class bigtop_toolchain::gnupg {
case $operatingsystem {
- /(?i:(centos|fedora))/: {
+ /(?i:(centos|fedora|redhat))/: {
$pkg = "gnupg2"
$cmd = "gpg2"
}
diff --git a/bigtop_toolchain/manifests/jdk.pp
b/bigtop_toolchain/manifests/jdk.pp
index c17588c..d369ee6 100644
--- a/bigtop_toolchain/manifests/jdk.pp
+++ b/bigtop_toolchain/manifests/jdk.pp
@@ -16,8 +16,27 @@
class bigtop_toolchain::jdk {
case $::operatingsystem {
/Debian/: {
- package { 'openjdk-8-jdk' :
- ensure => present,
+ if $::operatingsystemmajrelease =~ /^\d$/ {
+ # Up to Debian 9
+ package { 'openjdk-8-jdk' :
+ ensure => present,
+ }
+ } else {
+ # We need JDK 8, but Debian 10 only provides the openjdk-11-jdk
package in the official repo.
+ # So we use AdoptOpenJDK instead, following the steps described on:
+ # https://adoptopenjdk.net/installation.html#linux-pkg
+ include apt
+
+ apt::source { 'adoptopenjdk':
+ location => 'https://adoptopenjdk.jfrog.io/adoptopenjdk/deb/',
+ key => {
+ id => '8ED17AF5D7E675EB3EE3BCE98AC3B29174885C03',
+ source =>
'https://adoptopenjdk.jfrog.io/adoptopenjdk/api/gpg/key/public',
+ },
+ } ->
+ package { 'adoptopenjdk-8-hotspot' :
+ ensure => present,
+ }
}
}
/Ubuntu/: {
@@ -27,7 +46,7 @@ class bigtop_toolchain::jdk {
ensure => present,
}
}
- /(CentOS|Amazon|Fedora)/: {
+ /(CentOS|Amazon|Fedora|RedHat)/: {
package { 'java-1.8.0-openjdk-devel' :
ensure => present
}
diff --git a/bigtop_toolchain/manifests/packages.pp
b/bigtop_toolchain/manifests/packages.pp
index 7882b50..76440a9 100644
--- a/bigtop_toolchain/manifests/packages.pp
+++ b/bigtop_toolchain/manifests/packages.pp
@@ -15,8 +15,8 @@
class bigtop_toolchain::packages {
case $operatingsystem{
- /(?i:(centos|fedora))/: {
- $pkgs = [
+ /(?i:(centos|fedora|redhat))/: {
+ $_pkgs = [
"unzip",
"rsync",
"curl",
@@ -35,7 +35,6 @@ class bigtop_toolchain::packages {
"fuse-devel",
"cppunit-devel",
"openssl-devel",
- "python-devel",
"python2-pip",
"libxml2-devel",
"libxslt-devel",
@@ -64,6 +63,12 @@ class bigtop_toolchain::packages {
"bison",
"libffi-devel"
]
+ if ($operatingsystem != 'Fedora' and $operatingsystemmajrelease !~
/^[0-7]$/) {
+ # The python-devel package does not exist any longer from CentOS (and
RHEL) 8
+ $pkgs = concat($_pkgs, "python2-devel")
+ } else {
+ $pkgs = concat($_pkgs, "python-devel")
+ }
}
/(?i:(SLES|opensuse))/: { $pkgs = [
"unzip",
@@ -220,11 +225,20 @@ class bigtop_toolchain::packages {
package { 'epel-release':
ensure => installed
}
+ # On CentOS 8, EPEL requires that the PowerTools repository is enabled.
+ # See
https://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F
+ if $operatingsystemmajrelease !~ /^[0-7]$/ {
+ yumrepo { 'PowerTools':
+ ensure => 'present',
+ enabled => '1'
+ }
+ Yumrepo<||> -> Package<||>
+ }
}
# Install Python packages using pip
case $operatingsystem{
- /(?i:(centos|fedora))/: {
+ /(?i:(centos|fedora|redhat))/: {
$pip = 'python2-pip'
} /(?i:(SLES|opensuse))/: {
$pip = 'python-pip'
@@ -236,7 +250,7 @@ class bigtop_toolchain::packages {
}
file { '/usr/bin/pip-python':
ensure => 'link',
- target => '/usr/bin/pip',
+ target => '/usr/bin/pip2',
}
package { 'setuptools':
ensure => 'latest',
diff --git a/bigtop_toolchain/manifests/puppet-modules.pp
b/bigtop_toolchain/manifests/puppet_modules.pp
similarity index 93%
rename from bigtop_toolchain/manifests/puppet-modules.pp
rename to bigtop_toolchain/manifests/puppet_modules.pp
index 38ecca2..ef20f27 100644
--- a/bigtop_toolchain/manifests/puppet-modules.pp
+++ b/bigtop_toolchain/manifests/puppet_modules.pp
@@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-class bigtop_toolchain::puppet-modules {
+class bigtop_toolchain::puppet_modules {
exec { 'install-puppet-stdlib':
path => '/usr/bin:/bin',
@@ -39,5 +39,5 @@ class bigtop_toolchain::puppet-modules {
stage { 'first':
before => Stage['main'],
}
- class { 'bigtop_toolchain::puppet-modules-prereq': stage => 'first' }
+ class { 'bigtop_toolchain::puppet_modules_prereq': stage => 'first' }
}
diff --git a/bigtop_toolchain/manifests/puppet-modules-prereq.pp
b/bigtop_toolchain/manifests/puppet_modules_prereq.pp
similarity index 88%
rename from bigtop_toolchain/manifests/puppet-modules-prereq.pp
rename to bigtop_toolchain/manifests/puppet_modules_prereq.pp
index 7b6a8cf..3d18049 100644
--- a/bigtop_toolchain/manifests/puppet-modules-prereq.pp
+++ b/bigtop_toolchain/manifests/puppet_modules_prereq.pp
@@ -13,9 +13,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-class bigtop_toolchain::puppet-modules-prereq {
+class bigtop_toolchain::puppet_modules_prereq {
if versioncmp($::puppetversion,'3.0.0') < 0 {
- fail("bigtop_toolchain::puppet-modules requires Puppet 3.0.0+, but found:
$::puppetversion")
+ fail("bigtop_toolchain::puppet_modules requires Puppet 3.0.0+, but found:
$::puppetversion")
}
}
diff --git a/bigtop_toolchain/manifests/renv.pp
b/bigtop_toolchain/manifests/renv.pp
index 4d54434..9ab1441 100644
--- a/bigtop_toolchain/manifests/renv.pp
+++ b/bigtop_toolchain/manifests/renv.pp
@@ -18,7 +18,7 @@ class bigtop_toolchain::renv {
require bigtop_toolchain::packages
case $operatingsystem{
- /(?i:(centos|fedora|Amazon))/: {
+ /(?i:(centos|fedora|redhat|Amazon))/: {
$pkgs = [
"R",
"R-devel",
diff --git a/build.gradle b/build.gradle
index b1c07b7..05ab843 100644
--- a/build.gradle
+++ b/build.gradle
@@ -231,7 +231,7 @@ task toolchain(type:Exec,
if ('3.7' <= version && version < '4') {
command.addAll(['--parser', 'future'])
}
-
command.addAll(["--modulepath=${projectDir.absolutePath}:/etc/puppet/modules:/usr/share/puppet/modules",
+
command.addAll(["--modulepath=${projectDir.absolutePath}:/etc/puppet/modules:/usr/share/puppet/modules:/etc/puppetlabs/code/environments/production/modules",
'-e', 'include bigtop_toolchain::installer'])
workingDir '.'
commandLine command
@@ -242,7 +242,7 @@ task "toolchain-puppetmodules"(type:Exec,
def command = [
'sudo', 'puppet', 'apply', '-d',
"--modulepath=${projectDir.absolutePath}", '-e',
- 'include bigtop_toolchain::puppet-modules'
+ 'include bigtop_toolchain::puppet_modules'
]
workingDir '.'
commandLine command
@@ -253,7 +253,7 @@ task "toolchain-devtools"(type:Exec,
def command = [
'sudo', 'puppet', 'apply', '-d',
"--modulepath=${projectDir.absolutePath}", '-e',
- 'include bigtop_toolchain::development-tools'
+ 'include bigtop_toolchain::development_tools'
]
workingDir '.'
commandLine command
