ignite-7107 Apache Ignite RPM packages Signed-off-by: Andrey Gura <[email protected]>
Project: http://git-wip-us.apache.org/repos/asf/ignite/repo Commit: http://git-wip-us.apache.org/repos/asf/ignite/commit/33ad0ad8 Tree: http://git-wip-us.apache.org/repos/asf/ignite/tree/33ad0ad8 Diff: http://git-wip-us.apache.org/repos/asf/ignite/diff/33ad0ad8 Branch: refs/heads/ignite-zk Commit: 33ad0ad84a25a0d65ba408accfb1429cb96ff840 Parents: 3c5d376 Author: Ivanov Petr <[email protected]> Authored: Thu Dec 28 16:08:21 2017 +0300 Committer: Andrey Gura <[email protected]> Committed: Thu Dec 28 16:08:21 2017 +0300 ---------------------------------------------------------------------- DEVNOTES.txt | 198 +++++++++++++++++--------- packaging/rpm/SOURCES/name.service | 17 +++ packaging/rpm/SOURCES/service.sh | 28 ++++ packaging/rpm/SPECS/apache-ignite.spec | 206 ++++++++++++++++++++++++++++ 4 files changed, 381 insertions(+), 68 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ignite/blob/33ad0ad8/DEVNOTES.txt ---------------------------------------------------------------------- diff --git a/DEVNOTES.txt b/DEVNOTES.txt index 340153e..6ec0f39 100644 --- a/DEVNOTES.txt +++ b/DEVNOTES.txt @@ -4,81 +4,132 @@ Ignite Fabric Maven Build Instructions 2) Compile and install: - mvn clean install -Pall-java,all-scala,licenses -DskipTests + mvn clean install -Pall-java,all-scala,licenses -DskipTests - or if you have built Apache Ignite.NET on the first step use following command: - (Note that 'doxygen' should be installed before running this command.) + or if you have built Apache Ignite.NET on the first step use following command: + (Note that 'doxygen' should be installed before running this command.) - mvn clean install -Pall-java,all-scala,licenses -DskipTests -DclientDocs + mvn clean install -Pall-java,all-scala,licenses -DskipTests -DclientDocs 3) Javadoc generation (optional): - mvn initialize -Pjavadoc + mvn initialize -Pjavadoc 4) Assembly Apache Ignite fabric: - mvn initialize -Prelease + mvn initialize -Prelease Look for apache-ignite-fabric-<version>-bin.zip in ./target/bin directory. + Ignite Fabric with LGPL Maven Build Instructions -====================================== +================================================ 1) Optional: build Apache Ignite.NET as described at modules/platforms/dotnet/DEVNOTES.txt. 2) Compile and install: - mvn clean install -Pall-java,all-scala,licenses -DskipTests + mvn clean install -Pall-java,all-scala,licenses -DskipTests - or if you have built Apache Ignite.NET on the first step use following command: - (Note that 'doxygen' should be installed before running this command.) + or if you have built Apache Ignite.NET on the first step use following command: + (Note that 'doxygen' should be installed before running this command.) - mvn clean install -Pall-java,all-scala,licenses -DskipTests -DclientDocs + mvn clean install -Pall-java,all-scala,licenses -DskipTests -DclientDocs 3) Javadoc generation with LGPL (optional): - mvn initialize -Pjavadoc,lgpl + mvn initialize -Pjavadoc,lgpl 4) Assembly Apache Ignite fabric with LGPL dependencies: - mvn initialize -Prelease,lgpl -Dignite.edition=fabric-lgpl + mvn initialize -Prelease,lgpl -Dignite.edition=fabric-lgpl + + Look for apache-ignite-fabric-lgpl-<version>-bin.zip in ./target/bin directory. -Look for apache-ignite-fabric-lgpl-<version>-bin.zip in ./target/bin directory. Ignite Hadoop Accelerator Maven Build Instructions -============================================ +================================================== 1) Compile and install: - mvn clean install -Pall-java,all-scala,licenses -DskipTests + mvn clean install -Pall-java,all-scala,licenses -DskipTests -Use 'hadoop.version' parameter to build Ignite against a specific Hadoop version. -Use 'spark.version' parameter to build ignite-spark module for a specific Spark version. Version should be >= 2.0.0. -For example: + Use 'hadoop.version' parameter to build Ignite against a specific Hadoop version. + Use 'spark.version' parameter to build ignite-spark module for a specific Spark version. Version should be >= 2.0.0. + For example: - mvn clean install -Pall-java,all-scala,licenses -DskipTests -Dhadoop.version=2.4.2 -Dspark.version=2.1.1 + mvn clean install -Pall-java,all-scala,licenses -DskipTests -Dhadoop.version=2.4.2 -Dspark.version=2.1.1 2) Assembly Hadoop Accelerator: - mvn initialize -Prelease -Dignite.edition=hadoop -Look for apache-ignite-hadoop-<version>-bin.zip in ./target/bin directory. Resulting binary -assembly will also include integration module for Apache Spark. + mvn initialize -Prelease -Dignite.edition=hadoop + + Look for apache-ignite-hadoop-<version>-bin.zip in ./target/bin directory. Resulting binary + assembly will also include integration module for Apache Spark. + + NOTE: JDK version should be 1.7.0-* or >= 1.8.0-u40. -NOTE: JDK version should be 1.7.0-* or >= 1.8.0-u40. Ignite C++ Build Instructions +============================= +Instructions can be found at modules/platforms/cpp/DEVNOTES.txt. + + +Apache Ignite RPM Package Build Instructions +============================================ +1) Install dependencies + + yum install -y rpmdevtools rpm-build + +2) Create directory layout + + rpmdev-setuptree + +3) Copy build specification and additional sources required for build + + cp -rfv packaging/rpm/* ~/rpmbuild + +4) Copy built binary from target/bin to sources directory + + cp -rfv target/bin/apache-ignite-<version>-bin.zip ~/rpmbuild/SOURCES/apache-ignite.zip + +5) Run build process + + rpmbuild -bb ~/rpmbuild/SPECS/apache-ignite.spec + + NOTE: to build also SRPM package, run with -ba flag instead + + Package will be available in ~/rpmbuild/RPMS/noarch + + +Install and Run Apache Ignite from RPM ====================================== - Instructions can be found at modules/platforms/cpp/DEVNOTES.txt. +1) Install package + + yum localinstall -y apache-ignite-<version>.rpm + +2) Start Apache Ignite service + + systemctl start apache-ignite@<config name> + + NOTE: <config name> - relative to /etc/apache-ignite configuration file name + + Example: + + systemctl start [email protected] + +3) See logs in /var/log/apache-ignite or journalctl -xe + Ignite Release Instructions =========================== -Use your people.apache.org/keys/committer/<username>.asc key to generate KEYS file. -Download https://dist.apache.org/repos/dist/release/ignite/KEYS and append you key using commands: +1) Use your people.apache.org/keys/committer/<username>.asc key to generate KEYS file. + Download https://dist.apache.org/repos/dist/release/ignite/KEYS and append you key using commands: -gpg --list-sigs <keyname> >> KEYS -gpg --armor --export <keyname> >> KEYS + gpg --list-sigs <keyname> >> KEYS + gpg --armor --export <keyname> >> KEYS -Upload modified KEYS file. + Upload modified KEYS file. -Specify gpg profile at settings.xml. It will be used to sign sources and artifacts. +2) Specify gpg profile at settings.xml. It will be used to sign sources and artifacts. <profile> <id>gpg</id> @@ -88,88 +139,99 @@ Specify gpg profile at settings.xml. It will be used to sign sources and artifac </properties> </profile> -Ensure you have RELEASE (not SNAPSHOT) version at Ignite poms. -Maven release plugin release:prepare goal can be used to make release tag. + Ensure you have RELEASE (not SNAPSHOT) version at Ignite poms. + Maven release plugin release:prepare goal can be used to make release tag. + +3) Deploy Ignite release candidate to maven repository and dev-svn, make tag: + + 3.1) Deploy Ignite to maven repository, prepares sources and fabric edition binaries. -Deploy Ignite release candidate to maven repository and dev-svn, make tag: + mvn deploy -Papache-release,gpg,all-java,all-scala,licenses,deploy-ignite-site -Dignite.edition=fabric -DskipTests - 1) Deploy Ignite to maven repository, prepares sources and fabric edition binaries. - mvn deploy -Papache-release,gpg,all-java,all-scala,licenses,deploy-ignite-site -Dignite.edition=fabric -DskipTests + 3.2) Javadoc generation: - 2) Javadoc generation: - mvn initialize -Pjavadoc + mvn initialize -Pjavadoc - 3) Assembly Apache Ignite Fabric: - mvn initialize -Prelease + 3.3) Assembly Apache Ignite Fabric: - 4) Assembly Hadoop Accelerator: - mvn initialize -Prelease -Dignite.edition=hadoop + mvn initialize -Prelease - Binary artifact name can be changed by setting additional property -Dignite.zip.pattern. Binary artifact will be + 3.4) Assembly Hadoop Accelerator: + + mvn initialize -Prelease -Dignite.edition=hadoop + + NOTE: Binary artifact name can be changed by setting additional property -Dignite.zip.pattern. Binary artifact will be created inside /target/bin folder when release profile is used. - Sources artifact name is fixed. Sources artifact will be created inside /target dir when apache-release profile is used. + NOTE: Sources artifact name is fixed. Sources artifact will be created inside /target dir when apache-release profile is used. - Nexus staging (repository.apache.org) should be closed with appropriate comment contains release version and + NOTE: Nexus staging (repository.apache.org) should be closed with appropriate comment contains release version and release candidate number, for example "Apache Ignite 1.0.0-rc7", when mvn deploy finished. - Checkout https://dist.apache.org/repos/dist/dev/ignite svn. Create release candidate folder with name - equals to release version with "-rc*" ending, for example "1.0.0-rc7", at svn root. - Copy /target/site folder content to svn/ignite/<rc-version> folder and commit with appropriate comment. + 3.5) Checkout https://dist.apache.org/repos/dist/dev/ignite svn. Create release candidate folder with name + equals to release version with "-rc*" ending, for example "1.0.0-rc7", at svn root. + Copy /target/site folder content to svn/ignite/<rc-version> folder and commit with appropriate comment. - Make appropriate git tag for release candidate, for example "ignite-X.Y.Z-rc1". + 3.6) Make appropriate git tag for release candidate, for example "ignite-X.Y.Z-rc1". -Start vote based on https://dist.apache.org/repos/dist/dev/ignite/<rc-version>. +4) Start vote based on https://dist.apache.org/repos/dist/dev/ignite/<rc-version>. -Release nexus staging, move binaries and sources from https://dist.apache.org/repos/dist/dev/ignite/<rc-version> -to https://dist.apache.org/repos/dist/release/ignite/<version> when version accepted. -Use svn mv ^/dev/ignite/<rc-version> ^/release/ignite/<version> command for proper moving. +5) Release nexus staging, move binaries and sources from https://dist.apache.org/repos/dist/dev/ignite/<rc-version> + to https://dist.apache.org/repos/dist/release/ignite/<version> when version accepted. + Use svn mv ^/dev/ignite/<rc-version> ^/release/ignite/<version> command for proper moving. -Make appropriate git tag for released version, for example "ignite-X.Y.Z". +6) Make appropriate git tag for released version, for example "ignite-X.Y.Z". -Send an email to [email protected] contains release svn url. +7) Send an email to [email protected] contains release svn url. JCache TCK compliance ====================== To test compliance with JCache TCK use: -mvn test -P-release,jcache-tck -pl :ignite-core -am + mvn test -P-release,jcache-tck -pl :ignite-core -am + Ignite Mesos Maven Build Instructions -============================================ -cd to ./modules/mesos +===================================== -mvn clean package + cd ./modules/mesos + mvn clean package Look for ignite-mesos-<version>.jar in ./target directory. + Ignite Yarn Maven Build Instructions -============================================ -cd to ./modules/yarn +==================================== -mvn clean package + cd ./modules/yarn + mvn clean package Look for ignite-yarn-<version>.jar in ./target directory. + Run tests -========== +========= To run tests locally use: -mvn clean test -U -Plgpl,examples,-clean-libs,-release -Dmaven.test.failure.ignore=true -DfailIfNoTests=false -Dtest=%TEST_PATTERN% + mvn clean test -U -Plgpl,examples,-clean-libs,-release -Dmaven.test.failure.ignore=true -DfailIfNoTests=false -Dtest=%TEST_PATTERN% For example, %TEST_PATTERN% can be 'org.apache.ignite.testsuites.IgniteBasicTestSuite' or 'GridCacheLocalAtomicFullApiSelfTest#testGet' + Apache RAT Instructions ======================= To check license headers use: -mvn clean validate -Pcheck-licenses + mvn clean validate -Pcheck-licenses For more information (e.g. exclude list) see "check-licenses" profile in "parent/pom.xml". Useful Release Commands -======================== -ZIP: zip -r9 apache-ignite-X.X.X-src.zip apache-ignite-X.X.X-src -GPG: gpg --armor --output apache-ignite-X.X.X-src.zip.asc --detach-sig apache-ignite-X.X.X-src.zip +======================= +ZIP: + zip -r9 apache-ignite-X.X.X-src.zip apache-ignite-X.X.X-src +GPG: + gpg --armor --output apache-ignite-X.X.X-src.zip.asc --detach-sig apache-ignite-X.X.X-src.zip + http://git-wip-us.apache.org/repos/asf/ignite/blob/33ad0ad8/packaging/rpm/SOURCES/name.service ---------------------------------------------------------------------- diff --git a/packaging/rpm/SOURCES/name.service b/packaging/rpm/SOURCES/name.service new file mode 100644 index 0000000..d03a7fc --- /dev/null +++ b/packaging/rpm/SOURCES/name.service @@ -0,0 +1,17 @@ +[Unit] +Description=Apache Ignite In-Memory Computing Platform Service +After=syslog.target network.target + +[Service] +Type=forking +User=#user# +WorkingDirectory=/usr/share/#name#/work +PermissionsStartOnly=true +ExecStartPre=-/usr/bin/mkdir /var/run/#name# +ExecStartPre=-/usr/bin/chown #user#:#user# /var/run/#name# +ExecStartPre=-/usr/bin/env bash /usr/share/#name#/bin/service.sh set-firewall +ExecStart=/usr/share/#name#/bin/service.sh start %i +PIDFile=/var/run/#name#/#name#.%i.pid + +[Install] +WantedBy=multi-user.target http://git-wip-us.apache.org/repos/asf/ignite/blob/33ad0ad8/packaging/rpm/SOURCES/service.sh ---------------------------------------------------------------------- diff --git a/packaging/rpm/SOURCES/service.sh b/packaging/rpm/SOURCES/service.sh new file mode 100644 index 0000000..4163676 --- /dev/null +++ b/packaging/rpm/SOURCES/service.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash + +firewallCmd="firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT 0" + +# Define function to check whether firewalld is present and started and apply firewall rules for grid nodes +setFirewall () +{ + if [[ "$(type firewall-cmd &>/dev/null; echo $?)" -eq 0 && "$(systemctl is-active firewalld)" == "active" ]] + then + for port in s d + do + ${firewallCmd} -p tcp -m multiport --${port}ports 11211:11220,47500:47509,47100:47109 -j ACCEPT &>/dev/null + ${firewallCmd} -p udp -m multiport --${port}ports 47400:47409 -j ACCEPT &>/dev/null + done + ${firewallCmd} -m pkttype --pkt-type multicast -j ACCEPT &>/dev/null + + systemctl restart firewalld + fi +} + +case $1 in + start) + /usr/share/#name#/bin/ignite.sh /etc/#name#/$2 & echo $! >> /var/run/#name#/#name#.$2.pid + ;; + set-firewall) + setFirewall + ;; +esac http://git-wip-us.apache.org/repos/asf/ignite/blob/33ad0ad8/packaging/rpm/SPECS/apache-ignite.spec ---------------------------------------------------------------------- diff --git a/packaging/rpm/SPECS/apache-ignite.spec b/packaging/rpm/SPECS/apache-ignite.spec new file mode 100644 index 0000000..45d72b7 --- /dev/null +++ b/packaging/rpm/SPECS/apache-ignite.spec @@ -0,0 +1,206 @@ +%define __jar_repack %{nil} +%define user ignite + + +#------------------------------------------------------------------------------- +# +# Packages' descriptions +# + +Name: apache-ignite +Version: 2.4.0 +Release: 1%{?dist} +Summary: Apache Ignite In-Memory Computing Platform +Group: Development/System +License: ASL 2.0 +URL: https://ignite.apache.org/ +Source: %{name}.zip +Requires: java-1.8.0, chkconfig +Requires(pre): shadow-utils +Provides: %{name} +AutoReq: no +AutoProv: no +BuildArch: noarch +%description +Apache Ignite⢠is the in-memory computing platform composed of a strongly +consistent distributed database with powerful SQL, key-value and processing APIs + + +#------------------------------------------------------------------------------- +# +# Prepare step: unpack sources +# + +%prep +%setup -q -c -n %{name} + + +#------------------------------------------------------------------------------- +# +# Preinstall scripts +# $1 can be: +# 1 - Initial install +# 2 - Upgrade +# + + +#------------------------------------------------------------------------------- +# +# Postinstall scripts +# $1 can be: +# 1 - Initial installation +# 2 - Upgrade +# + +%post +case $1 in + 1) + # Add user for service operation + useradd -r -d %{_datadir}/%{name} -s /usr/sbin/nologin %{user} + # Change ownership for work and log directories + chown -vR %{user}:%{user} %{_sharedstatedir}/%{name} %{_var}/log/%{name} + # Install alternatives + # Commented out until ignitevisorcmd / ignitesqlline is ready to work from any user + #update-alternatives --install %{_bindir}/ignitevisorcmd ignitevisorcmd %{_datadir}/%{name}/bin/ignitevisorcmd.sh 0 + #update-alternatives --auto ignitevisorcmd + #update-alternatives --display ignitevisorcmd + #update-alternatives --install %{_bindir}/ignitesqlline ignitesqlline %{_datadir}/%{name}/bin/sqlline.sh 0 + #update-alternatives --auto ignitesqlline + #update-alternatives --display ignitesqlline + ;; + 2) + : + ;; +esac + + +#------------------------------------------------------------------------------- +# +# Pre-uninstall scripts +# $1 can be: +# 0 - Uninstallation +# 1 - Upgrade +# + +%preun +case $1 in + 0) + # Remove alternatives + # Commented out until ignitevisorcmd / ignitesqlline is ready to work from any user + #update-alternatives --remove ignitevisorcmd /usr/share/%{name}/bin/ignitevisorcmd.sh + #update-alternatives --display ignitevisorcmd || true + #update-alternatives --remove ignitesqlline /usr/share/%{name}/bin/sqlline.sh + #update-alternatives --display ignitesqlline || true + ;; + 1) + : + ;; +esac + + +#------------------------------------------------------------------------------- +# +# Post-uninstall scripts +# $1 can be: +# 0 - Uninstallation +# 1 - Upgrade +# + +%postun +case $1 in + 0) + # Remove user + userdel %{user} + # Remove service PID directory + rm -rfv /var/run/%{name} + # Remove firewalld rules if firewalld is installed and running + if [[ "$(type firewall-cmd &>/dev/null; echo $?)" -eq 0 && "$(systemctl is-active firewalld)" == "active" ]] + then + for port in s d + do + firewall-cmd --permanent --direct --remove-rule ipv4 filter INPUT 0 -p tcp -m multiport --${port}ports 11211:11220,47500:47509,47100:47109 -j ACCEPT &>/dev/null + firewall-cmd --permanent --direct --remove-rule ipv4 filter INPUT 0 -p udp -m multiport --${port}ports 47400:47409 -j ACCEPT &>/dev/null + done + firewall-cmd --permanent --direct --remove-rule ipv4 filter INPUT 0 -m pkttype --pkt-type multicast -j ACCEPT &>/dev/null + systemctl restart firewalld + fi + ;; + 1) + : + ;; +esac + + +#------------------------------------------------------------------------------- +# +# Prepare packages' layout +# + +%install +cd $(ls) + +# Create base directory structure +mkdir -p %{buildroot}%{_datadir}/%{name} +mkdir -p %{buildroot}%{_libdir}/%{name} +mkdir -p %{buildroot}%{_datadir}/doc/%{name}-%{version}/bin +mkdir -p %{buildroot}%{_var}/log/%{name} +mkdir -p %{buildroot}%{_sharedstatedir}/%{name} +mkdir -p %{buildroot}%{_sysconfdir}/systemd/system +mkdir -p %{buildroot}%{_bindir} + +# Copy nessessary files and remove *.bat files +cp -rf benchmarks bin platforms %{buildroot}%{_datadir}/%{name} +cp -rf docs/* examples %{buildroot}%{_datadir}/doc/%{name}-%{version} +mv -f %{buildroot}%{_datadir}/%{name}/bin/ignitevisorcmd.sh %{buildroot}%{_datadir}/doc/%{name}-%{version}/bin/ +find %{buildroot}%{_datadir}/%{name}/ -name *.bat -exec rm -rf {} \; + +# Copy libs to /usr/lib and map them to IGNITE_HOME +cp -rf libs/* %{buildroot}%{_libdir}/%{name} +ln -sf %{_libdir}/%{name} %{buildroot}%{_datadir}/%{name}/libs + +# Setup configuration +cp -rf config %{buildroot}%{_sysconfdir}/%{name} +ln -sf %{_sysconfdir}/%{name} %{buildroot}%{_datadir}/%{name}/config + +# Setup systemctl service +cp -rf %{_sourcedir}/name.service %{buildroot}%{_sysconfdir}/systemd/system/%{name}@.service +cp -rf %{_sourcedir}/service.sh %{buildroot}%{_datadir}/%{name}/bin/ +chmod +x %{buildroot}%{_datadir}/%{name}/bin/service.sh +for file in %{buildroot}%{_sysconfdir}/systemd/system/%{name}@.service %{buildroot}%{_datadir}/%{name}/bin/service.sh +do + sed -i -r -e "s|#name#|%{name}|g" \ + -e "s|#user#|%{user}|g" \ + ${file} +done + +# Map work and log directories +ln -sf %{_sharedstatedir}/%{name} %{buildroot}%{_datadir}/%{name}/work +ln -sf %{_var}/log/%{name} %{buildroot}%{_sharedstatedir}/%{name}/log + + +#------------------------------------------------------------------------------- +# +# Package file list check +# +%files +%dir %{_datadir}/%{name} +%dir %{_sysconfdir}/%{name} +%dir %{_sharedstatedir}/%{name} +%dir %{_var}/log/%{name} + +%{_datadir}/%{name}/benchmarks +%{_datadir}/%{name}/bin +%{_datadir}/%{name}/config +%{_datadir}/%{name}/libs +%{_datadir}/%{name}/platforms +%{_datadir}/%{name}/work +%{_libdir}/%{name} +%{_sysconfdir}/systemd/system/%{name}@.service +%{_sharedstatedir}/%{name}/log + +%config(noreplace) %{_sysconfdir}/%{name}/* + +%doc %{name}-*/README.txt +%doc %{name}-*/NOTICE +%doc %{name}-*/RELEASE_NOTES.txt +%license %{name}-*/LICENSE
