mneethiraj commented on code in PR #377:
URL: https://github.com/apache/ranger/pull/377#discussion_r1747539323
##########
dev-support/ranger-docker/README.md:
##########
@@ -30,9 +30,13 @@ Docker files in this folder create docker images and run
them to build Apache Ra
3. Set ```dev-support/ranger-docker``` as your working directory.
-4. Execute following command to download necessary archives to setup
Ranger/HDFS/Hive/HBase/Kafka/Knox services:
+4. Execute following command to download necessary archives to setup
Ranger/HDFS/Hive/HBase/Kafka/Knox/Ozone services:
~~~
- chmod +x download-archives.sh && ./download-archives.sh
+ chmod +x download-archives.sh
+ source download-archives.sh
+
+ # use a subset of the below to download specific services
+ download_hadoop; download_hbase; download_hive; download_kafka;
download_knox; download_ozone
Review Comment:
above `source` command exports methods defined in download-archives.sh -
`download_hadoop()`, `download_hbase()`, `download_..()`. These methods don't
need to be available in the shell. Instead of this approach, consider updating
`download-archives.sh` to take the list of components to download as arguments,
as shown below:
`./download-archives.sh hadoop hbase kafka`
##########
dev-support/ranger-docker/Dockerfile.ranger-ozone:
##########
@@ -0,0 +1,31 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+ARG OZONE_RUNNER_IMAGE
+ARG OZONE_RUNNER_VERSION
+FROM ${OZONE_RUNNER_IMAGE}:${OZONE_RUNNER_VERSION}
+
+ARG OZONE_HOME
+ARG OZONE_PLUGIN_VERSION
+
+USER root
+
+RUN useradd -g hadoop -ms /bin/bash ozone
+RUN mkdir -p -m 755 /var/log/ozone
+RUN chown om:om -R /var/log/ozone
Review Comment:
Shouldn't the username be `ozone`, instead of `om` - refer to #26 above?
##########
dev-support/ranger-docker/Dockerfile.ranger-ozone:
##########
@@ -0,0 +1,31 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+ARG OZONE_RUNNER_IMAGE
+ARG OZONE_RUNNER_VERSION
+FROM ${OZONE_RUNNER_IMAGE}:${OZONE_RUNNER_VERSION}
+
+ARG OZONE_HOME
+ARG OZONE_PLUGIN_VERSION
+
+USER root
+
+RUN useradd -g hadoop -ms /bin/bash ozone
Review Comment:
User `ozone` is already created in Dockerfile.ranger-base. Is it necessary
to create here as well? Does this fail if the user already exists?
##########
dev-support/ranger-docker/README.md:
##########
@@ -42,44 +46,66 @@ Docker files in this folder create docker images and run
them to build Apache Ra
export RANGER_DB_TYPE=postgres
~~~
-6. Build Apache Ranger in containers using docker-compose
-
- 1. Execute following command to build Apache Ranger:
- ~~~
- docker-compose -f docker-compose.ranger-base.yml -f
docker-compose.ranger-build.yml up
- ~~~
-
- Time taken to complete the build might vary (upto an hour), depending on
status of ```${HOME}/.m2``` directory cache.
-
- 2. Alternatively, the following commands can be executed from the parent
directory
- 1. To generate tarballs:```mvn clean package -DskipTests```
-
- 2. Copy the tarballs and version file to
```dev-support/ranger-docker/dist```
- ~~~
- cp target/ranger-* dev-support/ranger-docker/dist/
- cp target/version dev-support/ranger-docker/dist/
- ~~~
-
- 3. Build the ranger-base image:
- ~~~
- docker-compose -f docker-compose.ranger-base.yml build --no-cache
- ~~~
-7. To enable file based sync source for usersync execute: ```export
ENABLE_FILE_SYNC_SOURCE=true```
-
-8. Execute following command to start Ranger, Ranger Usersync, Ranger Tagsync,
Ranger enabled HDFS/YARN/HBase/Hive/Kafka/Knox and dependent services (Solr,
DB) in containers:
- ~~~
- docker-compose -f docker-compose.ranger-base.yml -f
docker-compose.ranger.yml -f docker-compose.ranger-${RANGER_DB_TYPE}.yml -f
docker-compose.ranger-usersync.yml -f docker-compose.ranger-tagsync.yml -f
docker-compose.ranger-kms.yml -f docker-compose.ranger-hadoop.yml -f
docker-compose.ranger-hbase.yml -f docker-compose.ranger-kafka.yml -f
docker-compose.ranger-hive.yml -f docker-compose.ranger-knox.yml up -d
- ~~~
-
- - valid values for RANGER_DB_TYPE: mysql or postgres
-9. To run ranger enabled Trino in containers (Requires docker build with JDK
11):
- ~~~
- docker-compose -f docker-compose.ranger-base.yml -f
docker-compose.ranger.yml -f docker-compose.ranger-${RANGER_DB_TYPE}.yml -f
docker-compose.ranger-trino.yml up -d
- ~~~
-
-10. To rebuild specific images and start containers with the new image, use
following command:
- ~~~
- docker-compose -f docker-compose.ranger-base.yml -f
docker-compose.ranger.yml -f docker-compose.ranger-usersync.yml -f
docker-compose.ranger-tagsync.yml -f docker-compose.ranger-kms.yml -f
docker-compose.ranger-hadoop.yml -f docker-compose.ranger-hbase.yml -f
docker-compose.ranger-kafka.yml -f docker-compose.ranger-hive.yml -f
docker-compose.ranger-trino.yml -f docker-compose.ranger-knox.yml up -d
--no-deps --force-recreate --build <service-1> <service-2>
- ~~~
-
-9. Ranger Admin can be accessed at http://localhost:6080 (admin/rangerR0cks!)
+### Apache Ranger Build
Review Comment:
I suggest to apply the reformatting for earlier steps as well (1 to 5).
##########
dev-support/ranger-docker/download-archives.sh:
##########
@@ -43,14 +41,49 @@ downloadIfNotPresent() {
fi
}
-downloadIfNotPresent hadoop-${HADOOP_VERSION}.tar.gz
https://archive.apache.org/dist/hadoop/common/hadoop-${HADOOP_VERSION}
-downloadIfNotPresent hbase-${HBASE_VERSION}-bin.tar.gz
https://archive.apache.org/dist/hbase/${HBASE_VERSION}
-downloadIfNotPresent kafka_2.12-${KAFKA_VERSION}.tgz
https://archive.apache.org/dist/kafka/${KAFKA_VERSION}
-downloadIfNotPresent apache-hive-${HIVE_VERSION}-bin.tar.gz
https://archive.apache.org/dist/hive/hive-${HIVE_VERSION}
-downloadIfNotPresent hadoop-${HIVE_HADOOP_VERSION}.tar.gz
https://archive.apache.org/dist/hadoop/common/hadoop-${HIVE_HADOOP_VERSION}
-downloadIfNotPresent postgresql-42.2.16.jre7.jar
https://search.maven.org/remotecontent?filepath=org/postgresql/postgresql/42.2.16.jre7
-downloadIfNotPresent mysql-connector-java-8.0.28.jar
https://search.maven.org/remotecontent?filepath=mysql/mysql-connector-java/8.0.28
+downloadIfNotPresent postgresql-42.2.16.jre7.jar
"https://search.maven.org/remotecontent?filepath=org/postgresql/postgresql/42.2.16.jre7";
+downloadIfNotPresent mysql-connector-java-8.0.28.jar
"https://search.maven.org/remotecontent?filepath=mysql/mysql-connector-java/8.0.28";
downloadIfNotPresent log4jdbc-1.2.jar
https://repo1.maven.org/maven2/com/googlecode/log4jdbc/log4jdbc/1.2
-downloadIfNotPresent knox-${KNOX_VERSION}.tar.gz
https://archive.apache.org/dist/knox/${KNOX_VERSION}
+download_hadoop() {
+ downloadIfNotPresent hadoop-${HADOOP_VERSION}.tar.gz
https://archive.apache.org/dist/hadoop/common/hadoop-${HADOOP_VERSION}
+}
+
+download_hbase() {
+ downloadIfNotPresent hbase-${HBASE_VERSION}-bin.tar.gz
https://archive.apache.org/dist/hbase/${HBASE_VERSION}
+}
+
+download_hive() {
+ downloadIfNotPresent apache-hive-${HIVE_VERSION}-bin.tar.gz
https://archive.apache.org/dist/hive/hive-${HIVE_VERSION}
+ downloadIfNotPresent hadoop-${HIVE_HADOOP_VERSION}.tar.gz
https://archive.apache.org/dist/hadoop/common/hadoop-${HIVE_HADOOP_VERSION}
+}
+
+download_kafka() {
+ downloadIfNotPresent kafka_2.12-${KAFKA_VERSION}.tgz
https://archive.apache.org/dist/kafka/${KAFKA_VERSION}
+}
+
+download_knox() {
+ downloadIfNotPresent knox-${KNOX_VERSION}.tar.gz
https://archive.apache.org/dist/knox/${KNOX_VERSION}
+}
+
+download_ozone() {
+ downloadIfNotPresent ozone-${OZONE_VERSION}.tar.gz
https://archive.apache.org/dist/ozone/${OZONE_VERSION}
+
+ if [ ! -d downloads/ozone-${OZONE_VERSION} ]
+ then
+ tar xvfz downloads/ozone-${OZONE_VERSION}.tar.gz --directory=downloads/
+ fi
+}
+
+setup_ozone() {
Review Comment:
why is `setup_ozone()` in `download-archives.sh`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
