This is an automated email from the ASF dual-hosted git repository.
jihoonson pushed a commit to branch 0.21.1
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/0.21.1 by this push:
new 6e15b53 Fix permission problems in docker (#11299) (#11321)
6e15b53 is described below
commit 6e15b535e5ec3bbee2a02319d433671217d7fba7
Author: Clint Wylie <[email protected]>
AuthorDate: Tue Jun 1 23:02:33 2021 -0700
Fix permission problems in docker (#11299) (#11321)
* Create /opt/data to fix permission problem
* eliminate symlink to avoid compatibility problem on AWS Fargate
* Add a workaround section
* Update instruction for named volume
* Use named volume in docker-compose
* Revert some doc change
* Resolve review comments
Co-authored-by: frank chen <[email protected]>
---
distribution/docker/Dockerfile | 12 ++++++++----
distribution/docker/docker-compose.yml | 7 ++++---
distribution/docker/environment | 6 +++---
docs/tutorials/docker.md | 6 +++++-
4 files changed, 20 insertions(+), 11 deletions(-)
diff --git a/distribution/docker/Dockerfile b/distribution/docker/Dockerfile
index 1d09594..8bdbcc2 100644
--- a/distribution/docker/Dockerfile
+++ b/distribution/docker/Dockerfile
@@ -35,7 +35,7 @@ RUN VERSION=$(mvn -B -q
org.apache.maven.plugins:maven-help-plugin:3.1.1:evaluat
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
- && ln -s /opt/apache-druid-${VERSION} /opt/druid
+ && mv /opt/apache-druid-${VERSION} /opt/druid
FROM amd64/busybox:1.30.0-glibc as busybox
@@ -50,9 +50,13 @@ RUN addgroup -S -g 1000 druid \
COPY --chown=druid:druid --from=builder /opt /opt
COPY distribution/docker/druid.sh /druid.sh
-RUN mkdir /opt/druid/var \
- && chown druid:druid /opt/druid/var \
- && chmod 775 /opt/druid/var
+
+# create necessary directories which could be mounted as volume
+# /opt/druid/var is used to keep individual files(e.g. log) of each Druid
service
+# /opt/shared is used to keep segments and task logs shared among Druid
services
+RUN mkdir /opt/druid/var /opt/shared \
+ && chown druid:druid /opt/druid/var /opt/shared \
+ && chmod 775 /opt/druid/var /opt/shared
USER druid
VOLUME /opt/druid/var
diff --git a/distribution/docker/docker-compose.yml
b/distribution/docker/docker-compose.yml
index a492b9e..c2343ea 100644
--- a/distribution/docker/docker-compose.yml
+++ b/distribution/docker/docker-compose.yml
@@ -25,6 +25,7 @@ volumes:
broker_var: {}
coordinator_var: {}
router_var: {}
+ druid_shared: {}
services:
@@ -49,7 +50,7 @@ services:
image: apache/druid:0.21.0
container_name: coordinator
volumes:
- - ./storage:/opt/data
+ - druid_shared:/opt/shared
- coordinator_var:/opt/druid/var
depends_on:
- zookeeper
@@ -81,7 +82,7 @@ services:
image: apache/druid:0.21.0
container_name: historical
volumes:
- - ./storage:/opt/data
+ - druid_shared:/opt/shared
- historical_var:/opt/druid/var
depends_on:
- zookeeper
@@ -98,7 +99,7 @@ services:
image: apache/druid:0.21.0
container_name: middlemanager
volumes:
- - ./storage:/opt/data
+ - druid_shared:/opt/shared
- middle_var:/opt/druid/var
depends_on:
- zookeeper
diff --git a/distribution/docker/environment b/distribution/docker/environment
index 2023138..7bb9214 100644
--- a/distribution/docker/environment
+++ b/distribution/docker/environment
@@ -39,12 +39,12 @@ druid_metadata_storage_connector_password=FoolishPassword
druid_coordinator_balancer_strategy=cachingCost
druid_indexer_runner_javaOptsArray=["-server", "-Xmx1g", "-Xms1g",
"-XX:MaxDirectMemorySize=3g", "-Duser.timezone=UTC", "-Dfile.encoding=UTF-8",
"-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager"]
-druid_indexer_fork_property_druid_processing_buffer_sizeBytes=268435456
+druid_indexer_fork_property_druid_processing_buffer_sizeBytes=256MiB
druid_storage_type=local
-druid_storage_storageDirectory=/opt/data/segments
+druid_storage_storageDirectory=/opt/shared/segments
druid_indexer_logs_type=file
-druid_indexer_logs_directory=/opt/data/indexing-logs
+druid_indexer_logs_directory=/opt/shared/indexing-logs
druid_processing_numThreads=2
druid_processing_numMergeBuffers=2
diff --git a/docs/tutorials/docker.md b/docs/tutorials/docker.md
index 33dc8e7..b64b636 100644
--- a/docs/tutorials/docker.md
+++ b/docs/tutorials/docker.md
@@ -36,7 +36,11 @@ The Druid source code contains [an example
`docker-compose.yml`](https://github.
### Compose file
-The example `docker-compose.yml` will create a container for each Druid
service, as well as Zookeeper and a PostgreSQL container as the metadata store.
Deep storage will be a local directory, by default configured as `./storage`
relative to your `docker-compose.yml` file, and will be mounted as `/opt/data`
and shared between Druid containers which require access to deep storage. The
Druid containers are configured via an [environment
file](https://github.com/apache/druid/blob/{{DRUIDVERS [...]
+The example `docker-compose.yml` will create a container for each Druid
service, as well as ZooKeeper and a PostgreSQL container as the metadata store.
+
+It will also create a named volumes `druid_shared`, which is mounted as
`opt/shared` in container, as deep storage to keep and share segments and task
logs among Druid services.
+
+The Druid containers are configured via an [environment
file](https://github.com/apache/druid/blob/{{DRUIDVERSION}}/distribution/docker/environment).
### Configuration
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
