[accumulo-docker] branch master updated: Dockerfile & README updates (#9)
This is an automated email from the ASF dual-hosted git repository.
mwalch pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/accumulo-docker.git
The following commit(s) were added to refs/heads/master by this push:
new aa2a60c Dockerfile & README updates (#9)
aa2a60c is described below
commit aa2a60cacb052fb8e763dcb76a511e97f1228b6e
Author: Mike Walch
AuthorDate: Tue Mar 19 16:23:09 2019 -0400
Dockerfile & README updates (#9)
---
Dockerfile | 16 +---
README.md | 4 +++-
2 files changed, 16 insertions(+), 4 deletions(-)
diff --git a/Dockerfile b/Dockerfile
index 551046a..0350ae0 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -23,6 +23,8 @@ ARG HADOOP_VERSION=3.1.1
ARG ZOOKEEPER_VERSION=3.4.13
ARG HADOOP_USER_NAME=accumulo
ARG ACCUMULO_FILE=
+ARG HADOOP_FILE=
+ARG ZOOKEEPER_FILE=
ENV HADOOP_USER_NAME $HADOOP_USER_NAME
@@ -33,7 +35,7 @@ ENV APACHE_DIST_URLS \
https://www.apache.org/dist/ \
https://archive.apache.org/dist/
-COPY README.md $ACCUMULO_FILE /tmp/
+COPY README.md $ACCUMULO_FILE $HADOOP_FILE $ZOOKEEPER_FILE /tmp/
RUN set -eux; \
download() { \
@@ -50,8 +52,16 @@ RUN set -eux; \
[ -n "$success" ]; \
}; \
\
- download "hadoop.tar.gz"
"hadoop/core/hadoop-$HADOOP_VERSION/hadoop-$HADOOP_VERSION.tar.gz"; \
- download "zookeeper.tar.gz"
"zookeeper/zookeeper-$ZOOKEEPER_VERSION/zookeeper-$ZOOKEEPER_VERSION.tar.gz"; \
+ if [ -z "$HADOOP_FILE" ]; then \
+download "hadoop.tar.gz"
"hadoop/core/hadoop-$HADOOP_VERSION/hadoop-$HADOOP_VERSION.tar.gz"; \
+ else \
+cp "/tmp/$HADOOP_FILE" "hadoop.tar.gz"; \
+ fi; \
+ if [ -z "$ZOOKEEPER_FILE" ]; then \
+download "zookeeper.tar.gz"
"zookeeper/zookeeper-$ZOOKEEPER_VERSION/zookeeper-$ZOOKEEPER_VERSION.tar.gz"; \
+ else \
+cp "/tmp/$ZOOKEEPER_FILE" "zookeeper.tar.gz"; \
+ fi; \
if [ -z "$ACCUMULO_FILE" ]; then \
download "accumulo.tar.gz"
"accumulo/$ACCUMULO_VERSION/accumulo-$ACCUMULO_VERSION-bin.tar.gz"; \
else \
diff --git a/README.md b/README.md
index 9d34287..2a5a117 100644
--- a/README.md
+++ b/README.md
@@ -3,7 +3,9 @@
**This is currently a work in progress that depends on unreleased features of
Accumulo and will not be ready
for use until after Accumulo 2.0.0 is released.** Sometime after Accumulo
2.0.0 is released this project
will make its first release. Eventually, this will project will create a
`apache/accumulo` image at DockerHub.
-Until then, you will need to build your own image.
+Until then, you will need to build your own image. The master branch of this
repo creates a Docker image for
+Accumulo 2.0+. If you want to create a Docker image for Accumulo 1.9, there is
a
+[1.9 branch](https://github.com/apache/accumulo-docker/tree/1.9) for that.
## Obtain the Docker image
[accumulo] branch master updated: Improve use of DistributedCacheHelper. Closes #896 (#1035)
This is an automated email from the ASF dual-hosted git repository.
mmiller pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to refs/heads/master by this push:
new 6f4c772 Improve use of DistributedCacheHelper. Closes #896 (#1035)
6f4c772 is described below
commit 6f4c7721809ce4e15e2296e83b54d003cea8e9a5
Author: Mike Miller
AuthorDate: Tue Mar 19 13:33:44 2019 -0400
Improve use of DistributedCacheHelper. Closes #896 (#1035)
---
.../mapreduce/lib/partition/RangePartitioner.java | 6 +--
.../mapreduce/lib/DistributedCacheHelper.java | 2 -
.../mapreduce/partition/RangePartitioner.java | 5 +--
.../hadoopImpl/mapreduce/lib/ConfiguratorBase.java | 1 +
.../mapreduce/lib/DistributedCacheHelper.java | 51 --
5 files changed, 5 insertions(+), 60 deletions(-)
diff --git
a/core/src/main/java/org/apache/accumulo/core/client/mapreduce/lib/partition/RangePartitioner.java
b/core/src/main/java/org/apache/accumulo/core/client/mapreduce/lib/partition/RangePartitioner.java
index 9f890fe..6b32130 100644
---
a/core/src/main/java/org/apache/accumulo/core/client/mapreduce/lib/partition/RangePartitioner.java
+++
b/core/src/main/java/org/apache/accumulo/core/client/mapreduce/lib/partition/RangePartitioner.java
@@ -91,8 +91,7 @@ public class RangePartitioner extends
Partitioner implements Conf
private synchronized Text[] getCutPoints() throws IOException {
if (cutPointArray == null) {
String cutFileName = conf.get(CUTFILE_KEY);
- Path[] cf =
org.apache.accumulo.core.clientImpl.mapreduce.lib.DistributedCacheHelper
- .getLocalCacheFiles(conf);
+ Path[] cf = Job.getInstance().getLocalCacheFiles();
if (cf != null) {
for (Path path : cf) {
@@ -131,8 +130,7 @@ public class RangePartitioner extends
Partitioner implements Conf
*/
public static void setSplitFile(Job job, String file) {
URI uri = new Path(file).toUri();
-
org.apache.accumulo.core.clientImpl.mapreduce.lib.DistributedCacheHelper.addCacheFile(uri,
-job.getConfiguration());
+job.addCacheFile(uri);
job.getConfiguration().set(CUTFILE_KEY, uri.getPath());
}
diff --git
a/core/src/main/java/org/apache/accumulo/core/clientImpl/mapreduce/lib/DistributedCacheHelper.java
b/core/src/main/java/org/apache/accumulo/core/clientImpl/mapreduce/lib/DistributedCacheHelper.java
index f115ee7..f99a4ad 100644
---
a/core/src/main/java/org/apache/accumulo/core/clientImpl/mapreduce/lib/DistributedCacheHelper.java
+++
b/core/src/main/java/org/apache/accumulo/core/clientImpl/mapreduce/lib/DistributedCacheHelper.java
@@ -24,9 +24,7 @@ import org.apache.hadoop.fs.Path;
/**
* @since 1.6.0
- * @deprecated since 2.0.0
*/
-@Deprecated
public class DistributedCacheHelper {
/**
diff --git
a/hadoop-mapreduce/src/main/java/org/apache/accumulo/hadoop/mapreduce/partition/RangePartitioner.java
b/hadoop-mapreduce/src/main/java/org/apache/accumulo/hadoop/mapreduce/partition/RangePartitioner.java
index abe01cc..7304904 100644
---
a/hadoop-mapreduce/src/main/java/org/apache/accumulo/hadoop/mapreduce/partition/RangePartitioner.java
+++
b/hadoop-mapreduce/src/main/java/org/apache/accumulo/hadoop/mapreduce/partition/RangePartitioner.java
@@ -29,7 +29,6 @@ import java.util.Base64;
import java.util.Scanner;
import java.util.TreeSet;
-import org.apache.accumulo.hadoopImpl.mapreduce.lib.DistributedCacheHelper;
import org.apache.hadoop.conf.Configurable;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
@@ -90,7 +89,7 @@ public class RangePartitioner extends
Partitioner implements Conf
private synchronized Text[] getCutPoints() throws IOException {
if (cutPointArray == null) {
String cutFileName = conf.get(CUTFILE_KEY);
- Path[] cf = DistributedCacheHelper.getLocalCacheFiles(conf);
+ Path[] cf = Job.getInstance().getLocalCacheFiles();
if (cf != null) {
for (Path path : cf) {
@@ -129,7 +128,7 @@ public class RangePartitioner extends
Partitioner implements Conf
*/
public static void setSplitFile(Job job, String file) {
URI uri = new Path(file).toUri();
-DistributedCacheHelper.addCacheFile(uri, job.getConfiguration());
+job.addCacheFile(uri);
job.getConfiguration().set(CUTFILE_KEY, uri.getPath());
}
diff --git
a/hadoop-mapreduce/src/main/java/org/apache/accumulo/hadoopImpl/mapreduce/lib/ConfiguratorBase.java
b/hadoop-mapreduce/src/main/java/org/apache/accumulo/hadoopImpl/mapreduce/lib/ConfiguratorBase.java
index 810320d..5155b76 100644
---
a/hadoop-mapreduce/src/main/java/org/apache/accumulo/hadoopImpl/mapreduce/lib/ConfiguratorBase.java
+++
b/hadoop-mapreduce/src/main/java/org/apache/accumulo/hadoopImpl/mapreduce/lib/ConfiguratorBase.java
@@ -27,6 +27,7 @@ import java.util.Scanner;
import org.apache.accumulo.core.Constants;
import
[accumulo-docker] branch 1.9 created (now f80b503)
This is an automated email from the ASF dual-hosted git repository. mwalch pushed a change to branch 1.9 in repository https://gitbox.apache.org/repos/asf/accumulo-docker.git. at f80b503 Support Accumulo 1.9 docker image This branch includes the following new commits: new 4e8d47c First commit new f80b503 Support Accumulo 1.9 docker image The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[accumulo-docker] 02/02: Support Accumulo 1.9 docker image
This is an automated email from the ASF dual-hosted git repository.
mwalch pushed a commit to branch 1.9
in repository https://gitbox.apache.org/repos/asf/accumulo-docker.git
commit f80b5036d909b125cab3d781dba88d933b5b5bd8
Author: Mike Walch
AuthorDate: Tue Mar 19 11:07:59 2019 -0400
Support Accumulo 1.9 docker image
---
CONTRIBUTING.md| 30
Dockerfile | 92
LICENSE| 202 +
NOTICE | 5 ++
README.md | 57 ++-
accumulo-site.xml | 51 ++
generic_logger.xml | 67 ++
monitor_logger.xml | 48 +
8 files changed, 551 insertions(+), 1 deletion(-)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
new file mode 100644
index 000..7daf3d0
--- /dev/null
+++ b/CONTRIBUTING.md
@@ -0,0 +1,30 @@
+
+
+# Contributing to the Accumulo Docker Image
+
+Contributions to the Accumulo Docker Image can be made by creating a pull
request to
+this repo on GitHub.
+
+Before creating a pull request, follow the instructions in the [README.md] to
build
+the image and use it to run Accumulo in Docker.
+
+For general instructions on contributing to Accumulo projects, check out the
+[Accumulo Contributor guide][contribute].
+
+[README.md]: README.md
+[contribute]: https://accumulo.apache.org/contributor/
diff --git a/Dockerfile b/Dockerfile
new file mode 100644
index 000..e4b8808
--- /dev/null
+++ b/Dockerfile
@@ -0,0 +1,92 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM centos:7
+
+RUN yum install -y java-1.8.0-openjdk-devel make gcc-c++ wget
+ENV JAVA_HOME /usr/lib/jvm/java-1.8.0-openjdk
+
+ARG ACCUMULO_VERSION=1.9.2
+ARG HADOOP_VERSION=2.8.5
+ARG ZOOKEEPER_VERSION=3.4.13
+ARG HADOOP_USER_NAME=accumulo
+ARG ACCUMULO_FILE=
+ARG HADOOP_FILE=
+ARG ZOOKEEPER_FILE=
+
+ENV HADOOP_USER_NAME $HADOOP_USER_NAME
+
+ENV APACHE_DIST_URLS \
+ https://www.apache.org/dyn/closer.cgi?action=download= \
+# if the version is outdated (or we're grabbing the .asc file), we might have
to pull from the dist/archive :/
+ https://www-us.apache.org/dist/ \
+ https://www.apache.org/dist/ \
+ https://archive.apache.org/dist/
+
+COPY README.md $ACCUMULO_FILE $HADOOP_FILE $ZOOKEEPER_FILE /tmp/
+
+RUN set -eux; \
+ download() { \
+local f="$1"; shift; \
+local distFile="$1"; shift; \
+local success=; \
+local distUrl=; \
+for distUrl in $APACHE_DIST_URLS; do \
+ if wget -nv -O "$f" "$distUrl$distFile"; then \
+success=1; \
+break; \
+ fi; \
+done; \
+[ -n "$success" ]; \
+ }; \
+ \
+ if [ -z "$HADOOP_FILE" ]; then \
+download "hadoop.tar.gz"
"hadoop/core/hadoop-$HADOOP_VERSION/hadoop-$HADOOP_VERSION.tar.gz"; \
+ else \
+cp "/tmp/$HADOOP_FILE" "hadoop.tar.gz"; \
+ fi; \
+ if [ -z "$ZOOKEEPER_FILE" ]; then \
+download "zookeeper.tar.gz"
"zookeeper/zookeeper-$ZOOKEEPER_VERSION/zookeeper-$ZOOKEEPER_VERSION.tar.gz"; \
+ else \
+cp "/tmp/$ZOOKEEPER_FILE" "zookeeper.tar.gz"; \
+ fi; \
+ if [ -z "$ACCUMULO_FILE" ]; then \
+download "accumulo.tar.gz"
"accumulo/$ACCUMULO_VERSION/accumulo-$ACCUMULO_VERSION-bin.tar.gz"; \
+ else \
+cp "/tmp/$ACCUMULO_FILE" "accumulo.tar.gz"; \
+ fi;
+
+RUN tar xzf accumulo.tar.gz -C /tmp/
+RUN tar xzf hadoop.tar.gz -C /tmp/
+RUN tar xzf zookeeper.tar.gz -C /tmp/
+
+RUN mv /tmp/hadoop-$HADOOP_VERSION /opt/hadoop
+RUN mv /tmp/zookeeper-$ZOOKEEPER_VERSION /opt/zookeeper
+RUN mv /tmp/accumulo-$ACCUMULO_VERSION /opt/accumulo
+
+RUN cp /opt/accumulo/conf/examples/2GB/native-standalone/* /opt/accumulo/conf/
+RUN /opt/accumulo/bin/build_native_library.sh
+
+ADD ./accumulo-site.xml /opt/accumulo/conf
+ADD ./generic_logger.xml /opt/accumulo/conf
+ADD ./monitor_logger.xml /opt/accumulo/conf
+
+ENV HADOOP_HOME /opt/hadoop
+ENV ZOOKEEPER_HOME /opt/zookeeper
+ENV ACCUMULO_HOME /opt/accumulo
+ENV PATH "$PATH:$ACCUMULO_HOME/bin"
+
+ENTRYPOINT ["accumulo"]
+CMD ["help"]
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 000..d645695
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,202 @@
+
+ Apache License
+ Version 2.0, January 2004
+
[accumulo-docker] 01/02: First commit
This is an automated email from the ASF dual-hosted git repository. mwalch pushed a commit to branch 1.9 in repository https://gitbox.apache.org/repos/asf/accumulo-docker.git commit 4e8d47c3731eed67b3ef37d9d691b59ed1c955f1 Author: Mike Walch AuthorDate: Tue Mar 19 10:51:46 2019 -0400 First commit --- .gitignore | 1 + README.md | 1 + 2 files changed, 2 insertions(+) diff --git a/.gitignore b/.gitignore new file mode 100644 index 000..335ec95 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +*.tar.gz diff --git a/README.md b/README.md new file mode 100644 index 000..6fbad74 --- /dev/null +++ b/README.md @@ -0,0 +1 @@ += Accumulo Docker =
[accumulo-docker] branch 1.9 deleted (was f0a1fa7)
This is an automated email from the ASF dual-hosted git repository. mwalch pushed a change to branch 1.9 in repository https://gitbox.apache.org/repos/asf/accumulo-docker.git. was f0a1fa7 Dockerfile updates (#7) The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
