Author: lewismc
Date: Sun Sep 20 12:54:49 2015
New Revision: 1704129
URL: http://svn.apache.org/viewvc?rev=1704129&view=rev
Log:
NUTCH-2050 Upgrade HBase and Hadoop versioning on 2.X HBase Docker
Modified:
nutch/branches/2.x/docker/hbase/Dockerfile
nutch/branches/2.x/docker/hbase/README.md
Modified: nutch/branches/2.x/docker/hbase/Dockerfile
URL:
http://svn.apache.org/viewvc/nutch/branches/2.x/docker/hbase/Dockerfile?rev=1704129&r1=1704128&r2=1704129&view=diff
==============================================================================
--- nutch/branches/2.x/docker/hbase/Dockerfile (original)
+++ nutch/branches/2.x/docker/hbase/Dockerfile Sun Sep 20 12:54:49 2015
@@ -35,8 +35,8 @@ RUN echo oracle-java7-installer shared/a
RUN apt-get install -y ant openssh-server zookeeperd vim telnet subversion
rsync curl build-essential maven
# Download Hadoop
-RUN wget -q
'https://archive.apache.org/dist/hadoop/core/hadoop-2.4.0/hadoop-2.4.0.tar.gz'
-RUN wget -q 'https://github.com/apache/hbase/archive/0.94.14.tar.gz' && mv
0.94.14.tar.gz hbase-0.94.14.tar.gz
+RUN wget -q
'https://archive.apache.org/dist/hadoop/core/hadoop-2.5.1/hadoop-2.5.1.tar.gz'
+RUN wget -q
'http://archive.apache.org/dist/hbase/hbase-0.98.8/hbase-0.98.8-hadoop2-bin.tar.gz'
RUN wget -q 'https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz'
RUN svn checkout http://svn.apache.org/repos/asf/nutch/branches/2.x/
nutch-sources
@@ -55,11 +55,11 @@ RUN chown hduser /home/hduser/.ssh/confi
RUN sed -ri 's/session[[:blank:]]+required[[:blank:]]+pam_loginuid.so/session
optional pam_loginuid.so/g' /etc/pam.d/sshd
# Deploy and setup file permissions
-RUN tar xvfz /root/hadoop-2.4.0.tar.gz -C /opt && \
- ln -s /opt/hadoop-2.4.0 /opt/hadoop && \
- chown -R root:root /opt/hadoop-2.4.0 && \
- mkdir /opt/hadoop-2.4.0/logs && \
- chown -R hduser:hadoop /opt/hadoop-2.4.0/logs
+RUN tar xvfz /root/hadoop-2.5.1.tar.gz -C /opt && \
+ ln -s /opt/hadoop-2.5.1 /opt/hadoop && \
+ chown -R root:root /opt/hadoop-2.5.1 && \
+ mkdir /opt/hadoop-2.5.1/logs && \
+ chown -R hduser:hadoop /opt/hadoop-2.5.1/logs
# Unpack and compile Google protobuf
RUN tar xvfz /root/protobuf-2.5.0.tar.gz -C /opt && \
@@ -67,14 +67,11 @@ RUN tar xvfz /root/protobuf-2.5.0.tar.gz
RUN cd /opt/protobuf-2.5.0 && ./configure && make && make check && make install
# Deploy and setup file permissions
-RUN tar xvfz /root/hbase-0.94.14.tar.gz -C /opt && \
- chown -R root:root /opt/hbase-0.94.14
-# http://hbase.apache.org/book.html#basic.prerequisites - 4.1.1 - HBase has to
be compiled from sources :(
-RUN vim -c '%s/2.4.0a/2.5.0/g' -c '%s/2.0.0-alpha/2.4.0' -c 'x'
/opt/hbase-0.94.14/pom.xml
-RUN cd /opt/hbase-0.94.14/ && mvn clean install assembly:single
-Dhadoop.profile=2.0 -DskipTests
+RUN tar xvfz /root/hbase-0.98.8-hadoop2-bin.tar.gz -C /opt && \
+ chown -R root:root /opt/hbase-0.98.8-hadoop2
# link binaries, create logs directory
-RUN ln -s /opt/hbase-0.94.14/target/hbase-0.94.14/hbase-0.94.14 /opt/hbase &&
mkdir /opt/hbase/logs && \
+RUN ln -s /opt/hbase-0.98.8-hadoop2 /opt/hbase && mkdir /opt/hbase/logs && \
chown -R hduser:hadoop /opt/hbase/logs
# Deploy and setup file permissions
Modified: nutch/branches/2.x/docker/hbase/README.md
URL:
http://svn.apache.org/viewvc/nutch/branches/2.x/docker/hbase/README.md?rev=1704129&r1=1704128&r2=1704129&view=diff
==============================================================================
--- nutch/branches/2.x/docker/hbase/README.md (original)
+++ nutch/branches/2.x/docker/hbase/README.md Sun Sep 20 12:54:49 2015
@@ -1,6 +1,6 @@
# Nutch Dockerfile #
-This directory contains Dockerfile of [Nutch](http://nutch.apache.org) for
[Docker](https://www.docker.com/)'s [automated
build](https://registry.hub.docker.com/u/dockerfile/elasticsearch/) published
to the [Hub Registry](https://registry.hub.docker.com/).
+This directory contains a Dockerfile of [Nutch 2.X](http://nutch.apache.org)
for [Docker](https://www.docker.com/).
## What is Nutch?
@@ -14,9 +14,9 @@ Nutch can run on a single machine, but g
Current configuration of this image consists of components:
-* Hadoop 2.4.0
-* HBase 0.94.14
-* Nutch 2.x
+* Apache Hadoop 2.5.1
+* Apache HBase 0.98.8-hadoop2
+* Apache Nutch 2.X HEAD (this will ensure that you are always running off
of bleeding edge)
## Base Image
@@ -57,3 +57,7 @@ Review configuration in /opt/nutch/conf/
/opt/nutch/bin/nutch parse <batchId>
/opt/nutch/bin/nutch updatedb <batchId>
[...]
+
+## Resources
+
+For more information on Nutch 2.X please see the
[tutorials](http://wiki.apache.org/nutch/#Nutch_2.X_tutorial.28s.29) and [Nutch
2.X wiki space](http://wiki.apache.org/nutch/#Nutch_2.x).
