Here is my set up: orientdb running in a docker container on an AWS server
in Frankfurt, and a similar container on AWS in Sydney. IPSEC tunneling
between them. My docker file looks like this:
FROM dockerfile/java:oracle-java8
MAINTAINER Simon Gemmell
# Update the default application repository sources list
RUN apt-get update
# Install supervisord
RUN apt-get -y install supervisor
RUN mkdir -p /var/log/supervisor
# Install orientdb
ENV ORIENT_VERSION orientdb-community-2.0.3
ENV ORIENT_TAR ${ORIENT_VERSION}.tar.gz
ENV ORIENT_URL
http://www.orientechnologies.com/download.php?file=${ORIENT_TAR}
ENV ROOT /opt
RUN cd ${ROOT} \
&& curl -o ${ORIENT_TAR} ${ORIENT_URL} \
&& tar -xzf ${ORIENT_TAR} \
&& ln -s ${ROOT}/${ORIENT_VERSION} ${ROOT}/orientdb \
&& ln -s ${ROOT}/orientdb/bin/*.sh /usr/local/bin/ \
&& mkdir /usr/local/log
# cleanup
RUN apt-get -y -qq --force-yes clean \
&& rm -rf /opt/downloads/linux /var/lib/apt/lists/* /tmp/* /var/tmp/*
# use supervisord to start orientdb
ADD supervisord.conf /etc/supervisor/conf.d/supervisord.conf
#EXPOSE 2424
EXPOSE 2480
EXPOSE 5701
# Set the user to run OrientDB daemon
USER root
# Default command when starting the container
CMD ["/usr/bin/supervisord", "-c",
"/etc/supervisor/conf.d/supervisord.conf"]
I don't expose 2424 as it is "linked" to another container.
Then I run it like this:
#!/bin/bash
sudo docker run --name orientdb -d -v
`pwd`/orientdb/config:/opt/orientdb/config -v
/mnt/EBS/databases:/opt/orientdb/databases -v
/mnt/EBS/backup:/opt/orientdb/backup -v `pwd`/orientdb/log:/var/log -p
2424:2424 -p 2480:2480 -p 5701:5701 tkbt/orientdb
cd orientdb
./updateip.sh
cd ..
updateip.sh does this:
#!/bin/bash
IP=`sudo docker inspect --format '{{ .NetworkSettings.IPAddress }}'
orientdb`
sed -i 's/<property name="hazelcast.local.localAddress">.*/<property
name="hazelcast.local.localAddress">'$IP'<\/property>/' config/hazelcast.xml
My supervisord:
[supervisord]
nodaemon=true
[program:orientdb]
directory=/opt/orientdb/bin/
command=server.sh -Ddistributed=true -Xmx1024m
autostart=true
autorestart=true
stderr_logfile=/var/log/orientdb.err.log
stdout_logfile=/var/log/orientdb.out.log
; startsecs 5 delays starting orient by 5 secs, which will allow any
external scripts to update IPs etc
startsecs = 5
stopwaitsecs = 120
; make sure we send the signals to any process started by the top scripts
stopasgroup=true
My hazelcastconfig:
....
<properties>
<property name="hazelcast.mancenter.enabled">false</property>
<property name="hazelcast.memcache.enabled">false</property>
<property name="hazelcast.rest.enabled">false</property>
<property name="hazelcast.wait.seconds.before.join">0</property>
<property name="hazelcast.logging.type">jdk</property>
<property
name="hazelcast.local.localAddress">172.17.0.22</property>
<property
name="hazelcast.socket.server.bind.any">false</property>
<property name="hazelcast.socket.client.bind">false</property>
</properties>
<network>
<!-- CHANGE: This is the public ip address of the host: -->
<public-address>10.0.1.96</public-address>
<!-- CHANGE: This port should be mapped on the docker level,
e.g. -p 5701:5701: -->
<port auto-increment="false" port-count="100">5701</port>
<outbound-ports>
<!--
Allowed port range when connecting to other nodes.
0 or * means use system provided port.
-->
<ports>0</ports>
</outbound-ports>
<join>
<!-- CHANGE: disable multicast -->
<multicast enabled="false">
<multicast-group>224.2.2.3</multicast-group>
<multicast-port>54327</multicast-port>
</multicast>
<!-- CHANGE: enable tcp-ip and specify the public ip
addresses of all the hosts that should communicate -->
<tcp-ip enabled="true">
<interface>172.31.6.12</interface>
</tcp-ip>
</join>
</network>
<executor-service>
<pool-size>16</pool-size>
</executor-service>
...
On Thursday, March 5, 2015 at 6:32:36 AM UTC+11, Tobie Morgan Hitchcock
wrote:
>
> Hi Luca,
>
> Docker differences
>
> 1. Using `centos:7` not `debian`
> 2. Using `java-1.7.0-openjdk-headless` instead of `java-1.7.0-openjdk`
> 3. The OrientDB install tar file (
> https://abcum-deploy.s3.amazonaws.com/orient/orientdb-community-2.0.4.tar.gz)
> is an exact copy of the one available at
> http://www.orientechnologies.com/download/ except that it has had the
> demo database removed to make it smaller.
> 4. The OrientDB configs are included in the docker image in the /conf/
> folder, and do not need to be included on a shared volume.
> 5. The OrientDB config and startup script has been simplified slightly so
> that it runs in the foreground on CoreOS in a distributed setup.
> 6. Running the docker image will launch a 'ready to go' distributed
> OrientDB instance using either TCP or AWS node discovery.
> 7. The OrientDB instance is launched in the foreground (not as a daemon)
> so that it can be controlled by Fleet on CoreOS
> 8. Using Hazelcast variables (
> http://blog.hazelcast.com/2013/08/14/xml-variables/) for putting the
> docker environment variables into the hazelcast.xml file.
>
> If running OrientDB on a CoreOS cluster in Amazon EC2/VPC you should be
> able to launch any number of OrientDB instances which connect with each
> other using the Fleetctl service file, which is included in the repository.
>
> I think those are the main differences off the top of my head!
>
> Tobie
>
> On Wednesday, 4 March 2015 18:24:24 UTC, Lvc@ wrote:
>>
>> Hi Tobie,
>> How your project compares to
>> https://github.com/orientechnologies/orientdb-docker ?
>>
>>
>> Lvc@
>>
>>
>> On 4 March 2015 at 14:41, Tobie Morgan Hitchcock <[email protected]> wrote:
>>
>>> Hi Esen,
>>>
>>> I've just been through the same problem, but have a solution.
>>>
>>> Basically, Hazelcast attaches itself to the private ip address of the
>>> docker container, and notifies other nodes of this ip address (meaning that
>>> it does not ever match the members). There are two solutions...
>>>
>>> 1. Use *docker run --net=host* to run the docker container (works ok,
>>> but the container is therefore not running in it's separate network).
>>> 2. Pass the private/public IP address of the machine to the docker
>>> container so that Hazelcast knows which IP address to bind to.
>>>
>>> 1. Add <properties><property name="hazelcast.local.localAddress"
>>> >IPV4ADDRESSHERE</property></properties> into hazelcast.xml
>>> 2. Add <network><public-address>IPV4ADDRESSHERE</public-address></
>>> network>
>>>
>>> I've setup a docker container to do exactly this, which can be used to
>>> setup a distributed OrientDB running on CoreOS using Fleet (fleetctl). This
>>> will work on Vagrant or Amazon EC2/VPC.
>>>
>>> The repository is available here:
>>> https://github.com/abcum/docker-orientdb.git
>>>
>>> Tobie
>>>
>>>
>>> On Tuesday, 6 January 2015 07:19:42 UTC, Esen Sagynov wrote:
>>>>
>>>> I can successfully telnet to each server from any other server because
>>>> incoming requests are allowed. In fact, each server successfully receives
>>>> a
>>>> connection request, however, there seems to be some kind of a validation.
>>>> When the host A receives a connection request from host B, host A checks
>>>> if
>>>> the requested IP address is host A's IP address. Now here this validation
>>>> fails according to the above error message.
>>>>
>>>> I wonder if it is necessary to do such validations in ODB?
>>>>
>>>> On Monday, January 5, 2015 9:27:31 PM UTC+9, Lvc@ wrote:
>>>>>
>>>>> Hi Esen,
>>>>> Seems OrientDB servers can't see each other. I suggest you to try if
>>>>> connections are allowed between hosts with CURL.
>>>>>
>>>>> Lvc@
>>>>>
>>>>>
>>>>> On 5 January 2015 at 07:19, Esen Sagynov <[email protected]> wrote:
>>>>>
>>>>>> Using ODB 2.0-SNAPSHOT (Jan 5, 2015).
>>>>>>
>>>>>> My nodes in Docker containers on separate physical machines cannot
>>>>>> join to each other. The establish a connection however eventually they
>>>>>> refuse joining because some conditions aren't met.
>>>>>>
>>>>>> The following are logs on Host 2:
>>>>>>
>>>>>> 2015-01-05 06:07:15:691 INFO [172.17.0.3]:2434 [orientdb] [3.3]
>>>>>> Accepting socket connection from /123.123.123.124:55647 [
>>>>>> SocketAcceptor]
>>>>>> 2015-01-05 06:07:15:692 INFO [172.17.0.3]:2434 [orientdb] [3.3]
>>>>>> Established socket connection between /172.17.0.3:2434 and /123.123.
>>>>>> 123.124:55647 [TcpIpConnectionManager]
>>>>>> 2015-01-05 06:07:15:694 WARNING [172.17.0.3]:2434 [orientdb] [3.3]
>>>>>> Wrong bind request from Address[172.17.0.12]:2434! This node is not
>>>>>> requested endpoint: Address[123.123.123.124]:2434 [
>>>>>> TcpIpConnectionManager]
>>>>>>
>>>>>> Host 1 (123.123.123.123) running ODB in a container 172.17.0.3:2434.
>>>>>> Host 2 (123.123.123.124) running ODB in a container 172.17.0.12:2434.
>>>>>> Host 3 (123.123.123.125) running ODB in a container 172.17.0.10:2434.
>>>>>>
>>>>>> Because docker on each host manages its own bridge each ODB container
>>>>>> doesn't see it directly. This is why in config/hazelcast.xml I specify
>>>>>> the
>>>>>> IP of the host machines.
>>>>>>
>>>>>> <tcp-ip enabled="true">
>>>>>> <member> 123.123.123.123:2434<
>>>>>> /member>
>>>>>> <member> 123.123.123.124:2434<
>>>>>> /member>
>>>>>> <member> 123.123.123.125:2434<
>>>>>> /member>
>>>>>> </tcp-ip>
>>>>>>
>>>>>> All ODB containers expose all 3 ports (2424, 2480, 2434). I can
>>>>>> telnet and ping. However, because each container requests a dynamic
>>>>>> docker
>>>>>> IP address, each node cannot join the same cluster because host IP
>>>>>> doesn't
>>>>>> not match the actual Docker IP inside the container as shown in the
>>>>>> above
>>>>>> error log.
>>>>>>
>>>>>> Any solutions for this?
>>>>>>
>>>>>> --
>>>>>>
>>>>>> ---
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "OrientDB" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "OrientDB" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
