Hi Graeme,

I can connect to 5060 port in 10.0.1.2 from sprout container.

root@bb818f0a5535:/# nc -v -z 10.0.1.2 5060
Connection to 10.0.1.2 5060 port [tcp/sip] succeeded!
root@bb818f0a5535:/#


Docker PS on the cluster shows bono container exposing port 5060


CONTAINER ID        IMAGE                                        COMMAND
               CREATED             STATUS              PORTS


            NAMES
bb818f0a5535        swarm-node:5000/clearwaterdocker_sprout
 "/usr/bin/supervisord"   47 hours ago        Up 47 hours         5052/tcp,
5054/tcp, 10.109.190.10:32822->22/tcp

                      swarm-node/clearwaterdocker_sprout_1
9ff210721451        swarm-node:5000/clearwaterdocker_homer
"/usr/bin/supervisord"   47 hours ago        Up 47 hours         7888/tcp,
10.109.190.9:32788->22/tcp

                       swarm-master/clearwaterdocker_homer_1
51653420c979        swarm-node:5000/clearwaterdocker_homestead
"/usr/bin/supervisord"   47 hours ago        Up 47 hours
8888-8889/tcp, 10.109.190.9:32787->22/tcp

                                swarm-master/clearwaterdocker_homestead_1
e994b17b4563        swarm-node:5000/clearwaterdocker_ellis
"/usr/bin/supervisord"   47 hours ago        Up 47 hours
10.109.190.10:80->80/tcp, 10.109.190.10:32821->22/tcp

                                swarm-node/clearwaterdocker_ellis_1
9837c4dab241        swarm-node:5000/clearwaterdocker_bono
 "/usr/bin/supervisord"   47 hours ago        Up 47 hours
10.109.190.9:3478->3478/tcp, 10.109.190.9:3478->3478/udp,
10.109.190.9:5060->5060/tcp,
10.109.190.9:5062->5062/tcp, 10.109.190.9:5060->5060/udp, 5058/tcp,
10.109.190.9:32786->22/tcp   swarm-master/clearwaterdocker_bono_1
3db967c58754        swarm-node:5000/clearwaterdocker_ralf
 "/usr/bin/supervisord"   47 hours ago        Up 47 hours
10888/tcp, 10.109.190.10:32820->22/tcp

                                 swarm-node/clearwaterdocker_ralf_1
c499d05af8e7        quay.io/coreos/etcd:v2.2.5                   "/etcd
-name etcd0 -a"   47 hours ago        Up 47 hours         2379-2380/tcp,
4001/tcp, 7001/tcp

                  swarm-node/clearwaterdocker_etcd_1
5fe0c51979e7        ubuntu:14.04.5
"/bin/bash"              8 days ago          Up 8 days



But I can't reach the 5060 port from the host machine where it is launched.
Though the port seems to be open.


root@swarm-master:~# nc -v -z 10.109.190.9 5060
nc: connect to 10.109.190.9 port 5060 (tcp) failed: Connection refused
root@swarm-master:~#
root@swarm-master:~# netstat -anp | grep 5060
tcp6       0      0 :::5060                 :::*                    LISTEN
     21818/docker-proxy
udp6       0      0 :::5060                 :::*
     21829/docker-proxy
root@swarm-master:~#


Any suggestion?

-Sarbajit


On Thu, Sep 22, 2016 at 4:08 PM, Graeme Robertson (projectclearwater.org) <
g...@projectclearwater.org> wrote:

> Hi Sarbajit,
>
>
>
> That output all looks fine, but it sounds as though the port mapping has
> failed – i.e. port 5060 on the Bono container hasn’t been exposed as port
> 5060 on the host. I’m not sure why that would have failed. Can you run nc
> -z -v 10.0.1.2 5060 inside the Bono container (this should work, but it’s
> worth doing as a sanity check!). Then can you run docker ps in your
> clearwater-docker checkout? The output should include a line that looks
> something like 0b4058027844        clearwaterdocker_bono
> "/usr/bin/supervisord"   About a minute ago   Up About a minute
> 0.0.0.0:3478->3478/tcp, 0.0.0.0:3478->3478/udp, *0.0.0.0:5060->5060/tcp*,
> 0.0.0.0:5062->5062/tcp, *0.0.0.0:5060->5060/udp*, 5058/tcp, 
> 0.0.0.0:42513->22/tcp
> clearwaterdocker_bono_1, which should indicate that the port mapping is
> active. If this all looks fine it might be worth also running nc -v -z
> 10.109.190.9 5060 on the host that’s running the Bono container.
>
>
>
> Thanks,
>
> Graeme
>
>
> ------------------------------
>
> *From:* Clearwater [mailto:clearwater-boun...@lists.projectclearwater.org]
> *On Behalf Of *Sarbajit Chatterjee
> *Sent:* 21 September 2016 19:39
>
> *To:* clearwater@lists.projectclearwater.org
> *Subject:* Re: [Project Clearwater] Deploy Clearwater in a Swarm cluster
> using docker-compose
>
>
>
> Hi Graeme,
>
>
>
> I'm using following command to run the livetest -
>
>
>
> rake test[example.com] TESTS="Basic*" SIGNUP_CODE=secret
> PROXY=10.109.190.9 ELLIS=10.109.190.10
>
>
>
> here PROXY ip is where the bono container is launched and ELLIS ip is
> where the ellis container is launched.
>
>
>
> The bono service seems to be running in the container -
>
>
>
> root@9837c4dab241:/# ps -eaf | grep bono
>
> root       122     1  0 Sep19 ?        00:00:10 /usr/share/clearwater/
> clearwater-cluster-manager/env/bin/python 
> /usr/share/clearwater/bin/clearwater-cluster-manager
> --mgmt-local-ip=10.0.1.2 --sig-local-ip=10.0.1.2 --local-site=site1
> --remote-site= --remote-cassandra-seeds= --signaling-namespace=
> --uuid=18c7daf3-a098-47ae-962f-a3d57c0cff6f --etcd-key=clearwater
> --etcd-cluster-key=bono --log-level=3 
> --log-directory=/var/log/clearwater-cluster-manager
> --pidfile=/var/run/clearwater-cluster-manager.pid
>
> root       124     1  0 Sep19 ?        00:00:00 /bin/bash /etc/init.d/bono
> run
>
> root       139   124  0 Sep19 ?        00:00:00 /bin/bash
> /usr/share/clearwater/bin/run-in-signaling-namespace start-stop-daemon
> --start --quiet --exec /usr/share/clearwater/bin/bono --chuid bono --chdir
> /etc/clearwater -- --domain=example.com --localhost=10.0.1.2,10.0.1.2
> --alias=10.0.1.2 --pcscf=5060,5058 --webrtc-port=5062
> --routing-proxy=scscf.sprout,5052,50,600 --ralf=ralf:10888 --sas=0.0.0.0,
> bono@10.0.1.2 --dns-server=127.0.0.11 --worker-threads=4
> --analytics=/var/log/bono --log-file=/var/log/bono --log-level=2
>
> bono       140   139  0 Sep19 ?        00:11:15
> /usr/share/clearwater/bin/bono --domain=example.com
> --localhost=10.0.1.2,10.0.1.2 --alias=10.0.1.2 --pcscf=5060,5058
> --webrtc-port=5062 --routing-proxy=scscf.sprout,5052,50,600
> --ralf=ralf:10888 --sas=0.0.0.0,bono@10.0.1.2 --dns-server=127.0.0.11
> --worker-threads=4 --analytics=/var/log/bono --log-file=/var/log/bono
> --log-level=2
>
> root       322   293  0 17:48 ?        00:00:00 grep --color=auto bono
>
> root@9837c4dab241:/#
>
> root@9837c4dab241:/# netstat -planut | grep 5060
>
> tcp        0      0 10.0.1.2:5060           0.0.0.0:*
> LISTEN      -
>
> udp        0      0 10.0.1.2:5060           0.0.0.0:*
>       -
>
> root@9837c4dab241:/#
>
>
>
>
>
> But the connection to bono is failing from livetest container as you had
> predicted.
>
>
>
> root@40efba73deb5:~/clearwater-live-test# nc -v -z 10.109.190.9 5060
>
> nc: connect to 10.109.190.9 port 5060 (tcp) failed: Connection refused
>
> root@40efba73deb5:~/clearwater-live-test#
>
>
>
>
>
> On checking the bono log, I see a series of errors like below
> in beginning of the log -
>
>
>
> 19-09-2016 15:41:44.612 UTC Status utils.cpp:591: Log level set to 2
>
> 19-09-2016 15:41:44.612 UTC Status main.cpp:1388: Access logging enabled
> to /var/log/bono
>
> 19-09-2016 15:41:44.613 UTC Warning main.cpp:1435: SAS server option was
> invalid or not configured - SAS is disabled
>
> 19-09-2016 15:41:44.613 UTC Warning main.cpp:1511: A registration expiry
> period should not be specified for P-CSCF
>
> 19-09-2016 15:41:44.613 UTC Status snmp_agent.cpp:117: AgentX agent
> initialised
>
> 19-09-2016 15:41:44.613 UTC Status load_monitor.cpp:105: Constructing
> LoadMonitor
>
> 19-09-2016 15:41:44.613 UTC Status load_monitor.cpp:106:    Target latency
> (usecs)   : 100000
>
> 19-09-2016 15:41:44.613 UTC Status load_monitor.cpp:107:    Max bucket
> size          : 1000
>
> 19-09-2016 15:41:44.613 UTC Status load_monitor.cpp:108:    Initial token
> fill rate/s: 100.000000
>
> 19-09-2016 15:41:44.613 UTC Status load_monitor.cpp:109:    Min token fill
> rate/s    : 10.000000
>
> 19-09-2016 15:41:44.613 UTC Status dnscachedresolver.cpp:144: Creating
> Cached Resolver using servers:
>
> 19-09-2016 15:41:44.613 UTC Status dnscachedresolver.cpp:154:
> 127.0.0.11
>
> 19-09-2016 15:41:44.613 UTC Status sipresolver.cpp:60: Created SIP resolver
>
> 19-09-2016 15:41:44.637 UTC Status stack.cpp:419: Listening on port 5058
>
> 19-09-2016 15:41:44.637 UTC Status stack.cpp:419: Listening on port 5060
>
> 19-09-2016 15:41:44.638 UTC Status stack.cpp:855: Local host aliases:
>
> 19-09-2016 15:41:44.638 UTC Status stack.cpp:862:  10.0.1.2
>
> 19-09-2016 15:41:44.638 UTC Status stack.cpp:862:  172.18.0.2
>
> 19-09-2016 15:41:44.638 UTC Status stack.cpp:862:  10.0.1.2
>
> 19-09-2016 15:41:44.638 UTC Status stack.cpp:862:  10.0.1.2
>
> 19-09-2016 15:41:44.638 UTC Status stack.cpp:862:
>
> 19-09-2016 15:41:44.639 UTC Status httpresolver.cpp:52: Created HTTP
> resolver
>
> 19-09-2016 15:41:44.641 UTC Status httpconnection.cpp:114: Configuring
> HTTP Connection
>
> 19-09-2016 15:41:44.641 UTC Status httpconnection.cpp:115:   Connection
> created for server ralf:10888
>
> 19-09-2016 15:41:44.641 UTC Status httpconnection.cpp:116:   Connection
> will use a response timeout of 500ms
>
> 19-09-2016 15:41:44.642 UTC Status connection_pool.cpp:72: Creating
> connection pool to scscf.sprout:5052
>
> 19-09-2016 15:41:44.642 UTC Status connection_pool.cpp:73:   connections =
> 50, recycle time = 600 +/- 120 seconds
>
> 19-09-2016 15:41:44.649 UTC Status bono.cpp:3314: Create list of PBXes
>
> 19-09-2016 15:41:44.649 UTC Status pluginloader.cpp:63: Loading plug-ins
> from /usr/share/clearwater/sprout/plugins
>
> 19-09-2016 15:41:44.649 UTC Status pluginloader.cpp:158: Finished loading
> plug-ins
>
> 19-09-2016 15:41:44.652 UTC Warning (Net-SNMP): Warning: Failed to connect
> to the agentx master agent ([NIL]):
>
> 19-09-2016 15:41:44.653 UTC Error pjsip:  tcpc0x14f3df8 TCP connect()
> error: Connection refused [code=120111]
>
> 19-09-2016 15:41:44.653 UTC Error pjsip:  tcpc0x14f5c38 TCP connect()
> error: Connection refused [code=120111]
>
>
>
>
>
> I have observed that in the bono container 5060 port is not listening in
> all interfaces while 5062 port is listening in all interfaces.
>
>
>
> root@9837c4dab241:/var/log/bono# netstat -planut | grep LISTEN
>
> tcp        0      0 10.0.1.2:4000           0.0.0.0:*
> LISTEN      -
>
> tcp        0      0 10.0.1.2:5058           0.0.0.0:*
> LISTEN      -
>
> tcp        0      0 127.0.0.11:43395        0.0.0.0:*
> LISTEN      -
>
> tcp        0      0 10.0.1.2:5060           0.0.0.0:*
> LISTEN      -
>
> tcp        0      0 0.0.0.0:5062            0.0.0.0:*
> LISTEN      -
>
> tcp        0      0 127.0.0.1:8080          0.0.0.0:*
> LISTEN      -
>
> tcp        0      0 10.0.1.2:3478           0.0.0.0:*
> LISTEN      -
>
> tcp        0      0 0.0.0.0:22              0.0.0.0:*
> LISTEN      9/sshd
>
> tcp6       0      0 :::22                   :::*                    LISTEN
>      9/sshd
>
> root@9837c4dab241:/var/log/bono#
>
>
>
>
>
> Is it right to have 5060 port listen on only local port? Please help me to
> debug the issue.
>
>
>
> Thanks,
>
> Sarbajit
>
>
>
>
>
> On Wed, Sep 21, 2016 at 10:01 PM, Graeme Robertson (projectclearwater.org)
> <gra...@projectclearwater.org> wrote:
>
> Hi Sarbajit,
>
>
>
> I’ve had another look at this, and actually I think clearwater-live-test
> checks it can connect to Bono before it tries to provision numbers from
> Ellis, and it’s actually that connection that’s failing – apologies!
>
>
>
> Can you do similar checks for the Bono container, i.e. connect to the Bono
> container and run ps -eaf | grep bono and run nc -z -v <ip> 5060 from
> your live test container (where <ip> is the IP of your Bono)?
>
>
>
> One other thought – what command are you using to run the tests? You’ll
> need to set the PROXY option to your Bono IP and the ELLIS option to your
> Ellis IP.
>
>
>
> Thanks,
>
> Graeme
>
>
>
> *From:* Clearwater [mailto:clearwater-boun...@lists.projectclearwater.org]
> *On Behalf Of *Sarbajit Chatterjee
> *Sent:* 21 September 2016 16:41
> *To:* clearwater@lists.projectclearwater.org
> *Subject:* Re: [Project Clearwater] Deploy Clearwater in a Swarm cluster
> using docker-compose
>
>
>
> Thanks Graeme for your reply. Here are the command outputs that you had
> asked -
>
>
>
> root@e994b17b4563:/# ps -eaf | grep ellis
>
> root       177     1  0 Sep19 ?        00:00:10 /usr/share/clearwater/
> clearwater-cluster-manager/env/bin/python 
> /usr/share/clearwater/bin/clearwater-cluster-manager
> --mgmt-local-ip=10.0.1.7 --sig-local-ip=10.0.1.7 --local-site=site1
> --remote-site= --remote-cassandra-seeds= --signaling-namespace=
> --uuid=18c7daf3-a098-47ae-962f-a3d57c0cff6f --etcd-key=clearwater
> --etcd-cluster-key=ellis --log-level=3 
> --log-directory=/var/log/clearwater-cluster-manager
> --pidfile=/var/run/clearwater-cluster-manager.pid
>
> root       180     1  0 Sep19 ?        00:00:00 /bin/sh /etc/init.d/ellis
> run
>
> ellis      185   180  0 Sep19 ?        00:00:05
> /usr/share/clearwater/ellis/env/bin/python -m metaswitch.ellis.main
>
> root       287   253  0 15:18 ?        00:00:00 grep --color=auto ellis
>
> root@e994b17b4563:/#
>
> root@e994b17b4563:/# ps -eaf | grep nginx
>
> root       179     1  0 Sep19 ?        00:00:00 nginx: master process
> /usr/sbin/nginx -g daemon off;
>
> www-data   186   179  0 Sep19 ?        00:00:16 nginx: worker process
>
> www-data   187   179  0 Sep19 ?        00:00:00 nginx: worker process
>
> www-data   188   179  0 Sep19 ?        00:00:16 nginx: worker process
>
> www-data   189   179  0 Sep19 ?        00:00:16 nginx: worker process
>
> root       289   253  0 15:19 ?        00:00:00 grep --color=auto nginx
>
> root@e994b17b4563:/#
>
> root@e994b17b4563:/# netstat -planut | grep nginx
>
> tcp6       0      0 :::80                   :::*                    LISTEN
>      179/nginx -g daemon
>
> root@e994b17b4563:/#
>
>
>
>
>
> I think both ellis and nginx are running fine inside the container. I can
> also open the ellis login page from a web browser. I also checked the MySQL
> DB in ellis container. I can see livetest user entry in users table and
> 1000 rows in numbers table.
>
>
>
> I can also connect to ellis (host IP 10.109.190.10) from my livetest
> container -
>
>
>
> root@40efba73deb5:~/clearwater-live-test# nc -v -z 10.109.190.10 80
>
> Connection to 10.109.190.10 80 port [tcp/http] succeeded!
>
> root@40efba73deb5:~/clearwater-live-test#
>
>
>
>
>
> Is this happening because Clearwater containers are spread across multiple
> hosts? What other areas I should check?
>
>
>
>
>
> Thanks,
>
> Sarbajit
>
>
>
>
>
> On Wed, Sep 21, 2016 at 6:09 PM, Graeme Robertson (projectclearwater.org)
> <gra...@projectclearwater.org> wrote:
>
> Hi Sarbajit,
>
>
>
> I don’t think we’ve never tried deploying Project Clearwater in a Docker
> Swarm cluster, but I don’t see any reason why it couldn’t work. The tests
> are failing very early – they’re not able to connect to Ellis on port 80. I
> can think of a couple of reasons for this – either Ellis isn’t running or
> the Ellis port mapping hasn’t worked for some reason.
>
>
>
> Can you connect to the Ellis container and run ps –eaf | grep ellis and ps
> –eaf | grep nginx to confirm that NGINX and Ellis are running? Can you
> also run sudo netstat -planut | grep nginx or something equivalent to
> check that NGINX is listening on port 80? If there’s a problem with either
> NGINX or Ellis we probably need to look in the logs at /var/log/nginx/ or
> /var/log/ellis/ on the Ellis container.
>
>
>
> If however that all looks fine, then it sounds like the port mapping has
> failed for some reason. Can you run nc -z <ip> 80 from the box you’re
> running the live tests on? This will scan for anything listening at
> <ip>:80 and will return successfully if it finds anything.
>
>
>
> Thanks,
>
> Graeme
>
>
> ------------------------------
>
> *From:* Clearwater [mailto:clearwater-boun...@lists.projectclearwater.org
> <clearwater-boun...@lists.projectclearwater.org>] *On Behalf Of *Sarbajit
> Chatterjee
> *Sent:* 20 September 2016 15:05
> *To:* clearwater@lists.projectclearwater.org
> *Subject:* [Project Clearwater] Deploy Clearwater in a Swarm cluster
> using docker-compose
>
>
>
> Hello,
>
>
>
> I am following the instructions from https://github.com/
> Metaswitch/clearwater-docker. I can successfully deploy it on a single
> Docker node but, the compose file does not work with Swarm cluster.
>
>
>
> I did try to modify the compose file like this -
>
>
>
>
>
> version: '2'
>
> services:
>
>   etcd:
>
>     image: quay.io/coreos/etcd:v2.2.5
>
>     command: >
>
>       -name etcd0
>
>       -advertise-client-urls http://etcd:2379,http://etcd:4001
>
>       -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001
>
>       -initial-advertise-peer-urls http://etcd:2380
>
>       -listen-peer-urls http://0.0.0.0:2380
>
>       -initial-cluster etcd0=http://etcd:2380
>
>       -initial-cluster-state new
>
>   bono:
>
>     image: swarm-node:5000/clearwaterdocker_bono
>
>     ports:
>
>       - 22
>
>       - "3478:3478"
>
>       - "3478:3478/udp"
>
>       - "5060:5060"
>
>       - "5060:5060/udp"
>
>       - "5062:5062"
>
>   sprout:
>
>     image: swarm-node:5000/clearwaterdocker_sprout
>
>     networks:
>
>       default:
>
>         aliases:
>
>           - scscf.sprout
>
>           - icscf.sprout
>
>     ports:
>
>       - 22
>
>   homestead:
>
>     image: swarm-node:5000/clearwaterdocker_homestead
>
>     ports:
>
>       - 22
>
>   homer:
>
>     image: swarm-node:5000/clearwaterdocker_homer
>
>     ports:
>
>       - 22
>
>   ralf:
>
>     image: swarm-node:5000/clearwaterdocker_ralf
>
>     ports:
>
>       - 22
>
>   ellis:
>
>     image: swarm-node:5000/clearwaterdocker_ellis
>
>     ports:
>
>       - 22
>
>       - "80:80"
>
>
>
>
>
> where swarm-node:5000 is the local docker registry and it hosts the
> pre-built images of Clearwater containers. Even though the deployment
> succeeded, clearwater-livetests are failing with following error -
>
>
>
>
>
> Basic Registration (TCP) - Failed
>
>   Errno::ECONNREFUSED thrown:
>
>    - Connection refused - connect(2)
>
>      - 
> /usr/local/rvm/gems/ruby-1.9.3-p551/gems/quaff-0.7.3/lib/sources.rb:41:in
> `initialize'
>
>
>
>
>
> Any suggestions on how I can deploy Clearwater on a Swarm cluster?
>
>
>
> Thanks,
>
> Sarbajit
>
>
>
>
> _______________________________________________
> Clearwater mailing list
> Clearwater@lists.projectclearwater.org
> http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.
> projectclearwater.org
>
>
>
>
> _______________________________________________
> Clearwater mailing list
> Clearwater@lists.projectclearwater.org
> http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.
> projectclearwater.org
>
>
>
> _______________________________________________
> Clearwater mailing list
> Clearwater@lists.projectclearwater.org
> http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.
> projectclearwater.org
>
>
_______________________________________________
Clearwater mailing list
Clearwater@lists.projectclearwater.org
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org

Reply via email to