A little more on this.
I have two systems, installed in an identical manner as is possible.
One works fine, on the other I can't connect to services.

For instance, from the master node I try to connect the docker-registry service on the infrastructure node. If I try:

curl -I https://<ip of docker-registry service>:5000/healthz

It works on the working environment, but gets a "No route to host" error on the failing one.

If I try:

sudo traceroute -T -p 5000 <ip of docker-registry service>

it confirms the problem. On the working environment:

$ sudo traceroute -T -p 5000 172.30.145.23
traceroute to 172.30.145.23 (172.30.145.23), 30 hops max, 60 byte packets
 1  docker-registry.default.svc.cluster.local (172.30.145.23)  3.044 ms  2.723 ms  2.307 ms

On the failing one:

$ sudo traceroute -T -p 5000 172.30.76.145
traceroute to 172.30.76.145 (172.30.76.145), 30 hops max, 60 byte packets
 1  docker-registry.default.svc.cluster.local (172.30.76.145) 3004.572 ms !H  3004.517 ms !H  3004.502 ms !H

The !H means the host is unreachable.
If I run the same commands from the infrastructure node where the service is actually running then it works OK.

The security group for both servers leaves all TCP traffic open. e.g.

ALLOW IPv4 1-65535/tcp to 0.0.0.0/0
ALLOW IPv4 1-65535/tcp from 0.0.0.0/0

Any thoughts on what is blocking the traffic?

Tim



On 27/03/18 21:54, Tim Dudgeon wrote:

Sorry, I am using port 5000. I wrote that bit incorrectly.
I did do some more digging based on what's here (https://docs.openshift.org/latest/admin_guide/sdn_troubleshooting.html) and it looks like there's something wrong with the node to node communications.
From the master I try to contact the infrastructure node:

$ ping 192.168.253.126
PING 192.168.253.126 (192.168.253.126) 56(84) bytes of data.
64 bytes from 192.168.253.126: icmp_seq=1 ttl=64 time=0.657 ms
64 bytes from 192.168.253.126: icmp_seq=2 ttl=64 time=0.588 ms
64 bytes from 192.168.253.126: icmp_seq=3 ttl=64 time=0.605 ms
^C
--- 192.168.253.126 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.588/0.616/0.657/0.041 ms

$ tracepath 192.168.253.126
 1?: [LOCALHOST]                                         pmtu 1450
 1:  no reply
 2:  no reply
 3:  no reply
 4:  no reply
^C

I can ping the node but treacepath can't reach it. On a working claster tracepath has no problems.

I don't know the cause. Any ideas?


On 27/03/18 21:46, Louis Santillan wrote:
Isn't the default port for your Registry 5000? Try `curl -kv https://docker-registry.default.svc:5000/healthz` <https://docker-registry.default.svc:5000/> [0][1].

[0] https://access.redhat.com/solutions/1616953#health
[1] https://docs.openshift.com/container-platform/3.7/install_config/registry/accessing_registry.html#accessing-registry-metrics

___________________________________________________

LOUIS P.SANTILLAN

Architect, OPENSHIFT, MIDDLEWARE & DEVOPS

Red Hat Consulting, <https://www.redhat.com/> Container and PaaS Practice

[email protected] <mailto:[email protected]>  M: 3236334854 <tel:3236334854>

<https://red.ht/sig>      
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>




On Tue, Mar 27, 2018 at 6:39 AM, Tim Dudgeon <[email protected] <mailto:[email protected]>> wrote:

    Something strange has happened in my environment which has
    resulted in not being able to route to any of the services.
    Earlier this was all working fine. The install was done using the
    ansible installer and this is happening with 3.6.1 and 3.7.1.
    The services are all there are running fine, and DNS is working,
    but I can't reach them. e.g. from the master node:

    $ host docker-registry.default.svc
    docker-registry.default.svc.cl
    <http://docker-registry.default.svc.cl>uster.local has address
    172.30.243.173
    $ curl -k https://docker-registry.default.svc/healthz
    <https://docker-registry.default.svc/healthz>
    curl: (7) Failed connect to docker-registry.default.svc:443; No
    route to host

    Any ideas on how to work out what's gone wrong?


    _______________________________________________
    users mailing list
    [email protected]
    <mailto:[email protected]>
    http://lists.openshift.redhat.com/openshiftmm/listinfo/users
    <http://lists.openshift.redhat.com/openshiftmm/listinfo/users>




_______________________________________________
users mailing list
[email protected]
http://lists.openshift.redhat.com/openshiftmm/listinfo/users

Reply via email to