We're you testing weave or running g it for real?
On Jul 4, 2017 7:28 AM, "Itamar O" <itamar...@gmail.com> wrote:
> solved it.
> I found a collision in the routing table of the 10.240.0.2 host...
> # route
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use
> Iface
> default 10.240.0.1 0.0.0.0 UG 0 0 0
> eth0
> *10.32.0.0 * 255.240.0.0 U 0 0 0
> weave*
> 10.240.0.1 * 255.255.255.255 UH 0 0 0
> eth0
> 172.17.0.0 * 255.255.0.0 U 0 0 0
> docker0
> 172.18.0.0 * 255.255.0.0 U 0 0 0
> docker_gwbridge
>
> thanks.
>
> On Mon, Jul 3, 2017 at 10:45 AM Itamar O <itamar...@gmail.com> wrote:
>
>> here's a slightly redacted snippet of my GCP routes:
>>
>> > gcloud compute routes list
>> NAME NETWORK
>> DEST_RANGE NEXT_HOP
>> PRIORITY
>> default-route-2aab1c1d780bf8ed datalab 10.240.0.0/24 1000
>> default-route-91098cb127e5377b default 10.240.0.0/16 1000
>> gke-foo1 default 10.44.4.0/24 us-central1-f/instances/gke-foo-pool1-a
>> 1000
>> gke-foo2 default 10.44.7.0/24 us-central1-f/instances/gke-foo-pool1-b
>> 1000
>> gke-foo3 default 10.44.5.0/24 us-central1-f/instances/gke-
>> foo-pool1-c 1000
>> gke-foo4 default 10.44.2.0/24 us-central1-f/instances/gke-
>> foo-pool1-d 1000
>>
>> I don't see overlaps in the 10.44.* space, but there's something with
>> 10.240.*.
>> Removing the "datalab" route didn't change anything,
>> and I'm also not sure how it would explain the inconsistent behavior
>> between 10.240.0.2 & 10.240.0.35.
>>
>> On Sun, Jul 2, 2017 at 7:21 PM 'Tim Hockin' via Kubernetes user
>> discussion and Q&A <kubernetes-users@googlegroups.com> wrote:
>>
>>> Check for duplicate or overlapping routes in the cloud console?
>>>
>>> On Jul 2, 2017 9:14 AM, "Itamar O" <itamar...@gmail.com> wrote:
>>>
>>>> Hi,
>>>> I'm investigating a weird routing behavior on our production GKE
>>>> cluster (nodes & master running 1.6.6), not quite sure how to proceed at
>>>> this point.
>>>>
>>>> The essence of the issue: containers on the GKE cluster can communicate
>>>> with *some* GCE instances, but *not others*, and there's no (apparent)
>>>> difference between the accessible and inaccessible GCE instances.
>>>>
>>>> More details:
>>>> All GCE instances and GKE clusters co-exist happily within 10.240.*.*
>>>> Pods on the GKE cluster are assigned addresses in the 10.44.*.* space.
>>>> I noticed one of the services on GKE misbehaving, and the logs
>>>> indicated that it is timing out on accessing a service that we have serving
>>>> from a GCE instance (at 10.240.0.2 for that matter).
>>>> I `kubectl exec` into a bash session in that pod, and indeed I saw that
>>>> `ping 10.240.0.2` is timing out, but `ping 10.240.0.35` is working just
>>>> fine (10.240.0.35 being another GCE machine, being 100% equivalent to
>>>> 10.240.0.2 AFAIK).
>>>> We also have another, older, GKE cluster up ("previous prod"), from
>>>> which I tried the same pings with no such issues. The differences I can map
>>>> include the previous cluster running k8s 1.6.2, and pods assigned under
>>>> 10.88.*.*.
>>>> I could not notice any meaningful related rule in GCE Routing or FW
>>>> (they seem to be equivalent between the old and new GKE clusters).
>>>>
>>>> Any ideas on how to proceed with investigating this?
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Kubernetes user discussion and Q&A" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to kubernetes-users+unsubscr...@googlegroups.com.
>>>> To post to this group, send email to kubernetes-users@googlegroups.com.
>>>> Visit this group at https://groups.google.com/group/kubernetes-users.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Kubernetes user discussion and Q&A" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to kubernetes-users+unsubscr...@googlegroups.com.
>>> To post to this group, send email to kubernetes-users@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/kubernetes-users.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q&A" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.
