Yes, I tried that and get the same problem. Pinging the fully qualified name works fine (so I know it is the correct long name).
On 7 April 2017 at 13:22, Jan Tattermusch <[email protected]> wrote: > Out of curiosity, have you tried with the long DNS name of the service? > (I think that's something in the sense of myservice.default.svc. > cluster.local) > > On Wednesday, March 15, 2017 at 6:37:30 PM UTC+1, [email protected] wrote: >> >> Hi, >> This is about the interplay of gRPC and Kubernetes, but I'm pretty sure >> the problem is with the gRPC name resolution. >> I have a gRPC application running on a kubernetes cluster exposed using a >> headless service. For those that don't know, this means (as I understand >> it) kubernetes sets up a DNS entry pointing to the application container. >> If I then try and run the client in the same cluster it hangs during name >> resolution to the kubernetes service. Connecting directly to the IP >> address of the service works fine. >> E.g. with the server already running and listening on port 50051 >> >> kubectl run -it --rm --image=myimage --env="GRPC_TRACE=all" >> --env="GRPC_VERBOSITY=DEBUG" testclient --command /bin/sh >> >> > ping myservice >> PING myservice (10.0.0.13): 56 data bytes >> 64 bytes from 10.0.0.13: seq=0 ttl=64 time=0.121 ms >> (etc; service name resolves fine) >> >> > myapp -c 10.0.0.13:50051 # connect as client to the given server and >> port using the ip address resolved by ping >> (loads of debug output, but I get the expected response and everything >> works fine) >> >> > myapp -c myservice:50051 >> (hangs indefinitely trying to resolve the name, full output below) >> >> The fact it hangs indefinitely appears to be related to >> https://github.com/grpc/grpc/issues/9481 (DNS resolver in C core never >> gives up resolving a nonexistent hostname), except in my case the hostname >> is valid, as shown by the ping. The container image is busybox plus the >> C++ application and required libraries. Is there anything gRPC needs in >> the OS (i.e. my container image) in order to perform name resolution? Ping >> manages it fine, why not gRPC? Forgive my ignorance of how the underlying >> name resolution is provided. >> >> Thanks, >> >> Mark. >> >> Output when trying to resolve to the service: >> >> I0315 17:08:46.596502176 15 ev_epoll_linux.c:85] epoll engine >> will be using signal: 36 >> D0315 17:08:46.596889365 15 ev_posix.c:106] Using >> polling engine: epoll >> I0315 17:08:46.597180247 15 init.c:193] >> grpc_init(void) >> I0315 17:08:46.597458188 15 completion_queue.c:139] >> grpc_completion_queue_create(reserved=(nil)) >> I0315 17:08:46.597765727 14 init.c:193] >> grpc_init(void) >> I0315 17:08:46.598071841 14 channel_create.c:235] >> grpc_insecure_channel_create(target=0xaf2e48, args=0x7ffd594b2fd0, >> reserved=(nil)) >> I0315 17:08:46.598358396 14 init.c:193] >> grpc_init(void) >> I0315 17:08:46.598675472 14 init.c:198] >> grpc_shutdown(void) >> I0315 17:08:46.598951207 14 channel.c:263] >> grpc_channel_register_call(channel=0xaeba20, >> method=/grpcendpoint.Endpoint/endpointVersion, host=(null), >> reserved=(nil)) >> I0315 17:08:46.599216112 14 channel.c:263] >> grpc_channel_register_call(channel=0xaeba20, >> method=/grpcendpoint.Endpoint/listCollections, host=(null), >> reserved=(nil)) >> I0315 17:08:46.599476727 14 channel.c:263] >> grpc_channel_register_call(channel=0xaeba20, >> method=/grpcendpoint.Endpoint/getCollection, host=(null), reserved=(nil)) >> I0315 17:08:46.599896046 14 channel.c:291] >> grpc_channel_create_registered_call(channel=0xaeba20, >> parent_call=(nil), propagation_mask=ffff, completion_queue=0x7fa7c8001b40, >> registered_call_handle=0xaf1910, deadline=gpr_timespec { tv_sec: >> 9223372036854775807, tv_nsec: 0, clock_type: 1 }, reserved=(nil)) >> I0315 17:08:46.600314315 14 grpc_context.c:41] >> grpc_census_call_set_context(call=0xaf4ec0, census_context=(nil)) >> I0315 17:08:46.600804961 14 call.c:1690] >> grpc_call_start_batch(call=0xaf4ec0, ops=0x7ffd594b2cf0, nops=3, >> tag=0xaf0318, reserved=(nil)) >> I0315 17:08:46.601111821 14 call.c:1366] ops[0]: >> SEND_INITIAL_METADATA >> I0315 17:08:46.601392365 14 call.c:1366] ops[1]: >> SEND_MESSAGE ptr=0xaebc90 >> I0315 17:08:46.601653874 14 call.c:1366] ops[2]: >> SEND_CLOSE_FROM_CLIENT >> I0315 17:08:46.602006491 14 client_channel.c:109] >> OP[client-channel:0xaf54c8]: SEND_INITIAL_METADATA{key=3a 70 61 74 68 >> ':path' value=2f 67 72 70 63 65 6e 64 70 6f 69 6e 74 2e 45 6e 64 70 6f 69 >> 6e 74 2f 65 6e 64 70 6f 69 6e 74 56 65 72 73 69 6f 6e >> '/grpcendpoint.Endpoint/endpointVersion', key=3a 61 75 74 68 6f 72 69 74 >> 79 ':authority' value=74 65 6c 65 68 69 73 74 3a 35 30 30 35 31 >> 'myservice:50051', key=67 72 70 63 2d 65 6e 63 6f 64 69 6e 67 >> 'grpc-encoding' value=69 64 65 6e 74 69 74 79 'identity', key=67 72 70 63 >> 2d 61 63 63 65 70 74 2d 65 6e 63 6f 64 69 6e 67 'grpc-accept-encoding' >> value=69 64 65 6e 74 69 74 79 2c 64 65 66 6c 61 74 65 2c 67 7a 69 70 >> 'identity,deflate,gzip'} SEND_MESSAGE:flags=0x00000000:len=0 >> SEND_TRAILING_METADATA{} >> I0315 17:08:46.602499932 14 call.c:1690] >> grpc_call_start_batch(call=0xaf4ec0, ops=0x7ffd594b2d30, nops=3, >> tag=0xaf03b8, reserved=(nil)) >> I0315 17:08:46.602729139 14 call.c:1366] ops[0]: >> RECV_INITIAL_METADATA ptr=0xaf03e0 >> I0315 17:08:46.603054019 14 call.c:1366] ops[1]: >> RECV_MESSAGE ptr=0xaf0408 >> I0315 17:08:46.603314856 14 call.c:1366] ops[2]: >> RECV_STATUS_ON_CLIENT metadata=0xaf0428 status=0xaf0440 details=0xaf0448 >> I0315 17:08:46.603639848 14 client_channel.c:109] >> OP[client-channel:0xaf54c8]: RECV_INITIAL_METADATA RECV_MESSAGE >> RECV_TRAILING_METADATA >> I0315 17:08:46.603924761 15 completion_queue.c:333] >> grpc_completion_queue_next(cc=0x7fa7c8001b40, deadline=gpr_timespec { >> tv_sec: 9223372036854775807, tv_nsec: 0, clock_type: 1 }, reserved=(nil)) >> D0315 17:08:46.604521071 16 dns_resolver.c:192] dns >> resolution failed: {"created":"@1489597726.604500252","description":"OS >> Error","errno":-2,"file":"src/core/lib/iomgr/resolve_address >> _posix.c","file_line":115,"os_error":"Name or service not >> known","syscall":"getaddrinfo","target_address":"myservice:50051"} >> D0315 17:08:46.604820308 16 dns_resolver.c:201] retrying >> immediately >> D0315 17:08:46.605088317 16 connectivity_state.c:156] SET: >> 0xaebb68 client_channel: IDLE --> TRANSIENT_FAILURE [new_lb+resolver] >> error=0x7fa7c0000b90 {"created":"@1489597726.605077608","description":"No >> load balancing policy","file":"src/core/ext/c >> lient_config/client_channel.c","file_line":188} >> D0315 17:08:47.605552537 17 dns_resolver.c:192] dns >> resolution failed: {"created":"@1489597727.605530416","description":"OS >> Error","errno":-2,"file":"src/core/lib/iomgr/resolve_address >> _posix.c","file_line":115,"os_error":"Name or service not >> known","syscall":"getaddrinfo","target_address":"myservice:50051"} >> D0315 17:08:47.606272692 17 dns_resolver.c:201] retrying >> immediately >> D0315 17:08:47.606719769 17 connectivity_state.c:156] SET: >> 0xaebb68 client_channel: TRANSIENT_FAILURE --> TRANSIENT_FAILURE >> [new_lb+resolver] error=0x7fa7c0000bf0 >> {"created":"@1489597727.606709040","description":"No >> load balancing policy","file":"src/core/ext/c >> lient_config/client_channel.c","file_line":188} >> D0315 17:08:49.501768132 18 dns_resolver.c:192] dns >> resolution failed: {"created":"@1489597729.501745552","description":"OS >> Error","errno":-2,"file":"src/core/lib/iomgr/resolve_address >> _posix.c","file_line":115,"os_error":"Name or service not >> known","syscall":"getaddrinfo","target_address":"myservice:50051"} >> D0315 17:08:49.502278763 18 dns_resolver.c:201] retrying >> immediately >> D0315 17:08:49.502683440 18 connectivity_state.c:156] SET: >> 0xaebb68 client_channel: TRANSIENT_FAILURE --> TRANSIENT_FAILURE >> [new_lb+resolver] error=0x7fa7c0000b90 >> {"created":"@1489597729.502672595","description":"No >> load balancing policy","file":"src/core/ext/c >> lient_config/client_channel.c","file_line":188} >> D0315 17:08:52.737487943 19 dns_resolver.c:192] dns >> resolution failed: {"created":"@1489597732.737457632","description":"OS >> Error","errno":-2,"file":"src/core/lib/iomgr/resolve_address >> _posix.c","file_line":115,"os_error":"Name or service not >> known","syscall":"getaddrinfo","target_address":"myservice:50051"} >> D0315 17:08:52.737695112 19 dns_resolver.c:201] retrying >> immediately >> D0315 17:08:52.737751585 19 connectivity_state.c:156] SET: >> 0xaebb68 client_channel: TRANSIENT_FAILURE --> TRANSIENT_FAILURE >> [new_lb+resolver] error=0x7fa7c0000bf0 >> {"created":"@1489597732.737742549","description":"No >> load balancing policy","file":"src/core/ext/c >> lient_config/client_channel.c","file_line":188} >> D0315 17:08:56.890396846 20 dns_resolver.c:192] dns >> resolution failed: {"created":"@1489597736.890373998","description":"OS >> Error","errno":-2,"file":"src/core/lib/iomgr/resolve_address >> _posix.c","file_line":115,"os_error":"Name or service not >> known","syscall":"getaddrinfo","target_address":"myservice:50051"} >> D0315 17:08:56.890592615 20 dns_resolver.c:201] retrying >> immediately >> D0315 17:08:56.890664375 20 connectivity_state.c:156] SET: >> 0xaebb68 client_channel: TRANSIENT_FAILURE --> TRANSIENT_FAILURE >> [new_lb+resolver] error=0x7fa7c0000b90 >> {"created":"@1489597736.890640201","description":"No >> load balancing policy","file":"src/core/ext/c >> lient_config/client_channel.c","file_line":188} >> (etc in a seemingly infinite loop) >> > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAHq3_RnD%2BzvMeqx7g97BfTNhZ%2Bbygkb0bo09jQbR-nEBr5QDmQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
