2018-04-05 10:00:24 UTC - Colm O'Reilly: @Colm O'Reilly has joined the channel
----
2018-04-05 22:47:02 UTC - Yuwei Jiang: Hi guys, I’m fairly new to Pulsar, I’m
playing with having it installed on a Kubernetes cluster. I followed the
documentation and have all the components running on K8s correctly, I have the
proxy running as `NodePort` and exposing port 6650 through port 30002.
I encountered an issue when running a subscriber via python client to connect
to the service url `pulsar://<K8s node ip>:30002`, from outside of K8s. I
received the following error:
```
15:12:38.146 [0x700003fd0000] ERROR
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientImpl.cc(240) - Error
Checking/Getting Partition Metadata while Subscribing- 5
```
I thought I can gain some insight on the issue by asking the question here.
Thank you!
----
2018-04-05 22:50:58 UTC - Matteo Merli: @Yuwei Jiang When connecting from
outside the K8S cluster, the problem is to have direct connectivity to the
Pulsar broker pods. There are few ways to solve that :
1. Configure broker to use host network so they will bind on an IP that is
reachable from outside K8S. This also involves using the host IP as the
`advertisedAddress` for the brokers
2. Run an instance of Pulsar proxy. The proxy is completely stateless and can
be exposed through load-balancer, NodePort, etc..
----
2018-04-05 23:00:43 UTC - Yuwei Jiang: Hi @Matteo Merli, thanks for the prompt
response! I didn’t encounter the issue yesterday, meaning I was able to create
topic and subscribe from /publish to it, with python client. I noticed the
error message and wondering if there something special about it. Is it checking
the metadata stored in zookeeper?
----
2018-04-05 23:02:30 UTC - Matteo Merli: the client doesn’t connect directly to
zookeeper, though, in absence of the Pulsar proxy, the client needs to have
direct connectivity to brokers
----
2018-04-05 23:03:00 UTC - Matteo Merli: can you share the complete logs with
the error?
----
2018-04-05 23:06:58 UTC - Yuwei Jiang: Yes, I have proxy with 3 replicas
running. Below is the output when I run my subscriber:
```
15:12:38.033 [0x7fffb4790380] INFO
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ConnectionPool.cc(63) - Created
connection for <pulsar://172.29.203.188:30002>
15:12:38.146 [0x700003fd0000] ERROR
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(323) -
[<none> -> <pulsar://172.29.203.188:30002>] Failed to establish
connection: Connection refused
15:12:38.146 [0x700003fd0000] INFO
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(1126) -
[<none> -> <pulsar://172.29.203.188:30002>] Connection closed
15:12:38.146 [0x700003fd0000] ERROR
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientImpl.cc(240) - Error
Checking/Getting Partition Metadata while Subscribing- 5
15:12:38.146 [0x700003fd0000] INFO
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(189) -
[<none> -> <pulsar://172.29.203.188:30002>] Destroyed connection
Traceback (most recent call last):
File "c8streams_consumer1.py", line 36, in <module>
consumer = client.subscribe(TOPIC, SUBSCRIPTION)
File "/usr/local/lib/python3.6/site-packages/pulsar.py", line 427, in
subscribe
c._consumer = self._client.subscribe(topic, subscription_name, conf)
Exception: Pulsar error: ConnectError
```
----
2018-04-05 23:08:27 UTC - Yuwei Jiang: Just ran again and I got the following
error:
```
16:06:27.639 [0x7fffb4790380] INFO
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ConnectionPool.cc(63) - Created
connection for <pulsar://172.29.203.188:30002>
16:06:27.665 [0x700001a85000] INFO
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(279) -
[172.29.219.211:55393 -> 172.29.203.188:30002] Connected to broker
16:06:27.693 [0x700001a85000] INFO
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/HandlerBase.cc(53) -
[<persistent://mm/global/c8/fake>, test-sub, 0] Getting connection from pool
16:06:27.719 [0x700001a85000] INFO
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ConnectionPool.cc(63) - Created
connection for <pulsar://10.243.65.70:6650>
16:06:28.738 [0x700001a85000] ERROR
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(323) -
[<none> -> <pulsar://172.29.203.188:30002>] Failed to establish
connection: Connection refused
16:06:28.738 [0x700001a85000] INFO
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(1126) -
[<none> -> <pulsar://172.29.203.188:30002>] Connection closed
16:06:28.738 [0x700001a85000] INFO
/Users/mmerli/prg/pulsar/pulsar-client-cpp/lib/ClientConnection.cc(189) -
[<none> -> <pulsar://172.29.203.188:30002>] Destroyed connection
Traceback (most recent call last):
File "c8streams_consumer1.py", line 36, in <module>
consumer = client.subscribe(TOPIC, SUBSCRIPTION)
File "/usr/local/lib/python3.6/site-packages/pulsar.py", line 427, in
subscribe
c._consumer = self._client.subscribe(topic, subscription_name, conf)
Exception: Pulsar error: ConnectError
```
----
2018-04-05 23:21:57 UTC - Sijie Guo: @Yuwei Jiang can you try “telnet
172.29.203.188 30002”?
----
2018-04-05 23:27:07 UTC - Matteo Merli: > Created connection for
<pulsar://10.243.65.70:6650>
That’s what I was mentioning, the client is directed to connect to the broker
pod, at the pod IP which is internal to K8S
----
2018-04-05 23:27:49 UTC - Matteo Merli: take a look at
<https://github.com/apache/incubator-pulsar/blob/master/deployment/kubernetes/generic/proxy.yaml>
----
2018-04-05 23:28:09 UTC - Matteo Merli: to deploy a proxy service in the K8S
cluster and use that as your endpoint
----
2018-04-05 23:28:16 UTC - Matteo Merli: for the NodePort
----
2018-04-05 23:35:23 UTC - Yuwei Jiang: Thank you guys for the direction! Looks
like something with my proxy configuration; let me look into it.
----
