2019-11-15 12:23:02 UTC - Igor Zubchenok: up! @Sijie Guo could you please reply?
----
2019-11-15 12:24:37 UTC - Igor Zubchenok: up!
----
2019-11-15 14:41:08 UTC - Kabeer Ahmed: Thank you for the response @Sijie Guo.
I will have a look at it.
----
2019-11-15 15:31:56 UTC - Ryan Samo: Hey guys, is there a maximum cap on the
amount of connections a Pulsar broker can handle at the software level? I see
the proxy has a maxConcurrentInboundConnections parameter but just wondering if
the broker itself has any limits.
----
2019-11-15 15:32:35 UTC - Ryan Samo: Thinking about this from a Spark
perspective and how it can bombard your servers with parallel connections
----
2019-11-15 16:08:40 UTC - tihomir: Hi, my team is using pulsar in standalone
mode, with 1 broker, 1 zoo and 1 bookie, but when the broker crash it is unable
to start properly again because we are getting the following error: Caused by:
org.apache.pulsar.broker.PulsarServerException: Broker-znode owned by different
zk-session 72076037952439175
at
org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl.start(ModularLoadManagerImpl.java:798)
~[org.apache.pulsar-pulsar-broker-2.3.2.jar:2.3.2]
We are suspecting that this is caused because when the broker goes down we
don't have any active brokers left then when the broker starts again it has new
session id from Zookeeper and this check fails
if (ownerZkSessionId != 0 && ownerZkSessionId !=
zkClient.getSessionId())
and raises the exception.
We are using ModularLoadManager but in the doc it says that it is working with
1 or more brokers so in general, it shouldn't be a problem when we have a setup
of 1 broker and when it fails it should be able to start again.
Could you please advise what should we do to be able to get the broker working
even after restart
----
2019-11-15 16:15:04 UTC - Addison Higham: I *think* once the zk-session times
out, it should automatically fix itself...
----
2019-11-15 16:36:08 UTC - tihomir: do you know if there is somehow to handle
this before session expires because that means that there wouldn't be any
brokers up until session expires
----
2019-11-15 17:16:32 UTC - Sunil Sattiraju: @Sunil Sattiraju has joined the
channel
----
2019-11-15 17:30:27 UTC - Sunil Sattiraju: Hi All, I am planning to port an
existing Kafka Streams application into Pulsar
Below is what the Kafka Streams logic does
1. Add key to the messages from Original topics ( initially no key )
2. Group values for the same key and append values into a list
3. Process the list and send output to a new topic
How should I approach this in Pulsar using Python API, should I be using Pulsar
functions and submit to cluster ? or should I consume and publish by running
multiple instances of the the python script.
Performance is key, kafka streams scala api is used and currently runs on 5
nodes with 6 threads to process data
----
2019-11-15 17:42:42 UTC - David Kjerrumgaard: @Sunil Sattiraju Might I suggest
the following approach; for step 1 you could use a simple Pulsar function
provided that the logic is fairly straight-forward and does NOT require
interaction with an external system for look-ups, i.e. the key can be
determined by the contents of the message itself. Make sure that these keyed
messages are all published to the same output topic. For steps 2 & 3, you
should leverage Pulsars key-shared subscription type to ensure that messages
with the same key are routed by the same consumer. You can use whatever type of
consumer you like, Python, Java, etc for the grouping.
----
2019-11-15 17:44:44 UTC - David Kjerrumgaard: HTH
----
2019-11-15 17:44:47 UTC - Matteo Merli: @tihomir In standalone mode it would be
using a "NoopLoadManageer" since there cannot be more brokers anyway.
Is the process running on a laptop (that sleeps and comes back) ?
----
2019-11-15 20:13:33 UTC - tihomir: no it is on a machine in Digital Ocean but
they had networking issues recently and the broker pod had to restart ~ 150
times after the issue was resolved before it starts processing messages again
----
2019-11-15 20:15:52 UTC - tihomir: and it was using the default load manader
ModularLoadManager. Also it had loadBalancer enabled because that is the
default. I have disabled that since there isn't much sense in having
loadBalancer enabled in standalone setup
----
2019-11-15 23:02:55 UTC - Matt Cohen: @Matt Cohen has joined the channel
----
2019-11-16 02:58:46 UTC - Nicolas Ha: Are tenants supposed to be automatically
created when a subscription is made?
I see only
```
[
"public",
"sample"
]
```
in `/admin/v2/tenants` - but I am using topics like
`<persistent://my-tenant/standalone/my-channel/some-id>`
I never created a tenant explicitly, but I was expecting to see `my-tenant`
in the endpoint above
----
