2020-07-14 09:33:35 UTC - VanderChen: I also have tried to change JVM settings
in file *bkenv.sh*, but still not work. Even could not start up successfully.
----
2020-07-14 10:08:32 UTC - SIMON: @SIMON has joined the channel
----
2020-07-14 11:23:59 UTC - Meyappan Ramasamy: still getting this error :
java.lang.UnsatisfiedLinkError: no
org_apache_pulsar_shade_netty_transport_native_epoll_x86_64 in java.library.path
----
2020-07-14 11:37:41 UTC - Meyappan Ramasamy: hi team, getting this error when
connecting pulsar java client to pulsar running in docker container
```java.lang.UnsatisfiedLinkError: no
org_apache_pulsar_shade_netty_transport_native_epoll_x86_64 in
java.library.path```
----
2020-07-14 13:43:00 UTC - Matt Mitchell: Will try again, but probably not until
later in the week (will get back to you w/results). Thanks for your help!
----
2020-07-14 13:43:04 UTC - puneetkhanal: @puneetkhanal has joined the channel
----
2020-07-14 16:20:02 UTC - Addison Higham: Correct, auto recovery looks at what
the quorum size of each ledger should be and replicates until it reaches that
number
----
2020-07-14 16:24:50 UTC - Alfredo Prada Giorgi: @Alfredo Prada Giorgi has
joined the channel
----
2020-07-14 16:25:40 UTC - Alfredo Prada Giorgi: :wave:
wave : Addison Higham
----
2020-07-14 18:11:03 UTC - Raghavender M: @Raghavender M has joined the channel
----
2020-07-14 18:49:55 UTC - Joshua Eric: Does Pulsar IO support Python
sinks/sources? If not, any plans to add this?
----
2020-07-14 19:02:00 UTC - Raghav: @Raghav has joined the channel
----
2020-07-14 21:15:20 UTC - Charlie Flowers: @Charlie Flowers has joined the
channel
----
2020-07-14 22:10:22 UTC - Sahil Sawhney: Team I am using KOP over pulsar and
deployed over Kubernetes. Now when I try to consume the message from the kafka
consumer, as per the doc
```bin/kafka-console-consumer.sh --bootstrap-server
[pulsar-broker-address]:9092 --topic sam --from-beginning```
It does not outputs anything
But when I try to consume these messages(produced by kafka broker) from pulsar
consumer(from toolset pod), It works fine
```bin/pulsar-client consume -s sub public/default/sam -n 0```
Note -> I have tls enabled for proxy and broker only.
----
2020-07-15 00:01:20 UTC - Hiroyuki Yamada: @Addison Higham Thank you !
So, if the failed Bookie node has 1 TB of data, it will try to recover that
much data, right ?
Does it prioritize new data ? or pick ledger randomly ?
----
2020-07-15 01:52:32 UTC - Addison Higham: I am not sure of the order it uses,
that's a good question. And while that is effectively what will happen, just to
clarify, autorecovery really doesn't care about which bookie died. It just
looks at under replicated ledgers and re-replicates them until the desired
number of replicas exist.
----
2020-07-15 01:54:19 UTC - Hiroyuki Yamada: @Addison Higham Thank you very much
for the reply.
It is getting clear now.
----
2020-07-15 05:01:58 UTC - Sijie Guo: Bookies is using the same port for both
non-tls and tls. It uses start-tls command to promote a non-tls port to tls.
@Anup Ghatage @Enrico Olivelli helped with one of similar questions before in
bookkeeper slack channel. They might be able to provide you some of the
references.
----
2020-07-15 05:02:46 UTC - Sijie Guo: No it doesn’t.
----
2020-07-15 05:03:05 UTC - Sijie Guo: I will reply to your questions in
<#C0106RFKAHM|kop> channel.
----
2020-07-15 05:10:07 UTC - Sijie Guo: this is an warning message. it will not
cause any issues. Any other logs?
----
2020-07-15 05:10:40 UTC - Sijie Guo: It seems to be at the broker side
----
2020-07-15 05:22:46 UTC - Hiroyuki Yamada: @Addison Higham It would be great if
you can teach me the priorities once you know it.
Also, I’m wondering what is going to happen when the failed bookie is back to
normal from temporary down.
In that case, there might be a case where ledger data is replicated more than
specified ?
Or the just back bookie’s ledger data is not counted as a replica by zookeeper
metadata ? (I feel the latter sounds reasonable to me but I want to make sure)
----
2020-07-15 08:12:15 UTC - VanderChen: @Sijie Guo @Penghui Li Excuse me! I have
several questions about settings for 300k - 500k Topics requirement.
Before we deploy the production, we set up a test platform on 3 servers (128Gb
memory per machine) with settings:
*bkenv.sh*:`BOOKIE_MEM=${BOOKIE_MEM:-${PULSAR_MEM:-"-Xms8g -Xmx8g
-XX:MaxDirectMemorySize=8g"}}`
*pulsar_nev.sh*:`PULSAR_MEM=${PULSAR_MEM:-"-Xms8g -Xmx8g
-XX:MaxDirectMemorySize=16g"}`
When I add our number of topics to about 30k, the client crashed (Memory usage
about 7-8Gb).
Exception message as follow:
``` org.apache.pulsar.client.api.PulsarClientException$TimeoutException: 8997
lookup request timedout after ms 30000
at
org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:821)
at
org.apache.pulsar.client.impl.ConsumerBuilderImpl.subscribe(ConsumerBuilderImpl.java:101)
at org.vander.consumer.ConsumerRunnable.run(ConsumerRunnable.java:42)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)```
But soon after, I restart the client of consumer, exception become:
```org.apache.pulsar.client.api.PulsarClientException:
java.util.concurrent.ExecutionException:
org.apache.pulsar.client.api.PulsarClientException: Connection already closed
at
org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:873)
at
org.apache.pulsar.client.impl.ConsumerBuilderImpl.subscribe(ConsumerBuilderImpl.java:101)
at org.vander.consumer.ConsumerRunnable.run(ConsumerRunnable.java:42)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)```
I don't know how to support more topics. I really need some help about settings
and cluster size.
By the way, it's there any way to re-start crashed components automatically,
such as bookie or broker.
----
