2019-05-27 09:31:23 UTC - divyasree: Hi..
----
2019-05-27 09:32:05 UTC - divyasree: I am working with venu on Pulsar POC..
Thanks for the guidelines for geo replication. I worked fine now.
----
2019-05-27 09:33:10 UTC - divyasree: We are trying to enabling authentation
using this link..<https://pulsar.apache.org/docs/en/security-token-admin/>
+1 : jia zhai
----
2019-05-27 09:33:28 UTC - Sijie Guo: great to hear that it worked out well!
feel free to ping me if you need more helps.
----
2019-05-27 09:35:03 UTC - divyasree: after changing the configuration in
broker.conf as below authenticationEnabled=true
authorizationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken
----
2019-05-27 09:35:29 UTC - divyasree: tokenSecretKey=<secret key path>
----
2019-05-27 09:35:51 UTC - divyasree: broker is not getting started..
----
2019-05-27 09:36:18 UTC - Sijie Guo: Did you see any errors in the log?
----
2019-05-27 09:36:27 UTC - divyasree: Can you help us with details procedure to
enable authentication and authorization in apache pulsar?
----
2019-05-27 09:36:57 UTC - Shivji Kumar Jha: It seems there is an environment
issue with pulsar builds on jenkins.
----
2019-05-27 09:37:51 UTC - Sijie Guo: @divyasree sure. did you see any errors in
the log?
----
2019-05-27 09:37:59 UTC - divyasree: ya
----
2019-05-27 09:38:00 UTC - divyasree: 09:29:44.398 [pulsar-io-23-2] WARN
org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:40928] Unable to
authenticate: Unsupported authentication mode: none
09:29:44.398 [pulsar-io-23-2] INFO org.apache.pulsar.broker.service.ServerCnx
- Closed connection from /127.0.0.1:40928
09:29:44.398 [pulsar-client-io-41-1] WARN
org.apache.pulsar.client.impl.ClientCnx - [127.0.0.1/127.0.0.1:6650] Got
exception IllegalArgumentException : null
java.lang.IllegalArgumentException: null
at
com.google.common.base.Preconditions.checkArgument(Preconditions.java:108)
~[com.google.guava-guava-21.0.jar:?]
at
org.apache.pulsar.client.impl.ClientCnx.handleError(ClientCnx.java:527)
~[org.apache.pulsar-pulsar-client-original-2.3.1.jar:2.3.1]
at
org.apache.pulsar.common.api.PulsarDecoder.channelRead(PulsarDecoder.java:157)
~[org.apache.pulsar-pulsar-common-2.3.1.jar:2.3.1]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
[io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
[io.netty-netty-all-4.1.32.Final.jar:4.1.32.Final]
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
[io.netty-netty-all-4.1.32.Final.ja
----
2019-05-27 09:38:30 UTC - Shivji Kumar Jha: I can see consecutive builds
failing with the same error. here is one such error
----
2019-05-27 09:43:56 UTC - Sijie Guo: @divyasree did you enable functions worker?
----
2019-05-27 09:49:47 UTC - divyasree: yes functionsWorkerEnabled=true
----
2019-05-27 09:50:19 UTC - Sijie Guo: and also geo-replication enabled?
----
2019-05-27 09:56:02 UTC - divyasree: i dont see any field named
"geo-replication" in broker.conf
----
2019-05-27 09:56:18 UTC - Sijie Guo: if you have geo-replication enabled, you
need configure following settings in conf/broker.conf
```
brokerClientTlsEnabled=false
brokerClientAuthenticationPlugin=
brokerClientAuthenticationParameters=
brokerClientTrustCertsFilePath=
```
these settings are for brokers to connect to other brokers. since you enabled
authentication, you need to configure them.
----
2019-05-27 09:56:52 UTC - Sijie Guo: > i dont see any field named
“geo-replication” in broker.conf
you setup multiple clusters with geo-replication, right?
----
2019-05-27 09:57:09 UTC - divyasree: ok.. Is that mean, tls need to be enabled?
----
2019-05-27 10:00:33 UTC - Sijie Guo: no, if your broker doesn’t enable tls, you
don’t need to.
----
2019-05-27 10:01:20 UTC - Sijie Guo: but if you are doing geo replication, you
are recommended to enable TLS to encrypt the channels between clusters.
----
2019-05-27 10:01:58 UTC - Sijie Guo: @Shivji Kumar Jha I also see this
exception as well.
----
2019-05-27 10:03:12 UTC - divyasree: ok.. Even if tls is disabled,
authentication should work right?
----
2019-05-27 10:03:34 UTC - divyasree: Any configuration, that we are missing to
get authentication enabled?
----
2019-05-27 10:05:02 UTC - Sijie Guo: > Even if tls is disabled,
authentication should work right?
correct.
> Any configuration, that we are missing to get authentication enabled?
You are missing the following settings. since you clusters are doing
geo-replication.
```
brokerClientTlsEnabled=false
brokerClientAuthenticationPlugin=
brokerClientAuthenticationParameters=
brokerClientTrustCertsFilePath=
```
----
2019-05-27 10:14:33 UTC - Shivji Kumar Jha: Hi, MultiVersionSchemaInfoProvider
maps to 1 topic and all the schemas on that topic are cached. It works well
with a consumer that consumers from one topic.
But for a consumer that consumes on List<topic> or regex topic , i dont
see a way to use a MultiVersionSchemaInfoProvider cache. I am looking for a
cache something on the lines of map<topic,
MultiVersionSchemaInfoProvider>. Any suggestions there?
----
2019-05-27 10:14:42 UTC - Shivji Kumar Jha: @Penghui Li @Sijie Guo ^
----
2019-05-27 10:23:06 UTC - Sijie Guo: > But for a consumer that consumes on
List<topic> or regex topic , i dont see a way to use a
MultiVersionSchemaInfoProvider cache.
when you are using `List<topc>` or a regex, you will have a map from
topic to consumer, and each consumer wll have a MultiVersionSchemaInfoProvider
cache, no?
----
2019-05-27 10:27:09 UTC - Sijie Guo: /cc @CongBo in this thread, since he was
the main author of this piece of code.
----
2019-05-27 12:25:06 UTC - Shivji Kumar Jha: @Sijie Guo
```map from topic to consumer```
Isnt this the other way? Consumer is composed of List<topic>
Also,
```consumer wll have a MultiVersionSchemaInfoProvider cache```
No
In fact now I see this
```
private final LoadingCache<String, SchemaInfoProvider>
schemaProviderLoadingCache = CacheBuilder.newBuilder().maximumSize(100000)
.expireAfterAccess(30, TimeUnit.MINUTES).build(new
CacheLoader<String, SchemaInfoProvider>() {
@Override
public SchemaInfoProvider load(String topicName) {
return newSchemaProvider(topicName);
}
});
```
But then getter to this is private so i dont see a way to get this cache
either.. Shouldnt this getter be public.
----
2019-05-27 12:27:47 UTC - Sijie Guo: @Shivji Kumar Jha ah sorry. the cache is
maintained per client. it is a map from topic to the schema loader. this would
work for list of topics and regex, no?
----
2019-05-27 12:27:58 UTC - Sijie Guo: I am a bit confused about the original
question now.
----
2019-05-27 12:38:07 UTC - Martijn Dekkers: @Martijn Dekkers has joined the
channel
----
2019-05-27 13:18:47 UTC - Thomas: @Thomas has joined the channel
----
2019-05-27 13:24:37 UTC - Thomas: Hello, I am developing a websocket Node.JS
connector for Pulsar.
It is working fine, however there is one thing that block us:
After 1000 nacked messages, no messages are sent anymore.
I saw in the doc that it is the receiverQueueSize:
`By default (pullMode=false), the consumer endpoint will use the
receiverQueueSize parameter both to size its internal receive queue and to
limit the number of unacknowledged messages that are passed to the WebSocket
client. In this mode, if you don't send acknowledgements, the Pulsar WebSocket
service will stop sending messages after reaching receiverQueueSize unacked
messages sent to the WebSocket client`
The nack-ing of message is part of our workflow and we expect to retry all
messages until eventually they get ACKed. But as soon as it reaches 1000
messages, it just comes to a halt.
How to work around it?
----
2019-05-27 15:22:31 UTC - mathieu longtin: @mathieu longtin has joined the
channel
----
2019-05-28 05:53:27 UTC - divyasree: i have followed this link
<https://pulsar.apache.org/docs/en/security-tls-transport/> to enabled tls in
broker node.
----
2019-05-28 05:54:06 UTC - divyasree: and changed the above mentioned
configuration in broker.conf
----
2019-05-28 05:54:26 UTC - divyasree: i am getting the below error when starting
the broker
----
2019-05-28 05:54:26 UTC - divyasree: 05:43:50.300 [main] ERROR
org.apache.pulsar.PulsarBrokerStarter - Failed to start pulsar service.
org.apache.pulsar.broker.PulsarServerException: javax.net.ssl.SSLException:
failed to set cipher suite: [TLS_DH_RSA_WITH_AES_256_GCM_SHA384,
TLS_DH_RSA_WITH_AES_256_CBC_SHA]
----
2019-05-28 06:18:34 UTC - Prasad Ghangal: @Prasad Ghangal has joined the channel
----
2019-05-28 06:22:05 UTC - dba: Just to let you all know, at Danske Commodities
we are working on a client for C#/.NET. It will be written entirely in C# and
released under the Apache License v2.0 at around the same time as .NET Core
3.0, which is in late September.
+1 : Sijie Guo
100 : Sijie Guo
----
2019-05-28 06:22:32 UTC - Ali Ahmed: @dba that’s great to hear
+1 : Ali Ahmed
----
2019-05-28 06:24:45 UTC - Ali Ahmed: @Thomas not sure about the issue but you
can try the native nodejs client <https://github.com/apache/pulsar-client-node>
----
2019-05-28 06:24:50 UTC - Prasad Ghangal: Hi folks,
Not sure if this is the correct channel to post. I just wanted to know, is
there any specific reason behind setting updateStrategy type to `OnDelete`
instead of `RollingUpdate` for Zookeeper and Bookkeeper in helm charts. Can I
change it to `RollingUpdate`? Are there any risks?
(<https://github.com/apache/pulsar/blob/master/deployment/kubernetes/helm/pulsar/values.yaml#L61>)
```
updateStrategy:
type: OnDelete
```
----
2019-05-28 06:28:45 UTC - Ali Ahmed: @Prasad Ghangal Rolling updates would not
be recommended, pulsar is generally core infra that’s needs constant uptime,
manual control for zk and bk will be better to make sure upgrades are stopped
in something is going wrong , if it’s a test env rolling updates should be fine.
----
2019-05-28 06:37:25 UTC - Prasad Ghangal: Thanks @Ali Ahmed. So what is the
recommended way to upgrade bk/zk in production?
----
2019-05-28 06:40:43 UTC - Ali Ahmed: zk guides are available online
bk guides are dependent on the version
<https://bookkeeper.apache.org/docs/4.7.2/admin/upgrade/#upgrade-guides>
----
2019-05-28 06:41:52 UTC - Prasad Ghangal: Thanks @Ali Ahmed, will go through it
----
2019-05-28 08:24:31 UTC - Sijie Guo: @divyasree: `A list of supported cipher
can be acquired from the openssl ciphers command, i.e. openssl ciphers -tls_v2.`
----
2019-05-28 08:24:56 UTC - Sijie Guo: can you run `openssl ciphers -tls_v2` to
list to ciphers that your machine supports
----
2019-05-28 08:25:12 UTC - Sijie Guo: if you are not sure about it, you can let
that field to be empty
----
2019-05-28 09:06:57 UTC - divyasree: i am getting this error Error in cipher
list
140243610089360:error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher
match:ssl_lib.c:1383:
----
2019-05-28 09:07:17 UTC - divyasree: so shall i give it blank?
----
2019-05-28 09:07:33 UTC - Sijie Guo: I would suggest just giving it blank
----
