2018-05-02 16:01:22 UTC - Karthik Palanivelu: @Karthik Palanivelu uploaded a file: <https://apache-pulsar.slack.com/files/U7VRE0Q1G/FAH96AXJ9/-.txt|Untitled> and commented: Hi All, I am getting Null Pointer Exception while granting permissions to a Existing Namespace. Can you please help me on what I am missing? ---- 2018-05-02 16:01:53 UTC - Karthik Palanivelu: @Karthik Palanivelu uploaded a file: <https://apache-pulsar.slack.com/files/U7VRE0Q1G/FAGTW0M52/-.txt|Untitled> ---- 2018-05-02 16:03:29 UTC - Karthik Palanivelu: I tried role as card, card*. All received NPE. ---- 2018-05-02 16:19:11 UTC - Matteo Merli: Hi @Karthikeyan Palanivelu is this with 1.22 or current master? ---- 2018-05-02 16:22:16 UTC - Matteo Merli: Ok, looks 1.22 from code line numbers. I think the problem is that authorization is not enabled in broker ---- 2018-05-02 16:22:53 UTC - Matteo Merli: therefore the AuthorizationService is not initialized ---- 2018-05-02 16:23:01 UTC - Matteo Merli: (the error message could be better) ---- 2018-05-02 16:23:44 UTC - Matteo Merli: You should set `authorizationEnabled=true` in `conf/broker.conf` ---- 2018-05-02 16:38:26 UTC - Vasily Yanov: Hi! Is it possible to customize JMX agent configuration for metrics export? Like here: <https://github.com/prometheus/jmx_exporter/blob/master/example_configs/zookeeper.yaml> ---- 2018-05-02 16:46:08 UTC - Matteo Merli: Hi @Vasily Yanov, we generally don’t rely on JMX since it gives a lot of issues when the amount of metrics increases.
Every component in Pulsar deployment supports Prometheus format out of the box. Even in case of ZooKeeper servers, the version that ships with Pulsar can be probed on `<http://hostname:8080/metrics>` by Prometheus directly ---- 2018-05-02 16:47:32 UTC - Vasily Yanov: so no way to customize JMX output? ---- 2018-05-02 16:49:24 UTC - Matteo Merli: sure, you can always enable JMX and collect that as well. What do you mean by customize? the example above is on the Prometheus scraper config, which sure you can do as well ---- 2018-05-02 16:50:18 UTC - Matteo Merli: my point was that you don’t have to pass through JMX to get metrics into Prometheus. Then, on Prometheus side you can do all desired transformations on the metrics name, add labels and so on. ---- 2018-05-02 17:07:25 UTC - Vasily Yanov: I mean aggregate metrics values by namespace. Prometheus can do it but very slowly. It's better to expose already aggregated metrics ---- 2018-05-02 17:15:56 UTC - Karthik Palanivelu: Hi @Matteo Merli that works but it should provide a relative exception or error message and not throw null pointer exception. ---- 2018-05-02 17:16:17 UTC - Matteo Merli: Oh I see. Yes there’s a setting to control the granularity of the aggregation for the metrics. The default is to send per-topic metrics to Prometheus. If you have many topic, you might want to switch to per-namespace aggregation done by Pulsar. The setting in `broker.conf` is : ``` # Enable topic level metrics exposeTopicLevelMetricsInPrometheus=true ``` +1 : Vasily Yanov ---- 2018-05-02 17:18:02 UTC - Sijie Guo: @Karthikeyan Palanivelu good point. will improve the exception output to provide more accurate information ---- 2018-05-02 17:18:32 UTC - Karthik Palanivelu: :+1: ---- 2018-05-02 17:53:13 UTC - Karthik Palanivelu: @Matteo Merli Quick Question, I had the CN as card and assigned role as ```*card*```; It did not work for me and expects exactly as card. Can you please explain the behaviour? ---- 2018-05-02 17:56:13 UTC - Matteo Merli: I checked that in 1.22, we used the `DefaultHostnameVerifier` defined by jersey client. It’s supposed to work with wildcards but I’m not sure of the extent of it. In `pulsar-admin` I have replaced to have a flag to enable/disable hostname verification (default is false as in Java client library). The verifier we use there does work with wildcards and we have tests around that. ---- 2018-05-02 18:01:26 UTC - Karthik Palanivelu: Let me explain further, I created a namespace with role access to card with asterisks on both sides. I created client certificate with card as CN, hostname verification is disabled on pulsar+ssl route. This set up should work based on the wild card match. Based on your answer, I believe wild cards will work either before or after and not on both sides. Please confirm ---- 2018-05-02 18:05:09 UTC - Matteo Merli: Ok, the client certificate is set to, say, `card` and you want to authorize `card*` to publish on a given namespace, is that correct? ---- 2018-05-02 18:05:44 UTC - Matteo Merli: (note: the broker doesn’t do any hostname verification of the client certificate) ---- 2018-05-02 18:05:48 UTC - Karthik Palanivelu: I want to authorize namespace as `*card*` ---- 2018-05-02 18:06:15 UTC - Karthik Palanivelu: This setting failed.. ---- 2018-05-02 18:06:16 UTC - Matteo Merli: there is a setting to allow wildcards in matching the “roles” (or principals) ---- 2018-05-02 18:06:27 UTC - Karthik Palanivelu: I enabled that setting in broker ---- 2018-05-02 18:06:50 UTC - Karthik Palanivelu: authorizationAllowWildcardsMatching=true ---- 2018-05-02 18:07:31 UTC - Matteo Merli: ok ---- 2018-05-02 18:07:47 UTC - Karthik Palanivelu: I have enabled in discovery as well as in broker ---- 2018-05-02 18:08:24 UTC - Matteo Merli: then, there should be some error log around the failed authorization. does that tell anything? ---- 2018-05-02 18:08:54 UTC - Matteo Merli: and, I was not clear about this point: if you just grant permission to `card`, does that work? ---- 2018-05-02 18:10:00 UTC - Karthik Palanivelu: Yes ---- 2018-05-02 18:10:09 UTC - Karthik Palanivelu: ```WARN - [pulsar-io-55-1:PulsarWebResource@655] - [<persistent://card-tenant/a/card-namespace/my-card-topic>] Role card is not allowed to lookup topic``` ---- 2018-05-02 18:10:44 UTC - Karthik Palanivelu: This error happens if I have authorization on a topic with `*card*` ---- 2018-05-02 18:11:54 UTC - Matteo Merli: Ok, given the log above, what URL are you using to connect to brokers, `<https://host:8843>` or `<pulsar+ssl://host:6651>`? ---- 2018-05-02 18:12:10 UTC - Karthik Palanivelu: <pulsar+ssl://host:6651> ---- 2018-05-02 18:13:14 UTC - Karthik Palanivelu: When I added `card`, it worked; I can publish and consume messages to the topic. ---- 2018-05-02 18:13:35 UTC - Karthik Palanivelu: Just want to understand and capture the behaviour ---- 2018-05-02 18:16:08 UTC - Matteo Merli: I am not super-familiar with that implementation ---- 2018-05-02 18:16:38 UTC - Matteo Merli: have you tried to do either prefix or suffix: eg: `card*` ---- 2018-05-02 18:17:15 UTC - Karthik Palanivelu: Let me try that and post here ---- 2018-05-02 18:17:33 UTC - Matteo Merli: I’m seeing all tests have single `*` : <https://github.com/apache/incubator-pulsar/blob/02eff26259b43d3fac646d80f8256f9308baba1d/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/AuthorizationTest.java#L110> ---- 2018-05-02 19:09:50 UTC - Karthik Palanivelu: @Matteo Merli `*card` and `card*` authorization works for `CN=card`. This is good understanding and will call to myself open wildcard is not supported... +1 : Sijie Guo ---- 2018-05-02 21:47:47 UTC - Byron: How many proxies are recommended relative to brokers? ---- 2018-05-02 21:50:01 UTC - Matteo Merli: Proxy CPU and memory usage is very low ---- 2018-05-02 21:50:31 UTC - Matteo Merli: proxy degrades itself to a TCP proxy after the initial handshake ---- 2018-05-02 21:50:54 UTC - Matteo Merli: number of instances would be mostly related to network bandwidth ---- 2018-05-02 21:51:52 UTC - Byron: Ok thanks ----
