2019-03-22 11:18:41 UTC - bossbaby: In this issue: <https://github.com/apache/pulsar/issues/3882> Has anyone been able to fix it? I really need it ---- 2019-03-22 13:03:09 UTC - Laurent Chriqui: HI @Jerry Peng, I get this error when trying to use the compiled library... File “/Users/laurentchriqui/Documents/data-stream-router/pulsar-functions-bypass/src/pulsar_producer_test.py”, line 1, in <module> import pulsar File “/Users/laurentchriqui/Documents/data-stream-router/.venv/lib/python3.7/site-packages/pulsar_client-2.4.0-py3.7-macosx-10.9-x86_64.egg/pulsar/__init__.py”, line 273, in <module> class Client: File “/Users/laurentchriqui/Documents/data-stream-router/.venv/lib/python3.7/site-packages/pulsar_client-2.4.0-py3.7-macosx-10.9-x86_64.egg/pulsar/__init__.py”, line 361, in Client schema=schema.BytesSchema(), File “/Users/laurentchriqui/Documents/data-stream-router/.venv/lib/python3.7/site-packages/pulsar_client-2.4.0-py3.7-macosx-10.9-x86_64.egg/pulsar/schema/schema.py”, line 53, in __init__ super(BytesSchema, self).__init__(bytes, _pulsar.SchemaType.BYTES, None, ‘BYTES’) File “/Users/laurentchriqui/Documents/data-stream-router/.venv/lib/python3.7/site-packages/pulsar_client-2.4.0-py3.7-macosx-10.9-x86_64.egg/pulsar/schema/schema.py”, line 32, in __init__ json.dumps(schema_definition, indent=True)) TypeError: __init__() should return None, not ‘NoneType’ ---- 2019-03-22 13:40:04 UTC - Daniel Ferreira Jorge: Hi! We are developing API endpoints that will produce messages to pulsar. Which is the better approach:
1. Instantiate the producer at request time, publish the message and then, close the producer at the end of the request 2. Instantiate all needed producers when the API starts and leave them open forever until the API process is closed There are many, many endpoints so the approach 2 will end up having thousands of open producers and I do not know how optimal this is. The approach 1 will have to instantiate a producer on every request, increasing the latency. I'm inclined to go with the second approach unless it is not optimal or recommended. Thanks ---- 2019-03-22 14:36:20 UTC - ou: @ou has joined the channel ---- 2019-03-22 15:41:40 UTC - Ryan Samo: Hey guys, If you want to enable the functions worker on your broker nodes and you have TLS enabled on the brokers, which parameters need to be set in the functions_worker.yml file to allow it to boot? Also do I need to add anything to the trust store? ---- 2019-03-22 15:43:52 UTC - Ryan Samo: It also seems to be ignoring the pulsarServiceUrl parameter when it set it to the 6651 TLS port ---- 2019-03-22 15:58:04 UTC - Matteo Merli: @Daniel Ferreira Jorge it’s better to reuse the producers throughout all the requests. If you don’t to create them all upfront, you can still cache the instances and reused then after the first time. There would be no problem to have thousands of producers from a single client instance. ---- 2019-03-22 16:05:56 UTC - Ryan Samo: I also get “Unable to authenticate: Client unable to authenticate with TLS certificate” in the broker logs and startup ---- 2019-03-22 16:07:54 UTC - Sijie Guo: yes. MultiVersionGenreicSchemaProvider is introduced recently but it is not used yet. It will be used in 1) AUTO_CONSUME and 2) normal `struct` schema when introducing reader & writer schema (<https://github.com/apache/pulsar/pull/3876>). ---- 2019-03-22 16:21:05 UTC - Ryan Samo: I figured it out! Just needed to set useTls: true Pulsar ignores certain parameters in the functions_worker.yml it seems the the service and web urls. It switches them based on the useTls parameter ---- 2019-03-22 16:44:24 UTC - Daniel Ferreira Jorge: @Matteo Merli Thanks! That's great! ---- 2019-03-22 16:46:13 UTC - Grant Wu: Created <#CH4SG2ZK2|dev-websocket> for Websocket stuff ---- 2019-03-22 16:46:34 UTC - Grant Wu: This way <#CFTUA9AHK|dev-node> can be strictly for the native node client ---- 2019-03-22 17:44:18 UTC - Matteo Merli: @bossbaby there are multiple steps in the process of ensuring the ordering: 1. Producer API - When a client calls send() or sendAsync() messages are enqueued to a pending queue from which they are pushed to the broker, in the same order - If there’s a disconnection between client and broker, the client will resend all pending messages, again in same order 2. Broker - Broker receives messages and appends them into a BooKkeeper ledger. The sequence is established by the order in which the messages are received by broker. - This order is defined across all producers publishing on the topic +1 : Shivji Kumar Jha ---- 2019-03-22 17:56:01 UTC - Shivji Kumar Jha: Thanks @Sijie Guo I will go through that patch :slightly_smiling_face: ---- 2019-03-22 18:39:23 UTC - Jerry Peng: How are you trying to compile? ---- 2019-03-22 18:40:19 UTC - Jerry Peng: After you pull in the change: ``` $ cd pulsar-client-cpp $ cmake . $ make _pulsar -j8 $ cd python $ python setup.py bdist_wheel ``` ---- 2019-03-22 18:41:05 UTC - Jerry Peng: and the pulsar-client wheel file will be in dist/ ---- 2019-03-22 21:45:05 UTC - Chris DiGiovanni: Is there a way to get additional logging from the offloader? ---- 2019-03-22 21:50:03 UTC - Chris DiGiovanni: Just seems to stall after it uploads a ledger ---- 2019-03-23 00:14:08 UTC - Ben k: @Ben k has joined the channel ---- 2019-03-23 02:34:42 UTC - Ben S: Is there direct request-reply support in Pulsar like Kafka offers? See here for pattern: <https://callistaenterprise.se/blogg/teknik/2018/10/26/synchronous-request-reply-over-kafka/> ---- 2019-03-23 02:35:52 UTC - Ben S: I‘m still thinking about alternate design choices, but it looks like that is what will fit best for my scenario ---- 2019-03-23 02:36:02 UTC - Matteo Merli: It doesn’t seem to me a Kafka feature though. This is something implemented on top of it ---- 2019-03-23 02:36:55 UTC - Matteo Merli: You can certainly use messages properties in Pulsar to indicate a topic where you want to receive a response, and some kind of “request-id” to correlate the response with a request ---- 2019-03-23 02:39:24 UTC - Ben S: Thanks, I‘ll give it a try. ---- 2019-03-23 02:40:58 UTC - Ali Ahmed: @Benjamin spring just adds correlation id’s to achieve this ---- 2019-03-23 02:44:50 UTC - Ben S: Something completely unrelated: I followed the guide to start a standalone instance of Pulsar downloaded from the main web site on OS X Mojave for testing, but the server did not start. Using the docker works fine. There is no error message, it just looks like it‘s trying to establish a connection to zk again and again. Tried to build and start from source with the same issue. Eventually I missed something obvious, but I just followed the steps without much further investigation. ---- 2019-03-23 02:46:25 UTC - Ali Ahmed: try “pulsar standalone --wipe-data” ---- 2019-03-23 02:46:30 UTC - Ali Ahmed: check ports ---- 2019-03-23 02:46:40 UTC - Ali Ahmed: also you can use this “<https://github.com/streamlio/homebrew-formulae>” ---- 2019-03-23 02:46:56 UTC - Ali Ahmed: and they just “brew services start pulsar” ---- 2019-03-23 02:51:45 UTC - Ben S: Thanks, will try that later. I think it‘s not the ports, and both instanced (source build and downloaded package) were clean. But will give it a try. ---- 2019-03-23 08:09:17 UTC - Shivji Kumar Jha: This seems like a nice initiative : <https://opensource.googleblog.com/2019/03/introducing-season-of-docs.html> ---- 2019-03-23 08:10:16 UTC - Shivji Kumar Jha: Perhaps pulsar can ride this to get more technical writers to improve our docs :smile: ----
