Re: [E] Re: [Discuss] PIP to add system topic for topic creation/deletion events

2021-04-22 Thread Joe Francis
the system itself into it, (eg: security events like failed auth etc), but you will need to provide an aggregate view for the cluster as a whole from all the brokers Joe On Thu, Apr 22, 2021 at 6:10 AM Joe Francis wrote: > Completely disagree that we have accepted this risk with PIP-39. Tha

Re: [E] Re: [Discuss] PIP to add system topic for topic creation/deletion events

2021-04-22 Thread Joe Francis
Completely disagree that we have accepted this risk with PIP-39. That is different because it is an admin flow. A failure in a namespace policy change does not affect data flow. What you are proposing is in the data path. Topics and subs are created in the data flow path. Failure means outages.

Re: [E] Re: [PIP-78] Split the individual acknowledgments into multiple entries

2021-01-22 Thread Joe Francis
Let me take a step back and explain how I am looking at this from a high-level design viewpoint Bookkeeper (BK) is like an LSM implementation of a KV store. Writes to all keys are appended to a single file; deletes are logical. Compaction reclaims space. An Index is used locate entries, tracki

Re: [DISCUSS] PIP 64: Introduce REST endpoints for producing, consuming and reading messages

2020-05-14 Thread Joe Francis
We do have concerns, and we have implemented throttles on even our existing REST APIs in the broker. So we would like to be able to turn this off, and validation that the implementation changes do not degrade existing protobuf based throughput Joe On Thu, May 14, 2020 at 12:35 AM Sijie Guo wrote

Re: PIP 36: Max message size

2019-05-14 Thread Joe Francis
They are not unrelated, because msg size automatically server double duty as a resource unit now, for all buffer allocations, and decoupling the two is good. It's perfectly reasonable to have one resource unit and be able to support small and large messages, instead of having to run separate instan

Re: [PROPOSAL] Remove deprecated client APIs in 2.4 release

2018-12-13 Thread Joe Francis
-1. The amount of changes this will need, especially around tools and code developed around the old API is enormous. And no, remaining with the old lib is not an option, unless there is a plan to maintain the old lib, with dependencies up-to-date with security fixes I would like the team to dis

Re: Limitations on No.Of Producers and Consumers

2018-11-12 Thread Joe Francis
>>The real limit in pulsar is the amount of metadata that can be stored in zookeeper. After, PIP-8, It is now possible to run multiple Pulsar clusters under one service URL (going beyond the limits of a what a single ZK can support). Joe On Mon, Nov 12, 2018 at 7:36 AM, Dave Fisher wrote: > M

Re: No space on H21 (Was: Fwd: Build failed in Jenkins: guacamole-client-coverity #73)

2018-10-01 Thread Joe Francis
We should cleanup all the bk/zk dirs created by unit tests. Almost 16gig of junk On Mon, Oct 1, 2018, 11:31 AM Matteo Merli wrote: > Chris, are these `ws-cleanup` jobs something that is created internally by > Jenkins? The job we have is `pulsar_precommit_cpp`. > > Is there any setting in Jenkin

Re: Pulsar release 2.2

2018-09-27 Thread Joe Francis
Good. I will get this going. Joe On Thu, Sep 27, 2018 at 8:30 PM, Sijie Guo wrote: > Hi Joe, > > I think all the issues for 2.2 are merged :) we are ready to go with the > release :) > > - Sijie > > > On Fri, Sep 21, 2018 at 5:54 PM Matteo Merli > wrote: > > > Hopefully there should be no issu