Hi Jerry, thank you for sharing the information. Do you know if the number of pulsar functions with localrun can be seamlessly increased? Say, A single instance (or kube pod) of FunctionA with localrun is consuming a topic and all the sudden (there may be a peak time for my service) I need to increase it to 10 instances. Would there be a downtime (rebalancing etc.)? Is the seamlessness same for degrading the number of function instances?
On Sat, May 5, 2018 at 11:49 PM, Jerry Peng <[email protected]> wrote: > Hi, > > Currently, submitting a Pulsar functions to run in an existing Pulsar > cluster does not support resource allocations/enforcements or quotas. We > may add such features in the future. However, you can run functions in > existing Kubernetes clusters via localrun mode. Kubernetes will provide > the resource allocations and enforcement mechanisms. > > On Sat, May 5, 2018 at 11:42 PM jun aoki <[email protected]> wrote: > > > (I switched to use users@. I should've started so) > > Thank you Sanjeev for explaining. It is exciting. > > Is it true that pulsar cluster does some kind of resource > management/scheduling? > > For example, when there are many functions registered and many messages > come in, does each function gets certain resource (memory/cpu per node) > allocations, dynamically, based on its tenant/namespace, similar way to > what Spark does? > > > On Sat, May 5, 2018 at 12:04 AM, Sanjeev Kulkarni <[email protected]> > wrote: > > >> Hi, > >> While introducing stream native computing in Pulsar, we wanted to start > >> with the most basic primitives aka functions. Thus 2.0 added Pulsar > >> Functions. This > >> <https://pulsar.incubator.apache.org/docs/latest/functions/quickstart/> > is > >> a good starting page to get started with functions. Advanced libraries > >> based on the functions primitive(like WindowFunction and streamlet apis) > >> are targetted for 2.1. > >> Thanks! > > >> On Fri, May 4, 2018 at 10:53 PM, jun aoki <[email protected]> wrote: > > >> > I did a 10 min google search for pulsar stream API, something > comparable to > >> > kafka stream API. (https://kafka.apache.org/documentation/streams/) > >> > but I haven't been able to find any. Is there such thing exists? Or am > I > >> > conceptually mistaken? > >> > > >> > -- > >> > -jun > >> > > > > > > > -- > > -jun > -- -jun
