Hi Atri,
Picked a blocking queue to do a small prototype to check if it helps
in improvement. Blocking queue will handle these by default:
- Stop pushing data from publisher when queue is full. For example if
the downstream operator is slow, we need to suspend publisher for a while.
- Wait on the receiver thread if queue is empty.
Once we have a flow going, we can certainly try out different Queue
implementations and choose which suits the purpose best!
Thanks,
Isha
On Fri, Nov 13, 2015 at 12:04 PM, Pramod Immaneni <[email protected]>
wrote:
> It is same process only till the buffer server thereafter the data is
> destined to go to a separate process.
>
> Thanks
>
> On Fri, Nov 13, 2015 at 11:30 AM, Munagala Ramanath <[email protected]>
> wrote:
>
> > If it is the same container, why bother serializing ?
> >
> > Ram
> >
> > On Fri, Nov 13, 2015 at 11:19 AM, Isha Arkatkar <[email protected]>
> > wrote:
> > > Hi all,
> > >
> > > For APEX-259 (https://malhar.atlassian.net/browse/APEX-259), I am
> > > exploring option of passing serialized tuples from publisher to buffer
> > > server through a blocking queue.
> > >
> > > Right now, publisher and buffer server reside within the same
> container,
> > > however, communication between the two goes though sockets. We want to
> > > check if we get any performance benefit by changing this communication
> to
> > > queue-based one.
> > >
> > > This is in exploration phase right now, but if we do see improvement,
> we
> > > may want to provide it as a pluggable option.
> > >
> > > Please let me know your thoughts!
> > >
> > > Thanks,
> > > Isha
> >
>
