If it takes t to emit one tuple and t*x < 0.5 sec, emitBatchSize can be
set to x. This will help with CPU utilization as platform will put the
operator thread to sleep if it does not emit and does not have
handleIdleTime() callback defined.
Thank you,
Vlad
On 4/7/17 18:53, Bhupesh Chawda wrote:
I think the understanding is wrong.
Platform calls emitTuples multiple times in a window. This number is
unknown; it depends on the window time. We can limit this to x.
emitBatchSize controls the number of tuples emitted in one such call.
Set this to 1.
This should result in at most x tuples per window. Note that it can be
less than x as well.
~ Bhupesh
On Apr 8, 2017 01:56, "Ambarish Pande" <ambarish.pande2...@gmail.com
<mailto:ambarish.pande2...@gmail.com>> wrote:
Hello Bhupesh Sir,
But does that mean I am emitting only 'x' lines from the file?.
Because from what I understood, emitTuples() emits multiple lines
in a single call and emitBatchSize controls number of times
emitTuples is called in a window. Am I right?. I inferred this
from the following
The platform invokes the |emitTuples()| callback multiple time
in each streaming window; within a single such call, if a
large number of tuples are emitted,there is some risk that
they may overwhelm the downstream operators especially if they
are performing some compute intensive operation.
Thank You.
On Fri, Apr 7, 2017 at 1:44 PM, Bhupesh Chawda
<bhup...@datatorrent.com <mailto:bhup...@datatorrent.com>> wrote:
You can set emitBatchSize to 1 and make sure emitTuples is
called just 'x' times within a window. You can do this
manually by keeping a count and resetting it in beginWindow().
~ Bhupesh
_______________________________________________________
Bhupesh Chawda
E: bhup...@datatorrent.com <mailto:bhup...@datatorrent.com>|
Twitter: @bhupeshsc
www.datatorrent.com <http://www.datatorrent.com> |
apex.apache.org <http://apex.apache.org>
On Fri, Apr 7, 2017 at 1:38 PM, Ambarish Pande
<ambarish.pande2...@gmail.com
<mailto:ambarish.pande2...@gmail.com>> wrote:
Yes i tried. That just gives me control on how many times
emitTuples is called. I want control on number of tuples
emitted.
Thank you.
Sent from my iPhone
On 07-Apr-2017, at 8:08 AM, Yogi Devendra
<devendra.vyavah...@gmail.com
<mailto:devendra.vyavah...@gmail.com>> wrote:
Have you tried /emitBatchSize /as mentioned
https://apex.apache.org/docs/malhar/operators/fsInputOperator/
<https://apex.apache.org/docs/malhar/operators/fsInputOperator/>
~ Yogi
On 3 April 2017 at 00:05, Ambarish Pande
<ambarish.pande2...@gmail.com
<mailto:ambarish.pande2...@gmail.com>> wrote:
Hi,
How can i make the AbstractFileInputOperator emit
only 'x' number of lines per window. Is there a hook
for that. Or i have to do it manually?
Thank You.
