Re: splitIntoBundles vs. generateInitialSplits

2017-04-13 Thread Etienne Chauchot
ter creating readers and waiting to use splitAtFraction. S On Sun, Jan 8, 2017 at 6:06 AM Stas Levin <stasle...@gmail.com> wrote: Hi, A short terminology question regarding "bundle", and particularly splitIntoBundles vs. generateInitialSplits. In *BoundedSource* we have: List>

Re: splitIntoBundles vs. generateInitialSplits

2017-04-13 Thread Jean-Baptiste Onofré
to be able to split up the work before creating readers rather than after creating readers and waiting to use splitAtFraction. S On Sun, Jan 8, 2017 at 6:06 AM Stas Levin <stasle...@gmail.com> wrote: Hi, A short terminology question regarding "bundle", and p

Re: splitIntoBundles vs. generateInitialSplits

2017-04-13 Thread Etienne Chauchot
y would be able to use splitIntoBundles during job startup to be able to split up the work before creating readers rather than after creating readers and waiting to use splitAtFraction. S On Sun, Jan 8, 2017 at 6:06 AM Stas Levin <stasle...@gmail.com> wrote: Hi, A short te

Re: splitIntoBundles vs. generateInitialSplits

2017-03-21 Thread Stas Levin
r naming discuss > >>>> > >>>> whether > >>>>> > >>>>> it was intentional or not. Conceptually, the way that bounded vs > >>>> > >>>> streaming > >>>>> > >>>>>

Re: splitIntoBundles vs. generateInitialSplits

2017-03-21 Thread Jean-Baptiste Onofré
nd waiting to use splitAtFraction. S On Sun, Jan 8, 2017 at 6:06 AM Stas Levin <stasle...@gmail.com> wrote: Hi, A short terminology question regarding "bundle", and particularly splitIntoBundles vs. generateInitialSplits. In *BoundedSource* we have: List> *splitIntoBundle

Re: splitIntoBundles vs. generateInitialSplits

2017-03-20 Thread Ismaël Mejía
> the >>>> >>>> streaming source is creating conceptual divisions of the data that will >>> >>> be >>>> >>>> used later. I'm not sure that's worth the confusion caused by the >>>> differences. >>>> >>>>

Re: splitIntoBundles vs. generateInitialSplits

2017-01-11 Thread Stas Levin
; > efficiency would be able to use splitIntoBundles during job startup to be > > able to split up the work before creating readers rather than after > > creating readers and waiting to use splitAtFraction. > > > > S > > > > On Sun, Jan 8, 2017 at 6:06 AM

Re: splitIntoBundles vs. generateInitialSplits

2017-01-09 Thread Stas Levin
use splitIntoBundles during job startup to be > able to split up the work before creating readers rather than after > creating readers and waiting to use splitAtFraction. > > S > > On Sun, Jan 8, 2017 at 6:06 AM Stas Levin <stasle...@gmail.com> wrote: > > > H

Re: splitIntoBundles vs. generateInitialSplits

2017-01-09 Thread Stephen Sisk
ction. S On Sun, Jan 8, 2017 at 6:06 AM Stas Levin <stasle...@gmail.com> wrote: > Hi, > > A short terminology question regarding "bundle", and > particularly splitIntoBundles vs. generateInitialSplits. > > In *BoundedSource* we have: > List>

splitIntoBundles vs. generateInitialSplits

2017-01-08 Thread Stas Levin
Hi, A short terminology question regarding "bundle", and particularly splitIntoBundles vs. generateInitialSplits. In *BoundedSource* we have: List> *splitIntoBundles*(...) In *UnboundedSource* we have: List> *generateInitialSplits*(...) I was wondering if the names were in