Hi Amit,
as the purpose is to remove IOChannelFactory, then I would suggest it's
a runner concern. The Read.Bounded should "locate" the bundles on a
executor close to the read data (even if it's not always possible
depending of the source).
My $0.01
Regards
JB
On 09/22/2016 02:26 PM, Amit Sela wrote:
It's not new that batch pipeline can optimize on data locality, my question
is regarding this responsibility in Beam.
If runners should implement a generic Read.Bounded support, should they
also implement locating the input blocks ? or should it be a part
of IOChannelFactory implementations ? or another way to go at it that I'm
missing ?
Thanks,
Amit.
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
