I end up using a shared folder to allow both parties to share resources both input and output
On Wed, Jul 6, 2016 at 10:37 AM, Bryan Fok <[email protected]> wrote: > Understood, moving to that direction now. Thank you for your advance. > > BR > Bryan > > On Wed, Jul 6, 2016 at 1:29 AM, haosdent <[email protected]> wrote: > >> I afraid protobuf don't allow so large message size? I think you may >> store your result in HDFS or other distributed filesystem. And executor >> just notify framework the progress of store. >> >> On Tue, Jul 5, 2016 at 2:17 PM, Bryan Fok <[email protected]> >> wrote: >> >>> Hi all >>> >>> What is the best practice to have a custom executor to return large size >>> (~200mb) value (e.g. matrix array, or numpy array) ? >>> >>> B.R >>> Bryan >>> >> >> >> >> -- >> Best Regards, >> Haosdent Huang >> > >
