as you said we can use hierarchical spawning job in spark. there is remained another issue. for example if we have load data from ignite cache and a job on spark has been spawned. this job spawn some other jobs. this jobs can be execute on ignite too? if this job fail how about the other job spawned? spark must handle them or ignite?
On Sat, Dec 22, 2018 at 6:28 PM zaleslaw <[email protected]> wrote: > I will try to answer these questions > 1) We can using ignite as cache storage and preparing data for processing > on > dl4j? > - yes you can, it's main case, to load data to ignite caches correctly > 2) Job on spark can spawn hirarchy job for accelerate execution. > - Ignite can compute piece pf code in parallel and etc, but you need to > implement correct infrastrucure about code tasks spreading and so on over > ignite. > Look at our TF integration and distributing training > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
