Hello once more!
I have prepared a patch. I think it won't be hard for you to try it out
since you are accustomized to building Ignite. Please try it, tell if it
solves the current issue. ignite-7080.patch
<http://apache-ignite-users.70518.x6.nabble.com/file/t1312/ignite-7080.patch>
I was trying to set Ignite up on AWS EMR today, without much success. I
wonder if you could provide me with some directions.
> IGNITE_YARN_JAR=/mnt/ignite/apache-ignite-2.3.0-src/modules/yarn/target/ignite-yarn-2.3.0.jar
> yarn jar ${IGNITE_YARN_JAR} ${IGNITE_YARN_JAR}
> /mnt/ignite/ignite_yarn.properties
Can you tell me what were you using to launch this command? I have tried
executing "Custom JAR" step on EMR cluster, after copying Ignite properties
and JAR to /mnt on master manually. So I was informed that an application
was started, and is now running, but no indication about execution results.
I suspect it hung with the same problem that you outlined (or earlier), but
I haven't found a way to view application logs, neither "node manager" logs
that you are referring to. Can you please point me where I can watch
intermediate execution results?
Also there's two data access questions. First, Ignite YARN jar expects two
files on local FS, the jar itself, and properties file. Is there a way to
make sure that these files are on file system, at predictable locations,
when starting EMR cluster? To be able to add "Custom JAR" step to cluster
initialization? E.g. specify an s3 bucket to be mounted to master node?
Another question is that you are specifying ignite zip and ignite config are
on hdfs. How would you put these files to hdfs when cluster is created? Or,
for that matter, after it is created? I haven't found any access to HDFS in
EMR console, neither any instructions on how to access it.
Maybe we should change this process in order to use Ignite YARN with EMR
without extensive boilerplate?
Regards,
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
