The only way I can think of is through some kind of wrapper. For java/scala,
use JNI. For Python, use extensions. There should not be a lot of work if you
know these tools.
From: Robin East
To: Annabel Melongo
Cc: Jia ; Dewful ; "user @spark"
; "d...@spark.apache.org"
Sent: Monday,
Thanks Josh and Marcelo! It now works!
BTW, just wondering, is there any perf difference between running spark in
standalone mode and under yarn? The only goal that I created this cluster is to
run spark jobs. So I can set up spark in standalone mode if it runs slow in
yarn.
best.
From: Jo