This is exactly what we want Jeff! A hook to plug in our own interpreters.
(I am on same team as Jhon btw)
Right now there are too many concrete references and injecting stuff is not
possible.
Eg of customizations -
1) Spark UI which works differently on EMR than standalone, so that logic
Hi Jeff,
#3 is not about different spark versions.
Same spark versions but multiple clusters. So based on logged in user, we may
want to route to different spark cluster or even let user choose the spark he
wants to connect to.
Will work with Jhon to create tickets on other #2.
What is the
Thanks for your replies Jeff,
We mention the deference between spark standalone and spark on yarn because
of the way in which spark UI URL is build by zeppelin, not for the the
possibility of connect multiple cluster:
*Spark UI URL on Yarn client mode in EMR*
In yarn client mode the access to
>>> Same spark versions but multiple clusters. So based on logged in user,
we may want to route to different spark cluster or even let user choose the
spark he wants to connect to.
If you use standalone, you can set `master` in interpreter setting for
different standalone cluster. If using yarn,
1) Spark UI which works differently on EMR than standalone, so that logic
will be in an interpreter specific to emr.
Could you create a ticket for that, and please add details of that ? I
don't know exactly what the difference between EMR and standalone, we can
expose api to allow customization
Hi Jhon,
Do you mind to share what kind of custom function you want to add to spark
interpreter ? One idea in my mind is that we could add extension point to
the existing SparkInterpreter, and user can enhance SparkInterpreter via
these extension point. That means we just open some interfaces and
Hello!
I'm a software developer and as part of a project I require to extend the
functionality of SparkInterpreter without modifying it. I need instead
create a new interpreter that extends it or wrap its functionality.
I also need the spark sub-interpreters to use my new custom interpreter,
but
