Hello Livy Developers,
I am new in livy community, and I start to understand how it
works under the hood. I see in the source code of livy
( server/src/main/scala/org/apache/livy/LivyConf.scala )
that the command to submit to spark cluster a request is "spark-submit" in
function sparkSubmit :
/** Return the path to the spark-submit executable. */
def sparkSubmit(): String = {
sparkHome().map { _ + File.separator + "bin" + File.separator +
"spark-submit" }.get
}
I think a good idea is to use a variable instead of a literal , for some
reasons :
1) some distributions of hadoop/spark use sym link to spark-submit like
spark2-submit and so on
2) some users could have spark 1 and spark 2, and they have customized with
sym link to switch between version easily and to be sure what versions they
was running, instead to inspect SPARK_HOME variable and PATH .
3) in the future it's possible they will change the name of the command
What do you think about ? If you are agree, Could I open a feature request
about this on Jira
https://issues.apache.org/jira/projects/LIVY/issues/LIVY-449?filter=allopenissues
?
I try to wrote a little patch based on the branch on github of apache livy,
I read on https://livy.incubator.apache.org/community/ that to propose a
patch I have to include tests.
I try to find a document about hoe to do tests correctly, but I don't find
it.
Could you help me?
Kind Regards
Matteo Durighetto
