[
https://issues.apache.org/jira/browse/SPARK-11264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Naisbitt updated SPARK-11264:
-------------------------------------
Description:
Some GREP_OPTIONS will modify the output of the grep commands that are looking
for the assembly jars in bin/spark-class.
For example, if the -n option is specified, the grep output will look like:
{code}5:spark-assembly-1.5.1-hadoop2.4.0.jar{code}
This will not match the regular expressions, and so the jar files will not be
found. We could improve the regular expression to handle cases like this and
trim off extra characters, but it is difficult to know which options may or may
not be set. Unsetting GREP_OPTIONS within the script handles all the cases and
gives the desired output.
By the way, the actual error seen from the commandline was this:
{code}Error: Could not find or load main class
org.apache.spark.launcher.Main{code}
was:
Some GREP_OPTIONS will modify the output of the grep commands that are looking
for the assembly jars in bin/spark-class.
For example, if the -n option is specified, the grep output will look like:
{code}5:spark-assembly-1.5.1-hadoop2.4.0.jar{code}
This will not match the regular expressions, and so the jar files will not be
found. We could improve the regular expression to handle cases like this and
trim off extra characters, but it is difficult to know which options may or may
not be set. Unsetting GREP_OPTIONS within the script handles all the cases and
gives the desired output.
> ./bin/spark-class can't find assembly jars with certain GREP_OPTIONS set
> ------------------------------------------------------------------------
>
> Key: SPARK-11264
> URL: https://issues.apache.org/jira/browse/SPARK-11264
> Project: Spark
> Issue Type: Bug
> Components: Spark Shell
> Affects Versions: 1.5.1
> Reporter: Jeffrey Naisbitt
> Priority: Minor
>
> Some GREP_OPTIONS will modify the output of the grep commands that are
> looking for the assembly jars in bin/spark-class.
> For example, if the -n option is specified, the grep output will look like:
> {code}5:spark-assembly-1.5.1-hadoop2.4.0.jar{code}
> This will not match the regular expressions, and so the jar files will not be
> found. We could improve the regular expression to handle cases like this and
> trim off extra characters, but it is difficult to know which options may or
> may not be set. Unsetting GREP_OPTIONS within the script handles all the
> cases and gives the desired output.
> By the way, the actual error seen from the commandline was this:
> {code}Error: Could not find or load main class
> org.apache.spark.launcher.Main{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
