Re: Unable to submit an application to standalone cluster which on hdfs.

[samuel281]

Actually I tried them both.  (hdfs:///, hdfs://)
Even I tried test to create java.net.URL instance by writing test code.

/URL test = new URL("hdfs://namenode:8020/path/to/jar");/

And it throws java.net.MalformedURLException. The message says that it
doesn't support hdfs protocol.

In the source code, ClientArguments just tries to instantiate URL object and
that's all. No URLStreamHandler either.
(https://github.com/apache/incubator-spark/blob/v0.9.0-incubating/core/src/main/scala/org/apache/spark/deploy/ClientArguments.scala)

Anybody come across the same issue?


Akhil Das wrote
> It says "*not a valid URL*"
> 
> *hdfs:///  - Invalid*
> *hdfs://   - Valid*
> 
> Hope that helps!
> 
> 
> Thanks
> Best Regards.
> 
> 
> On Wed, Feb 19, 2014 at 10:05 AM, samuel281 
> <
> samuel...@gmail.com
> >
>  wrote:
> 
>> I'm trying to launch application inside the cluster (standalone mode)
>>
>> According to docs, jar-url can be either file:// or hdfs:// format. (
>> https://spark.incubator.apache.org/docs/latest/spark-standalone.html)
>>
>> But, when I tried to run spark-class It seemed unable to parse hdfs://xx
>> format.
>>
>> 
> <command>
>> spark-class org.apache.spark.deploy.Client launch \
>>     cds-test05:7077 \
>>     hdfs:///namenode:8020/user/datalab/filename.jar \
>>     my.package.Runner \
>>     -i /user/myself/input -o /user/myself/output -m
>> spark://sparkmaster:7077
>>
>> 
> <output>
>> Jar url 'hdfs:///namenode:8020/user/datalab/filename.jar' is not a valid
>> URL.
>> Jar must be in URL format (e.g. hdfs://XX, file://XX)
>>
>> I've found that *ClientArguments class is using java.net
>> 
> &lt;
> http://java.net/
> &gt;
> .URL class to parse jar-url, and It doesn't support hdfs
>> protocol.*
>>
>> ------------------------------
>> View this message in context: Unable to submit an application to
>> standalone cluster which on
>> hdfs.
> &lt;
> http://apache-spark-user-list.1001560.n3.nabble.com/Unable-to-submit-an-application-to-standalone-cluster-which-on-hdfs-tp1730.html
> &gt;
>> Sent from the Apache Spark User List mailing list
>> archive
> &lt;
> http://apache-spark-user-list.1001560.n3.nabble.com/
> &gt;
> at
>> Nabble.com.
>>
> 
> 
> 
> -- 
> Thanks
> Best Regards

Quoted from: 
http://apache-spark-user-list.1001560.n3.nabble.com/Unable-to-submit-an-application-to-standalone-cluster-which-on-hdfs-tp1730p1731.html




--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Unable-to-submit-an-application-to-standalone-cluster-which-on-hdfs-tp1730p1739.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.