Re: Submitted applications does not run.

2015-09-01 Thread Madawa Soysa 
Hi Jeff,

I solved the issue by following the given instructions. Thanks for the help.

Regards,
Madawa.

On 1 September 2015 at 14:12, Jeff Zhang  wrote:

> You need to make yourself able to ssh to localhost without password,
> please check this blog.
>
> http://hortonworks.com/kb/generating-ssh-keys-for-passwordless-login/
>
>
>
> On Tue, Sep 1, 2015 at 4:31 PM, Madawa Soysa 
> wrote:
>
>> I used ./sbin/start-master.sh
>>
>> When I used ./sbin/start-all.sh the start fails. I get the following
>> error.
>>
>> failed to launch org.apache.spark.deploy.master.Master:
>> localhost: ssh: connect to host localhost port 22: Connection refused
>>
>> On 1 September 2015 at 13:41, Jeff Zhang  wrote:
>>
>>> Did you start spark cluster using command sbin/start-all.sh ?
>>> You should have 2 log files under folder if it is single-node cluster.
>>> Like the following
>>>
>>> spark-jzhang-org.apache.spark.deploy.master.Master-1-jzhangMBPr.local.out
>>> spark-jzhang-org.apache.spark.deploy.worker.Worker-1-jzhangMBPr.local.out
>>>
>>>
>>>
>>> On Tue, Sep 1, 2015 at 4:01 PM, Madawa Soysa 
>>> wrote:
>>>
 There are no logs which includes apache.spark.deploy.worker in file
 name in the SPARK_HOME/logs folder.

 On 1 September 2015 at 13:00, Jeff Zhang  wrote:

> This is master log. There's no worker registration info in the log.
> That means the worker may not start properly. Please check the log file
> with apache.spark.deploy.worker in file name.
>
>
>
> On Tue, Sep 1, 2015 at 2:55 PM, Madawa Soysa 
> wrote:
>
>> I cannot see anything abnormal in logs. What would be the reason for
>> not availability of executors?
>>
>> On 1 September 2015 at 12:24, Madawa Soysa 
>> wrote:
>>
>>> Following are the logs available. Please find the attached.
>>>
>>> On 1 September 2015 at 12:18, Jeff Zhang  wrote:
>>>
 It's in SPARK_HOME/logs

 Or you can check the spark web ui. http://[master-machine]:8080


 On Tue, Sep 1, 2015 at 2:44 PM, Madawa Soysa <
 madawa...@cse.mrt.ac.lk> wrote:

> How do I check worker logs? SPARK_HOME/work folder does not exist.
> I am using the spark standalone mode.
>
> On 1 September 2015 at 12:05, Jeff Zhang  wrote:
>
>> No executors ? Please check the worker logs if you are using
>> spark standalone mode.
>>
>> On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa <
>> madawa...@cse.mrt.ac.lk> wrote:
>>
>>> Hi All,
>>>
>>> I have successfully submitted some jobs to spark master. But the
>>> jobs won't progress and not finishing. Please see the attached 
>>> screenshot.
>>> These are fairly very small jobs and this shouldn't take more than 
>>> a minute
>>> to finish.
>>>
>>> I'm new to spark and any help would be appreciated.
>>>
>>> Thanks,
>>> Madawa.
>>>
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>>> For additional commands, e-mail: user-h...@spark.apache.org
>>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
>
> *_**Madawa Soysa*
>
> Undergraduate,
>
> Department of Computer Science and Engineering,
>
> University of Moratuwa.
>
>
> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
> madawa...@cse.mrt.ac.lk
> LinkedIn  | Twitter
>  | Tumblr
> 
>



 --
 Best Regards

 Jeff Zhang

>>>
>>>
>>>
>>> --
>>>
>>> *_**Madawa Soysa*
>>>
>>> Undergraduate,
>>>
>>> Department of Computer Science and Engineering,
>>>
>>> University of Moratuwa.
>>>
>>>
>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>>> madawa...@cse.mrt.ac.lk
>>> LinkedIn  | Twitter
>>>  | Tumblr
>>> 
>>>
>>
>>
>>
>> --
>>
>> *_**Madawa Soysa*
>>
>> Undergraduate,
>>
>> Department of Computer Science and Engineering,
>>
>> University of Moratuwa.
>>
>>
>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>> madawa...@cse.mrt.ac.lk
>> LinkedIn  | Twitter
>>

Re: Submitted applications does not run.

2015-09-01 Thread Jeff Zhang 
You need to make yourself able to ssh to localhost without password, please
check this blog.

http://hortonworks.com/kb/generating-ssh-keys-for-passwordless-login/



On Tue, Sep 1, 2015 at 4:31 PM, Madawa Soysa 
wrote:

> I used ./sbin/start-master.sh
>
> When I used ./sbin/start-all.sh the start fails. I get the following error.
>
> failed to launch org.apache.spark.deploy.master.Master:
> localhost: ssh: connect to host localhost port 22: Connection refused
>
> On 1 September 2015 at 13:41, Jeff Zhang  wrote:
>
>> Did you start spark cluster using command sbin/start-all.sh ?
>> You should have 2 log files under folder if it is single-node cluster.
>> Like the following
>>
>> spark-jzhang-org.apache.spark.deploy.master.Master-1-jzhangMBPr.local.out
>> spark-jzhang-org.apache.spark.deploy.worker.Worker-1-jzhangMBPr.local.out
>>
>>
>>
>> On Tue, Sep 1, 2015 at 4:01 PM, Madawa Soysa 
>> wrote:
>>
>>> There are no logs which includes apache.spark.deploy.worker in file
>>> name in the SPARK_HOME/logs folder.
>>>
>>> On 1 September 2015 at 13:00, Jeff Zhang  wrote:
>>>
 This is master log. There's no worker registration info in the log.
 That means the worker may not start properly. Please check the log file
 with apache.spark.deploy.worker in file name.



 On Tue, Sep 1, 2015 at 2:55 PM, Madawa Soysa 
 wrote:

> I cannot see anything abnormal in logs. What would be the reason for
> not availability of executors?
>
> On 1 September 2015 at 12:24, Madawa Soysa 
> wrote:
>
>> Following are the logs available. Please find the attached.
>>
>> On 1 September 2015 at 12:18, Jeff Zhang  wrote:
>>
>>> It's in SPARK_HOME/logs
>>>
>>> Or you can check the spark web ui. http://[master-machine]:8080
>>>
>>>
>>> On Tue, Sep 1, 2015 at 2:44 PM, Madawa Soysa <
>>> madawa...@cse.mrt.ac.lk> wrote:
>>>
 How do I check worker logs? SPARK_HOME/work folder does not exist.
 I am using the spark standalone mode.

 On 1 September 2015 at 12:05, Jeff Zhang  wrote:

> No executors ? Please check the worker logs if you are using spark
> standalone mode.
>
> On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa <
> madawa...@cse.mrt.ac.lk> wrote:
>
>> Hi All,
>>
>> I have successfully submitted some jobs to spark master. But the
>> jobs won't progress and not finishing. Please see the attached 
>> screenshot.
>> These are fairly very small jobs and this shouldn't take more than a 
>> minute
>> to finish.
>>
>> I'm new to spark and any help would be appreciated.
>>
>> Thanks,
>> Madawa.
>>
>>
>>
>> -
>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>> For additional commands, e-mail: user-h...@spark.apache.org
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



 --

 *_**Madawa Soysa*

 Undergraduate,

 Department of Computer Science and Engineering,

 University of Moratuwa.


 Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
 madawa...@cse.mrt.ac.lk
 LinkedIn  | Twitter
  | Tumblr
 

>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>>
>> --
>>
>> *_**Madawa Soysa*
>>
>> Undergraduate,
>>
>> Department of Computer Science and Engineering,
>>
>> University of Moratuwa.
>>
>>
>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>> madawa...@cse.mrt.ac.lk
>> LinkedIn  | Twitter
>>  | Tumblr 
>>
>
>
>
> --
>
> *_**Madawa Soysa*
>
> Undergraduate,
>
> Department of Computer Science and Engineering,
>
> University of Moratuwa.
>
>
> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
> madawa...@cse.mrt.ac.lk
> LinkedIn  | Twitter
>  | Tumblr 
>



 --
 Best Regards

 Jeff Zhang

>>>
>>>
>>>
>>> --
>>>
>>> *_**Madawa Soysa*
>>>
>>> Undergraduate,
>>>
>>> Department of Computer Science and Engineering,
>>>
>>> University of Moratuwa.
>>>
>>>

Re: Submitted applications does not run.

2015-09-01 Thread Madawa Soysa 
I used ./sbin/start-master.sh

When I used ./sbin/start-all.sh the start fails. I get the following error.

failed to launch org.apache.spark.deploy.master.Master:
localhost: ssh: connect to host localhost port 22: Connection refused

On 1 September 2015 at 13:41, Jeff Zhang  wrote:

> Did you start spark cluster using command sbin/start-all.sh ?
> You should have 2 log files under folder if it is single-node cluster.
> Like the following
>
> spark-jzhang-org.apache.spark.deploy.master.Master-1-jzhangMBPr.local.out
> spark-jzhang-org.apache.spark.deploy.worker.Worker-1-jzhangMBPr.local.out
>
>
>
> On Tue, Sep 1, 2015 at 4:01 PM, Madawa Soysa 
> wrote:
>
>> There are no logs which includes apache.spark.deploy.worker in file name
>> in the SPARK_HOME/logs folder.
>>
>> On 1 September 2015 at 13:00, Jeff Zhang  wrote:
>>
>>> This is master log. There's no worker registration info in the log. That
>>> means the worker may not start properly. Please check the log file
>>> with apache.spark.deploy.worker in file name.
>>>
>>>
>>>
>>> On Tue, Sep 1, 2015 at 2:55 PM, Madawa Soysa 
>>> wrote:
>>>
 I cannot see anything abnormal in logs. What would be the reason for
 not availability of executors?

 On 1 September 2015 at 12:24, Madawa Soysa 
 wrote:

> Following are the logs available. Please find the attached.
>
> On 1 September 2015 at 12:18, Jeff Zhang  wrote:
>
>> It's in SPARK_HOME/logs
>>
>> Or you can check the spark web ui. http://[master-machine]:8080
>>
>>
>> On Tue, Sep 1, 2015 at 2:44 PM, Madawa Soysa > > wrote:
>>
>>> How do I check worker logs? SPARK_HOME/work folder does not exist. I
>>> am using the spark standalone mode.
>>>
>>> On 1 September 2015 at 12:05, Jeff Zhang  wrote:
>>>
 No executors ? Please check the worker logs if you are using spark
 standalone mode.

 On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa <
 madawa...@cse.mrt.ac.lk> wrote:

> Hi All,
>
> I have successfully submitted some jobs to spark master. But the
> jobs won't progress and not finishing. Please see the attached 
> screenshot.
> These are fairly very small jobs and this shouldn't take more than a 
> minute
> to finish.
>
> I'm new to spark and any help would be appreciated.
>
> Thanks,
> Madawa.
>
>
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>



 --
 Best Regards

 Jeff Zhang

>>>
>>>
>>>
>>> --
>>>
>>> *_**Madawa Soysa*
>>>
>>> Undergraduate,
>>>
>>> Department of Computer Science and Engineering,
>>>
>>> University of Moratuwa.
>>>
>>>
>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>>> madawa...@cse.mrt.ac.lk
>>> LinkedIn  | Twitter
>>>  | Tumblr
>>> 
>>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
>
> *_**Madawa Soysa*
>
> Undergraduate,
>
> Department of Computer Science and Engineering,
>
> University of Moratuwa.
>
>
> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
> madawa...@cse.mrt.ac.lk
> LinkedIn  | Twitter
>  | Tumblr 
>



 --

 *_**Madawa Soysa*

 Undergraduate,

 Department of Computer Science and Engineering,

 University of Moratuwa.


 Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
 madawa...@cse.mrt.ac.lk
 LinkedIn  | Twitter
  | Tumblr 

>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>>
>> --
>>
>> *_**Madawa Soysa*
>>
>> Undergraduate,
>>
>> Department of Computer Science and Engineering,
>>
>> University of Moratuwa.
>>
>>
>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>> madawa...@cse.mrt.ac.lk
>> LinkedIn  | Twitter
>>  | Tumblr 
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 

*_**Madawa Soysa*

Undergraduate,

Department of Computer Science and Engineering,

University of Moratuwa.


Mobile: +94

Re: Submitted applications does not run.

2015-09-01 Thread Jeff Zhang 
Did you start spark cluster using command sbin/start-all.sh ?
You should have 2 log files under folder if it is single-node cluster. Like
the following

spark-jzhang-org.apache.spark.deploy.master.Master-1-jzhangMBPr.local.out
spark-jzhang-org.apache.spark.deploy.worker.Worker-1-jzhangMBPr.local.out



On Tue, Sep 1, 2015 at 4:01 PM, Madawa Soysa 
wrote:

> There are no logs which includes apache.spark.deploy.worker in file name
> in the SPARK_HOME/logs folder.
>
> On 1 September 2015 at 13:00, Jeff Zhang  wrote:
>
>> This is master log. There's no worker registration info in the log. That
>> means the worker may not start properly. Please check the log file
>> with apache.spark.deploy.worker in file name.
>>
>>
>>
>> On Tue, Sep 1, 2015 at 2:55 PM, Madawa Soysa 
>> wrote:
>>
>>> I cannot see anything abnormal in logs. What would be the reason for not
>>> availability of executors?
>>>
>>> On 1 September 2015 at 12:24, Madawa Soysa 
>>> wrote:
>>>
 Following are the logs available. Please find the attached.

 On 1 September 2015 at 12:18, Jeff Zhang  wrote:

> It's in SPARK_HOME/logs
>
> Or you can check the spark web ui. http://[master-machine]:8080
>
>
> On Tue, Sep 1, 2015 at 2:44 PM, Madawa Soysa 
> wrote:
>
>> How do I check worker logs? SPARK_HOME/work folder does not exist. I
>> am using the spark standalone mode.
>>
>> On 1 September 2015 at 12:05, Jeff Zhang  wrote:
>>
>>> No executors ? Please check the worker logs if you are using spark
>>> standalone mode.
>>>
>>> On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa <
>>> madawa...@cse.mrt.ac.lk> wrote:
>>>
 Hi All,

 I have successfully submitted some jobs to spark master. But the
 jobs won't progress and not finishing. Please see the attached 
 screenshot.
 These are fairly very small jobs and this shouldn't take more than a 
 minute
 to finish.

 I'm new to spark and any help would be appreciated.

 Thanks,
 Madawa.



 -
 To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
 For additional commands, e-mail: user-h...@spark.apache.org

>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>>
>> --
>>
>> *_**Madawa Soysa*
>>
>> Undergraduate,
>>
>> Department of Computer Science and Engineering,
>>
>> University of Moratuwa.
>>
>>
>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>> madawa...@cse.mrt.ac.lk
>> LinkedIn  | Twitter
>>  | Tumblr 
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



 --

 *_**Madawa Soysa*

 Undergraduate,

 Department of Computer Science and Engineering,

 University of Moratuwa.


 Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
 madawa...@cse.mrt.ac.lk
 LinkedIn  | Twitter
  | Tumblr 

>>>
>>>
>>>
>>> --
>>>
>>> *_**Madawa Soysa*
>>>
>>> Undergraduate,
>>>
>>> Department of Computer Science and Engineering,
>>>
>>> University of Moratuwa.
>>>
>>>
>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>>> madawa...@cse.mrt.ac.lk
>>> LinkedIn  | Twitter
>>>  | Tumblr 
>>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
>
> *_**Madawa Soysa*
>
> Undergraduate,
>
> Department of Computer Science and Engineering,
>
> University of Moratuwa.
>
>
> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
> madawa...@cse.mrt.ac.lk
> LinkedIn  | Twitter
>  | Tumblr 
>



-- 
Best Regards

Jeff Zhang

Re: Submitted applications does not run.

2015-09-01 Thread Madawa Soysa 
There are no logs which includes apache.spark.deploy.worker in file name in
the SPARK_HOME/logs folder.

On 1 September 2015 at 13:00, Jeff Zhang  wrote:

> This is master log. There's no worker registration info in the log. That
> means the worker may not start properly. Please check the log file
> with apache.spark.deploy.worker in file name.
>
>
>
> On Tue, Sep 1, 2015 at 2:55 PM, Madawa Soysa 
> wrote:
>
>> I cannot see anything abnormal in logs. What would be the reason for not
>> availability of executors?
>>
>> On 1 September 2015 at 12:24, Madawa Soysa 
>> wrote:
>>
>>> Following are the logs available. Please find the attached.
>>>
>>> On 1 September 2015 at 12:18, Jeff Zhang  wrote:
>>>
 It's in SPARK_HOME/logs

 Or you can check the spark web ui. http://[master-machine]:8080


 On Tue, Sep 1, 2015 at 2:44 PM, Madawa Soysa 
 wrote:

> How do I check worker logs? SPARK_HOME/work folder does not exist. I
> am using the spark standalone mode.
>
> On 1 September 2015 at 12:05, Jeff Zhang  wrote:
>
>> No executors ? Please check the worker logs if you are using spark
>> standalone mode.
>>
>> On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa > > wrote:
>>
>>> Hi All,
>>>
>>> I have successfully submitted some jobs to spark master. But the
>>> jobs won't progress and not finishing. Please see the attached 
>>> screenshot.
>>> These are fairly very small jobs and this shouldn't take more than a 
>>> minute
>>> to finish.
>>>
>>> I'm new to spark and any help would be appreciated.
>>>
>>> Thanks,
>>> Madawa.
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>>> For additional commands, e-mail: user-h...@spark.apache.org
>>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
>
> *_**Madawa Soysa*
>
> Undergraduate,
>
> Department of Computer Science and Engineering,
>
> University of Moratuwa.
>
>
> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
> madawa...@cse.mrt.ac.lk
> LinkedIn  | Twitter
>  | Tumblr 
>



 --
 Best Regards

 Jeff Zhang

>>>
>>>
>>>
>>> --
>>>
>>> *_**Madawa Soysa*
>>>
>>> Undergraduate,
>>>
>>> Department of Computer Science and Engineering,
>>>
>>> University of Moratuwa.
>>>
>>>
>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>>> madawa...@cse.mrt.ac.lk
>>> LinkedIn  | Twitter
>>>  | Tumblr 
>>>
>>
>>
>>
>> --
>>
>> *_**Madawa Soysa*
>>
>> Undergraduate,
>>
>> Department of Computer Science and Engineering,
>>
>> University of Moratuwa.
>>
>>
>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>> madawa...@cse.mrt.ac.lk
>> LinkedIn  | Twitter
>>  | Tumblr 
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 

*_**Madawa Soysa*

Undergraduate,

Department of Computer Science and Engineering,

University of Moratuwa.


Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
madawa...@cse.mrt.ac.lk
LinkedIn  | Twitter
 | Tumblr

Re: Submitted applications does not run.

2015-09-01 Thread Jeff Zhang 
This is master log. There's no worker registration info in the log. That
means the worker may not start properly. Please check the log file
with apache.spark.deploy.worker in file name.



On Tue, Sep 1, 2015 at 2:55 PM, Madawa Soysa 
wrote:

> I cannot see anything abnormal in logs. What would be the reason for not
> availability of executors?
>
> On 1 September 2015 at 12:24, Madawa Soysa 
> wrote:
>
>> Following are the logs available. Please find the attached.
>>
>> On 1 September 2015 at 12:18, Jeff Zhang  wrote:
>>
>>> It's in SPARK_HOME/logs
>>>
>>> Or you can check the spark web ui. http://[master-machine]:8080
>>>
>>>
>>> On Tue, Sep 1, 2015 at 2:44 PM, Madawa Soysa 
>>> wrote:
>>>
 How do I check worker logs? SPARK_HOME/work folder does not exist. I am
 using the spark standalone mode.

 On 1 September 2015 at 12:05, Jeff Zhang  wrote:

> No executors ? Please check the worker logs if you are using spark
> standalone mode.
>
> On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa 
> wrote:
>
>> Hi All,
>>
>> I have successfully submitted some jobs to spark master. But the jobs
>> won't progress and not finishing. Please see the attached screenshot. 
>> These
>> are fairly very small jobs and this shouldn't take more than a minute to
>> finish.
>>
>> I'm new to spark and any help would be appreciated.
>>
>> Thanks,
>> Madawa.
>>
>>
>> -
>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>> For additional commands, e-mail: user-h...@spark.apache.org
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



 --

 *_**Madawa Soysa*

 Undergraduate,

 Department of Computer Science and Engineering,

 University of Moratuwa.


 Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
 madawa...@cse.mrt.ac.lk
 LinkedIn  | Twitter
  | Tumblr 

>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>>
>> --
>>
>> *_**Madawa Soysa*
>>
>> Undergraduate,
>>
>> Department of Computer Science and Engineering,
>>
>> University of Moratuwa.
>>
>>
>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>> madawa...@cse.mrt.ac.lk
>> LinkedIn  | Twitter
>>  | Tumblr 
>>
>
>
>
> --
>
> *_**Madawa Soysa*
>
> Undergraduate,
>
> Department of Computer Science and Engineering,
>
> University of Moratuwa.
>
>
> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
> madawa...@cse.mrt.ac.lk
> LinkedIn  | Twitter
>  | Tumblr 
>



-- 
Best Regards

Jeff Zhang

Re: Submitted applications does not run.

2015-08-31 Thread Madawa Soysa 
I cannot see anything abnormal in logs. What would be the reason for not
availability of executors?

On 1 September 2015 at 12:24, Madawa Soysa  wrote:

> Following are the logs available. Please find the attached.
>
> On 1 September 2015 at 12:18, Jeff Zhang  wrote:
>
>> It's in SPARK_HOME/logs
>>
>> Or you can check the spark web ui. http://[master-machine]:8080
>>
>>
>> On Tue, Sep 1, 2015 at 2:44 PM, Madawa Soysa 
>> wrote:
>>
>>> How do I check worker logs? SPARK_HOME/work folder does not exist. I am
>>> using the spark standalone mode.
>>>
>>> On 1 September 2015 at 12:05, Jeff Zhang  wrote:
>>>
 No executors ? Please check the worker logs if you are using spark
 standalone mode.

 On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa 
 wrote:

> Hi All,
>
> I have successfully submitted some jobs to spark master. But the jobs
> won't progress and not finishing. Please see the attached screenshot. 
> These
> are fairly very small jobs and this shouldn't take more than a minute to
> finish.
>
> I'm new to spark and any help would be appreciated.
>
> Thanks,
> Madawa.
>
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>



 --
 Best Regards

 Jeff Zhang

>>>
>>>
>>>
>>> --
>>>
>>> *_**Madawa Soysa*
>>>
>>> Undergraduate,
>>>
>>> Department of Computer Science and Engineering,
>>>
>>> University of Moratuwa.
>>>
>>>
>>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>>> madawa...@cse.mrt.ac.lk
>>> LinkedIn  | Twitter
>>>  | Tumblr 
>>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
>
> *_**Madawa Soysa*
>
> Undergraduate,
>
> Department of Computer Science and Engineering,
>
> University of Moratuwa.
>
>
> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
> madawa...@cse.mrt.ac.lk
> LinkedIn  | Twitter
>  | Tumblr 
>



-- 

*_**Madawa Soysa*

Undergraduate,

Department of Computer Science and Engineering,

University of Moratuwa.


Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
madawa...@cse.mrt.ac.lk
LinkedIn  | Twitter
 | Tumblr

Re: Submitted applications does not run.

2015-08-31 Thread Madawa Soysa 
Following are the logs available. Please find the attached.

On 1 September 2015 at 12:18, Jeff Zhang  wrote:

> It's in SPARK_HOME/logs
>
> Or you can check the spark web ui. http://[master-machine]:8080
>
>
> On Tue, Sep 1, 2015 at 2:44 PM, Madawa Soysa 
> wrote:
>
>> How do I check worker logs? SPARK_HOME/work folder does not exist. I am
>> using the spark standalone mode.
>>
>> On 1 September 2015 at 12:05, Jeff Zhang  wrote:
>>
>>> No executors ? Please check the worker logs if you are using spark
>>> standalone mode.
>>>
>>> On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa 
>>> wrote:
>>>
 Hi All,

 I have successfully submitted some jobs to spark master. But the jobs
 won't progress and not finishing. Please see the attached screenshot. These
 are fairly very small jobs and this shouldn't take more than a minute to
 finish.

 I'm new to spark and any help would be appreciated.

 Thanks,
 Madawa.


 -
 To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
 For additional commands, e-mail: user-h...@spark.apache.org

>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>>
>>
>> --
>>
>> *_**Madawa Soysa*
>>
>> Undergraduate,
>>
>> Department of Computer Science and Engineering,
>>
>> University of Moratuwa.
>>
>>
>> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
>> madawa...@cse.mrt.ac.lk
>> LinkedIn  | Twitter
>>  | Tumblr 
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 

*_**Madawa Soysa*

Undergraduate,

Department of Computer Science and Engineering,

University of Moratuwa.


Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
madawa...@cse.mrt.ac.lk
LinkedIn  | Twitter
 | Tumblr 
Spark Command: /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -cp 
/opt/spark-1.4.1-bin-hadoop2.6/ml/org.wso2.carbon.ml.core_1.0.1.SNAPSHOT.jar:/opt/spark-1.4.1-bin-hadoop2.6/ml/org.wso2.carbon.ml.commons_1.0.1.SNAPSHOT.jar:/opt/spark-1.4.1-bin-hadoop2.6/sbin/../conf/:/opt/spark-1.4.1-bin-hadoop2.6/lib/spark-assembly-1.4.1-hadoop2.6.0.jar:/opt/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/opt/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/opt/spark-1.4.1-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar
 -Xms512m -Xmx512m -XX:MaxPermSize=256m org.apache.spark.deploy.master.Master 
--ip [master-url] --port 7077 --webui-port 8080

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/09/01 11:32:14 INFO Master: Registered signal handlers for [TERM, HUP, INT]
15/09/01 11:32:14 WARN Utils: Your hostname, ubuntu resolves to a loopback 
address: 127.0.1.1; using 10.8.108.92 instead (on interface wlan0)
15/09/01 11:32:14 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another 
address
15/09/01 11:32:15 WARN NativeCodeLoader: Unable to load native-hadoop library 
for your platform... using builtin-java classes where applicable
15/09/01 11:32:15 INFO SecurityManager: Changing view acls to: ubuntu
15/09/01 11:32:15 INFO SecurityManager: Changing modify acls to: ubuntu
15/09/01 11:32:15 INFO SecurityManager: SecurityManager: authentication 
disabled; ui acls disabled; users with view permissions: Set(ubuntu); users 
with modify permissions: Set(ubuntu)
15/09/01 11:32:16 INFO Slf4jLogger: Slf4jLogger started
15/09/01 11:32:16 INFO Remoting: Starting remoting
15/09/01 11:32:16 INFO Remoting: Remoting started; listening on addresses 
:[akka.tcp://sparkMaster@ubuntu:7077]
15/09/01 11:32:16 INFO Utils: Successfully started service 'sparkMaster' on 
port 7077.
15/09/01 11:32:17 INFO Utils: Successfully started service on port 6066.
15/09/01 11:32:17 INFO StandaloneRestServer: Started REST server for submitting 
applications on port 6066
15/09/01 11:32:17 INFO Master: Starting Spark master at spark://ubuntu:7077
15/09/01 11:32:17 INFO Master: Running Spark version 1.4.1
15/09/01 11:32:17 INFO Utils: Successfully started service 'MasterUI' on port 
8080.
15/09/01 11:32:17 INFO MasterWebUI: Started MasterWebUI at 
http://10.8.108.92:8080
15/09/01 11:32:17 INFO Master: I have been elected leader! New state: ALIVE
15/09/01 11:32:57 INFO Master: Registering app 
ML-SPARK-APPLICATION-0.9453989189195968
15/09/01 11:32:57 INFO Master: Registered app 
ML-SPARK-APPLICATION-0.9453989189195968 with ID app-20150901113257-

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Re: Submitted applications does not run.

2015-08-31 Thread Jeff Zhang 
It's in SPARK_HOME/logs

Or you can check the spark web ui. http://[master-machine]:8080


On Tue, Sep 1, 2015 at 2:44 PM, Madawa Soysa 
wrote:

> How do I check worker logs? SPARK_HOME/work folder does not exist. I am
> using the spark standalone mode.
>
> On 1 September 2015 at 12:05, Jeff Zhang  wrote:
>
>> No executors ? Please check the worker logs if you are using spark
>> standalone mode.
>>
>> On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa 
>> wrote:
>>
>>> Hi All,
>>>
>>> I have successfully submitted some jobs to spark master. But the jobs
>>> won't progress and not finishing. Please see the attached screenshot. These
>>> are fairly very small jobs and this shouldn't take more than a minute to
>>> finish.
>>>
>>> I'm new to spark and any help would be appreciated.
>>>
>>> Thanks,
>>> Madawa.
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>>> For additional commands, e-mail: user-h...@spark.apache.org
>>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
>
> *_**Madawa Soysa*
>
> Undergraduate,
>
> Department of Computer Science and Engineering,
>
> University of Moratuwa.
>
>
> Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
> madawa...@cse.mrt.ac.lk
> LinkedIn  | Twitter
>  | Tumblr 
>



-- 
Best Regards

Jeff Zhang

Re: Submitted applications does not run.

2015-08-31 Thread Madawa Soysa 
How do I check worker logs? SPARK_HOME/work folder does not exist. I am
using the spark standalone mode.

On 1 September 2015 at 12:05, Jeff Zhang  wrote:

> No executors ? Please check the worker logs if you are using spark
> standalone mode.
>
> On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa 
> wrote:
>
>> Hi All,
>>
>> I have successfully submitted some jobs to spark master. But the jobs
>> won't progress and not finishing. Please see the attached screenshot. These
>> are fairly very small jobs and this shouldn't take more than a minute to
>> finish.
>>
>> I'm new to spark and any help would be appreciated.
>>
>> Thanks,
>> Madawa.
>>
>>
>> -
>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>> For additional commands, e-mail: user-h...@spark.apache.org
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 

*_**Madawa Soysa*

Undergraduate,

Department of Computer Science and Engineering,

University of Moratuwa.


Mobile: +94 71 461 6050 <%2B94%2075%20812%200726> | Email:
madawa...@cse.mrt.ac.lk
LinkedIn  | Twitter
 | Tumblr

Re: Submitted applications does not run.

2015-08-31 Thread Jeff Zhang 
No executors ? Please check the worker logs if you are using spark
standalone mode.

On Tue, Sep 1, 2015 at 2:17 PM, Madawa Soysa 
wrote:

> Hi All,
>
> I have successfully submitted some jobs to spark master. But the jobs
> won't progress and not finishing. Please see the attached screenshot. These
> are fairly very small jobs and this shouldn't take more than a minute to
> finish.
>
> I'm new to spark and any help would be appreciated.
>
> Thanks,
> Madawa.
>
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>



-- 
Best Regards

Jeff Zhang