Re: Not getting event logs >= spark 1.3.1

2015-06-16 Thread Tsai Li Ming
Forgot to mention this is on standalone mode.

Is my configuration wrong?

Thanks,
Liming

On 15 Jun, 2015, at 11:26 pm, Tsai Li Ming  wrote:

> Hi,
> 
> I have this in my spark-defaults.conf (same for hdfs):
> spark.eventLog.enabled  true
> spark.eventLog.dir  file:/tmp/spark-events
> spark.history.fs.logDirectory   file:/tmp/spark-events
> 
> While the app is running, there is a “.inprogress” directory. However when 
> the job completes, the directory is always empty.
> 
> I’m submitting the job like this, using either the Pi or world count examples:
> $ bin/spark-submit 
> /opt/spark-1.4.0-bin-hadoop2.6/examples/src/main/python/wordcount.py 
> 
> This used to be working in 1.2.1 and didn’t test 1.3.0.
> 
> 
> Regards,
> Liming
> 
> 
> 
> 
> 
> 
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
> 


-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org



Not getting event logs >= spark 1.3.1

2015-06-15 Thread Tsai Li Ming
Hi,

I have this in my spark-defaults.conf (same for hdfs):
spark.eventLog.enabled  true
spark.eventLog.dir  file:/tmp/spark-events
spark.history.fs.logDirectory   file:/tmp/spark-events

While the app is running, there is a “.inprogress” directory. However when the 
job completes, the directory is always empty.

I’m submitting the job like this, using either the Pi or world count examples:
$ bin/spark-submit 
/opt/spark-1.4.0-bin-hadoop2.6/examples/src/main/python/wordcount.py 

This used to be working in 1.2.1 and didn’t test 1.3.0.


Regards,
Liming






-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org