BTW, is there an option to set file permission for spark event logs? Jianshi
On Thu, May 28, 2015 at 11:25 AM, Jianshi Huang <jianshi.hu...@gmail.com> wrote: > Hmm...all files under the event log folder has permission 770 but > strangely my account cannot read other user's files. Permission denied. > > I'll sort it out with our Hadoop admin. Thanks for then help! > > Jianshi > > On Thu, May 28, 2015 at 12:13 AM, Marcelo Vanzin <van...@cloudera.com> > wrote: > >> Then: >> - Are all files readable by the user running the history server? >> - Did all applications call sc.stop() correctly (i.e. files do not have >> the ".inprogress" suffix)? >> >> Other than that, always look at the logs first, looking for any errors >> that may be thrown. >> >> >> On Wed, May 27, 2015 at 9:10 AM, Jianshi Huang <jianshi.hu...@gmail.com> >> wrote: >> >>> Yes, all written to the same directory on HDFS. >>> >>> Jianshi >>> >>> On Wed, May 27, 2015 at 11:57 PM, Marcelo Vanzin <van...@cloudera.com> >>> wrote: >>> >>>> You may be the only one not seeing all the logs. Are you sure all the >>>> users are writing to the same log directory? The HS can only read from a >>>> single log directory. >>>> >>>> On Wed, May 27, 2015 at 5:33 AM, Jianshi Huang <jianshi.hu...@gmail.com >>>> > wrote: >>>> >>>>> No one using History server? :) >>>>> >>>>> Am I the only one need to see all user's logs? >>>>> >>>>> Jianshi >>>>> >>>>> On Thu, May 21, 2015 at 1:29 PM, Jianshi Huang < >>>>> jianshi.hu...@gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I'm using Spark 1.4.0-rc1 and I'm using default settings for history >>>>>> server. >>>>>> >>>>>> But I can only see my own logs. Is it possible to view all user's >>>>>> logs? The permission is fine for the user group. >>>>>> >>>>>> -- >>>>>> Jianshi Huang >>>>>> >>>>>> LinkedIn: jianshi >>>>>> Twitter: @jshuang >>>>>> Github & Blog: http://huangjs.github.com/ >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Jianshi Huang >>>>> >>>>> LinkedIn: jianshi >>>>> Twitter: @jshuang >>>>> Github & Blog: http://huangjs.github.com/ >>>>> >>>> >>>> >>>> >>>> -- >>>> Marcelo >>>> >>> >>> >>> >>> -- >>> Jianshi Huang >>> >>> LinkedIn: jianshi >>> Twitter: @jshuang >>> Github & Blog: http://huangjs.github.com/ >>> >> >> >> >> -- >> Marcelo >> > > > > -- > Jianshi Huang > > LinkedIn: jianshi > Twitter: @jshuang > Github & Blog: http://huangjs.github.com/ > -- Jianshi Huang LinkedIn: jianshi Twitter: @jshuang Github & Blog: http://huangjs.github.com/
