Hi Benjamin, I changed the file permissions to 777 for my work directories and log files. Still I don't see any server logs in webui. Also I am using mesos-slaves.sh and mesos-master.sh command to start the master and slave ( so I am not running binaries directly).
Only way I can see my slave logs is my creating the links ( the thing that I was doing before). But I can't see master logs by anyway. I am getting the same error with and without links. Regards, Praveenesh On Tue, Sep 25, 2012 at 12:05 AM, Benjamin Mahler <[email protected]>wrote: > Ah, I was confused by the asterisks in your commands. > > So you're running: > $ ln -s lt-mesos-slave.INFO mesos-slave.INFO > because your log file is named lt-mesos-slave.INFO? > > This suggests you're running the lt-mesos-slave binary directly, instead of > mesos-slave.sh. Google-glog will use basename(argv0) as the logging > destination, so running mesos-slave.sh will produce a log file named > mesos-slave.INFO. > > Regardless, I have a change that should fix this issue for you: > https://reviews.apache.org/r/7061/ > It's not currently in trunk, so in the interim I would suggest looking at > permissions of the work directory you created on the master, to ensure it's > readable by the master process. > > Hope this helps! > > On Mon, Sep 24, 2012 at 3:32 AM, praveenesh kumar <[email protected] > >wrote: > > > Hi Benjamin, > > > > So I did, what you suggest: > > > > 1. -wipe your existing symbolic links, etc > > -remove the ln commands from your workflow > > -run again, and see if the webui master LOG works, if not please > attach > > the master logs and the commands you ran > > > > 2. I can't still see the webui master Logs...On web UI, I am seeing > "Failed > > to initialize .. retrying" > > > > Error : > > E0924 05:10:30.077483 10025 files.cpp:92] Error attaching path > > '/usr/local/mesos-spark/mesos/work/mesos-master.INFO': Failed to > > canonicalize /usr/local/mesos-spark/mesos/work/mesos-master.INFO into an > > absolute path > > > > > > Its the same error as I was getting before. > > > > 3. I deleted the links I created for slave nodes also. When I start slave > > nodes, now I am not able to see the slave node logs also. > > > > * tail: cannot open > > `/usr/local/mesos-spark/mesos/work/mesos-slave.INFO' for reading: No such > > file or directory* > > > > Slave error logs -- > > > > E0924 05:21:00.418911 11332 files.cpp:92] Error attaching path > > '/usr/local/mesos-spark/mesos/work/mesos-slave.INFO': Failed to > > canonicalize /usr/local/mesos-spark/mesos/work/mesos-slave.INFO into an > > absolute path > > > > E0924 05:21:00.525163 11332 slave.cpp:362] Failed to attach log file: > > Failed to canonicalize /usr/local/mesos-spark/mesos/work/mesos-slave.INFO > > into an absolute path > > > > > > After reading the above errors only, I created those symlinks and it > > allowed me to see slave logs, so I thought of doing the same for master > > also, but couldn't succeed. > > > > Thanks for any help. > > > > Regards, > > Praveenesh > > > > > > On Fri, Sep 21, 2012 at 10:40 PM, Benjamin Mahler <[email protected] > > >wrote: > > > > > 1. I would suggest looking into the 'daemon' linux command man page. > > > > > > 3. What are you trying to accomplish with the symbolic links? They > appear > > > to conflict with the logging files google-glog will create. The error > > from > > > the master log indicates that the log files cannot be attached (and > hence > > > viewing logs from the webui won't work). > > > > > > I would suggest: > > > -wipe your existing symbolic links, etc > > > -remove the ln commands from your workflow > > > -run again, and see if the webui master LOG works, if not please > attach > > > the master logs and the commands you ran > > > > > > Hope this helps! > > > > > > On Thu, Sep 20, 2012 at 11:42 PM, praveenesh kumar < > [email protected] > > > >wrote: > > > > > > > Thanks Benjamin. > > > > > > > > 1. I was wondering how can I run mesos-master and slaves as daemon > > > process. > > > > Nohup is giving me issues to launch them from remotely. > > > > > > > > 3. I have build mesos from trunk and starting meso master and slave > by > > > --- > > > > > > > > Mesos-master > > > > > > > > *nohup /usr/local/mesos-spark/mesos/bin/mesos-master.sh > > > > --log_dir=/usr/local/mesos-spark/mesos/work >> /tmp/mesos-master.txt > &* > > > > * > > > > * > > > > *ln -s lt-mesos-master.INFO mesos-master.INFO* > > > > *ln -s lt-mesos-master.ERROR mesos-master.ERROR* > > > > *ln -s lt-mesos-master.WARNING mesos-master.WARNING* > > > > > > > > > > > > > > > > Mesos-slave --- (Note I need to create links to all log files to > > enable > > > > logging from mesos web url) > > > > > > > > *nohup /usr/local/mesos-spark/mesos/bin/mesos-slave.sh > > --master=br9:5050 > > > > --log_dir=/usr/local/mesos-spark/mesos/work >> /tmp/mesos-slave.txt > &* > > > > > > > > *cd /usr/local/mesos-spark/mesos/* > > > > *chmod 777 -R work* > > > > *cd work* > > > > *ln -s lt-mesos-slave.ERROR mesos-slave.ERROR* > > > > *ln -s lt-mesos-slave.WARNING mesos-slave.WARNING* > > > > *ln -s lt-mesos-slave.INFO mesos-slave.INFO* > > > > > > > > > > > > I see only these suspicious lines in lt-mesos-master.ERROR > > > > > > > > Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg > > > > E0918 04:28:37.111768 28729 files.cpp:92] Error attaching path > > > > '/usr/local/mesos-spark/mesos/work/mesos-master.INFO': Failed to > > > > canonicalize /usr/local/mesos-spark/mesos/work/mesos-master.INFO into > > an > > > > absolute path > > > > E0918 04:28:37.112239 28731 master.cpp:535] Failed to attach log > file: > > > > Failed to canonicalize > > > /usr/local/mesos-spark/mesos/work/mesos-master.INFO > > > > into an absolute path > > > > > > > > Regards, > > > > Praveenesh > > > > > > > > > > > > On Wed, Sep 19, 2012 at 12:26 PM, Benjamin Mahler < > [email protected] > > > > >wrote: > > > > > > > > > 1. You could run it as a daemon, what you're doing with nohup is > fine > > > as > > > > > well. > > > > > > > > > > 2. The webui shows various resource statistics: [image: Inline > image > > 2] > > > > > > > > > > *Total memory* indicates the total amount of memory available on > all > > > the > > > > > slaves. > > > > > Total Memory should equal Used + Idle + Offered > > > > > * > > > > > * > > > > > *Used:* Memory used by the frameworks. > > > > > *Offered:* This memory has been offered but the framework hasn't > > > > > responded yet. > > > > > *Idle:* Any remaining memory in the system. > > > > > > > > > > 3. Are you running off trunk or a release? What is the command line > > > > you're > > > > > running to start the master? Can you provide the master's log > output? > > > > > > > > > > On Tue, Sep 18, 2012 at 11:57 PM, praveenesh kumar < > > > [email protected] > > > > >wrote: > > > > > > > > > >> Hi all, > > > > >> > > > > >> I have been able to successfully build and install mesos on my > > > physical > > > > >> nodes. I can see the mesos web-url with list of mesos slaves. > > However > > > I > > > > am > > > > >> finding few difficulties w.r.t to following things: > > > > >> > > > > >> *1. How to start mesos-master.sh and mesos-slave.sh as a > background > > > > >> process > > > > >> > > > > >> ? * > > > > >> I am seeing all my output on the console. Since I want to remotely > > > start > > > > >> the service on the cluster, I tried using nohup. Its working for > > now, > > > > but > > > > >> is it the right way? I am hoping there should be better way to do > > > that. > > > > >> > > > > >> *2.* *Whatever memory I am seeing on the mesos-web url, is it the > > > total > > > > >> memory or the free memory available on my system ?* > > > > >> > > > > >> I am seeing very low memory (0.001 GB) -- Does it indicate the > > current > > > > >> available memory ? > > > > >> > > > > >> 3. I have enable the "log_dir" directory, while starting the > > > > mesos-master > > > > >> and slaves. I can see the logs for my slave machines from mesos > web > > > url, > > > > >> but not able to see logs for my master. Its saying *"Failed to > > > > Initialize > > > > >> .... Retrying"*. I can see work directory is there as well as the > > logs > > > > are > > > > >> > > > > >> also there. Set the permission to 777 also. Still no success. > > > > >> > > > > >> > > > > >> Any help on the above to understand mesos, would be very helpful. > > > > >> > > > > >> Regards, > > > > >> Praveenesh > > > > >> > > > > > > > > > > > > > > > > > > > >
